No title

ARTICLE

Communicated by Bard Ermentrout

Asynchronous States and the Emergence of Synchrony in Large Networks of Interacting Excitatory and Inhibitory Neurons D. Hansel [email protected] Laboratoire de Neurophysique et de Physiologie du Syst`eme Moteur, UniversitÂe RenÂe Descartes, 75270 Paris Cedex 06, France, and InterdisciplinaryCenter for Neural Computation, Hebrew University of Jerusalem, 91904 Jerusalem, Israel

G. Mato [email protected] ComisiÂon Nacional de EnergÂõ a AtÂomica and CONICET, Centro AtÂomico Bariloche and Instituto Balseiro (CNEA and UNC), 8400 San Carlos de Bariloche, R.N., Argentina

We investigate theoretically the conditions for the emergence of synchronous activity in large networks, consisting of two populations of extensively connected neurons, one excitatory and one inhibitory. The neurons are modeled with quadratic integrate-and-Žre dynamics, which provide a very good approximation for the subthreshold behavior of a large class of neurons. In addition to their synaptic recurrent inputs, the neurons receive a tonic external input that varies from neuron to neuron. Because of its relative simplicity, this model can be studied analytically. We investigate the stability of the asynchronous state (AS) of the network with given average Žring rates of the two populations. First, we show that the AS can remain stable even if the synaptic couplings are strong. Then we investigate the conditions under which this state can be destabilized. We show that this can happen in four generic ways. The Žrst is a saddle-node bifurcation, which leads to another state with different average Žring rates. This bifurcation, which occurs for strong enough recurrent excitation, does not correspond to the emergence of synchrony. In contrast, in the three other instability mechanisms, Hopf bifurcations, which correspond to the emergence of oscillatory synchronous activity, occur. We show that these mechanisms can be differentiated by the Žring patterns they generate and their dependence on the mutual interactions of the inhibitory neurons and cross talk between the two populations. We also show that besides these codimension 1 bifurcations, the system can display several codimension 2 bifurcations: Takens-Bogdanov, Gavrielov-Guckenheimer, and double Hopf bifurcations.

Neural Computation 15, 1–56 (2003)

° c 2002 Massachusetts Institute of Technology

2

D. Hansel and G. Mato

1 Introduction Synchrony can be deŽned in several ways. In its broader deŽnition, the activities of two neurons are said to be synchronized if some kind of temporal correlations exist between them. In this sense, in any system of interacting neurons, some level of synchrony should exist. In this respect, observing synchrony between directly interacting neurons should not be surprising. In fact, several studies in the 1960s and 1970s were devoted to determining what the shape of the cross-correlations of a pair of neurons can tell us about their interactions (Abeles, 1990). The implicit assumption in this approach was that correlations are dominated by direct interactions between the neurons. Other types of synchronous states can emerge through cooperative effects. In these states, the correlations in the activity of a pair of neurons in the network depend weakly on their direct interaction (Hansel & Sompolinsky, 1996). Instead, they are mostly mediated by the inputs to the neurons coming from all other neurons belonging to the part of the network with which they interact. To deŽne this type of synchronous state, it is appropriate to consider macroscopic observables, for example, the instantaneous activity of the neurons averaged across assemblies of neurons (multi-unit activities) or local Želd potentials that presumably reect electrical activity induced by many neurons. In an asynchronous state, the temporal uctuations of these quantities (suitably normalized) are small—on the order of 1 / M where M is the size of the neuronal assembly. When synchrony emerges, these uctuations become Žnite even if the population is large. Understanding the conditions for the emergence of these states in the brain is one of the goals of recent research in the Želd of collective neuronal dynamics. Central issues concern the roles of the cellular properties of the neurons; the nature (excitatory, inhibitory), the strength, and the kinetics of the synaptic interactions; and the local and global architectures of the networks. The ways in which noise, heterogeneities, and spatial uctuations of connectivity patterns affect synchrony are also among the fundamental questions investigated. One possible approach to these questions is to study cooperative dynamical states in simpliŽed network models of local neuronal populations in the brain. Within limits, these models can be studied in depth analytically. Subsequently, these analytical investigations can be extended using numerical simulations beyond the limits that can be treated analytically. This approach has led to substantial progress in our understanding of the general mechanisms of neuronal synchrony. However, most of the theoretical studies have addressed the case of one population of neurons, much less is known about networks comprising several populations. The goal of this article is to study the emergence of synchronous activity in large networks consisting of two interacting populations of neurons, one excitatory and the other inhibitory. We investigate the conditions

Asynchronous States and the Emergence of Synchrony

3

under which asynchronous states (AS) become unstable due to interactions. More speciŽcally, we inquire: (1) Under which conditions can the network display a stable asynchronous state? Is this possible when the couplings are strong? (2) What are the different mechanisms through which the AS can become unstable in the network? (3) In what ways do they depend on the recurrent interactions in the excitatory neurons, the mutual interactions of the inhibitory neurons, and the excitatory-inhibitory feedback connections between the two populations? (4) What patterns of synchrony emerge when the AS is unstable? In particular, can one differentiate unambiguously between the regimes of parameters in which neurons spike in synchrony from regimes in which they burst in synchrony? (5) More quantitatively, what determines the period of the synchronous oscillations and the phase shift between the oscillations in the two populations? The article is organized as follows. The model is introduced in section 2. In section 3, the method of AS stability analysis is presented, and a spectral equation for the eigenmodes of the linearized dynamics is established. Assuming that the synaptic time constants of the excitatory and inhibitory synapses are identical simpliŽes the analysis of this equation. This allows us in section 4 to study the instabilities of AS to a large extent completely analytically. The various bifurcations of the AS are characterized. The results of section 4 are subsequently generalized in section 5. Results from numerical simulations are presented in section 6. Section 7 is devoted to a discussion of our results. 2 The model 2.1 The Quadratic Integrate-and-Fire Model. Near the onset of Žring, it can be shown that the detailed dynamics of any type I conductance-based model can be replaced by the reduced model (Ermentrout & Kopell, 1986; Ermentrout, 1996), ¡ ¢2 dV CN D A V ¡ V ? C I ¡ Ic , dt

(2.1)

where Ic is the rheobase of the neuron. Analytical formulas can be derived N the constant A, and V ?. Using these to compute the effective capacitance C, formulas, one can compute these parameters numerically as functions of the biophysical parameters of the full conductance-based model (see appendixes A and B). The solution of equation 2.1 diverges in Žnite time. This divergence corresponds to the Žring of an action potential. If one supplements equation 2.1 with the condition that the variable V has to be reset to ¡1 following a spike, this yields a reduced model that accurately describes the dynamics of the full conductance-based model in the limit I ! Ic . In

4

D. Hansel and G. Mato

this model, the relationship between the Žring rate º and the current I is given by p º D B I ¡ Ic ,

(2.2)

p N where B D A / p C. When I ¡ Ic is not small, this reduction is no longer exact. However, one can generalize it heuristically as follows. We assume that action potentials are Žred whenever V crosses some threshold, Vt (from below), and that it is immediately reset to a value Vr . The relationship between the discharge rate and the external current of the model is now given by p

p

Vt A Vr A 1 1 arctan( pI¡Ic ) ¡ arctan( pI¡Ic ) p . D pB º I ¡ Ic

(2.3)

The two parameters, Vt and Vr , are phenomenological. They are not determined by the exact reduction method near the Žring onset, and additional constraints are required to determine them. In the following, we impose that the º ¡ I curve of the quadratic integrate-and-Žre (QIF) model matches the º ¡ I curve of the full model as much as possible. It is convenient to rewrite the reduced model in terms of the dimensionless variables VN and IN deŽned by A VN D t0 (V ¡ V ?) CN A 2 IN D It0 , CN 2

(2.4) (2.5)

where t0 has a dimension of a time. The dynamics are governed by t0

¯ dV N 2 C IN ¡ INc . DV dt

(2.6)

The dynamical variable VN is reset to VNr D A ( Vr ¡ V ?)t 0 / CN whenever N A similar model has it reaches the threshold value: VN t D A ( Vt ¡ V ?)t0 / C. been used by Latham, Richmond, Nelson, and Nirenberg (2000) to study with numerical simulations a two-population network model. In this article, we use a QIF model with parameters derived from the Wang-Buszaki (WB) conductance-based model described in appendix A. Using the approach described above, one Žnds that Ic D 0.1601 mA / cm2 , V ? D ¡59.5462 mV, CN D 0.9467 m F / cm2 , and A D 0.012875 mS / cm2 / mV. Requiring that the º-I curve of the reduced and the full model agrees well up to Žring rates as large as ºmax D 100 spikes per second yields for the other parameters Vr ¡ V ? D ¡4.6 mV and Vt ¡ V ? D 33.2 mV (VN r D ¡0.626,

Asynchronous States and the Emergence of Synchrony

5

100

n (Hz)

75 50 25 0 0.0

0.5

1.0 1.5 I (mA/cm 2)

2.0

Figure 1: Firing rate of a neuron versus the external current. Circles: WangBusza ki model. Solid line: QIF model with A D 0.012875 mS / cm2 / mV, Vr ¡V ¤ D N D 0.9467 m F / cm2 . ¡4.6 mV, Vt ¡ V ¤ D 33.2 mV, Ic D 0.1601 muA / cm2 and C

VN t D 4.52). The º-I curves of the full model and the QIF model with these parameters are plotted in Figure 1. Clearly the Žt is excellent over a broad range of Žring rates. The traces of the membrane potential for neurons Žring at 50 spikes per second and 10 spikes per second are shown in Figures 2A and 2B, respectively, for this QIF model. For comparison, we have also plotted the traces of a WB neuron that Žres at the same frequencies. For a frequency of 10 Hz, the traces are very similar. When the frequency increases, the traces of the QIF neuron deviate from those of the WB neuron. 2.2 The Two-Population Network Model. We deal with a network of two heterogeneous populations: one excitatory and the other inhibitory. The dynamics of the neurons is modeled by equation 2.6, dV ia syn 2 th C Iia C Iia ( t ) ¡ Ia D Via dt

(2.7)

where i D 1, . . . , Na are the indices of the neurons in population a (a D E, I). For simplicity, the Žring threshold of the neurons Vt , the resetting value Vr , and the rheobase currents are the same for all the neurons. Time is measured in units of t0 D 10 ms. The current Iia represents the effect of an external stimulus that is assumed to be constant in time but different from one neuron to another. We consider the case where Na is very large and the external input currents to the neurons in population a are distributed according to a density syn distribution Qa ( I ) . The synaptic current, Iia ( t) , on the ith cell, is the sum of

6

D. Hansel and G. Mato A

V (mv)

20 20 60 100

0

20 t (msec)

40 B

V (mv)

20 20 60 100

0

100

200 t (msec)

300

400

Figure 2: Traces of a WB neuron (solid line) and QIF neuron (dashed line). Parameters of the model are the same as in Figure 1. (A) The neuron Žres at 50 Hz. (B) The neuron Žres at 10 Hz. syn

all contributions Iijab from all the presynaptic cells on it: syn Iia (t) D

X

syn

Iijab (t ).

(2.8)

jb syn

In this work, the current Iijab ( t) is modeled by syn

syn

Iijab ( t) D Gijab sjb (t) ,

(2.9)

syn

where Gijab measures the strength of the synapses that neuron j in population b makes on neuron i in population a and sjb evolves with time according to sjb ( t) D

X spikes

fb ( t ¡ tspike,j ) ,

(2.10)

Asynchronous States and the Emergence of Synchrony

7

where the summation is performed over all the spikes emitted by the presynaptic neuron with an index j at times tspike,j . The function fb ( t) is fb ( t ) D

t2b

µ ³ ´ ³ ´¶ 1 t t exp ¡ ¡ exp ¡ H (t) . ¡ t1b t1b t2b

(2.11)

Here, H ( t) is the Heaviside function, and the normalization is such that the integral of fb ( t) is one. The characteristic times t1b and t2b are the rise and decay times of the synapse, respectively. In case t1b D t2b D tb , one obtains the so-called alpha function (Rall, 1967): fb ( t ) D

³ ´ t t exp ¡ . tb tb

(2.12)

syn

Note that Gijab has the dimension of a current density and that excitatory syn

syn

(resp. inhibitory) interactions correspond to Gijab > 0 (resp. Gijab < 0). The synaptic current, which a cell i of the ath population receives from a cell j from the bth population, is syn

Iiab ( t) D

X

syn

Iijab ( t) ,

(2.13)

j

where a D E, I, i D 1, . . . , Na , j D 1, . . . , Nb , and Na is the number of neurons in the ath population. As a limiting case of a network with extensive connectivity, we assume full connectivity between the neurons, which simpliŽes considerably our analytical approach. For simplicity, we also assume that the strength of a synapse between two neurons depends on only the populations to which they belong. Therefore, syn

syn

Gijab D Gab .

(2.14)

In the case of an extensively connected network, the average number of synaptic inputs per neuron varies proportionally to the size of the system, and the synaptic strengths are varied in inverse proportion to the system size so that (see Hansel & Sompolinsky, 1996) syn

Gab D

gab , Nb

(2.15)

where the normalized synaptic strengths, gab are independent of the Na ’s. Our approach can be applied to the case where the kinetics of excitatory (resp. inhibitory) synapses are different when the postsynaptic neuron belongs to the excitatory or the inhibitory population. However, this entails

8

D. Hansel and G. Mato

an increase in the number of parameters of the model. Therefore, in this article, we consider that the synaptic rise time and decay time depend on only the nature, excitatory or inhibitory, of the synapses, and not on the synapse target. The rise and decay times of the synapses are denoted by t1b , t2b , b D E, I. We neglect axonal propagation delays. 3 The Asynchronous State and Its Stability 3.1 The Distribution of Neuron Firing Rates in the Asynchronous State. In the asynchronous state, the synaptic current is the same for all neurons in a population and does not depend on time. It is given by X syn syn (3.1) Iia (t) D gabºb ´ Ia , b D E,I

where ºb is the average Žring rate of the neurons in population b and gab is a 2 £ 2 matrix that characterizes the strengths of the synaptic interactions within and between the populations. Integrating the dynamical equations, equation 2.7, one calculates the Žring rate of each of the neurons in the network. This yields "Z ºia D

Vt Vr

Á ´ F Iia

# ¡1

dV syn

V2 C Iia C Ia ¡ Iath ! X C gabºb ¡ Iath ,

(3.2)

b D E,I

where F (I) D

p

I p arctan(Vt / I ) ¡ arctan(Vr / I ) p

(3.3)

if I > 0 and F ( I ) D 0 otherwise. These equations, together with ºa D

1 X ºia, Na i

(3.4)

determine the distribution of Žring rates in each population. Note that neusyn rons with an external current smaller than Iamin D Ith a ¡ Ia are below threshold and cannot Žre. The distribution Pa (º) of the Žring rates of the neurons in population a in the asynchronous state can be related to the distribution of external currents via the input-output relation, Z 0

P a (º) D Qa ( F ¡1 (º) ) |F ¡1 (º) | C d (º)

Imin a ¡1

dIQ a (I ) ,

(3.5)

Asynchronous States and the Emergence of Synchrony

9 A

Q(I)

1

0.5

0 -1

0

1 I (mA/cm 2 )

2 B

0.03

P(n)

0.02

0.01

0

0

25

50 n (Hz)

75

100

Figure 3: (A) Gaussian distribution of external current with a standard deviation s D 0.4. (B) Distribution of Žring rates for the QIF model (parameters as in Figure 1) and a distribution of currents as in A. The average Žring rate is ºN D 30 Hz. The delta function at º D 0 is not shown.

where the last contribution comes from the neurons that receive a total input too small to make them Žre. The distribution of Žring rates is plotted in Figure 3 for a gaussian distribution of currents:

Qa ( I ) D p

³ ´ ( I ¡ Ia ) 2 exp ¡ . 2sa2 2p sa2 1

(3.6)

We emphasize that the distribution of Žring rates can be kept constant even if the coupling strengths change. This can be done simply by adjusting the distribution of external currents with an offset that depends on the coupling and the average Žring rate of the population. In this way, one can separate the effect of coupling strength and the effect of the average rate on synchronization properties.

10

D. Hansel and G. Mato

3.2 Stability Analysis of the Asynchronous State. The stability of the AS in a homogeneous network of leaky integrate-and-Žre neurons has been studied by Abbott and van Vreeswijk (1993). We have generalized their approach to the case of heterogeneous network of QIF neurons (Hansel & Mato, 2001). For each of the active neurons in the population a, the variable yia can be deŽned as Z Via dx P (3.7) yia D ºia , 2 C I C x ia Vr b gab ºb where ºi and ºa are the Žring rates of neuron i and the average population Žring rate in the AS, respectively, and gab are the strengths of the synapses from population b to population a. This dynamical variable evolves in time according to X dyia D ºia C gab Kb ( yia ) 2 dt b

b ( t) ,

(3.8)

where 2 b ( t) D

Nb 1 X (ºjb (t) ¡ ºb ) Nb j D1

(3.9)

is the deviation of the Žring rate of neuron j at time t from its value in the AS, and Ka ( yia ) D

ºia P . 2 CI C Via i b gabºb

(3.10)

The probability density, r a ( y, º, t ) satisŽes the continuity equation, @r a ( y, º, t) @t

D ¡

@Ja ( y, º, t ) @y

,

(3.11)

where the ux Ja ( y, º, t) is Á X Ja ( y, º, t) D ºa C gab Kb ( y )2

! b ( t)

r a ( y, º, t) .

(3.12)

b

In the asynchronous state, Ja ( y, º, t) D ºa . The deviations from this value, ja ( y, º, t) D Ja ( y, º, t) ¡ ºa , due to perturbations from the AS state satisfy @ja ( y, º, t) @t

D

X b

gab Kb ( y )

d2

b ( t)

dt

¡ ºa

@ja ( y, º, t) @y

.

(3.13)

Asynchronous States and the Emergence of Synchrony

11

For synaptic interactions modeled with a difference of exponentials (see equations 2.11), it is straightforward to show that equation 3.13 is equivalent to a ( t) D ¡c 1a2 a ( t) C ha ( t) dt X dha ( t) D ¡c 2a ha ( t) C c 1ac 2a ja (1, º, t) , dt º

d2

(3.14) (3.15)

where c ka D 1 / tka for k D 1, 2. The perturbation ja has to be evaluated at y D 1 because this corresponds to the crossing of the threshold, V D Vt (see equation 3.7). The AS is stable if small perturbations from the AS eventually decay. Assuming that ja ( y, º, t) , 2 a ( t) , ha (t) are proportional to exp(lt) and integrating equation 3.13, one Žnds after a straightforward but tedious calculation that l satisŽes Y µ (l C c 1a ) (l C c 2a ) ¡ c 1ac 2a gaa Ua ¶ (3.16) D gEI gIE UI UE , c 1ac 2a aD E,I where Z Ua (l) D

1 0

dº

Ha (º) 1 ¡ exp(¡l /º)

(3.17)

and Pa (º)º Ha (º) D 2F ¡1 (º)

" 1 ¡ exp(¡l /º)

(cos( A C w ) C sin(A C w ) Aº ) ) (cos(w ) C sin(w ) Aº ) l ¡ exp(¡l /º l C 2 ( 1 C Aº/ l)

# (3.18)

with p A D 2 F ¡1 (º) /º

p w D 2 arctan(Vr / F ¡1 (º) ) .

(3.19) (3.20)

The spectral equation, equation 3.16, determines the eigenvalues of the dynamical equations linearized around the AS. Note that this spectral equation also holds if a fraction of the neurons are subthreshold and do not Žre in the AS. Indeed, an inŽnitesimal perturbation cannot make these neurons Žre. Therefore, they do not contribute to the destabilization of the AS.

12

D. Hansel and G. Mato

3.3 The Spectral Equation at the Onset of an Instability. The AS is stable if all the eigenvalues, the solutions to equation 3.16, have a negative real part. Therefore, continuous onset of instabilities occurs when at least one of the eigenvalues crosses the imaginary axis when some parameter is changed. At this onset, l D im and singularities appear in the integral in equation 3.17. These singularities can be isolated, and one Žnds: ³

´ p m A1a C A2a ¡ i A3a , 2 m 2

(3.21)

dºHa (º)

(3.22)

Ua ( im ) D where Z A1a D A2a D A3a D

1 0

1 X

± m ² ± m ²2 2p n 2p n

Ha

nD 1

1 Z X nD 1

p

dy ctg

± y ² Ha ( Cm ) y 2np 2

¡p

Z

C

p

dy ctg 0

(3.23)

( y C 2np ) 2

± y ² Ha ( m ) y 2

y2

.

(3.24)

These integrals are nonsingular. In general, they have to be evaluated numerically. The onset of instabilities of the AS is determined by taking the real and imaginary part of equation 3.16. The simultaneous solution of these equations determines m and the coupling at the onset of instabilities as a function of the other parameters of the model. At instability onset, the synaptic currents in the unstable mode oscillate with a frequency given by the imaginary part of the eigenvalue, m . The dephasing d between the oscillations of the two populations is exp(id) / 2

/

I 2 E

D

gEE UE ¡ (l /c 1E C 1) (l /c 2E C 1) . | gEI |UI

(3.25)

A positive (resp. negative) value of the phase lag d means that the oscillation of the inhibitory population is in advance (resp. delayed) over the excitatory population. The solutions of equation 3.16 can be classiŽed according to the temporal 6 0, the properties of the corresponding unstable modes of instability. If m D instability occurs through a Hopf bifurcation (Strogatz, 2000). This can happen in two ways. In the Žrst, called a supercritical Hopf bifurcation, the AS loses stability, and simultaneously stable oscillations of the network activity appear with vanishingly small amplitude and Žnite frequency. The latter

Asynchronous States and the Emergence of Synchrony

13

is equal to m / 2p . The phase lag between the inhibitory and the excitatory population activity is d (see equation 3.25). In the second, called a subcritical Hopf bifurcation, the AS loses its stability by coalescing with an unstable state in which the activity of the network oscillates. This is a necessary (but not sufŽcient) condition for having a region in the parameter space where the stable AS coexists with two states of synchronous oscillations—one stable and one unstable. The stable oscillatory state persists in some regions where the AS is unstable and the unstable oscillatory state no longer exists. The values of m and d determined by the stability analysis now correspond to the frequency and the phase lag between the two populations in the unstable oscillatory state at bifurcation. However, if the amplitude of the stable oscillations at the bifurcation is small, these values should also provide a good estimate of the frequency and the phase lag in the stable oscillatory state near the bifurcation. This point needs to be veriŽed with numerical simulations (see section 6). Finally, if m D 0, there is a saddle node (SN) bifurcation (Strogatz, 2000). To study how the stability of the asynchronous states depends on the synaptic properties, we construct phase diagrams for a Žxed distribution of the Žring rates of the two populations, PE (º) and PI (º) , taking as parameters the strength of the four interactions. This implies that when the interaction strengths are changed, the external average inputs (or the Žring thresholds) have to be modiŽed accordingly to satisfy equation 3.2 for the two populations. Because of the normalization of the synaptic interactions, changing the synaptic time constants does not affect the Žring-rate distribution in the AS. To study their role on the emergence of synchrony, the synaptic time constants can be varied while maintaining constant all the other parameters. 4 Synchrony in a Two-Population Model: The Symmetric Case A detailed analysis of the conditions of the AS stability as a function of synaptic coupling strength, synaptic kinetics, and distributions of Žring rates is a formidable task. In this section, we focus on a situation where, t1E D t1I D t1 , t2E D t2I D t2 , and P E (º) D PI (º). We term this situation the symmetric case. It turns out that to a large extent, this case can be investigated analytically and that the results thus obtained are highly instructive for understanding the general properties of the two-population network. 4.1 The Instabilities of the Asynchronous State. In the symmetric case, for all a, Ua (l) D U (l) where U (l) is given by equation 3.17. Therefore equation 3.16 becomes (im C c 1 ) 2 ( im C c 2 ) 2 ¡c 1c 2 ( im C c 1 ) ( im C c 2 ) UT C c 12c 22 U 2 D D 0,

(4.1)

where U is a function of m , T D gEE C gII and D D gEE gII ¡ gEI gIE . The latter parameters are the trace and the determinant of the matrix gab , respectively.

14

D. Hansel and G. Mato

Figure 4: Phase diagram of a two-population QIF network in the symmetric case. The determinant and the trace of the coupling matrix are D D gEE gII ¡gEI gIE and T D gEE C gII , respectively. The standard deviation of the current distribution is sE D sI D 0.1. The average Žring rates in the AS are ºE D ºI D 50 Hz. The synaptic time constants are t1E D t1I D 1 msec, t2E D t2I D 4 msec. In the gray region, the AS is stable. See the text for an explanation of the different lines. The dotted line is the limit of L4 in the limit of inŽnite synaptic time constants (see section 5.3).

Therefore, one can completely describe the role of the synaptic strength in a two-dimensional plane spanned by these two effective parameters. Let us consider point X 0 D (D D 0, T D 0). A particular realization of point X 0 is obtained in the absence of coupling. Therefore, at X 0, the AS is marginally stable if the system is homogeneous, that is, for sE D sI D s D 0. If one introduces any level of heterogeneities, the AS becomes stable in some region around this point. The size of this region depends on s (or, equivalently, on the dispersion of the Žring rates in the AS). This result holds independent of the parameters and the Žring rates distribution provided the symmetry is preserved. It shows that there is always some domain in the D , T plane where the AS is stable. What are the boundaries of this region, and what are the instabilities on these boundaries? An example of the network phase diagram is shown in Figure 4. It was obtained by solving equation 4.1 for synaptic rise and decay times of 1 msec and 4 msec, respectively, the average Žring rate of the two populations, º D 50 Hz, and the standard deviation of the external input distribution, s D 0.1. The distribution of Žring rates for these parameters is shown in Figure 3. The region of stability of the AS is bounded by four lines.

Asynchronous States and the Emergence of Synchrony

15

Line L1 is deduced from equation 4.1 by taking m D 0. It is straightforward to see that the equation of the line is given by

T D U (0) D C 1 / U (0)

(4.2)

with Z U (0) D

1 0

dº

µ ¶ sin(A C w ) ¡ sin(w ) P (º)º C 1 . 2F¡1 (º) A

(4.3)

For the parameters of Figure 4, U (0) D 0.36. On this line, an SN bifurcation occurs. Above the line, the AS with ºE D ºI D 50 Hz is unstable through an SN bifurcation. At the instability onset, the unstable mode grows exponentially but does not oscillate. This corresponds to a change in the average Žring rates of the two populations, ºE and ºI , but not to the emergence of synchrony in the network. Before discussing the general meaning of transition line L2, we consider two cases: 1. P ´ gEI gIE D 0, gII D 0, gEE > 0. The two populations are noninteracting, and the inhibitory neurons are decoupled. For such parameters, the states of the network are represented in the (D , T ) plane by the line segment: D D 0, T > 0. There is one instability with real eigenvalue for the unstable mode on this segment at point XE D (0, 1 / U (0) ) , which belongs to L1. All other instabilities in this segment occur in the region above L1 (not shown in Figure 4). Therefore, for the parameters of the Žgure, the excitatory population cannot develop synchrony by itself. 2. P D gEI gIE D 0, gEE D 0, gII < 0. As in the previous case, the two populations are noninteracting, but now the excitatory neurons, rather than the inhibitory ones, are decoupled. This situation is mapped in the phase diagram on segment D D 0, T < 0. An instability with a pure imaginary eigenvalue l D im c occurs on this segment at point XI D (0, gc ) where gc and m c are solutions to the complex equation: ( im c C c 1 ) ( im c C c 2 ) D c 1c 2 gc U ( im c ).

(4.4)

Solving this equation yields gc D ¡0.98, m c D 3.02, that is, m c / (2p ºI ) ¼ 0.96. Therefore, at XI , the mutual inhibitory interactions I-I become strong enough to destabilize the AS with a Hopf bifurcation where synchrony emerges on the timescale of the spiking average period. Other solutions of equation 4.4 exist, but they are not relevant since they occur in the region below XI , where the AS is already unstable. This mechanism, where synchrony of neural activity emerged in one population of neurons, has been studied by several authors

16

D. Hansel and G. Mato

(van Vreeswijk,1996; Wang & Busz Âaki, 1996; White, Chow, Ritt, SotoTrevino, ˜ & Kopell, 1998; Neltner, Hansel, Mato, & Meunier, 2000; Golomb & Hansel, 2000; Golomb, Hansel, & Mato, 2001). 6 We now consider the general case, P D gEI gIE D 0, in which the two populations interact. It is straightforward to see that if T and D satisfy the relationship

T D

D C gc , gc

(4.5)

then equation 4.1 is satisŽed with m D m c . Equation 4.5 deŽnes a straight line (L2) in the (T , D ) plane on which a Hopf bifurcation occurs. Obviously XI belongs to this line. The intersection of L1 and L2 deŽnes a point, A ¼ (¡2.72, 1.8) , where the real parts of the two eigenvalues vanish. One corresponds to an SN bifurcation and the other to a Hopf bifurcation. At that point, the AS becomes unstable through a codimension 2 bifurcation called a Gavrielov-Guckenheimer bifurcation (Kuznetsov, 1998). The points on L1 and L2 to the left of A are not relevant as instability onset since they are inside a region where the AS is already unstable. The pattern of synchrony into which population a (a D E, I) settles depends in general on the ratio Ra D

m . 2p ºa

(4.6)

Here Ra is close to one, and neurons Žre on the average about one spike per period in the unstable mode. However, the phase relationship between the spikes and the collective oscillations can change from cycle to cycle. Consequently, the phase distribution is broad but unimodal. It can be easily shown that on L2, 2 I

2 E

D

gEE ¡ gc . | gEI |

(4.7)

Therefore, on this line, the phase lag is constant, d D 0, that is, at the instability onset, the activities in the two populations oscillate in phase. Point B on L2 deŽned by D D g2c ´ DB , T D 2gc ´ TB is a multicritical point where two eigenvalues have a vanishing real part. This can be shown by expanding equation 3.16 in the vicinity of B. One Žnds two other critical lines, L3 and L4, tangent to L2 in B. SufŽciently close to B, these lines are given by the equation

T ¡ TB D b1 ( D ¡ DB ) C b2 (D ¡ DB ) 2 ,

(4.8)

Asynchronous States and the Emergence of Synchrony

17

where b1 D

1 gc

b2 D ¡

|W 0 (m c ) | 2 . 4g3c <e(W 0 (m c ) ) 2

(4.9) (4.10)

Here we have deŽned W (m ) D c 1c 2 U ( im ) / ( im C c 1 ) ( im C c 2 ) , and W 0 (m ) is the derivative of W with respect to m . For the parameters of Figure 4, b1 D ¡1.02 and b2 D 47.9. Hence, the expansion, equation 4.8, truncated at second order, is a good approximation to L3 and L4, only very close to B. This is why it seems to be a cusp in point B. Beyond B, L2 is no longer relevant for the instability of the AS. We conclude that on AB, the AS undergoes an instability to spike-to-spike synchrony. This instability, driven by the I-I interactions, is the extension in the framework of two populations of the instability that occurs in one population of inhibitory neurons. The interaction between the two populations makes this mechanism less efŽcient. Indeed, from equation 4.5, one sees that when the feedback between the two populations increases, a stronger mutual inhibitory coupling, gII , is required for the AS to be destabilized on L2. Point D is deŽned as the intersection of L4 and L1. At that point, the AS is unstable through a codimension 2 bifurcation called a Takens-Bogdanov bifurcation (Kuznetsov, 1998). The coordinates of D can also be obtained by expanding equation 3.16 in the limit m ! 0. One Žnds that in D, D D U (0) 2 , T D 2U (0) . The frequency of the unstable mode, m , varies continuously on L3 and L4, as shown in Figure 5. On L4, m decreases continuously when D increases, from m D m cII D 3.02 at B, to m D 0 at D. On L3, m increases slowly when D increases. In the limit of large D , m converges asymptotically to a Žnite value, m 1 , and L3 is asymptotic to the straight line deŽned by T D D / g1 C gc , where g1 and m 1 satisfy the equation (im 1 C c 1 ) (im 1 C c 2 ) D c 1c 2 g1 U ( im 1 ).

(4.11)

Note that this equation is identical to the spectral equation describing the stability of the AS in one population of neurons. For the parameters of Figure 4, m 1 ¼ 3.82 and g1 ¼ 6.5. The ratio between the frequency of the unstable mode and the average Žring rate of the neurons, R ´ RE D RI (see equation 4.6), is plotted in Figure 5. This ratio is larger than 1 on L3. This means that the unstable mode oscillates faster than the average frequency of the neurons. However R increases by less than 25% on L3. Therefore, the synchrony that emerges is on the timescale of the spikes. In contrast, on L4, R varies from R ¼ 1 near B to R D 0 in the vicinity of D. This means that along L4, the pattern

18

D. Hansel and G. Mato

Figure 5: The ratio R D m / 2p º versus the determinant of the coupling matrix, D, on lines L3 (dashed line) and L4 (dashed-dotted line) shown in Figure 4. Points B and D correspond to B and D shown in Figure 4.

of synchrony changes continuously from spike-to-spike synchrony to burst synchrony in which the neurons have the time to Žre several spikes on average during one period of oscillation of the unstable mode. The phase lag, d, on L3 and L4 can be expressed analytically as a function of D , T , and gEE . From equation 4.1 one Žnds that ( im C c 1 ) (im C c 2 ) D

p c 1c 2 U (im ) ( T § T 2 ¡ 4D ). 2

(4.12)

Analyzing the numerical solutions of the spectral equation, one Žnds that it is negative on L4 and positive on L3. Substitution in equation 3.25 gives for the phase lag the equation p =m ( T 2 ¡ 4 D ) d D § arctan , gEE ¡ gII

(4.13)

where the positive (resp. negative) sign is for the phase lag on L4 (resp. on L3). The condition T 2 ¡ 4D D 0 deŽnes a line in the phase diagram to which B, D and the point T D D D 0 belong. Therefore, T 2 ¡ 4D < 0 on L3, except in B and D where T 2 ¡ 4D D 0 (see deŽnition of points B and D). Since gEE ¡ gII > 0, cosd > 0.

(4.14)

The phase lag, d, is a function of the three variables gEE , gII , and gEI gIE and not only the reduced variables T and D . In contrast to what happens on L2,

Asynchronous States and the Emergence of Synchrony

19 A

1.6

d

1.2 0.8 0.4 0 -8

0

-4 g

4 g

II

8

EE

B

0

d

-0.4 -0.8 -1.2 -1.6 -8

-4 g

II

0

4 g EE

8

Figure 6: Phase lag of the population oscillations. Solid lines were obtained from equation 3.25. The squares are results from numerical simulations of a twopopulation network with NE D NI D 1600. The positive (resp. negative) part of the x-axis corresponds to variation of gEE with gII D 0 (resp. gII with gEE D 0). (A) In the vicinity of L3. The simulations were done keeping a Žxed distance equal to 1 from L3 and with gIE D ¡gEI . The positive (resp. negative) part of the x-axis corresponds to variation of gEE (resp. gII ). The standard deviation of the current distribution is s D 0.1. Other parameters are as in Figure 4. (B) In the vicinity of L4. The simulations were done keeping a Žxed distance equal to 2 from L4 and with gIE D ¡gEI . The standard deviation of the current distribution is s D 0.4. Other parameters are as in Figure 4.

on L3 and L4, it changes continuously and nonmonotonically. It vanishes in B and D. This is shown, for example, in Figure 6A, where d is plotted against gEE (with gII D 0) along the two-line segment L3 and L4 and against gII (with gEE D 0) for the same two lines. Equation 4.14 implies that in general, |d| < p / 2. In particular, the activity of the two populations can never be in antiphase. From equation 4.13, one

20

D. Hansel and G. Mato

Figure 7: Phase diagram of a two-population QIF network in the symmetric case. The determinant and the trace of the coupling matrix are D D gEE gII ¡gEI gIE and T D gEE C gII , respectively. The standard deviation of the current distribution is sE D sI D 0.4. Other parameters are as in Figure 4. In the gray region, the AS is stable. See the text for an explanation of the different lines. The dotted line is the limit of L4 for very large synaptic time constants (see section 5.3).

sees that the maximum phase lag at the onset of instability, |d| D p / 2, is achieved only for gEE D gII D 0 (d D ¡p / 2 on L3 and d D p / 2 on L4). Finally, on line L3, in the limit of very large D and T , the phase lag returns to 0, as can be seen in equation 3.25 using equation 4.11. These results are also conŽrmed in the example of Figure 6B. The two line segments, L3 and L4, intersect at point C where D ( C) ¼ 7.9 and T ( C) ¼ 4.3. Note that m has different values on the two lines at C, as can be seen from Figure 5. Therefore, at C, the AS is unstable through a double-Hopf bifurcation (Kuznetsov, 1998). Whether L3 or L4 is the border of stability of the AS depends on T . This is shown in Figure 4 for T > T ( C) . The AS loses stability on L4, whereas for T < T ( C) , this occurs on L3. 4.2 The Effect of Heterogeneities. We now consider how the phase diagram for the stability of the AS is modiŽed when the level of heterogeneities varies. Figure 7 shows the phase diagram in the symmetric case for sE D sI D 0.4, with all other parameters the same as in Figure 4. Comparing these two Žgures, one sees that in spite of a signiŽcant increase in the level of heterogeneities, the transition line (L1) has moved only slightly. In contrast, L2 has substantially moved toward more negative values of T and D . This can be easily understood because to compensate for the increased desynchronizing effect of the heterogeneities, stronger synapses are required to destabilize the AS. Line segment L4 is barely modiŽed in the neighborhood of D, its crossing point with L1. A stronger effect occurs in the vicinity of

Asynchronous States and the Emergence of Synchrony

21

Figure 8: The ratio R D m / 2p º versus the trace of the coupling matrix, T , on line L4 shown in Figure 7. Points B and D correspond to B and D shown in Figure 7.

B, which moves this point toward more negative values of T and D . Most of the changes in line segment L4 when s increases can be explained by this stronger effect. A more dramatic change occurs for the line segment L3. Indeed, when the heterogeneities are strong enough, L3 moves completely to the right of L4. Moreover, on L4, T increases very slowly with D . Therefore, all L3 becomes located well into the region where the AS is already unstable, and this instability scenario is now irrelevant. As in Figure 4, along line L4, the frequency of the unstable mode depends continuously on the location. However, in contrast to that case, L4 is relevant as an instability line all the way between points B and C. In Figure 8, we plotted the ratio of the frequency of the emerging oscillations to the average Žring rate showing the continuous and monotonic variation of this quantity along L4 from a value slightly larger than 1 at B corresponding to spike-tospike synchrony to 0 at D in the vicinity of which the network Žres in synchronous bursts. To characterize further how the instabilities of the AS depend on the level of heterogeneities, we computed the value of D on the four lines L1, L2, L3, L4 as functions of the heterogeneity level, s, for T with all other parameters constant. The results are shown in Figure 9. As already noted from the comparison of Figures 4 and 7, the location of L1 depends only very weakly on the level of the heterogeneities. In contrast, the destabilization of line L2 is much more sensitive to this parameter. In Figure 9B, the line segment L3 is clearly much more sensitive to heterogeneities than line segment L4. Indeed, the coupling strength required on the former line increases much more rapidly with s than on the latter line. A consequence of this difference in sensitivity to heterogeneities is that, as already noted, the two line segments

22

D. Hansel and G. Mato

Figure 9: Determinant of the coupling matrix, D, versus the standard deviation, s on the bifurcation lines L1, L2, L3, L4 for T D 1. The synaptic time constants are as in Figure 4. (A) L1 (solid line) and L2 (long dashed line). (B) L3 (dashed line) and L4 (dashed-dotted line).

L3 and L4 interchange their positions in the (D ,T ) phase diagram when s is large enough. This happens at s ¼ 0.15. For s larger than this value, line L3 is located beyond the instability line L4 in a region where the AS is already unstable. Therefore, in that case only, one instability scenario driven by the E-I-E interaction loop is relevant. 4.3 The Effect of the Firing Rate. For a Žxed architecture and coupling strength, the average Žring rate of the network depends on the external input. To investigate how the stability of the AS depends on this input, we studied the solutions of the spectral equation as a function of the average Žring rate, keeping the variance of the threshold distribution constant, s D 0.1. Figure 10A shows how the minimal coupling strength gc required for the destabilization of the AS in one inhibitory population determined by equa-

Asynchronous States and the Emergence of Synchrony

23

Figure 10: (A) The synaptic coupling, gII , at which the AS becomes unstable in one inhibitory population versus the average Žring rate of the neurons. The standard deviation of the input distribution is s D 0.1. The synaptic time constants are as in Figure 4. (B) The determinant, D, on the bifurcation lines L3 (dashed line) and L4 (dashed-dotted line) versus the average Žring rate of the neurons in a two-population network (symmetric case) for T D 1. Other parameters are as in Figure 4.

tion 4.4 varies with the average Žring rate. One Žnds that gc is a nonmonotonic function of the average Žring rate. The critical coupling is minimum (gc D ¡0.98) for a Žring rate of º D 50 spikes per second. Note, however, that the extremum of the curve is very shallow. Already for gc D ¡1.5, the AS is unstable in a broad range of average Žring rates. Figure 10B plots D c , the value of D at the instability onset on L3, and L4, for T D 1. This Žgure shows that on L3, Dc decreases monotonically with increasing Žring rate. Moreover, this instability disappears for º < ºmin D 20 spikes per second at which value Dc diverges. In contrast, on L4, Dc is a nonmonotonic function of the average Žring rate and exists at all values of the Žring rate. The two lines intersect for º ¼ 40 spikes per second. Below

24

D. Hansel and G. Mato

this value of the Žring rate, the relevant instability of the AS is on L3, whereas above this value, it is on L4. 4.4 Summary of the Results in the Symmetric Case. Our analysis of the symmetric case reveals that in a two-population heterogeneous network, the AS can lose stability in four ways. Two of these are not speciŽc to two-population networks and are an extension in this framework of instabilities that already exist for one population: the rate instability that occurs on L1 and the spike-to-spike synchrony instability that occurs on L2. At the onset of the former type of instability, which depends only weakly on heterogeneities, no synchrony occurs. In the latter case, which is sensitive to heterogeneities, at the onset of the instability, activities in the two populations are synchronized on the timescale of the spikes and oscillate in phase. The two other types of instability that occur on L3 and L4 are speciŽc to two-population networks. On L3, the activities in the two populations oscillate at a frequency on the order of the average Žring rate, and the inhibitory population is oscillating ahead of the excitatory population. On L4, the frequency of the population activity oscillations can become very slow. This happens if both the recurrent excitation and the recurrent inhibition (the feedback between the two populations) are strong. On L4, the oscillation of the excitatory population is ahead of the oscillation of the inhibitory population. Finally, L3 and L4 differ in the way they depend on the level of heterogeneities. L3 becomes irrelevant if the heterogeneities are large or the average Žring rates are small. 5 Synchrony in Two-Population Networks: The General Case If the distributions of the Žring rates of the two populations are different in the AS or the synaptic time constants of the excitation and inhibition are not the same, the stability of the AS (at given Žring-rate distributions) depends on three synaptic coupling parameters gEE , gII , gEI gIE and also on the synaptic time constants. 5.1 The Effect of the Synaptic Coupling Strength. Figure 11A displays the phase diagram of the AS in the plane ¡gEI gIE , gEE when gII D 0. The synaptic time constants are t1E D 1 msec, t2E D 3 msec, t1I D 1 msec, and t2I D 6 msec. The average Žring rate of the excitatory (resp. inhibitory) population is 20 Hz (resp. 40 Hz), and sE D sI D 0.2 for both populations. As in the symmetric case, if the recurrent excitation is strong enough, SN instability occurs. In the plane ¡gEI gIE , gEE , this instability occurs on a straight line, the equation of which is

gEE D

1 ¡ gEI gIE UI (0) , UE (0)

(5.1)

Asynchronous States and the Emergence of Synchrony

25

Figure 11: Phase diagram in the asymmetric case. The average Žring rate of the excitatory (resp. inhibitory) population is ºE D 20 Hz (resp. ºI D 40 Hz). The synaptic time constants are t1E D t1I D 1 msec, t2E D 3 msec, t2I D 6 msec. The standard deviations of the input distributions are sE D sI D 0.2. The AS is stable in the gray region. In this Žgure, only the physical branches of the instability lines are drawn. In particular, L3, which branches at B, is not drawn. (A) gII D 0. A saddle-node bifurcation occurs on L1 (solid line) and a Hopf bifurcation on L4 (dashed-dotted line). (B) gEE D 0. Hopf bifurcations occur on L2 (long-dashed line) and L4 (dashed-dotted line). The difference between these two bifurcation lines is explained in the text. At E, the instability of the AS is due to the interactions between the two populations since gEE D gII D 0 at that point.

26

D. Hansel and G. Mato

where Ua (0) , a D E, I is given by equation 4.3, replacing the distribution P by Pa , the Žring-rate distribution in population a. For the parameters of the Žgure, one Žnds UE (0) D 0.53 and UI (0) D 0.4. When gEE increases, this instability can be prevented by increasing the negative feedback between the two populations measured by gEI gIE < 0. When this feedback is strong enough, the AS becomes unstable through a Hopf bifurcation, which occurs on the line to the right of the phase diagram. On this line, the period of the population rhythm varies continuously from m ¼ 1.25 at its intersection with the x-axis, E, (gEE ( E ) D 0) to m D 0 at D. The value of m at E corresponds to a pattern in which the inhibitory (resp. excitatory) neurons Žre approximately two spikes (resp. one spike) per period of the population oscillation. Near D, it corresponds to a pattern in which neurons tend to Žre synchronous bursts with many spikes. The properties of the instability on this line are similar to those found on line L4 in the phase diagrams of the symmetric case. Because of these similarities, we have denoted this line by L4 here as well. Figure 11B displays the phase diagram of the model in the plane ¡gEI gIE , gII when gEE D 0. The other parameters are the same as in Figure 11A. The region of stability of the AS is bounded by two lines. One, L2, corresponds to the destabilization of the AS when gII is strong enough. Like line L2 in Figures 4 and 7, this line corresponds to the emergence of synchrony through the I-I interaction. In contrast to the symmetric case, here m / 2p and d are not constant on L2. However, m / 2p always remains just slightly smaller, by less than 4%, than the average Žring rate of the inhibitory population, and d is always smaller than 15% of the total period. Another difference with the symmetric case is that L2 is not straight anymore. However, it deviates only slightly from a straight line. In similarity with the symmetric case, the slope of L2 is negative. This means that the feedback between the two populations plays against synchrony through the I-I coupling. A stronger I-I coupling is required to compensate for this feedback. For strong enough feedback, gEI gIE , the AS can lose stability even if gII is small. This transition occurs on the line to the right in Figure 11B. On this line, the population frequency varies signiŽcantly. Indeed, at B, m / 2p ¼ 40 Hz, and it is two times smaller at point E. Embedded into the three-dimensional space, (gEE , gII , gEI gIE ), this line connects to L4 from Figure 11A. We also denote this line by L4 since the instabilities occurring on these two lines can be thought of as corresponding to a similar synchrony mechanism. Note that the slope of L4 in Figure 11B is negative. In other words, increasing gII plays against the destabilization of the AS through this mechanism. 5.2 The Effect of Synaptic Time Constants. In this section, we analyze the effect of synaptic kinetics on the stability of the AS for Žxed distributions of Žring rates. First, we focus on the effect of the decay times.

Asynchronous States and the Emergence of Synchrony 0.02

g2E(1/msec)

A

0.01

0.00 0.0

AS

0.1

0.2

g2I(1/msec)

g2E(1/msec)

0.8

27

B

0.6

AS

0.4 0.2 0.0 0.0

0.1

0.2

g2I(1/msec)

Figure 12: Phase diagrams in the plane c 2I ´ 1 / t2I , c 2E ´ 1 / t2I for two different strengths of the I-I interactions. The average Žring rate of the excitatory and inhibitory populations are ºE D 20 Hz and ºI D 30 Hz, respectively. (A) gII D 0. (B) gII D ¡7.4, Other parameters are t1E D t1I D 1 msec, gEE D 6, gEI gIE D ¡36 and sE D sI D 0.2. In the gray region, the AS is stable. On L2 (long-dashed line) and L4 (dashed-dotted line), Hopf bifurcations occur. Along L4, the frequency of the unstable mode m decreases with c 2I . For instance, in A, m / 2p varies from 11 Hz for c 2I D 0.3 msec ¡1 to 0 for c 2I D 0. Along L2, m remains Žnite and, its variation is smaller.

Figure 12 displays the phase diagram in the (c 2I ,c 2E ) plane for gEE D 6, gEI gIE D ¡36 and two different values of gII . We consider Žrst the case where gII D 0. In the limit t2E ! 1, the AS is always stable. This is because in that limit, the excitatory input to the inhibitory population is not modulated in time. Therefore, it is equivalent to a positive constant current that excites the inhibitory population. The only way synchrony can occur in this case is through the I-I interactions, which we have assumed to be zero. For Žnite t2E , the AS is unstable provided t2I is sufŽciently large. This deŽnes a transition line c 2E D f (c 2I ) where function f (c ) is an increasing function that vanishes linearly when c ! 0. On this line, which is plotted in Figure 12A, m is a decreasing continuous

28

D. Hansel and G. Mato

function of c 2I . It goes to 0 in the limit c 2I ! 0. Since the decay of the inhibition cannot be faster than its rise time (see equation 2.11), m cannot be larger than m D 0.79, its value for t2I D 1 msec, on the transition line in Figure 12A. Therefore, all along the line, the transition leads to a synchronous bursting state. The bursts are short for fast inhibition and long when the inhibition is slow. In Figure 12A, the AS stabilizes for fast inhibition and slow enough excitation. In particular, the fastest excitation compatible with a stable AS (attained for very fast inhibition with t1I D t2I D 1 msec) is t2E ¼ 50 msec, in the range of NMDA synaptic time constants. The phase diagram in the (c 2I ,c 2E ) plane remains qualitatively the same as in Figure 12A, provided that gII is sufŽciently small. If gII is large enough, a second instability line appears in the region of the phase diagram corresponding to fast inhibition and slow excitation. This is because for a large enough gII , the I-I interaction can induce spike-to-spike synchrony in the inhibitory population. This situation is shown in Figure 12B. Note that the transition line found for gII D 0 (see Figure 12A) is still present but has been shifted to much higher values of c 2E . In contrast to what happens for small values of gII , when the inhibition is fast (t2I < 9 msec), the AS is destabilized for slow enough excitation. For instance, for t2I D 6 msec, this happens for t2E larger than 20 msec. Note that for the parameters of Figure 12B, the AS is stable if the excitatory and inhibitory synapses are in a range corresponding to AMPA and GABAA synapses, respectively. 5.3 The Limit of Slow Synapses. The spectral equation, equation 3.16, can be simpliŽed if one assumes that the synapses have inŽnitely long decay time constants. We assume that c 2E D gcN2E and c 2I D gcN2I where g ¿ 1 and cN2E , cN2I are Žnite and we make the ansatz l D glN .

(5.2)

Substituting in equation 3.16, expanding at the leading order in g, and usR1 ing the fact that 0 dyK ( y ) D F0 (F ¡1 (º) ) , one Žnds that at the onset of an instability lN D imN with mN solution of the equation, ( ( imN C cN2E ) ¡cN2E gEE UE ) ( ( imN C cN2I ) ¡cN2I gII UI ) D gEI gIEcN2EcN2I UE UI ,

(5.3)

where the quantities Z UN a D

1 0

dºP a (º) F0 (F ¡1 (º) )

do not depend on mN .

(5.4)

Asynchronous States and the Emergence of Synchrony

29

Equation 5.3 is quadratic in mN . For mN D 0, one recovers the condition for the SN bifurcation obtained above (line L1). Another transition line is 6 found for mN D 0 on which synchronous oscillations emerge and have a low frequency, m / g ¿ 1. During one period of these oscillations, the neurons burst synchronously since they have the time to Žre several spikes in one oscillation of the populations. If one assumes c 2E D c 2I D c , it is convenient to deŽne X D gEE UE C gII UI and Y D D UI UE . One Žnds that an instability to oscillations occurs on the line X D 2 in the ( X, Y ) plane and that m / g ¿ 1 varies continuously on this line. Therefore, sufŽciently strong gEE and not too large gII are required for burst synchrony. More 6 c 2I , this onset of slow bursting occurs on a surface generally, when c 2E D in the three-dimensional space ( gEE , gII , ¡gEI gIE ) . One also Žnds here that a sufŽciently strong gEE is required for this instability. In general, one also Žnds that increasing gII prevents the emergence of synchronous bursting. Equation 5.3 is also the spectral equation for the stability of the Žxedpoint solutions of the system of two coupled equations, 1 dºE D ¡ºE C GE ( gEEºE C gEIºI ) cN2E dt 1 dºI D ¡ºI C GI ( gIEºE C gIIºI ) , cN2I dt

(5.5) (5.6)

where the functions Ga ( hE , hI ) are Z Ga ( x) D

1 ¡1

dIQ a (I ) F ( I C x ).

(5.7)

Let us consider a two-population rate model in which the neurons have an input-output transfer function F (x ) and receive a random external input with probability density function Qa (I ) (a D E, I). In this model, the Žring rate of neuron i in population a, ºia, follows the equation 1 dºia syn D ¡ºia C F ( Iia C Iia ) , cN2a dt syn

(5.8)

where the total synaptic current, Iia , is given by equation 3.1 and Iia is its external input. It is easy to check that the dynamics of this model can be reduced to two effective degrees of freedom: the average Žring rate in the two populations, ºa ( t) , a D E, I, which satisfy equations 5.5 and 5.6. These two equations are nothing other than a two-population mean-Želd model with transfer function Ga (x ) given by equation 5.7. An example of such a function is plotted in Figure 13. Therefore, in the limit of very slow synaptic decay, the instabilities of our spiking model are the same as those of a rate model (see also Ermentrout, 1994; Bressloff & Coombes, 2000).

30

D. Hansel and G. Mato

n (Hz)

150

100

50

0

1

0

1 2 I (mA/cm2 )

3

Figure 13: Input-output transfer function of the rate model that corresponds to the QIF model for s D 0.4. Other parameters as in Figure 1.

Finally, one can show that if the rise time of the synapses is also very slow and on the same order of magnitude as their decay time, the quadratic equation for m , equation 5.5, is replaced by a quartic equation. This situation is similar to the previous one except that now the instability to burst synchrony does not require mutual interactions between the excitatory neurons. Here too the spectral equation can be derived from the mean-Želd equations of a rate model. However, now it is deŽned by four coupled equations—two equations for each of the two populations. 5.4 The Effect of the Average Firing Rates. The effect of the Žring rates of the two populations on the stability of the AS has been studied above in the symmetric case in which the average Žring rates of the two populations were necessarily equal. Here, we consider the case in which the distributions of Žring rates in the two populations are different to understand how this affects the stability of the AS. In particular, we want to investigate whether some speciŽc relationship (“resonance”) between the average Žring rates in the two populations is favorable to the emergence of synchrony in the network. The SN instability (m D 0) depends on the Žring rates through UE (0) and UI (0). The maximal value of the recurrent excitatory feedback, gEE , is given by gEE D

1 UI (0) C |gEI | gIE . 1 C | gII |UI (0) UE (0)

(5.9)

The quantities Ua (0), a D E, I are decreasing functions of the Žring rate. For large Žring rates, they converge to a Žnite value. Therefore, increasing

Asynchronous States and the Emergence of Synchrony

31

the Žring rate of the excitatory population or decreasing the Žring rate of the inhibitory population increases gEE (i.e., stabilizes the AS with respect to the SN instability). In particular, the maximal value of gEE at this onset of instability, when ºI changes, is 1 / UE (0) C | gEI | gIE / gII . It is reached in the limit of very small Žring rates of the inhibitory neurons. This is because the sensitivity of the neurons to changes in their input increases when their Žring rate decreases. Therefore, the inhibitory neurons are more efŽcient at low Žring rates than at high Žring rates at preventing the runaway of the activity of the excitatory neurons due to the excitatory feedback. In the symmetric case, we have found that the interaction with an excitatory population stabilizes the AS with respect to the spike-to-spike instability generated by the I-I interactions. Figure 11B provides an example that this remains true for the general case. In order to gain more insight into this property, we computed the value of gII on L2 for different values of ºE and ºI with all the other parameters of the model held constant. The result for ºI D 50 Hz is plotted in Figure 14A; in the limit, ºE ! 0, gII ! gc , where gc is the critical value of the I-I coupling at which the AS would lose stability in the inhibitory population if it were isolated from the excitatory population. This is expected since in this limit, the excitatory population has no effect because it is inactive. In the limit of large ºE , also, gII ! gc . This is because in this limit, the modulation of the excitatory drive on the inhibitory population becomes small, and therefore the excitatory population has no dynamical effect on the inhibitory population. For intermediate values of the excitatory Žring rate, gII is a nonmonotonic function of ºE . The minimum value is reached for ºE ¼ ºI . It is about 25% more negative than gc . Note that the position and the maximum of gII depend on ºI . As a rule, we have found that when ºI is large, this maximum is more pronounced, and it is located around a value of ºE < ºI as depicted in Figure 14A. These results show that in general, the feedback between the two populations disrupts the emergence of synchrony through the mutual interactions between the inhibitory neurons unless the average frequencies of the two populations differ sufŽciently. The frequencies of the two populations affect the instability on L4 in a different way, as shown in Figure 14B. Here, we have plotted the critical value of the feedback, ¡gEI gIE , between the two populations versus the frequencies ºE and ºI for gEE D gII D 0. The feedback required to destabilize the AS is minimum, around ºE D ºI ¼ 27.5 Hz. Note that this minimum is deep but that its curvature is small. Therefore, in contrast to what happens on L2, synchrony is favored on L4 if the average frequencies in the two populations are similar. The behavior for line L3 is similar to the one found for L4. The critical value of the feedback is smaller when the two populations have the same Žring rate. However, in this case, the system is much more sensitive to the difference in the Žring rates. For instance, a factor of 2 between ºE and ºI generates an increase of a factor of 10 in the critical

32

D. Hansel and G. Mato

A

-2.4 -2.6 -2.8 -3 g

II

-3.2 -3.4 -3.6 -3.8

0

20

40 60 n (Hz)

80

100

E

B -g g

IE EI

24 16 8 0 0

20 n E (Hz)

40

60 0

20

40

60

80

n (Hz) I

Figure 14: (A) The strength of the mutual inhibition, gII , for which the AS becomes unstable on line L2 versus the average frequency of the excitatory population, ºE . The average frequency of the inhibitory population is ºI D 50 Hz. Other parameters are gEE D 2 and gEI gIE D ¡4. (B) ¡gEI gIE versus the average Žring rates of the two populations for which a Hopf bifurcation (L4) occurs. Here, gEE D gII D 0. In A and B, the synaptic time constants are t1E D t1I D 1 msec, t2E D 3 msec, t2I D 6 msec and the standard deviations of the input distributions are sE D sI D 0.2.

¡gEI gIE (data not shown here), while the change on L4 is less than 100%. This behavior is similar to the one found in the previous section: line L3 is a much less robust transition than line L4.

Asynchronous States and the Emergence of Synchrony

33

6 Numerical Simulations The study of the solutions of equation 3.16 has allowed us to determine in the parameter space the AS stability region and the different ways this state can become unstable. The goal of this section is to complete the characterization of these instabilities and the patterns of synchrony that emerge beyond their onset. We focus on the following questions: • Are the transitions at the onset of instabilities sub- or supercritical when they occur through a Hopf bifurcation? • To what extent is the imaginary part of the unstable eigenvalue on the instability lines a good estimator of the frequency of the population in the stable oscillatory state in which the network settles beyond but close to these lines? Is the phase lag, d, computed from the linearized dynamics, a good estimator for the actual phase lag between the oscillations of the activities in the two populations in the stable state near the instability onset? The answer to these two questions depends critically on the nature of the Hopf bifurcation. • Which one of the excitatory or inhibitory populations is more synchronized near onset of synchrony? How can their relative level of synchrony be controlled? In principle, the answer to the Žrst question can be determined analytically by expanding the dynamics beyond the linear order. However, this calculation is extremely involved and is beyond the scope of this article. Therefore, to clarify all these issues, we relied on numerical simulations. These were performed by integrating the dynamics of the network using a second-order Runge-Kutta scheme with a Žxed time step, dt D 0.1 msec. This algorithm was supplemented by an interpolation for the determination of the Žring times of the neurons (Hansel, Mato, Neltner, & Meunier, 1998; Shelley & Tao, 2001). 6.1 The Nature of the Hopf Bifurcations. In a subcritical Hopf bifurcation, the stable Žxed point, a stable limit cycle, and an unstable limit cycle can coexist in some range of the control parameter near the onset of bifurcation. This does not happen if the bifurcation is supercritical. One can use this criterion to characterize the nature of a Hopf bifurcation in numerical simulations by checking whether the system displays a hysteresis when the control parameter is changed continuously from a value where only the Žxed point exists up to values where only the limit cycle is stable and backward. We performed numerical simulations in the vicinity of the three lines L2, L3, and L4. The state of the network was characterized by computing the synchrony level  in the two populations, as explained in appendix C. Simulations with different network sizes were compared to check that syn-

34

D. Hansel and G. Mato

chrony was not due to a Žnite-size effect. An example of the results of this study is shown in Figures 6.1A through 15C for s D 0.1 and network size NE D NI D 12, 560 near instability points chosen on L2, L3, and L4, respectively. No hysteresis is apparent in the vicinity of the instability points on L2 (see Figure 15A). Although a small hysteresis cannot be completely excluded, our simulations were unable to detect it. In contrast, near L3 and L4,

Asynchronous States and the Emergence of Synchrony

35

the network displays some hysteresis (see Figures 15B and 15C). However, this is a small effect since the range of coupling strength in which it occurs, as well as the value of the synchrony parameters, Â, on the left branches of the hysteresis, are small. When s increases, a hysteresis also appears near the instability points on L2. This effect can be seen by comparing Figures 15A and 15D, which were obtained for s D 0.1 and s D 0.4, respectively. However, this is a small effect, which can hardly be distinguished from sample-to-sample uctuations if the network size is too small (result not shown). Near L4, for this value of s, the hysteresis has shrunk and cannot be seen. 6.2 The Frequency of the Synchronous Oscillations and the Phase Lag Between the Two Populations. Our simulations indicate that a supercritical Hopf bifurcation occurs on L4 for large levels of heterogeneities. Therefore, one expects that the pattern of synchrony should be correctly predicted by our linear stability analysis near the bifurcation. This is conŽrmed in Figure 16A, where simulation results for the frequency of the population oscillations near L4 are plotted for the parameters of the phase diagram in Figure 7. The simulations were performed along a line parallel to line L4, and the results are compared with the value predicted from a linear stability analysis. In Figure 17A, the potentials of one excitatory neuron and one inhibitory neuron are plotted for T D 5.25, D D 8. On this point, the global frequency is much smaller than the Žring rate of the neurons. This means that the system must be bursting, as observed in the simulation.

Figure 15: Facing page. The level of synchrony in the excitatory population close to the Hopf bifurcations lines in the symmetric case. For each value of the bifurcation control parameter (corresponding to the x-axis), the network was simulated during 60,000 time steps. At the end of the run, the Žnal state of the network was kept and used as initial conditions to run a new simulation after the control parameter was incremented. In each Žgure, two lines are shown. One corresponds to a sweep positive increment and the other to a negative increment. The standard deviation of the input is s D 0.1 (A) The strength of the recurrent excitation is Žxed gEE D 2 and gII was changed. For each value of gII , gEI gEI is varied to keep D D 0. Predicted transition on L2 at T D ¡1.11 (vertical arrow). (B) The E-E and I-I couplings are Žxed: gEE D gII D 0. The E-I and I-E coupling are varied with gEI D ¡gIE . Predicted transition on line L3 at D D 1.98 (vertical arrow). (C) The E-E and the I-I interactions are Žxed, gEE D 5, gII D 0 and gEI and gIE are changed simultaneously with gEI D ¡gIE . Predicted transition on L4 at D D 7.9 (vertical arrow). (D) gII and gEI , gIE are changed such that gEI D ¡gIE and D D ¡1. Predicted transition on L2 at T D ¡4.2 (vertical arrow). (E) The E-E and the I-I interactions are Žxed: gEE D gII D 0 and gEI and gIE are changed simultaneously with gEI D ¡gIE . Predicted transition on L4 at D D 8.11.

36

D. Hansel and G. Mato

Figure 16: The frequency of the population activity oscillations near the onset of instability of the AS. The lines represent the predicted frequency from the solution of the spectral equation, equation 3.16. The squares are the results from simulations. A network of NE D NI D 1600 was simulated during 60,000 time steps and the frequency oscillation was obtained by evaluating the autocorrelation of the average membrane potential. (A) Parameters as in Figure 7. The simulations were done near L4 at distance 2 away from L4 with gIE D ¡gEI and keeping gII D ¡2. (B) Parameters as in Figure 4. The simulations were done at a distance 1 away from line L3 with gIE D ¡gEI and keeping gII D ¡2. (C) Parameters of Figure 7. The simulations were done at a distance 2 away from line L2 varying gIE D ¡gEI and keeping gEE D 0.

The Hopf bifurcations that occur on L3 (for small levels of heterogeneities) as well as on L2 if the level of heterogeneities is sufŽcient were found to be subcritical. Therefore, one expects that the actual oscillation frequencies and phase lags near these lines should deviate from the values obtained for the

Asynchronous States and the Emergence of Synchrony

37

A

20 mV

0

100 200 300 400 500 t (msec) B

10 mV

0

100 200 300 400 500 t(msec)

Figure 17: Synchronous oscillations in which neurons Žre bursts of spikes. Parameters as in Figure 7 with T D 5.25 and D D 8. (A) Excitatory neuron (lower trace) and inhibitory neuron (upper trace). (B) The membrane potential of the excitatory (resp. inhibitory) neurons averaged over all the population (lower line, resp. upper line).

unstable mode in the linear stability analysis. In spite of this, a reasonable agreement between the predictions and the simulation values was found as shown in Figures 16B and 16C. This is because the hysteresis near L2 has a small range and a small amplitude (see, e.g., Figure 15B). In Figure 6, the phase lag between the oscillations of the activity of the two populations is plotted as a function of gEE for gII D 0 (A) and as a function of gII for gEE D 0 (B). The solid and dashed lines correspond to the predictions from our analytical results. Results from numerical simulations

38

D. Hansel and G. Mato

performed in the vicinity of the instability lines are represented by squares. They were estimated from the position of the cross-correlation peak of the activities of the two populations. In agreement with the linear theory, the phase lag on L3 and L4 are nonmonotonic functions of the coupling. For instance, on line L4, it reaches a maximum of p / 2 when gEE D gII D 0 and then decreases as one approaches points B or D in the phase diagram. The negative sign of the phase lag means that the oscillation of the activity of the excitatory population is ahead of the inhibitory one. Note that on L3, the lag has the opposite sign, that is, the inhibitory population oscillates ahead of the excitatory one. On line L4 near D, the linear theory predicts that the phase lag should become small and vanish at D. This is conŽrmed by our numerical simulations, as shown in Figure 17B. Finally, the linear theory predicts that at the instability onset on L2, the phase lag should be zero. Therefore, d is expected to be small near L2 when synchrony occurs. Our simulations are in line with this prediction. 6.3 The Level of Synchrony in the Two Populations. For strong enough gII , the activity of the inhibitory neurons can synchronize as a result of their mutual coupling. If the feedback between the two populations, ¡gEI gIE is weak, the dynamical state of the inhibitory population should not be perturbed much. However, the excitatory population is now entrained by the inhibitory one. Therefore, it develops synchrony. The level of synchrony in the excitatory population depends on the coupling gEI . For small gEI , one expects the excitatory population to be weakly synchronized, and the level of synchrony should increase with gEI . These expectations were conŽrmed by our numerical simulations. Moreover, under these conditions, the inhibitory population was always substantially more synchronized than the excitatory one (results not shown). More generally, we found this property to be true in the vicinity of L2 even for large values of ¡gEI gIE . This can be seen in Figure 18, where the synchrony measures, ÂE and ÂI , are plotted against ¡gIE gEI , for gEI D ¡gIE / 4 (A) and for gEI D ¡4gIE (B). In both cases, the transition point on L2 occurs for the same value of ¡gIE gEI , namely, ¡gIE gEI ¼ 3.25. This agrees with the fact that the spectral equation depends on only gEI and gIE through their product. However, the level of synchrony in the two populations also depends on the ratio gEI / gIE . On both curves, the inhibitory population is always more synchronized than the excitatory one. The synchrony in the excitatory population varies nonmonotonically with gEI gIE , with an intermediate region where the AS is stable. This is because the feedback between the two populations has two opposing effects. On the one hand, it drives the excitatory population, and on the other, it disrupts the synchrony induced by the mutual I-I interactions in the inhibitory population. Finally, increasing gEI (for Žxed gEI gIE ) increases ÂE . This is because the larger the | gEI |, the stronger is the inhibitory drive on the excitatory population.

Asynchronous States and the Emergence of Synchrony

39 A

0.5 0.4 0.3 c

L2

0.2

L4

0.1 0

1

3

5 -g

7

g

EI IE

B

0.5 0.4

c

0.3 L2

0.2

L4

0.1 0

1

3

5 -g

EI

g

7

IE

Figure 18: The synchrony level of the excitatory (solid line) and the inhibitory populations (dashed line) versus ¡gEI gIE . The strength of the E-E and I-I interactions are Žxed: gEE D 2, gII D ¡2. Other parameters ºE D ºI D 50 Hz, sE D sI D 0.2. A network with NE D NI D 1600 neurons was simulated for different values of gEI and a Žxed ratio gEI / gIE . (A) gEI / gIE D ¡1 / 4. (B) gEI / gIE D ¡4. The level of synchrony was calculated as explained in appendix C (average over 6 seconds). The theory predicts that the AS becomes unstable at ¡gEI gIE D 3.25 (on L2) and ¡gEI gIE D 5.7 (on L4) as indicated by the arrows.

In all our numerical simulations, we found that in the vicinity of L2, the level of synchrony is larger in the inhibitory population than in the excitatory one. This is in contrast with what happens near L3. This is shown in Figure 19. Here, for a given value of ¡gEI gIE , the relative level of synchrony depends strongly on the balance between gEI and gIE . For large gEI and small gIE , the synchrony in the excitatory population is higher than in the inhibitory one. The opposite happens if gIE is large and | gEI | is small. The same Žgure also shows that the synchrony levels near line L4 are affected in the same way by the balance between gEI and gIE . Making the excitatory to

40

D. Hansel and G. Mato A

0.5 0.4 0.3 c

L2

0.2

L3

0.1 0

1

3

5 -g

EI

g

7

IE

B

0.5 0.4 0.3 L3

c

L2

0.2 0.1 0

1

3

5

7

-g g

EI EI

Figure 19: The synchrony level of the excitatory (solid line) and the inhibitory populations (dashed line) vs. ¡gEI gIE . The strength of the E-E and I-I interactions is Žxed: gEE D 2, gII D ¡2. Other parameters ºE D ºI D 50 Hz, sE D sI D 0.1. A network with NE D NI D 1600 neurons was simulated for different values of gEI and a Žxed ratio gEI / gIE . (A) gEI / gIE D ¡1 / 4 (B) gEI / gIE D ¡4. The level of synchrony of the two populations was calculated as explained in appendix C (average over 6 seconds). The theory predicts that the AS becomes unstable at ¡gEI gIE D 2.96 (on L2) and ¡gEI gIE D 5.42 (on L3) as indicated by the arrows.

inhibitory interaction stronger makes the inhibitory population more synchronized, and vice versa. 7 Discussion In this work, we have analyzed the synchronization properties of a fully connected two-population system. We have not intended to present a complete model of the cortex, but a simple network model that is amenable to analytical treatment. A natural but nontrivial extension of our work would

Asynchronous States and the Emergence of Synchrony

41

be to study the stability of asynchronous states in networks with more structured patterns of connectivity such as the ones found in the visual cortex, which include local or intracolumnar connections between neurons with similar feature preference, vertical connections between different layers, and feedback connections from higher cortical areas. For instance, one expects that in a network with space-dependent interactions, instabilities of the AS may lead to traveling waves or other spatiotemporal patterns in which hot spots of activity move across the system (Ben Yshai, Hansel, & Sompolinsky, 1997; Hansel & Sompolinsky, 1998; Golomb, 1998; Golomb & Ermentrout, 1999). 7.1 Three Mechanisms for Synchrony in Two-Population Networks. 7.1.1 The Mutual Inhibition Mechanism. The activity of inhibitory neurons can be synchronized on the timescale of the spikes through their mutual interactions. It has been proposed that hippocampal c rhythms can be driven by populations of inhibitory interneurons whose activity is synchronized through this mechanism (Buzs a ki & Chrobak, 1995). Synchrony in one population of inhibitory neurons has been extensively studied in recent years theoretically (van Vreeswijk, Abbott, & Ermentrout, 1994; Hansel, Mato, & Meunier, 1995; van Vreeswijk, 1996; Wang & Busz a ki, 1996; White et al., 1998; Chow, 1998; Chow, White, Ritt, & Kopell, 1998; Golomb & Hansel, 2000; Neltner et al., 2000; Whittington, Traub, Kopell, Ermentrout, & Buhl, 2000) and experimentally (Whittington, Traub, & Jefferys, 1995; Traub, Whittington, Colling, Buzs Âaki, & Jefferys, 1996; Traub, Jefferys, & Whittington, 1999, for a review). Here, we have generalized this mechanism to take into account the effect of a second excitatory population. When the cross-talk interaction between the two populations is not too large, we have found that the AS becomes unstable if the mutual inhibition is sufŽciently strong. Excitation plays against this destabilization in two ways. For a given value of the feedback between the two populations (Žxed value of | gEI gIE |), the strength of the mutual inhibition gII required to destabilize the AS increases with gEE . Also, for a given gEE , a larger mutual inhibition is required if the feedback between the two populations is increased. If |gEI gIE | is too strong, another mechanism for the destabilization of the AS can occur, as discussed below. These desynchronizing effects of the excitatory population are stronger when the average Žring rates of the two populations are more similar. In the example shown in Figure 14, | gII | is maximal when the Žring rates of the two populations are similar. Therefore, as far as the value of gII is concerned, the excitatory population has a strong inuence. In contrast, we have found that the frequency of the unstable mode at the transition is mainly determined by the average Žring rate of the inhibitory population and depends only weakly on the properties of the excitatory population and on the way the two populations interact. Related to this fact, we have shown that the phase shift between the two populations is very small even

42

D. Hansel and G. Mato

if their average Žring rates are very different. In view of these features, this mechanism can arguably be based on the synchronization of the inhibitory population. Our analysis focuses on the mechanisms for destabilization of the AS. We address this problem by studying local bifurcations of the AS. Therefore, we did not address the issue of rhythm properties under strong synchrony. Strongly synchronized states could coexist with the weakly synchronous states or even the AS. The only way to address this question (and the question of the size of the attraction basins of the stable states) is with extensive numerical simulations. The fact that the frequency of the unstable mode is determined to a large extent by the average Žring rate of the neurons in the AS does not contradict previous results indicating that when the synchrony is substantial, the rhythm frequency can depend on the synaptic time constant of mutual inhibition (Whittington et al., 1995; Chow et al., 1998; White et al., 1998). 7.1.2 The Cross-Talk Mechanisms. We have demonstrated that the AS can be destabilized with two qualitatively different mechanisms by the crosstalk interactions between the two populations. In the Žrst mechanism (on L4), the inhibitory population lags behind the excitatory one. Therefore, in the unstable mode, an increase in activity of the excitatory population makes the inhibitory neurons increase their activity. This generates in return an increase in inhibition of the excitatory neurons, which tends to prevent simultaneous Žring of many of these neurons. SufŽciently strong feedback between the two populations will prevent the perturbation from decaying, and synchrony will emerge. Mutual excitation of excitatory neurons favors the development of this instability since it helps to amplify perturbations in the E population. Mutual inhibition of inhibitory neurons disrupts the development of this instability since it reduces the excitability of this population. In this mechanism, synchrony is induced by recurrent inhibition of the excitatory population. It has been investigated mostly by numerical simulations in previous studies and has been proposed as a mechanism for c oscillations (Jefferys, Traub, & Whittington, 1996). However, as shown here, this mechanism can lead to slow oscillations. The oscillations can be so slow that the neurons develop bursts of action potentials. In the second type of cross-talk instability (on L3), the inhibitory population tends to Žre in advance with respect to the excitatory one, preventing the excitatory neurons from Žring. In return, this deexcitation reduces the activity of the inhibitory neurons. This instability does not decay provided that the feedback between the two populations is sufŽciently strong. The global oscillations near this instability are on the order of or faster than the average Žring rate of the neurons. These two mechanisms are qualitatively different from the mutual inhibition mechanism. Indeed, we have found that the frequency of population

Asynchronous States and the Emergence of Synchrony

43

activity as well as the phase shift between the oscillations in the two populations vary substantially and continuously on the corresponding instability surfaces. Also, the emerging oscillations in the two populations can display a large phase shift, d. However, in all the examples we have studied, the numerical solution of the spectral equation reveals that |d| < p2 . We did not prove this statement in general. However, we have proved that in the symmetric case, |d| D p / 2 on onset instability lines L3 and L4 at gEE D gII D 0. We have also shown that d has a local maximum at that point. Without the assumption of symmetry between the two populations, we have also shown that the oscillations of the two populations become in phase when the emerging rhythm becomes very slow. 7.1.3 Robustness of Synchrony to Heterogeneities. A systematic and comparative study of the robustness of the various modes of destabilization would be a very tedious task. Nevertheless, some general trends emerge from the study of the spectral equation in the symmetric case. The coupling strengths required for destabilization on L2 and L3 depend crucially on the heterogeneity level (see Figure 9). This is in contrast with what happens for L1 and L4 where the dependence is much weaker. Thus, the recurrent inhibition is the most robust mechanism for synchronous oscillations. Another difference between line L3 and the other lines is that for sufŽciently large s, L3 lies completely inside a region, which is beyond instability line L4. Therefore, when the level of heterogeneities increases, this line becomes physically irrelevant. In this sense, the mechanism corresponding to this instability line is much less robust than the other mechanisms. Similar trends were also found in all the examples we investigated after relaxing the symmetry assumption. Recent studies have suggested that because of heterogeneities, synchrony in inhibitory networks can be difŽcult to achieve (White et al. 1998; Chow et al., 1998). Our study provides an example of a model in which the desynchronizing effect of large levels of heterogeneities can be overcome provided that the external excitatory drive and the synaptic interactions are sufŽciently strong. Simulation results in the framework of a conductance-based model were found to be consistent with this conclusion (Neltner et al., 2000). This is in contrast with the behavior of inhibitory networks of leaky integrate-andŽre models where for sufŽciently large level of heterogeneities, the AS is always stable (Golomb et al., 2001). For sufŽciently strong coupling, the AS is stable even if the network is homogeneous. This is yet another difference with the present model. Below, we elaborate further on these differences. 7.2 Getting Rid of Synchrony. The frequent occurrence of synchronous neuronal activity in physiopathological states in the brain (see Bergman et al., 1998; McCormick & Contreras, 2001) suggests that in many situations, synchronous states of activity need to be prevented. The issue is to understand how synchrony can be prevented in particular in systems

44

D. Hansel and G. Mato

where massive feedback interactions are observed. Our analysis shows that even if the interactions are strong, a suitable balance between excitation and inhibition stabilizes the AS. We have proved this statement for our model in full generality and analytically under the assumption of symmetry between the two populations. Indeed, for D D T D 0, the AS is marginally stable without heterogeneities, and it becomes stable when heterogeneities are added. Besides the presence of heterogeneities, this implies that if the excitatory feedback connections, gEE , are strong, the two populations must interact strongly, and the mutual interactions of the inhibitory neurons must be strong to ensure that the AS is stable. In particular, if gII is not strong enough, the system tends to Žre synchronous bursts. Increasing gII reduces the excitability of the inhibitory population, preventing the synchronous bursts from developing. This desynchronizing effect of mutual inhibition also holds in the nonsymmetric case, as shown in Figure 11.

7.3 Application to the Stability of Persistent Activity. Hebb (1949) proposed that the representation of an object in short-term memory consists of the set of cortical cells that this object activates when it is presented. According to this hypothesis, the reverberation of excitation across this set is able to maintain activity even when the stimulus has been removed, thus preserving a memory of the object. In other words, the network displays multistability. The fact that persistent activity requires strong excitatory feedback poses the problem of control of neural activity in persistent states at Žring rates in the range 10 to 30 spikes per second, as observed in experiments. Wang (1999) has argued, mostly on the basis of numerical simulations, that to achieve a stable persistent state with activities in a physiological range (10–50 Hz), recurrent excitatory synapses must be dominated by a slow component. It was subsequently proposed that NMDA synapses were required to achieve such states. Another possibility is that inhibition controls the Žring rate (Rubin and Sompolinsky, 1989). However, recurrent inhibition can also induce synchrony, and this can be incompatible with low rate persistent activity. Indeed, if the neurons Žre synchronously in some short time window, the fast excitatory feedback will decay before the neurons Žre their next spike. Note, however, that weak synchrony may be compatible with persistent activity. The results shown in Figure 12 indicate that strong mutual inhibitory interactions with synaptic time constants in a range corresponding to GABAA synapses stabilize the AS even if the recurrent excitation is mediated by nonsaturating AMPA synapses. Therefore, strong GABAA mutual inhibition should help achieve a stable persistent state. This is in line with the conclusions of our previous study (Hansel & Mato, 2001), in which the threshold distribution had a Žnite support. Here, we show that this is true for a gaussian distribution of threshold without truncation (see Figure 3).

Asynchronous States and the Emergence of Synchrony

45

At Žrst glance, this contradicts the results of Wang (1999). However, I-I interactions were not included in Wang’s study. This may explain why it concluded that stable persistent activity was impossible to achieve with AMPA excitatory synapses and that NMDA synapses were necessary. One should note that recent anatomical and physiological studies have revealed that I-I connections are numerous and strong in the cortex (Sik, Penttonen, Ylinen, & Buzs a ki, 1995; Tarczy-Hornoch, Martin, Jack, & Stratford, 1998; Gupta, Wang, & Markram, 2000). The role we suggest for these connections in the stability of persistent activity Žts nicely with these experimental results, without excluding that NMDA synapses could also contribute to delay period activity in cortex. 7.4 Comparison with Networks of Leaky (Linear) Integrate-and-Fire Networks. In the absence of noise, a very precise Žne tuning of the external input is required to get a LIF neuron to Žre below 8 spikes per second. This is due to the logarithmic behavior of the º-I curve near current threshold. Furthermore, there is an exponential divergence of the function H (º) (the equivalent of equation 3.18 for the leaky integrate-and-Žre neuron; Golomb et al. 2001) when º ! 0 unless P ( x) vanishes at least exponentially fast when x ! 0. In this situation, U also diverges, and the spectral equation is not well deŽned. This divergence is due to the fact that at very low rates, the LIF model spends an exponentially long time near the threshold. This is because leaky integrate-and-Žre neurons integrate their inputs in a purely passive way. As a consequence, if the external currents the neurons receive are distributed according to a gaussian, the resulting distribution of Žring rate does not vanish sufŽciently fast at zero Žring rate for the integrals in H (º) to converge. To avoid this problem, a gaussian distribution of period was chosen in Golomb et al. (2001). In contrast, for the QIF model in the limit of very low rates, H (º) goes as (exp(¡l /º) ¡1) P (º) /º. Therefore, in this limit, U is proportional to the average period of the population, which is deŽned even if the distribution of Žring rates does not vanish at zero frequency. Another difference between the two models is that in LIF inhibitory networks, instability of the AS can lead to cluster states (van Vreeswijk, 1996; Neltner et al., 2000; Golomb & Hansel, 2000). The number of clusters thus generated can be rather large, especially at low Žring rates. This does not happen in our model, since the numerical solution of the spectral equation reveals that the onsets to clustering instabilities are always within the regions where the AS is already unstable. These instabilities are therefore irrelevant for the AS. Of course, this does not exclude the possibility that clustering occurs far from the onset of the instabilities in our model as well. The origin of this different behavior can be clariŽed in the weak coupling limit. In this limit, the instabilities of the AS can be related to the Fourier expansion of the response function, Z (w ), of the neurons, where w is the phase variable describing the position of the neuron on its limit cycle (Kuramoto, 1984; Neltner et al., 2000). Destabilization of the AS through an n-cluster

46

D. Hansel and G. Mato

instability mode in a heterogeneous network requires a large Fourier coefŽcient of order n. In the LIF model, the function Z is an exponential and is discontinuous at w D p / 2. Therefore, all the Fourier coefŽcients are large and even diverge in the limit of small Žring rates. One can further show that in this limit, the order of the unstable leading mode increases. In contrast, for the QIF, the amplitude of the response function diverges but proportionally to cos(w ) (up to a constant). Therefore, the dominant mode becomes the Žrst one. This difference has its origin in the way the membrane potential approaches the threshold potential at low rate. In the LIF the derivative of the membrane potential that is just before the spike threshold goes to zero with the Žring rate. This is not the case for the QIF where this derivative is 6 0. proportional to Vt2 D 7.5 Relation to Previous Studies. Mechanisms of neuronal synchrony have been addressed in recent years, following two different and complementary theoretical approaches. One has been to study the conditions under which fully synchronized states or more general phase-locked states are stable in fully connected networks of identical neurons (Wang & Rinzel, 1992; Hansel, Mato, & Meunier, 1993b; van Vreeswijk et al., 1994; Hansel, et al., 1995; White et al., 1998; Chow, 1998; Crook, Ermentrout, & Bower, 1998). The other approach has been to investigate the conditions of the stability of the asynchronous state of a large neuronal system (Abbott & van Vreeswijk, 1993; Gerstner & van Hemmen, 1993; Treves, 1993; van Vreeswijk, 1996, 2000; Brunel & Hakim, 1999; Brunel, 2000; Golomb & Hansel, 2000; Neltner et al., 2000; Gerstner, 2000). The latter approach was followed here. This article aims to contribute to a comprehensive understanding of the patterns of synchrony in two-population heterogeneous neuronal networks with full connectivity. In order to make the model more amenable to analytical calculations, noise was not introduced. An expected consequence of adding input noise is to increase the parameter domain where the AS is stable. Still, if the noise is not too strong, simulations indicate that three instabilities may occur for the AS. However, another type of instability can also be found if the noise is strong enough. It leads to a state in which the population oscillations are faster than the average Žring rate of the neurons. In this state, the Žring pattern of the neurons can be very irregular, and averaging over a large number of neurons is necessary to reveal the existence of the underlying rhythmic component in the network activity. Such patterns of synchrony, called fast oscillations, were studied Žrst by Brunel and Hakim (1999) in sparse networks of inhibitory leaky integrate-and-Žre neurons. The results of this study were subsequently generalized to the case of two populations with sparse connectivity (Brunel, 2000; N. Brunel, C. Geisler, & X.-J. Wang, private communication Dec. 2001). More recently, Brunel and Hansel (2002) have shown that sparse connectivity was not required for fast oscillations to take place but that they can occur in a fully connected network of inhibitory neurons in the presence

Asynchronous States and the Emergence of Synchrony

47

of strong input noise. Increasing the coupling destabilizes the AS, leading to a state where the neurons Žre synchronously at a rate that can be signiŽcantly smaller than the population oscillations. Therefore, when noise is also present, the AS can become unstable through four generic mechanisms. The all-to-all connectivity considered here can be thought of as the extreme case of a system in which the connectivity is high. Indeed, we have checked that our results still hold when the connectivity is random and sufŽciently dense (results not shown). However, an important difference exists between the collective behavior of densely and sparsely connected networks. In densely connected networks in the absence of input noise, the neurons discharge in a very regular manner. This is because the uctuations in the synaptic input are small—of the order of O (1 / K ) , where K is the connectivity. In contrast, in sparse networks with strong excitatory and inhibitory synapses, the Žring of the neurons can be highly irregular even if no input noise is present (van Vreeswijk & Sompolinsky, 1996, 1998; Brunel, 2000). In this state, the stochasticity of the Žring is of deterministic origin, and the uctuations in the synaptic input remain large due to a balance between excitation and inhibition. We have shown that in the presence of an excitatory population, a stronger coupling is required to synchronize activity through mutual inhibition. A similar conclusion was reported in Traub, Jefferys, and Whittington (1997) relying on a simulation of a detailed conductance-based model. These authors found that excitation can destroy a synchronous state generated by mutual inhibition. We should also remark that the desynchronizing effect of excitation is a characteristic feature of type I neurons. For type II neurons (such as in the Hodgkin-Huxley model), excitation can have a strongly synchronizing effect, especially at low Žring rates (Hansel, Mato, & Meunier, 1993b; Hansel et al., 1993a, 1995). The fact that mutual inhibition can prevent recurrent inhibition to induce synchrony was noted by Tsodyks, Skaggs, Sejnowski, and McNaughton (1997) in the framework of a two-population rate model. In this work, strengthening the I-I interactions was shown to prevent the two populations from developing synchronous oscillatory rate modulations. Since we have seen that in the limit of inŽnitely slow synapses, the spectral equation becomes equivalent to the spectral equation of a rate model, it is not surprising that in that limit, our conclusions agree with the results of Tsodyks et al. (1997). However, mutual inhibition is a powerful way to generate synchrony on a timescale comparable to the timescale of AMPA and GABAA interactions. Therefore, it was not clear a priori that this mechanism would still hold for these synapses. Our detailed stability analysis of the AS shows that this is indeed the case. A desynchronizing effect of mutual inhibition was also reported by Bush and Sejnowski (1996) in simulations of a two-population model of a column in primary visual cortex. It was found that when the inhibition between the

48

D. Hansel and G. Mato

basket cells is increased, the temporal modulation of the local Želd potential is decreased and the neurons are less bursty. However, in this work, the Žring rate of the neurons changed when the mutual inhibition was increased. Another difference with our work is that in the simulations presented by Bush and Sejnowski, the neurons were intrinsic bursters. Therefore, in their case, the desynchronizing effect of mutual inhibition could also have been the consequence of a disruption of the intrinsic bursting when the inhibitory synaptic input became stronger. Excitatory cells in cortex display strong spike adaptation, whereas interneurons do not. To reduce the number of parameters of our two-population model, we assumed the dynamics of the two populations were identical and spike adaptation was not introduced in the excitatory dynamics. Recent theoretical studies have investigated the dynamics of excitatory networks of neurons with spike adaptation (van Vreeswijk & Hansel, 2001; Fuhrman, Markram, & Tsodyks, 2002). In these works, it was demonstrated, mostly analytically, that excitation and spike adaptation can cooperate to create synchronous oscillatory activity and that the frequency of these oscillations depends on the kinetics of the adaptation process. In particular, if the excitatory recurrent feedback and the adaptation is sufŽciently strong and slow, the network settles into a state in which the neurons Žre synchronous bursts (van Vreeswijk & Hansel, 2001). How an additional population of inhibitory neuron modiŽes this type of synchronous state was subsequently studied, relying on numerical simulations (van Vreeswijk & Hansel, 2001). The main conclusion was that recurrent inhibition of the excitatory population (E-I-E feedback loop) plays against the emergence of the synchronous bursts but that mutual inhibition (I-I interactions) favors it. In contrast, we have found in our model that recurrent inhibition favors the synchronous bursting that emerges on instability lines L3 and L4 but that mutual inhibition disrupts it. Keeping in mind that spike-to-spike synchrony can be induced by mutual inhibition, inhibitory interactions can thus have a spectrum of qualitatively different effects in two-population networks. Appendix A: The Wang-Busz a ki Model The dynamical equations for the conductance-based neuron model used in this work are (Wang & Busz a ki, 1996):

C

dV D Iext ¡gNa m31 h ( V ¡VNa ) ¡gK n4 ( V ¡ VK ) ¡gl ( V ¡Vl ) C Isyn dt dh h1 ( V ) ¡ h D th ( V ) dt dn n1 ( V ) ¡ n . D tn ( V ) dt

(A.1) (A.2) (A.3)

Asynchronous States and the Emergence of Synchrony

49

The parameters gNa , gK , and gl are the maximum conductances per surface unit for the sodium, potassium, and leak currents, respectively, and VNa , VK , and Vl are the corresponding reversal potentials. The capacitance per surface unit is denoted by C. The external stimulus on the neuron is represented by an external current Iext , and the synaptic current due to the interactions with other neurons in the network is Isyn. The functions m1 ( V ), h1 (V ) , and n1 ( V ) and the characteristic times (in milliseconds) tm , tn , th , are given by x1 ( V ) D ax / (ax C bx ) , tx D 1 / (ax C bx ) with x D m, n, h and: am D ¡0.1(V C 35) / (exp(¡0.1( V C 35) ) ¡ 1) bm D 4 exp(¡(V C 60) / 18)

ah D 0.35 exp(¡(V C 58) / 20)

bh D 5 / (exp(¡0.1( V C 28) ) C 1).

(A.4) (A.5) (A.6) (A.7)

The other parameters of the sodium current are gNa D 35 mS / cm2 and VNa D 55 mV. The delayed rectiŽer current is described in a similar way as in the HH model with an D ¡0.05 (V C 34) / (exp(¡0.1( V C 34) ) ¡ 1) bn D 0.625 exp(¡(V C 44) / 80)

(A.8) (A.9)

and gK D 9 mS / cm2 and VK D ¡90 mV. Other parameters of the model are C D 1 m F/cm2 , gl D 0.1 mS/cm2 , and Vl D ¡65 mV. This neuron model displays an SN bifurcation at the current threshold to repetitive Žring, Ic , in the vicinity of which the relationship between p the current, I, injected in the neuron and its Žring rate behaves like f / I ¡ Ic for I > Ic . Therefore, they are able to Žre at an arbitrarily small rate. This is also seen in Figure 1, which represents the frequency-current relation of this model. Appendix B: Reduction of a Conductance-Based Model to the QIF Model We consider a type I conductance-based model. The equations of the singleneuron dynamics can be written in compact form: dX D F(X) C G(X) . dt

(B.1)

50

D. Hansel and G. Mato

In the case of the WB model, X, F, and G are three-dimensional vectors with: X1 D V

(B.2)

X2 D h

(B.3)

X3 D n

(B.4)

F1 (X) D ( Ic ¡gNa m31 h ( V ¡VNa ) ¡gK n4 (V ¡VK ) ¡gl ( V ¡Vl ) ) / C F2 (X) D ( h1 (V ) ¡ h ) / th (V )

F3 (X) D ( n1 (V ) ¡ n ) / tn (V ) G1 D I ¡ Ic

(B.5) (B.6) (B.7) (B.8)

and G2 D G3 D 0. Following Ermentrout (1996), one can study this dynamics in the vicinity of Žring onset. One writes: G(X) D 2 N(X) ,

(B.9)

where 2 ¿ 1. For 2 < 0, the neuron is not Žring. It displays a stable Žxed ¯ . For 2 > 0, the neuron Žres action potentials. At 2 D 0, a point: X D X(I) bifurcation occurs. The Taylor expansion of F (X) around X? D X¯ (Ic ) is F (X) D F(X?) C A(X ¡ X?) C Q(X ¡ X?, X ¡ X?) C ¢ ¢ ¢

(B.10)

where A and Q are, respectively, the Jacobian and the Hessian matrices of F evaluated at X?. Since an SN bifurcation occurs at X?, the matrix A has one zero eigenvalue. The corresponding normalized eigenvector will be denoted by e. Similarly, AT , the transpose of A, has a zero eigenvalue. We will denote by f the corresponding eigenvector satisfying f.e D 1. For small 2 > 0, the solution X(t) of equation B.1 can be expanded, X (t ) D X? C

p

2 z (t) e

(B.11)

where the scalar quantity z satisŽes (Ermentrout & Kopell, 1986) dz p 2 D 2 (g C qz ) , dt

(B.12)

where: g D f.N(X?) and q D f.Q(e, e). This equation can easily be solved. Assuming that g and q have the same sign, one Žnds r z ( t) D

g p tan gq2 ( t ¡ t0 ) , q

(B.13)

where t0 is a constant. Function z diverges when the argument of the tangent is a multiple of p / 2. These divergences correspond to the Žring of an action

Asynchronous States and the Emergence of Synchrony

51

potential in the original system (Ermentrout & Kopell, 1986). Therefore, in the limit 2 ! 0, the Žring frequency of the neuron vanishes as p º D B I ¡ Ic,

(B.14)

p gq . BD p

(B.15)

with

CoefŽcient B can be calculated by diagonalizing matrix A. It turns out that due to the particular structure of this matrix, expressions for e, f, Q(e, e) can be derived analytically in terms of the gating functions (this fact has apparently escaped attention in previous papers). In particular, for the WB model, one Žnds that 0

0

e D K (1, h1 , n1 ) ³ ´ 1 @F1 @F1 fD 1, th , tn LK @h @n

(B.16) (B.17)

where KD p

1

(B.18)

0 1 C ( h1 ) 2 C (n1 ) 2

L D 1 C th

0

@F1 @h

0

h1 C tn

@F1 @n

0

n1 .

(B.19) 0

The derivative of function f (V ) with respect to V has been denoted f . In these expressions, all the functions have to be evaluated for V, n1 , h1 at the Žxed point of the dynamics. Finally we Žnd: v Á ! u 2F 2F X 0 @2 F1 X 00 @F1 1 u 1 @ @ 1 1 0 t C2 C ( n1 ) 2 C . BD Xi Xi p L 2 @V 2 @V @Xi @n2 iD 2,3 @Xi i D2,3

(B.20)

The Taylor expansion of the dynamics, equation B.10, is valid in the limit of low Žring rate except during an action potential where the variable z diverges. If I is sufŽciently close to threshold, an approximate trajectory for the vector X(t) can be computed from z ( t) and the vector e. This approximation is a good description of the behavior of the neuron except during the spikes, which are of short duration. It becomes exact in the limit I ! Ic . Therefore, near the Žring onset, one can replace the full model, equation B.1, by the reduced model, equations B.11 and B.12, where all the parameters can be

52

D. Hansel and G. Mato

computed from the full model. In particular, the shape of the f-I curve near threshold is given by equation B.14. When I ¡ Ic is not small, this reduction is no longer exact. However, one can generalize it heuristically as follows. We consider that action potentials are Žred whenever z crosses (from below) some threshold, zt . Then z is immediately reset to a value zr and it evolves again according to equation B.12. The two phenomenological parameters, zt and zr , cannot be determined by the exact reduction method detailed above. Instead, we estimate them by requiring that the f ¡ I curve Žts the f ¡ I curve of the full model as well as possible. Appendix C: A Measure for the Synchrony Level To characterize the degree of synchrony in a population of N neurons in the simulations, we measure the temporal uctuations of the membrane potential averaged over the population (Hansel & Sompolinsky, 1992; Golomb & Rinzel, 1993, 1994; Ginzburg & Sompolinsky, 1994). The quantity

V ( t) D

N 1 X Vi (t) N iD 1

(C.1)

is evaluated over time, and the variance sV2 D h[V (t )]2 it ¡ [hV (t) it ]2 of its temporal uctuations is computed where h. . .it denotes time averaging. After normalization of sV to the average over the population of the single cell membrane potentials sV2i D h [Vi ( t) ]2 it ¡ [hVi ( t) it ]2 , one deŽnes  ( N ) ,  (N) D

1 N

sV2 PN

2 i D 1 sVi

,

(C.2)

which is between 0 and 1. The central limit theorem implies that in the limit, N ! 1  ( N ) behaves as  ( N ) D  (1) C

³ ´ 1 a CO , N N

(C.3)

where a > 0 is a constant. In particular,  (N ) D 1, if the system is fully synchronized (i.e., Vi ( t) D V ( t) for all i), and  (N) D O (1 / N ) if the state of the system is asynchronous. In the asynchronous state,  (1) D 0. More generally, the larger  (1) > 0, the more the population is synchronized. In networks with more than one population of neurons, we deŽne a synchrony measure  for each population separately.

Asynchronous States and the Emergence of Synchrony

53

Acknowledgments This work was partially supported by ECOS. The work of G.M. was partially supported by grant PICT97 03-00000-00131 from ANPCyT and by Fundaci Âon Antorchas. Fruitful discussions with N. Brunel, B. Ermentrout, D. Golomb, H. Sompolinsky, and C. van Vreeswijk are acknowledged. We thank N. Brunel, D. Golomb, and C. Meunier for careful and critical reading of the manuscript. References Abbott, L. F., & van Vreeswijk, C. (1993). Asynchronous state in networks of pulse-coupled oscillators. Phys. Rev., E48, 1483–1490. Abeles, M. (1990). Corticonics: Neural circuits of the cerebral cortex. Cambridge: Cambridge University Press. Ben Yshai, R., Hansel, D., & Sompolinsky, H. (1997). Traveling waves and processing of weakly tuned inputs in cortical module. J. Comput. Neurosci., 4, 57–77. Bergman, H., Feingold, A., Nini, A., Raz, A., Slovin, H., Abeles, M., & Vaadia E. (1998). Physiological aspects of information processing in the basal ganglia of normal and parkinsonian primates. Trends in Neuroscience, 21, 32–38. Bressloff, P. C., & Coombes, S. (2000). Dynamics of strongly-coupled spiking neurons. Neural Comput., 12, 91–129. Brunel, N. (2000). Dynamics of sparsely connected networks of excitatory and inhibitory spiking neurons. J. Comput. Neurosci., 8, 183–208. Brunel, N., & Hakim, V. (1999). Fast global oscillations in networks of integrateand-Žre neurons with low Žring rates. Neural Comput., 11, 1621–1671. Brunel, N., & Hansel, D. (2002). How noise affects synchronization in recurrent networks of inhibitory neurons. Unpublished manuscript. Bush, P., & Sejnowski, T. (1996). Inhibition synchronizes sparsely connected cortical neurons within and between columns of realistic network models. J. Comput. Neurosci., 3, 91–110. BuzsÂa ki, G., & Chrobak, J. J. (1995). Temporal structure in spatially organized neuronal ensembles: A role for interneuron networks. Curr. Opin. Neurobiol., 5, 504–510. Chow, C. C. (1998). Phase-locking in weakly heterogeneous neuronal networks. Physica, D118, 343–370. Chow, C. C., White, J. A., Ritt, J., & Kopell, N. (1998). Frequency control in synchronized networks of inhibitory neurons. J. Comput. Neurosci., 5, 407– 420. Crook, S. M., Ermentrout, G. B., & Bower, J. M. (1998). Spike frequency adaptation affects the synchronization properties of networks of cortical oscillators. Neural Comput., 10, 837–854. Ermentrout, G. B. (1994). Reduction of conductance based models with slow synapses to neural nets. Neural Comput., 6, 679–695.

54

D. Hansel and G. Mato

Ermentrout, G. B. (1996). Type I membranes, phase resetting curves, and synchrony. Neural Comput., 8, 979–1001. Ermentrout, G. B., & Kopell, N. (1986). Parabolic bursting in an excitable system coupled with a slow oscillation. SIAM J. Appl. Math., 46, 233–253. Fuhrman, G., Markram, H., & Tsodyks, M. (2002). Spike frequency adaptation and neocortical rhythms. J. Neurophysiol., 88, 761–770. Gerstner, W. (2000). Population dynamics of spiking neurons: Fast transients, asynchronous state, and locking. Neural Comput., 12, 43–89. Gerstner, W., & van Hemmen, J. L. (1993). Coherence and incoherence in a globally coupled ensemble of pulse-emitting units. Phys. Rev. Lett., 71, 312–315. Ginzburg, I., & Sompolinsky, H. (1994). Theory of correlations in stochastic neuronal networks. Phys. Rev., E50, 3171–3191. Golomb, D. (1998). Models of neuronal transient synchrony during propagation of activity through neocortical circuitry. J. Neurophysiol., 79, 1–12. Golomb, D., & Ermentrout, G. B. (1999). Continuous and lurching traveling pulses in neuronal networks with delay and spatially decaying connectivity. Proc. Natl. Acad. Sci. USA. 96, 13480–13485. Golomb, D., & Hansel, D. (2000). The number of synaptic inputs and the synchrony of large sparse neuronal networks. Neural Comput., 12, 1095–1139. Golomb, D., Hansel, D., & Mato, G. (2001). Theory of synchrony of neuronal activity. In S. Gielen & F. Moss (Eds.), Handbook of biological physics. Amsterdam: Elsevier. Golomb, D., & Rinzel, J. (1993).Dynamics of globally coupled inhibitory neurons with heterogeneity. Phys. Rev., E48, 4810–4814. Golomb, D., & Rinzel, J. (1994). Clustering in globally coupled inhibitory neurons. Physica, D72, 259–282. Gupta, A., Wang, Y., & Markram, H. (2000). Organizing principles for a diversity of GABAergic interneurons and synapses in the neocortex. Science, 287, 273– 278. Hansel, D., & Mato, G. (2001). Existence and stability of persistent states in large neuronal networks. Phys. Rev. Lett., 86, 4175–4178. Hansel, D., Mato, G., & Meunier, C. (1993a).Phase dynamics for weakly coupled Hodgkin-Huxley neurons. Europhys. Lett., 23, 367–372. Hansel, D., Mato, G., & Meunier, C. (1993b). Phase reduction and neural modeling. in functional analysis of the brain based on multiple-site recordings. Concepts in Neuroscience, 4, 192–210. Hansel, D., Mato, G., & Meunier, C. (1995). Synchrony in excitatory neural networks. Neural Comput., 7, 307–337. Hansel, D., Mato, G., Neltner, L., & Meunier, C. (1998).On numerical simulations of integrate-and-Žre neural networks. Neural Comput., 10, 467–483. Hansel, D., & Sompolinsky, H. (1992). Synchrony and computation in a chaotic neural network. Phys. Rev. Lett., 68, 718–721. Hansel, D., & Sompolinsky, H. (1996). Chaos and synchrony in a model of hypercolumn in visual cortex. J. Comput. Neurosci., 3, 7–34. Hansel, D., & Sompolinsky, H. (1998) Modeling feature selectivity in local cortical circuits. In C. Koch & I. Segev. (Eds.), Methods in neuronal modeling: from ions to networks. Cambridge, MA: MIT Press.

Asynchronous States and the Emergence of Synchrony

55

Hebb, D. (1949). The organization of behavior. New York: Wiley. Jefferys, J. G. R., Traub, R. D., & Whittington, M. A. (1996). Neuronal networks for induced “40 Hz” rhythms. Trends Neurosci., 19, 202–208. Kuramoto, Y. (1984). Chemical oscillations, waves and turbulence. New York: Springer-Verlag. Kuznetsov, Y. (1998). Elements of applied bifurcation theory (2nd ed.). New York: Springer-Verlag. Latham, P. E., Richmond, B. J., Nelson, P. G., & Nirenberg, S. (2000). Intrinsic dynamics in neuronal networks. I. Theory. J. Neurophysiol., 83, 808–827. McCormick, D. A., & Contreras, D. (2001). On the cellular and network bases of epileptic seizures. Annu. Rev. Physiol., 64, 815–846. Neltner, L., Hansel, D., Mato, G., & Meunier, C. (2000). Synchrony in heterogeneous networks of spiking neurons. Neural Comput., 12, 1607–1641. Rall, W. (1967). Distinguishing theoretical synaptic potentials computed for different some-dendritic distribution of synaptic input. J. Neurophysiol.,30, 1138– 1168. Rubin, N., & Sompolinsky, H. (1989). Neural networks with low local Žring rates. Europhys. Lett., 10, 465–470. Shelley, M. J., & Tao, L. (2001). EfŽcient and accurate time-stepping schemes for integrate-and-Žre neuronal networks. J. Comput. Neurosci., 11, 111–119. Sik, A., Penttonen, M., Ylinen, A., & BuzsÂa ki G. (1995). Hippocampal CA1 interneurons: An in vivo intracellular labeling study. J. Neurosci., 15, 6651–6665. Strogatz, H. S. (2000). Nonlinear dynamics and chaos. Cambridge: Perseus. Tarczy-Hornoch, K., Martin, K. A., Jack J. J., & Stratford K. J. (1998). Synaptic interactions between smooth and spiny neurones in layer 4 of cat visual cortex in vitro. J. Physiol., 508, 351–363. Traub, R. D., Jefferys, J. G., & Whittington, M. A. (1997). Simulation of gamma rhythms in networks of interneurons and pyramidal cells. J. Comput. Neurosci., 4, 141–150. Traub, R. D., Jefferys, J. G. R., & Whittington, M. A. (1999). Fast oscillations in cortical circuits. Cambridge, MA: MIT Press. Traub, R. D., Whittington, M. A., Colling, S. B, BuzsÂaki, G., & Jefferys, J. G. R. (1996). Analysis of gamma rhythms in the rat hippocampus in vitro and in vivo. J. Physiol. (London), 493, 471–484. Treves, A. (1993). Mean Želd analysis of neuronal spike dynamics. Network, 4, 259–284. Tsodyks, M. V., Skaggs, W. E., Sejnowski, T. J., & McNaughton B. L. (1997). Paradoxical effects of external modulation of inhibitory interneurons. J. Neurosci., 17, 4382–4388. van Vreeswijk, C. (1996). Partial synchronization in populations of pulsecoupled oscillators. Phys. Rev., E54, 5522–5537. van Vreeswijk, C. (2000). Analysis of the asynchronous state in networks of strongly coupled oscillators. Phys. Rev. Lett., 84, 5110–5113. van Vreeswijk, C., Abbott, L. F., & Ermentrout, G. B. (1994). When inhibition not excitation synchronizes neural Žring. J. Comput. Neurosci., 1, 313–321. van Vreeswijk, C., & Hansel, D. (2001). Patterns of synchrony in neural networks with adaptation. Neural Comput., 13, 959–992.

56

D. Hansel and G. Mato

van Vreeswijk, C., & Sompolinsky, H. (1996). Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science, 274, 1724–1726. van Vreeswijk, C., & Sompolinsky, H. (1998). Chaotic balanced state in a model of cortical circuits. Neural Comput., 10, 1321–1372. Wang, X.-J. (1999). Synaptic basis of cortical persistent activity: The importance of NMDA receptors to working memory. J. Neurosci., 19, 9587–9603. Wang, X.-J., & BuzsÂaki, G. (1996). Gamma oscillation by synaptic inhibition in a hippocampal interneuronal network model. J. Neurosci., 16, 6402–6413. Wang, X.-J., & Rinzel, J. (1992). Alternating and synchronous rhythms in reciprocally inhibitory model neurons. Neural Comput., 4, 84–97. White, J. A., Chow, C. C., Ritt, J., Soto-Trevino, ˜ C., & Kopell, N. (1998). Synchronization and oscillatory dynamics in heterogeneous, mutually inhibited neurons. J. Comput. Neurosci., 5, 5–16. Whittington, M. A., Traub, R. D., & Jefferys, J. G. R. (1995). Synchronized oscillations in interneuron networks driven by metabotropic glutamate receptor activation. Nature, 373, 612–615. Whittington, M. A., Traub, R. D., Kopell, N., Ermentrout, B., & Buhl, E. H. (2000). Inhibition-based rhythms: Experimental and mathematical observations on network dynamics. Int. J. of Psychophysiology,38, 315–336. Received April 25, 2002; accepted June 27, 2002.

NOTE

Communicated by Erkki Oja

A Constrained EM Algorithm for Principal Component Analysis Jong-Hoon Ahn [email protected]

Jong-Hoon Oh [email protected] Department of Physics, Pohang University of Science and Technology, Pohang, KyÆ ongbuk, Korea

We propose a constrained EM algorithm for principal component analysis (PCA) using a coupled probability model derived from single-standard factor analysis models with isotropic noise structure. The single probabilistic PCA, especially for the case where there is no noise, can Žnd only a vector set that is a linear superposition of principal components and requires postprocessing, such as diagonalization of symmetric matrices. By contrast, the proposed algorithm Žnds the actual principal components, which are sorted in descending order of eigenvalue size and require no additional calculation or postprocessing. The method is easily applied to kernel PCA. It is also shown that the new EM algorithm is derived from a generalized least-squares formulation. 1 Introduction Principal component analysis (PCA) is a statistical technique that analyzes the covariance structure of multivariate data. It takes a data set in a given orthonormal basis in Rd and Žnds a new orthonormal basis Rq with its axes ordered (d > q). This new basis is rotated in such a way that the axes are oriented along the direction in which the data have their highest variance. The principal components are the new coordinates, ordered by their corresponding variances. The reduced data can best preserve the variance of data; thus, they are widely used in data compression, image analysis, visualization, pattern recognition, regression, and time-series prediction. Situations dealing with high-dimensional data with low-dimensional structure require an efŽcient PCA technique as a dimensionality-reduction method, and a variety of methods for PCA have been introduced in the literature (Jolliffe, 1986). Nevertheless, the existing PCA methods had one common shortcoming: the absence of an associated probability density model. The derivation of PCA from the perspective of density estimation offers a number of important advantages, and the PCA model in the noiseless case provides a very Neural Computation 15, 57–65 (2003)

° c 2002 Massachusetts Institute of Technology

58

J. Ahn and J. Oh

efŽcient method for extracting a few principal components from a highdimensional data set (Tipping & Bishop, 1999; Roweis, 1997). Classic eigenvalue analysis is a quick and exact method for use with low-dimensional data, but it is unsuitable for very high-dimensional data sets. Computing a complete covariance matrix is laborious and inefŽcient when only a few principal components are required. However, the EM algorithm for PCA has rotational ambiguity, which means that it Žnds only vector sets that are linear superpositions of the principal components. In this article, we propose a coupled latent variable model for extracting the actual principal components. 2 Probabilistic Models 2.1 The Single Latent Variable Model. The single latent variable model for PCA is referred to as the probabilistic model (Tipping & Bishop, 1999). The latent variable model is linear: t D Wx C m C 2 ,

(2.1)

where the q-dimensional column vector x has a unit isotropic distribution N (0, I ). The noise is given as a normal distribution such that 2 » N (0, Y) , with Y diagonal, the d £ q matrix W contains the factor loadings, and m is a constant vector with a maximum likelihood estimator that is the mean of the given data set ftkg. Given this formulation, t is also modeled as the normal density N (m , Y C WW T ) . It is assumed that the diagonal elements of the covariance matrix Y should be identical. When the diagonal elements of Y are equal, the factor loadings eventually become the principal axes. Tipping and Bishop showed that by considering a model with an isotropic noise structure, the maximum likelihood estimator W ML is a matrix whose columns are the scaled and rotated principal eigenvectors of the sample’s covariance matrix S, W D Uq (L q ¡ s 2 I ) 1 / 2 R,

(2.2)

where the q-column vectors in Uq are eigenvectors of S, with corresponding eigenvalues in the diagonal matrix L q , and R is an arbitrary q £q orthogonal matrix. The true principal axes can be recovered when the columns of RT are equal to the eigenvectors of the W T W matrix. Although postprocessing needs few costs, it may be unwanted from a theoretical viewpoint. In order to remove the rotational ambiguity in a probabilistic PCA model, we shall consider a coupled latent variable model constructed from single models under some constraints. 2.2 A Coupled Latent Variable Model. It was shown that the q-dimensional single latent variable model Žnds q-dimensional principal subspace.

A Constrained Algorithm for PCA

59

Now let us consider a ( q ¡ 1)-dimensional single model prior to the qdimensional model. This certainly recovers (q ¡ 1)-dimensional principal subspace. Assuming that the columns of Uq are sorted in descending order of eigenvalue size, we can constrain the q £ q rotational matrix R to Rqi D 0, where i D 1, . . . , q ¡ 1. This is because a ( q ¡ 1)-dimensional principal subspace should be spanned by the set of Žrst ( q ¡ 1) principal axes. Because matrix R is orthogonal, the condition leads to Riq D 0 and Rqq D 1. Thus, we can obtain an exact qth principal vector through the coupled model constructed from two single models. We can then deduce that a coupled model constructed from single latent variable models with full dimensionality should recover the exact principal components on their convergence. The ith model in the coupled model is given by yi D WIi x C m C 2

(2.3)

i

where the q£q diagonal matrix Ii contains the factor selection with Ii ( j, j) D 1 for j D 1, . . . , i and Ii ( j, j) D 0, for j D i C 1, . . . , q. For the ith isotropic noise model 2 i » N (0, si2 I ) , the coupled model implies a probability distribution over multiple y spaces for x, given by p ( y1 D t, . . . , yq D t | x ) D D

q Y

p (t | xI i)

iD 1 q Y

(2p si2 ) ¡d / 2

iD 1

(2.4) µ

¶ 1 2 exp ¡ 2 kt ¡WIi x¡m k , 2si

(2.5)

where p ( t | xI i) is the conditional density for the ith single model. Note that R 6 1. With the latent variables x deŽned by p ( t, . . . , t | x) dt D µ ¶ 1 p (x ) D (2p ) ¡q/ 2 exp ¡ xT x , 2

(2.6)

we obtain the marginal distribution of coupled y spaces in the form of Z p (t, . . . , t) D D

p ( t, . . . , t | x) p ( x ) dx µ ¶ 1 (2p si2 ) ¡d / 2 |M| ¡1 / 2 exp ¡ ( t ¡ m ) T C ¡1 ( t ¡ m ) , 2 iD 1

q Y

(2.7)

where the model covariance is " q #¡1 X I ¡1 T T ¡ WQM Q W CD , s2 iD 1 i

(2.8)

60

J. Ahn and J. Oh

Pq Pq where M D I C iD 1 IiT W T WIi / si2 and Q D iD1 Ii / si2 . Using Bayes’ rule, the posterior distribution of the latent variables x may be calculated: µ 1 ¡q / 2 1/ 2 ( ) (2p ) | |M| . . . exp ¡ fx ¡ M ¡1 QT W T ( t ¡ m )gT p x t, ,t D 2 ¶ (2.9) ¡1 T T ( £ Mfx ¡ M Q W t ¡ m ) g , where the posterior covariance matrix is given by MDIC

q X Ii W T WIi . si2 i D1

(2.10)

M is q £ q matrix while C is d £ d matrix. 3 Derivation of Constrained EM Algorithm An expectation-maximization (EM) algorithm for PCA Žnds the principal subspace as the noise level sq2 becomes inŽnitely small (Roweis, 1997). The coupled model for the new EM algorithm also comes to be valid for inŽnitely small noises si2 . Further, we need to consider priorities among the single models that are associated with the coupled model. By the constrained algorithm, we mean that the ith single model should be treated prior to the (i C 1)th single model so that the columns of Uq are sorted in descending order of eigenvalue size. The priorities for such models are decided by the limits si2 si2C 1

¡! 0.

(3.1)

These conditions certainly give the constrained EM algorithm for PCA. 3.1 EM Algorithm. As the latent variable x is considered missing data in the EM approach, it can be expected by the posterior density, equation 2.9, and the noise condition, equation 2.11, Z | E[x t] D xp (x | t, . . . , t) dx D M ¡1 QT W T ( t ¡ m )

(3.2)

! fL(W T W ) g¡1 W T ( t ¡ m ) , where elementwise operator L is deŽned as L ( aij ) D aij for i ¸ j and L ( aij ) D 0 for i < j. The E-step may be also represented in matrix form, H D fL(W T W ) g¡1 W T V,

(3.3)

A Constrained Algorithm for PCA

61

where H is q £ n matrix, which has column vectors fx kg, and V is d £ n matrix, which has column vectors ftk ¡m g. In the M-step, the new parameter value W new that maximizes the expected value of the log likelihood for the complete data given at each iteration is determined. The part that includes P W in the complete data log likelihood LC D nkD 1 ln[p ( tk, . . . , tk | x k ) p ( xk ) ] is given by LW

" # ´ q ³ X 1 HHT T T T T D tr 2VH Q W ¡ W Ii 2 Ii W . 2 si iD 1

(3.4)

If we consider condition 3.1, this is maximized by W new as follows, ( W new

T

D VH Q

! VH

T

´) ¡1 q ³ X HHT I i 2 Ii si i D1

T fU(HHT )g ¡1

(3.5)

,

where elementwise operator U is deŽned as U ( aij ) D aij for i · j and U ( aij ) D 0 for i > j. 4 Discussion We treat the coupled latent variable model constructed from single probabilistic models in order to remove the rotational ambiguity involved in the single probabilistic model. The constrained EM algorithm derived from them gradually recovers the actual principal components, and it is naturally applied to the kernel PCA, H D fL(C T KC ) g ¡1 C T K

C

new

T

D H fU(HH

T )g ¡1

,

(4.1) (4.2)

which are derived in appendix A. This article proposes using the coupled latent variable model to perform the PCA. The source vector x is considered a hidden variable that is linearly related to the observed vectors yi . Different isotropic noises 2 i are added to the observed data vectors under different model conditions Ii , but all the observed vectors are modeled by the same vector t. By the coupled model, we mean this model description. The EM algorithm is employed to Žt linear combinations of these source distributions to the observed data. The parameter to be estimated is the factor loading W (Dempster, Laird, & Rubin, 1977). The forms of iteration rules are similar to those used in the previous EM algorithm, where simple pseudo-inverse solutions are used for the leastsquare formulation (Roweis, 1997). But it is clearly distinguished by the

62

J. Ahn and J. Oh

lower and upper operators that leave the matrix elements unchanged or changed to zero.1 Thus, there is virtually no additional or postprocessing cost in using the method. The cost per iteration is the same for both algorithms, via elapsed time or total oating-point operations. Ignoring constants, they are both O ( n £ d £ q) , where n = data number, d = original dimensionality, and q = reduced dimensionality. The postprocessing cost for the EM algorithm to Žnd the orthogonal vectors is O (n £ q2 C d £ q2 ) , which is smaller than the cost of one iteration. The previous EM algorithm for PCA minimizes the squared error function monotonically. However, the constrained EM algorithm shows a slightly different convergent behavior. It is also derived from the solution for minimizing the generalized squared error function, which is given by a weighted squared error function summed over the required dimensionality with positive constants ai :

JD

q X iD 1

ai kV ¡ WIi Hk2 .

(4.3)

Using the generalized least-squares formulation and condition 3.1 as a ( i C 1) / ai ! 0, we can obtain the same iteration rules 3.3 and 3.5 that are derived in appendix B. Thus, we can learn that the iterative rules B.5 and B.6 monotonically minimize the generalized squared error function 4.3. Although the proposed method for squared cost function is generally slower than the EM algorithm while converging, the two methods Žnally recover the principal subspace in the same number of iterations. The iterative number required to recover the orthogonal axis is based on the ratios of eigenvalues li / li C 1 . Appendix A: Constrained EM Algorithm for Kernel PCA The constrained EM algorithm for PCA can be naturally applied to kernel PCA (Rosipal & Girolami, 2001). The basic idea is that the factor loadings W can be expressed as W D VW C,

(A.1)

where VW is a nonlinearly mapped data matrix for observed data V 2 Rd£n and C 2 Rn£q contains coefŽcients. Using this expression and the kernel 1 The explicit forms of the operators are related to our assumption that the columns of Uq are sorted in descending order of the eigenvalue size. Thus, we can obtain well-ordered principal components.

A Constrained Algorithm for PCA

63

matrix K, we can write the matrix W T W and W T VW as T W T W D C T VW VW C

D C T KC

(A.2)

D C T K.

(A.3)

T W T VW D C T VW VW

Thus, from the E-step, equation 3.3, the matrix H 2 Rq£n can be updated as H D fL(C T KC )g ¡1 C T K.

(A.4)

Considering expression A.1, we can write the M-step, equation 3.5, in the matrix form VW C new D VW H T fU(HHT )g ¡1 . Then the M-step follows: C new D HT fU(HHT )g ¡1 .

(A.5)

Appendix B: The Generalized Squared Error Formulation The treatment of a coupled model is supplanted by introducing a generalized squared error function. Our problem is to seek the weight matrix W and the hidden matrix H that minimize the error 4.3. The problem of minimizing the single squared error is a classic one that can be solved by a gradient search procedure. Similarly, the generalized squared error problem is solved by forming the gradient,

rH J D rW J D

q X iD 1 q X iD 1

2ai ( WIi ) T (V ¡ WIi H )

(B.1)

2ai ( V ¡ WIi H ) ( Ii H ) T ,

(B.2)

and setting it equal to zero. This yields the necessary condition (

q X

) ai ( WIi

)T

VD

iD 1

(

V

q X i D1

(

) ai ( Ii H

)T

q X

) ai ( WIi

)T(

WI i ) H

i D1

(

D W

q X i D1

(B.3)

) ai ( Ii H ) ( Ii H

)T

.

(B.4)

64

J. Ahn and J. Oh

Pq Pq Because iD 1 ai (WIi ) T (WIi H ) and iD1 ai ( WIi H ) ( Ii H ) T are nonsingular, we can solve for H and W uniquely as ( q )¡1 Á q ! X X T H D ai ( WIi ) (WIi ) ai Ii W T V iD 1

D

Á

f(W T W ) ¯ Ag¡1 2

D

4(WTW ) ¯

D

f(W T

D

T

q X

ai Ii W T V

iD 1

8Á q < X :

i D1

!

i D1

W ) ¯ Dg

¡1

Á q X

!(

!¡1 93 ¡1 = ai Ii A 5 WT V ;

(B.5)

WTV

and W

D

VH VH

T

i D1 Á q X

D

a i ( Ii H ) ( Ii H

)T

)¡1

iD 1

!

ai Ii f(HHT ¯ Ag ¡1

8 Á !¡1 93¡1 q < = X 5 VH T 4 ( HH T ) ¯ A ai Ii : ; D1 2

D

i D1

ai Ii

q X

i

VH T f(HHT )

¯

DT g¡1 ,

where ¯ denotes the Hadamard product and A is given by Pq Pq 0 Pq 1 ¢ ¢ ¢ aq i D 1 ai i D 2 ai i D3 ai B Pq C Pq Pq B ¢ ¢ ¢ aq C i D 2 ai i D3 ai B iD 2 ai C B Pq C Pq Pq B C ¢ ¢ ¢ a a a a qC A D B iD 3 i iD 3 i i D3 i B C B . C .. .. .. B .. C . . . @ A aq

aq

and D is represented as Pq 0 ai 1 P iqD 2 a B iD 1 i B B B1 1 B DDB B B. .. B. . @. 1

(B.6)

1

aq

Pq a PiqD 3 i iD 1

ai

Pq a PiqD 3 i iD 2

ai

¢¢¢

¢¢¢

Paqq

¢¢¢

Paqq

.. .

.. .

¢¢¢

¢¢¢

aq 1 ai C

C C C C a iD 2 i C . C .. C C . A iD 1

1

(B.7)

(B.8)

A Constrained Algorithm for PCA

65

These iterative rules can minimize the generalized squared error most efŽciently. If we take condition 3.1 as ai C 1 / ai ! 0 , all upper elements in the D matrix approach zero, and the iteration rules become identical to equations 3.3 and 3.5. Acknowledgments This research was funded by the Brain Science and Engineering Research Program, Ministry of Science and Technology, in Korea. References Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm Journal of Royal Statistical Society, 39, 1–38. Jolliffe, I. T. (1986). Principal component analysis. New York: Springer-Verlag. Rosipal, R., & Girolami, M. (2001). An expectation-maximization approach to nonlinear component analysis. Neural Computation, 13, 505–510. Roweis, S. R. (1997). EM algorithms for PCA and SPCA. In M. Kearns, M. Jordan, & S. Solla (Eds.), Advances in neural information processing systems, 10 (pp. 626– 632). Cambridge, MA: MIT Press. Tipping, M. E., & Bishop, C. M. (1999). Mixtures of probabilistic principal component analyzers. Neural Computation, 11, 443–482. Received July 26, 2001; accepted June 20, 2002.

LETTER

Communicated by Emilio Salinas

Higher-Order Statistics of Input Ensembles and the Response of Simple Model Neurons Alexandre Kuhn [email protected] Ad Aertsen [email protected] Stefan Rotter [email protected] Neurobiology and Biophysics, Biology III, Albert-Ludwigs-University, D-79104 Freiburg, Germany

Pairwise correlations among spike trains recorded in vivo have been frequently reported. It has been argued that correlated activity could play an important role in the brain, because it efŽciently modulates the response of a postsynaptic neuron. We show here that a neuron’s output Žring rate critically depends on the higher-order statistics of the input ensemble. We constructed two statistical models of populations of spiking neurons that Žred with the same rates and had identical pairwise correlations, but differed with regard to the higher-order interactions within the population. The Žrst ensemble was characterized by clusters of spikes synchronized over the whole population. In the second ensemble, the size of spike clusters was, on average, proportional to the pairwise correlation. For both input models, we assessed the role of the size of the population, the Žring rate, and the pairwise correlation on the output rate of two simple model neurons: a continuous Žring-rate model and a conductance-based leaky integrate-and-Žre neuron. An approximation to the mean output rate of the Žring-rate neuron could be derived analytically with the help of shot noise theory. Interestingly, the essential features of the mean response of the two neuron models were similar. For both neuron models, the three input parameters played radically different roles with respect to the postsynaptic Žring rate, depending on the interaction structure of the input. For instance, in the case of an ensemble with small and distributed spike clusters, the output Žring rate was efŽciently controlled by the size of the input population. In addition to the interaction structure, the ratio of inhibition to excitation was found to strongly modulate the effect of correlation on the postsynaptic Žring rate.

Neural Computation 15, 67–101 (2003)

° c 2002 Massachusetts Institute of Technology

68

A. Kuhn, A. Aertsen, and S. Rotter

1 Introduction

The brain consists of a huge number of neurons that are heavily interconnected. These cells interact by producing rapid depolarizations of their membrane, the action potentials (spikes) that are propagated along the axon and excite or inhibit postsynaptic neurons by inducing postsynaptic potentials (PSPs). The integration of PSPs can bring the postsynaptic neurons, in turn, to Žre a spike. Such interactions within the neuronal network lead to temporal correlations between spike trains of different neurons. Pairwise correlations of spike trains are routinely observed in virtually all regions of the brain. They have been related to the representation of sensory information (DeCharms & Merzenich, 1996) and motor behavior (Baker, Spinks, Jackson, & Lemon, 2001), but also to more abstract processing levels (Steinmetz et al., 2000; Salinas & Sejnowski, 2001). The effect of correlated synaptic input on the neuronal response has been investigated in several recent theoretical and numerical studies (Feng & Brown, 2000; Salinas & Sejnowski, 2000; Svirskis & Rinzel, 2000; Feng & Zhang, 2001; Stroeve & Gielen, 2001). They showed that temporal correlation can have a tremendous effect on the rate and variability of the output of model neurons. Pairwise correlations are generally used to characterize the statistical dependence of parallel spike trains. However, as pointed out, for example, by Bohte, Spekreijse, and Roelfsema (2000), an ensemble of spike trains is not completely described by pairwise interactions. Thus, the effect of pairwise correlation cannot be studied without also taking into account correlations among three or more spike trains (i.e., higher-order interactions). This point is illustrated by the following example. Under the general assumption that only the timing of presynaptic spikes and not their form is important, we can model series of incoming spikes as point processes (Tuckwell, 1988; Cox & Isham, 1980). A presynaptic ensemble of N spike trains can then be thought of as N parallel point processes. We deŽne an interaction as the occurrence of simultaneous spikes in different spike trains. The interactions can be classiŽed as second, third, . . . , Nth-order, corresponding to clusters of 2, 3, . . . , N simultaneous spikes. For simplicity, we assume that the ensemble is homogeneous, that is, the Žring rates of all spike trains are identical and, similarly, the rates of second, third, . . . , Nth-order interactions are identical for any subset of 2, 3, . . . , N spike trains picked from the ensemble. We now consider a homogeneous ensemble of N parallel spike trains where all spikes are associated with pairwise (second-order) interactions: there are no interactions of order 3 or higher, nor are there isolated spikes, that is, spikes in one spike train without a counterpart in another one. If k is the rate of each second-order interaction, the discharge rate r of an individual spike train is r D ( N ¡ 1)k . The correlation coefŽcient between two point processes in a homogeneous system is the rate of interactions with simultaneous spikes on both processes, divided by the

Higher-Order Statistics of Input Ensembles

69

discharge rate of an individual process (see appendix A). Thus, in the case of an ensemble with only second-order interactions, the correlation coefŽcient is c D 1 / ( N ¡1) . It is approximately inversely proportional to the number of presynaptic inputs. For N D 100 correlated spike trains, the correlation coefŽcient will be about 10¡2 . Stronger pairwise correlations can be obtained only by introducing higher-order interactions. A single third-order interaction (or triplet), for instance, contributes to the pairwise correlations in three pairs of spike trains, while introducing only one spike per process. The same second-order interaction is obtained by having one pair of spikes in each of the three pairs of spike trains. This results in two spikes in each single process, that is, twice as many as in the triplet case, which results in a halved correlation coefŽcient. In a homogeneous high-dimensional system, certain combinations of rates and pairwise correlations thus imply the existence of higher-order interactions. Biological neural networks are typically not homogeneous (in the sense deŽned above), but very little is known about their actual interaction structure. The assessment of the statistical signiŽcance of simultaneous events across more than two neurons is a nontrivial problem. Different statistical methods have been proposed (Martignon, von Hasseln, Grun, ¨ Aertsen, & Palm, 1995; Martignon et al., 2000; Gutig, ¨ Aertsen, & Rotter, 2002). Here, we report on a study of the effects of higher-order interactions on the response of model neurons. SpeciŽcally, we constructed two different spike train ensembles and compared their effects on the response of a Žringrate (FR) neuron model and a conductance-based leaky integrate-and-Žre (IF) model. For both neuron models, we assessed the effects of the pairwise correlation in the input on the output Žring rate of the neuron. Analytical results were derived for the FR neuron. Preliminary results have been presented elsewhere (Kuhn, Rotter, & Aertsen, 2002). 2 Two Models for Correlated Neuronal Populations

The realization x (t) of a stochastic point process consists of a sequence of points in time . . . , tk, tkC 1 , . . . that can be represented by a pulse train, X d ( t ¡ tk ) , x ( t) D k

where d (t ) is the Dirac delta function. To reduce the number of parameters deŽning the input ensemble, we model spike trains as parallel Poisson point processes with homogeneous interaction structures. Moreover, we restrict our treatment to instantaneous statistical dependence, that is, the interaction among spike trains originates from simultaneous spikes only. 2.1 Single Interaction Process Model. The single interaction process (SIP) model was motivated by results of Deppisch et al. (1993) and Bohte et al. (2000). They simulated a network of integrate-and-Žre neurons and

70

A. Kuhn, A. Aertsen, and S. Rotter

observed that it could show periods of random looking, unsynchronized neuronal activities, but also population bursts, with almost all neurons Žring synchronously. The SIP model is deŽned as follows. First, a single realization wu ( t) of a Poisson process with stationary rate a is generated. Each process xi (t ) in a set of N spike trains is then deŽned by xi ( t) D wu ( t) C wi (t )

(i D 1, 2, . . . , N ) ,

where wi (t) is an independent realization of a Poisson process with rate b. The superposition of independent Poisson processes is again a Poisson process with a rate equal to the sum of the individual rates (see Cox & Isham, 1980). Thus, the rate of xi (t ) is r D a C b. Figure 1A shows the realization wu ( t) (top) that is copied into each individual spike train xi ( t) (bottom), introducing a statistical dependence between them. We characterized the statistical dependence between two spike trains by the count correlation coefŽcient, c D a/ (a C b ) . Note that the ensemble model is completely constrained by the rate r and the pairwise count correlation coefŽcient c. The relations a D rc and b D r (1 ¡c) are used for the numerical simulation of a set of spike trains with given rate and given pairwise correlation. In the literature, the pairwise dependence is frequently expressed by the cross-correlation. The cross-correlation function R ( D t) of two spike trains xi ( t) and xj ( t) is R (D t) D ad ( D t) C (a C b ) 2 . A derivation of the cross-correlation function and the count correlation coefŽcient is presented in appendix A. When using the SIP model as input to a point neuron, all individual spike trains xi ( t) are summed up. The compound process can, in turn, be decomposed into two independent processes: a Poisson process with rate a, in which each point accounts for N simultaneous spikes, and a Poisson process of rate Nb, resulting from the sum of all independent realizations wi (t) . The dependence between spike trains in the SIP model is thus described by a single interaction process (of order N). We can write X X X [wu (t) C wi ( t) ] D Nwu (t ) C xi ( t ) D wi (t) . i

i

i

The decomposition into two independent processes greatly facilitates the analysis of the neuronal response. This model of correlated spike trains also appears in Feng and Brown (2000) and Stroeve and Gielen (2001).

Higher-Order Statistics of Input Ensembles

A

Single Interaction Process

B

71

Multiple Interaction Process

w

u

wg

x1 x2

x1 x2

x

x

N

N

time

time

C

SIP

­1

sum (s )

spike train index

1

D

MIP 1

10

10

20

20

30

30

40

40

104

104

0

0

0.1 0.2 0.3 0.4 0.5 time (s)

0 0

0.1 0.2 0.3 0.4 0.5 time (s)

Figure 1: Two models of spike train ensembles with identical rates and pairwise correlations but different higher-order statistics. (A) The single interaction process (SIP) model. Each spike train xi ( t) of the ensemble is composed of a Žxed realization wu ( t) of a Poisson process with rate a and an independent realization of a Poisson process with rate b. (B) The multiple interaction process (MIP) model. Each spike train xi ( t) is realized by randomly deleting spikes from a Žxed realization w g ( t) . The generating process is a realization of a Poisson process with rate c . (C) Fifty spike trains with a rate of 20 spikes per second and pairwise correlation coefŽcient 0.1, generated according to the SIP model. Their summed activity is shown below (bin size 5 ms). (D) Fifty spike trains with the same rate and pairwise correlation as in A, but generated according to the MIP model.

72

A. Kuhn, A. Aertsen, and S. Rotter

2.2 Multiple Interaction Process Model. In the second model of a spiking population, the interaction structure is composed of spike clusters involving only subsets of the spike trains. The multiple interaction process (MIP) model is constructed as follows. Each individual spike train is a thinned version (Cox & Isham, 1980) of one and the same realization w g ( t) of a Poisson process. Individual spike trains xi (t ) are thus produced by random deletions of pulses in w g ( t) . The deletions are independent. The thinning process is repeated for all spike trains in the ensemble. If the rate of w g ( t) is c and the probability of deletion is (1 ¡ 2 ), then the rate of each individual spike train is

r Dc2 . Figure 1B shows the generating process w g ( t) (top) and the resulting correlated spike trains x1 (t) , . . . , xN (t ) (bottom). Note that the pulses in each individual spike train are Poisson (Snyder & Miller, 1991). The count correlation coefŽcient is cD2 . Again, the rate r and correlation coefŽcient c together completely constrain the model, and the relations 2 D c and c D r/ c are used for the numerical simulation of the ensemble of spike trains. The cross-correlation function of any two spike trains xi ( t) and xj (t ) is R (D t) D c 2 2d ( D t) C (c 2 ) 2 . The derivation of the cross-correlation function and the count correlation coefŽcient is again given in appendix A. The sum of the N individual spike trains results in a pulse train similar to the generating realization wg ( t) , but with each pulse scaled differently. The scalar amplitude n represents the number of (synchronous) spikes at this point in time. As the deletion of a certain pulse in wg ( t) occurs with a Žxed probability and is independent for each of the N individual spike trains, the number n follows a binomial distribution: Á ! N n n » B(nI 2 , N ) D 2 (1 ¡ 2 ) N¡n . n The scalar n is independent for each point in the compound process. The sum process can thus be decomposed into N C 1 independent Poisson processes (Snyder & Miller, 1991), each process containing only points with identical multiplicity. Such a process is called an interaction process of order n, with n 2 f0, 1, . . . , Ng. Evidently, the rate of the interaction process of order n is c B(nI 2 , N ) . Figures 1C and 1D show realizations of an ensemble of 50 spike trains, generated according to the SIP and the MIP model, respectively. In both

Higher-Order Statistics of Input Ensembles

73

cases, the individual input rate r and the count correlation coefŽcient c were identical (r D 20 spikes per second, c D 0.1). The compound activity of all spike trains is shown below the spike rasters. Although any single spike train, as well as any pair of spike trains picked from the two ensembles, would be statistically indistinguishable, the population activity is clearly different for both models. 3 Results

The effects of both ensemble models were tested on two neuron models. To reduce the complexity of the input, we Žrst focused on a simpliŽed setting. We assume the input population to be composed of two subpopulations, one excitatory and one inhibitory, with the same size and rate of the excitatory and the inhibitory populations. The excitatory presynaptic spike trains were correlated, generated according to the SIP or the MIP model, whereas the inhibitory presynaptic spike trains were modeled as independent Poisson processes. We assessed the inuence of the input parameters, and in particular of the excitatory pairwise correlation, on the mean output rate of two types of model neurons: the FR neuron and the conductance-based leaky IF neuron. The output of the FR neuron is represented by a continuous function of time, the instantaneous Žring rate of the neuron, and could be treated analytically. The response of the IF neuron was studied by means of numerical simulations. In addition, the latter model was studied for inputs with different proportions of excitation and inhibition. 3.1 Response of the Firing Rate Neuron. The continuous FR neuron is widely used in neural network studies. Its membrane potential U ( t) is described by

C

1 d U ( t) D ¡ U ( t) C I ( t) , dt R

(3.1)

with C the membrane capacitance, R the membrane resistance, and I ( t) the input current. The membrane potential U (t) is a low-pass version of the input current I ( t) . The output Žring rate ro ( t) of the neuron is given by the activation function, which is assumed to be an instantaneous function of the potential. Here, we used the Heaviside step function H ( x ) , ( H ( x) D

0 1

for x < 0 for x ¸ 0.

(3.2)

In this case, the output rate of the neuron is expressed by ro ( t) D KH ( U ( t) ¡ h ) ,

(3.3)

74

A. Kuhn, A. Aertsen, and S. Rotter

where h represents the threshold and K is a constant with dimension spike per second. Alternatively, a sigmoidal function is often used as activation function. In our case, however, we used the step function because it greatly facilitates the analysis of the neuronal response. Since we are interested only in the qualitative behavior of the output rate, the constant K will not be considered any further, and ro ( t) will be dimensionless. An example of the response of the membrane potential U ( t) to the input described above is shown in Figure 2. Synapses were modeled by current pulses. The PSPs were thus represented by an instantaneous jump of potential (positive or negative, according to the excitatory or inhibitory nature of the synapse), followed by an exponential decay toward rest, with time constant t D RC D 10 ms. Excitatory and inhibitory PSPs had the same absolute amplitude, Žxed at a value of 1. The excitatory and inhibitory input population size was N D 100 in this example, and the individual excitatory and inhibitory input Žring rate was r D 20 spikes per second. The excitatory input was modeled by the SIP model (see Figure 2A), or by the MIP model (see Figure 2B), and the pairwise correlation coefŽcient was c D 0.4, in both cases. Observe that for both input models, the membrane potential shows large depolarizations caused by the occurrence of synchronous excitatory presynaptic spikes. These depolarizing events had, however, different characteristics for both input models, as we will see in more detail in the following. 3.1.1 Amplitude Distribution of the Membrane Potential. The probability density function (pdf) of the amplitude distribution of the membrane potential is represented in Figure 2 next to sample time courses of the membrane potential. The histogram is an estimate generated from a 200 s simulation, with a bin size of 4 (in units of the PSP amplitude). The thick black lines represent an analytical approximation of part of the pdf. As the output rate of the FR neuron is an instantaneous function of the membrane potential, knowledge of the pdf of the potential is sufŽcient to derive the mean output rate of the neuron. We now describe the main assumptions made to obtain the approximation of the pdf; a full derivation for the potential and the mean output rate is given in appendix B. The approximation of the pdf is based on the decomposition of the input into independent components (cf. section 2) and the use of a classical result of shot noise theory. The membrane potential of a FR neuron can be considered as a Žltered point P process. Assuming identical synapses, it can be represented by U (t ) D k h ( t ¡ tk ) , with t k denoting the times of occurrences of synaptic events and h ( t) the impulse response of the Žlter realized by the synapse and the membrane. For synapses modeled as current pulses, h (t) is simply the impulse response of the membrane, given by ( h ( t) D

0 Ae¡t / t

for t < 0 for t ¸ 0,

(3.4)

Higher-Order Statistics of Input Ensembles

A

120

B

SIP

120

40 0

C

120

0.2 0.4 time (s)

­ 40 0

10­ 3 10­ 1 p.d.f.

SIP, approximated

D

120

0.2 0.4 time (s)

10­ 3 10­ 1 p.d.f.

MIP, approximated

80 potential

potential

40 0

80 40 0 ­ 40 0

MIP

80 potential

potential

80

­ 40 0

75

40 0

0.2 0.4 time (s)

10­ 3 10­ 1 p.d.f.

­ 40 0

0.2 0.4 time (s)

10­ 3 10­ 1 p.d.f.

Figure 2: Time course (left) and probability density function (pdf, right) of the membrane potential of the Žring rate (FR) neuron. The membrane potential is expressed in units of the PSP amplitude. (A) The excitatory input to the FR neuron consisted of 100 excitatory spike trains, generated according to the SIP model (individual Žring rate is 20 spikes per second, pairwise correlation 0.4). The inhibitory input was composed of 100 independent Poisson processes with a rate of 20 spikes per second. The pdf estimated from a 200 s simulation of the membrane potential is shown on the right (histogram with a bin width of 4). A segment of the analytical approximation of the pdf (see text) is overlaid (thick black line). The dotted line indicates the Žring threshold. (B) Same input rate and pairwise correlation as in A, but using the MIP model for excitatory inputs. (C) Approximation (see text for details) of the membrane potential with the SIP model as excitatory input. The approximation retained the large uctuations caused by clusters of synchronous input spikes (compare with A). The estimated pdf of the approximated membrane potential (from a 200 s simulation of the approximated membrane potential, bar histogram) matched the analytically calculated pdf (thick black line, same as in A). (D) Approximation of the membrane potential using the MIP model for excitatory inputs. Again, the estimated pdf (histogram) matched the analytical expression (thick black line, same as in B).

76

A. Kuhn, A. Aertsen, and S. Rotter

with A being the postsynaptic potential amplitude and t the membrane time constant. If the synaptic events follow a Poisson statistics, the Žltered process is called a shot noise (Papoulis, 1991). If the rate of the incoming events is large compared to the time constant of the Žlter, the distribution of membrane potentials is approximately gaussian (see Papoulis, 1991). In the case of our two input models, however, the interaction structure does not admit a normal approximation. The right panels in Figures 2A and 2B show that the membrane potential densities are clearly not gaussian. Gilbert and Pollak (1960) derived an exact expression for the pdf of a shot noise with exponential impulse response. They showed that for A D 1 and t D 1, the shot noise U (t) with underlying rate of pulses l D 1, has density ( for 0 · U < 1 e ¡g P (U ) D ¡g e (1 ¡ log U ) for 1 · U < 2, R1 where g D ¡ 0 e ¡t log t dt, Euler ’s constant. In fact, the analytic expression of P ( U ) changes at U D 1, 2, 3, . . . The expression for a given segment is derived recurrently from the expression of the preceding segment. The case with parameters A, t , and l arbitrary can be extended from the results of Gilbert and Pollak (1960) and is shown in appendix B.

3.1.2 Analytic Approximation of Membrane Potential Distribution. What is the use of this result for the derivation of the membrane potential density of the FR neuron to the SIP and MIP input models? If the proportions of excitation and inhibition are equal (i.e., the input is balanced), the neuron’s output Žring rate is driven by the uctuations of the membrane potential above the threshold. Synchronous clusters of input spikes induce large excursions of the membrane potential that are likely to constitute the major contribution to the neuronal output rate. If, in addition, the nonsynchronous input spikes (excitatory or inhibitory) do not signiŽcantly contribute to the membrane potential excursions, but only to the mean membrane potential level, we can approximate the membrane potential by the Žltered interaction process (i.e., the process with simultaneous spikes), together with a constant membrane potential offset due to the nonsynchronous input spikes. For the input with the SIP model, the interaction process has Žxed amplitude N and rate rc, and the nonsynchronous input consists of an excitatory spike train with rate Nr (1 ¡ c) and an inhibitory spike train with rate Nr. The resulting mean membrane potential offset is given by Campbell’s theorem and amounts to ¡Nrct (see appendix B). We can write P (U ) ¼ PeN ( U C Nrct ) , where the pdf of the potential response to the SIP model is approximated by a shifted version of the pdf of the Žltered Nth-order interaction process PeN ( U ) . The latter can be derived on the basis of Gilbert and Pollak’s method.

Higher-Order Statistics of Input Ensembles

77

The rigorous justiŽcation of this approximation is given in appendix B. Figure 2C shows an example of the time course of the approximated membrane potential, with the same parameters as in Figure 2A (left panel). Observe that the small uctuations caused by the nonsynchronous input spikes disappeared, but their mean level was retained. A similar approximation can be made for input deŽned by the MIP model. To be able to use the shot noise density directly, we make the additional assumption that the stochastic spike clusters generated by the MIP model can be approximated by the average cluster; we replace the N C 1 interaction processes of order 0 to N by a single interaction process of order (amplitude) Nc and rate r/ c. The density of the potential response to the MIP model thereby becomes P (U ) ¼ PenN ( U C Nrt ) , where PenN (U ) is the pdf of the Žltered average interaction process and ¡Nrt the mean membrane potential offset, caused by the independent inhibitory input. Again, we refer to appendix B for a complete derivation of this result. An example of the time course of the approximated membrane potential response to the input with the MIP model is shown in Žgure 2D (left panel). Because of the piecewise expression of the shot noise pdf, the convenience of our approximation depends on the number of segments that must be taken into account. For a membrane’s impulse response of amplitude A (see equation 3.4), the expression changes its form at integer multiples of A. For the SIP model (see Figure 2C, right), the neuron’s response to a synchronized spike cluster has amplitude N. We used the Žrst segment of the pdf, extending over [¡Nrct, ¡N ( rct ¡ 1) [. (We use the conventional mathematical notation x 2 [a, b[ to indicate a · x < b.) For the parameter set considered here, the approximation of the potential holds for U 2 [¡8, 92[ in units of A. Comparison with the histogram estimated from 200 second simulation of the approximated membrane potential shows an excellent correspondence. For the MIP model (see Figure 2D, right), we used the Žrst two segments. As we reduced the model’s interaction structure to the average spike cluster, the neuron’s response to a synchronized cluster has amplitude Nc, and the Žrst two segments cover [¡Nrt, ¡N(rt ¡ 2c) [, which corresponds to [¡20, 60[. Note that the change of the analytic form of the pdf can be recognized from the sharp bend at U D ¡N ( rt ¡ c) D 20. Again, a numerical estimate from the simulation of the approximated membrane potential (see the histogram in Figure 2D, right) shows excellent correspondence. If we now compare the analytical approximation of the pdf with the pdf estimated from a simulation of the full input model (Figures 2A and 2B, right), the agreement is very good. A small discrepancy can be observed only near the lower bound of the domain considered. There, our approximation neglected the uctuations due to the nonsynchronous input spikes that spread the membrane potential around the mean value.

78

A. Kuhn, A. Aertsen, and S. Rotter

3.1.3 Mean Response Rate. We can now consider the mean output rate ro for the FR neuron, Z 1 (3.5) ro ´ E[ (ro (t) ] D P ( U ) dU D 1 ¡ Q (h ) , h

Rx

where Q ( x ) D ¡1 P ( U ) dU, the cumulative probability distribution function of the membrane potential. We thus obtain an approximation of the mean output rate by integrating over the approximated density, which has to be known between ¡1 and h . In Figure 2, the threshold h is represented by a dotted line and was set to 30. The number of segments of the pdf that must enter the calculation of ro is thus determined by the value of the input parameters, the threshold, and the membrane time constant. For the SIP model, an approximation of the mean output rate based on the Žrst segment of the probability distribution is µ ³ ´rct ¶ 1 e h C rct ro ¼ 1 ¡ D , N rct

(3.6)

R e D e ¡rct g / C ( rct ), where C ( z) D 1 tz¡1 e¡t dt is the gamma function. with D 0 It is valid for ¡rct · h / N < ¡(rct ¡ 1) . For larger h , additional segments of the distribution have to be included. For the MIP model, the approximation based on the Žrst two segments is given in appendix B in equation B.2. We used these approximations to study the output rate of the FR neuron in response to both types of input. The input was described by three parameters: the size N of the input population, the rate r of individual inputs, and the correlation coefŽcient c between any two individual excitatory inputs. N and r were identical for the excitatory and the inhibitory populations. We examined how the input correlation coefŽcient c inuenced the mean output rate ro of the FR neuron. In addition, we systematically varied N and r. The results are shown in Figure 3. Analytical approximations are represented by thick gray lines, and results obtained by numerical simulations are given by black dots. Figure 3A shows the output rate ro as a function of the input correlation coefŽcient c for several input population sizes N D 40, 100, 500, 1000 and a Žxed input rate of r D 20 spikes per second. The SIP model was used as the excitatory input ensemble. For all N, ro increased monotonically with the correlation coefŽcient. Indeed, as c grows larger, the rate of occurrence of simultaneous excitatory input spikes, given by rc, increases, and membrane potential excursions above threshold become more frequent, leading to an increase of the mean output rate. Nevertheless, observe that the overall increase of ro does not scale with N and tends to saturate rapidly for larger input population sizes. In the limit N ! 1, we have ro ¼ 1 ¡

( rct ) rct ¡1 ¡rct g . e C ( rct )

Higher-Order Statistics of Input Ensembles

79

This limit is plotted as a dashed line in Figure 3A. Figure 3B features the output rate of the FR neuron for identical input conditions, with an interaction structure of the excitatory population generated according to the MIP model. For small values of c, the response of the FR neuron showed a considerably stronger dependence on the ensemble size than in the SIP model. For N D 40, for instance, the initial increase of ro was smaller than with the SIP model. By contrast, for N D 1000, a small input correlation induced a much larger output rate with the MIP model. For larger N, a further increase of c led to a decrease of ro . Note that for c D 1, both correlated input models are statistically identical, and the responses of the FR neuron were thus the same. For both the SIP and MIP input models, the analytical approximations of ro for the FR neuron model were in very good agreement with the simulation results, showing that the assumptions made were appropriate. As hypothesized, the response of the neuron is driven by the occurrence of excitatory spike clusters for the largest part of the correlation range. If c approaches 0, the rate of these synchronous events tends to 0, and the approximate treatment of the SIP model degrades. The reason is that it neglects the contribution of nonsynchronous excitatory input spikes, the rate of which is Nr (1¡c), increasing for smaller c. This is shown in the inset of Figure 3A, depicting the output rate of the FR neuron for c D 0.005, 0.01, . . . , 0.1. Whereas the approximation tends to 0, the neuron response to the full SIP input model has nonzero values at c D 0, for N D 1000. The success of the approximation near c D 0 thus critically depends on the values of r and N. For the response to the MIP model, we limited the approximation to the Žrst two segments of the pdf and thus could not approximate the response of the FR neuron over the whole range of c (see Figure 3B). For small correlation coefŽcients, the size of the mean synchronous spike cluster Nc is small compared to the threshold, and many segments of the shot noise density must be taken into consideration. The success of the analytical approximation here indicates that the stochastic nature of the size of synchronous clusters does not play a signiŽcant role for the mean output rate. This independence of the stochastic nature of the cluster size allows us to explain the observed effects in terms of the average cluster. For small values of c, the rate r/ c of the mean cluster is large. The frequent occurrence of small clusters (with size proportional to c) results in their accumulation in the membrane potential, which drives the potential above threshold h (see Figure 2B) and increases the output rate. Nevertheless, the rate of occurrence of the synchronous input events decreases with increasing c. The clusters become larger but less frequent with increasing c. If their mean inter-event interval (inverse rate) is smaller than the membrane time constant, they do not accumulate, and the increase of the cluster size cannot compensate for their less frequent occurrence. For a large input population size, these two opposing effects result in a non-monotonic dependence of the output rate on the input correlation (see Figure 3B).

80

0.05

0.3

0 0

0.05 0.1

0.2

100

0.1 0 0

C

0.5

D

SIP 90 s­ 1 ­1 50 s

output

20 s­ 1 10 s

0.1

0.3

1000 500

0.2 0 0

0.2 0.4 0.6 0.8 1 correlation coefficient

0.3

0 0

MIP

100

0.1

40

0.4 0.2

0.5 0.4

¥ 1000 500

output

output

0.4

B

SIP

0.5

­1

0.5 0.4

output

A

A. Kuhn, A. Aertsen, and S. Rotter

0.3 0.2

40

0.2 0.4 0.6 0.8 1 correlation coefficient MIP 90 s­­ 11 50 s 20 s­ 1 ­1

10 s

0.1 0.2 0.4 0.6 0.8 1 correlation coefficient

0 0

0.2 0.4 0.6 0.8 1 correlation coefficient

Figure 3: Mean output of the Žring rate (FR) neuron as a function of the pairwise correlation coefŽcient in the population of excitatory inputs, for the two statistical input models. Relative units are used (see text). Thick gray lines are obtained from analytical approximations (see text); the black dots represent the results of simulations (duration 500 s each). (A) The excitation was generated according to the SIP model, and the inhibition was represented by a set of independent Poisson processes. Excitatory and inhibitory population size and rate were identical. The output rate of the FR neuron is shown for different input population sizes (N D 40, 100, 500, 1000). The individual input rate was held constant (r D 20 spikes per second). The analytically computed output rate of the FR neuron for inŽnite population size is plotted as a dashed line. Inset: Zoom-in of the graph for low pairwise correlation coefŽcients (0 · c · 0.1). (B) The excitation was generated according to the MIP model, everything else as in A. (C) Same input setting as in A, but for a Žxed input population size (N D 500) and different individual input rates (r D 10, 20, 50, 90 spikes per second). (D) The excitation was generated according to the MIP model. Everything else as in C. Note that for c D 0 and c D 1, the SIP and the MIP models coincide.

Figure 3C shows the output rate ro for a Žxed population size of N D 500 and for different input rates r D 10, 20, 50, 90 spikes per second, with the SIP model being used as excitatory input ensemble. The output rate ro increased with increasing c, and more so for larger r. The form of the dependence of ro on c was more saturating for larger r. The output rate of the FR neuron with identical input parameters, but using the MIP model as excitatory input, is

Higher-Order Statistics of Input Ensembles

81

shown in Figure 3D. We observe a distinct difference with the response to the SIP input, particularly for small input correlations. Here, for small c, an increase of c leads to a large increase of ro due to the large N. A consecutive decrease of ro for larger c occurred for small r only. Indeed, for r D 50, 90 spikes per second, the inter-cluster interval did not become small compared to the membrane time constant. After having explored the effects of each of the three input parameters, we now consider how modiŽcations of the FR neuron and, more precisely, of the membrane time constant t , the thresholdh of the activation function, and the form of the activation function could affect these results. With regard to t , note that the output rate of the FR neuron with the SIP model (see equation 3.6) depends only on the product rt . Thus, an increase (decrease) of the time constant has the same effect as the proportional increase (decrease) of the input rate. This is the case for the density function of any shot noise with exponential kernel (Gilbert & Pollak, 1960). Taking into consideration that the membrane potential of the FR neuron response (for both the SIP and the MIP models) can be decomposed into a sum of several elementary shot noise processes with different amplitudes and rates (see appendix B), it can be concluded that the equivalence of t and r is valid for all input parameters of both input models. An analogous but somewhat weaker statement can be made concerning the threshold h of the activation function. The output rate of the neuron depends only on the quotient h / N (see equation 3.6). This is the case for the approximated response to both the SIP and the MIP models (see appendix B). Thus, as long as the approximation is good, the effect of an increase of h is equivalent to the proportional decrease of N. Finally, we turn to the effects of the form of the activation function. For an arbitrary function, it would be necessary to know the entire density function of the potential to be able to derive the mean output rate of the FR neuron. The piecewise deŽnition of the density function, however, makes it difŽcult to derive an approximation of the output rate. Nevertheless, our method would still be applicable, and an approximation of the neuronal rate could be derived, if the input clusters had a low rate. Indeed, in this case, the entire density function is well approximated by the Žrst few segments. Note that if the activation function was linear, the mean output rate of the FR neuron would not be sensitive to any aspect of the input statistics other than the mean input rate, because the mean output of a linear system is determined by its mean input only. A nonlinear activation function is thus a prerequisite for the sensitivity of the neuron to input correlations and, a fortiori, to higherorder statistics. In turn, any sufŽciently nonlinear activation function will result in different mean responses of the FR neuron to both input models. Indeed, the response potential densities to both the SIP and MIP model are different, and the mean output rate will typically be different. 3.2 Response of the Integrate-and-Fire Neuron. In order to compare the effects of different input ensemble statistics under more realistic condi-

82

A. Kuhn, A. Aertsen, and S. Rotter

tions, we simulated a conductance-based leaky IF neuron (Tuckwell, 1988; Koch, 1999). Its membrane potential was (on average) depolarized with respect to the resting state, and it uctuated. Its input conductance was also increased, similarly to what is typically observed in vivo (ParÂe, Shink, Gaudreau, Destexhe, & Lang, 1998). The essential difference to the membrane potential dynamics of the FR neuron is that synapses here were modeled not as current pulses but as transient membrane conductance changes. The subthreshold membrane potential of the IF neuron was described by 1 d U ( t) D [Ur ¡ U ( t) ] C Ge ( t) [Ue ¡ U (t )] C Gi ( t) [Ui ¡ U (t) ]. dt R Ur is the resting membrane potential. Ge ( t) and Gi ( t) are the excitatory and inhibitory synaptic conductances, and Ue and Ui are the excitatory and inhibitory synaptic reversal potentials, respectively. We use ge (t) and gi (t ) to denote the conductance changes elicited by a single excitatory or inhibitory presynaptic spike. Both excitatory and inhibitory conductance changes were modeled by an a-function (Koch, 1999; Rotter & Diesmann, 1999), X ge ( t) D ge ate1¡at H ( t) , Ge (t) D ge (t ¡ tek ) , C

gi ( t) D gi ate1¡at H ( t) ,

Gi (t) D

k X k

gi ( t ¡ tik ) ,

where ge (gi ) is the peak excitatory (inhibitory) conductance change, and 1 / a is the time constant of the conductance change. Both types of synapses had the same time constant. The times tek and tik represent the occurrences of excitatory and inhibitory input spikes, respectively. The voltage-dependent conductances responsible for the action potential generation were not included in the model. Instead, when the membrane potential reached a threshold Uh , an action potential was generated and the membrane potential was clamped to a reset value Ureset for a period trefr . In addition, all synaptic currents were shunted. This latter mechanism modeled the refractory period. The various model parameter values were chosen to be in line with the experimental literature (McCormick, Connors, Lighthall, & Prince, 1985): C D 500 pF, R D 30 MV, Ur D ¡70 mV, Uh D ¡50 mV, Ureset D ¡60 mV (Troyer & Miller, 1997), and trefr D 2 ms. The excitatory synapses modeled fast glutamatergic (AMPA) synapses, and the inhibitory synapses modeled fast GABAergic (GABAA ) synapses. We set Ue D 0 mV and Ui D ¡70 mV (Koch, 1999). The time constant of the synaptic conductance changes was 1 / a D 1 ms, and the peak conductances were ge D 1 nS and gi D 3.4 nS, respectively. 3.2.1 Membrane Potential Response. A depolarized membrane potential was obtained by bombarding the neuron with Poisson spike trains. We used independent excitatory and inhibitory spike trains whose rates were

Higher-Order Statistics of Input Ensembles

83

adjusted such that the membrane potential uctuated around a mean value of ¡54.3 mV (LÂeger, Stern, Aertsen, & Heck, 2002). We used an excitatory rate of 9000 spikes per second and an inhibitory rate of 5500 spikes per second, which is equivalent to 9000 independent individual excitatory inputs and 5500 independent individual inhibitory inputs, both with a rate of 1 spike per second. We refer to this input as background input. The membrane potential uctuated with a standard deviation of 1.5 mV, and the model neuron Žred with a rate of 1 spike per second under these conditions. The excitatory postsynaptic potential (EPSP) or inhibitory postsynaptic potential (IPSP) produced by an additional excitatory (inhibitory) input spike had, on average, an amplitude of 0.16 mV (¡0.15 mV). This is in accordance with Matsumura, Chen, Sawaguchi, Kubota, & Fetz (1996), who found that EPSPs and IPSPs recorded intracellularly in vivo had similar amplitudes and time courses. We let two additional populations of excitatory and inhibitory inputs impinge on the neuron. Both populations had the same size, and the individual excitatory and inhibitory discharge rates were identical, as with the FR neuron. The excitatory spike trains were correlated, with parameters represented by either the SIP or the MIP model; the inhibitory inputs were always independent. Figure 4A shows a 500 ms realization of the excitatory and inhibitory population activity (bottom), with N D 100 and r D 20 spikes per second. Both the input and the IF neuron were simulated with a time step of 0.1 ms. In the Žgure, the number of excitatory input spikes per time step is plotted as a positive integer and the number of inhibitory input spikes per time step as a negative integer. The SIP model was used for the excitatory population, and the correlation coefŽcient was c D 0.1. Note that the background synaptic input is not shown. The membrane potential response of the IF neuron is plotted in the upper part of the Žgure. It uctuated randomly, characterized by occasional fast depolarizations leading to an output spike. These were mostly caused by spike clusters involving the whole correlated excitatory population. Note that the second spike in Figure 4A was due to the background of independent synaptic input. The same input parameters were used for Figure 4B, but here the MIP model replaced the SIP model. The excitatory input activity consisted of a large number of clusters of smaller size; the output spikes resulted from the summation of several input clusters. This summation mechanism was rendered stochastic by the background uctuations of the membrane potential. Due to the reduced size of the input clusters, the depolarization preceding a response spike was slower than with the SIP model. In Figure 4C, we used the SIP model but increased the pairwise correlation coefŽcient to 0.4. As a consequence, the rate of occurrence of the input clusters increased, accompanied by a corresponding increase of the output rate. Figure 4D shows the response of the neuron to the MIP model, with c D 0.4. The size of the clusters increased compared to c D 0.1 (see Figure 4B), but their rate of occurrence decreased, as already explained in section 3.1.

84

A. Kuhn, A. Aertsen, and S. Rotter

­ 70 100 50 0 0

input

­ 60

0.1 0.2 0.3 time (s)

0.4 0.5

­ 50 ­ 60 ­ 70 100 50 0 0

0.1 0.2 0.3 time (s)

MIP, c = 0.1 ­ 50 ­ 60 ­ 70 100 50 0 0

0.1

D

SIP, c = 0.4

U (mV)

U (mV)

U (mV)

­ 50

C

input

B

SIP, c = 0.1

input

input

U (mV)

A

0.4 0.5

0.2 0.3 0.4 time (s)

0.5

MIP, c = 0.4 ­ 50 ­ 60 ­ 70 100 50 0 0

0.1

0.2 0.3 0.4 time (s)

0.5

Figure 4: Time course of the membrane potential of the leaky integrate-andŽre (IF) neuron (top) and the corresponding input (bottom), with the excitation generated by either the SIP or the MIP model. (A) The excitation was generated according to the SIP model, with a pairwise correlation coefŽcient of 0.1; the inhibition was a population of independent Poisson processes. The input population size (N D 100) and individual Žring rate (r D 20 spikes per second) were the same for the excitation and the inhibition. The number of excitatory (inhibitory) input spikes per bin (size 0.1 ms) is plotted as a positive (negative) number in the lower panel. Note that the additional independent background inputs that kept the membrane potential in a depolarized state (see text) are not shown. The dotted line in the upper panel indicates the Žring threshold. (B) The excitation was generated according to the MIP model. The pairwise correlation and other input parameters were identical to A. (C) The excitation was generated according to the SIP model, with an excitatory input pairwise correlation coefŽcient of 0.4. Other parameters as in A. (D) The excitation was generated according to the MIP model. The pairwise correlation and other input parameters were identical to C.

The output rate was larger than with c D 0.1. However, the clusters were still not large enough to reliably elicit an output spike individually. 3.2.2 Mean Response Rate. Similarly to the FR neuron, we compared the response of the IF neuron for the two different input models SIP and

Higher-Order Statistics of Input Ensembles

85

MIP by studying the dependence of the output Žring rate on the pairwise correlation of the excitatory input. In addition, we systematically varied N and r. Figure 5A shows the output rate ro of the IF neuron to SIP input as a function of c, for a Žxed input rate r D 20 spikes per second, and for different input population sizes N D 20, 40, 100, 1000. The output rate slowly increased with the input correlation coefŽcient. For population sizes beyond 100, the output rate ro did not increase any further. This behavior was caused by the membrane potential resetting of the IF neuron. Evidently, if the input population size is large enough that the depolarizations induced by an input cluster reach the threshold, a further increase of N has no impact on the output rate because the rate of occurrence of the input clusters does not depend on N. In that case, the mean output rate ro of the IF neuron can thus be approximated by (see Murthy & Fetz, 1994) ro D r (1 ¡ trefr ro ) ,

(3.7)

where r is the rate of input clusters. For the SIP model, we have r D rc. This approximation is plotted as a thick gray line in Figure 5A and expresses that the output rate ro is equal to the rate of spike clusters r, subtracting the clusters arriving while the neuron is refractory. Of course, the approximation is not valid if output spikes are (also) caused by the background bombardment of non-synchronized excitatory spikes (as is the case for c D 0, for instance). Figure 5B shows the response of the IF neuron as a function of c but with the MIP model for the excitation and for r D 20 spikes per second and N D 20, 40, 100, 300, 500, 1000. For small values of c, the dependence of the output rate on c was far more strongly modulated than with the SIP model. For N D 1000, for instance, a small c generated a very large output Žring rate, much larger than with the SIP model. Only for very small populations (N D 20) was the output rate smaller than with the SIP model, over the whole range of c (see Figure 5A, inset). For large N, as for the FR neuron, the dependence of ro on c was non-monotonic. The initial increase of ro with c was caused by the integration of increasingly large input clusters. However, when the clusters were large enough that the occurrence of a single input cluster reliably triggered an output spike, a further increase of the input correlation was detrimental to the output Žring rate. Indeed, a further increase of c, resulting in even larger input clusters, causes ro to decrease, because the occurrence rate of clusters decreases with increasing c. For these larger values of c, the mean output rate ro of the IF neuron can again be approximated by equation 3.7. We assume that the stochastic nature of the spike cluster size does not play a role and replace each cluster by one of average amplitude. The rate of the mean amplitude clusters is r/ c. The approximation given by equation 3.7, with r D r/ c, is represented as a thick gray line in Figure 5B. The amplitude of the mean input spike cluster is proportional to N, and the range of validity of the approximation is larger

86

A. Kuhn, A. Aertsen, and S. Rotter

for large values of N. Note that for larger values of ro , the approximation slightly overestimated the output rate. This is due to the Žnite rise time of the conductances used to model the synapses, which allowed synchronized clusters to impinge on the cell during the depolarization caused by a Žrst cluster. Such an event happened more frequently at higher rates. Figure 5C shows the mean output rate of the IF neuron with the SIP model, for N D 500, and r D 10, 20, 50, 90 spikes per second. N was large enough so that each input cluster elicited an output spike. The relation between ro and c depended on r, because the rate of occurrence of the input clusters is proportional to r. The mean output rate was approximated by equation 3.7, with r D rc (thick gray lines). Again, note that the approximation is more precise for small values of ro . We replaced the a-function conductance with a conductance pulse and repeated these simulations. In that case, the (small) discrepancy between the simulated output rate and the approximation, observed for larger values of ro , disappeared completely (data not shown). The response to exactly the same input parameters, but with the MIP model instead of the SIP model, is shown in Figure 5D. As N was large, a non-monotonic dependence of ro on c could be observed. This was the case for all r. The approximation of the mean output rate ro (see equation 3.7, r D r/ c) is plotted as thick gray lines. For the different input rates r, the left bound of the range of the approximated output rate was chosen as the value of c producing the maximal output rate in the numerical simulations. The approximation gave better results for small values of output rates, as explained above. For large r (50, 90 spikes per second), two local maxima appeared in the simulation results for the dependence between ro and c. For the smaller value of c, the interaction structure was thus (locally) optimal with regard to the generation of output spikes, so that both an increase and a decrease of c caused a decrease of ro . To Žrst approximation, we can make the same assumption here as for the response of the FR neuron and explain this effect in terms of the mean cluster. The mean cluster generated by the MIP model, for a value of c producing the locally maximal output rate, was not big enough to depolarize the membrane potential up to the spiking threshold alone and thus needed support from other inputs. An efŽcient arrangement consists of pairs of clusters whose sizes were such that within the integration time of the membrane, they jointly cause a depolarization that can just bring the neuron to Žre a spike. A pair of larger clusters would result in the same effect, but it would occur less frequently, as the mean cluster rate is inversely related to its mean size. Note that the positions of both peaks are shifted to higher values of c for increasing input rate r. This is because not only excitation but also inhibition increases with r, necessitating larger clusters to reach the threshold. Interestingly, this can lead to a nonmonotonic dependence of ro on r. For c around 0.35, for instance, an increase of the input rate from 50 to 90 spikes per second resulted in a decreased output rate.

Higher-Order Statistics of Input Ensembles

140 120 100 80 60 40 20 0 0

B

SIP 2 0 0

SIP®

­1

output rate (s )

4

®MIP N=20 0.5 1 100, 1000 40 20

0.2 0.4 0.6 0.8 1 correlation coefficient

D

SIP

­1

­1

output rate (s )

C

120 100 80 60 40 20 0 0

output rate (s )

­1

output rate (s )

A

90 s­ 1 ­1

50 s

s­­ 11

20 10 s

0.2 0.4 0.6 0.8 1 correlation coefficient

87

MIP 120 100 1000 80 60 40 20 40 20 0 0 0.2 0.4 0.6 0.8 1 correlation coefficient 140 120 100 80 60 40 20 0 0

MIP

90 s­ 1 ­1

50 s

20 s­­ 11 10 s

0.2 0.4 0.6 0.8 1 correlation coefficient

Figure 5: Mean output rate of the leaky integrate-and-Žre (IF) neuron as a function of the pairwise correlation coefŽcient of the excitatory input, for the two different input ensemble statistics. Thick gray lines represent an analytical approximation of the output Žring rate under the assumption of large spike clusters in the input (see text for details). (A) The excitation was generated according to the SIP model; the inhibition was a set of independent Poisson processes. Excitatory and inhibitory population size and rate were identical. The output rate is shown for different input population sizes (N D 20, 40, 100, 1000) and constant individual input rate (r D 20 spikes per second). The output rate for N D 100 and N D 1000 produced undistinguishable curves. The inset compares the response of the IF neuron for the SIP and the MIP input model as excitation, for N D 20 and r D 20 spikes per second. It shows that for small population sizes, the output rate with the SIP model was larger over the whole range of c. (B) The excitation was generated according to the MIP model. The output rate is shown for different input population sizes (N D 20, 40, 100, 300, 500, 1000) and a Žxed input rate (r D 20 spikes per second). (C) Same input setting as in A, for different individual input rates (r D 10, 20, 50, 90 spikes per second) and a Žxed population size (N D 500). (D) The excitation was generated according to the MIP model. All input parameters as in C. Again, note that for c D 0 and c D 1 the SIP and the MIP model coincide.

88

A. Kuhn, A. Aertsen, and S. Rotter

The input scenarios discussed thus far were composed of equal proportions of excitation and inhibition. The ratio of excitation and inhibition received by a cortical neuron in vivo, however, can only be roughly estimated. Anatomical data suggest that excitation amounts to more than four times the inhibition (Braitenberg & Schuz, ¨ 1998). On the other hand, it has been argued on the basis of the irregularity of the discharge of cortical neurons that they operate with equal proportions of excitation and inhibition (van Vreeswijk & Sompolinsky, 1996; Shadlen & Newsome, 1998). We deŽne the balance of the input, bD

Ni ri , Ne re

where Ne (Ni ) is the excitatory (inhibitory) input population size and re (ri ) the Žring rate of individual excitatory (inhibitory) input lines. We varied the input balance of the IF neuron by changing the amount of inhibition (the product Ni ri ). Note that the same variation of b could be obtained by changing either Ne or re , not necessarily leading to exactly the same effects. Figure 6A shows the mean output rate of the IF neuron as a function of the excitatory pairwise correlation, for different levels of inhibition, using the SIP model for the excitatory input. The excitation was composed of Ne D 300 and re D 20 spikes per second. The inhibition was Ni ri D 0, 1200, 3000, 6000, or 9000 spikes per second, corresponding to b D 0, 0.2, 0.5, 1, 1.5. For low levels of inhibition (b D 0, 0.2, 0.5), the output Žring rate for weak pairwise correlations in the input was much larger than for the balanced input. For these values of b, an increase of c caused a decrease of the output rate. The occurrence of excitatory clusters of size Ne induced large depolarizations—larger than actually necessary to reach the spiking threshold. The excess excitatory spikes hence did not contribute to generating output spikes, which had a detrimental effect on the output rate. Observe that for b D 0.5, ro decreased down to a value of 10 spikes per second at c D 0.45 and increased up to about 20 spikes per second at c D 1. Thus, the initial decrease of ro , caused by spikes participating in the (too) large synchronization clusters, could be partially compensated for by the increased cluster rate. Using more inhibition than excitation (b D 1.5), no output spikes were produced for independent inputs. Due to the hyperpolarized membrane potential, the uctuations of the potential were not large enough to induce spontaneous spiking. Nevertheless, an increase of the input correlation and the associated occurrence of synchronized clusters brought the neuron to Žre, with an output rate again tending to the maximal rate of 20 spikes per second for c D 1. This is shown by the overlap with the thick gray line, representing the approximation expressed by equation 3.7 (with r D rc). The response of the IF neuron to the same input parameters, but with the MIP model instead of the SIP model for the excitatory input, is shown in Figure 6B. We again observe considerable differences with the response

Higher-Order Statistics of Input Ensembles

80

SIP b=0

60 40 20 0 0

B ­1

100

output rate (s )

­1

output rate (s )

A

¬b=1,1.5

0.2 0.4 0.6 0.8 1 correlation coefficient

89

100 80

MIP b=0

60 40 20 0 0

b=1.5

0.2 0.4 0.6 0.8 1 correlation coefficient

Figure 6: Mean output rate of the leaky integrate-and-Žre (IF) neuron as a function of the pairwise correlation coefŽcient of the excitatory input, for the two different input ensemble statistics, with different ratios of inhibition to excitation. Thick gray lines represent an analytical approximation of the output Žring rate, assuming that output spikes are caused only by large spike clusters in the input. (A) The excitation was generated according to the SIP model. The excitatory population size was Ne D 300, and the excitatory individual Žring rate was re D 20 spikes per second. The product of the inhibitory population size and rate was Ni ri D 0, 1200, 3000, 6000, 9000, corresponding to an input balance (see text) of b D 0, 0.2, 0.5, 1, 1.5. The response of the IF neuron for b D 1.5 and b D 1 differed only for very low excitatory input correlations. (B) The excitation was generated according to the MIP model. All input parameters were identical to A.

to the SIP input, particularly for small input correlations. At small values of c, a small increment induced a large increase of ro for b D 1 and 1.5. For b D 0.2, a small increase of c had little inuence on ro , and for b D 0, ro decreased with c. Small values of c generated small synchronized clusters that could drive the output rate up or down, depending on the net excitation level. This is because for high inhibition levels, the membrane potential uctuates strongly, and an increase of excitatory input synchrony results in an even higher variance of the membrane potential, leading to more frequent threshold crossings. By contrast, if the excitation level is high, the synaptic input, on average, tends to depolarize the neuron above threshold (see Shadlen & Newsome, 1998). In that case, the summation of rapidly occurring excitatory spikes is efŽcient, and synchrony does not greatly accelerate the depolarization of the membrane potential to threshold. For b D 0.2, ro was initially insensitive to changes in c, because the loss of spikes in the synchronized clusters was exactly compensated by the greater efŽcacy of synchronous input. The degree of input balance at which this effect occurs depends on the input parameters N and r. For larger c, as N was large, the effect of synchrony was detrimental for all b values. As the output spikes were then exclusively caused by clusters of synchronous

90

A. Kuhn, A. Aertsen, and S. Rotter

input spikes, the mean response rate was well described by the approximation expressed by equation 3.7 (with r D r/ c), plotted as a thick gray line in Figure 6B. The interaction structure of the input can thus have a dramatic inuence on the output Žring rate. A striking example is given by the response of the IF neuron for b D 0.5. With the interaction structure of the SIP type, an increase of c from 0 provoked a decrease of the output Žring rate, but with the MIP model, for the same input balance level, the same increase of correlation induced an increase of the output rate. 4 Discussion

Individual Žring rates and pairwise correlations of a population of neurons do not determine the ensemble uniquely. We showed that neurons can be very sensitive to the details of the interaction structure of their input ensembles. Thus, it is difŽcult to draw any deŽnitive conclusions about the effect of particular input parameters (like pairwise correlation) on a neuron’s output rate, as long as physiological input ensembles are not characterized in a more complete manner, in particular, by specifying their higher-order correlation structure. 4.1 Input Ensemble Statistics. In model studies, it is often assumed that the total input or the membrane potential can be represented by a process with a gaussian probability density function (Tuckwell, 1988; Feng & Brown, 2000; Svirskis & Rinzel, 2000). Doing so has the advantage that the statistical properties of the model can be constrained easily. Nevertheless, as the third and higher cumulants are forced to be zero, this model is obviously not suited to study the impact of the full ensemble statistics. As we have shown in this work, it is crucial to take all moments of the input ensemble into account. Several studies have tackled the impact of correlated spike trains on the output of neuron models. Shadlen and Newsome (1998) and Salinas and Sejnowski (2000) generated correlated spike trains by simulating neurons with partial common input. Two sets of inputs were randomly drawn from a Žnite pool, resulting in a certain fraction of the inputs being used for both neurons. This introduces a correlation in the output spike trains of the two neurons. Such spike trains have broader cross-correlograms, which is more realistic than the exact synchrony used in this study. However, the higher-order interactions in such ensembles of spike trains were not considered. Another drawback is that the ensembles were not deŽned analytically. Destexhe and ParÂe (1999) used correlated synaptic input to reproduce the neuronal properties observed in vivo. They used Poisson processes as inputs and introduced stochastic dependence by drawing a smaller number of input processes than there were synapses (similarly to Salinas & Sejnowski, 2000), a single Poisson process being used as input to several synapses.

Higher-Order Statistics of Input Ensembles

91

Again, the relation between the pairwise correlation and the higher-order correlation structure was not stated explicitly. Statistically, the interaction structure of an input ensemble has a large number of degrees of freedom. We picked two extreme situations. In the SIP model, all of the doublets of spikes on any pair of spike trains occur simultaneously, resulting in spike clusters synchronized over the whole population. In the MIP model, the cluster size is a stochastic variable with a mean increasing linearly with the pairwise correlation. To realize a pairwise correlation coefŽcient identical to the SIP model, the average cluster rate must be higher than in the SIP model. Some studies used an input ensemble related to the MIP model (Murthy & Fetz, 1994; Bernander, Koch, & Usher, 1994) in order to investigate the effects of input synchrony. Their input ensemble differed from ours in that the rate of occurrence of spike clusters was held constant. As a consequence, they parameterized their ensemble in terms of the absolute synchrony (the size of spike clusters), and not in terms of the correlation coefŽcient. Interestingly, Bernander et al. (1994) also introduced a temporal jitter of the synchronization and showed that the precision of the spike correlation can strongly modulate the output Žring rate, depending on the values of the input parameters. The issue of the ensemble statistics was speciŽcally tackled by Bohte et al. (2000), who studied an input distribution with maximal entropy. Assuming homogeneous rates and homogeneous pairwise correlations, they derived a maximally at distribution of input clusters. The size of an input cluster was deŽned as the number of neurons spiking within a single time bin. They showed that under the assumption of maximal entropy, spike clusters involving a large number of neurons already exist for a small pairwise correlation. The SIP and MIP models were chosen because they represent two extreme cases of input ensembles. The question arises of how similar they are to the inputs that a real cortical neuron receives. To date, this question remains largely unanswered. Let us mention two lines of research that could help solve this problem. The Žrst is the massively parallel recording of multiple single cell activities in vivo (Nicolelis, 1998). Statistical methods suited to the analysis of the corresponding data are beginning to be available (Gerstein & Aertsen, 1985; Martignon et al., 1995, 2000; Gutig ¨ et al., 2002). Nevertheless, the pool of neurons forming the actual presynaptic input ensemble to a single cortical neuron cannot be inferred from such studies. Therefore, a second source of answers could come from the detailed investigation of membrane potential uctuations gained from intracellular recordings in vivo (Borg-Graham, Monier, & Fre gnac, 1998; Azouz & Gray, 1999; LÂeger, Stern, Aertsen, & Heck, 2002). Analysis of membrane potential uctuations or their time courses, possibly combined with simultaneous recordings from neighboring cells, could help to shed light on the interaction structure of the input ensembles seen by cortical neurons.

92

A. Kuhn, A. Aertsen, and S. Rotter

4.2 Dependence of Response on Input Ensemble Statistics. We used the SIP and MIP models of spike train ensembles to describe the excitatory input into neurons. The inhibitory inputs were assumed independent throughout this study. With the SIP model and for a Žxed pairwise input correlation, an increase of the input population size induced only a limited increase of the output rate for two very different neuron models (see Figures 3A and 5A). This was true at all input rates and over the entire range of input correlations. Interestingly, this was the case not only for the IF neuron but also for the FR neuron, whose membrane potential did not undergo any reset following a threshold crossing. Probing the model neurons with an interaction structure generated by the MIP model, we observed that for any Žxed pairwise correlation in the input, the range of output rates obtained by varying the size of the input ensemble was much larger than with the SIP model (see Figures 3B and 5B). For small input correlations and in contrast to what happened with an interaction structure of the SIP type, the output rate increased steadily with the size of the input ensemble and could thus attain much larger values. The most striking difference between the response to both interaction structures was observed for large input populations. By large, we mean that an input spike cluster involving the whole population elicits a membrane depolarization that reaches the Žring threshold. With the SIP model, the rate of spike clusters was proportional to both the individual input rate and the pairwise correlation. For a large input population, the output rate of the IF neuron was thus approximately linearly related to these two input parameters (see Figure 5C). By contrast, with input statistics of the MIP type, the dependence of the IF response on the input Žring rate of a large input ensemble could be non-monotonic (see Figure 5D). Note that for the FR neuron, the variance of the membrane potential was the same with the SIP or the MIP input model. Indeed, the autocovariance function (and thereby the variance) of a linear system’s output is entirely determined by the autocovariance function of its input (Papoulis, 1991). As the autocovariance of the summed input ensemble is given by the sum of the autocovariances and cross-covariances of the individual spike trains (both of which were identical for the SIP and the MIP input models), the autocovariance of the membrane potential is identical for both input ensembles. Thus, the size of the uctuations, as measured by the variance of the membrane potential, does not alone explain the output rate response, as shown by the very different response of the FR neuron to both input ensembles (see Figure 3). This also implies that the assumption of a gaussian distribution of the membrane potential, which requires knowing only the Žrst and second moments of the potential, may fail to predict the mean response rate. Moreover, it can be easily shown that the variance of the membrane potential increases (monotonically) with the input pairwise correlation c (see appendix B for the complete expression of the variance of the membrane potential). Thus, despite an increasing vari-

Higher-Order Statistics of Input Ensembles

93

ance of the membrane potential, the response rate can decrease (see Figures 3B and 3D), depending on the higher-order interactions of the input ensemble. Many of the neuronal response properties investigated here were similar for both the FR and the IF neuron. The differences could originate from either the postspike reset mechanism built into the IF neuron or from the use of conductances to model the synapses. Tiesinga, JosÂe, and Sejnowski (2000) studied the differences in the neuronal response to an uncorrelated input ensemble when using currents or conductances to model synapses. In the case of the MIP model, we have previously shown that the mean output of a continuous FR neuron with reversal potentials can be in very good correspondence to the output rate observed in an otherwise identical neuron that included a membrane potential reset mechanism (Kuhn, Rotter, & Aertsen, 2002). Hence, depending on the higher-order statistics of the input ensemble, input parameters can have very different effects on the neuron’s output rate. We studied the effects of pairwise correlation in the excitatory population across its entire range from 0 to 1. However, in experimental studies, the correlation coefŽcient is rarely found to be larger than 0.1 to 0.2 (Aertsen & Gerstein, 1985; Das & Gilbert, 1999; Eggermont & Smith, 1995; Nelson, Salin, Munk, Arzi, & Bullier, 1992). Considering this range of input correlations, it appears that the two different realizations of input correlations considered here impose different constraints on the way the neuronal response can be modulated. If the activity of a presynaptic network is characterized by the synchronization of large pools of neurons, the neuronal output is approximately linearly modulated by the input rate. By contrast, if the population activity of the presynaptic network is composed of small, distributed clusters, the size of the input population constitutes an ideal parameter to modulate the output Žring rate of the postsynaptic neuron over a large range. The latter case would thereby suggest an operation mode based on the dynamic recruitment of input lines. 4.3 Balance of Excitation and Inhibition. We changed the mean ratio of inhibition to excitation of the input (the input balance) and observed that the effects of pairwise correlation in the excitatory population on the output rate could be reversed. With the SIP model as excitatory input structure, the monotonic increase of the output rate as a function of the correlation, observed for equal proportions of excitation and inhibition, could change into a monotonically decreasing relation (see Figure 6A). Such modulation of the relative effect of synchrony by the net excitatory level was also observed with the MIP model as interaction structure for low values of the input correlation coefŽcient (see Figure 6B). This detrimental effect of input synchrony on the postsynaptic Žring, at high levels of (net) excitation, has already been reported (Murthy & Fetz, 1994), albeit with a different input ensemble (see above).

94

A. Kuhn, A. Aertsen, and S. Rotter

Stroeve and Gielen (2001) used an input population identical to the SIP model and reported that the pairwise correlation of excitation affects the postsynaptic Žring rate only if the inhibitory Žring rate is sufŽciently high. It must be pointed out that this statement follows from their particular choice of input parameters. They used a small input size (N D 120), which was effectively smaller compared to the input used here because the membrane potential of their model neuron was not in a depolarized state. As shown in this study, the effects of input correlation depend strongly on the size of the population. More important, their input parameters were such that a compensation effect, similar to what was described for Figure 6B (b D 0.2, c < 0.2), was in effect. This is clearly demonstrated by the irregularity of inter-spike intervals (Stroeve & Gielen, 2001) that increased with the input correlation. This shows that the neuron’s Žring was increasingly caused by large, randomly occurring spike clusters. As such input clusters induced membrane depolarizations beyond the Žring threshold, the corresponding “loss of spikes” had to be compensated for by the increased efŽcacy of synchrony. Hence, as demonstrated in Figure 6B for low input correlation, it is appropriate to consider the net excitation level as a powerful means of modulating the effects of input correlation. This possibility was mentioned earlier by Salinas and Sejnowski (2000). Finally, we would like to stress that both input and output investigated here were stationary and that an important and obvious step toward more realistic conditions would be to study the impact of nonstationary input ensemble statistics. Appendix A

In this appendix, we derive the cross-correlation function and count correlation coefŽcient for pairs of spike trains generated by either the SIP or the MIP model. Consider a homogeneous Poisson point process x ( t) with rate E[x(t)] D l. The autocorrelation Rxx ( D t) of this process is (Papoulis, 1991) Rxx ( D t) D E[x(t)x(t C D t) ] D ld ( D t) C l2 . For two independent Poisson point processes x (t) and y ( t) with rates l and m , respectively, the cross-correlation function Rxy ( D t) is Rxy ( D t) D E[x(t) y ( t C D t) ] D lm . We now consider dependent processes. Let v ( t) and w (t) be deŽned as v ( t) D x ( t) C z ( t) w ( t) D y ( t) C z (t) , where x ( t) , y ( t) , and z ( t) are independent homogeneous Poisson processes with rate l, m , and º, respectively. The cross-correlation function of v ( t) and

Higher-Order Statistics of Input Ensembles

95

w (t) is Rvw (D t) D Rxy ( D t) C Rxz ( D t) C Ryz (D t) C Rzz ( D t) 2 D ºd ( D t) C º C lº C mº C lm

D ºd ( D t) C (º C l) (º C m ) .

The count correlation coefŽcient for the two point processes v (t) and w (t) with respect to a Žxed observation interval of duration T is deŽned in terms of the stochastic variables V and W giving the respective numbers of points encountered during the observation interval: cVW D p

cov(V, W ) . var ( V ) var (W )

For arbitrary point processes, the count correlation coefŽcient depends in general on the observation interval. For the two dependent Poisson processes v ( t) and w (t) considered above, we have in particular var(V) D var (X ) C var (Z ) var(W) D var (Y ) C var(Z) cov(V, W ) D var (Z ) , with X, Y, and Z the stochastic variables giving the respective numbers of points in the processes x (t) , y ( t) , and z ( t) , in the interval considered. For an observation interval of duration T, we have var(X) D lT,

var(Y) D m T,

var (Z ) D ºT,

and thus obtain cVW D p

º . (º C l) (º C m )

Note that this is independent of the observation length T. Thus, we may also simply use the term “pairwise correlation coefŽcient” for “count correlation coefŽcient.” Normalizing the cross-correlation function of the two processes by the geometric mean of their respective rates, we obtain an expression of the cross-correlation as a function of the count correlation coefŽcient, p

p Rvw ( D t) D cVWd ( D t) C (º C l) (º C m ) . (º C l) (º C m )

The count correlation coefŽcient thus corresponds to the net integral over the central peak in the (normalized) cross-correlation function. The general case represented by v ( t) and w ( t) is now readily applied to our two models of spike train ensembles. For the SIP model, we have º D a

96

A. Kuhn, A. Aertsen, and S. Rotter

and l D m D b. Hence, the cross-correlation function and count correlation coefŽcient between any two spike trains are R (D t) D ad ( D t) C (a C b ) 2 , a . cD aC b For the MIP model, it holds that the rate of coincident spikes in any pair of spike trains is º D c 2 2 and l D m D c 2 ¡c 2 2 . The cross-correlation function and count correlation coefŽcient thus are R (D t) D c 2 2d ( D t) C (c 2 ) 2 , cD2 . Appendix B

In this appendix, we develop an analytical approximation of the response of the FR neuron to the two models of spike train ensembles (SIP and MIP). The FR neuron was described above (see equations 3.1 through 3.3). The corresponding synaptic input current is described by a stationary stochastic process, and we are interested in the stationary output rate of the neuron. The knowledge of the stationary membrane potential distribution is sufŽcient to derive the mean output rate for this particular neuron model (see equation 3.5). If we use linear synapses and Poisson-driven synaptic events, the membrane potential is a shot noise. For pulse-like synaptic currents, the membrane realizes a Žrst-order low-pass Žlter with time constant t D RC (see equation 3.4). Gilbert and Pollak (1960) derived an integral equation for the shot noise distribution and solved it for several particular choices of the Žlter. For the Žlter h ( t) deŽned in equation 3.4, with A D 1 and t D 1, the shot noise S (t ) with underlying rate l has an amplitude distribution with density P (S ) given by P (S ) D dSl¡1

for 0 · S < 1.

For larger values of S, P (S ) is found by recursion from the integral equation, " # Z P (S ) D Sl¡1 d ¡ l The constant d is given by dD

e ¡lg , C (l)

S

1

P (x ¡ 1) x¡l dx .

Higher-Order Statistics of Input Ensembles

97

R1 R1 where g D ¡ 0 e¡t log t dt is Euler ’s constant and C ( z ) D 0 tz¡1 e ¡t dt is the gamma function. Thus, the analytic expression of P ( S ) changes at integer values of S. For t arbitrary, the density is easily shown to be ( P? ( S ) D

DSlt ¡1 DSlt ¡1 w (lt , S )

where

"³

w (lt , S) D 1 ¡

1 S ¡1

for 0 · S < 1

(B.1)

for 1 · S < 2

´¡lt

# F (lt , lt, 1 C ltI 1 ¡ S )

and F is the hypergeometric function F (a, b, cI z ) D

1 X ( a ) k ( b) k z k ( c) k k! kD 0

with ( a) k :D a ( a C 1) ¢ ¢ ¢ ( a C k ¡ 1). The constant D is DD

e ¡lt g . C (lt )

If lt D 1, w (lt , S ) D 1 ¡ log S, and P? (S ) simpliŽes to ( for 0 · S < 1 e¡g P? (S ) D ¡g for 1 · S < 2, e (1 ¡ log S ) which is the expression presented in Gilbert and Pollak (1960). We now formulate the approximation of the membrane potential density, P ( U ) , for correlated excitatory and independent inhibitory input. The neuron’s impulse response h ( t) is given explicitly in equation 3.4. We Žx the PSP amplitude at A D 1. Let us Žrst consider the membrane response to the SIP model. Let Ne (Ni ) be the number of excitatory (inhibitory) input spike trains and re (ri ) the rate of each individual excitatory (inhibitory) spike train. The excitation can be decomposed into two independent Poisson processes (see section 2.1). Let UeN ( t) be the Žltered Ne th-order interaction process and Ue1 (t) the Žltered Žrst-order process. Ui ( t) is the Žltered inhibitory input spike train with total rate Ni ri . Thus, if tekN , tek1 , and tik denote the times of events in the corresponding pulse trains, we have UeN (t) D Ue1 (t) D Ui (t) D

X k

X k

X k

Ne h ( t ¡ tekN ) h ( t ¡ tek1 ) ¡h(t ¡ tik ) .

98

A. Kuhn, A. Aertsen, and S. Rotter

Note that we chose the same absolute amplitude for the excitatory and inhibitory synaptic currents. As we are dealing with a linear system, we have U ( t) D UeN ( t) C Ue1 (t ) C Ui ( t) , and thus P ( U ) D ( PeN ¤ Pe1 ¤ Pi ) ( U ) , the convolution product of the densities of the three independent components. We now useR Campbell’s theorem to obtain the mean and variance of each R density. As h ( t) dt D t and h ( t) 2 dt D t / 2, we have E (UeN ) D Ne at D Ne re ct, E (Ue1 ) D Ne bt D Ne re (1¡ c)t , E (Ui ) D ¡Ni ri t,

var (UeN ) D Ne2 at / 2 D Ne2 re ct / 2, var (Ue1 ) D Ne bt / 2 D Ne re (1¡ c) t / 2, var (Ui ) D Ni ri t / 2.

For a large range of parameters, in particular for large excitatory populations, we have var (UeN ) À var ( Ue1 ) C var (Ui ) , and we can approximate the convolution (Pe1 ¤ Pi ) ( U ) by a Dirac delta function located at E(Ue1 C Ui ) . We obtain P (U ) ¼ PeN ( U ¡ Ne re (1 ¡ c) t C Ni ri t ) . The density of the membrane potential response of the FR neuron to the MIP model is approximated in a similar manner. Let Ue ( t) be the Žltered excitatory input. The interacting point processes in the MIP model can be decomposed into Ne C 1 independent interaction processes (see section 2.2). Let Uen ( t) be the Žltered nth-order interaction process, with rate B(nI c, Ne ) re / c. The membrane potential of the FR neuron is then given by U ( t ) D Ue ( t ) C Ui ( t ) D

N X nD 0

Uen ( t) C Ui (t ).

The variance of Ue (t ) is var (Ue ) D Ne re (1 ¡ c C cNe )t / 2. If the relation var(Ue ) À var (Ui ) holds true, we can approximate P i (U ) by a Dirac delta function located at E(Ui ) and write P (U ) ¼ Pe (U C Ni ri t ) .

P In addition, we approximate Ue ( t) by UenN ( t) D Ne c k h (t ¡ teknN ) , where teknN are the times of pulses in the generating process (wg ( t) ; see section 2.2). Effectively, we let the order of interaction be deterministic and approximate the N interaction processes by a single process with amplitude E(n) D Ne c and with the same rate re / c as the generating process. We then get P (U ) ¼ PenN ( U C Ni ri t ). The membrane potential density for both input models can thus be approximated by the expression of the shot noise density (see equation B.1), using a linear variable transformation U D kS ¡ l. We Žnally obtain ³ ´ 1 UCl P ( U ) ¼ P? , k k

Higher-Order Statistics of Input Ensembles

99

with an appropriate choice of k, l, and of the parameters of the density function P? (see equation B.1). The mean output rate ro of the FR neuron is then obtained by integrating over the tail of the potential distribution (see equation 3.5) and is expressed in a piecewise manner too. The Žrst two terms are 8 > > >1 ¡ D > > > > > > > > > > > 1¡D > > > < ro ¼

> > > > > > > > > > > > > > > > > :

´ h C l lt for ¡l · h < k ¡ l k ³ ´ ´ ³ ´¡lt Zh ³ 1 h C l lt U C l lt ¡1 k ¡ > k k U C l¡k : lt k¡l ³ ´ UCl ¢F lt, lt, 1 C lt I 1 ¡ k 9 > dU = ¢ for k ¡ l · h < 2k ¡ l. k > ; 1 lt 8 > <

³

(B.2)

It remains to Žx values for l, k, and l, in the expressions above, with the parameters from the input models. For the SIP model, l D re c, k D Ne , and l D ¡Ne re (1 ¡ c) t C Ni ri t . For the MIP model, l D re / c, k D Ne c, and l D Ni ri t . Note that if N D Ne D Ni and r D re D ri , the expression (h C l) / k simpliŽes to h / N C rct for the SIP model and to h / (Nc) C rt / c for the MIP model. Hence, the threshold h and the size of the input population N appear only as the ratio h / N. Acknowledgments. This work was supported in part by the Human Frontier Science Program, the Deutsche Forschungsgemeinschaft, and the German-Israeli Foundation for ScientiŽc Research and Development. References Aertsen, A., & Gerstein, G. L. (1985). Evaluation of neuronal connectivity: Sensitivity of cross-correlation. Brain Research, 340, 341–354. Azouz, R., & Gray, C. M. (1999). Cellular mechanisms contributing to response variability of cortical neurons in vivo. J. Neurosci., 19(6), 2209–2223. Baker, S. N., Spinks, R., Jackson, A., & Lemon, R. N. (2001). Synchronization in monkey motor cortex during a precision grip task. I. Task-dependent modulation in single-unit synchrony. J. Neurophysiol., 85(2), 869–885. ¨ Koch, C., & Usher, M. (1994). The effect of synchronized inputs Bernander, O., at the single neuron level. Neural. Comput., 6, 622–641. Bohte, S. M., Spekreijse, H., & Roelfsema, P. R. (2000). The effects of pair-wise and higher-order correlations on the Žring rate of a postsynaptic neuron. Neural. Comput., 12, 153–179.

100

A. Kuhn, A. Aertsen, and S. Rotter

Borg-Graham, L. J., Monier, C., & FrÂegnac, Y. (1998). Visual input evokes transient and strong shunting inhibition in visual cortical neurons. Nature, 393, 369–373. Braitenberg, V., & Schuz, ¨ A. (1998). Cortex: Statistics and geometry of neuronal connectivity (2nd ed.). Berlin: Springer-Verlag. Cox, D. R., & Isham, V. (1980). Point processes. London: Chapman & Hall. Das, A., & Gilbert, C. D. (1999).Topography of contextual modulations mediated by short-range interactions in primary visual cortex. Nature, 399(6737), 655– 661. DeCharms, R. C., & Merzenich, M. M. (1996). Primary cortical representation of sounds by the coordination of action-potential timing. Nature, 381, 610–613. Deppisch, J., Bauer, H.-U., Schillen, T., Konig, ¨ P., Pawelzik, K., & Geisel, T. (1993). Alternating oscillatory and stochastic states in a network of spiking neurons. Network, 4, 243–257. Destexhe, A., & ParÂe, D. (1999). Impact of network activity on the integrative properties of neocortical pyramidal neurons in vivo. J. Neurophysiol., 81(4), 1531–1547. Eggermont, J. J., & Smith, G. M. (1995). Rate covariance dominates spontaneous cortical unit-pair correlograms. Neuroreport, 6(16), 2125–2128. Feng, J., & Brown, D. (2000). Impact of correlated inputs on the output of the integrate-and-Žre model. Neural Comput., 12(3), 671–692. Feng, J., & Zhang, P. (2001). Behavior of integrate-and-Žre and Hodgkin-Huxley models with correlated inputs. Phys. Rev. E, 63, 051902:1–11. Gerstein, G. L., & Aertsen, A. M. H. J. (1985). Representation of cooperative Žring activity among simultaneously recorded neurons. J. Neurophysiol., 54(6), 1513–1528. Gilbert, E. N., & Pollak, H. O. (1960). Amplitude distribution of shot noise. Bell Syst. Tech. J., 39, 333–350. Gutig, ¨ R., Aertsen, A., & Rotter, S. (2002). Analysis of higher-order neuronal interactions based on conditional inference. Manuscript submitted for publication. Koch, C. (1999). Biophysics of computation: Information processing in single neurons. New York: Oxford University Press. Kuhn, A., Rotter, S., & Aertsen, A. (2002). Correlated input spike trains and their effects on the leaky integrate-and-Žre neuron. Neurocomputing, 44–46, 121–126. LÂeger, J.-F., Stern, E. A., Aertsen, A., & Heck, D. (2002). Synaptic integration in the cortex shaped by network activity. Manuscript submitted for publication. Martignon, L., Deco, G., Laskey, K., Diamond, M., Freiwald, W., & Vaadia, E. (2000). Neural coding: Higher-order temporal patterns in the neurostatistics of cell assemblies. Neural. Comput., 12, 2621–2653. Martignon, L., von Hasseln, H., Grun, ¨ S., Aertsen, A., & Palm, G. (1995). Detecting higher-order interactions among the spiking events in a group of neurons. Biol. Cybern., 73, 69–81. Matsumura, M., Chen, D., Sawaguchi, T., Kubota, K., & Fetz, E. E. (1996). Synaptic interactions between primate precentral cortex neurons revealed by spiketriggered averaging of intracellular membrane potentials in vivo. J. Neurosci., 16(23), 7757–7767.

Higher-Order Statistics of Input Ensembles

101

McCormick, D. A., Connors, B. W., Lighthall, J. W., & Prince, D. A. (1985). Comparative electrophysiology of pyramidal and sparsely spiny neurons of the neocortex. J. Neurophysiol., 54, 782–806. Murthy, V. N., & Fetz, E. E. (1994). Effects of input synchrony on the Žring rate of a three-conductance cortical neuron model. Neural Comput., 6, 1111–1126. Nelson, J. I., Salin, P. A., Munk, M. H. J., Arzi, M., & Bullier, J. (1992). Spatial and temporal coherence in cortico-cortical connections: A cross-correlation study in areas 17 and 18 in the cat. Visual Neurosci., 9, 21–37. Nicolelis, M. A. L. (Ed.). (1998).Methodsfor neural ensemble recordings. Boca Raton, FL: CRC Press. Papoulis, A. (1991). Probability, random variables, and stochastic processes (3rd ed.). New York: McGraw-Hill. ParÂe, D., Shink, E., Gaudreau, H., Destexhe, A., & Lang, E. (1998). Impact of spontaneous synaptic activity on the resting properties of cat neocortical pyramidal neurons in vivo. J. Neurophysiol., 79, 1450–1460. Rotter, S., & Diesmann, M. (1999). Exact digital simulation of time-invariant linear systems with applications to neuronal modeling. Biol. Cybern., 81, 381– 402. Salinas, E., & Sejnowski, T. J. (2000). Impact of correlated synaptic input on output Žring rate and variability in simple neuronal models. J. Neurosci., 20(16), 6193–6209. Salinas, E., & Sejnowski, T. J. (2001). Correlated neuronal activity and the ow of neural information. Nat. Rev. Neurosci., 2(8), 539–550. Shadlen, M. N., & Newsome, W. T. (1998). The variable discharge of cortical neurons: Implications for connectivity, computation, and information coding. J. Neurosci., 18(10), 3870–3896. Snyder, D. L., & Miller, M. I. (1991). Random point processes in time and space (2nd ed.). New York: Springer-Verlag. Steinmetz, P., Roy, A., Fitzgerald, P., Hsiao, S., Johnson, K., & Niebur, E. (2000). Attention modulates synchronized neuronal Žring in primate somatosensory cortex. Nature, 404(6774), 187–190. Stroeve, S., & Gielen, S. (2001). Correlation between uncoupled conductancebased integrate-and-Žre neurons due to common and synchronous presynaptic Žring. Neural Comput., 13(9), 2005–2029. Svirskis, G., & Rinzel, J. (2000). Inuence of temporal correlations of synaptic input on the rate and variability of Žring in neurons. Biophys. J., 79, 629–637. Tiesinga, P. H. E., JosÂe, J. V., & Sejnowski, T. J. (2000). Comparison of currentdriven and conductance-driven neocortical model neurons with HodgkinHuxley voltage-gated channels. Phys. Rev. E, 62(6), 8413–8419. Troyer, T. W., & Miller, K. D. (1997).Physiological gain leads to high ISI variability in a simple model of a cortical regular spiking cell. Neural Comput., 9, 971–983. Tuckwell, H. C. (1988). Introduction to theoreticalneurobiology (Vol. 2). Cambridge: Cambridge University Press. van Vreeswijk, C., & Sompolinsky, H. (1996). Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science, 274, 1724–1726. Received May 10, 2002; accepted June 4, 2002.

LETTER

Communicated by Ad Aertsen

Duality of Rate Coding and Temporal Coding in Multilayered Feedforward Networks Naoki Masuda [email protected] Department of Mathematical Engineering and Information Physics, Graduate School of Engineering, University of Tokyo, Tokyo, Japan Kazuyuki Aihara [email protected] Department of Complexity Science and Engineering, Graduate School of Frontier Sciences, University of Tokyo, Tokyo, Japan A functional role for precise spike timing has been proposed as an alternative hypothesis to rate coding. We show in this article that both the synchronous Žring code and the population rate code can be used dually in a common framework of a single neural network model. Furthermore, these two coding mechanisms are bridged continuously by several modulatable model parameters, including shared connectivity, feedback strength, membrane leak rate, and neuron heterogeneity. The rates of change of these parameters are closely related to the response time and the timescale of learning. 1 Introduction

Experimental data on information coding by Žring rates have been widely reported, but it is still controversial whether the brain really uses such coding for information processing. Calculating Žring rates on the basis of spike trains, if it requires time averaging, may consume more time than is biologically acceptable to account for brain functions (Thorpe, Fize, & Marlot, 1996). Accordingly, functional roles of precisely timed spiking and synchronous Žring have been explored. Our research is motivated by experimental results showing that neurons with overlapping receptive Želds Žre synchronously when activated by common preferred stimuli in the cat primary visual cortex (Eckhorn et al., 1988; Gray, Konig, ¨ Engel, & Singer, 1989). Synchronous Žring is also experimentally observed in the cat superior colliculus (Brecht, Singer, & Engel, 1999), the monkey primary motor cortex (Hatsopoulos, Ojakangas, Paninski, & Donoghue, 1998), and the monkey somatosensory cortex (Steinmetz et al., 2000). Furthermore, the Žring of a single neuron is also precise to the millisecond order in, for example, rat neocortical slices (Mainen & Sejnowski, 1995) and b-lobe neurons (MacLeod, B¨acker, & LauNeural Computation 15, 103–125 (2003)

° c 2002 Massachusetts Institute of Technology

104

N. Masuda and K. Aihara

rent, 1998). Synchronous Žring has been examined by various mathematical models with pulse couplings to clarify physiological mechanisms (Domany, van Hemmen, & Schulten, 1994; Bottani, 1995; Bressloff & Coombes, 2000; Guardiola, DÂõ az-Guilera, Llas, & PÂerez, 2000; Feng, Brown, & Li, 2000; Masuda & Aihara, 2001). It has been shown that network behavior is sensitive to model details and that stable asynchronous and clustered states may also appear. Temporal coding models, alternatives to the rate coding, have been presented on the basis of precisely timed spiking, coincidence, and synchronization. Among them are the multilayered synŽre chain model (Abeles, 1991; Domany et al., 1994; Diesmann, Gewaltig, & Aertsen, 1999), the detectors of periodic Žring patterns (Fujii, Ito, Aihara, Ichinose, & Tsukada, 1996), a learning algorithm model for coincidence detection by updating synaptic strengths (Gerstner, Kempter, van Hemmen, & Wagner, 1996), and the chaotic attractor reconstructor model with intersynchronization intervals (Tokuda & Aihara, 2000; Masuda & Aihara, 2002b; Aihara & Tokuda, 2002). These spatiotemporal computation models can process information at higher speeds and require less energy than is required by the rate coding (Abeles, 1991). Nevertheless, the functional roles of precisely timed neural activities are still unclear. On the other hand, short-term Žring rates can be estimated reliably by a population average of 100 or fewer neurons without resorting to temporal averaging or measurement of precisely timed Žring (Shadlen & Newsome, 1998; Mar, Chow, Gerstner, Adams, & Collins, 1999; van Rossum, Turrigiano, & Nelson, 2002). The population rate coding with such spatial averaging produces a biologically plausible coding scheme with Žring rates. Balanced excitation and inhibition models (Shadlen & Newsome, 1998) can also reproduce the experimentally observed high variability of Žring time, which was once thought to be generated more easily with temporal coding schemes with coincidence detectors (Abeles, 1991; Softky & Koch, 1993). Although these two coding mechanisms are usually considered alternatives, they may work in a complementary rather than exclusive way (Abeles, 1982, 1991; Tsodyks & Markram, 1997). It is shown that both the temporal coding and the population rate coding can work in a single neural network and that they are bridged by changing the intensity of additive gaussian noise (Masuda & Aihara, 2002b; van Rossum et al., 2002). In the feedforward network model depicted in Figure 1, each sensory neuron integrates the external stimulus S ( t) to encode its different aspect. Each cortical neuron gathers small contributions from many sensory neurons to encode signal information (Shadlen & Newsome, 1998; Masuda & Aihara, 2002a). When sufŽciently weak noise is applied to the cortical neurons, they tend to synchronize and encode a signal crudely but robustly in intersynchronization intervals (synchronous coding), but moderately large noise desynchronizes them (Tokuda & Aihara, 2000; Masuda & Aihara, 2002b; Aihara & Tokuda, 2002). In the latter case, each cortical neuron represents a different feature of

Rate Coding and Temporal Coding

105

IF x1

LIF v1

x2

v2

synchronous coder population rate coder

S(t) xn1 sensory layer

vn2 cortical layer

Figure 1: Architecture of the feedforward neural network model. The sensory layer and the cortical layer consist of n1 integrate-and-Žre (IF) neurons and n2 leaky integrate-and-Žre (LIF) neurons with leak rate c , respectively. Each sensory neuron integrates the external stimulus S ( t ) and Žres when the membrane potential reaches the threshold. A cortical neuron receives spatiotemporal incident spikes with amplitude 2 from n01 sensory neurons where n01 · n1 . After Žring, it sends a feedback spike with amplitude 2 and synaptic delay t to all the other cortical neurons. The spatiotemporal spike trains from all the cortical neurons are used to evaluate Žring synchrony and the instantaneous Žring rate.

S ( t) , and Žne approximation to the temporal waveform of S ( t) is obtained by the population rate coding deŽned by the instantaneous ensemble average of spike counts (Shadlen, Britten, Newsome, & Movshon, 1996; Shadlen & Newsome, 1998; Mar et al., 1999; van Rossum et al., 2002). Our results suggest that the two coding schemes can work dually, depending on physiological conditions and anatomical structure. Noise intensity is not the only physical cause of switch between the population rate coding and the synchronous coding. For example, a theoretical study suggests that transmembrane leak can also change rapidly with background activities (Bernander, Douglas, Martin, & Koch, 1991). Figure 2 suggests that the rate code and synchronous code modes can coexist when heterogeneity in Žring thresholds is modulated. We continuously bridge these two coding schemes in this article to illuminate their complementary roles. We identify dominant coding schemes determined by the correlation of inputs to cortical neurons, the feedback coupling strength, the membrane leak rate, and inhomogeneity in constituent neurons. We also consider the

106

N. Masuda and K. Aihara B 1

1

0.75

0.75 s _ 2 (i)

s _ 2 (i)

A

0.5 0.25 0

0.5 0.25

0

200

400

600

0

800

0

200

time (ms) C

600

800

D 30

30

20

20

n eu ron

n eu ron

400 time (ms)

10 0

0

200

400 time (ms)

600

800

10 0

0

200

400

600

800

time (ms)

Figure 2: The histograms (A, B) and rastergrams (C, D) in the regime of the synchronous coding (A, C) when D H D 0 and the population rate coding (B, D) when D H D 0.25, with the deŽnition of D H given in section 3.4. The other parameters are equal to those used for Figure 6C.

combination of these effects in multilayered neural networks where both feedforward and feedback connections exist. We describe the model neural network and the methods that we use to evaluate the functional coding schemes in section 2. In section 3, we examine how the functional coding schemes switch when various model parameters are modulated. Section 4 is devoted to the uniŽed effects of simultaneous parameter changes. In section 5, we discuss the relations between our results and dual coding schemes, multilayered feedforward networks, synŽre chains, and the timescale of learning. 2 Model 2.1 Neural Network. The neural network model used in this work is depicted in Figure 1. The network consists of two layers: the sensory layer with n1 neurons and the cortical layer with n2 neurons. Each sensory neuron is an integrate-and-Žre (IF) neuron (Tuckwell, 1988; Koch, 1999) that receives a common continuous external input S ( t) . The membrane potential of the ith sensory neuron is obtained by the perfect integration of suprathreshold S (t) . The ith sensory neuron Žres when the membrane potential sum reaches 1, af-

Rate Coding and Temporal Coding

107

ter which the potential is instantaneously reset to a resting potential of 0 and the neuron restarts integration. The initial phases of the neurons are chosen randomly from a uniform distribution on [0, 1]. This is a key assumption required for the network to operate as an efŽcient population rate coder. The correlation between the signal and the population rate would be reduced if the phases were not random (Masuda & Aihara, 2002a; van Rossum et al., 2002). Dynamical noise could disperse the phases in real situations to make them uniformly distributed even in the absence of randomness of initial phases (Masuda & Aihara, 2002b). We adopt this neuron model because of its simplicity and the theoretical evidence that interspike intervals of the IF neuron can encode the deterministic information on S (t) (Sauer, 1994). The results below can be extended to more realistic sensory neurons with a leak (Masuda & Aihara, 2002a). When a sensory neuron Žres, it sends a spike to some cortical neurons. Every cortical neuron receives instantaneous incident spike trains from n01 · n1 sensory neurons. A pair of cortical neurons generally shares in average n012 / n1 incident spatiotemporal spike trains from the sensory neurons connecting to both neurons. Therefore, we deŽne the shared connectivity r ´ ( n012 / n1 ) / n01 D n01 / n1 . Generally, the network dynamics of coupled leaky integrate-and-Žre (LIF) neurons are more sensitive to network conŽguration (Knight, 1972; Bressloff & Coombes, 2000) in contrast to the dynamics of coupled IF neurons. We treat the cortical neurons as LIF neurons without an external bias (Tuckwell, 1988; Masuda & Aihara, 2002a, 2002b) since dynamical transitions of the network states are our primary concern. The ith cortical neuron (1 · i · n2 ) has a membrane potential vi ( t) , a threshold for Žring (vi (t) D H i , E[H i ] D H 0 D 1, where E means the expectation), and a resting potential (vi ( t) D 0). After Žring, this cortical neuron emits an instantaneous feedback spike with amplitude 2 and synaptic delay t to the other cortical neurons. The dynamics of the ith cortical neuron are described by n2 XX X X dvi 0 D 2 d ( t ¡ Ti0 , j ) C 2 d ( t ¡ Ti0 , j ¡ t ) ¡ c vi ( t ) , dt 6 i,i0 D 1 j i0 D i0 2S j i

(1 · i · n2 ) ,

(2.1)

where c > 0 is the membrane leak rate and d is the delta function. Ti0 , j is the time when the jth spike generated by the i0 th sensory neuron arrives at the ith cortical neuron. Here, the delay between the sensory and cortical layers is uniformly zero or equivalently uniformly constant. Ti00 , j is the jth Žring time of the i0 th cortical neuron, and 2 is the amplitude of a feedforward spike. The ith cortical neuron receives incident spikes only from the i0 th sensory neuron with i0 2 Si where |Si | (D n01 ) is the number of randomly selected sensory neurons. Although we do not consider noise sources such as ongoing activity (Arieli, Sterkin, Grinvald, & Aertsen, 1996; Azouz & Gray, 1999) composed of spikes from neurons outside the network, they

108

N. Masuda and K. Aihara

would also modulate the network behavior and preferred coding schemes in a similar way to dynamical gaussian white noise (Masuda & Aihara, 2002b; van Rossum et al., 2002). The delay t D 2.6 ms (Abeles, 1991; Diesmann et al., 1999), and the Žring rate is kept at 20 Hz (de Oliveira, Thiele, & Hoffmann, 1997; Zohary, Shadlen, & Newsome, 1994). The membrane leak rate should be around 0.05 to 0.1 ms¡1 (Abeles, 1982; Koch, 1999), but we use larger values to emphasize the duality of the two complementary coding schemes. We analyze the network dynamics in the high-input regime by setting n01 D 240 and n2 D 30, where 100 to 200 incident spikes are necessary for a cortical neuron to Žre (Gerstner et al., 1996; Shadlen & Newsome, 1998; Diesmann et al., 1999). The spike trains emitted by the cortical neurons are observed by the virtual synchronous coder and the virtual population rate coder, as shown in Figure 1. We measure the degree of synchrony by calculating syn, which is deŽned as the ratio of the number of synchronous Žring events to the average number of single neuron spikes. A synchronous Žring event is deŽned as spike arrivals from more than n2 / 2 cortical neurons within tw D 1.5 ms (Softky & Koch, 1993; Hatsopoulos et al., 1998; Aihara & Tokuda, 2002). Full synchrony gives syn D 1, whereas fully asynchronous Žring results in syn D 0. The population rate of synchronous Žring usually represents the input signal more robustly but less accurately than that of asynchronous Žring. The accuracy of the synchronous coding is equal to that of coding by a single neuron (Masuda & Aihara, 2002b; van Rossum et al., 2002), although the noise torelance is better (Aihara & Tokuda, 2002). The performance of population rate coding is evaluated by the correlation coefŽcient corr between the input signal S ( t) and the short-time population Žring rate of the cortical neurons, that is, the number of spikes from all the cortical neurons in each bin of width w D 3.8 ms (Shadlen et al., 1996). Given Z 1 iw ( ) s1 i D S (t ) dt w ( i¡1) w and s2 ( i) D the number of spikes observed for t 2 [(i ¡ 1)w, iw], the correlation coefŽcient between the two time series fs1 ( i)g and fs2 ( i)g deŽnes corr. 2.2 External Inputs. The chaotic input S ( t) was generated from the Lorenz system, the dynamics of which is represented by the following equations: 8 dx > < dt D a (10y ¡ 10x) , dy (2.2) D a (28x ¡ y ¡ xz ) , dt > : dz 8 D a (¡ 3 z C xy ). dt

Rate Coding and Temporal Coding

109

The continuous input signal is deŽned as S ( t) D 2.25£10¡2 C 7.00£10¡4 x and models nonrandom dynamical rules with complexity contained in complex external stimuli such as visual scenes and sounds. The change rate a is Žxed at 0.03 so that the characteristic timescale of S ( t) is smaller than the interspike interval. As a result, the synchronous coding cannot capture rapid signal change, and the Žring rate can encode only roughly the temporal waveform of S ( t). This fact is revealed by low values of corr for the synchronous Žring. Population Žring rates with asynchronous Žring are more strongly correlated with S ( t). Our results qualitatively remain the same when we apply stochastic inputs, which are often used to study information processing by neural networks (see section 3.4). 3 Selection of Effective Coding Modes

We investigate which coding mechanism is effective when values of model parameters are changed. The parameters examined here have different functional relevance and affect the choice of the optimal coding in different ways. 3.1 Shared Connectivity. Two neurons often share incident connections, especially when the neurons are geometrically or functionally close to each other (Aertsen, Gerstein, Habib, & Palm, 1989; Zohary et al., 1994; Shadlen et al., 1996). They usually share only a part of inputs, and the feedforward and also feedback coupling structure is far from all-to-all. Incomplete coupling introduces quenched noise that disperses the membrane potentials of two neurons starting from the same initial level. The shared connectivity r D n01 / n1 , that is, the ratio of the average number of the shared connections to that of the total connections (n01 ), is biologically between 0.1 and 0.3 (Zohary et al., 1994; Shadlen et al., 1996; Shadlen & Newsome, 1998; Hatsopoulos et al., 1998; Diesmann et al., 1999). Shared connections, or common incident spikes, are likely to drive the neural population to synchrony, as do common feedback spikes delivered to many neurons (Bottani, 1995). Synchronous Žring can occur theoretically even without feedback couplings (Knight, 1972; Feng, Brown, & Li, 2000), though biologically it may be produced by cooperation of feedback interactions and correlated inputs (Eckhorn et al., 1988; Hatsopoulos et al., 1998; Brecht et al., 1999). We examine how r inuences network behavior and the efŽciency of the two codings. As shown in Figure 3, we change the number of shared connections between the sensory layer and the cortical layer by changing n1 and keeping all the other parameters constant. Since the simulation results depend on initial conditions, syn and corr shown in this Žgure, as well as in the following Žgures, are the averages of 50 trials. Figure 3 demonstrates that the population rate coding performs better for smaller r, whereas the neurons are more synchronous for larger r. This result holds regardless of the feedback strength within the cortical layer (see Figure 4).

110

N. Masuda and K. Aihara

Figure 3: Change in the degree of synchrony measured by syn and performance of the population rate coder measured by corr when the shared connectivity r varies. We set n1 D n01 / r, n2 D 30, n01 D 240, 2 D 0.007, c D 40 ms ¡1 , and 2 D 0.002.

A

B 1 0.4 0.3

0.6 0.4

r=0.50 r=0.25 r=0.03

0.2 0

0

0.0025 0.005 0.0075 0.01 coupling strength

c o rr

syn

0.8

0.2 r=0.50 r=0.25 r=0.03

0.1 0

0

0.0025 0.005 0.0075

0.01

coupling strength

Figure 4: Change in syn (A) and corr (B) when the feedback strength 2 varies. n2 D 30, n01 D 240, 2 D 0.007,c D 40 ms ¡1 , and 2 D 0.002. The lines in each panel correspond to the three examined cases, with n1 D 480, n1 D 960, and n1 D 8000, corresponding to r D 0.50, r D 0.25, and r D 0.03, respectively.

Rate Coding and Temporal Coding

111

Correlated inputs with large r produce relatively invariant responses of the cortical neurons and interfere with efŽcient signal averaging, which results in biases and a loss of reliability in signal estimation. The information transmission capacity from the sensory layer to the cortical layer is decreased in this situation (Stein, 1967; Shadlen & Newsome, 1998). However, robust temporal coding with synchronous Žring is possible based on coincidence detection. When r is small, cortical neurons are likely to desynchronize to participate in reliable rate coding. Effective pooling of independent information from asynchronous cortical neurons leads to better estimation of S (t ) (Shadlen et al., 1996; Shadlen & Newsome, 1998). Changing r continuously inuences the efŽciency of the synchronous coding and the population rate coding in opposite directions. 3.2 Feedback Strength. We examine the inuence of feedback coupling 2 among cortical neurons. Feedback interactions generally compel networks

toward characteristic states such as full synchrony, clustered, and phaselocked states (Domany et al., 1994; Bressloff & Coombes, 2000). Figure 4 shows that the cortical layer is a reliable rate coder when 2 is small over a wide range of shared connectivity, whereas the cortical neurons synchronize when 2 is large. 3.3 Membrane Leak Rate. The membrane leak rate c of the cortical neurons is another modulatable parameter. Simulation studies suggest that it can vary rapidly in response to the background activity (Bernander et al., 1991). Networks of LIF neurons with large c synchronize more readily than LIF neurons with small c or nonleaky (c D 0) neurons (Knight, 1972; Bressloff & Coombes, 2000; Feng et al., 2000; Masuda & Aihara, 2001). Therefore, we expect that the preferred coding mode would change gradually from the population rate coding to the synchronous coding as c increases. Changes in Žring rates cause signiŽcant biases in the performance measures syn and corr when a in equation 2.2 is Žxed. Consequently, to make the Žring rates Žxed for the sake of fair comparisons, we adjust 2 for each c in the numerical analyses. The simulation result shown in Figure 5, however, supports our conjecture only for c 2 [0.00, 0.02]. EfŽcient population rate coding as well as a certain level of synchrony is achieved for larger c . The simultaneous large values of syn and corr for large c are contrasted with the opposed effects on these quantities of varying r or 2 . The relevant parameter range includes the biologically plausible leak rate of c D 0.05 ¡ 0.1 ms ¡1 (Abeles, 1982; Diesmann et al., 1999; Koch, 1999). Although large c could cause signal estimation to deteriorate because of the weighted integration of the signal performed by the LIF neuron (Knight, 1972; Masuda & Aihara, 2002a), its contribution to corr is actually marginal. However, c also modulates the input preference. An LIF neuron with large c operates as a coincidence detector that Žres only when multiple spikes arrive in a short

112

N. Masuda and K. Aihara

Figure 5: Change in syn and corr when the membrane leak c varies. n1 D 480, n01 D 240, n2 D 30, and 2 D 0.003. We adjust 2 to keep the Žring rates constant.

time window (Abeles, 1991; Softky & Koch, 1993; Fujii et al., 1996; Koch, 1999). Such a cortical neuron detects the peaks of S ( t) at which the sensory neurons send relatively many coincident spikes to the cortical neurons (Masuda & Aihara, 2002a). As a result, the cortical neurons Žre synchronously for increased c , and the rate of synchronous Žring events encodes the amplitude of S (t ). Since the temporal waveform of S (t ) is effectively characterized by the location and the amplitude of its peaks, corr becomes large. Introduction of a refractory period will prevent synchronous bursting with high frequency and therefore eliminate the emergence of simultaneously large syn and corr. 3.4 Heterogeneity. Unlike the homogeneous model neurons considered above, real neurons are naturally heterogeneous. Heterogeneity drives neural populations out of synchrony (White, Chow, Ritt, Soto-Trevino, ˜ & Kopell, 1998; Tiesinga & JosÂe, 2000). We consider heterogeneity in feedforward coupling strength, membrane leak rate, and Žring threshold. For each neuron or synapse, one of these parameters is assigned a value randomly according to the uniform distributions [2 0 (1 ¡ D2 ) , 2 0 (1 C D 2 ) ], [c 0 (1 ¡ Dc ) , c 0 (1 C D c ) ], or [H 0 (1 ¡ D H ) , H 0 (1 C D H ) ], where 2 0 D 0.007, c 0 D 0.025 ms¡1 , and H 0 D 1. The degrees of heterogeneity are parameterized by D2 , D c , and D H . Figure 6 shows that in all three cases, the cortical neurons prefer the synchronous coding when the network is close to homogeneous, while the population rate coding is more efŽcient with increased heterogeneity. Figure 6D, in comparison with Figure 6C, shows that the conclusion remains the same when a stochastic instead of chaotic input is applied. The stochas-

Rate Coding and Temporal Coding

113

A

B 0.8

0.8

syn corr

0.6

0.6

0.4

0.4

0.2

0.2

0

0

0.5

1

1.5

2

syn corr

2.5

0

3

normalized heterogeneity in feedforward coupling C

0

0.05

0.1

0.15

0.2

normalized heterogeneity in membrane leak D

0.8

0.8

syn corr

0.6

0.6

0.4

0.4

0.2

0.2

0

0

0.05

0.1

0.15

syn corr

0.2

0.25

normalized heterogeneity in threshold

0

0

0.05

0.1

0.15

0.2

0.25

normalized heterogeneity in threshold

Figure 6: Change in syn and corr when the cortical neurons are heterogeneous. We consider heterogeneity in (A) the feedforward coupling strength 2 , (B) the membrane leak rate c , and (C, D) the Žring threshold H of the cortical neurons. The Lorenz and Žltered gaussian noise inputs are applied in A, B, C, and in D, respectively. We set n1 D 480, n01 D 240, n2 D 30, and 2 D 0.003. The horizontal axes show the width of the uniform distribution normalized by the average.

tic input signal is generated from the Ornstein-Uhlenbeck process, or the Žltered gaussian noise (Mainen & Sejnowski, 1995; van Rossum et al., 2002), deŽned by dx x C j ( t) , D ¡ tou dt S (t) D 2.25 £ 10¡2 C 4.00 £ 10¡4 x,

(3.1)

114

N. Masuda and K. Aihara

where tou D 8.0 ms and j ( t) is gaussian white noise applied at every time step dt D 0.02 ms. Heterogeneous resetting (Mar et al., 1999) and transmission delays of feedforward couplings (Gewaltig, Diesmann, & Aertsen, 2001) would introduce additional jitter. The biases in estimating S ( t) that heterogeneity could cause are negligible. Actually, corr grows to fairly large values (corr ¸ 0.6) for strong heterogeneity, compared to that when the network is homogeneous or when r or 2 is small. Although the Žring rates of the cortical neurons differ with heterogeneity, S ( t) can be estimated reliably from the collection of heterogeneous spike trains. The network states easily escape from constrained states such as full synchrony or clustered states (Mar et al., 1999; Guardiola et al., 2000). Homogeneous neurons tend to synchronize no matter how small r, 2 , and c are (Knight, 1972). Even when r D 0, two spike trains arriving at two cortical neurons still resemble each other because they encode the same stimulus, just with different phases. Synchronous Žring of coupled oscillators is considered important (Eckhorn et al., 1988; Gray, Konig, ¨ Engel, & Singer, 1989; Hatsopoulos et al., 1998; Brecht et al., 1999; Steinmetz et al., 2000), and mathematical studies have explored how much heterogeneity in, for example, the Žring thresholds and coupling strengths (Bottani, 1995) a stable synchronization can endure. However, it has been shown that even less than 10% heterogeneity in the Žring rates easily desynchronizes neurons (White et al., 1998; Tiesinga & JosÂe, 2000). Heterogeneous neurons do not necessarily synchronize all the time. Asynchronous Žring can provide accurate estimates of the instantaneous Žring rate (Knight, 1972; van Rossum et al., 2002).

4 Parameter UniŽcation

We have seen the effects of changing several model parameters on coding modes. Cortical neurons are likely to prefer the synchronous coding when the shared connectivity is large, strong feedbacks are present, the membrane leak is large, or the cortical neurons are fairly homogeneous. The performance as a population rate coder is highest in the opposite extremes except when the membrane leak is changed. The leak rate determines how inputs are Žltered, as well as the excitation dynamics. Selection of the coding scheme depends on the parameter values in a continuous manner. We deŽne an order parameter s that is essential in determining the preferred coding mode. The value of s measures the accumulated dispersion of the membrane potentials of two neurons after the duration of a typical interspike interval. We assume that the arrival of presynaptic spikes is a Poisson process with parameter l and that the input is approximated by a diffusion process (Lansky, 1984; Tuckwell, 1988). The membrane potential vi of the ith neuron

Rate Coding and Temporal Coding

115

follows the stochastic differential equation, dvi ( t) D (m ¡ c i vi ( t) ) dt C

p

DdBi (t ) ,

(4.1)

m D lE2 [2

i, j ],

(4.2)

D D lE2 [2

2 i, j ],

(4.3)

lD

220 N D , 40 ms T

where c i and 2 i, j are the leak rate of the ith cortical neuron and the amplitude of an incident spike from the sensory layer to the cortical neuron, respectively. The Brownian motion is denoted by Bi ( t) , E is the expectation, T is the typical interspike interval, and N is the number of incident spikes necessary for a cortical neuron to Žre. We assume that 2 i, ji ,c i , and also H i , the Žring threshold of the ith neuron, are taken from uniform distributions on [2 0 (1¡D2 ) , 2 0 (1 C D 2 ) ], [c 0 (1¡Dc ) , c 0 (1 C Dc )], and [H 0 (1¡D H ) , H 0 (1 C D H )], respectively. Consequently, equations 4.2 and 4.3 reduce to m D l2 0 , Á D D l 1C

D2

2

3

(4.4) ! .

(4.5)

Supposing vi (0) D 0, it follows from equations 4.1 and 4.4 that Z

vi (t) D

t 0

0

e ¡c i ( t¡t ) (l2 0 dt C

p

DdBi ( t) ) dt 0 .

(4.6)

We Žrst consider the uctuation of the interspike interval T caused mainly by heterogeneous H i and c i . We calculate T and its standard deviation s1 without considering input jitter. Substituting vi (t) D H i into equation 4.6 and ignoring the diffusive term, the interspike interval T is estimated by ³ ´ 1 c i Hi . (4.7) T D ¡ log 1 ¡ ci l2 0 Expansion of the right-hand side of equation 4.7 to the Žrst order in D H and D c yields T D A C (¡A C B )

ci ¡c 0 Hi ¡ H 0 CB , c0 H0

(4.8)

where ³ ´ 1 c 0H0 A D ¡ log 1 ¡ D E H,c [T], c0 l2 0

(4.9)

116

N. Masuda and K. Aihara

and BD¡

l2

0

H0 . ¡ c 0H 0

(4.10)

On the basis of equation 4.8 and the assumption that Hi and c i are uniformly distributed, we have s12 D s12 (c , D H , Dc ) ´ E H,c [(T¡A) 2 ] D

D c2 3

( A C B) 2 C

D 2H 3

B2 .

(4.11)

We next deŽne s2 , which estimates how the membrane potentials of two neurons starting from the same resting potential disperse when they come close to the threshold. To evaluate the difference between vi1 ( T ) and vi2 (T ) , it should be noted that Bi1 ( t) and Bi2 (t) are not independent of each other because of the shared connectivity. For any L2 ([0, T]) –bounded function f ( t) , it follows from the Itoˆ isometry that 2Á Z 4 Ev

!2 3

T

5D

f (t) dB ( t) 0

Z

T

f ( t) 2 dt,

(4.12)

0

where Ev [¢] denotes the average over the probability measure of the Brownian motion. We obtain "Z # Z Z T

Ev

0

T

f ( t) dBi1 ( t)

0

T

f ( t) dBi2 (t )

Dr

f ( t) 2 dt.

(4.13)

0

since the proportion r of the total incident spike inputs is shared by a pair of neurons (Shadlen & Newsome, 1998). We then normalize vi ( t) so that vi1 ( T ) and vi2 (T ) can be compared with the same effective threshold. By substituting vi (t ) D H i vQ i (t) / H 0 into equation 4.1, we obtain ³ dvQ i ( t) D

p ´ m H0 DH 0 ( ) ¡ c i vQ i t dt C dBi ( t) , Hi Hi

(4.14)

and the threshold for Žring is normalized to vQ D H 0 . Using E H,c ,v[vQ i1 ( t) ¡ vQ i2 ( t) ] D 0 and equations 4.5, 4.6, 4.12, and 4.13, the second-order approximation leads to s22 D s22 (r, c , D2 , D c , D H ) ´ E H,c ,v[ (vQ i1 (T ) ¡ vQ i2 (T ) ) 2 ] "ÁZ T e ¡c i1 ( T¡t ) H 0 (l2 D E H,c ,v H i1 0

0 dt C

p

DdBi1 ( t) )

Rate Coding and Temporal Coding

Z ¡

0

Z

0

0

e ¡c i1 ( T¡t) H 0 l2 Hi1

Z

T

T

¡r

Z 0

Á

1C

2Á

D2 2

¡l

!

3

ÁZ

T 0

Á ¡ r 1C

Z

T 0

5

e ¡2c i1 ( T¡t ) H 20D dt H2i1

e ¡c i2 (T¡t) H 0l2 H i2 3

T

dt 0

1 ¡ e ¡c i1 T 4 D 2l2 20 E c : 1 ¡ D 2H c i1 C

DdBi2 ( t) )

C

dt

!2 3

p

0

dt

e (c i1 Cc i2 ) ( T¡t ) H20 D dt5 H i1 Hi2

0

l

0 dt C

!2

e ¡c i1 ( T¡t ) H 0l2 H i1

T

¡

8 <

e ¡c i2 ( T¡t) H 0 (l2 Hi2

T 0

2Á Z 4 D 2E H,c ,v

117

1 1¡ D 2H

Z

T 0

!2 3 5 e ¡2c 0 (1¡D c ) t ¡ e ¡2c 0 (1CD c ) t dt 4c 0 Dc t

e ¡c 0 (1¡D c ) t ¡e ¡c 0 (1CD c ) t dt 2c 0 D c t

! D22 Z 3

T 0

³

!2

e ¡c 0 (1¡D c ) t ¡e ¡c 0 (1CD c ) t 2c 0D c t

´2

9 = dt . (4.15) ;

To combine s1 and s2 into the uniŽed parameter s, s1 should be transformed into a dispersion of membrane potentials. Using equation 4.14, the mean inŽnitesimal increase of vQ i at the Žring threshold is given by "³

Ec , H,v

#­ p ´ ­ m H0 DH 0 ­ ¡ c i vQ i ( t) dt C dBi ( t) ­ ­ Hi Hi

D (m ¡ c 0 H 0 ) dt D (l2 0 ¡ c 0 H 0 ) dt.

vQ i D H 0

(4.16)

Accordingly, we deŽne s by s D (l2

0

¡ c 0 H 0 )s1 C s2 .

(4.17)

We evaluate the integrals in equation 4.15 numerically. The relations between s and the coding performances are shown in Figure 7. The order parameter s parameterizes syn and corr well in spite of some deviations whose causes should be investigated in the future. Our method cannot incorporate 2 because its effect cannot be derived only by observing the dynamics of vi .

118

N. Masuda and K. Aihara

B 0.8

r g D Dg DQ

syn

0.6 0.4 0.2 0

0.01

0.1 degree of deviation

c o rr

A

0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

r g D Dg DQ 0.01

0.1 degree of deviation

Figure 7: (A) syn and (B) corr with respect to the order parameter s. The data points identiŽed by the symbols r, c , D 2 , D c , and D H are identical to those used for Figures 3, 5, 6A, 6B, and 6C, respectively.

5 Discussion 5.1 Dual Coding. The synchronous coding is more robust against disorders of Žring in each neuron or noise than the population rate coding (Masuda & Aihara, 2002b). The synchronous coding also consumes less power since downstream neurons Žre with a smaller number of incident spikes when they arrive synchronously than when asynchronously (Abeles, 1982, 1991; Bernander et al., 1991). It has also been shown numerically that a volley of synchronous Žring can stably propagate from layer to layer if the instantaneous synchronous population has at least 90 neurons (Diesmann et al., 1999; Cˆateau & Fukai, 2001; Gewaltig et al., 2001). However, the efŽciency of information transmission by the synchronous coding is limited by the duration of interspike or intersynchronization intervals. Signal information temporally Žner than this interval is discarded by time averaging over the interval. When the cortical neurons Žre asynchronously, the spike train from each cortical neuron encodes a different aspect of the signal. Thus, the Žner temporal waveform of S ( t) is restored by the population rate coding. Although the stimulus frequency can be much higher than the characteristic frequency of a single neuron, the cortical layer as a whole can preserve accurate information on signals (Knight, 1972; Mar et al., 1999). The population rate coding is slightly less robust than the synchronous coding in the sense that displacement in each spike timing directly affects the coding performance. This inuence is small, however, and the population rate coding in networks is sufŽciently stable with graceful degradation. We have shown that

Rate Coding and Temporal Coding

119

the functional role of asynchronous states is as important as that of synchronous states. The importance of analyses of complete asynchrony (Domany et al., 1994; van Vreeswijk & Sompolinsky, 1996; Bressloff & Coombes, 2000) should be recognized. Real neural networks may use the two coding mechanisms dually (Abeles, 1991; Araki & Aihara, 2001; Lu, Liang, & Wang, 2001; Tiesinga, Fellous, JosÂe, & Sejnowski, 2002). Actually, at the single-neuron level, neurotransmitter release probability mediates selection of the rate coding or the temporal coding by means of coincidence detection (Tsodyks & Markram, 1997). Selection between synchronous and asynchronous Žring is observed in experiments in which tasks or stimuli are switched (de Oliveira et al., 1997; Riehle, Grun, ¨ Diesmann, & Aertsen, 1997; Steinmetz et al., 2000). Animals may exploit sharply tuned population averaging under new or competitive circumstances so that they can gain as much external information as fast as possible. Once they adapt and get out of emergent situations, robust coding modes may be preferred even if they are not particularly efŽcient for reproducing the waveform of the external signal. Then the internal parameters may be modulated, or background activity may decrease, so that the synchronous coding replaces the population rate coding. The concept of dual coding also appears in different contexts in which more than one dynamic properties of Žring patterns are used simultaneously to encode different features of external stimuli or cognitive states. Dynamical cell assemblies may be linked to meanings while the Žring rate of a single neuron may represent the weight of its contribution to the meaning (Douglass & Martin, 1991; Kruger ¨ & Becker, 1991). The degree of synchrony may represent external or internal events, while the modulation of Žring rates may reect existence of external stimuli (Riehle et al., 1997; de Oliveira et al., 1997). The dual coding that we have proposed allows one of the two complementary codings selected appropriately for each situation (van Rossum et al., 2002). 5.2 Feedforward Networks with Many Layers. Our results can be generalized to feedforward neural networks with more than two layers. Feedback coupling among sensory neurons was ignored in this work, and the sensory layer worked as a population rate coder without synchronization. We have discussed only how the signal information passed through the population rate coder is processed in the second layer. We can consider another layer that receives spike trains from the cortical neurons of Figure 1. When the cortical layer works as a population rate coder, the arguments in the earlier sections apply to this additional layer, which works as a population rate coder or a synchronous coder, depending on the conŽguration or the value of s. Then how does the additional layer behave when the cortical neurons of Figure 1 Žre synchronously, or, equivalently, what information does the cortical layer encode when the sensory neurons Žre synchronously?

120

N. Masuda and K. Aihara

layer

A

B

C

D

E

F

(a)

R

R

R

R

R

S

(b)

R

R

S

S

S

S

(c)

R

S

S

S

R?

?

desynchronized Figure 8: Architectures of multilayered feedforward networks. The layers labeled by R and S work as population rate coders and synchronous coders, respectively. The mechanisms presented in a and b are realizable, while that in c is not.

If moderately strong noise is applied to each layer, then each layer works as an efŽcient population rate coder, and the information on the stimulus is stably transmitted through many layers (van Rossum et al., 2002). Once a layer Žres synchronously, the network loses the detailed information necessary for reliable signal estimation by population Žring rates (Abeles, 1991). Postsynaptic neurons Žre in unison when they receive inputs in the form of a synchronous volley (Abeles, 1991; MacLeod et al., 1998; Diesmann et al., 1999; Gewaltig et al., 2001; van Rossum et al., 2002), and successive layers would encode the signal by the synchronous coding. Although postsynaptic neurons could desynchronize given strong heterogeneity, for example, they cannot realize efŽcient population rate coding because accurate information on S ( t) is lost once the signal passes through a synchronous layer. The downstream layer receives signal information that is no more complete than that encoded by the synchronous coder in the previous layer. It is easy to switch from the population rate code to the synchronous code, whereas the reverse is difŽcult. Figure 8 shows how multilayered feedforward networks work. The network architectures shown in Figures 8a and 8b are easily realizable, with Figure 8b generalizing the synŽre chain model. A modal change from the popula-

Rate Coding and Temporal Coding

121

tion rate code to the synchronous code occurs at a certain stage. This transition does not have to be immediate; the networks can gradually shift the coding mode through multiple layers. Once the synchronous coding is reached, it is used in all the following layers. This is consistent with the Žnding that sensory neurons are more likely to use the rate coding, while cortical neurons tend to resort to temporal codings for, for example, feature binding (Domany et al., 1994; Fujii et al., 1996). The information coding scheme shown in Figure 8c is unlikely to encode the external signal faithfully because the last two layers represent nothing meaningful unless we use skillfully elaborated signal detectors that seem to be biologically implausible. Biological neural networks have global feedback connections among cortical layers. Although little is understood about the role of this feedback, it may return the global information stored in downstream or higher layers into upstream or lower layers that code local features of stimuli and promote further information processing. Model studies suggest that global feedback may enhance synchronization in upstream layers (Damasio, 1989). Here we rely on the general principle that correlated inputs help neurons synchronize. Global feedback from layer F to layer B in Figure 8a would compel the neurons in layer B to synchronize, and then synchronous Žring would propagate to layers C, D, E, and F. Even if the neurons in layer F are not synchronized, other factors such as high shared connectivity in the global feedback to layer B could trigger synchronous Žring in layer B, with layer F sending synthesized information on stimuli back to the lower layers. Global feedback from one layer to another, both of which do the population rate coding, may induce synchrony in the long run. The following speculation explains how synchrony is also stabilized by global feedback. Let us consider the global feedback from layer F to layer D in Figure 8b and assume that synchrony within each layer is not yet stable, possibly because of a paucity of the neurons in each layer (Diesmann, Gewaltig, & Aertsen, 1999; Cˆateau & Fukai, 2001; Gewaltig et al., 2001). Layer D receives partially synchronous volleys from layers C and F. If the synaptic delays are nearly homogeneous and the synchronous volleys are received with appropriate timing, synchrony is enhanced through layer D (Cˆateau & Fukai, 2001). Nonuniform delays inherent in real neural networks, however, will complicate the problem. 5.3 Learning and Its Timescale. Our work is motivated by experimental results showing the positive role of the desynchronization that occurs when cognitive states may be switched during tasks (Vaadia et al., 1995; de Oliveira et al., 1997; Riehle et al., 1997). The neuronal states switch from synchrony to asynchrony and vice versa within 1 to 10 ms in response to environmental changes. In addition to noise intensity (Masuda & Aihara, 2002b; van Rossum et al., 2002), threshold for Žring (Koch, 1999) is capable of rapid change to accommodate fast switching between two modes, and

122

N. Masuda and K. Aihara

the membrane leak rate is also predicted to change rapidly (Bernander et al., 1991). The shared connectivity and coupling strengths reect the wiring structure. They are mainly modulated through long-term potentiation (LTP) and long-term depression (LTD) that require tens of minutes or more (Koch, 1999). Synaptic learning on long timescales may be relevant to differentiation and long-term memory associated with selection of coding schemes as long-term strategies. Numerical studies have also shown that coupled LIF neurons can specialize in coincidence detection or synchronous coding after learning (Gerstner et al., 1996). Although much less is known about short-term synaptic dynamics, synaptic modulation that occurs in less than a second has been reported in the context of spike-timing-dependent plasticity (Markram, Lubke, ¨ Frotscher, & Sakmann, 1997; Abbott & Nelson, 2000), effective connectivity (Aertsen et al., 1989; Fujii et al., 1996), redistribution of synaptic efŽcacy (Markram & Tsodyks, 1996; Abbott & Nelson, 2000), and synaptic depression (Abbott, Varela, Sen, & Nelson, 1997; Tsodyks & Markram, 1997). Such short-term synaptic modulation may be capable of, for example, feature binding (von der Malsburg & Schneider, 1986), dynamic gain control (Tsodyks & Markram, 1997), and selection of the coding scheme (Tsodyks & Markram, 1997; Araki & Aihara, 2001; Tiesinga et al., 2002). In physiological experiments showing selection of synchrony and asynchrony during tasks, the population rate coding and the synchronous coding may have been properly switched as a result of short-term learning. It is important to conŽrm experimentally possible effects of synaptic modulation on neural coding. Acknowledgments

This work is supported by the Japan Society for the Promotion of Science and CREST, JST. References Abbott, L. F., & Nelson, S. B. (2000). Synaptic plasticity: Taming the beast. Nature Neuroscience (Suppl.), 3, 1178–1183. Abbott, L. F., Varela, J. A., Sen, K., & Nelson, S. B. (1997). Synaptic depression and cortical gain control. Science, 275, 220–224. Abeles, M. (1982).Role of the cortical neuron: Integrator or coincidence detector? Isr. J. Med. Sci., 18, 83–92. Abeles, M. (1991). Corticonics. Cambridge: Cambridge University Press. Aertsen, A. M. H. J., Gerstein, G. L., Habib, M. K., & Palm, G. (1989). Dynamics of neuronal Žring correlation: Modulation of “effective connectivity.” J. Neurophsiol., 61(5), 900–917. Aihara, K., & Tokuda, I. (2002). Possible neural coding with inter-event intervals of synchronous Žring. Physical Review E, 66, 026212.

Rate Coding and Temporal Coding

123

Araki, O., & Aihara, K. (2001). Dual information representation with stable Žring rates and chaotic spatiotemporal spike patterns in a neural network model. Neural Computation, 13, 2799–2822. Arieli, A., Sterkin, A., Grinvald, A., & Aertsen, A. (1996). Dynamics of ongoing activity: Explanation of the large variability in evoked cortical responses. Science, 273, 1868–1871. Azouz, R., & Gray, C. M. (1999). Cellular mechanisms contributing to response variability of cortical neurons in vivo. J. Neurosci., 19(6), 2209– 2223. ¨ Douglas, R. J., Martin, K. A. C., & Koch, C. (1991). Synaptic backBernander, O., ground activity inuences spatiotemporal integration in single pyramidal cells. Proc. Natl. Acad. Sci. USA, 88, 11569–11573. Bottani, S. (1995). Pulse-coupled relaxation oscillators: From biological synchronization to self-organized criticality. Phys. Rev. Lett., 74(21), 4189–4192. Brecht, M., Singer, W., & Engel, A. K. (1999). Patterns of synchronization in the superior colliculus of anesthetized cats. J. Neurosci., 19(9), 3567–3579. Bressloff, P. C., & Coombes, S. (2000). Dynamics of strongly coupled spiking neurons. Neural Computation, 12, 91–129. Caˆ teau, H., & Fukai, T. (2001). Fokker-Planck approach to the pulse packet propagation in synŽre chain. Neural Networks, 14, 675–685. Damasio, A. R. (1989). The brain binds entities and events by multiregional activation from convergence zones. Neural Computation, 1, 123–132. de Oliveira, S. C., Thiele, A., & Hoffmann, K-P. (1997). Synchronization of neuronal activity during stimulus expectation in a direction discrimination task. J. Neurosci., 17(23), 9248–9260. Diesmann, M., Gewaltig, M-O., & Aertsen, A. (1999). Stable propagation of synchronous spiking in cortical neural networks. Nature, 402, 529–533. Domany, E., van Hemmen, J. K., & Schulten, K. (Eds.). (1994). Models of neural networks II. New York: Springer-Verlag. Douglass, R. J., & Martin, K. A. C. (1991). Opening the grey box. Trends in Neurosciences, 14(7), 286–293. Eckhorn, R., Bauer, R., Jordan, W., Brosch, M., Kruse, W., Munk, M., & Reitboeck, H. J. (1988). Coherent oscillations: A mechanism of feature linking in the visual cortex? Biol. Cybern., 60, 121–130. Feng, J., Brown, D., & Li, G. (2000). Synchronization due to common pulsed input in Stein’s model. Phys. Rev. E, 61(3), 2987–2995. Fujii, H., Ito, H., Aihara, K., Ichinose, N., & Tsukada, M. (1996). Dynamical cell assembly hypothesis—theoretical possibility of spatio-temporal coding in the cortex. Neural Networks, 9(8), 1303–1350. Gerstner, W., Kempter, R., van Hemmen, J. L., & Wagner H. (1996). A neuronal learning rule for sub-millisecond temporal coding. Nature, 383, 76–78. Gewaltig, M-O., Diesmann, M., & Aertsen, A. (2001). Propagation of cortical synŽre activity: Survival probability in single trials and stability in the mean. Neural Networks, 14, 657–673. Gray, C. M., Konig, ¨ P., Engel, A. K., & Singer, W. (1989). Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reects global stimulus properties. Nature, 338, 334–337.

124

N. Masuda and K. Aihara

Guardiola, X., DÂõ az-Guilera, A., Llas, M., & PÂerez, C. J. (2000). Synchronization, diversity, and topology of integrate and Žre oscillators. Phys. Rev. E, 62(4), 5565–5570. Hatsopoulos, N. G., Ojakangas, C. L., Paninski, L., & Donoghue, J. P. (1998). Information about movement direction obtained from synchronous activity of motor cortical neurons. Proc. Natl. Acad. Sci. USA, 95, 15706– 15711. Knight, B. W. (1972). Dynamics of encoding in a population of neurons. Journal of General Physiology, 59, 734–766. Koch, C. (1999). Biophysics of computation. New York: Oxford University Press. Kruger, ¨ J., & Becker, D. (1991). Recognizing the visual stimulus from neuronal discharges. Trends in Neurosciences, 14(7), 282–286. Lansky, P. (1984). On approximations of Stein’s neuronal model. J. Theor. Biol., 107, 631–647. Lu, T., Liang, L., & Wang, X. (2001). Temporal and rate representations of timevarying signals in the auditory cortex of awake primates. Nature Neuroscience, 4(11), 1131–1138. MacLeod, K., B¨acker, A., & Laurent, G. (1998). Who reads temporal information contained across synchronized and oscillatory spike trains? Nature, 395, 693– 698. Mainen, Z. F., & Sejnowski, T. J. (1995). Reliability of spike timing in neocortical neurons. Science, 268, 1503–1506. Mar, D. J., Chow, C. C., Gerstner, W., Adams, R. W., & Collins, J. J. (1999). Noise shaping in populations of coupled model neurons. Proc. Natl. Acad. Sci. USA, 96, 10450–10455. Markram, H., Lubke, ¨ J., Frotscher, M., & Sakmann, B. (1997). Regulation of synaptic efŽcacy by coincidence of postsynaptic APs and EPSPs. Science, 275, 213–215. Markram, H., & Tsodyks, M. (1996). Redistribution of synaptic efŽcacy between neocortical pyramidal neurons. Nature, 382, 807–810. Masuda, N., & Aihara, K. (2001). Synchronization of pulse-coupled excitable neurons. Phys. Rev. E, 64, 051906. Masuda, N., & Aihara, K. (2002a). Spatio-temporal spike encoding of a continuous external signal. Neural Computation, 14, 1599–1628. Masuda, N., & Aihara, K. (2002b). Bridging rate coding and temporal spike coding by effect of noise. Phys. Rev. Lett., 88(24), 248101. Riehle, A., Grun, ¨ S., Diesmann, M., & Aertsen, A. (1997). Spike synchronization and rate modulation differently involved in motor cortical function. Science, 278, 1950–1953. Sauer, T. (1994). Reconstruction of dynamical systems from interspike intervals. Phys. Rev. Lett., 72(24), 3811–3814. Shadlen, M. N., Britten, K. H., Newsome, W. T., & Movshon, J. A. (1996). A computational analysis of the relationship between neuronal and behavioral responses to visual motion. J. Neurosci., 16(4), 1486–1510. Shadlen, M. N., & Newsome, W. T. (1998). The variable discharge of cortical neurons: Implications for connectivity, computation, and information coding. J. Neurosci., 18(10), 3870–3896.

Rate Coding and Temporal Coding

125

Softky, W. R., & Koch, C. (1993). The highly irregular Žring of cortical cells is inconsistent with temporal integration of random EPSPs. J. Neurosci., 13(1), 334–350. Stein, R. B. (1967). The information capacity of nerve cells using a frequency code. Biophysical Journal, 7, 797–826. Steinmetz, P. N., Roy, A., Fitzgerald, P. J., Hsiao, S. S., Johnson, K. O., & Niebur, E. (2000). Attention modulates synchronized neuronal Žring in primate somatosensory cortex. Nature, 404, 187–190. Thorpe, S., Fize, D., & Marlot, C. (1996). Speed of processing in the human visual system. Nature, 381, 520–522. Tiesinga, P. H. E., & JosÂe, J. V. (2000). Robust gamma oscillations in networks of inhibitory hippocampal interneurons. Network, 11, 1–23. Tiesinga, P. H. E., Fellous, J-M., JosÂe, J. V., & Sejnowski, T. J. (2002). Information transfer in entrained cortical neurons. Network, 13, 41–66. Tokuda, I., & Aihara, K. (2000). Inter-event interval reconstruction of chaotic dynamics. In Proc. of the Fifth International Symposium on ArtiŽcial Life and Robotics (AROB 5th’00) (pp. 177–180). Oita, Japan: Shubundo Insatsu. Tsodyks, M. V., & Markram, H. (1997). The neural code between neocortical pyramidal neurons depends on neurotransmitter release probability. Proc. Natl. Acad. Sci. USA, 94, 719–723. Tuckwell, H. C. (1988). Introduction to theoretical neurobiology (Vols. 1, 2). Cambridge: Cambridge University Press. Vaadia, E., Haalman, L., Abeles, M., Bergman, H., Prut, Y., Slovin, H., & Aertsen, A. (1995). Dynamics of neuronal interactions in monkey cortex in relation to behavioural events. Nature, 373, 515–518. van Rossum, M. C. W., Turrigiano, G. G., & Nelson, S. B. (2002). Fast propagation of Žring rates through layered networks of noisy neurons. J. Neurosci., 22(5), 1956–1966. van Vreeswijk, C., & Sompolinsky, H. (1996). Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science, 274, 1724–1726. von der Malsburg, Ch., & Schneider, W. (1986).A neural cocktail-party processor. Biol. Cybern., 54, 29–40. White, J. A., Chow, C. C., Ritt, J., Soto-Trevino, ˜ C., & Kopell, N. (1998). Synchronization and oscillatory dynamics in heterogeneous, mutually inhibited neurons. J. Comput. Neurosci., 5, 5–16. Zohary, E., Shadlen, M. N., & Newsome, W. T. (1994). Correlated neuronal discharge rate and its implications for psychophysical performance. Nature, 370, 140–143. Received January 14, 2002; accepted July 2, 2002.

LETTER

Communicated by Laura Martignon

Synchronous Firing and Higher-Order Interactions in Neuron Pool Shun-ichi Amari [email protected] Hiroyuki Nakahara [email protected] Laboratory for Mathematical Neuroscience, RIKEN Brain Science Institute, Wako-shi, Saitama, Japan Si Wu [email protected] Department of Computer Science, ShefŽeld University, ShefŽeld S1, 4DP, U.K. Yutaka Sakai [email protected] Department of Information and Computer Science, Saitama University, Saitama-shi, Saitama, Japan The stochastic mechanism of synchronous Žring in a population of neurons is studied from the point of view of information geometry. Higherorder interactions of neurons, which cannot be reduced to pairwise correlations, are proved to exist in synchronous Žring. In a neuron pool where each neuron Žres stochastically, the probability distribution q ( r ) of the activity r, which is the fraction of Žring neurons in the pool, is studied. When q ( r) has a widespread distribution, in particular, when q (r ) has two peaks, the neurons Žre synchronously at one time and are quiescent at other times. The mechanism of generating such a probability distribution is interesting because the activity r is concentrated on its mean value when each neuron Žres independently, because of the law of large numbers. Even when pairwise interactions, or third-order interactions, exist, the concentration is not resolved. This shows that higher-order interactions are necessary to generate widespread activity distributions. We analyze a simple model in which neurons receive common overlapping inputs and prove that such a model can have a widespread distribution of activity, generating higher-order stochastic interactions. 1 Introduction

Let us consider a pool of n binary neurons. Let xi be binary random variables, xi D 1 when the ith neuron Žres, and xi D 0 otherwise, and i D 1, . . . , n. The Neural Computation 15, 127–142 (2003)

° c 2002 Massachusetts Institute of Technology

128

S. Amari, H. Nakahara, S. Wu, and Y. Sakai

neurons Žre with a Žxed joint probability p ( x) of x D (x1 , . . . , xn ) , which may be correlated. The activity of this neuron pool is deŽned by rD

n 1X xi , n iD 1

(1.1)

which is the fraction of Žring neurons in the pool. This article studies the probability distribution of r called the joint Žring distribution. When n is large, we regard r as a continuous variable and denote its probability density function by q (r ). Then, q (r ) dr D

X P

p (x ) ,

(1.2)

xi D nr

P where the summation is taken over all x such that xi D nr and dr D 1 / n, reecting the fact that r is an integer multiple of 1 / n. When q ( r) is a unimodal distribution having a sharp peak at r0, the activity of the pool is mostly r0 , slightly uctuating around that. However, when q ( r) is widely distributed, in particular, when it has two peaks at r1 and r2 ( r1 < r2 ), the activity is sometimes around r2 and sometimes around r1 . When r2 is large and r1 is small, many neurons sometimes Žre simultaneously, while many neurons are quiescent at other times. This is the phenomenon called synchronous Žring of neurons (Abeles, 1991; Singer & Gray, 1995; Aertsen & Arndt, 1993). This was analyzed in Bohte, Spekreijse, and Roelfsema (2000) by using the maximum entropy principle. The higher-order interactions of neurons were also discussed in Martignon, Von Hasseln, Grun, ¨ Aertsen, & Palm (1995) and Bohte et al. (2000). See also Gerstein, Bedenbaugh, and Aertsen (1989) and Vaadia et al. (1995) for higher-order interactions in real neuron pools, and Amari (2001) and Nakahara and Amari (2002) for theoretical studies. We study the mechanism of synchronous Žring by using information geometry (Amari & Nagaoka, 2000) to show that higher-order interactions of neurons (Amari, 2001; Nakahara & Amari, 2002) are required to generate widespread synchronous Žring, and pairwise correlations cannot generate such a synchronous Žring distribution. We consider the case that n is large and that the distribution p ( x) is symmetric with respect to n variables x1 , . . . , xn , that is, the pool is homogeneous. The distribution q ( r) of activity (see Figure 1) reveals the inner structure of the neuron pool. We Žrst show two extreme cases. 1. Independent case: Each neuron Žres independently with probability a,

p ( x) D

n Y iD 1

fad ( xi D 1) C (1 ¡ a)d ( xi D 0) g,

(1.3)

Synchronous Firing and Higher-Order Interactions in Neuron Pool

129

A

40

q(r)

30 20 10 0 0.0

q(r)

3

0.2

0.4

0.2

0.4

0.2

0.4

B

r

0.6

0.8

1.0

0.6

0.8

1.0

0.6

0.8

1.0

2 1 0 0.0 8

C

r

q(r)

6 4 2 0 0.0

r

Figure 1: Typical joint Žring distributions: (A) concentrated, (B) widely spread, (C) bimodal.

where d is 1 when the event in the parentheses holds and 0 otherwise. In this case, xi is 1 with probability a, so that rD

1X xi n

(1.4)

converges to a as n ! 1. Hence, q ( r ) D d ( r ¡ a) ,

(1.5)

where d is the delta function, and the distribution is concentrated at r D a (see Figure 1A). This concentration is caused by the law of large numbers, when all the neurons Žre independently. Such concentration still holds even where there are pairwise correlations or triple-wise correlations among xi but no further higher-order correlations. Higher-order correlations are required for q ( r) to be widely distributed.

130

S. Amari, H. Nakahara, S. Wu, and Y. Sakai

2. Completely dependent case: All the neurons Žre or are silent at the same

time with probability a or 1 ¡ a, respectively. Then r D 1 with probability a and r D 0 with probability 1 ¡ a, q ( r ) D ad (r ¡ 1) C (1 ¡ a )d (r ).

(1.6)

The distribution has two concentrated peaks, at r D 0 and r D 1. This is the other extreme case where all the neurons synŽre completely. This article studies the intrinsic mechanism that generates a smooth distributed pattern q (r ) of Žring, as is seen in Figures 1B and 1C. Such a structure was observed in Bohte et al. (2000), which mentioned that higher-order interactions of neurons play a fundamental role. Information geometry (Amari & Nagaoka, 2000) is a useful tool for studying higher-order interactions and is applied to analysis of neuron populations (Amari, 2001; Nakahara & Amari, 2002). Here, we use a simple model to analyze the distributed Žring by using information geometry. We prove that weak but many higher-order interactions are necessary for generating a smooth distribution and a twopeak distribution. This elucidates intrinsic synchronous Žring mechanisms of neurons. We further show how to calculate the terms of higher-order interactions from a distribution of activity. The maximum entropy distribution is also studied, conŽrming the results of Bohte et al. (2000). 2 A Simple Model of Neuron Pool

A simple model we consider is a pool of neurons where the potentials ui of the neurons are subject to a joint gaussian distribution. Let us normalize the variance of ui equal to 1, and let a be the covariance of any two ui and uj . There are no higher-order correlations except for pairwise correlations in the gaussian case, but the nonlinear transformation, ( 1, ui > 0 (2.1) xi D 0, ui · 0, generates higher-order correlations among neural Žring x1 , . . . , xn . This distribution is realized when a pool receives common inputs s1 , . . . , sm . Let ui be the weighted sum of inputs to the ith neuron, ui D

m X jD 1

wij sj ¡ h,

(2.2)

where wij is the connection weight from the jth input to the ith neuron. The weights are randomly assigned, for example, ( 1, with probability c (2.3) wij D 0, with probability 1 ¡ c.

Synchronous Firing and Higher-Order Interactions in Neuron Pool

131

These ui are gaussian because of the central limit theorem. We normalize the scale of ui such that they are subject to the gaussian distribution N (¡h, 1) with mean ¡h and variance 1. Let the covariance of two variables be 1 ¸ 6 a ¸ 0 for any i D j. Then we may put ui D

p

1 ¡ avi C

p

ae ¡ h,

(2.4)

where v1 , . . . , vn and e are independent standard gaussian random variables subject to N (0, 1) . We prove that the activity distribution q (r ) D q ( r, a) has the following form under this model. Theorem 1.

The activity distribution q ( r, a) is given, in the limit n ! 1, by "

2a ¡ 1 q ( r, a) D C exp 2(1 ¡ a)

» F

¡1

p

a ( r) ¡ h 2a ¡ 1

where C is the normalization constant and Z 1 1 u2 ( ) e p F D e ¡ 2 du. p ae p 2p h¡ 1¡a

¼

2

#

,

(2.5)

(2.6)

Before giving a proof, we show the shape of q ( r) D q ( r, a) in terms of a 6 (see Figure 2). When a D 0, that is, ui are nearly independent, q (r, 0) is the delta function and is concentrated. (When a D 0 exactly, we cannot use this formula, because F ¡1 (r ) reduces to a constant.) In the range 0 < a

A probability distribution of a binary n-dimensional vector x is expanded in general as 8 9 <X = X hi xi C hij xi xj C ¢ ¢ ¢ C h1¢¢¢n x1 ¢ ¢ ¢ xn ¡ y , (3.1) p (x ) D exp : ; i< j

where hi , hij , . . . , h1¢¢¢n are parameters to specify p ( x) , and y is the normalizing factor. Such a model is called a full log-linear model in statistics. The

132

S. Amari, H. Nakahara, S. Wu, and Y. Sakai 8

q(r)

6

4

2

0 0.0

0.2

0.4

r

0.6

0.8

1.0

Figure 2: Joint Žring distributions of simple model for various a.

log-linear model is used in Gerstein, Bedenbaugh, and Aertsen (1989), Martignon et al. (1995), Vaadia et al. (1995), and others. Information geometry (Amari & Nagaoka, 2000; Amari, 2001; Nakahara & Amari, 2002) elucidates the meaning of this decomposition. The parameter fhi1 ¢¢¢ik g represents the kth-order interactions of k neurons i1 , . . . , ik, such that it is orthogonal to the parameters specifying the marginal distributions of m neurons ( m < k). When p ( x) is symmetric, all hi are equal, and we use h1 for representing 6 it. Also, all hij ( i D j) are equal, and we denote them by h2 , and so on. Hence, p ( x, µ ) D exp

nX

o hk Xk ¡ y ( µ ) ,

(3.2)

where µ D (h1 , . . . , hn ) is the new parameter vector to specify the symmetric distribution and Xk D

X

xi1 ¢ ¢ ¢ xik ,

(3.3)

with the summation taken over all combinations of k indices (i1 , . . . , ik ) . The set of all the probability distributions fp( x, µ )g is an exponential family, where the expectations of Xk are E[Xk] D n C kE[x1 , . . . , xk ], and E denotes expectation and n Ck is the binomial coefŽcients.

(3.4)

Synchronous Firing and Higher-Order Interactions in Neuron Pool

133

The probability q (r, µ ) of exactly k D nr neurons Žring is q ( r, µ ) dr D n CkProbfx1 D ¢ ¢ ¢ D x k D 1 I x kC 1 D ¢ ¢ ¢ D xn D 0g,

(3.5)

because there are n C k distinct patterns of exactly k neurons Žring whose probabilities are equal. When exactly k speciŽc neurons Žre, for example, x1 D ¢ ¢ ¢ D xk D 1 and all the others are 0, X (3.6) Xj D xi1 ,...,ij D k Cj , when j · k, when j > k,

Xj D 0,

(3.7)

so that Probfx1 D ¢ ¢ ¢ D x k D 1 I x kC 1 D ¢ ¢ ¢ D 0g 8 9 <X = k D exp k Cjhj ¡ y . : ;

(3.8)

jD 1

When n is large, we have an asymptotic formula, n Cnr

D p

1 expfnH ( r) g, 2p nr (1 ¡ r)

(3.9)

where H ( r) is the entropy, H ( r) D ¡r log r ¡ (1 ¡ r ) log(1 ¡ r) .

(3.10)

When k D nr, k X jD 1

k Cjhj

D nrh1 C

nr ( nr ¡ 1) nr ( nr ¡ 1) ( nr ¡ 2) h2 C h3 C ¢ ¢ ¢ 2 6

C nrhnr¡1 C hn ,

(3.11)

which is a polynomial in r. Taking these into account, we can rewrite equation 3.5 in the form q ( rI µ ) D exp[nfn (r, µ ) ¡ y ( µ ) ],

(3.12)

where fn ( r, µ ) D H ( r) C where kj,n ( r) D

nr Cj

1X hj kj,n ( r) , n

/ n is a polynomial in r.

(3.13)

134

S. Amari, H. Nakahara, S. Wu, and Y. Sakai

When fn ( r, µ ) has an isolated peak at r D r0 , there exists c > 0 such that fn ( r0 , µ ) ¡ fn (r, µ ) > c > 0

(3.14)

6 r 0, for |r ¡ r0 | > e for arbitrary small e > 0, and we have, for any r D

q ( r, µ ) D exp[nf fn ( r, µ ) ¡ fn ( r 0, µ )g] ! 0, q ( r 0, µ )

(3.15)

as n ! 1. In this case, q ( r, µ ) is concentrated at r0 . In order to have widely spread q ( r) , fn ( r, µ ) should be of order 1 / n. This gives the following theorem. When the activity distribution q ( r) is not concentrated, fn (r, µ ) D O ( n1 ) , and the interactions h2 , . . . , hn¡2 scale as Theorem 2.

h2 D O

³ ´ 1 , n

³ h3 D O ³ hk D O ³ hk D O

1 n2

´

1 nk¡1 1

,

...,

´ ,

k·

n 2

,

k>

n . 2

´

nn¡k

(3.16)

Moreover, µ satisŽes in the limit n ! 1, 1X hj kj,n ( r) . n!1 n

H ( r) D ¡ lim Proof.

(3.17)

Since fn ( r, µ ) is a polynomial, for a nonconcentrated distribution

q ( r) , lim n[ fn ( r, µ ) ¡ fn ( r0 , µ ) ] < 1

n!1

(3.18)

is required. This holds only when fn ( r, µ ) ¡ fn (r 0, µ ) is of order 1 / n, and hence all the hj scale as mentioned, and equation 3.16 holds. We now consider the case when there are no higher-order interactions. The simplest one is that all the hj D 0 except for h1 . The distribution p ( x, µ ) is independent, and q ( r) D d ( r ¡ r0 ) , where r0 D E[r], is easily determined from h1 . The second simplest case is that all hi are zero except for h1 and h2 , h3 D h4 D ¢ ¢ ¢ D 0. This is the case where only pairwise interactions exist.

Synchronous Firing and Higher-Order Interactions in Neuron Pool

135

The stationary distribution of the Boltzmann machine belongs to this class. The distribution is written as µ » ³ ´¼ ¶ 1 1 ¡y , (3.19) q ( r, µ ) D exp n H ( r) C h1 r C hN2 r r ¡ 2 n where hN2 D nh2 . The function fn ( r) D H ( r) C h1 r C hN2 r2

(3.20)

has a peak in the interval [0, 1] for hN2 < 0. Hence, we have a concentrated distribution. When hN2 is positive and large, there can be two concentrated peaks at r D 0 and r D 1, one of which dominates except for the symmetric case. By similar reasoning, we can prove that there is no widespread distribution q ( r) , when there is only a Žnite number of higher-order interactions. Weak higher-order interactions of almost all orders are required for realizing a widespread activity distribution. Theorem 3.

When h1 D nhN1 is of order n and h2 is of order 1, they dominate over the entropy term, and Remark.

µ q ( r, µ ) ¼ exp

¶ n2 fhN2 r2 ¡ hN1 r ¡ y g ¡ y . 2

(3.21)

Hence, for hN2 > 0, the distribution has two concentrated delta peaks at r D 0 and r D 1. This is the case of Bohte et al. (2000). For a symmetric distribution with E[xi ] D rN and Cov[xi , xj ] D r, the variance of r is "» ¼ 2# 1X 1 n¡1 ( xi ¡ rN ) r. (3.22) V[r] D E D V[xi ] C n n n Remark.

When r > 0, the distribution of r is not concentrated but is spread with variance r. This shows that whenever the covariance of xi and xj is a positive constant of order 1, q ( r) is widespread and higher-order interactions exist necessarily. 4 Calculations of Interactions

We calculate the higher-order interactions from a distribution q ( r) . To this Q ( ) Q ( ) Q ( ) end, we introduce P random variables X1 x , . . . , Xn x such that Xi x is equal to 1 when xj D i, that is, when exactly i neurons are excited, and is

136

S. Amari, H. Nakahara, S. Wu, and Y. Sakai

equal to 0 otherwise. This XQ i is different from Xi , and we have X XQ k D xi1 ,...,ik (1 ¡ xikC 1 ) , . . . , (1 ¡ xin ) ,

(4.1)

where the summation is taken over all permutations of indices. We put g k D E[Xk ],

(4.2)

gQ k D E[XQ k ].

(4.3)

We then have the following linear relations: gQ i D

n X jD 1

Theorem 4.

Aijgj I gi D

n X j D1

BijgQj .

(4.4)

The matrices A D ( Aij ) and B D (Bij ) are mutually inverse, and

Aij D (¡1) j¡ij Ci ,

(4.5)

Bij D j Ci .

(4.6)

Let us put

Proof.

xN i D 1 ¡ xi

(4.7)

and xi C xN i D 1. We then have the following relations: x1 , . . . , x k D x1 , . . . , x k ( x kC 1 C xN kC 1 ) , . . . , (xn C xN n ) ,

(4.8)

x1 , . . . , x k (1 ¡ xkC 1 ) , . . . , (1 ¡ xn ) D x1 , . . . , x kxN kC 1 , . . . , xN n .

(4.9)

By expanding the above relations and taking expectation, we have the relations that connect fE[x1 , . . . , x k]g and fE[x1 , . . . , x kxN kC 1 , . . . , xN n ]g. In order to obtain the explicit forms of A and B, we use the identity n C k n¡k Cj

D n CkC j kC j Ck .

Now we introduce new parameters hQ1 , . . . , hQn by X X hQi D hj D Bjihj , AijhQi .

(4.10)

(4.11)

Then we have another expression of probability distribution p (x ), p ( x, µQ ) D exp because

P

hi Xi D

nX

P

o hQj XQ j ¡ y ,

hQi XQ i .

(4.12)

Synchronous Firing and Higher-Order Interactions in Neuron Pool

137

The probability of exactly k neurons Žring is q ( k/ n ) dr D

X

x:

P

p ( x)

xi D k

D n Ck expfhQk ¡ y g,

(4.13)

because when exactly k neurons Žre, XQ i D 1 only when i D k, and all the other XQ i vanish, and there are n Ck such Žring patterns. From this, we have an explicit expression of hQk, hQk D ¡ log n Ck C log[q(k/ n ) / q (0) ],

(4.14)

because expf¡y g D

1 q (0) . n

(4.15)

When q ( r) is given, the corresponding higher-order interactions hk are given explicitly as hi D

X j

Aji f¡ log n Cj C log[q( j/ n ) / q (0) ]g.

(4.16)

5 Entropy Maximization

Bohte et al. (2000) treated the distribution q (r ) by maximizing the entropy under the condition that the Žrst-order and the second-order statistics are Žxed. We touch on this problem. The entropy of p ( x) is given by ¡

X

p ( x) log p ( x) X X gQ i log gQ i ¡ gQ 0 log gQ 0 C gQ i log n Ci , D ¡

(5.1)

where gQ 0 D 1 ¡

n X i D1

gQi .

(5.2)

This is the negative of the dual potential of the exponential family in information geometry (Amari & Nagaoka, 2000). Therefore, hQi are calculated by @ hQi D Q ( ´Q ) D log gQi ¡ log gQ 0 ¡ log n Ci . @gQ i

(5.3)

138

S. Amari, H. Nakahara, S. Wu, and Y. Sakai

The problem is written as: to maximize X SD¡ p (x ) log p (x )

(5.4)

under g1 D nE[x1 ] D const, g2 D n C2 E[x1 x2 ] D const.

(5.5) (5.6)

The solution is given by the distribution p ( x) D expfl1 X1 C l2 X2 ¡ y g,

(5.7)

which has only the pairwise correlations but no higher-order interactions. Now we rewrite the constraints: g1 D g2 D

n X jD 1 n X jD 2

Qj , j C1g

(5.8)

Qi. j C2g

(5.9)

The entropy is written in terms of ´Q as SD¡

n X iD 0

gQ i log gQ i C

X

gQ i log n Ci .

(5.10)

Now we Žx g1 and g2 . The maximal entropy distribution is given from @S @gQ i

D 0,

i D 3, 4, . . . , n.

The solution (Bohte et al., 2000) is ± i ² log Di D ¡ ¡ C 1 (i ¡ 1) log D 0 2 1 ¡ i (i ¡ 2) log D1 C i ( i ¡ 1) log D2 , 2

(5.11)

(5.12)

where Di D

1 gQ i . n Ci

(5.13)

The activity distribution has two peaks in general, but the bigger peak dominates as n ! 1.

Synchronous Firing and Higher-Order Interactions in Neuron Pool

139

6 Derivation of Activity Distribution

The probability that k D nr neurons Žre among n neurons is given by »

k q ( r) dr D Pr r D n

¼

D n Ck Prfx1 D x2 D ¢ ¢ ¢ D x k D 1, xk C 1 D ¢ ¢ ¢ D xn D 0g,

(6.1)

where dr D 1 / n. In order to calculate Prfx1 D ¢ ¢ ¢ D x k D 1, x kC 1 D ¢ ¢ ¢ D xn D 0g

D Prfu1 , . . . , u k > 0 I u k C 1 , . . . , un < 0g,

(6.2)

we Žrst Žx e in the expression ui in equation 2.4 and calculate the conditional probability. We then take the average over e . Since u1 , . . . , un are independent when e is Žxed, Prfx1 D ¢ ¢ ¢ D x k D 1, x kC 1 D ¢ ¢ ¢ D xn g

D Ee [Prfu1 , u2 , . . . , u k > 0, u k C 1 , . . . , un · 0 | e g] D Ee [n C k fPr(u > 0 | e )g k fPr(u · 0 | e ) gn¡k ],

(6.3) (6.4)

where Ee is the expectation taken with respect to a random variable e and Pr (¢ | e ) is the conditional probability conditioned on e —in other words, the probability for a Žxed e . In the above derivation, we used the homogeneity of neural activity, that is, symmetry of x1 , . . . , xn . Let us put F (e ) ´ Pr ( u > 0 | e ) p ¶ µ h ¡ ae D Pr ui > p 1¡a Z 1 t2 1 D p e ¡ 2 dt. p h¡ ae 2p p1¡a

(6.5)

We then have µ ¶ k Pr r D D Ee [n C k fF( e ) gk f1 ¡ F ( e ) gn¡k ] n ³ 2´ Z 1 1 e k n¡k ¢p exp ¡ D de. (6.6) n C k fF( e )g f1 ¡ F ( e ) g 2 2p ¡1

140

S. Amari, H. Nakahara, S. Wu, and Y. Sakai

To investigate further, by taking a large n limit, we approximate equation 6.6 as an equation taking a continuous variable r. In other words, we take µ ¶ k ¡¡¡¡! q ( r) dr, (6.7) P rD n n!1 where dr D we obtain

1 , n

and r on the right side is now a continuous variable. Then

µ ¶ k q (r ) ’ nPr r D n D nEe [n Cnr fF( e ) gnr f1 ¡ F ( e ) gn¡nr ]

r

» ³ ´¼ Z 1 1 ¡ F (e) n F (e) C (1 ¡ r ) log exp n r log D 2p r (1 ¡ r) ¡1 1¡r r ³ 2´ 1 e ¢p exp ¡ de 2 2p r ³ 2´ Z 1 1 e n expfnz (e ) g ¢ p exp ¡ (6.8) D de, 2p r (1 ¡ r) ¡1 2 2p where we used the following approximation, n Cnr

’ p

1 exp(¡nr log r ¡ n (1 ¡ r) log(1 ¡ r ) ) , 2p nr (1 ¡ r )

and deŽned z ( e ) ´ r log

1 ¡ F (e) F (e) C (1 ¡ r) log . 1 ¡r r

(6.9)

By Laplace’s approximation (also referred to as the saddle-point approximation), we get r q (r ) ’

s n ¢ 2p r (1 ¡ r )

Á ! e02 2p 1 exp(nz (e 0 ) ) ¢ p exp ¡ , (6.10) 2 n|z00 ( e0 ) | 2p

where e 0 is given by

e 0 ´ arg max z (e ) . Equation 6.11 is solved by setting e 0 as the solution of » ¼ 1 ¡r dz (e ) r ¡ D F0 ( e ) D 0, 1 ¡ F (e ) de F (e)

(6.11)

(6.12)

Synchronous Firing and Higher-Order Interactions in Neuron Pool

141

and the solution is

e 0 D F ¡1 ( r) .

(6.13)

For r ! 0, 1, we have e 0 ! § 1 or, equivalently, F ( e ) ! 0, 1. By some calculations, equation 6.10 is written by " q ( r) D C exp

# p ( h ¡ ae 0 ) 2 e 20 ¡ , 2(1 ¡ a) 2

(6.14)

where C is a normalization constant. Using equation 6.13, we Žnally obtain the required activity distribution. 7 Conclusion

The stochastic mechanism underlying the synchronous Žring of neurons is studied by using information geometry. When the probability distribution of the activity of neurons is concentrated on one value (the delta function), the fraction of Žring neurons is almost the same in each time. When the distribution has two peaks at r1 and r2 , the activity takes the higher value r2 where many neurons are synchronizing and the lower value r1 at other times. When the distribution is widespread and continuous, the neuron pool shows a variety of activities. It is a new Žnding that higher-order interactions are necessary for generating widespread activities, and independent, pairwise, or such a limited number of interactions cannot generate a widespread distribution. We have used a simple model of neuron pool that receives stimuli from overlapping common neurons to show that it can generate higher-order interactions and hence a widespread distribution of activity. Although the model is simple, it suggests many interesting features. One is the role of higher-order interactions in a neuron pool. This may be used to judge if there are such interactions in experimental data. We treated only symmetric and homogeneous interactions, but the essential point is the same even in unhomogeneous networks. It is also interesting to note that even when the neuronal potentials ui have no higher-order interactions, their nonlinear transforms can have them. We have not discussed dynamical aspects of synchronous Žring such as in synŽring chains, a subject for further research. References Abeles, M. (1991). Corticonics: Neural circuits of the cerebral cortex. Cambridge: Cambridge University Press. Aertsen, A., & Arndt, M. (1993). Response synchronization in the visual cortex. Current Opinion in Neurobiology, 3(4), 586–594.

142

S. Amari, H. Nakahara, S. Wu, and Y. Sakai

Amari, S. (2001).Information geometry on hierarchy of probability distributions. IEEE Transactions on Information Theory, 47(5), 1701–1711. Amari, S., & Nagaoka, H. (2000). Methods of information geometry. New York: Oxford University Press. Bohte, S. M., Spekreijse, H., & Roelfsema, P. R. (2000). The effects of pair-wise and higher-order correlations on the Žring rate of a postsynaptic neuron. Neural Computation, 12, 153–179. Gerstein, G. L., Bedenbaugh, P., & Aertsen, A. M. (1989). Neuronal assemblies. IEEE Transactions on Biomedical Engineering, 36(1), 4–14. Martignon, L., Von Hasseln, H., Grun, ¨ S., Aertsen, A., & Palm, G. (1995). Detecting higher-order interactions among the spiking events in a group of neurons. Biological Cybernetics, 73(1), 69–81. Nakahara, H., & Amari, S. (2002). Information geometric measure for neural spikes. Neural Computation, 14, 2269–2316. Singer, W., & Gray, C. M. (1995). Visual feature integration and the temporal correlation hypothesis. Annual Review of Neuroscience, 8, 555–586. Vaadia, E., Haalman, I., Abeles, M., Bergman, H., Prut, Y., Slovin, H., & Aertsen, A. (1995). Dynamics of neuronal interactions in monkey cortex in relation to behavioural events. Nature, 373, 515–518.

Received February 19, 2002; accepted July 3, 2002.

LETTER

Communicated by Bard Ermentrout

Determination of Firing Times for the Stochastic Fitzhugh-Nagumo Neuronal Model Henry C. Tuckwell [email protected] Department of Mathematics, University of California, Irvine, CA 92697, U.S.A., and Laboratory for Visual Neurocomputing, Riken Brain Science Institute, Wako-shi, Saitama 351-0198, Japan Roger Rodriguez [email protected] Centre de Physique ThÂeorique, CNRS, F13288 Marseille Cedex 9, France Frederic Y. M. Wan [email protected] Department of Mathematics, University of California, Irvine, CA 92697, U.S.A. We present for the Žrst time an analytical approach for determining the time of Žring of multicomponent nonlinear stochastic neuronal models. We apply the theory of Žrst exit times for Markov processes to the Fitzhugh-Nagumo system with a constant mean gaussian white noise input, representing stochastic excitation and inhibition. Partial differential equations are obtained for the moments of the time to Žrst spike. The observation that the recovery variable barely changes in the prespike trajectory leads to an accurate one-dimensional approximation. For the moments of the time to reach threshold, this leads to ordinary differential equations that may be easily solved. Several analytical approaches are explored that involve perturbation expansions for large and small values of the noise parameter. For ranges of the parameters appropriate for these asymptotic methods, the perturbation solutions are used to establish the validity of the one-dimensional approximation for both small and large values of the noise parameter. Additional veriŽcation is obtained with the excellent agreement between the mean and variance of the Žring time found by numerical solution of the differential equations for the one-dimensional approximation and those obtained by simulation of the solutions of the model stochastic differential equations. Such agreement extends to intermediate values of the noise parameter. For the mean time to threshold, we Žnd maxima at small noise values that constitute a form of stochastic resonance. We also investigate the dependence of the mean Žring time on the initial values of the voltage and recovery variables when the input current has zero mean. Neural Computation 15, 143–159 (2003)

° c 2002 Massachusetts Institute of Technology

144

H. Tuckwell, R. Rodriguez, and F. Wan

1 Introduction

Following the Žrst use of microelectrodes to observe the activity of individual neurons in the mammalian central nervous system, it was soon found that such activity was for the most part highly irregular and unpredictable (Burns, 1968). There have been proposed and analyzed many mathematical models whose aim is to provide quantitative theories of these phenomena (Softky & Koch, 1993; Stevens & Zador, 1998; Lee & Kim, 1999; Tuckwell, 1989, has a review of early modeling). In the past several years, there has been a vigorous renewed interest in such aspects of neuronal spiking activity, especially with regard to an understanding of information processing (Shadlen & Newsome, 1995; Konig, Engel, & Singer, 1996; Tiesinga, JosÂe, & Sejnowski, 2000). There have been many experimental studies of the spike trains emitted by cortical cells in response to natural and artiŽcial stimulation (Burns, 1968; Kreiter & Singer, 1996), but it is still uncertain which models reproduce experimental details (Softky & Koch, 1993; Plesser & Gerstner, 2000). Whereas many early studies of neuronal networks were based on the assumption that there are two possible states for each element of the network corresponding to a Žring or nonŽring nerve cell, models have since become increasingly realistic and have begun to include physiological details of neurons, including synaptic currents of various kinds. Methods of research have included phase analysis, simulation, or numerical integration of systems of ordinary differential equations, as well as direct analytical approaches to problems involving stochastic model neurons with synaptic input. We have recently reported a reductionist method for nonlinear noisy single neurons and networks in which differential equations are solved for the Žrst- and secondorder moments of the model dynamical variables (Tuckwell & Rodriguez, 1998). Although there has been considerable progress with mathematical analysis of the behaviors of model neurons, much of the research has involved simpliŽed models. In this direction, there have been advances with classical one-dimensional models, but these have mainly been for linear integrateand-Žre models such as lead to the subthreshold potential being represented as an Ornstein-Uhlenbeck process (Plesser & Tanaka, 1997; Plesser & Gerstner, 2000). There have been only a few analytical studies including noise in nonlinear models such as the Fitzhugh-Nagumo system and the HodgkinHuxley system (Rodriguez, 1995; Tanabe & Pakdaman, 2001). Although several software routines are available to implement simulation studies of model neurons with both linear and nonlinear dynamics, as, for example, in Liley, Alexander, Wright, and Aldous (1999), it is always desirable to have results obtained using analytical approaches. Using the theory of multidimensional (Markov) diffusion processes, we derive equations for the moments of random variables associated with the time to reach threshold in the Fitzhugh-Nagumo system. Although this sys-

Stochastic Neuronal Firing Times

145

tem does not have as Žrm a basis as conductance-based Hodgkin-Huxley type models, it is relatively simple and more directly amenable to analysis. We describe several methods of Žnding Žring times by solving associated ordinary or partial differential equations, including approximations suitable for small and large values of the noise parameter. Comparison will be made with results obtained by simulating the stochastic Fitzhugh-Nagumo system directly. In a subsequent article, we will conduct a similar analysis of the original Hodgkin-Huxley equations. The approach that we use can be employed for any conductance-based model. 2 The Stochastic Fitzhugh-Nagumo System

The Fitzhugh-Nagumo (FN) system of two differential equations has been employed to provide insight into the more complex Hodgkin-Huxley system of four equations. It shares with the latter the properties of subthreshold responses, action potentials or spikes in response to suitable stimuli, and repetitive activity (periodic solutions) in certain ranges of stimuli. There are associated Hopf bifurcations at the parameter values associated with the end points of these ranges. We study the following corresponding stochastic version of this model (Tuckwell, 1986, 1989; Feng & Brown, 2000) in which the components obey the following equations, dX D [ f ( X) ¡ Y C I]dt C sdW dY D b ( X ¡ c Y ) dt,

(2.1) (2.2)

where X D X ( t) is the “voltage” variable, Y D Y ( t) is the “recovery” variable, W is a standard (zero mean, variance t) Wiener process, s is a constant determining the overall input noise level, I D I ( t) is a deterministic input current (stimulus) that may be constant or time varying, and b and c are positive constants. The function f is a cubic f ( x ) D kx ( x ¡ a ) (1 ¡ x ) ,

(2.3)

where 0 < a < 1. Usually one takes a < 1 / 2 in order to obtain suitable supra-threshold responses. 2.1 Exact Equations for the Moments of the Time to a Threshold. Let T ( x, y ) be the Žrst exit time of the vector-valued process ( X, Y ) from an open set A in the phase-space for a starting point (x, y ) 2 A. Assuming that the stimulus I is constant so that the process is in fact temporally homogeneous, the theory of Markov diffusion processes (Tuckwell, 1976) yields the following partial differential equations (PDEs) for the Žrst moment

146

H. Tuckwell, R. Rodriguez, and F. Wan

F (x, y ) D E[T ( x, y )] and for the second moment G ( x, y) D E[T 2 (x, y )]: [ f ( x ) ¡y C I] [ f (x ) ¡y C I]

@F ( x, y ) @x

@G ( x, y ) @x

C b ( x¡c y)

C b ( x ¡ c y)

@F ( x, y) @y

@G ( x, y ) @y

C

s 2 @2 F (x, y ) D ¡1, 2 @x2

(2.4)

C

s 2 @2 G (x, y ) D ¡2F(x, y ) , 2 @x2

(2.5)

with suitable boundary conditions on the boundary of the set A. The set A will be chosen as an open rectangle Q containing ( x, y) . Since we are primarily interested in the time taken for the potential variable X to reach a threshold value h > 0 from the resting value X (0) D 0, the rectangle is taken to be (¡a, h ) £ ( y1 , y2 ) , where a is positive. The usual boundary conditions for Žrst exits from the open rectangle Q are that F and G vanish on its boundaries. In order to facilitate numerical solution, we modify these boundary conditions, but in such a way that accurate values are still obtained for the moments. The condition that F and G vanish along the line x D h is retained. For the parameter ranges and initial values of interest, most exits of ( X, Y ) from Q occur through the line x D h and, for the values of Y (0) considered here, away from the lines y D y1 and y D y2 . As a ! 1, the paths are less inuenced by the boundary condition at x D ¡a, with nearly all exits occurring at x D h rather than x D ¡a. Hence, one may use a reecting condition Fx (¡a, : ) D 0, where the subscript denotes partial differentiation, instead of F (¡a, : ) D 0. The zero-derivative conditions Fy ( : , y1 ) and Fy ( : , y2 ) may similarly be employed. These zero-derivative conditions facilitate numerical solution of the differential equations by avoiding sharp boundary layers at these three edges of Q and correspond to the Euler boundary conditions in a variational formulation of the problem. The time to Žrst threshold crossing or time to Žrst spike from an initial rest point (0, y 0 ) is not the same as the interspike interval in nonlinear models and has Žrst and second moments F (0, y 0 ) and G (0, y 0 ), respectively, where yo 2 ( y1 , y2 ) . 3 Methods of Solution

We have approached the solution of the moment equations in various ways. Approximate solutions can always be obtained by simulation, as described in the next section. However, accurate numerical solution of the moment equations is a useful and perhaps necessary check on simulation results, which are subject to sampling error. We have pursued the following methods of solution: (1) a simpliŽed approach, which considers the single component process X; (2) approximate solution of the PDEs 2.4 and 2.5 when s is small; and (3) the solution of the PDEs when s is large.

Stochastic Neuronal Firing Times

147

Figure 1: The value of the recovery variable is practically unchanged during the passage of X ( t) from 0 to the threshold value of h D 0.6. Here the input current is I D 1.5, and the noise parameter is s D 1. The values of the remaining parameters a D 0.1, b D 0.015, c D 0.2, k D 0.5, are the standard set throughout. 3.1 A SimpliŽed Approach. In this Žrst approach, we reduce the problem to a one-dimensional one. For the PDEs 2.4 and 2.5, this is tantamount to neglecting the Fy and Gy terms. The justiŽcation for this approach is the observation that during the initial stages of the interspike interval, the recovery variable Y (t ) is practically unchanged and may therefore be set as a constant equal to the initial value Y (0) D y0 . This is illustrated in Figure 1. We then consider the diffusion process satisfying

dX D [ f ( X) ¡ y 0 C I]dt C sdW.

(3.1)

Letting X (0) D x 2 (¡1, h ) , we consider the exit time Th ( x) , which is the time to reach the level h, identiŽed as the threshold for an action potential, given that the voltage started below this value. The Žrst and second moments, F (x ) and G ( x ) , respectively, of this exit time satisfy the differential equations s 2 d2 F ( x ) dF ( x) C [ f ( x) ¡ y0 C I] D ¡1, 2 2 dx dx

(3.2)

148

H. Tuckwell, R. Rodriguez, and F. Wan

and s 2 d2 G ( x) dG (x ) C [ f ( x ) ¡ y 0 C I] D ¡2F(x) , 2 dx2 dx

(3.3)

with the boundary conditions F (h ) D 0 and G (h ) D 0. For numerical solution, we make the interval of initial values Žnite with x 2 (¡a, h ) , where a > 0. As mentioned in the previous section, at the lower boundary, it is advantageous to choose zero-derivative conditions Fx (¡a) D 0 and Gx (¡a) D 0. The quantities of interest are the moments of the time to reach threshold from rest, F (0) and G (0). A further reduction is sometimes possible. The solutions of the secondorder ordinary differential equations 3.2 and 3.3 may be approximated when s is small by the solutions of the Žrst-order equations [ f ( x ) ¡ y 0 C I]

dF ( x ) D ¡1, dx

(3.4)

[ f ( x ) ¡ y 0 C I]

dG ( x) D ¡2F(x) . dx

(3.5)

and

These equations admit only a single boundary value, chosen as zero at x D h. A more accurate approach for small s is described in the next section. 3.2 Solution When s Is Small. When s is small, the PDEs 2.4 and 2.5 can be solved approximately using perturbation methods. Putting 2 D s 2 / 2, the equation satisŽed by the mean Žrst exit time can be written as 2 Fxx C L1 [F] D ¡1,

(3.6)

L1 [F] D P ( x, y ) Fx C Q ( x, y) Fy ,

(3.7)

with

and where P ( x, y ) D I ¡ y C kx ( x ¡ a) (1 ¡ x ) and Q ( x, y ) D b ( x ¡ c y) . The solution is obtained with F (h , y) D 0 and Fx (¡a, y) D 0. A regular perturbation expansion is sought in powers of 2 : F ( x, yI 2 ) D

1 X nD 0

Fn ( x, y) 2

n

.

(3.8)

Substitution in equation 3.6 gives Žrst-order PDEs for the coefŽcients fFn ( x, y ) g, which can be solved by the method of characteristics. For s ¿ 1, it is expected that the solution of equation 2.4 is F ( x, yI 2 ) ¼ F 0 ( x, y) C O (2 ) ,

Stochastic Neuronal Firing Times

149

where F 0 is the leading term in the regular perturbation expansion, equation 3.8. A similar development leads to G ( x, yI 2 ) ¼ G0 (x, y ) C O (2 ). For the leading terms F 0 and G 0, the relations 3.4 and 3.5 are applicable along with the characteristics of equations 2.4 and 2.5. Here, we have y D y0 ( x) , known from a separate determination of the characteristic projections for the linear PDEs for F and G. It follows that the vanishing slope conditions for F and G as x ! ¡1 are satisŽed by F 0 and G 0 . For s < 1, these leading term solutions (or the corresponding multiterm truncated perturbation series) differ by less than 5% from those obtained with the one-dimensional approach, validating the latter for such values of s. 3.3 Solution for Large s . It is convenient to rewrite equation 2.4 in diy s2 mensionless variables z D hx , l D 2h 2 , and u D h . The PDE becomes

lFzz C p ( z, u ) Fz C b ( z ¡ c u ) Fu D ¡1

(3.9)

for ¡a · z · 1, where p (z, u) D [I C kz (z ¡ a ) (1 ¡ hz) ¡ u] and where I D hI , k D kh, a D ha , a D F (1, u ) D 0 and Fz (¡a, u ) D 0. With Z

P ( z, u ) D

a h.

(3.10)

The boundary conditions are now

z

p ( w, u ) dw

µ ¶ h 4 1 1 2 3 ( ) (1 C ha) z ¡ az , D I¡u zC k ¡ z C 4 3 2

(3.11)

we multiply throughout by 1l eP/ l and write the PDE for F as [eP / l Fz ]z C

1 eP/ l [b(z ¡ c u ) eP/ l Fu ] D ¡ . l l

(3.12)

When l À 1, a regular perturbation solution is sought of the form F (z, uI l) D

1 X

Fn (z, u ) l ¡n .

(3.13)

nD 1

With the boundary conditions for the leading term given by F1 (1, u ) D 0 and F1,z (¡a, u ) D 0, the solution for F1 can be found as a double integral. The following expression is then obtained for the mean Žrst exit time: F (z, uI l) D

1 l

Z

1 z

e ¡P(g,u) / l

Z

g ¡a

eP (f ,u ) / l df dg[1 C O (l¡1 ) ].

(3.14)

150

H. Tuckwell, R. Rodriguez, and F. Wan

A similar analysis gives G ( z, uI l) D

1 l

Z

1

e ¡P(g,u) / l

Z

z

£ [1 C O (l ¡1 ) ].

g ¡a

( ) eP f ,u / l F1 (f , u ) df dg

(3.15)

In addition to yielding explicit accurate approximations for the moments of the Žrst passage time, the above leading term asymptotic solutions (or more accurate multiterm truncated series solutions) provide, for large s, validation of the one-dimensional approximation of section 3.1. The leading term of the perturbation solution for F is seen from equation 3.12 to be determined by equation 3.2. Similar remarks apply to G. 4 Results and Comparison with Simulation

We have determined the Žrst two moments of T (x, y ) by the above methods. We have additionally simulated the processes described by the evolution equations 2.1 and 2.2 using a standard strong Euler method. The use of this scheme is straightforward and is vindicated in the existing problem by virtue of the close agreement of the simulation results and those obtained using the analytical framework. To apply this method, with X ( kD t) and Y ( kD t) approximated by X k and Yk, respectively, the simulation scheme consists of the recursive relations p X kC 1 D Xk C [ f (X k ) ¡ Yk C I]D t C b D tN k Y kC 1 D Yk C b[Xk ¡ c Yk]D t, where k D 0, 1, 2, . . . and fNkg is a sequence of independent and identically distributed standard normal random variables, obtained from computer library routines. This was done principally with the set of parameter values a D 0.1, b D 0.015, c D 0.2, and k D 0.5 for various values of s and I. The threshold value h for spikes was estimated from sample paths with allowance for the discrete nature of simulation. The value of D t was pragmatically chosen between 0.00001 and .001, the value being determined in each case by requiring the minimum number of steps to threshold to be at least 20 and the maximum number several thousand. The initial value of X ( t) was set at 0 and that of Y ( t) usually set at 1.0 in the results to be reported. Other values of Y (0) were nevertheless considered for comparison, some being reported in the Žnal section. When I is less than about 1.3, the (expected) value of X ( t) rarely attains the designated threshold value of h D 0.6. For the chosen parameters, we therefore consider mainly values of I ¸ 1.3, although the value I D 0 is discussed later in the article. The values of the noise parameter s are chosen from 0.05 to 5.0. For small

Stochastic Neuronal Firing Times

151

Figure 2: Mean times to reach threshold versus noise parameter s. Parameter values are given in the text. Values are determined by simulation (circles) and by numerical solution of differential equation 2.4 (+).

values of s, the number of trials in simulations was 4500, whereas only 1500 trials were employed for larger values. Plots of the means of the times for X ( t) to reachh for various I as a function of s are shown in Figures 2 and 3. For a given value of I, results obtained by numerical solution are shown in the same Žgure for comparison. In Figure 2, the values of I are 1.3 and 1.5, whereas in Figure 3, the values of I are 2 and 3. The numerical results shown here were obtained using the simpliŽed approach from section 3.1. There is excellent agreement between the two sets of results, especially allowing for sampling error in the simulation results. The magnitudes of sampling errors ( § standard error) in the simulated means are summarized as follows, with best cases Žrst. When I D 1.3, for s D 0.05, § 0.17%; for s D 0.5, § 1.31%; for s D 0.75, §2.70%; and for s D 5, § 3.95%. When I D 1.5, for s D 0.05, § 0.13%; for s D 0.5, § 1.20%; for s D 0.75, §2.53%; and for s D 5, § 3.93%. When I D 2, for s D 5, § 0.10%; for s D 0.5, § 0.92%; for s D 0.75, §2.2%; and for s D 5, §4.50%. When I D 3, for s D 0.05, § 0.07%; for s D 0.5, § 0.66%; for s D 0.75, § 1.76%; and for s D 5, § 4.30%.

152

H. Tuckwell, R. Rodriguez, and F. Wan

Figure 3: Mean Žrst Žring time versus s when I D 2 and I D 3. Values are determined by simulation (circles) and by numerical solution of differential equation 2.4 (+).

In general, the means from simulation are slightly larger than those obtained by solution of the differential equation. It can be clearly seen in the top part of Figure 2, where I D 1.3, that as s increases away from zero, the mean Žrst passage time actually increases. After reaching a maximum when s is about equal to 0.25, the mean then declines, as is expected with increased variability. Similar behavior is observed for the larger values of the mean input current, except that the maximum of the mean interval occurs at larger values of s. This phenomenon is akin to stochastic resonance, well documented in nonlinear systems with noise, including neural systems such as that of Hodgkin-Huxley and Fitzhugh-Nagumo (Lee & Kim, 1999; Nozaki & Yamamoto, 1998; Massanes & Vicente, 1999). In Figures 4 and 5, we show the standard deviation of the time to reach threshold as determined by both simulation and solution of the differential equation 2.5 for the second moment. It can be seen that there is excellent agreement between the two sets of results for all four values of the input parameters I and s. In each of the four sets of results shown, the standard deviation attains a maximum as a function of s. The magnitudes of the sampling errors in the standard deviations in representative cases are as

Stochastic Neuronal Firing Times

153

Figure 4: Standard deviation of time to reach threshold versus noise parameter s. Parameter values are given in the text. Values are determined by simulation (circles) and by numerical solution (+) of the differential equation 2.4.

follows. When I D 1.3, for s D 0.05, § 2.0%; and for s D 5, § 2.6%. When I D 3, for s D 0.05, § 2.2%; and for s D 5, §4.8%. In Figure 6, we show the behavior of the coefŽcient of variation (CV, deŽned as standard deviation divided by the mean), calculated from computer simulations, as a function of the noise parameter s for the values I D 1.3 and I D 3. The results for I D 1.5 and I D 2 are similar to these, respectively. It is seen that the CV is greater than 1 over much of the range of parameters considered. 5 Discussion

Since interest Žrst arose in the theory of stochastic neuronal Žring over 30 years ago, analytical methods have been employed to determine properties of interspike intervals in many integrate-and-Žre model neurons in which the subthreshold membrane potential satisŽes a linear stochastic differential equation. In this regard, nonlinear models such as Fitzhugh-Nagumo and Hodgkin-Huxley have been studied mainly using computer simulations. We have used an analytical framework for the Žrst time in a nonlinear

154

H. Tuckwell, R. Rodriguez, and F. Wan

Figure 5: Standard deviation of Žrst Žring time versus s when I D 2 and I D 3. Values determined by simulation (circles) and by numerical solution of the differential equation 2.5 (+).

stochastic neural model. To this end, we have reduced the task of characterizing the Žring time to the solution of linear second-order partial differential equations. Some remarks about the nature of these equations and our methods of solution of them are in order when the noise term is small. In such singular perturbation problems, where the highest derivative term in the differential equation is multiplied by the small parameter, setting the small parameter equal to zero (or, equivalently, seeking the leading term of a regular perturbation solution in the small parameter) reduces the order of the equation and therefore leads to a loss of some of the components of the general solution of the equation. However, for problems with a certain combination of boundary conditions, such as prescribing the normal derivative equal to zero at negative inŽnity or at some large negative value of x and prescribing the value of the unknown (equal to zero in our case) at the opposite end of the interval, the missing solutions are irrelevant, as they would have been eliminated by the boundary condition at negative inŽnity anyway. That is, our less general solution, obtained after setting the small parameter s equal to zero, is made to satisfy all boundary conditions (in particular, the

Stochastic Neuronal Firing Times

155

Figure 6: CoefŽcient of variation of the time to threshold versus s for I D 1.3 and I D 3. Values are determined by simulation (circles) and by numerical solution of the differential equation 2.5 (+).

condition at negative inŽnity automatically), whereupon by a uniqueness theorem for linear PDEs, we have the correct solution of the problem without obtaining the missing solutions. A previous example of this technique is contained in our extensive investigation of the Ornstein-Uhlenbeck process representation of neuronal potential (Wan & Tuckwell, 1982). 5.1 Parameter and Initial Values: The Case I D 0. The parameters we have employed are necessarily in restricted particular ranges, and a brief consideration of other cases is warranted. The deterministic constant component of the input current I represents the drift or the mean synaptic drive, which depends on the relative magnitudes and frequencies of excitation and inhibition. The quantities we have determined using an analytical framework are the moments of the time for emergence of the Žrst spike. For values of I such as those employed in the calculations performed above, periodic Žring occurs; that is, subsequent to the Žrst spike, an approximately periodic spike train ensues. If the train is sufŽciently regular with a consistently deŽned apparent threshold, such as occurs when the noise amplitude is not excessive (see Figure 2 of Tuckwell & Rodriguez, 1998), then the methods

156

H. Tuckwell, R. Rodriguez, and F. Wan

Figure 7: Effects of changing the initial value of the voltage variable X (0) are illustrated for a Žxed value of Y (0) D 0. (Top) For the two indicated values of X (0) , the variation of the mean time to a spike as a function of noise parameter s is shown. (Bottom) Variations in mean time to a spike as the initial value X (0) changes for two Žxed values of the noise parameter.

employed here could be applied to the determination of the moments of the associated yet different random variable, namely, the interspike interval. Such a determination will be deferred to a later article. The initial value chosen above for the dynamical variables X to illustrate the usefulness of the analytical approach is X (0) D 0 because it corresponds to a natural resting or equilibrium value for the unperturbed system. The value Y (0) D 1 was based on previously published results as it was close to the values in sustained rhythmic Žring for chosen parameter values (Tuckwell & Rodriguez, 1998). However, other initial conditions are of interest and also the case where periodic Žring does not necessarily occur because the mean input current is not sufŽciently large. To study these aspects, we have determined, using simulation, the statistics of the time to Žrst spike in certain “excitable” cases when I D 0. First, we consider the effects of varying the initial value X (0) of the voltage variable for a Žxed value Y (0) D 0 of the recovery variable. Some results for the mean time to Žrst spike are shown in Figure 7. Comparison of the results

Stochastic Neuronal Firing Times

157

Figure 8: Effects of changing the initial value of the recovery variable Y (0) are illustrated for a Žxed value of X (0) D 0. (Top) For the indicated values of Y (0) , the variation of the mean time to a spike as a function of noise parameter s is shown. (Bottom) Variations in mean time to a spike as the initial value Y (0) changes for the two indicated values of the noise parameter.

in the top part of this Žgure with those in Figures 2 and 3 indicates that the variation in this mean as the noise parameter varies from 0.5 to 5.0 is similar for the three different values of X (0) even despite the different values of I and the fact that Figures 2 and 3 correspond to the periodic Žring mode with a different value for Y (0) . That the mean time to a spike is moderately sensitive to changes in X (0) for Žxed values of all the parameters and Y (0) can be seen in the lower part of Figure 7. Similar remarks apply to the effects of changing Y (0) , as can be gleaned from inspection of Figure 8. Noticeable here, however, is that when s D 4, the mean time to a spike increases only gradually as Y (0) increases from ¡0.4 to 1.0. In summary, we have investigated for the Žrst time the determination of times for a threshold crossing in a nonlinear stochastic neuronal model, that of Fitzhugh-Nagumo, with the addition of a white gaussian noise representing stochastic excitation and inhibition. We have found moments of threshold crossing times accurately by solving the associated partial differ-

158

H. Tuckwell, R. Rodriguez, and F. Wan

ential equations, obtained from the theory of Markov processes. The validity of the methods employed has been veriŽed by comparison with computer simulation results. An interesting structure in the mean Žrst passage time was noted at small values of the noise parameter. In future work, we will report our Žndings for the more complicated Hodgkin-Huxley system. Acknowledgments

H. C. T. appreciates support from the Riken Brain Science Institute, Saitama, Japan. References Burns, B. D. (1968). The uncertain nervous system. London: Arnold. Feng, J-F., & Brown, D. (2000). Integrate-and-Žre models with nonlinear leakage. Bulletin of Mathematical Biology, 62, 467–481. Konig, P., Engel, A. K., & Singer, W. (1996). Integrator or coincidence detector? The role of the cortical neuron revisited. Trends in Neurosciences, 19, 130–137. Kreiter, A. K., & Singer, W. (1996). Stimulus-dependent synchronization of neuronal responses in the visual cortex of the awake macaque monkey. Journal of Neuroscience, 16, 2381–2396. Lee, S-G., & Kim, S. (1999). Parameter dependence of stochastic resonance in the stochastic Hodgkin-Huxley neuron. Phys. Rev. E, 60, 826–830. Liley, D. T. J., Alexander, D. M., Wright, J. J., & Aldous, M. D. (1999). Alpha rhythm emerges from large-scale networks of realistically coupled multicompartmental model cortical neurons. Network: Comput. Neural Syst., 10, 79–92. Massanes, S. R., & Vicente, C. J. P. (1999). Nonadiabatic resonances in a noisy Fitzhugh-Nagumo neuron model. Phys. Rev. E, 59, 4490–4497. Nozaki, D., & Yamamoto, Y. (1998). Enhancement of stochastic resonance in a Fitzhugh-Nagumo neuronal model driven by colored noise. Phys. Lett. A, 243, 281–287. Plesser, H. E., & Gerstner, W. (2000). Noise in integrate-and-Žre neurons: From stochastic inputs to escape rates. Neural Computation, 12, 367–384. Plesser, H. E., & Tanaka, S. (1997). Stochastic resonance in a model neuron with reset. Physics Letters A, 225, 228–234. Rodriguez, R. (1995). Coupled Hodgkin-Huxley neurons with stochastic synaptic inputs. In J. Bertrand (Ed.), Modern group theoretical methods in physics (pp. 233–242). Norwood, MA: Kluwer. Shadlen, M. N., & Newsome, W. T. (1995). Noise, neural codes and cortical organization. Curr. Opin. Neurobiol., 4, 569–579. Softky, W. R., & Koch, C. (1993). The highly irregular Žring of cortical cells is inconsistent with temporal integration of random EPSPs. J. Neurosci., 13, 334–350. Stevens, C. F., & Zador, A. M. (1998). Input synchrony and the irregular Žring of cortical neurons. Nature Neuroscience, 1, 210–217.

Stochastic Neuronal Firing Times

159

Tanabe, S., & Pakdaman, K. (2001). Dynamics of moments of Fitzhugh-Nagumo neuronal models and stochastic bifurcations. Phys. Rev. E, 63, 031911. Tiesinga, P. H. E., JosÂe, J. V., & Sejnowski, T. J. (2000). Comparison of currentdriven and conductance-driven neocortical model neurons with HodgkinHuxley voltage-gated channels. Phys. Rev. E, 62, 8413–8419. Tuckwell, H. C. (1976). On the Žrst exit time problem for temporally homogeneous Markov processes. J. Appl. Prob., 13, 39–48. Tuckwell, H. C. (1986). Stochastic equations for nerve membrane potential. J. Theoret. Neurobiol., 5, 87–99. Tuckwell, H. C. (1989). Stochastic processes in the neurosciences. Philadelphia: SIAM. Tuckwell, H. C., & Rodriguez, R. (1998). Analytical and simulation results for stochastic Fitzhugh-Nagumo neurons and neural networks. J. Computational Neuroscience, 5, 91–113. Wan, F. Y. M., & Tuckwell, H. C. (1982). Neuronal Žring and input variability. J. Theoret. Neurobiol., 1, 197–218. Received August 24, 2001; accepted July 3, 2002.

LETTER

Communicated by Richard Zemel

Developmental Constraints Aid the Acquisition of Binocular Disparity Sensitivities Melissa Dominguez [email protected] Department of Computer Science, Universityof Rochester,Rochester,NY 14627,U.S.A. Robert A. Jacobs [email protected] Department of Brain and Cognitive Sciences, University of Rochester, Rochester, NY 14627, U.S.A. This article considers the hypothesis that systems learning aspects of visual perception may beneŽt from the use of suitably designed developmental progressions during training. We report the results of simulations in which four models were trained to detect binocular disparities in pairs of visual images. Three of the models were developmental models in the sense that the nature of their visual input changed during the course of training. These models received a relatively impoverished visual input early in training, and the quality of this input improved as training progressed. One model used a coarse-scale-to-multiscale developmental progression, another used a Žne-scale-to-multiscale progression, and the third used a random progression. The Žnal model was nondevelopmental in the sense that the nature of its input remained the same throughout the training period. The simulation results show that the two developmental models whose progressions were organized by spatial frequency content consistently outperformed the nondevelopmental and random developmental models. We speculate that the superior performance of these two models is due to two important features of their developmental progressions: (1) these models were exposed to visual inputs at a single scale early in training, and (2) the spatial scale of their inputs progressed in an orderly fashion from one scale to a neighboring scale during training. Simulation results consistent with these speculations are presented. We conclude that suitably designed developmental sequences can be useful to systems learning to detect binocular disparities. The idea that visual development can aid visual learning is a viable hypothesis in need of study. 1 Introduction

Human infants are born with limited perceptual, motor, and cognitive abilities relative to adults. Within the Želd of developmental psychology, there Neural Computation 15, 161–182 (2003)

° c 2002 Massachusetts Institute of Technology

162

M. Dominguez and R. Jacobs

are at least two perspectives regarding these limitations. The older and more commonplace view is that these limitations are barriers that must be overcome in order for a child to achieve adult function (Piaget, 1952). That is, they are deŽciencies or immaturities that serve no positive purpose. A newer view is that these apparent inadequacies are in fact helpful, perhaps necessary, stages in development. Limited mental abilities, according to this theory, reect simple neural representations that are useful stepping-stones or building blocks for the subsequent development of more complex representations (Turkewitz & Kenney, 1982). The idea that early developmental stages are useful or necessary precursors to more advanced stages is becoming increasingly studied in the cognitive neurosciences. The development of biological nervous systems is sometimes characterized as using a bootstrapping strategy. Greenough, Black, and Wallace (1987) speculated that asynchrony in brain development serves the useful function of stage setting. The developmental schedule for the maturation of different brain regions is staggered such that neural systems that develop relatively early provide a suitable framework for the development of later experience-sensitive systems. Harwerth, Smith, Duncan, Crawford, and von Noorden (1986) provided experimental evidence consistent with this hypothesis by performing behavioral studies of sensitive periods for visual development in monkeys. Their results suggest that these sensitive periods are organized into a hierarchy in which early visual functions requiring information processing in the earliest stages of the visual system have shorter sensitive periods than higher-level functions requiring more central neural processing. A second example is provided by the work of Shatz (1996). Within the lateral geniculate nucleus (LGN) of adult mammals, retinal ganglion cell axons from one eye are segregated from those arising from the other eye to form a series of alternating eye-speciŽc layers. These layers are not present initially in development. Moreover, they form during a period in which vision is not possible. Shatz argued that the development of eye-speciŽc layers is characterized by at least two important events. First, retinal ganglion cells spontaneously show waves of activity that sweep across the retina such that activity at nearby cells is more highly correlated than at distant cells. Second, LGN cells use a Hebb-style adaptation mechanism that sorts connections based on local correlations of activity in order to form eye-speciŽc layers. If so, then this is a clear example in which a developmental event at an earlier visual region (spontaneous waves of activity at the retina) sets the stage for an event at a later visual region (Hebb-style adaptation at the LGN). Newport (1990) hypothesized that children use a bootstrapping strategy when attempting to learn a language. Human languages are componential systems in which small linguistic components are systematically combined to form larger linguistic structures. According to Newport’s “less is more” hypothesis, the limited attentional and memorial abilities of children are useful when learning a language because they help children segment and

Acquisition of Binocular Developmental Sensitivities

163

identify the small components that comprise the language. Elman (1993) studied an implementation of this general idea and showed that a recurrent neural network whose memory capacity was initially limited but then gradually increased during the course of training learned aspects of an artiŽcial grammar better than a network whose memory capacity was never limited; that is, the second network’s memory capacity was always equal to that of the Žrst network at the end of training. Elman claimed that this outcome supports the idea that starting small is important to the subsequent acquisition of complex mental abilities. Rohde and Plaut (1999), however, were unable to replicate Elman’s simulation results, so it is difŽcult to know how to interpret these results. This article considers the hypothesis that systems learning aspects of visual perception may beneŽt from the use of suitably designed developmental progressions during training. We report the results of simulations in which four systems were trained to detect binocular disparities in pairs of visual images. Three of the systems were developmental models in the sense that the nature of their input changed during the course of training. These systems received a relatively impoverished visual input early in training, and the quality of this input improved as training progressed. The fourth system was a nondevelopmental model; the nature of its input remained constant during the course of training. The inputs to the systems were left and right retinal images Žltered with binocular energy Žlters tuned to various spatial frequencies. The training of the Žrst system, referred to as the coarse-scale-to-multiscale model (model C2M), included a developmental sequence such that the system was exposed only to lowspatial-frequency information at the start of training, and information at higher spatial frequencies was added to its input as training progressed. The training of the second system, referred to as the Žne-scale-to-multiscale model (model F2M), included an analogous developmental sequence with spatial frequency information added in the reverse order. This system received high-spatial-frequency information at the start of training, and information at lower spatial frequencies was added as training progressed. The third system, referred to as the random-developmental model (model RD), was similar to models C2M and F2M in the sense that its training included a developmental sequence. However, whereas the inputs received by models C2M and F2M at each developmental stage were organized by spatial frequency content, the inputs received by model RD at each stage were randomly selected. Finally, the fourth system, referred to as the nondevelopmental model (model ND), was not trained using a developmental sequence; it received information at all spatial frequencies throughout the training period. When comparing the three developmental models with the nondevelopmental model, there are at least two reasonable predictions that one could make about the simulation results. One prediction is that the nondevelopmental model should outperform the developmental models. The

164

M. Dominguez and R. Jacobs

nondevelopmental model received all input information throughout all stages of training, whereas the developmental models were deprived of portions of the input at certain training stages. If more information is better than less information, then the nondevelopmental model ought to perform best. This would be consistent with the traditional view of human infant development—that perceptual immaturities are barriers to be overcome. An alternative prediction, consistent with the general approach of the less-is-more hypothesis, is that the developmental models would show the better performance. If it is believed that too much information could lead a learning system in its early stages of training to form poor internal representations, then the developmental models ought to have an advantage. The nondevelopmental model had a greater number of inputs than the developmental models during the early stages of training and, thus, a greater number of modiŽable weights. Because learning in neural networks is a search in weight space, the nondevelopmental model needed to perform an unconstrained search in a high-dimensional weight space. Unconstrained searches frequently lead to the acquisition of poor representations. In contrast, the developmental models initially had fewer inputs and thus fewer modiŽable weights. During early stages of training, their searches in weight space were comparatively constrained. If the constraints were appropriate, this should have facilitated the acquisition of useful representations by the developmental models. When comparing the performances of the two developmental models whose stages are based on spatial frequency content (models C2M and F2M), there are at least three reasonable predictions one could make about the simulation results. One prediction is that model C2M should perform best. A motivation for this prediction comes from the Želd of computer vision. Consider the task of aligning two images of a scene where the images differ due to a small horizontal offset in their viewpoints. Roughly, this is known as the stereo correspondence problem. If this were something that you had never done before, you might try to align Žne details of each image. For example, you might pick a white dot in one image and repeatedly try aligning it with white dots in the other image. If the images are highly textured, such an approach would be inefŽcient because there is an intractable number of potential alignments that might need to be checked before the two images are properly aligned (see Figure 1A). If, however, the images are blurred, the Žne details that were the source of so much confusion would be removed, leaving a smaller number of larger features in each image. The problem of Žnding a good alignment is now signiŽcantly easier because there is a smaller number of potential alignments (see Figure 1B). After you have gained skill at aligning blurred images, you would then have a reliable foundation that you could use to learn to align clearer images. You might attempt to match the larger image features Žrst. Subsequent analysis of Žne details would seek to remove ambiguities that arise when aligning large features instead of being a starting point for locating correspondences.

Acquisition of Binocular Developmental Sensitivities

165

A

B Figure 1: (A) One image from a stereo pair depicting a textured object and background. Due to the Žne details in the stereo pair, solving the stereo correspondence problem is relatively difŽcult. (B) The image from A blurred so as to remove the Žne details. It is easier to solve the stereo correspondence problem when the stereo pair is blurred because image features tend to be larger, less numerous, and more robust to noise.

This style of processing is commonplace in the computer vision literature. Systems by Marr and Poggio (1979), Quam (1986), and Barnard (1987), among many others, initially search for correspondences within a pair of low-resolution images. Low-resolution images are used initially because these images contain fewer image features, larger image features, and image features that are relatively robust to noise. Next, these systems reŽne their estimates of corresponding image features using information from one or more higher-resolution pairs of images. The usefulness of a coarse-to-Žne processing strategy when searching for stereo correspondences suggests that a coarse-scale-to-multiscale developmental strategy might be useful for learning to detect binocular disparities. Before adopting this hypothesis, however, it is important to keep in mind the differences between these two strategies. Computer vision researchers use a coarse-to-Žne strategy when searching for correspondences in individual pairs of images. In contrast, we used the coarse-scale-to-multiscale developmental sequence while training a learning system to detect binocular disparities using many pairs

166

M. Dominguez and R. Jacobs

of images. It is not obvious that lessons from one situation can be applied to the other situation. If we assume that human intelligence provides a guide to creating machine intelligence, then a second motivation for the prediction that model C2M should perform best is the fact that human infants show a related developmental progression. Visual acuity is often measured using a grating, which is a visual pattern whose luminance values are sinusoidally modulated. Acuity is characterized by the highest-frequency grating, which is distinguishable from a solid gray pattern. Using this method, it has been found that newborns’ visual acuity is extremely poor. Whereas adults with normal vision (so-called 20 / 20 vision) can discriminate approximately 30 cycles per degree of arc, newborns can discriminate only 1 to 2 cycles per degree, giving them a visual acuity of about 20/ 400. Acuity improves approximately linearly from these low levels at birth to near adult levels by around eight months of age (Norcia & Tyler, 1985). Importantly for our purposes, infants are acquiring other visual abilities during this time period; in particular, sensitivity to binocular disparities appears at around four months of age (Atkinson & Braddick, 1976; Fox, Aslin, Shea, & Dumais, 1980; Held, Birch, & Gwiazda, 1980; Petrig, Julesz, Krop, Baumgartner, & Anliker, 1981). We speculate that the developments of visual acuity and binocular disparity sensitivity may be related in the sense that poor acuity at an early age aids in the acquisition of disparity sensitivity later in life. An alternative prediction is that model F2M should perform better than model C2M. A motivation for this prediction is the fact that computer vision researchers often Žnd it easier to solve the stereo correspondence problem by Žrst extracting edge information from left and right images, a form of highfrequency bandpass Žltering, and then searching for a good alignment of the images based on this information. This strategy is useful because edges can be sparse, large, and robust to noise relative to other image features. Analogous to computer vision systems’ initial use of high-frequency information, model F2M is initially trained solely with information extracted from highfrequency bandpass Žlters. If we are willing to assume that computer vision methods for Žnding stereo correspondences may provide lessons for how learning systems can learn to detect binocular disparities, then we might predict that model F2M has an advantage. A second motivation for the prediction that model F2M should perform best is the seemingly counterintuitive result that neural networks trained with input patterns that have been corrupted by noise frequently show better generalization than equivalent networks trained with input patterns that have not been corrupted (Sietsma & Dow, 1991). Training with noisy inputs has been shown analytically to be equivalent to a form of regularization (Bishop, 1995; Matsuoka, 1992; Webb, 1994). In other words, the learning process of networks trained with noisy inputs is more constrained than that of similar networks whose inputs are not corrupted by noise. If the retinal images contain noise and if model C2M tends to Žlter out the noise during

Acquisition of Binocular Developmental Sensitivities

167

early stages of training, then this model will not obtain the beneŽts of training with noisy inputs. We would therefore expect model F2M to perform best. Another possible prediction is that models C2M and F2M perform about equally well. If the relative advantages of model C2M (being able to use fewer image features, larger image features, and more noise-resistant image features at the start of training) and the relative advantages of model F2M (initial exposure to high-frequency bandpass information, being able to obtain the beneŽts of training with noisy inputs) are roughly balanced, then neither model would be expected to show superior performance. Furthermore, we outlined the logic of computer vision researchers who advocate a coarse-to-Žne processing strategy when analyzing stereo correspondences, and we tentatively speculated that the usefulness of this strategy suggests that the seemingly related coarse-scale-to-multiscale developmental strategy might be useful for learning to detect binocular disparities. If, however, we again make the assumption that human intelligence provides a guide to creating machine intelligence, then it is worth noting that humans often do not use a coarse-to-Žne strategy when analyzing stereo correspondences. Mallot, Gillner, and Arndt (1996) found that unambiguous information at a coarse scale is not always used by observers to disambiguate Žner-scale information and that observers can use unambiguous Žne-scale information to disambiguate coarse-scale information, meaning that observers are using a Žne-to-coarse processing strategy in these circumstances. Related Žndings have been reported by several other researchers (McKee & Mitchison, 1988; Mowforth, Mayhew, & Frisby, 1981; Smallman, 1995). Because human observers do not use a coarse-to-Žne strategy or a Žne-to-coarse strategy exclusively, we might not expect the exclusive use of a C2M developmental strategy or the exclusive use of a F2M strategy to yield a relative performance advantage. When comparing the performances of models C2M and F2M with that of model RD, the only reasonable prediction seems to be that model RD should perform no better than the other developmental models, and it is likely that it will perform worse. We have included model RD in this article in order to demonstrate that the use of a developmental sequence does not necessarily lead to performance advantages. Instead, the beneŽts of a developmental sequence are found only when the sequence incorporates constraints that are useful for learning the desired behavioral task. This article reports the results of computer simulations comparing the learning performances of the developmental and nondevelopmental models on the task of estimating binocular disparities in novel pairs of images. The results show that the developmental models whose stages are based on spatial frequency content (models C2M and F2M) consistently outperformed the nondevelopmental and random developmental models. We speculate that the superior performance of these models is due to two important features of their developmental progressions: (1) these

168

M. Dominguez and R. Jacobs

models were exposed to visual inputs at a single scale early in training, and (2) the spatial scale of their inputs progressed in an orderly fashion from one scale to a neighboring scale. Simulation results consistent with these speculations are presented. We conclude that suitably designed developmental sequences can be useful to systems learning to detect binocular disparities and that the general idea that visual development can aid visual learning is a viable hypothesis in need of future study. 2 Developmental and Nondevelopmental Models

Figure 2 illustrates the structure of the developmental and nondevelopmental models. This structure is based on a similar architecture studied by Gray, Pouget, Zemel, Nowlan, & Sejnowski (1998). The retinal layer consisted of two one-dimensional arrays 62 pixels in length for the left and right eye images. Each retina was treated as if it were shaped like a circle; the leftmost and right-most pixels were regarded as neighbors. This wraparound of the left and right edges was done to avoid edge artifacts. Although onedimensional retinas are a simpliŽcation, their use is justiŽed by the fact that the models were concerned only with horizontal disparities, as these are the ones that provide information about the three-dimensional conŽguration of the visual environment.1 The retinal inputs were Žltered using binocular energy Žlters. Based on neurophysiological studies, Ohzawa, DeAngelis, and Freeman (1990) proposed binocular energy Žlters as a way of modeling the binocular sensitivities of simple and complex cells in primary visual cortex. These Žlters are an extension of motion energy Žlters proposed by Adelson and Bergen (1985). A simple cell receives input from a pair of subunits, one for each retina. The receptive Želd proŽles of the subunits can be described mathematically as Gabor functions: ³ ´ x2 exp ¡ 2 sin(2p vx C w ) 2s 2p s ³ ´ 1 x2 exp ¡ 2 sin(2p vx C w C dw ) . gR ( x, w ) D p 2s 2p s gL ( x, w ) D p

1

(2.1) (2.2)

Each function is a sinusoid multiplied by a gaussian envelope. The parameter x is the distance to the center of the gaussian, s 2 is the variance of the 1

Vertical disparities provide information about viewing distance and angle of gaze but by themselves do not carry information about the three-dimensional structure of the visual environment. That is, by themselves, they cannot be used for making relative or absolute depth judgments. In contrast, horizontal disparities alone can be used for making relative depth judgments and, when scaled by viewing distance information (perhaps obtained via vertical disparities), for making absolute depth judgments.

Acquisition of Binocular Developmental Sensitivities

169

Output Layer Hidden Layer Input Layer

Binocular Energy Filters

Low SF

Med SF

Stage 1 Stage 2 Stage 3

High SF

Left Eye Retina

Right Eye Retina

Figure 2: The developmental and nondevelopmental models shared a common structure. The bottom portion of this structure is the left and right retinal images. These images are then Žltered by binocular energy Žlters. The outputs of these Žlters are the inputs to an artiŽcial neural network that is trained to estimate the disparity present in the images. The model illustrated here is model C2M in which low-spatial-frequency information was received during early stages of training, and information at higher frequencies was added as training progressed.

gaussian, v is the frequency of the sinusoid, and w and dw are referred to as the base phase and phase offset of the sinusoid, respectively. The Gabor functions associated with the left and right retinal subunits differ in that the phase of one is offset from the phase of the other. The output of a simple cell is formed in two stages. First, the convolution of the left retinal image with the left subunit Gabor is added to the convolution of the right retinal image with the right subunit Gabor; next, this sum is half-wave rectiŽed and squared (a negative sum is mapped to zero; a positive sum is mapped to its square). The magnitude of a simple cell’s output is related to the presence of a binocular disparity of a particular size in the retinal input. Simple cells formed from subunits with different phase offsets are sensitive to disparities of different sizes (Fleet, Wagner, & Heeger, 1996; Qian, 1994). The output of a complex cell is the sum of the outputs of four simple

170

M. Dominguez and R. Jacobs

cells. Because the base phases of these simple cells form quadrature pairs (the base phases are 0, p / 2, p , and 3p / 2), the complex cell’s output is relatively insensitive to the exact position of a disparity within its receptive Želd. In our simulations, there were 35 receptive Želd locations that received input from overlapping regions of the retina. At each location, there were 30 complex cells corresponding to three spatial frequencies and 10 phase offsets at each frequency. The three spatial frequencies were each separated by an octave: 0.25, 0.125, and 0.0625 cycles per pixel. The standard deviations of the Gabor functions were set to be inversely proportional to the frequency: 1.0 for 0.25 cycles per pixel, 2.0 for 0.125 cycles per pixel, and 4.0 for 0.0625 cycles per pixel. The 10 phase offsets were equally spaced over a range from 0 to p / 2. The outputs of the complex cells were normalized using a softmax nonlinearity, eEi ( x ) / t EO i (x ) D P E ( x ) / t , j je

(2.3)

where E i (x ) was the initial output of the complex cell, EO i ( x ) was the normalized output, t is a scaling parameter known as a temperature parameter (its value was set to 0.25), and j indexed the 10 complex cells with different phase offsets at a receptive Želd location within a single frequency band. As a result of this normalization, complex cells tended to respond to relative contrast in an image rather than absolute contrast. The normalized outputs of the complex cells were the inputs to an artiŽcial neural network. The network had 1050 input units (the complex cells had 35 receptive Želd locations, and there were 30 cells at each location). The hidden layer of the network contained 32 units organized into 8 groups of 4 units each. The connectivity to the hidden units was set so that each group had a limited receptive Želd; a group of hidden units received inputs from seven receptive Želd locations at the complex cell level. The hidden units used a logistic activation function. The output layer consisted of a single linear unit; this unit’s output was an estimate of the disparity present in the right and left retinal images. The weights of an artiŽcial neural network were initialized to small random values and were adjusted during the course of training to minimize a sum of squared error cost function using a conjugate gradient optimization procedure (Press, Teukolsky, Vetterling, & Flannery, 1992). This procedure was used because it tends to converge quickly and because it has no free parameters (e.g., no learning rate or momentum parameters). Weight sharing was implemented at the hidden unit level so that corresponding units within each group of hidden units had the same incoming and outgoing weight values and a hidden unit had the same set of weight values from each receptive Želd location at the complex unit level. This provided the

Acquisition of Binocular Developmental Sensitivities

171

network with a degree of translation invariance and also dramatically decreased the number of modiŽable weight values in the network. It therefore decreased the number of data items needed to train the network and the amount of time needed to train the network. Models were trained and tested using separate sets of training and test data items. Training sets contained 250 randomly generated data items; test sets contained 122 data items that were generated so as to cover the range of possible binocular disparities uniformly. Training was terminated after 35 iterations through the training set in order to minimize overŽtting of the training data. The results reported are based on the data items from the test set. Model C2M was trained using a coarse-scale-to-multiscale developmental sequence. This was implemented as follows. The training period was divided into three stages where the Žrst and second stages were each 10 iterations and the third stage was 15 iterations.2 During the Žrst stage, the neural network portion of the model received only the outputs of complex cells tuned to low spatial frequencies (the outputs of the other complex cells were set to zero). During the second stage, the network received the outputs of complex cells tuned to low and medium spatial frequencies; it received the outputs of all complex cells during the third stage. The training of model F2M was identical to that of model C2M except that its training used a Žne-scale-to-multiscale developmental sequence. During the Žrst stage of training, its network received the outputs of complex cells tuned to high spatial frequencies. This network received the outputs of complex cells tuned to high and medium frequencies during the second stage and received the outputs of all complex cells during the third stage. The training of model RD also used a developmental sequence, though this sequence was generated randomly and thus was not based on the spatial frequency tuning of the complex cells. The collection of complex cells was randomly partitioned into three equal-sized subsets with the constraint that each subset included all phase offsets at all receptive Želd locations. During the Žrst stage of training, the neural network portion of the model received the outputs of only the complex cells in the Žrst subset. It received the outputs of the cells in the Žrst and second subsets during the second stage of training and the outputs of all complex cells during the third stage. In contrast, the training period for the nondevelopmental model was not divided into separate stages; its neural network received the outputs of all complex cells throughout the training period.

2 The number of iterations in the training stages was roughly optimized using models C2M and F2M and the solid object data set described below. Models with stages of different sizes were tested, and the results suggest that models tended to show worse generalization performance if the number of iterations in the Žrst and second stages was decreased or increased.

172

M. Dominguez and R. Jacobs

3 Data Sets and Simulation Results

The performances of the four models were evaluated on three data sets. These data sets were based on related data sets used by Gray et al. (1998). In all cases the images were gray scale with luminance values between 0 and 1 and disparities with values between 0 and 3 pixels. Ten simulations of each model on each data set were conducted. In the solid object data set, images consisted of a single light or dark object on a gray background. The object’s gray-scale value was either between 0.0 and 0.1 or between 0.9 and 1.0, whereas the gray-scale value of the background was always 0.5. The location of the object was randomly chosen to be a real-valued location on the retina. The object’s disparity was randomly chosen to be a real value between 0 and 3 pixels. The object’s size was randomly chosen to be a real value between 10 and 25 pixels. Since the object’s size, location, and disparity were all real numbers, the ends of the object could fall at a real-valued location within a pixel. In these (common) cases, the value of the partially covered pixel was interpolated between the gray-scale value of the object and that of the background in proportion to the amount of the pixel covered by the object and background. An example of a right and left image is shown in the top panel of Figure 3. Given the right and left images, the task of a model was to estimate the object’s disparity. The upper left graph of Figure 4 illustrates the results. The horizontal axis gives the model, and the vertical axis gives the root mean squared error (RMSE) at the end of training on the data items from the test set (the error bars give the standard error of the mean). On average, developmental model C2M had a 16.5% smaller generalization error than the nondevelopmental model (the difference between the mean error rates is statistically signiŽcant; t D 3.77, p < 0.002 using a two-tailed t-test), and a 19.3% smaller error than the random developmental model (t D 4.60, p < 0.001). Developmental model F2M had a 12.2% smaller error than the nondevelopmental model (t D 23.74, p < 0.001) and a 15.2% smaller error than the random develop-

Figure 3: Examples of right and left images (top and bottom rows in each panel) from the solid object, noisy object, and planar data sets.

Acquisition of Binocular Developmental Sensitivities

1.2

solid object data set

1.0

1.2

173

noisy object data set

1.0

RMSE 0.8

0.6

0.8

C2M F2M

RD

1.2

0.6

ND

C2M F2M

RD

ND

planar data set

1.0 RMSE 0.8

0.6

C2M F2M

RD

ND

Figure 4: The four models’ RMSE on the test set data items after training on the three data sets (the error bars give the standard error of the mean).

mental model (t D 49.01, p < 0.001). Clearly, the two developmental models whose developmental progressions were organized by spatial frequency content outperformed both the nondevelopmental model and the random developmental model. A statistical comparison between models C2M and F2M shows that their performances were not signiŽcantly different. The variability in model C2M’s performance was notably greater than the variability in the performances of the other models. Although it is difŽcult to know why, it may have to do with the properties of binocular energy units. As discussed by Qian (1994), Zhu and Qian (1996), and Fleet et al. (1996), the responses of these units may peak even when the input disparity is outside the range of disparities to which the unit was thought to be responsive. In such cases, the large response is known as a false peak. False peaks occur for nearly all stimuli, including white noise. Moreover, some of the false peaks will be signiŽcantly larger than the peak at the disparity to which the unit

174

M. Dominguez and R. Jacobs

was thought to be selective. False peaks tend to occur at integer multiples of the wavelength to which the unit is tuned, meaning that units tuned to low frequencies tend to have false peaks that are relatively far from each other, and units tuned to high frequencies tend to have false peaks near to each other. Consequently, units tuned to low spatial frequencies tend to have false peaks at disparities that are signiŽcantly different from the disparity to which they were thought to be responsive, whereas units tuned to high frequencies tend to have false peaks at disparities that are closer to the disparity to which they were thought to be responsive (see Figure 5). The large variability in model C2M’s performance may be due to the fact that this model is sometimes misled early in training by false peaks at disparities that are far from the disparities to which units are thought to be selective. This is less likely to be as serious a problem for the other models because they are less likely than model C2M to emphasize information provided by units tuned to low spatial frequencies. The learning curves for the four models on the solid object data set are illustrated in the upper left graph of Figure 6. The horizontal axis gives the training time in epochs, and the vertical axis gives the RMSE on the test set data items. The solid black line is for model C2M, the solid gray line is for model F2M, the dashed black line is for model RD, and the dotted black line is for model ND. The learning curve for model ND falls quickly within the Žrst few epochs of training and then is relatively at for the remainder of the training period. In contrast, the developmental models learned relatively slowly, and models C2M and F2M eventually showed the best generalization performance. This result is consistent with the notion described above that apparent inadequacies in performance during early development are not necessarily bad; they sometimes suggest the use of internal representations, which are useful stepping-stones for the subsequent development of advanced behaviors. The images in the second data set, referred to as the noisy object data set, were meant to resemble random dot stereograms frequently used in behavioral experiments. Images contained a noisy object against a noisy background. The gray-scale values of the object pixels and the background pixels were set to random numbers between 0 and 1. The location of the object was randomly chosen to be a real-valued location on the retina. The object’s size was randomly chosen to be a real value between 10 and 25 pixels. The object’s disparity was a randomly chosen integer between 0 and 3 pixels. As before, the task was to map the left and right images to an estimate of the object’s disparity. An example of a left and right image is shown in the middle panel of Figure 3. The results are shown in the upper right panel of Figure 4. As before, the developmental models whose developmental progressions were organized by spatial frequency content performed best. On average, model C2M had a 7.1% smaller generalization error than model ND (t D 448.8, p < 0.001) and a 6.6% smaller error than model RD (t D 2.46, p < 0.05). Model F2M had a

Acquisition of Binocular Developmental Sensitivities

175

Low Spatial Frequency Left Eye

Right Eye

d d+l

High Spatial Frequency Left Eye

Right Eye

d d+l

Figure 5: The top two Žgures illustrate Gabor Žlters for the left-eye and righteye images that are tuned to a low spatial frequency (l is the wavelength). The right-eye Gabor is phase-shifted by d units relative to the left-eye Gabor. The sum of the Žlter responses tends to be large when the disparity in the images is d, d ¡ l, or d C l. Because the wavelength l is relatively large, the sum can peak at input disparities (e.g., d C l) that are far from the disparity (d) to which the sum was thought to be responsive. As illustrated in the bottom two Žgures, a similar situation holds for Žlters tuned to high spatial frequencies, but now the wavelength l is smaller and the false peaks occur at input disparities that are signiŽcantly closer to the disparity to which the sum was thought to be responsive. Consequently, false peaks are more misleading when using Žlters tuned to low frequencies than when using Žlters tuned to high frequencies.

176

M. Dominguez and R. Jacobs

solid object data set

2.2

C2M F2M RD ND

1.8

1.8

RMSE 1.4

1.4

1.0

1.0

0.6

noisy object data set

2.2

0.6 0

10 20 30 training time (epochs)

0

10 20 30 training time (epochs)

planar data set

2.2 1.8 RMSE 1.4 1.0 0.6 0

10 20 30 training time (epochs)

Figure 6: Learning curves for the four models on the three data sets.

4.3% smaller error than model ND (t D 1566.5, p < 0.001) and a 3.8% smaller error than model RD (this result is not statistically signiŽcant). Comparing the developmental models C2M and F2M, C2M had a 2.85% smaller error (t D 170.6, p < 0.001). The learning curves for the four models are shown in the upper right graph of Figure 6. A notable feature illustrated by this graph is that the random developmental model had highly unstable levels of performance during the course of training. The last data set, the planar data set, was different from the Žrst two data sets. Instead of an object in front of a background, the images depicted a frontoparallel plane. The values of the left-image pixels were randomly chosen to be either 0 or 1. The right image was generated by applying an integer shift to the left image of 0, 1, 2, or 3 pixels. Given the left and right images, the task was to estimate the size of the shift. An example of a left and right image is shown in the bottom panel of Figure 3.

Acquisition of Binocular Developmental Sensitivities

177

The results are shown in the bottom graph of Figure 4. Again, the developmental models whose developmental progressions were organized by spatial frequency content tended to outperform the other models. Model C2M had a 6.7% smaller generalization error than model ND (t D 2.27, p < 0.05) and a 1.3% smaller error than model RD (this result is not statistically signiŽcant). Model F2M had an 8.3% smaller error than model ND (t D 16.84, p < 0.001) and a 3.0% smaller error than model RD (t D 7.01, p < 0.001). The performances of models C2M and F2M were not statistically different. The bottom graph of Figure 6 gives the learning curves for the four models. In summary, the simulation results using the three data sets show that the developmental models whose progressions were organized by spatial frequency content (models C2M and F2M) signiŽcantly outperformed the nondevelopmental and random-developmental models. To understand the performances of models C2M and F2M better, we analyzed more carefully their behaviors using the solid object data set. We looked at how these models performed on images with disparities of different sizes (small, midsize, and large disparities) at the end of each developmental stage (stages 1, 2, and 3). These data are given in Figure 7. The three graphs in this Žgure correspond to the three disparity sizes. The horizontal axis of each graph gives the stage number for each model; the vertical axis gives each model’s RMSE at the end of the indicated stage on those test set data items with the indicated disparity size. For purposes of comparison, the graphs also include the data for the nondevelopmental model. Model ND did not signiŽcantly change its ability to detect disparities of a given size across training stages; its performance at detecting small, midsize, or large disparities remained nearly constant across the stages. Model C2M improved its ability to estimate disparities of all sizes as training progressed from one stage to the next. In contrast, model F2M appears to have learned about different-sized disparities in each of the different training stages. For example, it learned a great deal about detecting small disparities in stage 1; in stages 2 and 3, it learned more about detecting midsize and large disparities. Not surprisingly, its performance at detecting small disparities degraded as its performance at detecting midsize and large disparities improved. The question remains as to why models C2M and F2M showed superior performance. One logical possibility is that these models performed well due to the fact that they received relatively few inputs early in training. This possibility can be ruled out, however, because model RD received equally few inputs early in training, but this model did not perform well. We speculate that two important features of the developmental progressions of models C2M and F2M account for their superior performances. First, these models were exposed to visual inputs at a single scale early in training. Model C2M received only coarse-scale information at the start of training; model F2M received only Žne-scale information. In contrast, models RD and ND received information at all spatial scales at all stages of training.

178

M. Dominguez and R. Jacobs

small disparities (0-1 pixels)

mid-size disparities (1-2 pixels)

2.0

2.0

RMSE 1.0

1.0

0.0

123 C2M

123 F2M

0.0

123 ND

123 C2M

123 F2M

123 ND

large disparities (2-3 pixels) 2.0

RMSE 1.0

0.0

123 C2M

123 F2M

123 ND

Figure 7: Performance of models C2M, F2M, and ND on images with disparities of different sizes (small, midsize, and large disparities) at the end of each developmental stage (stages 1, 2, and 3). Training and test data items came from the solid object data set.

We conjecture that it might be advantageous for a learning system to receive inputs at a single scale early in training because this allows the system to combine and compare input features without the need to compensate for the fact that these features could be at different spatial scales. Second, models C2M and F2M might be at an advantage because the spatial scale of their inputs progressed in an orderly fashion from one scale to a neighboring scale. Consequently, when these models received inputs at a new spatial scale, this new scale was close to a scale with which the models were already familiar. If it is the case that this second feature of models C2M and F2M is important, then this leads to an interesting prediction. We predict that developmental models whose progressions do not proceed in an orderly manner from one scale to a neighboring scale ought to show poor performance. To test this prediction, two additional models were created and tested on the solid object data set. These models had developmental stages based

Acquisition of Binocular Developmental Sensitivities

179

solid object data set 1.4 RMSE 1.0

0.6

C2M

F2M

C-CF-CMF

F-CF-CMF

Figure 8: RMSE of models C2M, F2M, C-CF-CMF, and F-CF-CMF on the test items from the solid object data set.

on spatial frequency content (like models C2M and F2M), but the addition of new frequency bands to their inputs at each stage did not proceed in an orderly manner. The Žrst new model is referred to as model C-CF-CMF. It received the outputs of complex cells tuned to a low spatial frequency early in training. In stage 2, it received the outputs of complex cells tuned to low and high frequencies, and it received the outputs of cells tuned to low, medium, and high frequencies in stage 3. The second new model, referred to as model F-CF-CMF, was similar to model C-CF-CMF, but it started with high-frequency information. That is, it received the outputs of complex cells tuned to a high spatial frequency early in training. It received the outputs of cells tuned to low and high frequencies in stage 2, and the outputs of all complex cells in stage 3. Figure 8 shows the performances of models C2M, F2M, C-CF-CMF, and F-CF-CMF at the end of training on the solid object data set. In accord with our prediction, models C-CF-CMF and F-CF-CMF showed very poor performance. These data are consistent with the conjecture that it is advantageous to a developmental system for the spatial scale of its inputs to progress in an orderly fashion from one scale to a neighboring scale. 4 Summary and Conclusion

This article has considered the hypothesis that systems learning aspects of visual perception may beneŽt from the use of suitably designed develop-

180

M. Dominguez and R. Jacobs

mental progressions during training. We reported the results of simulations in which different systems were trained to detect binocular disparities in pairs of visual images. Three of the systems were developmental models in the sense that the nature of their input changed during the course of training. These systems received a relatively impoverished visual input early in training, and the quality of this input improved as training progressed. The fourth system was a nondevelopmental model; the nature of its input remained constant during the course of training. The results show that the developmental models whose stages were based on spatial frequency content, models C2M and F2M, consistently outperformed the nondevelopmental and random developmental models. We speculate that the superior performance of these models is due to two important features of their developmental progressions: (1) these models were exposed to visual inputs at a single scale early in training, and (2) the spatial scale of their inputs progressed in an orderly fashion from one scale to a neighboring scale. Simulation results consistent with these speculations were presented. We conclude that suitably designed developmental sequences can be useful to systems learning to detect binocular disparities, in accord with the less-is-more view of development. Moreover, the idea that visual development can aid visual learning is a viable hypothesis in need of study. With relatively few exceptions, the relationship between development and learning has been ignored by the neural computation community. We believe that this is unfortunate. It is well known that systems learn best when they are suitably constrained through the use of domain knowledge. Learning systems are inherently faced with the bias-variance dilemma (Geman, Bienenstock, & Doursat, 1995). Systems with little or no bias are often capable of learning many different sets of training items. Unfortunately, they tend to interpolate in unpredictable ways and thus generalize poorly to novel data items. In contrast, systems that are constrained through the use of domain knowledge and thus have large bias are not able to learn as wide a variety of training sets. However, they tend to show better generalization performance and less variable generalization performance when exposed to those training sets that they can adequately learn. The design of appropriate developmental progressions through the use of domain knowledge provides researchers with an effective means of biasing their learning systems so as to enhance their performances.

Acknowledgments

This work was supported by NSF graduate fellowship DGE9616170 and by NIH research grant R01-EY13149.

Acquisition of Binocular Developmental Sensitivities

181

References Adelson, E. H., & Bergen, J. R. (1985). Spatiotemporal energy models for the perception of motion. Journal of the Optical Society of America A, 2, 284– 299. Atkinson, J., & Braddick, O. (1976). Stereoscopic discrimination in infants. Perception, 5, 29–38. Barnard, S. T. (1987).Stereo matching by hierarchical,microcanonical annealing (Tech. Rep. 414). Menlo Park, CA: ArtiŽcial Intelligence Center, SRI International. Bishop, C. M. (1995). Training with noise is equivalent to Tikhonov regularization. Neural Computation, 7, 108–116. Elman, J. L. (1993). Learning and development in neural networks: The importance of starting small. Cognition, 43, 71–99. Fleet, D. J., Wagner, H., & Heeger, D. J. (1996). Neural encoding of binocular disparity: Energy models, position shifts, and phase shifts. Vision Research, 36, 1839–1857. Fox, R., Aslin, R. N., Shea, S. L., & Dumais, S. T. (1980). Stereopsis in human infants. Science, 207, 323–324. Geman, S., Bienenstock, E., & Doursat, R. (1995) Neural networks and the bias/variance dilemma. Neural Computation, 4, 1–58. Gray, M. S., Pouget, A., Zemel, R. S., Nowlan, S. J., & Sejnowski, T. J. (1998). Reliable disparity estimation through selective integration. Visual Neuroscience, 15, 511–528. Greenough, W. T., Black, J. E., & Wallace, C. S. (1987). Experience and brain development. Child Development, 58, 539–559. Harwerth, R. S., Smith, E. L., Duncan, G. C., Crawford, M. L. J., & von Noorden, G. K. (1986). Multiple sensitive periods in the development of the primate visual system. Science, 232, 235–238. Held, R., Birch, E., & Gwiazda, J. (1980). Stereoacuity in human infants. Proceedings of the National Academy of Sciences USA, 77, 5572–5574. Mallot, H. A., Gillner, S., & Arndt, P. A. (1996). Is correspondence search in human stereo vision a coarse-to-Žne process? Biological Cybernetics, 74, 95– 106. Marr, D., & Poggio, T. (1979). A computational theory of human stereo vision. Proceedings of the Royal Society of London B, 204, 301–328. Matsuoka, K. (1992). Noise injection into inputs in back-propagation learning. IEEE Transactions on Systems, Man, and Cybernetics, 22, 436–440. McKee, S. P., & Mitchison, G. J. (1988). The role of retinal correspondence in stereoscopic matching. Vision Research, 28, 1001–1012. Mowforth, P., Mayhew, J. E. W., & Frisby, J. P. (1981). Vergence eye movements made in response to spatial-frequency-Žltered random-dot stereograms. Perception, 10, 299–304. Newport, E. L. (1990). Maturational constraints on language learning. Cognitive Science, 14, 11–28. Norcia, A., & Tyler, C. (1985). Spatial frequency sweep VEP: Visual acuity during the Žrst year of life. Vision Research, 25, 1399–1408.

182

M. Dominguez and R. Jacobs

Ohzawa, I., DeAngelis, G. C., & Freeman, R. D. (1990). Stereoscopic depth discrimination in the visual cortex: Neurons ideally suited as disparity detectors. Science, 249, 1037–1041. Petrig, B., Julesz, B., Krop, W., Baumgartner, G., & Anliker, M. (1981). Development of stereopsis and cortical binocularity in human infants: Electrophysiological evidence. Science, 213, 1402–1405. Piaget, J. (1952). The origins of intelligence in children. New York: International Universities Press. Press, W. H., Teukolsky, S. A., Vetterling, W. T., & Flannery, B. P. (1992).Numerical recipes in C: The art of scientiŽc computing. Cambridge: Cambridge University Press. Qian, N. (1994). Computing stereo disparity and motion with known binocular cell properties. Neural Computation, 6, 390–404. Quam, L. H. (1986). Hierarchical warp stereo (Tech. Rep. 402). Menlo Park, CA: ArtiŽcial Intelligence Center, SRI International. Rohde, D. L. T., & Plaut, D. C. (1999). Language acquisition in the absence of explicit negative evidence: How important is starting small? Cognition, 72, 67–109. Shatz, C. J. (1996). Emergence of order in visual system development. Proceedings of the National Academy of Sciences USA, 93, 602–608. Sietsma, J., & Dow, R. J. F. (1991). Creating artiŽcial neural networks that generalize. Neural Networks, 4, 67–79. Smallman, H. S. (1995). Fine-to-coarse scale disambiguation in stereopsis. Vision Research, 34, 2971–2982. Turkewitz, G., & Kenney, P. A. (1982). Limitations on input as a basis for neural organization and perceptual development: A preliminary statement. Developmental Psychobiology, 15, 357–368. Webb, A. R. (1994). Functional approximation by feed-forward networks: A least-squares approach to generalization. IEEE Transactions on Neural Networks, 5, 363–371. Zhu, Y., & Qian, N. (1996). Binocular receptive Želd proŽles, disparity tuning, and characteristic disparity. Neural Computation, 8, 1611–1641. Received November 14, 2001; accepted June 4, 2002.

LETTER

Communicated by Terrence Sejnowski

A QuantiŽed Sensitivity Measure for Multilayer Perceptron to Input Perturbation Xiaoqin Zeng [email protected] Daniel S. Yeung [email protected] Department of Computing, Hong Kong Polytechnic University, Hong Kong The sensitivity of a neural network’s output to its input perturbation is an important issue with both theoretical and practical values. In this article, we propose an approach to quantify the sensitivity of the most popular and general feedforward network: multilayer perceptron (MLP). The sensitivity measure is deŽned as the mathematical expectation of output deviation due to expected input deviation with respect to overall input patterns in a continuous interval. Based on the structural characteristics of the MLP, a bottom-up approach is adopted. A single neuron is considered Žrst, and algorithms with approximately derived analytical expressions that are functions of expected input deviation are given for the computation of its sensitivity. Then another algorithm is given to compute the sensitivity of the entire MLP network. Computer simulations are used to verify the derived theoretical formulas. The agreement between theoretical and experimental results is quite good. The sensitivity measure can be used to evaluate the MLP’s performance. 1 Introduction

Theoretically, an artiŽcial neural network is aimed at realizing a mapping between its inputs and outputs, so the sensitivity of a neural network’s output to its input perturbation is an important measure for evaluating its performance, such as error tolerance and generalization ability. Obviously, a quantiŽed sensitivity measure will have both theoretical and practical value. How to deŽne and quantify an appropriate sensitivity for neural networks is still a problem in need of exploration. In the past decade, a variety of studies on the sensitivity of neural networks have emerged. For feedforward neural networks, there are two kinds of sensitivity that differ in the target function being studied: the sensitivity with respect to an objective (error) function and the sensitivity with respect to the neural networks’ output function (Engelbrecht, Fletcher, & Cloete, 1999). These two sensitivity measures seem in general to be different. The sensitivity of the error function is closely related to the process of training Neural Computation 15, 183–212 (2003)

° c 2002 Massachusetts Institute of Technology

184

X. Zeng and D. Yeung

and thus would be useful for training related issues, whereas the output sensitivity has been found useful in a number of problems both in and after training. This article focuses on the sensitivity with respect to the neural networks’ output function. Stevenson, Winter, and Widrow (1990) made use of a hypersphere as the mathematical model in analyzing the sensitivity of Adalines and Madalines. They deŽned sensitivity as the probability of erroneous output of Madalines under the assumption that input and weight perturbations are small and the number of Adalines per layer is sufŽciently large. Alippi, Piuri, and Sami (1995) generalized the hypersphere model by also considering the sensitivity as the probability of erroneous output of the feedforward multilayered neural networks with multiple-step symmetric activation functions. But their results are applicable only to networks with discrete neuron inputs and outputs and thus are not suitable for multilayer perceptrons (MLPs) with continuous inputs and outputs. Cheng and Yeung (1999) again used a geometrical technique to investigate the sensitivity of neocognitrons by modifying the hypersphere model. Unfortunately, the hypersphere model they used fails in the case of the MLP because its input vectors generally do not span a hypersphere surface or volume. PichÂe (1995) employed a statistical rather than a geometrical argument to analyze the effects of weight errors in Madalines. He made the assumption that inputs and weights as well as their errors are all independently and identically distributed with mean zero. Based on such a statistical model and under the condition that both input and weight errors are small enough, PichÂe derived an analytical expression for the sensitivity as the ratio of the variance of the output error to the variance of the output. Recently, Yeung and Sun (2002) generalized PichÂe’s method by deriving a universal expression of the MLP’s sensitivity for antisymmetric squashing activation functions, without any restriction on the magnitude of input and weight perturbations. However, this approach is applicable only to an ensemble of networks, not to any individual one, because the assumptions on the statistical model are too strong. There are basically two approaches to study the sensitivity of the MLP. The analytical approach deŽnes the sensitivity as a partial derivative of output to input. For example, Hashem (1992) and Fu and Chen (1993) computed the partial derivative one layer at a time, starting from the output layer and proceeding backward (using backward chaining) toward the input layer. But this approach is applicable only when the input deviation tends to zero. The other approach is statistically based. For example, Choi and Choi (1992) introduced a statistical sensitivity measure for the MLP with differentiable activation functions. The sensitivity is deŽned as the ratio of the standard deviation of output errors to the standard deviation of weight or input error under the condition that the latter tends to zero. They derived a formula for the sensitivity of a single-output MLP under a given input pattern. However, these two approaches are useful only to measure the output deviation with respect to a given input pattern. They are not

MLP’s Sensitivity to Input Perturbation

185

able to measure the expected output deviation with respect to the overall input patterns, an important factor affecting the performance measurement of networks like MLPs. For our study, instead of using a hypersphere as described in Stevenson et al. (1990), Alippi et al. (1995), and Cheng and Yeung (1999), we employed a hyperrectangle as the mathematical model to represent an MLP’s input space because it usually falls into a continuous interval. Our approach is different from those of the other studies and offers certain advantages over them. For example, it does not demand that the input deviation be very small as the derivative approach requests, and the mathematical expectation used in our approach reects the network’s output deviation more directly and exactly than the variance does; for example, some assumptions must be made for the expectation, as in Choi and Choi (1992), PichÂe (1995), and Yeung and Sun (2002). Moreover, our result is applicable to the MLP that deals with inŽnite input patterns, which is an advantage of the MLP over other discrete feedforward networks like Madalines. In Zeng and Yeung (2001) the sensitivity for an ensemble of MLPs due to input and weight perturbations was investigated. But this article is mainly concerned with an individual MLP with Žxed weights and thus Žxed network architecture. We note that the sensitivity analysis of an ensemble of MLPs with respect to weight perturbations provides some useful design guidelines prior to the hardware implementation of an MLP, whereas the sensitivity measure of a trained MLP to input perturbations is directly related to its application performance. The arrangement of this article is as follows. The MLP model is briey described in section 2. The deŽnitions of sensitivity are given in section 3 for different granules in the MLP from neuron to layer and Žnally to the entire network. Sections 4 and 5 deal with the computation of the sensitivity, in which algorithms based on some analytical expressions are given for the computation of the sensitivity of a single neuron and the sensitivity of the entire MLP. Simulation results that support the theoretical results are presented in section 6. Section 7 concludes the article.

2 The Multilayer Perceptron Model 2.1 Notation. Generally, an MLP can have L layers, and each layer l (1 · l · L ) has nl ( nl ¸ 1) neurons. The form n 0 ¡n1 ¡, . . . , ¡nL is used to represent an MLP with a certain structural conŽguration, in which each nl (0 · l · L ) not only indicates the number of neurons (n0 is an exception, which indicates the dimension of input), but also stands for a layer from left to right, including the input layer. n0 stands for the input layer and nL for the output layer. Since the number of neurons in layer l ¡ 1 is equal to the output dimension of that layer, which is also equal to the input dimension of layer l, the input dimension of layer l is nl¡1 . For a neuron i (1 · i · nl )

186

X. Zeng and D. Yeung

in layer l, its input vector is Xl D ( xl1 , . . . , xlnl¡1 ) T , its weight vector is W il D (wli1 , . . . , wl l¡1 ) T , and its output is yli D f ( Xl ¤W il ) , where f (¢) is an activation in function having the summation of weighted inputs as its variable. For each layer l, all of its neurons have the same input vector Xl . Its weight set is W l D fW 1l , . . . , Wnl l g, and its output vector is Yl D ( yl1 , . . . , ylnl ) T . For the entire MLP, its input is the vector X1 or Y 0 , its weight is W D W 1 [ ¢ ¢ ¢ [ W L , and its output is YL . Let D Xl D ( D xl1 , . . . , D xlnl¡1 ) T and D Yl D (D yl1 , . . . , D ylnl ) T be the corresponding deviations for input and output, respectively. 2.2 Structure. Neurons, as cells, in an MLP are organized into layers by linking them together. Links exist only between neurons of two adjacent layers; there is no link between neurons in the same layer or in any two nonadjacent layers. All neurons in a layer are fully linked from all the neurons in the immediately preceding layer and are fully linked to all the neurons in the immediately succeeding layer. At each layer except the input layer, the inputs of each neuron are the outputs of the neurons in the previous layer, Xl D Yl¡1 ( l ¸ 1). The commonly used logistic function is chosen as the activation function, that is,

f ( x ) D 1 / (1 C e¡x ) ,

(2.1)

with the following characteristics: 0 · f ( x) · 1

(2.2)

f ( x ) C f (¡x) D 1

(2.3)

f ( x0 ) < f ( x00 ) if x0 < x00 .

(2.4)

3 DeŽnition of the Sensitivity

The main issue addressed by the sensitivity study is what the effects of input deviation will be on the MLP network’s output. For the case of a single neuron, the most direct and natural way to express the output deviation arising from input deviation is the difference of its deviated and nondeviated output,

D yli

D g ( Xl , D Xl , W il ) D f ( ( Xl C

D Xl ) ¤Wil ) ¡ f (Xl ¤Wil ).

(3.1)

Since Wil is Žxed, D yli can be easily computed if the values of Xl and D Xl are known. But in most cases, especially in the performance evaluation of a network, it would not be appropriate to compute D yli for a speciŽc input. Rather, it would be more desirable to Žnd the relationship between

MLP’s Sensitivity to Input Perturbation

187

the output deviation D yli and its cause, the input deviation D Xl , in an ensemble sense by taking overall inputs into consideration. It is obvious that equation 3.1 reects the relationship between D yli and D Xl . The problem is to deŽne such a sensitivity that keeps the relationship without reference to Xl . Our approach is to consider the mathematical expectation of D yli with respect to all possible values of Xl , that is, to treat Xl as a statistical variable. Furthermore, since input deviation is usually randomly oriented, it is proper to replace D Xl in equation 3.1 by its expected absolute value, denoted as D Xl D (D xl1 , D xl2 , . . . , D xlnl¡1 ) D ( E (| D xl1 |) , E (| D xl2 | ) , . . . , E (| D xlnl¡1 |) ) . The following deŽnitions use a bottom-up way to deŽne the sensitivity of different granules in the MLP. First is the sensitivity of a single neuron, then the sensitivity of a layer, and Žnally the sensitivity of the entire MLP. For any input vector Xl 2 [a1 , b1 ] £[a2 , b2 ] £¢ ¢ ¢ £[anl¡1 , bnl¡1 ] µ [0, and its corresponding expected deviation D Xl and a given weight vector l¡1 Wil 2 Rn , the sensitivity of neuron i in layer l is deŽned as the mathematical expectation of the absolute value of D yli with respect to Xl , which is expressed as DeŽnition 1. l¡1 1]n

sli D E (| g ( Xl ) |) D E (| f ( (Xl C D Xl )¤Wil ) ¡ f ( Xl ¤Wil ) | ).

(3.2)

Obviously, sli is a function of D Xl , independent of any speciŽc Xl . Because of equation 2.2, the interval of the input of all layers except the input layer is l¡1 in [0, 1]n (l ¸ 2), and one can normalize the input of the Žrst layer to be in l¡1 0 [0, 1]n . So the assumption Xl 2 [a1 , b1 ]£[a2 , b2 ]£¢ ¢ ¢£[anl¡1 , bnl¡1 ] µ [0, 1]n in the sensitivity deŽnition is practically acceptable. For any input vector Xl 2 [a1 , b1 ] £[a2 , b2 ] £¢ ¢ ¢ £[anl¡1 , bnl¡1 ] µ [0, 1]n and its corresponding expected deviation D Xl and a given weight vector l¡1 set W l D fWil | Wil 2 Rn I 1 · i · nl g, the sensitivity of layer l is deŽned as DeŽnition 2. l¡1

Sl D ( sl1 , . . . , slnl ) T .

(3.3)

D Xl (l > 1) in fact can be regarded as the sensitivity of the previous layer, that is, D Xl D S 0 D ( E (| D x11 |) , E (| D x12 |) , . . . , E (| D x1n0 |) ) when l D 1 and D Xl D Sl¡1 when l > 1. It is noted that Sl is a vector, which consists

of the sensitivities of all neurons in layer l. Since sensitivity is generally agreed to be a scalar rather than a vector, one may choose, for instance, norm(Sl ) , max(Sl ) , avg(Sl ) , or min(Sl ) as the representation of a layer ’s sensitivity depending on the nature of the application involved. One popular strategy is to choose the one with a smaller value if the application requires a higher precision.

188

X. Zeng and D. Yeung

The sensitivity of an MLP is the sensitivity of its output layer,

DeŽnition 3.

that is, S D SL D ( sL1 , . . . , sLnL ) T .

(3.4)

4 Computation of the Sensitivity for a Single Neuron

In this section, we derive formulas for the computation of equation 3.2, i.e., the sensitivity of a single neuron. For simplicity, the superscript that marks the neuron’s layer and the subscript that marks the neuron’s order in the layer are omitted because only single neurons are considered and their positions in the network will be irrelevant. Hence, s, X, D X, and W are used to replace sli , Xl , D Xl , and Wil in this and the next section. Clearly equation 3.2 can be expressed as Z sD

Z V

Q (X ) | f ( (X C D X ) ¤W) ¡ f ( X¤W ) | dV,

¢¢¢

(4.1)

where Q (X ) is the density function and V is the region [a1 , b1 ] £ ¢ ¢ ¢ £ [an , bn ] spanned by X. By assuming that the components of X are independent from each other and thus each can be regarded as a coordinate axis, equation 4.1 can be further expressed as Z sD

b1 a1

Z

b2 a2

Z ¢¢¢

bn an

Q (X ) | f ( (X C D X ) ¤W) ¡ f ( X¤W ) | dx1 dx2 , . . . , dxn .

(4.2)

Furthermore, by equation 2.4, namely, f ( x) being a monotonic increasing function, and the fact that Q ( X ) ¸ 0, the absolute value sign can be moved to the outside of the integral. Thus, 8Z b Z b Z bn 1 2 > > > ¢ ¢ ¢ Q (X ) f ( ( X C D X ) ¤W) dx1 dx2 , . . . , dxn > > > a1 a2 an > > Z b1 Z b2 Z bn > > > > ¡ ¢¢¢ Q (X ) f ( X¤W ) dx1 dx2 , . . . , dxn , if D X¤W ¸ 0 > > > a1 a2 an < Z b1 Z b2 Z bn sD > ¢ ¢ ¢ Q (X ) f (X¤W ) dx1 dx2 , . . . , dxn > > > a1 a2 an > > Z b1 Z b2 Z bn > > > > ¡ ¢ ¢ ¢ Q ( X ) f ( ( X C D X ) ¤W) dx1 dx2 , . . . , dxn , > > > a1 a2 an > > : if D X¤W < 0

MLP’s Sensitivity to Input Perturbation

189

­ Z b Z b2 Z bn ­ ­1 ¢¢¢ Q ( X ) f ( ( X C D X )¤W ) dx1 dx2 , . . . , dxn D ­ ­a1 a2 an ­ Z b1 Z b2 Z bn ­ ­ ¡ ¢¢¢ Q ( X ) f ( X¤W ) dx1 dx2 , . . . , dxn ­ . ­ a1 a2 an

(4.3)

(¢) Now replacing the inner P product by a summation and f by the logistic function and let s D jnD1 D xj wj , equation 4.3 becomes ­ 0 1 ­ Z Z Z bn n X ­b1 b2 ¢¢¢ Q (X ) f @ (xj C D xj ) wj A dx1 dx2 , . . . , dxn sD ­ ­ a1 a2 an ­ j D1 ­ 0 1 ­ Z b1 Z b2 Z bn n X ­ ¡ ¢¢¢ Q (X) f @ xj wj A dx1 dx2 , . . . , dxn ­ ­ a1 a2 an ­ j D1 ­ ¿³ ´ Z b Z b2 Z bn Pn ­ ¡ ­1 j D1 xj wj ¡ s ¢¢¢ Q (X ) 1Ce D ­ dx1 dx2 , . . . , dxn ­ a1 a2 an Z ¡

b1

Z

a1

b2 a2

Z ¢¢¢

¿³

bn

Q (X ) an

1Ce

¡

Pn

j D1 xj wj

´

­ ­ ­ . dx1 dx2 , . . . , dxn ­ ­

(4.4)

Therefore, the computation of the sensitivity is essentially reduced to the calculation of the following integral, Z I ( A, B, W, s) D

b1 a1

Z

b2 a2

Z ¢¢¢

bn an

¿³ ´ Pn ¡ xj wj ¡s D 1 j Q (X) 1 1Ce dx1 dx2 , . . . , dxn ,

(4.5)

where A D fa1 , a2 , . . . , an g and B D fb1 , b2 , . . . , bn g are sets of lower and upper bounds of the integral, and W D fw1 , w2 , . . . , wn g is the set of the 6 weights, with bj > aj and wj D 0 for 1 · j · n. The following algorithm could be used to compute the sensitivity for a single neuron: Algorithm: NEURON SENSITIVITY ( A, B, Q ( X) , D X, W ) 1. s D D X¤W;

190

X. Zeng and D. Yeung

Z 2. I ( A, B, W, s) ¼

Z

a1

Z 3. I ( A, B, W, 0) ¼

b1

a2

b1 Z a1

b2

b2 a2

Z ¢¢¢

Q (X ) an

Z ¢¢¢

¿³

bn

1Ce

¡

Pn

j D1 xj wj

¡s

´

dx1 dx2 , . . . , dxn ; ¿³ ´ Pn bn ¡ j D 1 xj wj Q (X) 1Ce

an

dx1 dx2 , . . . , dxn ;

4. s D |I ( A, B, W, s) ¡ I ( A, B, W, 0) | is the sensitivity for a single neuron. The following sections present the derivation of an approximate solution for equation 4.5 under the assumption that Q (X ) D 1 (X being uniformly distributed in the hyperrectangle), and an algorithm for its computation. 4.1 Approximate Solution for Integration of the Logistic Function.

Since no primitive function can be found for the integrand in equation 4.5, the Taylor series,

1 / (1 C x ) ¼

p X

(¡1) kx k,

(4.6)

kD 0

under the convergent condition |x| < 1, is employed to approximate the p( ) integrand. Let i (X ) denote the integrand Pn and i X denote its Taylor expan¡ j D 1 xj wj ¡ s for x in equation 4.6, the sion of p terms. By substituting e integrand could be approximately expressed as

i ( X ) ¼ ip ( X ) D

p X

(¡1) ke

¡k

Pn

j D1 xj wj

¡ ks

,

(4.7)

kD 0

Pn Pn ¡ j D 1 xj wj ¡ s | < 1, namely, where |e j D 1 xj wj C s > 0, is required. But Pn how to deal with i ( X ) when jD 1 xj wj C s · 0? It is quite clear that when P X satisŽes jnD1 xj wj C s < 0, ip (X ) will certainly be divergent. Fortunately, (¡ ) ( ) the logistic function’s characteristic, expression 2.3, P f x D 1 ¡ f x , can be used to solve this problem. When X satisŽes jnD 1 xj wj C s D 0, ip (X ) oscillates between 0 and 1, that is, ip ( X ) D 0 when p is an odd number, say p1 , and ip (X ) D 1 when p is an even number, say p2 . But at X it actually has i (X ) D 1 / 2. PThus, it is necessary to amend the imprecision arising from X satisfying jnD 1 xj wj C s D 0 in V. This can be done by calculating equation 4.5 twice, once with p1 and once with p2 , and then taking the average of the two

MLP’s Sensitivity to Input Perturbation

191

results. Now equation 4.5 can be rewritten as Z I¼

Z V

¢¢¢

ip ( X ) dV

8Z ! Z ÁX Pn p > ¡k xj wj ¡ ks > k D 1 j > (¡1) e ¢¢¢ dV, > > > V > kD 0 > > > n > X > > > if (4.8a) xj wj C s > 0 > > > > D 1 j > > ³Z ´ Z Z Z > > 1 > p1 ( ) p2 ( ) > > C ¢ ¢ ¢ ¢ ¢ ¢ i X dV i X dV , > > 2 > V V > < n X D if (4.8b) xj wj C s D 0 > > > jD 1 > > > > > > > > ! >Z > Z Á Pn p > X > xj wj C ks > ¢¢¢ k k D 1 j > (¡1) 1¡ e dV, > > > V > kD 0 > > > n > X > > > if xj wj C s < 0. (4.8c) > : jD 1

PnFrom a geometric point of view, the hyperplane P that is deŽned by j D 1 xj wj C s D 0 can be regarded as a dividing plane. It divides the ndimensional space into three parts: the points P on P, the points below P, and the points above P. The points on P satisfy jnD 1 xj wj C s D 0, and each of P P the other two parts satisŽes either jnD1 xj wj C s > 0 or jnD 1 xj wj C s < 0. Pn They are two opposite cases. If one satisŽes jD 1 xj wj C s > 0, the other P must satisfy jnD1 xj wj C s < 0, and vice versa. Now the problem is how to Žnd out whether P cuts V. If it does, how can equations 4.8a and 4.8c be computed? A solution to this problem is proposed below. The idea is to locate the intersection points of P and those parallel lines that are the extensions of the edges of V. Each line must have one and only one intersection point 6 with P under the assumption wj D 0 for 1 · j · n. Then by comparing the coordinates on a given axis of those intersection points with the lower bound and the upper bound of V on that axis, it is easy to identify the intersection situations between P and V. It is known that V has 2n¡1 edges parallel to a given coordinate axis. Take the xn -axis as an example. Each edge parallel to the xn -axis can be determined by assigning the n ¡ 1 coordinates ( x1 , . . . , xn¡1 ) with either xj D aj or xj D bj . Thus, the xn -coordinate for a

192

X. Zeng and D. Yeung

given line’s intersection point can be calculated by 0 xn D @

1 n¡1 X jD 1

xj wj C s A / wn

(4.9)

with its n¡1 Žxed coordinates. This will yield a set En D fen,1 , en,2 , . . . , en,2n¡1 g representing 2n¡1 edges. Note that during the calculation of En , an order is set to select the 2n¡1 edges, which are represented by the second subscript of the elements of E n . This means that the second subscript uniquely corresponds to an edge and thus its n ¡ 1 Žxed coordinates, denoting as (x1 , . . . , xn¡1 ) or fx1 , . . . , xn¡1 g for en,i (1 · i · 2n¡1 ) . This property will be used later. Let en,s D minfen,1 , en,2 , . . . , en,2n¡1 g and en,l D maxfen,1 , en,2 , . . . , en,2n¡1 g, and compare the four values en,s , en,l , an , and bn . In the following we present Žve possible cases together with their corresponding formulas for calculating equation 4.8. Only three-dimensional Žgures are used for the illustration. 4.1.1 Case 1: en,s ¸ bn or en,l · an . In this case as illustrated by Figure 1, P does not cutP V. Hence, in V, only one convergent condition of equation 4.8 is satisŽed. If jnD 1 xj wj C s > 0, then Z I¼

V

Z D D D

b1

Z

a1 p X

p X

¡k (¡1) k

!

Pn

j D 1 xj wj ¡ ks

e

kD 0 b2

a2

Z

¢¢¢ Z

(¡1) k

bn

p X

an b1

a1

kD 0

Z

(¡1) ke

kD 0 b2 a2

¢¢¢

Z

¡k

bn

e

Pn

¡k

j D1 xj wj

Pn

j D 1 xj wj

dV ¡ ks

dx1 dx2 , . . . , dxn

¡ ks

dx1 dx2 , . . . , dxn

an

(¡1) kI1 (¡k, s, A, B, W ).

(4.10)

kD 0

Otherwise,

I¼

¢¢¢

Z ÁX p

Pn

p X

j D 1 xj wj C

s < 0, and it follows that

(¡1) kC 1 I1 ( k, s, A, B, W ) .

(4.11)

kD 1

4.1.2 Case 2:en,s ¸ an and en,l · bn . In this case, as illustrated by Figure 2, P cuts V by the side and divides V into two parts, V1 and V2 . Supposing

MLP’s Sensitivity to Input Perturbation

193

Figure 1: P does not cut V.

Pn

j D 1 xj wj C

s < 0 in V1 and

Z I¼

V1

D

kD 0

¢¢¢

Z X p Z

V1

p X kD 0

p X kD 1

C

(¡1) ke

kD 0

(¡1) kC 1

kD 1

C

D

V2

p X

j D 1 xj wj C

s > 0 in V2 , it follows that

! Z Á Pn p X k xj wj C ks k D 1 j (¡1) e ¢¢¢ 1¡ dx1 dx2 , . . . , dxn

Z C

Pn

Z

¢¢¢

Z

e

¡k

k

Z

(¡1) k V2

¢¢¢

e

Pn

j D1 xj wj

¡ ks

j D1 xj wj C

ks

Pn

¡k

Pn

j D 1 xj wj

dx1 dx2 , . . . , dxn

dx1 dx2 , . . . , dxn

¡ ks

dx1 dx2 , . . . , dxn

(¡1) kC 1 |I2 ( k, s, A, B ¡ fbn g, W ) |

p X kD 0

(¡1) k |I2 (¡k, s, B, A ¡ fan g, W ) |,

Figure 2: P cuts the four parallel sides of V.

(4.12)

194

X. Zeng and D. Yeung

where Z

Z V1

¢¢¢

e

Z D

b1

k

Pn

j D 1 xj wj C

Z

a1

b2 a2

Z ¢¢¢

bn¡1

ks

±P n¡1

Z

an¡1

dx1 dx2 , . . . , dxn

¡

jD 1

an

£e

k

² xj wj C s / wn

Pn

j D 1 xj wj C

ks

dx1 dx2 , . . . , dxn , (4.13)

and Z V2

¢¢¢

Z X p

Z D

(¡1) ke

kD 0 b1 a1

Z

b2 a2

Z ¢¢¢

¡k

Pn

bn¡1 an¡1

j D1 xj wj

Z

bn

p X kD 0

C

² xj wj C s / wn Pn ¡k j D1 xj wj ¡ ks jD 1

£e

I¼

dx1 dx2 , . . . , dxn

±P n¡1

¡

Similarly, the formula for I with in V2 is

¡ ks

Pn

j D 1 xj wj C s

dx1 dx2 , . . . , dxn . (4.14)

> 0 in V1 and

Pn

j D 1 xj wj C s

< 0

(¡1) k |I2 (¡k, s, A, B ¡ fbn g, W ) |

p X kD 1

(¡1) kC 1 |I2 ( k, s, B, A ¡ fan g, W ) |.

(4.15)

4.1.3 Case 3: en,s D an and en,l > bn or en,s < an and en,l D bn . In this case, as illustrated by Figure 3 , P cuts a prism from V, and there is a special intersection point that is a vertex of V. We assume that the point C(c1 , c2 , . . . , cn ) is the lower vertex of V on the edge corresponding to en,s and the point D(d1 , d2 , . . . , dn ) is the upper vertex of V on the edge corresponding to en,l . Then either C or D is the special intersection point, and it must be true that cn D an and dn D bn . The other n ¡ 1 coordinates of C and D can be determined by en,s and en,l , as mentioned above. The coordinates of these two points are critical in setting the boundary for the integral in calculating equation 4.8. For simplicity of expression, they are also regarded as two sets: C D fc1 , . . . , cn g and D D fd1 , . . . , dn g. Referring to the results in section 4.2, three regions V1 , V2 , and V3 are used to replace V, as indicated in Figures 4, 5, and 6. It is obvious that V1 is suitable for equation 4.24 and V2 for 4.23. But V3 , as Figure 7 illustrates, needs to be extended

MLP’s Sensitivity to Input Perturbation

Figure 3: P cuts a prism from V.

Figure 4: The region of V1 .

Figure 5: The region of V2 .

Figure 6: The region of V3 .

Figure 7: The extended region of V3 .

195

196

X. Zeng and D. Yeung

with 2n¡1 ¡ 2 small prisms in order to be suitable for equation 4.24. In fact, this situation may be true for V1 too. In order to give a general solution, equation 4.24 is Žrst used to calculate the integral by regarding V1 and V3 as prisms, and then the results are adjusted by subtracting the integral in each small prism under the condition that the integrand expansion in that small prism is convergent. The integrand expansion in each of the small prisms will diverge if V1 and V3 do not need such a prism for the extension. Thus, according to whether cn D en,s or dn D en,l , and the satisfaction of the convergent conditions of equations 4.8a and 4.8c in V1 , V2 , and V3 , four groups of formulas for different conditions could P be derived. Below, the derivation for only one group is given: cn D en,s , jnD 1 xj wj C s > 0 in V1 and Pn j D 1 xj wj C s < 0 in V 2 and V3 . Those for the other three groups can be found in Zeng (2001).

Z I¼

V1

¢¢¢

D

p X

¡ D

C

¡

¡ ks

dx1 dx2 , . . . , dxn

kD 0

kD 0

Z

Z

(¡1) k V1

¢¢¢

p X kD 1

kD 0

e

¡k

Z

Pn

j D1 xj wj

Z

(¡1) kC 1 V2

kD 1

kD 0

j D 1 xj wj

V3

p X

p X

Pn

! Z Á Pn p X k xj wj C ks k D 1 j (¡1) e ¢¢¢ 1¡ dx1 dx2 , . . . , dxn

kD 0

C

¡k

V2

Z ¡

(¡1) ke

! Z Á Pn p X k xj wj C ks k D 1 j (¡1) e ¢¢¢ 1¡ dx1 dx2 , . . . , dxn

Z C

Z X p

¢¢¢

Z

e Z

(¡1) kC 1 V3

¢¢¢

e

k

k

¡ ks

dx1 dx2 , . . . , dxn

Pn

j D1 xj wj C

ks

Pn

j D1 xj wj C

ks

dx1 dx2 , . . . , dxn

dx1 dx2 , . . . , dxn

0

(¡1) k @ |I3 (¡k, s, C ¡ fcn g) [ fdn g, W ) | ¡

p X kD 1 p X kD 1

2n¡1 X

1 dh A

6 s&h D 6 l hD 1&hD

(¡1) kC 1 |I2 ( k, s, C, D ¡ fdn g, W ) | 0 (¡1) kC 1 @|I

3 ( k,

s, D, W ) | ¡

n¡1 2X

6 s&hD 6 l h D1&hD

1

eh A ,

(4.16)

MLP’s Sensitivity to Input Perturbation

where Z V1

Z ¢¢¢

e

¡k

Pn

j D 1 xj wj

¡ ks

197

dx1 dx2 , . . . , dxn

² ­±P n¡1 ­ Z ­ j D 2 cj wj C dn wn C s / w1 D­ ­ c1 ­ ± ² Z x1 w1 C Pn¡1 cj wj C dn wn C s / w2 j D3 £ c2 ± ² Z Pn¡2 xj wj C dn wn C s / wn¡1 Z dn j D1 ±P ¢¢¢ n¡1

² xj wj C s / wn ­ ­ Pn ­ ¡k j D 1 xj wj ¡ ks dx dx , . . . , dx ­ £e 1 2 n­ ­

cn¡1

¡ Z ¢¢¢

(4.17)

e

k

Pn

j D 1 xj wj C

ks

dx1 dx2 , . . . , dxn

± ² ­ ­ Z b1 Z b2 Z bn¡1 Z ¡ Pn¡1 xj wj C s / wn p X ­ D 1 j kC 1 ¢¢¢ D­ ­ (¡1) a1 a2 an¡1 an ­ kD 1 ­ ­ Pn ­ C k x w ks j j D 1 j £e dx1 dx2 , . . . , dxn ­ , ­ ­

Z

Z V3

dh ,

6 s&hD 6 l h D1&h D

Z V2

and

n¡1 2 X

j D1

¢¢¢

e

k

Pn

j D 1 xj wj C

ks

(4.18)

dx1 dx2 , . . . , dxn

­ ­ Z Z d2 ­d1 ± ² ± ² D­ P Pn ­ n C C C s ¡ s d w w x w d w ­¡ / / w2 1 1 1 j j j j j D2 jD 3 ±P ² n¡1 Z dn¡1 Z C s x w / wn j j jD 1 ² ¢ ¢ ¢ ±Pn¡2 ¡ j D 1 xj wj C dn wn C s / wn¡1 dn ­ ­ n¡1 Pn 2 X ­ C k x w ks j j j D1 £e ¡ eh . (4.19) dx1 dx2 , . . . , dxn ­ ­ ­ h D 1&hD6 s&hD6 l

198

X. Zeng and D. Yeung

Further, 8­±P ² n¡1 ­ Z > C dh wh C dn wn C s / w1 c w > j j ­ 6 D 2&j D j h > > ­ > ¢¢¢ > ­ > > ­c1 > > > ± ² > > Z Ph¡1 xj wj C Pn¡1 cj wj C dn wn C s / wh > > D1 Dh C 1 j j > > > ¢¢¢ > > > dh > ± ² > P > n¡2 < Z Z dn j D1 xj wj C dn wn C s / wn¡1 dh D ±P ² > n¡1 > C ¡ s c x w / wn > n¡1 j j jD 1 > > > ­ > > ­ Pn > > ­ > ¡k > j D 1 xj wj ¡ ks dx , . . . , dx ­ > £ e , 1 h > ­ > > ­ > > > > > > if it is convergent for large p > > : 0, if it is divergent for large p 8 |I3 (¡k, s, ( C ¡ fch , cn g) [ fdh , dn g, W ) | > > > > p > X > > > (¡1) k |I3 (¡k, s, ( C ¡ fch , cn g) [ fdh , dn g, W ) | if < D

kD 0

> > > > > > > > :

< 0,

p X kD 0

(¡1) k |I

3 (¡k,

s, (C ¡ fcn g) [ fdn g, W ) |

otherwise

and 8­ ­ Z > > ­d1 > > ­± ² ¢¢¢ > Pn > ­ > > ­¡ ch wh C jD2&jD6 h dj wj C s / w1 > > > Z c > h > > > ±P ² ¢¢¢ > P > h¡1 > ¡ > x1 w1 C jnDh C 1 dj wj C s / wh > D 1 j > > ± ² > > Z Pn¡1 xj wj C s / wn < Z dn¡1 jD 1 eh D ±P ² n¡2 > > C C ¡ s x w d w / wn¡1 dn > n n j j jD 1 > > > ­ > > ­ Pn > > ­ > > £ ek jD1 xj wj C ks dx1 , . . . , dxn ­ , > > ­ > > ­ > > > > > if it is convergent for large p > > > :0, if it is divergent for large p

(4.20)

MLP’s Sensitivity to Input Perturbation

199

Figure 8: V is divided into two smaller rectangles.

Figure 9: V is extended to a bigger rectangle with two extra rectangles.

D

8 |I3 ( k, s, (D ¡ fdh g) [ fch g, W ) |, > > > > p > X > > > (¡1) kC 1 |I3 ( k, s, ( D ¡ fdh g) [ fch g, W ) | if > < > > > > > > > > > :

k D1

< 0,

p X

(4.21)

(¡1) k C 1 |I3 ( k, s, D, W ) |

kD 1

otherwise.

In a similar way, the other three groups of formulas could be derived. 4.1.4 Case 4: en,s > an and en,l > bn or en,s < an and en,l < bn . In this case, P cuts V as Figure 8 illustrates. It can be reduced to cases 1 and 3 by cutting V with the hyperplane xn D en,s or xn D en,l into two smaller hyperrectangles. 4.1.5 Case 5: en,s < an and en,l > bn . This case, as Figure 9 illustrates, can be reduced to cases 2 and 3 by considering a bigger hyperrectangle Žrst, as case 2 does, and then dealing with two extra hyperrectangles on the top and bottom, as case 3 does. 4.2 Three Integral Formulas Used in the Derivations. Below are three integration formulas that are used in the derivations of equations 4.10 to

200

X. Zeng and D. Yeung

4.21. These formulas can be established by mathematical induction on n. It is assumed that k is an integer Pn and s is a real number. These formulas all k j D 1 xj wj C ks , but different integration regions. have the same integrand; e Each region represents a closed subregion within the hyperrectangle V. Integration Formula Z b1 Z I1 ( k, s, A, B, W ) D a1

D

b2 a2

Z ¢¢¢

bn

e

k

Pn

j D 1 xj wj C

ks

an

8Y n > > ( bj ¡ aj ) > > < jD 1

dx1 dx2 , . . . , dxn

if k D 0 (4.22)

n Y > > ks > (e kbj wj ¡ e kaj wj ) / ( kwj ) , > e :

6 0 if k D

j D1

The integration region of this formula is the entire V, which is enclosed by 2n planes perpendicular to each other, xj D aj and xj D bj (1 · j · n ) , as Figure 10 illustrates. Integration Formula Z I2 ( k, s, A, B ¡ fbn g, W ) D

b1 a1

Z

b2 a2

Pn

Z ¢¢¢

bn¡1

Z

±P

¡

n¡1 j D 1 xj wj C

² s / wn

an¡1

j D1 xj wj C

ks dx1 dx2 , . . . , dxn 10 1 8 00 n¡1 n¡1 > X Y > > > ¡ @@ wj (bj C aj ) / 2A C an wn C s A @ ( bj ¡ aj ) A / wn , > > > > jD 1 jD 1 > > > < if k D 0 0 1 (4.23) D > n¡1 n¡1 n > Y Y Y > > > (bj ¡ aj ) / ( kwn ) ¡ ekan wn C ks ( ekbj wj ¡ ekaj wj ) / @kn wj A , > > > > D 1 D 1 D 1 j j j > > : 6 0. if k D £e 1

k

The integration P region of this formula is the lower part of V cut by the hyperplane P, jnD 1 xj wj C s D 0, through the side parallel to an axis, say xn axis, as Figure 11 illustrates. This formula is also suitable for the upper part of V by adjusting the integration limits. Furthermore, if one is interested only in the scalar of the integral, the condition bj > aj can be removed because the lower limit and the upper limit for each variable in the integral can be arbitrarily exchanged without affecting the absolute value of the integral.

MLP’s Sensitivity to Input Perturbation

201

Figure 10: The entire V is the integration region.

The previous formula, 4.22, and the next formula, 4.24, also possess this characteristic. Integration Formula ± Z Pn I3 ( k, s, A, W ) D

j D 2 aj wj C

² ± ² P s / w1 Z x1 w1 jnD 3 aj wj C s / w2

a1

a

¢¢¢

2 ± ² ± ² Z Pn¡2 xj wj C an wn C s / wn¡1 Z Pn¡1 xj wj C s / wn jD 1 jD 1

an¡1 k

Pn

an

j D 1 xj wj C ks

£e dx1 dx2 , . . . , dxn 8 0 1n ,0 1 n n > X Y > > > @n! (¡1) n @ aj wj C s A wj A , > > > > j D1 jD 1 > > > 0 1l ,0 1 > > <X n¡1 n n X Y @kn¡l l! D (¡1) n C 1 @ aj wj C s A wj A > > > lD 1 j D1 j D1 > > 0 1 > > ³ ´, Pn > n > Y > > n C 1 1 ¡ ek j D 1 aj wj C ks @ kn > wj A , > C (¡1) : jD 1

if k D 0 (4.24)

6 0. if k D

The integration region of this formula is a prism part of V, as Figure 12 illustrates, which is enclosed by P with the n planes perpendicular to each other: xj D aj (1 · j · n ) . This formula is suitable for any prism part of a

Figure 11: The lower part of V is the integration region.

202

X. Zeng and D. Yeung

Figure 12: The prism part of V is the integration region.

hyperrectangle cut by a hyperplane. The hyperplane part in the prism is its base, and the coordinate ( a1 , a2 , . . . , an ) indicates the vertex not on the base. 4.3 Algorithm for the Computation of the Integral. Based on the above derivations, the following algorithm outlines the computation of equation 4.5 with Q (X ) D 1:

Algorithm: I ( A, B, W, s) P n¡1 with either 1. Calculate en,i D ¡( jn¡1 D 1 xj wj C s) / wn for 1 · i · 2 xj D aj or xj D bj ; 2. Find s and l satisfying en,s D minfen,1 , . . . , en,2n¡1 g and en,l D maxfen,1 , . . . , en,2n¡1 g; 3. Branch to the following four cases according to the values of en,s , en,l , an , and bn : case1. en,s ¸ bn or en,l · an : If

n X jD 1

If

n X jD 1

xj wj C s > 0 then I ¼ xj wj C s < 0 then I ¼

p X

(¡1) kI1 (¡k, s, A, B, W ) ;

kD 0 p X

(¡1) kC 1 I1 ( k, s, A, B, W ) ;

kD 1

case2. en,s ¸ an and en,l · bn : If

n X jD 1

I¼

xj wj C s < 0 in V1 and p X kD 1

C

n X j D1

xj wj C s > 0 in V2 then

(¡1) kC 1 |I2 ( k, s, A, B ¡ fbn g, W ) |

p X kD 0

(¡1) k |I2 (¡k, s, B, A ¡ fan g, W ) |I

MLP’s Sensitivity to Input Perturbation

If

n X j D1

I¼

xj wj C s > 0 in V1 and p X kD 0

C

n X jD 1

203

xj wj C s < 0 in V2 then

(¡1) k |I2 (¡k, s, A, B ¡ fbn g, W ) |

p X kD 1

(¡1) kC 1 |I2 ( k, s, B, A ¡ fan g, W ) |I

case3. en,s D an and en,l > bn or en,s < an and en,l D bn : Let C D fc1 , . . . , cn g D fx1 , . . . , xn¡1 gs [ fan g and D D fd1 , . . . , dn g D fx1 , . . . , xn¡1 gl [ fbn g; If cn D en,s ,

I¼

p X kD 0

C

¡ where

dh D

and

eh D

n X j D1

xj wj C s > 0 in V1 , and 0

n X jD 1

xj wj C s < 0 in V2 and V3 then

(¡1) k @|I3 (¡k, s, (C ¡ fcn g) [ fdn g, W ) | ¡

p X kD 1 p X

2n¡1 X 6 s&hD 6 l hD 1&hD

(¡1) kC 1 |I2 ( k, s, C, D ¡ fdn g, W ) | 0 (¡1) kC 1 @ |I3 ( k, s, D, W ) |

kD 1

2n¡1 X

1

eh A ,

6 s&h D 6 l hD 1&h D

8 |I3 (¡k, s, ( C ¡ fch , cn g) [ fdh , dn g, W ) |, > > > > p > X > > > (¡1) k |I3 (¡k, s, ( C ¡ fch , cn g) [ fdh , dn g, W ) | if > < > > > > > > > > > :

kD 0

< 0,

p X kD 0

(¡1) k |I3 (¡k, s, (C ¡ fcn g) [ fdn g, W ) |

otherwise

8 |I3 ( k, s, (D ¡ fdh g) [ fch g, W ) | > > > > p > X > > > (¡1) kC 1 |I3 ( k, s, ( D ¡ fdh g) [ fch g, W ) | if > < > > > > > > > > > :

k D1

< 0,

p X kD 1

(¡1) kC 1 |I3 ( k, s, D, W ) |

otherwise

1 dh A

204

X. Zeng and D. Yeung

If cn D en,s ,

I¼

p X kD 1

C

¡ where

j D1

xj wj C s < 0 in V1 , 0

n X jD 1

xj wj C s > 0 in V2 and V3 then

(¡1) kC 1 @|I3 ( k, s, (C ¡ fcn g) [ fdn g, W ) | ¡

p X kD 0 p X kD 0

2n¡1 X

1 dh A

6 s&h D 6 l hD 1&h D

(¡1) kI2 (¡k, s, C, D ¡ fdn g, W ) 0 (¡1) k @|I3 (¡k, s, D, W ) | ¡

1

2n¡1 X

eh A ,

6 s&h D 6 l hD 1&h D

8 |I3 ( k, s, ( C ¡ fch , cn g) [ fdh , dn g, W ) |, > > > > p > X > > > (¡1) kC 1 |I3 ( k, s, ( C ¡ fch , cn g) [ fdh , dn g, W ) | if > <

dh D

kD 1

> > > > > > > > > :

and

eh D

<

p X kD 1

0,

(¡1) kC 1 |I3 ( k, s, (C ¡ fcn g) [ fdn g, W ) |

otherwise

8 |I (¡ s, ( D ¡ fdh g) [ fch g, W ) |, > > 3 k, > > p > X > > > (¡1) k |I3 (¡k, s, (D ¡ fdh g) [ fch g, W ) | if > < > > > > > > > > > :

kD 0

<

p X kD 0

C

¡

p X

(¡1) k |I3 (¡k, s, D, W ) |

kD 0

0,

If dn D en,l ,

I¼

n X

otherwise n X jD 1

xj wj C s > 0 in V1 , 0

n X jD 1

xj wj C s < 0 in V2 and V3 then

(¡1) k @ |I3 (¡k, s, ( D ¡ fdn g) [ fcn g, W ) | ¡

p X kD 1 p X kD 1

2n¡1 X 6 s&h D 6 l hD 1&hD

(¡1) kC 1 |I2 ( k, s, D, C ¡ fcn g, W ) | 0 (¡1) kC 1 @|I

3 ( k,

s, C, W ) |

2n¡1 X 6 s&h D 6 l h D1&h D

1

eh A ,

1 dh A

MLP’s Sensitivity to Input Perturbation

205

where 8 |I3 (¡k, s, ( D ¡ fdh , dn g) [ fch , cn g, W ) |, > > > > p > X > > > (¡1) k |I3 (¡k, s, ( D ¡ fdh , dn g) [ fch , cn g, W ) | if > <

dh D

> > > > > > > > > :

kD 0

<

p X kD 0

0,

(¡1) k |I3 (¡k, s, (D ¡ fdn g) [ fcn g, W ) |

otherwise

and

eh D

8 ( ) |, |I ( > > 3 k, s, C ¡ fch g) [ fdh g, W > > p > X > > > (¡1) kC 1 |I3 ( k, s, ( C ¡ fch g) [ fdh g, W ) | if > < > > > > > > > > > :

k D1

p X kD 1

C

¡

(¡1) kC 1 |I3 ( k, s, C, W ) |

kD 1

0,

If dn D en,l ,

I¼

p X

<

otherwise n X j D1

xj wj C s < 0 in V1 , and

n X j D1

xj wj C s > 0 in V2 and V3 then

0 (¡1) kC 1 @|I3 ( k, s, ( D ¡ fdn g) [ fcn g, W ) | ¡

p X kD 0 p X kD 0

n¡1 2 X

6 s&hD 6 l h D 1&hD

(¡1) k |I2 (¡k, s, D, C ¡ fcn g, W ) | 0 (¡1) k @ |I3 (¡k, s, C, W ) | ¡

2n¡1 X

1

eh A ,

6 s&h D 6 l hD 1&hD

where

dh D

8 |I3 ( k, s, (D ¡ fdh , dn g) [ fch , cn g, W ) |, > > > > p > X > > > (¡1) kC 1 |I3 ( k, s, ( D ¡ fdh , dn g) [ fch , cn g, W ) | if > < > > > > > > > > > :

kD 1

< 0,

p X kD 1

(¡1) kC 1 |I3 ( k, s, ( D ¡ fdn g) [ fcn g, W ) |

otherwise

1 dh A

206

X. Zeng and D. Yeung

and

eh D

8 |I (¡ s, ( C ¡ fch g) [ fdh g, W ) |, > > 3 k, > > p > X > > > (¡1) k |I3 (¡k, s, ( C ¡ fch g) [ fdh g, W ) | if > < > > > > > > > > > :

kD 0

< 0,

p X

(¡1) k |I3 (¡k, s, C, W ) |

kD 0

otherwise

case4. en,s > an and en,l > bn or en,s < an and en,l < bn : Reduce to case1 and case3; case5. en,s < an and en,l > bn : Reduce to case2 and case3. 5 Computation of the Sensitivity for the Entire MLP

This section discusses the computation of the sensitivity for an entire MLP. From a global point of view, the sensitivity of an MLP reects its output deviation, or exactly its output layer ’s output deviation due to the Žrst layer’s input deviation. As the inputs of each neuron in a layer are the outputs of all neurons in its immediately preceding layer, the Žrst layer ’s input deviation will be propagated through all internal layers to inuence the output layer’s output. The sensitivity of the output layer can be computed neuron by neuron from the Žrst layer to the last layer. In more detail, S1 can be computed with D X1 and W 1 by NEURON SENSITIVITY(A, B, Q ( X1 ) , D X1 , Wi1 ) (1 · i · n1 ) , then S2 with D X2 D S1 and W 2 , and so on to, Žnally, SL with D XL D SL¡1 and WL . Logically, since the input space and the distribution of a neuron in layer l (2 · l · L ) may be different from the neurons in its previous layers, the input bounds A and B and the density function Q ( Xl ) for each layer l (2 · l · L ) need to be adjusted in the computation. An algorithm to compute the MLP’s sensitivity is given below. The expected input deviation D X1 , that is, S 0 , and the computed sensitivity Sl (1 · l · L ) will form a two-dimensional array. By controlling the mapping between the iterative time and the arrays’ subscript, it would be easier to compute the sensitivity for an MLP. Algorithm: MLP SENSITIVITY(A, B, Q ( X1 ) , D X1 , W ) 1. Put the given expected input deviation D X1 (i.e., S 0 ) and the weight set W into array forms; 2. Compute the sensitivity neuron by neuron and layer by layer: For l D 1 step 1 to L

MLP’s Sensitivity to Input Perturbation

207

If l > 1 then adjusting A, B, and Q ( Xl ) ; For i = 1 step 1 to nl Call NEURON SENSITIVITY(A, B, Q (Xl ) , Sl¡1 , Wil ) and save result sli ; 3. SL is the computed sensitivity for an MLP. In this algorithm, the adjustment of the input bounds for layer l (2 · l · L ) can be easily done with the bounds and the weight information of layer l ¡1 because of the linear feature of the summation of weighted inputs and the monotonic increasing feature of the sigmoid function. But the adjustment of the density function is generally very complex. A reasonable distribution function to be employed as an approximation is the uniform distribution. Other distribution functions would most likely make the integration of equation 4.5 virtually impossible. Our experimental results presented in section 6 support the use of uniform distribution as being a reasonably good one. 6 Experimental VeriŽcation

To verify the derived theoretical results, a number of experiments have been conducted. Again, a single neuron is considered Žrst. In the experiments, for simplicity and without losing the generality of the spatial dimension and direction, eight neurons with four different input dimensions from n D 2 to n D 5 and different weight values are used. Each neuron has either all positive weight values or separate positive and negative weight values. As for the case of the neurons with all negative weight values, it could be reduced to the case of all positive elements because of equation 2.3. Some weight values for these neurons are given in Table 1. Both theoretical computations and the computer simulations are based on the data given by this table. The theoretical results, that is, the sensitivities, for each neuron are computed with the algorithm NEURON SENSITIVITY(A, B, Q (X ) , D X, W ) with the parameters A D f0g, B D f1g, Q ( X ) D 1, and the elements of D X being all identical and ranging from 0 to 0.1 with a step size of 0.01. The simulation results, that is, the average output deviations for each neuron, are obtained with respect to more than 100 million inputs selected randomly and uniformly from the interval [0, 1]n (2 · n · 5) . Theoretical and simulation results are shown in Figures 13 through 16. It is clear that the agreement between the two results is very good. The experiments on the networks were carried out for three MLPs: 2 ¡2 ¡1, 2 ¡3 ¡1, and 2 ¡2 ¡2 ¡1. Their weight values are given in Table 2. Similar to the veriŽcation of individual neurons, the theoretical computations and the computer simulations were run with the given weight sets, and the expected input deviations ranged from 0 to 0.1 with a step size of 0.01. The sensitivity for each network is approximately computed by the algorithm MLP SENSITIVITY(A, B, Q ( X1 ) , D X1 , W ) with

208

X. Zeng and D. Yeung

Table 1: Weight Dimension and Weight Values for Neurons. Parameter

n

w(1)

w(2)

Neuron 1 Neuron 2 Neuron 3 Neuron 4 Neuron 5 Neuron 6 Neuron 7 Neuron 8

2 2 3 3 4 4 5 5

1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0

2.0 ¡2.0 2.0 ¡2.0 2.0 ¡2.0 2.0 ¡2.0

w(3)

3.0 3.0 3.0 3.0 3.0 3.0

w(4)

4.0 ¡4.0 4.0 ¡4.0

w(5)

5.0 5.0

Figure 13: Sensitivity of the neurons with two-dimensional weight to expected input deviation.

A D f0g, B D f1g, Q (X1 ) D 1. They are not adjusted during the computations. The average output deviations for each network are with respect to more than 100 million inputs selected randomly and uniformly from the interval [0, 1]2 . The theoretical and simulation results are shown in Figure 17. 7 Conclusion

We have proposed a quantiŽed measure and its computation for the sensitivity of the MLP to its input perturbation. The theoretical results obtained

MLP’s Sensitivity to Input Perturbation

209

Figure 14: Sensitivity of the neurons with three-dimensional weight to expected input deviation.

Figure 15: Sensitivity of the neurons with four-dimensional weight to expected input deviation.

210

X. Zeng and D. Yeung

Figure 16: Sensitivity of the neurons with Žve-dimensional weight to expected input deviation. Table 2: Three Sets of Weight Values.

W

2 ¡2 ¡1 2 ¡3 ¡1 2 ¡2 ¡2 ¡1

w(1I 1, 1) w(2I 1, 1) w(3I 1, 1)

w(1I 1, 2) w(2I 1, 2) w(3I 1, 2)

w(1I 2, 1) w(2I 1, 3)

w(1I 2, 2) w(2I 2, 1)

¡5.0 ¡1.0

6.0 2.0

¡3.0

4.0

1.0 7.0

¡2.0 ¡8.0

3.0 9.0

¡4.0

10.0 ¡5.0 ¡1.0

¡9.0 6.0 2.0

8.0 ¡3.0

¡7.0 4.0

w(1I 3, 1) w(2I 2, 2)

w(1I 3, 2)

5.0

¡6.0

are in reasonable agreement with the simulation results. The sensitivity measure is something like an analog rule that network users can use as a tool to evaluate their networks’ performance. For example, Lee and Oh (1994) indicated that in pattern classiŽcation applications, their sensitivity (misclassiŽcation probability) result could be applied to select the optimal weight set among many weight sets acquired through a number of learning

MLP’s Sensitivity to Input Perturbation

211

Figure 17: Sensitivity of the three MLPs to expected input deviation.

trials. Among these weight sets, an MLP with the optimal set will have the best generalization capability and the best input noise immunity. Zurada, Malinowski, and Usui (1997) made use of analytical sensitivity to delete redundant inputs to an MLP for pruning its architecture. Engelbrecht and Cloete (1999) also used analytical sensitivity to dynamically select training patterns. Some theoretical issues as well as applications of the MLP, such as improving error tolerance, measuring generalization ability, and pruning the network architecture, would beneŽt from this study. Acknowledgments

This research work is supported by a Hong Kong CERG 5114/97E. References Alippi, C., Piuri, V., & Sami, M. (1995). Sensitivity to errors in artiŽcial neural networks: A behavioral approach. IEEE Transactions on Circuits and Systems-I: Fundamental Theory and Applications, 42(6), 358–361. Cheng, A. Y., & Yeung, D. S. (1999). Sensitivity analysis of neocognitron. IEEE Transactions on System, Man, and Cybernetics—Part C:Applications and Reviews, 29(2), 238–249.

212

X. Zeng and D. Yeung

Choi, J. Y., & Choi, C. H. (1992).Sensitivity analysis of multilayer perceptron with differentiable activation functions. IEEE Transactions on Neural Networks, 3(1), 101–107. Engelbrecht, A. P., & Cloete, I. (1999). Incremental learning using sensitivity analysis. In Proceedings of IEEE IJCNN’99 (Vol. 2, pp. 1350–1355). Washington, DC. Engelbrecht, A. P., Fletcher, L., & Cloete, I. (1999). Variance analysis of sensitivity information for pruning multilayer feedforward neural networks. In Proceedings of IEEE IJCNN’99 (Vol. 3, pp. 1829–1833). Washington, DC. Fu, L., & Chen, T. (1993). Sensitivity analysis for input vector in multilayer feedforward neural networks. In Proceedings of IEEE International Conference on Neural Networks (Vol. 1, pp. 215–218). San Francisco. Hashem, S. (1992). Sensitivity analysis for feedforward artiŽcial neural networks with differentiable activation functions. In Proceedings of IJCNN’92 (Vol. 1, pp. 419–424). Baltimore, MD. Lee, Y., & Oh, S. H. (1994). Input noise immunity of multilayer perceptrons. ETRI Journal, 16(1), 35–43. PichÂe, S. W. (1995). The selection of weight accuracies for madalines. IEEE Transactions on Neural Networks, 6(2), 432–445. Stevenson, M., Winter, R., & Widrow, B. (1990). Sensitivity of feed forward neural networks to weight errors. IEEE Transactions on Neural Networks, 1(1), 71–80. Yeung, D. S., & Sun, X. (2002). Using function approximation to analyze the sensitivity of MLP with antisymmetric squashing activation function. IEEE Transactions on Neural Networks, 13(1), 34–44. Zeng, X. (2001). Output sensitivity of MLPs derived from statistical expectation. Unpublished doctoral dissertation, Hong Kong Polytechnic University. Zeng, X., & Yeung, D. S. (2001). Sensitivity analysis of multilayer perceptron to input and weight perturbations. IEEE Transactions on Neural Networks, 12(6), 1358–1366. Zurada, J. M., Malinowski, A., & Usui, S. (1997). Perturbation method for deleting redundant inputs of perceptron networks. Neurocomputing, 14, 177–193. Received July 24, 2001; accepted May 30, 2002.

LETTER

Communicated by Hagai Attias

Variational Mixture of Bayesian Independent Component Analyzers R. A. Choudrey [email protected] S. J. Roberts [email protected] University of Oxford, Robotics Research, Oxford, U.K. There has been growing interest in subspace data modeling over the past few years. Methods such as principal component analysis, factor analysis, and independent component analysis have gained in popularity and have found many applications in image modeling, signal processing, and data compression, to name just a few. As applications and computing power grow, more and more sophisticated analyses and meaningful representations are sought. Mixture modeling methods have been proposed for principal and factor analyzers that exploit local gaussian features in the subspace manifolds. Meaningful representations may be lost, however, if these local features are nongaussian or discontinuous. In this article, we propose extending the gaussian analyzers mixture model to an independent component analyzers mixture model. We employ recent developments in variational Bayesian inference and structure determination to construct a novel approach for modeling nongaussian, discontinuous manifolds. We automatically determine the local dimensionality of each manifold and use variational inference to calculate the optimum number of ICA components needed in our mixture model. We demonstrate our framework on complex synthetic data and illustrate its application to real data by decomposing functional magnetic resonance images into meaningful—and medically useful—features. 1 Introduction

The goal of pattern analysis and recognition is to extract information from some data. In order for this information to be useful, the distribution of data must be represented in some meaningful way. In many cases, insight may be gained by dividing the data into self-similar areas and analyzing each of these clusters under some informative framework, for example, using some understanding of the assumed data-generating process. One such method is to model the data as being produced by a mixture of data generators (also called analyzers), where each component generator is responsible for generating a particular cluster. The problems to overcome in this mixture Neural Computation 15, 213–252 (2003)

° c 2002 Massachusetts Institute of Technology

214

R. Choudrey and S. Roberts

modeling are to decide how many generators are needed, where to place them, and how to adjust them to best represent the data. Mixtures of gaussians (MoG) are widely used throughout the Želds of machine learning and statistics for data modeling, where each generator is a gaussian density. Despite their popularity, however, MoGs suffer from two serious drawbacks. The Žrst is that as the dimensionality S of the problem space increases, the size of each covariance matrix, S2 , becomes prohibitively large. This can be dealt with by assuming isotropic gaussians (i.e., ignoring the covariance structure), but this greatly reduces the exibility of the model class. This problem has been solved by Tipping and Bishop (1999), who replaced each gaussian with a probabilistic principal component analyzer (PCA). This allowed the dimensionality of each covariance to be effectively reduced while maintaining the richness of the model class. This was formulated under a maximum-likelihood framework, which, although efŽcient, does not allow one to infer the optimum number of generators needed. This mixture was modiŽed into a mixture of factor analyzers (FA) by Ghahramani and Beal (2000), where Bayesian inference was used to infer the optimum number of analyzers. The second problem with MoGs is that each component is a gaussian, a strong assumption that is often violated in many natural clustering problems (Lee, Lewicki, & Sejnowski, 2000). Although MoGs are capable of modeling any distribution given enough components, the problem still remains of automatically grouping gaussians that together describe some larger-scale feature. It is this second problem that we address in this article. A solution is reached by extending the mixtures of probabilistic PCA / FA model to an independent component analysis (ICA) mixture model. We improve on previous work on mixtures of ICA by Lee et al. (2000) and Penny and Roberts (2001) by incorporating a very exible ICA model that can generate arbitrary densities using MoGs and by bringing the formalism into the Bayesian arena. We use Bayesian inference to infer the optimum number of ICAs needed and to determine their ideal dimensionalities automatically. 1.1 ICA. Independent component analysis seeks to extract salient features and structure from data assumed to be a linear mixture of independent underlying (hidden) features

x D As,

(1.1)

where x is an observed S-dimensional data vector, s is the L-dimensional source vector, and A is the SxL bases or mixing matrix.1 An ICA model formulated in this way is well posed only if L · S; otherwise, there are effectively more unknowns than equations in the model. The overcomplete 1 The term mixing refers to the linear mixing in the ICA model; mixture refers to the stochastic generating method in a mixture model.

215

S o u rc e 2

S e n so r 3

Variational Mixture of Bayesian Independent Component Analyzers

Source 1

Sensor 2

Sensor 1

Figure 1: The underlying and observed data densities.

problem (L ¸ S) is an area of ongoing research and will not be dealt with here. Which of A or s represents the features depends on one’s view of the data. If the data are considered to be constructed from varying amounts of L static features, then the columns of A represent the L static basis vectors and s represents the amount of each basis used for a given data vector. For example, if the data represent an image, that image may be considered a mixture of underlying (maybe fewer) independent edges, textures, and so forth. The aim of ICA in this case is to “unmix” the data set and recover these representative features. If the data are time varying, they can be seen as a linear mixing of underlying (possibly lower-dimensional) signals represented by s. For example, the data may be signals received at S microphones at a cocktail party. These signals are then simply mixings of L people talking. ICA is used to blindly (i.e., without access to s and A) separate the sensor signals into the underlying source signals s. This is known as blind source separation (BSS) in the signal processing literature. In either case, ICA can be recast as data density modeling. The data density p ( x) is a linear transform (i.e., scaling, rotation and possibly higher-dimensional projection) of an unknown, nongaussian manifold—the source density p (s ) (see Figure 1). The goal of ICA is to recover the source density and estimate the transform matrix. ICA has traditionally been performed in the noiseless limit (Bell & Sejnowski, 1995; Lee, Girolami, Bell, & Sejnowski, 1998), with limited exibility in the source density model and with noise often being dealt with as an extra source. More recently, however, Attias (1999b) extended ICA by using a more exible source model and incorporating full covariance noise explicitly into the ICA framework. The model, which Attias dubbed a independent factor analysis (IFA), was subsequently learned through a maximum likelihood expectation-maximization (EM) algorithm. Bayesian

216

R. Choudrey and S. Roberts

formalisms have also been introduced, most notably (under a variational framework) by Attias (1999a), Lappalainen (1999), Choudrey, Penny, and Roberts (2000), and Miskin and MacKay (2000). For a comprehensive review of ICA, see Hyv¨arinen (1999) or Roberts and Everson (2001). 1.2 ICA Mixture Model. ICA was originally developed as a solution to the BSS problem in auditory signal analysis by HÂerault and Jutten (1986). Since then, the BSS approach has been extended to human electroencephalograms (EEGs) (Makeig, Bell, Jung, & Sejnowski, 1996), Žnancial data analysis (Back & Weigend, 1998), and telecommunications (Ristaniemi & Joutsensalo, 1999), among many other applications. Recently, however, ICA’s remit has broadened to one of representation in general. The way data are presented very much inuences what patterns are found and how much information can be extracted from them. A primary goal in pattern recognition, then, is to Žnd some intrinsic coordinate system from which structure is easier to extract. In this sense, ICA is a higherorder extension of second-order methods such as PCA and FA. PCA/FA Žnd representations of data that preserve the maximum amount of variance by effectively Žnding an intrinsic coordinate system for the covariance matrix. Projections of the data on to the bases found lead to decorrelation. ICA seeks a loftier goal: a representation in which the data are maximimally statistically independent. In this way, ICA aims to repackage data, which may contain structure across its constituent dimensions, into independent signals that contain structure only within components, thereby making patterns (hopefully) more cogent. This application of ICA has led to interesting work in representation of text corpora (Isbell & Viola, 1999; Kolenda, Hansen, & Sigurdsson, 2000), of images such as face representation (Bartlett, Lades, & Sejnowski, 1998), and of natural scenes (Bell & Sejnowski, 1997). This, in turn, has hinted at possible links between ICA and coding schemes in the primary visual cortex (Olshausen & Field, 1997; Lewicki & Olshausen, 1999), although we stress that this connection has generated some controversy (van Hateren & van der Schaaf, 1998). It is this representational viewpoint that motivates our extension of ICA into a mixture of ICAs. Decomposing and representing data using ICA assumes the whole data distribution is adequately described by one coordinate frame. This may not be appropriate for many problems, however. In the BSS problem, ICA assumes each data signal carries a constant mixture of source signals. Consider the scenario proposed by Lee, Lewicki, and Sejnowski (1999). There are two people talking to each other, but never at the same time, and there is music in the background. This cacophany is picked up by two microphones. At any one time, the microphones pick up a mixture of one voice and the music or the other voice and the music, but never all three at the same time. Clearly in this case, standard ICA is an inadequate model. As Lee et al. (1999) showed, a mixture of ICAs is a more appropriate generative model. More generally, if the sensor density in Figure 1 consists of

Variational Mixture of Bayesian Independent Component Analyzers

217

various self-similar, nongaussian manifolds, enforcing a single global representation is not appropriate and will produce a suboptimal representation. A more intuitive way of representing the data is the use of a number of local coordinate frames, each constructed under local conditions. This leads to a mixture of independent component analyzers. ICA mixture models were Žrst formulated by Lee et al. (1999) using the extended Infomax algorithm (Lee, Girolami, & Sejnowski, 1999) to switch the source model between subgaussian and supergaussian regimes. The model was learned through a combination of maximum likelihood and gradient ascent. Although well demonstrated, the source model could switch only between Laplacian and bimodal densities and thus lacked exibility. Penny and Roberts (2001) relaxed this constraint by using generalized exponential sources that can model a wide variety of kurtoses by the adjustment of a parameter. A basic method of model selection was also incorporated using the Bayesian information criterion (BIC) to infer the number of hidden sources (i.e., the intrinsic dimensionality of the local manifold). Due to the problem formulation used, only two-dimensional or higher manifolds could be modeled. Although more exible, the densities could only be unimodal, and the learning scheme was also maximum likelihood. In this article, we present an ICA mixture model trained using Bayesian methods. In line with Attias (1999b), Lappalainen (1999), Choudrey et al. (2000), and Miskin and MacKay (2000), we choose a fully adaptable factorial MoG as the source model for our ICA components, allowing us to recover arbitrary source densities. Essentially, each ICA component will model selfsimilar areas (the features) as a mixture of gaussian subfeatures. To overcome the heavy computational load associated with Bayesian learning, we use the variational framework reviewed by Jaakkola and Jordan (2000) to make assumptions about the posterior, giving tractability to the Bayesian model. The variational framework is a whole class of methods for bounding intractable integrals. In particular, we use free-energy minimization (Hinton & van Camp, 1993) or ensemble learning (MacKay, 1995a), a variational method for Bayesian parameter estimation. We further extend the ensemble learning results of Choudrey et al. (2000) and Miskin and MacKay (2000) by using an MoG posterior density in a similar fashion to Attias (1999b)—rather than a single gaussian—letting us capture much richer posteriors. This, in turn, leads to greater robustness under uncertainty (Choudrey & Roberts, 2001). The variational Bayesian method is carried through to the ICA mixture model, allowing model comparison, incorporation of prior knowledge of the mixture process, and—by integrating out nuisance parameters— control of model complexity and thus overŽtting. By monitoring the variational free energy of our models, we can compare model assumptions, allowing us, for example, to infer the optimum number of components needed in our ICA mixture model (Attias, 1999a). We also employ automatic relevance determination (ARD) (MacKay, 1995b) to suppress unsupported sources and thus effectively infer the number of latent dimensions of each

218

R. Choudrey and S. Roberts

ICA component as part of the learning process. This leads to the variational Bayesian independent component analyzers mixture model (vbICA-MM). 2 The Model

The probability of generating a data vector xn from a C-component mixture model given assumptions M is p ( xn | M ) D

C X

p ( c | M 0 ) p (xn | Mc , c ).

(2.1)

cD 1

A data vector is generated by choosing one of the C components stochastically under p (c | M0 ) and then drawing from p ( xn | Mc , c) . M D fM 0 , M1 , . . . , MC g is the vector of component model assumptions, Mc , and assumptions about the mixture process, M 0 . The assumptions represent everything that essentially deŽnes the model: values of Žxed parameters, model structure, details of the component switching method, any prior information, and so forth. p ( xn | M ) is known as the evidence for model M and quantiŽes the likelihood of the observed data under model M. The variable c indicates which component of the mixture model is chosen to generate a given data vector x. If p (c | M 0 ) is a vector of probabilities and each component p ( xn | Mc , c) is a gaussian, then equation 2.1 simply describes an MoG. If the MoG is adapted through a maximum likelihood approach, then M represents a list of point estimates for the corresponding parameters. In our mixture model, however, each component has a nongaussian density derived from the ICA model presented in the next section, and M represents assumptions concerning the distribution of possible parameter values. Figure 2 shows a generative model for a data vector x. 2.1 The ICA Model. In common with ICA in the literature, we choose a generative model to work with. The observed variables, x, of dimension S are modeled as a linear combination of statistically independent latent variables, sc , of dimension Lc with added gaussian noise,

x D A c sc C y c C ec

(2.2)

where y c is an S-dimensional bias vector, ec is S-dimensional additive noise, and c represents the cth ICA model. In signal processing nomenclature, S is the number of (observed) sensors, and Lc is the number of latent (hidden) sources. Equation 2.2 acts as a complete description for cluster c in the data density. The bias vector, y c , deŽnes the position of the cluster in the S-dimensional data space, Ac describes its orientation, and sc describes the underlying manifold. Using ICA, data produced by mixing gaussian sources cannot be

Variational Mixture of Bayesian Independent Component Analyzers

M1

M2

MC

ICA 1

ICA 2

ICA C

x

c

219

M0

Figure 2: ICA mixture model.

separated as they only have 2-moment distributions, so can never be more than decorrelated. The noise, ec , is assumed to be zero-mean gaussian and isotropic, p (ec | 0, L c , c) D N

( ec I 0, lc I ) ,

(2.3)

where lc I is the precision.2 The noise essentially absorbs any (spherical) gaussianity present in the cluster. The probability of observing data vector xn under component c is then given by ³ p ( x n | hc , c ) D

lc 2p

´S 2

exp[¡Ec ],

(2.4)

where hc D fAc , snc , ¸c g and

lc n ( x ¡ Ac snc ¡ y c ) T ( xn ¡ Ac snc ¡ y c ) . 2 Since the sources sc D fsc,1 , .., sc,i , .., sc,Lc g are by deŽnition mutually independent, the distribution over sc for data point n can be written as Ec D

p (snc | Msc , c) D

Lc Y

p ( snc,i | Msc, i , c) ,

(2.5)

iD 1

2 Conventional notation speciŽes covariance rather than precision. We have found precisions easier to manipulate in the math, so have abused the notation somewhat to make subsequent equations easier to read.

220

R. Choudrey and S. Roberts

where the product runs over the Lc sources of component c and Msc is the vector of source model assumptions. p (snc | Msc , c) is the source model for ICA component c. Traditionally a reciprocal-cosh (Bell & Sejnowski, 1995) or unimodal density (Miskin & MacKay, 2000) has been used. These, however, have limited exibility in kurtosis representation or capturing multiple modes. Methods have been proposed for switching between super- and subGaussian regimes (Girolami, 1998; Lee et al., 2000; Lee, Lewicki et al., 1999), although these depend on heuristic stability analyses. More recently, Attias (1999b) introduced an MoG source model into his ICA formalism, potentially allowing any distribution to be modeled. This is the route we take. 2.2 ICA Source Model. The choice of a exible and mathematically attractive (tractable) source model is crucial if a wide variety of source distributions are to be modeled; in particular, the source model should be capable of encompassing both super- and subgaussian distributions (distributions with positive and negative kurtosis respectively) and complex multimodal distributions. One such distribution is a factorized mixture of one-dimensional gaussians with Lc factors (i.e., sources) and mi components per source (see Figure 3),

p ( snc | ’c , c) D D

Lc X mi Y i D 1 qi D 1 Lc X mi Y i D 1 qi D 1

p ( qni D qi | ¼ i , c) p ( snc,i | Qc,i , c) p i,qi N

(snc,i I m i,qi , bi,qi ) ,

(2.6)

where, for brevity, the ICA component subscript c has been dropped from parameters that can be seen to belong to ICA c from context. From now on, all subscripted parameters should be assumed to belong to the cth ICA model unless otherwise stated. Equation 2.6 essentially describes the local features of cluster c: m i,qi is the position of feature qi with regard to the cluster center, b i,qi is its size, and p i,qi its prominance with regard to other features. The mixture proportions p i,qi D p ( qni D qi | ¼ i ) are the prior probabilities of choosing component qi of the ith source (of the cth ICA model). qni is a variable indicating which component of the ith source is chosen for generating snc,i and takes on values of fqi D 1, . . . , qi D mi g (where mi , of course, depends on ICA model c). The mean and precision of gaussian qi in source i are m i,qi and bi,qi respectively. The parameters of source i are Qc,i D f¼ c,i , ¹c,i , ¯ c,i g where boldface type indicates the vector of mi parameters. The complete parameter set of the source model is ’c D fQc,1 , Qc,2 , . . . , Qc,Lc g.

Variational Mixture of Bayesian Independent Component Analyzers

M s,1

M s,2

Gaussian1

221

M s,m

Gaussian2

Gaussianm

s

q

M s,0

Figure 3: MoG source model.

The complete collection of possible Qsource states is denoted q c D fq c,1 , q c,2 , . . . , q c,m g and runs over all m D i mi possible combinations of source states. The probability of state q nc being chosen and generating source vector snc is p (snc , q nc | ’c , c) D

Lc Y iD 1

p ( qni D qi | ¼ i , c ) p ( snc,i | qi , m i,qi , bi,qi , c)

(2.7)

D p ( q nc | ¼ c , c ) p ( snc | q nc , ’c , c ) ,

where ¼ c D f¼ c,1 , ¼ c,2 , . . . , ¼ c,Lc g (similarly for ¹ and ¯). Note that the product of Lc one-dimensional MoGs in equation 2.6 is equivalent to a single MoG in Lc -dimensional space with m states. By integrating and summing over the hidden variables, fsc , q c g, the likelihood of the IID data X D fx1 , x2 , . . . , xN g given the model parameters H c D fAc , y c , lc , ’c g can now be written as p (X | H c , c) D

N X m Z Y

p ( xn , snc , q nc | Hc , c) dsc ,

(2.8)

n D 1 qD 1

Q where dsc D i ds c,i . If we stipulate a form for p (c | M 0 ) in equation 2.1, p (X | M ) D

C X c D1

p ( c | ·) p ( X | H c , c ) ,

(2.9)

222

R. Choudrey and S. Roberts

where p (c | ·) D fp(c D 1) D k1 , p ( c D 2) D k2 , . . . , p (c D C) D kC g, then equations 2.4 through 2.5 and 2.6 can be substituted into equation 2.9 to yield a maximum likelihood model. This can then be learned through an iterative process such as gradient descent (Lee et al., 2000) or the EM algorithm (Penny & Roberts, 2001). In this article, however, we take the Bayesian route by also integrating out the parameters f·, H c g in equation 2.9. 3 Bayesian Inference and Variational Learning

The maximum likelihood approach to learning the parameters of a model is well documented (for an introduction, see Pearlmutter & Parra, 1996; Cardoso, 1997; and Attias, 1999b), as are the pitfalls (e.g., overŽtting, no quantiŽcation of uncertainty in the model or model comparison). We choose to take the Bayesian approach and integrate out the parameters f·, H c g and hidden variables fsc , q c g. This allows us to take our assumptions one stage higher; rather than assume point estimates for the parameters (very harsh and speciŽc), we assume prior distributions over all possible parameter values. This is much more exibile because it allows us to tailor the distributions to avoid overŽtting the data, reduce the parameter search space to reasonable values, and code our inevitable uncertainty in the parameters into the model. First, we will state the prior distributions over the hidden variables and model parameters. 3.1 The Priors. The priors are chosen to be appropriately conjugate to allow tractibility. All boldface parameters without sub- or superscripts indicate the collection of C parameter vectors. The prior over the ICA mixture indicator variables, c D fc1 , c2 , . . . , cN g, simply factorizes over the N data vectors:

p ( c | ·) D

N Y

k cn .

(3.1)

nD 1

The prior over the ICA mixture coefŽcients k is a Dirichlet p ( ·) D

P C Y C ( c0 ic0 ) ic ¡1 kc . C (ic ) c D1

(3.2)

Because of source independence, it follows that the distribution over the MoG component indicator variables, p (q c | ¼c , c) , is a product over all p i,qni where i indexes the sources and n the data: p ( q c | ¼ c , c) D

Lc N Y Y nD 1 iD 1

p i,qni .

(3.3)

Variational Mixture of Bayesian Independent Component Analyzers

223

For a given state q , the distribution over the sources is p ( sc | q c , ’ c , c ) D

N Y Lc Y

( snc,i I m i,qi , bi,qi ) .

N

nD 1 iD 1

(3.4)

The prior over the source model (MoG) parameters is a product of priors over ¼ c , ¹c , ¯c : p (’) D

C Y

p ( ¼ c ) p (¹c ) p ( ¯ c ).

(3.5)

cD 1

The prior over the mixture proportions, ¼ c , for the ith source is a Dirichlet with parameter ri,qi : p (¼ ) D

C Y Lc Y mi C ( Y

P q0i

ri,q0i )

c D 1 i D1 qi D 1

C (r i,qi )

r i,q ¡1

p i,qi i

.

(3.6)

The prior over each MoG mean, m i,qi , is a gaussian with mean mi0 and precision ti0 , and the prior over the associated precision, bi,qi , is a gamma with width and scale parameters bi0 and ci0, respectively: p (¹ ) D p(¯ ) D

Lc Y mi C Y Y cD 1 i D 1 q i D 1 Lc Y mi C Y Y cD 1 i D 1 q i D 1

(m i,qi I mi0 , ti0 )

N

G (b i,qi I bi0 , ci0 ) ,

(3.7)

(3.8)

where we deŽne the gamma distribution as

G ( wI b, c) D

± w² 1 wc¡1 exp ¡ . C ( c) bc b

(3.9)

The prior over the bias vector, y c D fy1 , y2 , . . . , yS g, is a product over S zero-mean gaussians with precision tyj : p (y ) D

C Y S Y cD 1 jD 1

N

( yc, j I 0, tyj ) .

(3.10)

The prior over the sensor noise precision, lc , is a gamma distribution with width and scale parameters blc and clc : p (¸ ) D

C Y cD 1

G (lc I blc , clc ) .

(3.11)

224

R. Choudrey and S. Roberts

The prior over each element of the mixing matrix, Aji , is a zero-mean gaussian with precision ai for each column: p ( A) D

Lc Y C Y S Y

N

(aji | 0, ai ) .

(3.12)

c D 1 i D1 j D1

By monitoring the precisions ®c D fa1 , . . . , aLc g, the relevance of each source may be automatically determined (ARD). If ai is large, column i of Ac will be close to zero, indicating source i is irrelevant. Finally, the prior over each ai is a gamma(bai , cai ): p (®) D

Lc C Y Y cD 1 i D 1

D G (ac,i I bai , cai ) .

(3.13)

Figure 4 shows the graphical model for the variational Bayes ICA (vbICA) model. Bayesian inference in such a model is computationally intensive and often intractable. An important and efŽcient tool in approximating posterior distributions is the variational method (see Jordan, Ghahramani, Jaakkola, & Saul, 1999, for an excellent tutorial). In particular, we take the variational Bayes approach detailed by Attias (1999a) and Jaakkola and Jordan (2000). 3.2 Variational Bayesian Learning. Consider the log evidence for

data X, log p ( X ) D log

p (X , W ) , p (W | X )

(3.14)

which simply follows from Bayes’ rule and where the explicit dependence on model assumptions has been dropped for brevity. The term W is the vector of all hidden variables and unknown parameters. Because the log evidence does not depend on W , this can be rewritten as Z

p0 ( W ) p ( X , W ) dW p0 ( W ) p (W | X ) Z Z p(X , W ) p0 ( W ) C p0 ( W ) log D p0 ( W ) log d W dW p0 ( W ) p (W | X )

log p ( X ) D

p0 ( W ) log

0 D F[W ] C KL[p kp],

(3.15)

where p0 ( W ) is some approximation to the posterior p ( W | X ) and F[W ] D hlog p ( X , W ) ip0 ( W ) C H[p0 ( W ) ] Z p0 ( W ) 0 KL[p kp] D p0 ( W ) log dW . p (W | X )

(3.16) (3.17)

Variational Mixture of Bayesian Independent Component Analyzers

m10 t10 r1,q

m1,1

1

m1,m

q1

p1

mL0tL0

b10 c10 b1,1

1

b1,m

1

s1

rL,q

L

pL

mL,1

225

bL0 cL0

mL,m

L

qL

bL,m

bL,1

L

sL

ba1 ca1 a11

a1

l

x1

l

x2

l

xS

bl cl

y1

bl cl

y2

bl cl

yS

ty1

ty2

tyS

Figure 4: Graphical model of vbICA. Circles represent random variables, and boxes denote parameters not governed by distributions (initially set by hand).

H [p0 (W ) ] is the entropy of p0 ( W ) . The Žrst term in equation 3.15, F, is known as the mean-Želd bound to the log evidence. Alternatively, ¡F is called the Helmholtz energy or variational free energy. The second term is the Kullback-Leibler divergence (KL divergence), a pseudodistance that measures the difference between two densities. This term is strictly positive, which means that F is a strict lower bound on the log evidence. By maximizing F, not only do we minimize the KL divergence between the approximating and true posterior, we also implicitly integrate out the unknowns W . By choosing an appropriate form for the approximation p0 ( W ), we perform tractable Bayesian learning. As F is a strict lower bound to the model (log) evidence, a wide variety of models and assumptions can be compared and contrasted by calculating F for each model. The higher F is, the higher the likelihood of the data under that model, and therefore the better that model is at explaining the data. 3.3 Variational Learning for vbICA-MM. In our model, W D fc, s, q , ·, £ g. By choosing p0 ( W ) such that it factorizes, terms in each hidden vari-

226

R. Choudrey and S. Roberts

able can be maximized individually. We choose the following factorization, p0 ( W ) D p0 ( c) p0 ( sc | q c , c) p0 ( q c | c) p0 (· ) p0 (y ) £ p0 (¸ ) p0 ( A ) p0 ( ® ) p0 ( ’ ) ,

(3.18)

where p0 ( ’ ) D p0 (¼ ) p0 (¹ ) p0 (¯ ) and p0 ( a | b) is the approximating density of p ( a | b, X ). We will also stipulate that the posteriors over the sources factorize such that p0 ( sc , q c | c) D

Lc Y

p0 (qi | c) p0 (sc,i | qi , c).

(3.19)

i D1

This additional factorization allows efŽcient scaling of computation with the number of hidden sources, with little loss of accuracy (Miskin & MacKay, 2000). The term p0 ( sc,i | qi , c) in equation 3.19 implies a mixture posterior source density for source i in ICA component c, p0 ( snc,i | c) D D

mi X qi D 1 mi X qi D 1

p0 (qni D qi | c) p0 (snc,i | qni , c) n cOi,q N i

n ) ( snc,i I mO ni,qi , bOi,q , i

(3.20)

(3.21)

where a posterior MoG density follows from from Bayes’ rule. By substituting p ( X , W ) and equation 3.18 into 3.16, we obtain expressions for the bound, F, to our model Ftot D Fmix C

C X

FICAc ,

(3.22)

cD 1

where Fmix D F[c, ·] FICAc D F[sc , q c , ¸c , Ac , ®c , ’c ].

(3.23)

The terms in equation 3.23 can be further factorized into contributions from each parameter. By monitoring subsets of F, we can infer the effect of local assumptions, for example, FICAc can be used instead of ARD to infer the most likely number of sources in ICA component c. One may now proceed by specifying functional forms of each of the approximating posteriors and using these in equation 3.16, as shown by Lappalainen (1999). As shown by MacKay (1995a), however, there is no

Variational Mixture of Bayesian Independent Component Analyzers

227

need to specify functional forms for (all) the approximating posteriors. The functional forms “fall out” of the maximization process, helped by the factorized form of p0 ( W ) and the conjugacy of the priors. The optimal form for each posterior is simply given by p0 (W k ) / p ( W k ) exp[hlog p (X , W ) iQ

lD 6 k

p0 ( Wl )

],

(3.24)

where the index k refers to the kth parameter in W . If p0 ( sc | q c , c) D p0 ( sc ), free-form optimization gives the ICA algorithms Žrst presented by Choudrey et al. (2000). This factorization gives a gaussian posterior over s. Using equation 3.18, however, is less restrictive and, as discussed below, more robust for an ICA mixture model. The posterior expectations of the data likelihood, equation 2.3, under the mixture density, equation 3.21, are given by hsnc,i | ci D hsnc,i 2 | ci D

mi X qi D 1 mi X qi D 1

p0 ( qni D qi | c)hsc,i n | qi , ci

(3.25)

2 p0 ( qni D qi | c)hsnc,i | qi , ci,

(3.26)

where n p0 (qni D qi | c) D cOi,q i

(3.27)

hsnc,i | qni , ci D mO ni,qi

(3.28)

2 hsnc,i | qni , ci D (mO ni,qi ) 2 C

1 . n O b i,q i

(3.29)

The advantages of an MoG posterior over the sources instead of a single gaussian (Choudrey et al., 2000; Miskin & MacKay, 2000) are twofold. First, it allows us to capture arbitrary densities, something not possible under a gaussian posterior. Second, an MoG posterior is more robust under uncertainty, especially in the context of an ICA mixture model. To understand this more clearly, consider the update equations for p0 ( snc,i | c) and p0 (snc,i | qni , c ) presented below (see the appendix for notation), mO ni /

mi X qi D 1

n cOi,q hbi,qi ihm i,qi i C gO cn hlc i i

mO ni,qi / hbi,qi ihm i,qi i C gO cn hlc i

S X jD 1

S X jD 1

n | ci¡hyj i) haji i(xjn ¡hxOj,k 6 i D

n | q nk , ci ¡ hyj i) , haji i(xjn ¡ hxOj,k 6 i D

(3.30)

(3.31)

228

R. Choudrey and S. Roberts

where the explicit dependence on c has been dropped for brevity. Equation 3.30 is the update under a gaussian source posterior (posterior gaussian mean), and equation 3.31 is the update under an MoG posterior (posterior mean for component qi ). The important feature to note is that the updates consist of prior (i.e., current) information from the MoG source model plus new data information. This information is combined and fed back up to the MoG source models. The MoGs then use this information to update their parameters. In uncertain situations (i.e., high noise), hlc i tends toward 0, drastically downweighting the data term. In the gaussian posterior case, the MoG source model gets fed a weighted average across MoG components of the current component parameters; this leads the MoG components to update their parameters toward common values. If there are few data or little data support, the MoGs will evolve toward the centroids of their respective densities rather than staying static. Over a number of iterations, they will effectively become the same single gaussian. If the source posterior is itself an MoG, then each component of the posterior MoG is responsible for its equivalent in the source prior MoG. In equation 3.31, as hlc i ! 0, separate information is fed back to each component qi —essentially current parameter values. In this case, if there are few data or little data support, the source MoGs will remain static. The difference is more acute in an ICA mixture model context. As well as the noise estimate, hlc i, the data term is also weighted by the current ICA’s responsibility for the current datum, gO cn . If ICA model c has little or no responsibility, its source parameters should remain static until it observes data deemed under its jurisdiction. Again, this is possible only under a mixture posterior distribution for the sources. 4 Implementing vbICA-MM

The measure F is maximized using the free-form approach of equation 3.24. All the derived posteriors require solving a set of coupled parameter update equations (see the appendix). In practice, this is best achieved by Žrst initializing the posterior component responsibilities (p0 ( c )), using these to initialize each ICA component, then beginning learning on each ICA component. These components are then used to calculate the new posterior responsibilities, and the learning process is repeated until convergence. These steps may be conveniently implemented in the algorithm shown in pseudocode form in Table 1. Once trained, the model can be used to reconstruct hidden source signals (to within a scaling and permutation) given a data set and the (now Žxed) model parameter distributions by calculating hci, hq c i, and hsc i under their respective posteriors. 4.1 Priors and Initialization. Broad priors have little or no effect on the results because these allow the data to “do the talking.” Such priors allow pa-

Variational Mixture of Bayesian Independent Component Analyzers

Table 1: Pseudocode for vbICA-MM Updates.

initialize; WHILE (D F (ica ¡mm ) < tolerance ) FOR every ICA component WHILE (D F (ica) < tolerance) update ICA observation model by cycling through equations A.12–A.17 until convergence; update ICA source model by cycling through equations A.18–A.24 until convergence; update ICA noise model using equations A.25–A.28; calculate Fnew ; (ica ) . |; calculate D F (ica ) D |Fnew ¡ Fold (ica ) (ica ) END WHILE; END FOR;

update ICA-MM indicator probabilities and parameters using equations A.29–A.30; calculate Fnew ; (ica¡mm ) . |; calculate D F (ica ¡mm ) D |Fnew ¡ Fold (ica ¡mm ) (ica ¡mm ) END WHILE;

229

230

R. Choudrey and S. Roberts

rameters and hidden variables to remain within sensible boundaries while encoding very little knowledge of what values they might take. Allowing the priors to become too broad, however, can reduce their regularizing abilities, leading to implausible magnitudes and dangers of overŽtting. Narrow priors, on the other hand, encode strong assumptions about possible parameter and variable values and have a discernible effect on the inference process. If the priors are too narrow, the model becomes inexible, and its ability to learn is compromised. For the ICA mixture model presented in this article, we examined a wide variety of priors (10¡6 · b · 106 , 10¡6 · c · 106 for all gamma distributions, scale parameter = 5–5000 for all Dirichlets, and precision = 10¡6 ¡ 106 for gaussians). For gamma distributions, the variance is given by b2 c and the mean by bc. Large values of c give distributions that are highly peaked around the mode, tending toward gaussianity. Values of c < 1 have no mode and are more like exponentials. Consequently, they encourage small values for the random variate while curtailing large values beyond the mean. We found 1 · b · 1000 with bc ¼ 1 a useful range for gamma distributions. Gamma distributions govern scale parameters, in this case, precisions. Values of bc > 10 have a narrowing effect on the posteriors, making them overconŽdent. Values of bc < 10¡1 with c > 1 have a smoothing effect on the posteriors, losing detail in the process. Similarly, we have found precisions of the order of 100 ¡ 10¡3 a good range. The value of Dirichlet scale parameters is as a pseudocounts, so high values are very constraining. Appropriate values depend on the number of data vectors. If N is the number of data vectors, then values between 0.01N and 0.1N stop components dying while allowing the data to set the posterior values accurately. If the algorithm was recast as an on-line scheme, these values would have to be reexamined because the results would necessarily become more sensitive to the choice of priors. The choice of initialization is also important. As with any other ICA formalism, the vbICA-MM algorithm Žnds a local maximum in the parameter space, and results depend on initialization. There are many ways to initialize; we recommend that readers explore the different options available to work out which is most suitable for them. We have investigated two initialization schemes: random and singular value decomposition (SVD). We initialize the model by Žrst running Ccomponent k-means on the data. The class responsibilities generated are used to partition the data into C segments, which are used to initialize C ICA models either randomly or by using SVD on each segment. If SVD is used, then the Žrst Lc columns of the SVD data matrix are used to initialize Lc MoGs using k-means, while residual singular values are used to estimate the noise. The Žrst Lc columns of the SVD projection matrix are used to initialize the mixing matrix. We have found random initialization the most useful overall, particularly for modeling clusters that are best described by nonorthogonal axes. Initializing using SVD (tantamount to initializing at the PCA solution)

Variational Mixture of Bayesian Independent Component Analyzers

231

yields a quicker result, but vbICA-MM can have problems with resolving directions in clusters exhibiting high correlation, often staying at or close to the PCA result. This is discussed further in section 6.1. One must also choose the number of components in each source MoG. Although this can be determined by monitoring the contribution to F by each source (F[Qc,i ]) as a function of components, this is time-consuming. We have found that three components are enough for most data sets, although Žve should be used for image data. 5 Results

We Žrst demonstrate the versatility of the vbICA-MM algorithm on two- and three- dimensional synthetic data, each grouped into three classes. The algorithm’s ability to model intricate, highly nongaussian data is highlighted, as is its structure determination using ARD and the mean-Želd bound, F, to the log evidence. We then use vbICA-MM on real functional magnetic resonance imaging (fMRI) data to decompose the images into interpretable, independent features and compare the results with previous methods. We set vague priors (b D 1000, c D 1e ¡ 3 for all gamma distributions, scale parameter = 5 for all Dirichlets and precision = 1e-3 for gaussians) to encode poor prior knowledge for both the synthetic and real data sets. 5.1 Test Data. We tested the vbICA-MM algorithm on two-dimensional and three-dimensional synthetic test data drawn from three classes and with 10% added gaussian noise. We set three components per source MoG and commenced learning using the algorithm presented in Table 1. Training continued until F changed by less than 0.01% or the number of iterations reached 200. Typically, iterations of each ICA component start high (approximately 100–200 for the nonorthogonal cluster presented below and less than 100 for the orthogonal clusters) and end with only a handful of iterations. In the 2D case, the underlying source manifolds were two-dimensional. Figure 5a shows the three clusters, with one gaussian cluster, one nongaussian cluster described by orthogonal directions, and one highly correlated cluster made up of six subclusters. For this data set, vbICA-MM was intialized randomly. Training took nine iterations of the vbICA-MM algorithm. Figure 5b shows the partition by vbICA-MM trained on 3000 points (1000 from each cluster) compared with the original data; only 12 (0.4%) data were assigned incorrectly. The Žrst deviations of the underlying MoG components are shown as ellipses. Note how the gaussian source is deemed one-dimensional, while the two-dimensional structure of the other classes has been clearly captured. A gaussian source is unidentiŽable from gaussian noise in ICA. Its principal direction is captured by the single source, while its spread is absorbed by the (gaussian) noise variance. The vbICA-MM data density model in Figure 5e shows the multimodal structure captured within

232

R. Choudrey and S. Roberts

the clusters. In comparison, an MoG (see Figure 5c) and the generalized exponential decorrelating ICA mixture model (gedecICA-MM) presented by Penny and Roberts (2001) (see Figure 5d) show no structure within the classes. Subsequently, vbICA-MM gives a more efŽcient representation of the true data density, with little probability mass wasted in regions of low density within each cluster. The data density of the 3D test data is shown in Figure 6a. The three clusters are intrinsically one-, two- and three-dimensional. We initialized a vbICA-MM using SVD and trained on 1500 points, 500 drawn from each cluster. Training took Žve iterations of the vbIVA-MM algorithm. Figure 6b depicts the model captured by vbICA-MM showing accurate representation of the clusters and cluster structures. The Hinton diagrams (Hinton, 1989) of the mixing matrices in Figures 6c through 6e show how vbICA-MM has used ARD to infer the latent dimensionalities correctly. Unsupported columns in the matrices have been suppressed by small ARD coefŽcients (inverse ai ’s shown in Figure 7a), effectively switching off unnecessary sources in the source reconstructions below. Another way of inferring the latent dimensionality of each ICA component is to monitor the free energy of the model. As implied by equation 3.23, this is equivalent to monitoring the contribution each ICA component makes to the overall free energy of the model. Figure 7b is a plot of the mean-Želd bound (i.e., the negative of the free energy) for each ICA component, FICAc . Each curve has had its maximum subtracted such that the trends in Figure 7b are more apparent. These curves conŽrm the latent dimensionality inferred by ARD. Plotting the overall mean-Želd bound, F, across components in Figure 7c correctly infers three clusters. There is a further peak at Žve components (and, indeed, progressively shallower ones at seven and nine) as ICAs latch on to different subclusters in the 3D cube. It must be noted, however, that F is a bound to the log evidence, so these further peaks disappear when exponentiated to yield the evidence bound. In comparison, a Bayesian MoG infers 21 clusters, while the BIC in gedecICA-MM predicts 9. Figure 8 shows how well the supported sources’ multimodal densities have been modeled by the MoGs. The ability of vbICA-MM to capture undulating manifolds is fundamental in correctly inferring the true number of clusters. As a comparison, Figure 7c also plots the Bayesian information criterion (BIC) approximation found to the log evidence using MAP estimates (BIC is the negative of maximum description length modeling; see Rissanen, 1994, for an introduction). Interestingly, this also picks out the correct model order, with a higher score for the log evidence. Attias (1999a) showed that the BIC is equivalent to variational Bayes in the inŽnite data limit. The BIC achieves a higher score, presumably because it is a much simpler model, with no model averaging, although we must admit surprise at the difference. We must point out, however, that Penny, Roberts, and Everson (2001) have suggested that penalized likelihood is more applicable to ICA than other models due to the likely violation of the independence assumption in overcomplex models,

Variational Mixture of Bayesian Independent Component Analyzers

(a) Data density

(b) vbICA-MM partition

(c) Gaussian mixture model

(d) gedec-ICA mixture model

233

(e) vbICA mixture model

Figure 5: Two-dimensional test data results. Contours plot 0.001 · p ( x ) · 1 in 250 intervals.

234

R. Choudrey and S. Roberts

(a) Data density

(c) 1D cluster

(b) vbICA-MM isosurface of p ( x) D 0.01

(d) 2D cluster

(e) 3D cluster

Figure 6: Three-dimensional test data results.

leading to a penalizing effect on the data likelihood itself. This suggests that a full variational integration is not necessary in the high data–low noise limit and that penalized likelihood MAP learning is adequate. With more noisy data, however, Penny et al. (2001) showed that model order selction using penalized likelihood quickly breaks down, so in uncertain situations, a full Bayesian approach would be more robust.

Variational Mixture of Bayesian Independent Component Analyzers

ARD Coeff.

1

0.5

0 1 2

3

S o u rce N u m b e r

2

3 1

IC A C om po n e nt

(a) ARD CoefŽcients 0 ­ 2 00

F(ICA )

­ 4 00 ­ 6 00 ­ 8 00

1d 2d 3d

­ 10 00 ­ 12 00 ­ 14 00

1

2

3

N um b e r of S o u rc es

(b) FICA for each ICA component 4 000 2 000

F

0 ­ 2 000 ­ 4 000 ­ 6 000 ­ 8 000

1

2

3

4

5

Number of Components

6

(c) Solid: vbICA-MM negative free-energy Dotted: BIC using MAP estimates

Figure 7: Structure determination using ARD and negative free energy.

235

236

R. Choudrey and S. Roberts

(a) ICA 1

(b) ICA 2

(c) ICA 3

Figure 8: (Top) Original source distributions. (Bottom) Learned distributions.

5.2 Real Data. We used the vbICA, variational Bayesian mixture of factor analyzers (vbMFA), gedecICA-MM, and vbICA-MM algorithms on fMRI images of a slice through a tumor patient’s brain. Data were collected using both T2 and proton density (PD) spin sequences, which are used directly to form a two-dimensional feature space. Two 100 £ 100 pixel images were

Variational Mixture of Bayesian Independent Component Analyzers

237

vectorized, and 2000 samples were randomly drawn from the subsequent 2D data. Models with a range of latent dimensions were trained on the 2000 data vectors, and the most likely models were then used to unmix the complete data set of 10,000 points into the corresponding independent features. Figure 9 shows the two original images used, along with the feature extracted by vbICA and the gedecICA mixture model. Due to the inherent limitations of ICA, no more than two features could be extracted from the fMRI images. The vbICA algorithm favored a two-source ICA model, shown in Figure 9b. The left-hand feature is simply a copy of one of the original images. The right-hand image has separated out a local feature: cerebrospinal uid. The vbMFA algorithm infers an eight-component model with F D ¡14813.75, giving the features in Figure 9c. The BIC of the gedecICA-MM chose a four-component model with two sources per ICA component, giving the eight overall features in Figure 9d. Although the gedecICA-MM has managed to separate out the cerebrospinal uid, most features are simply scaled copies of each other, and therefore the gedecICA-MM overrepresents the fMRI images. We trained one-component to Žve-component models using vbICA-MM, with each ICA component having the maximum two allowed sources. The negative free energy (F) plot in Figure 10a shows that a two-component model is preferred,3 while the Hinton plots in Figures 10b and 10c imply one-source and two-source ICA components. Compared to vbMFA, vbICAMM had F D ¡3618.8, a substantial improvement. The three features extracted by the most likely model are presented in Figure 11, along with their learned distributions. The single source of the Žrst ICA component, shown in Figure 11a, is global “background” brain tissue detail. The second ICA component represents more local features, where the central part of Figure 11b is, once again, the cerebrospinal uid. The two-tone background demarcates white and gray brain matter and has not been separated out, unlike some of the features shown in Figure 9. The reasons for this are not yet clear. More interesting, however, the second source has extracted two dark blobs. These are the tumors, which neither vbICA, nor vbMFA, nor gedecICA-MM picked out. The features’ respective MoGs can be interpreted as the distribution of gray-scale values in each feature. The tumors’ distribution is heavily peaked around darker values, with the left-hand tail capturing the lighter gray–level information. The other distributions similarly represent lighter to darker gray values from left to right. The ability of vbICA-MM to capture multimodal feature distributions is pivotal in allowing the complex background distribution to be separated from the rest. This representation is thus much more interpretable and efŽcient than either simple ICA or the gedecICA-MM.

3

The BIC preferred a one-component model.

238

R. Choudrey and S. Roberts

(a) Original data

(b) ICA features

(c) vbMFA features

(d) gedecICA-MM features

Figure 9: fMRI images and ICA, vbMFA and gedecICA-MM extracted features.

Variational Mixture of Bayesian Independent Component Analyzers

­ 33 50 ­ 34 00 ­ 34 50

F

­ 35 00 ­ 35 50 ­ 36 00 ­ 36 50 ­ 37 00 ­ 37 50 1

2

3

4

Num ber of Com ponents

(a) vbICA-MM negative free-energy

(b) 1D cluster

(c) 2D cluster

Figure 10: Inferred latent structure for fMRI images.

5

239

240

R. Choudrey and S. Roberts

(a) Background

(d) ICA 1

(b) C-S uid

(c) Tumors

(e) ICA 2

Figure 11: vbICA-MM features extracted from fMRI images and respective ICA component source distributions.

Figure 12 illustrates how the original data density has been modeled and partitioned by both gedecICA-MM and vbICA-MM. The lighter cluster in Figure 12d is described by the 1D ICA component, while the darker area is the 2D ICA cluster. vbICA-MM has managed to partition the data density into two clusters that are not linearly separable. 6 Discussion

In this article, we have presented an algorithm for modeling multidimensional, nongaussian data distributions. The vbICA-MM algorithm splits the distribution into self-similar areas and uses ICA components to learn representations of these areas. The ICAs model these local cluster manifolds by forming independent directions in the underlying distribution. Automatic relevance determination selects the appropriate dimensionality of each manifold, and a Bayesian learning scheme allows the optimum number of ICA components to be inferred. We have demonstrated the algorithm on two-dimensional and three-dimensional data by faithfully modeling intricate, discontinuous clusters while simultaneously inferring their intrinsic dimensions, something not possible under previous ICA mixture model. We

Variational Mixture of Bayesian Independent Component Analyzers

241

Figure 12: vbICA-MM model for fMRI images.

have shown how it is possible to compare assumptions underlying various models, in particular, using the negative free energy, F, to ascertain the correct number of clusters. We have used vbICA-MM to decompose real fMRI images into interpretable, self-similar features, blindly and automatically. 6.1 Problems to Overcome. Although we have comprehensively demonstrated the versatility and robustness of the vbICA-MM algorithm, there are practical issues to address, in particular, initialization and speed. As discussed in section 4.1, the choice of initialization can have an effect on the Žnal solution. SVD initialization always starts at the same place for

242

R. Choudrey and S. Roberts

a given data set and always ends at the same solution. In practice, we have found that different random initializations generally lead to the same solution as well. However, if parts of the data density are highly correlated, the ICA component most responsible for that area will stay close to the PCA solution if initialized using SVD. Different random initializations may lead to different solutions or may converge onto the PCA / FA solution. ICA can be seen as a generalization of principal component analysis and factor analysis. As shown by Attias (1999b), noiseless ICA with gaussian sources is equivalent to PCA, while FA is the same with nonisotropic noise. This means the PCA / FA solution is a subset of all possible ICA solutions for a given data set. As a particular cluster becomes more and more correlated (or, equivalently, its independent directions become less orthogonal), the second moment starts to dominate so the cluster becomes easier to describe by a gaussian density. This means the PCA / FA maxima in the ICA solution space become more prominent, so more and more initialization points will lead to them. The nonorthogonal cluster in Figure 5 was chosen because empirically, it seems to be the limit of correlation and nonorthogonality that vbICA-MM can resolve into independent directions. This particular cluster has a correlation of ¡0.67, and we needed on average Žve random initializations before vbICA-MM latched on, as indicated by the signiŽcantly higher value for F after one iteration for this initialization. More investigation is needed into initialization strategies to overcome this. One possible solution is to decorrelate, or whiten, each cluster identiŽed by the k-means step and then commence learning on these whitened data. We have had encouraging results using this admittedly rather ad hoc method. Also, the factorial approximation in equation 3.19 is no longer a good approximation for highly correlated data, so dropping this will also solve much of the problem, albeit with a corresponding reduction in speed. Speed is the other problem. Bayesian learning is inherently slower than non-Bayesian methods due to the extra parameters that have to be estimated. In fact, Bayesian inference can be shown to be NP-hard (Cooper, 1990). The variational approximation brings tractability to the Bayesian formulation and is speedier than the nondeterministic sampling regime, but it still remains understandably slower than its non-Bayesian counterparts. Interesting work by Valpola and Pajunen (2000) on speeding up Bayesian ICA may provide a basis for a faster vbICA-MM algorithm. Also, modelorder selection for the number of ICA components can become prohibitive if a large number are required. Some progress may be made in this area if, for example, some sort of birth-death (Roberts, Holmes, & Denison, 2001) and split-merge (Ueda, Nakano, Ghahramani, & Hinton, 1999) criteria could be enforced allowing active generating, culling, splitting, and merging of components during the learning process. This is an area of ongoing research. 6.2 Further Applications. Interesting data are, almost by deŽnition, not gaussian distributed. For data distributions that consist of self-similar non-

Variational Mixture of Bayesian Independent Component Analyzers

243

gaussian clusters, we believe this model will provide a better representation over current methods, such as gaussian mixture modeling, mixtures of factor analyzers, and plain-vanilla ICA. The kinds of data that can beneŽt from such an analysis range from the prosaic to the provocative. The most obvious area is in nonstationary blind source separation, as alluded to in section 1.2. Standard ICA is now the de facto method for blindly separating mixtures of signals. It has shown to be very robust, effective, and, in a probabilistic formulation, highly interpretable. However, ICA is essentially limited to stationary mixings of independent sources, which number fewer than the sensors picking up the observations. ICA mixture models are an important step in overcoming these restrictions. Sudden changes in the mixing amounts will be encoded by vbICA-MM as a switching between ICA components. In the cocktail party problem, for example, this would include different voices cutting in and out over a background cacophony. Provided the number of voices (plus background) active at any one time does not exceed the total number of sensors, vbICA-MM can capture more sources than sensors. This was demonstrated in the fMRI example, where three features were extracted from 2D data. Furthermore, the independence assumption at the core of ICA holds within components but is not necessary between components. Sources that are not independent will be represented as sources within different ICA components by vbICA-MM. The application of vbICA-MM to enhanced blind source separation does not have to stop at auditory signals. The multimodal nature of the source distributions lends itself readily to image separation and other BSS problems. Clustering in general is another area that naturally comes under the vbICA-MMs umbrella. Mixture modeling is the deŽnitive formalism for Žnding clusters in data distributions. Interesting clusters are very often not gaussian, so mixture models based on gaussianity will generally not model such clusters well. The synthetic clusters in section 5 were nongaussian and also contained structure within themselves. This confused both a gaussian mixture model and the mixture of factor analyzers model. The former did not model the data accurately, while the latter failed to infer the correct number of clusters. Lee et al. (2000) have shown that their ICA mixture model performed better at clustering the Iris data of Fisher (1936) than those based on gaussian methods. The grouping of documents into semantic clusters is currently a very active area of research (see Landaur, Foltz, & Laham, 1998, for an introduction). It is highly unlikely that such man-made corpora are intrinsically gaussian in nature. The reader may have noticed that in the use of ICA as a representative tool, we have verged on the evangelical. This, we strongly believe, is where ICA and its extensions will prove to be indispensable. Lee, Lewicki et al. (1999) have already shown that ICA mixture models based on Laplacian source densities are over 20% more efŽcient than PCA at coding natural scenes and images of newspaper text, and more efŽcient than standard

244

R. Choudrey and S. Roberts

ICA. As ICA is only beginning to leave its BSS beginnings behind, this area is as yet largely unexplored. 6.3 Future Directions. There are a number of modiŽcations that could be made to vbICA-MM to take into account further information that one may have on the data. For example, the source models could be restricted to being the same dimensionality for all ICA components if it is expected that the data lie on some dimensionally consistent manifold. Indeed, the source models could be shared among a number, if not all, of the ICA components if different regions of the data space are thought to be different projections of some handful of underlying manifolds. Temporal information may be introduced if the mixture process prior is conditioned across samples as in, for example, a Markov process, leading to a Bayesian formulation of hidden Markov ICA (Penny, Roberts, & Everson, 2000). vbICA-MM leads to a whole family of ICA mixture models depending on which assumptions are coded into the model. Because of the Bayesian formulation, the effectiveness of these models can be quantitatively compared. Again, this is an area of active research. Appendix: Update Equations

The parameter posteriors are given by p0 ( sc | q c , c) D p0 ( A ) D p0 ( ® ) D p0 ( q c | c ) D 0

p (¼ ) D p0 ( ¹ ) D p0 ( ¯ ) D

Lc N Y Y nD 1 i D1

N

n ) ( snc,i I mO ni,qi , bOi,q i

Lc Y C Y S Y

( aji I m O aji , aO ji )

N

c D1 i D 1 j D 1 Lc C Y Y c D1 i D 1 Lc N Y Y nD 1 i D1

(A.1)

(A.2)

G (ai I bOai , cOai )

(A.3)

n cOi,q i

(A.4)

Lc Y mi ( C Y Y

c D1 i D 1 qi D1 Lc Y mi C Y Y c D1 i D 1 qi D1

q i0

lO i,qi0 )

C ( lO i,qi )

c D1 i D 1 qi D1 Lc Y mi C Y Y

P

lO i,q ¡1

p i,qi i

(m i,qi I m O i,qi , tOi,qi )

N

G (b i,qi I bOi,qi , cOi,qi )

(A.5)

(A.6)

(A.7)

Variational Mixture of Bayesian Independent Component Analyzers

p0 ( y ) D p0 ( ¸ ) D p0 ( c ) D p0 ( · ) D

C Y S Y

( yj I mO yj , tOyj )

N

c D1 j D 1 C Y c D1

(A.8)

G (lcI bOlc , cOlc )

N Y C Y n D1 c D1 C Y ( c D1

(A.9)

gOcn

P

245

(A.10)

0 c0 Oic )

C (Oic )

k cOic ¡1 .

(A.11)

Updated distribution parameters are hatted versions of the original parameters, hai are expectations taken with regard to p0 ( a) , and ha | bi are expectations with regard to p0 (a | b) . A.1 Observation Model.

A.1.1 p0 ( sc | qc , c) .

mO ni,qi

2 S X 1 4hb i,qi ihm i,qi i C gO cn hlc i haji i D n bOi,q D1 j

i

3

n | q nk , ci ¡ hyj i) 5 £ ( xjn ¡ hxOj,k 6 i D

n bOi,q D hb i,qi i C gO cn hlc i i

S X jD 1

ha2ji i

where

n xOj,k 6 i D D

hxOj,nkD6 i | ci D hxOj,nkD6 i | q nk , ci D

Lc X

ajksnc, k

6 i kD

Lc X 6 i kD

Lc X 6 i kD

hajk ihsnc, k | ci hajk ihsnc, k | q nk , ci,

(A.12)

(A.13)

246

R. Choudrey and S. Roberts

and deŽne xOjn D hxOjn | ci D hxOjn | q ni , ci D

Lc X

aji snc,i

iD 1 Lc X iD 1 Lc X iD 1

haji ihsnc,i | ci haji ihsnc,i | q ni , ci.

The expectations for the sources s are given by the expressions in section 3.3. In practice, equation A.12 has to be iterated for every i a number of times 6 i. until mO ni,qi converges as it depends on every other k D Note the intuitive form of equations A.12 and A.13. Source i “sees” data xjn at sensor j and works out what it can “explain away” given information n in the rest of the model, that is, xOj,k . The residual information is then used 6 i D to update its own parameters in a bid to explain what is “left over.” This is analogous to the message-passing algorithms of Pearl (1988), whereby a n ) using messages node (sc,i ) updates its belief (encapsulated in mO ni,qi and bOi,q i passed to it by its parents (the MoG expectations), its children (the sensor data), and all other parents of its children (i.e., all other sources and quantiŽed by xOj, kD6 i ). This use of information from a variable’s Markov blanket runs through all the update equations. A.1.2 p0 (A) . mO aji D

N hlc i X gO n hsn | ci ( xjn ¡ hxOj,nkD6 i | ci ¡ hyj i) a O ji nD 1 c c,i

aO ji D hai i C hlc i

N X nD1

gO cn hsnc,i 2 | ci

(A.14)

(A.15)

In practice, equation A.14 has to be iterated for every i a number of times until mO aji converges. A.1.3 p0 ( ®c ) . 0

bOai

1¡1 S X 1 1 C ha2 iA D @ 2 jD 1 ji bai

cOai D cai C

S 2

(A.16)

(A.17)

Variational Mixture of Bayesian Independent Component Analyzers

247

A.2 Source Model.

A.2.1 p0 (qc | c) . Á n c i,q i

D pQ i,qi

bQi,qi n bOi,qi

!1 2

µ

1 n n2 (bO mO ¡ hbi,qi ihm 2i,qi i) exp 2 i,qi i,qi

¶

n c i,q i n P cOi,q D n i 0 c qi i,q0

(A.18)

(A.19)

i

where 2

0

pQ i,qi D exp 4Y (rOi,qi ) ¡ Y @

13 X q0i

rOi,q0i A5

bQi,qi D bOi,qi exp[Y ( cOi,qi ) ] and where equation A.19 ensures that function.

P

n qi cOi,qi

D 1. Y (.) is the digamma

A.2.2 p0 ( ¼ ) . rOi,qi D r i,qi C

N X nD 1

n cOi,q i

(A.20)

A.2.3 p0 ( ¹) . Á O i,qi D m

1 tOi,qi

ti0 mi0 C hbi,qi i

tOi,qi D ti0 C hbi,qi i

N X nD 1

N X nD 1

! n cOi,q hsnc,i i

|

qni ,

ci

n cOi,q . i

A.2.4 p0 ( ¯ ) . ³ ´ ¡1 Obi,q D 1 C 1 sQ i,q i 2 i bi0 cOi,qi D ci0 C

N 1X cO n , 2 nD1 i,qi

where we deŽne the average variance of component qi in source i sQ i,qi D

N X nD 1

n (h n 2 | n cOi,q sc,i qi , ci ¡ 2hm i,qi ihsnc,i | qni , ci C hm 2i,qi i). i

(A.21)

(A.22)

(A.23)

(A.24)

248

R. Choudrey and S. Roberts

A.3 Noise Model.

A.3.1 p0 (y) . mO yj D

N hlc i X gO n ( xn ¡ hxOjn | ci) tOyj nD 1 c j

tOyj D tyj C hlc i

N X nD 1

(A.25)

gO nc .

(A.26)

A.3.2 p0 (l) . 2

bOlc

3¡1 S X N 1 1X C gO n h(xn ¡ xOjn ¡ yj ) 2 i5 D 4 2 j D1 n D1 c j blc

cOlc D clc C

N SX gO n . 2 n D1 c

(A.27)

(A.28)

A.4 ICA Mixture Update.

A.4.1 p0 (c) . gcn

S

Q c2 D kQ c l

µ

¶ lc n 2 n ) exp h(xj ¡ xOj ¡ yj i 2 j D1

S Y

gn gO cn D PC c

n c D 1 gc

,

(A.29)

(A.30)

where "

kQ c D exp Y (Oic ) ¡ Y

Á X c0

!# Oic0

lQ c D bOlc exp[Y ( cOlc ) ], where equation A.30 ensures that

P

O cn cg

D 1.

A.4.2 p0 ( ·) . iOc D ic C

N X nD 1

gO cn

(A.31)

Variational Mixture of Bayesian Independent Component Analyzers

249

The relevent moments are given by hai D mean(a)

(A.32)

ha2 i D mean(a) 2 C variance(a) . Note that for a mixture with only one ICA component gcn D 1 for all n, and equations A.12 through A.24 reduce to the update equations for a single ICA model. Acknowledgments

We thank Iead Rezek, Peter Sykacek, and Evangelos Roussos for perceptive comments and discussions. Thanks to Mike Gibbs for solving computerrelated problems. Thanks also to the anonymous referees for very helpful and constructive criticism. References Attias, H. (1999a). Learning parameters and structure of latent variable models by variational Bayes. In Electronic Proceedings of the Fifteenth Annual Conference on Uncertainty in ArtiŽcial Intelligence (VAI-1999). Available on-line at: http://www2.sis.pitt.edu/»dsl/UAI/uai.html. Attias, H. (1999b). Independent factor analysis. Neural Computation, 11, 803–851. Back, A., & Weigend, A. (1998). A Žrst application of independent component analysis to extracting structure from stock returns. International Journal of Neural Systems, 8(4), 473–484. Bartlett, M., Lades, H., & Sejnowski, T. (1998). Independent components representations for face recognition. In Proceedings of the SPIE Symposium on Electronic Imaging: Science and Technology: Conference on Human Vision and Electronic Imaging III (pp. 528–539). San Jose, CA: SPIE. Bell, A., & Sejnowski, T. (1995). An information maximization approach to blind separation and blind deconvolution. Neural Computation, 7, 1129–1159. Bell, A., & Sejnowski, T. (1997).The “independent components” of natural scenes are edge Žlters. Vision Research, 37(23), 3327–3338. Cardoso, J.-F. (1997). Infomax and maximum liklihood for blind source separation. IEEE Letters on Signal Processing, 4, 112–114. Choudrey, R., Penny, W., & Roberts, S. (2000). An ensemble learning approach to independent component analysis. In Proceedings of Neural Networks for Signal Processing X. Sydney, 2000. Choudrey, R., & Roberts, S. (2001). Variational Bayesian independent component analysis with exible sources (Tech. Rep. No. PARG-01-05). Available on-line at: http://www.robots.ox.ac.uk/sjrob/pubs.html. Oxford: Oxford University. Cooper, G. (1990). Computational complexity of probabilistic inference using Bayesian belief networks. ArtiŽcial Intelligence, 42(3), 393–405. Fisher, R. (1936). The use of multiple measurements in taxonomic problem. Annals of Eugenics, 7(2), 179–188.

250

R. Choudrey and S. Roberts

Ghahramani, Z., & Beal, M. (2000). Variational inference for Bayesian mixtures of factor analysers. In S. A. Solla, T. K. Leen, & K.-R. Muller ¨ (Eds.), Advances in neural information processing systems, 12 (pp. 449–455). Cambridge, MA: MIT Press. Girolami, M. (1998). An alternative perspective on adaptive independent component analysis algorithms. Neural Computation, 10(8), 2103–2114. HÂerault, J., & Jutten, C. (1986). Space or time adaptive signal processing by neural models. In J. Denker (Ed.), Proceedings AIP Conference: Neural Networks for Computing (Vol. 151, pp. 206–211). Snowbird, UT: AIP. Hinton, G. (1989). Connectionist learning procedures. ArtiŽcial Intelligence, 40, 185–234. Hinton, G., & van Camp, D. (1993). Keeping neural networks simple by minimising the description length of the weights. In Proceedings of COLT-93. Santa Cruz, CA. Hyva¨ rinen, A. (1999). Survey on independent component analysis. Neural Computing Surveys, 2, 94–128. Available on-line at: http://www.icsi.berkeley. edu/jagota/NCS. Isbell, B., & Viola, P. (1999). Restructuring sparse high dimensional data for effective retrieval. In M. Kearns, S. Solla, & D. Cohn (Eds.), Advances in neural information processing systems, 11 (pp. 480–486). Cambridge, MA: MIT Press. Jaakkola, T., & Jordan, M. (2000). Bayesian parameter estimation via variational methods. Statistics and Computing, 10, 25–37. Jordan, M., Ghahramani, Z., Jaakkola, T., & Saul, L. (1999). An introduction to variational methods for graphical models. In M. Jordan (Ed.), Learning in graphical models. Cambridge, MA: MIT Press. Kolenda, T., Hansen, L., & Sigurdsson, S. (2000). Independent component analysis in text. In M. Girolami (Ed.), Advances in independent component analysis. Berlin: Springer-Verlag. Landaur, T., Foltz, P., & Laham, D. (1998). An introduction to latent semantic analysis. Discourse Processes, 25, 295–284. Lappalainen, H. (1999).Ensemble learning for independent component analysis. In Proceedings of the First International Workshop on Independent Component Analyis (pp. 7–12). Aussois, France. Lee, T., Girolami, M., Bell, A., & Sejnowski, T. (1998). A unifying informationtheoretic framework for independent component analysis. International Journal of Mathematical and Computer Modelling, 39, 1–21. Lee, T., Girolami, M., & Sejnowski, T. (1999). Independent component analysis using an extended infomax algorithm for mixed sub-gaussian and supergaussian sources. Neural Computation, 11(2), 409–433. Lee, T., Lewicki, M., & Sejnowski, T. (1999). ICA mixture models for unsupervised classiŽcation and automatic context switching. In International Workshop on Independent Component Analysis (pp. 209–214). Aussois, France. Lee, T., Lewicki, M., & Sejnowski, T. (2000). ICA mixture models for unsupervised classiŽcation of non-gaussian classes and automatic context switching

Variational Mixture of Bayesian Independent Component Analyzers

251

in blind signal separation. IEEE Transactions on Pattern Recognition and Machine Intelligence, 22(10), 1–12. Lewicki, M., & Olshausen, B. (1999). A probabilistic framework for the adaption and comparison of image codes. Journal of the Optical Society of America, 16(7), 1587–1601. MacKay, D. (1995a). Developments in probabilistic modelling with neural networks—ensemble learning. In Proceedings of the Third Annual Symposium on Neural Networks (pp. 191–198). Nijmegen, Netherlands: SpringerVerlag. MacKay, D. (1995b). Probable networks and plausible predictions—a review of practical Bayesian methods for supervised neural networks. Network: Computation in Neural Systems, 6, 469–505. Makeig, S., Bell, A., Jung, T.-P., & Sejnowski, T. (1996). Independent component analysis for electroencephalographic data. In D. Touretzky, M. Mozer, & M. Hasselmo (Eds.), Advances in neural information processing systems, 8 (pp. 145–151). Cambridge, MA: MIT Press. Miskin, J., & MacKay, D. (2000). Application of ensemble learning to infra-red imaging. In Proceedings of ICA2000. Helsinki, Finland. Olshausen, B., & Field, D. (1997). Sparse coding with an overcomplete basis set: A strategy employed by V1? Vision Research, 37, 3311–3325. Pearl, J. (1988). Probabilistic reasoning in intelligent systems (2nd ed.). San Mateo, CA: Morgan Kaufmann. Pearlmutter, B., & Parra, L. (1996). A context-sensitive generalization of ICA. In ICONIP ’96 (pp. 151–157). Berlin: Springer-Verlag. Penny, W., & Roberts, S. (2001). Mixtures of independent component analysers. In ArtiŽcial Neural Networks—ICANN2001 (pp. 527–534). Vienna, Austria. Penny, W., Roberts, S., & Everson, R. (2000). Hidden Markov independent components analysis. In M. Girolami (Ed.), Advances in independent component analysis. Norwell, MA: Kluwer. Penny, W., Roberts, S., & Everson, R. (2001). ICA: Model order selection and dynamic source models. In S. Roberts & R. Everson (Eds.), Independent component analysis: Principles and practice (pp. 299–314). Cambridge: Cambridge University Press. Rissanen, J. (1994). MDL modeling—an introduction. In P. Grassberger & J.P. Nadal (Eds.), From statistical physics to statistical inference and back (pp. 95– 104). Norwell, MA: Kluwer. Ristaniemi, T., & Joutsensalo, J. (1999). On the performance of blind source separation in CDMA downlink. In Proceedings of ICA ’99 (pp. 437–442). Aussois, France. Roberts, S., & Everson, R. (Eds.). (2001).Independent component analysis:Principles and practice. Cambridge: Cambridge University Press. Roberts, S., Holmes, C., & Denison, D. (2001). Minimum entropy data partitioning using reversible jump Markov chain Monte Carlo. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(8), 909–915. Tipping, M., & Bishop, C. (1999). Mixtures of probabilistic principal component analyzers. Neural Computation, 11(2), 443–482.

252

R. Choudrey and S. Roberts

Ueda, N., Nakano, R., Ghahramani, Z., & Hinton, G. (1999). SMEM algorithm for mixture models. In M. S. Kearns, S. Solla, & D. Cohn (Eds.), Advances in neural information processing systems, 11. Cambridge, MA: MIT Press. Valpola, H., & Pajunen, P. (2000). Fast algorithms for Bayesian independent component analysis. In Proceedings of ICA2000 (pp. 233–237). Helsinki, Finland. van Hateren, J., & van der Schaaf, A. (1998). Independent component Žlters of natural images compared with simple cells in primary visual cortex. Proceedings of the Royal Society of London, Series B, 265, 359–366. Received October 15, 2001; accepted July 8, 2002.

Communicated by Paul Bressloff

LETTER

Interspike Interval Correlations, Memory, Adaptation, and Refractoriness in a Leaky Integrate-and-Fire Model with Threshold Fatigue Maurice J. Chacron [email protected] Department of Physics, University of Ottawa, Ottawa, Canada K1N 6N5

Khashayar Pakdaman [email protected] Inserm, U444, 75571 Paris Cedex 21, France

AndrÂe Longtin [email protected] Department of Physics, University of Ottawa, Ottawa, Canada K1N 6N5

Neuronal adaptation as well as interdischarge interval correlations have been shown to be functionally important properties of physiological neurons. We explore the dynamics of a modiŽed leaky integrate-and-Žre (LIF) neuron, referred to as the LIF with threshold fatigue, and show that it reproduces these properties. In this model, the postdischarge threshold reset depends on the preceding sequence of discharge times. We show that in response to various classes of stimuli, namely, constant currents, step currents, white gaussian noise, and sinusoidal currents, the model exhibits new behavior compared with the standard LIF neuron. More precisely, (1) step currents lead to adaptation, that is, a progressive decrease of the discharge rate following the stimulus onset, while in the standard LIF, no such patterns are possible; (2) a saturation in the Žring rate occurs in certain regimes, a behavior not seen in the LIF neuron; (3) interspike intervals of the noise-driven modiŽed LIF under constant current are correlated in a way reminiscent of experimental observations, while those of the standard LIF are independent of one another; (4) the magnitude of the correlation coefŽcients decreases as a function of noise intensity; and (5) the dynamics of the sinusoidally forced modiŽed LIF are described by iterates of an annulus map, an extension to the circle map dynamics displayed by the LIF model. Under certain conditions, this map can give rise to sensitivity to initial conditions and thus chaotic behavior.

Neural Computation 15, 253–278 (2003)

° c 2002 Massachusetts Institute of Technology

254

M. Chacron, K. Pakdaman, and A. Longtin

1 Introduction Since the seminal work of Adrian and Zotterman (1926), adaptation has been demonstrated to be a key property of neurons. For example, photoreceptor neurons have to cope with very different light intensities (e.g., day and night) and must adapt to each (Kufer, Fitzhugh, & Barlow, 1957). Moreover, this adaptation can sometimes give rise to interspike interval (ISI) correlations in experimental recordings (Kufer et al., 1957; Goldberg, Adrian, & Smith, 1964; Yamamoto & Nakahama, 1983; Teich & Lowen, 1994; Sch¨afer, Braun, Peters, & Bretschneider, 1995; Longtin & Racicot, 1998; Chacron, Longtin, St.Hilaire, & Maler, 2000; Neiman & Russell, 2001; Liu & Wang, 2001; Brandman & Nelson, 2002). Information-theoretic approaches have recently shown that neural information transfer can either increase or decrease as a result of correlations between spikes (Abbott & Dayan, 1999; Panzeri, Petersen, Schultz, Lebedev, & Diamond, 2001; Tiesinga, Fellous, JosÂe, & Sejnowski, 2002) or ISIs (Chacron, Longtin, & Maler, 2001a). Also, correlations between neuronal inputs can increase or decrease information transfer (Abbott & Dayan, 1999; Panzeri et al., 2001). One prototype sensory system in which the role of ISI correlations has been addressed is the electrosensory system of weakly electric Žsh. Negative ISI correlations have been found in P-type electroreceptors (Longtin & Racicot, 1998). A modeling study of these receptors has further shown that these correlations enhance not only information transfer but signal detection as well (Chacron et al., 2001a). It is clear from those studies that ISI correlations depend on various biophysical model parameters, as well as on noise and periodic forcing. The goal of this article is to provide a foundation for understanding how ISI correlations arise from the interplay of these factors in the context of a simple model. The leaky integrate-and-Žre model (LIF) is one of the most elementary spiking models and has been widely used to gain a better understanding of information processing in neurons (see, e.g., Tuckwell, 1988). One feature that makes this model particularly suitable for theoretical analysis is that it is analytically tractable and retains two key neuronal properties of type I membranes: the all-or-none response and the postdischarge refractoriness. The tractability of the standard LIF is in part due to the fact that following a discharge, the dynamics of the LIF depends on only the present and future inputs. In other words, after each Žring, the LIF has no memory of the past: inputs and discharge times prior to the last one bear no inuence on the next discharge times. This property has been a key ingredient in the analysis of the response of the standard LIF to inputs such as sinusoidal current or white gaussian noise. Indeed, it allows the standard LIF to be described by iterates of orientation-preserving circle maps (Rescigno, Stein, Purple, & Popele, 1970; Keener, Hoppensteadt, & Rinzel, 1981; Coombes & Bressloff, 1999; Pakdaman, 2001). Moreover, the discharge times evoked by

Memory and Adaptation in a Model Neuron

255

gaussian white noise stimulation form a renewal process (for recent reviews on the noisy LIF, see Lansky & Sato, 1999; Ricciardi, Di Crescenzo, Giornio, & Nobile, 1999; Pakdaman, Tanabe, & Shimokawa, 2001). The lack of memory about past discharges, which constitutes an advantage in the theoretical analysis of the standard LIF, is also one of the shortcomings of this model. Indeed, this property can eliminate the correlation between successive interspike intervals (ISIs) of the LIF. For instance, the ISIs evoked by additive gaussian white noise stimulation of this model are independent from one another. As mentioned, this is not the case experimentally for many neurons. Moreover, the LIF model assumes that the threshold to Žring is constant in time. There is experimental evidence that the threshold for Žring is not constant but depends on the past spiking history of the neuron (Azouz & Gray, 1999). Thus, a physiologically plausible modiŽcation to the LIF neuron would be to make the threshold a dynamical variable also. In this article, we study a modiŽed LIF neuron in which the threshold is a dynamical variable that models threshold fatigue after an action potential. Using step currents as stimuli, we show that the model can give rise to adaptation (i.e., a gradual change in Žring rate), a feature that is absent from the standard LIF model. Under certain conditions, we show that the threshold fatigue can lead to a saturation in the Žring rate of the neuron as the depolarizing current is increased. This feature is also absent from the LIF model, where the Žring rate diverges as a function of input current. We then show that the modiŽed LIF incorporating threshold fatigue can reproduce ISI correlations similar to those found in experimental recordings. The choice of the threshold dynamics is guided by previous observations that ISI correlations can result from the progressive accumulation of refractory effects (Stein, 1965; Weiss, 1966; Geisler & Goldberg, 1966; Holden, 1976; Gerstner & van Hemmen, 1992; Chacron et al., 2000, 2001a; Chacron, Longtin, & Maler, 2001b; Liu & Wang, 2001). If the neuron Žres two spikes in a relatively short time interval, then the next action potential usually occurs after a longer time interval. Implementing threshold fatigue, that is, decreasing excitability during high-frequency Žring, in the LIF should thus be a candidate for increasing the ISI correlations in a satisfactory way. Our investigations provide another conŽrmation of this point. Although more detailed ionic models of neurons such as the Hodgkin-Huxley model (Hodgkin & Huxley, 1952) are capable of adaptation and ISI correlations with the addition of suitable ionic channels, such models seldom are analytically tractable due to their complexity. In contrast, our model retains enough of the simplicity of the LIF model to remain analytically tractable for the most part. Various integrate-and-Žre-like models with mechanisms analogous to threshold fatigue have been previously used—for instance: • In the analysis of mutually inhibiting neuronal models (Reiss, 1962; Wilson & Waldron, 1968: for an account of these studies, see MacGregor & Lewis, 1977)

256

M. Chacron, K. Pakdaman, and A. Longtin

• A stochastic model where an inhibitory conductance is being reset to a value dependent on past history to mimic the possible summation of hyperpolarizing afterpotentials following each discharge (Geisler & Goldberg, 1966; Wehmier, Dong, Koch, & van Essen, 1989; Treves, 1996; Liu & Wang, 2001) • Periodically and irregularly forced neuronal models (Segundo, Perkel, Wyman, Hegstad, & Moore, 1968) • Self-exciting models and interacting excitatory assemblies (Vibert, Pakdaman, & Azmy, 1994; Pakdaman & Vibert, 1995; Pakdaman, Vibert, Boussard, & Azmy, 1996) • An LIF model in which the threshold decayed exponentially and was incremented by a Žxed amount immediately after an action potential (Geisler & Goldberg, 1966; Holden, 1976; Chacron et al., 2000, 2001a, 2001b; Liu & Wang, 2001) Although the model used in this study bears similarities with previous ones, one key difference resides in the introduction of a memory parameter to adjust the level of fatigue. Indeed, previous studies dealt with neuron models with a Žxed level of fatigue, whereas this one is concerned with a systematic analysis of the inuence of this factor. This program is carried out in three stages involving the description of the response of the modiŽed LIF to (1) constant and step current stimulation, (2) white gaussian noise, and (3) sinusoidal forcing. The Žrst two stages are concerned with adaptation and the ISI correlations due to fatigue. The last stage examines how these factors affect the response of the modiŽed model to other inputs. The particular selection of sinusoidal forcing is motivated by its relevance for understanding information processing in extrinsically forced neurons (Chacron et al., 2000). Furthermore, comparison of our results with the extensive studies of sinusoidally forced LIFs and their variants (Rescigno et al., 1970; Glass & Mackey, 1979; Keener et al., 1981; Alstrøm & Levinsen, 1988; Coombes, 1999; Coombes & Bressloff, 1999; Pakdaman, 2001) provides the basis for the determination of speciŽc contributions of threshold fatigue to the dynamics of the model. 2 The Model We consider the model given by the following equations and Žring rule: dv D ¡v / tv C I ( t) if v ( t) < s ( t) , dt ds sr ¡ s if v ( t) < s ( t) , D ts dt v ( t C ) D v0 C

if v ( t) D s (t)

s ( t ) D s0 C W ( s ( t) , a)

(2.1) (2.2) (2.3)

if v (t) D s ( t) ,

(2.4)

Memory and Adaptation in a Model Neuron

257

Figure 1: Voltage (black solid line) and threshold (gray solid line) time series obtained with the model. An action potential occurs when voltage and threshold are equal. The Žring times tn thus satisfy v ( tn ) D s ( tn ) . Immediately after an action potential, the voltage is reset to zero while the threshold is set to a value s ( tnC ) D s 0 C W ( s ( tn ) , a) .

where v is the voltage, s is the threshold, I (t ) is the stimulation current, tv and ts are the time constants for voltage and threshold, respectively, and sr is the value at which the threshold stabilizes in the absence of Žring. Firing occurs when the voltage reaches the threshold. Following this, the voltage is reset to v 0 (see equation 2.3), and the threshold is set to s ( t C ) D s0 C W ( s ( t) , a), where s 0 is a parameter and W is a monotically increasing function of s and a with W ( s, 0) D 0. a is a positive parameter controlling the memory in the model. We will mostly look at the case W ( s, a) D W 1 ( s, a) ´ as, but other nonlinear forms could be used for W. Throughout, we assume that v 0 · 0 < sr · s0 . The model dynamics are graphically illustrated in Figure 1. Equation 2.4 with W D W1 represents threshold fatigue, with a being the memory parameter. Indeed, when a D 0, the threshold value immediately after a discharge is independent of the threshold value at that discharge, so that future discharges bear no memory of the Žring history. The case where a D 0 and s0 D sr corresponds to the standard LIF with constant threshold. Conversely, when W D W1 and a D 1, one recovers the threshold fatigue

258

M. Chacron, K. Pakdaman, and A. Longtin

implemented in previous studies, that is, after each discharge, the threshold is raised by a Žxed amount s0 (Segundo et al., 1968; Vibert et al., 1994; Pakdaman & Vibert, 1995; Pakdaman et al., 1996; Chacron et al., 2000, 2001a, 2001b; Liu & Wang, 2001). Other models used a conductance that was decremented by a Žxed amount immediately after an action potential to achieve similar effects (Wehmeier et al., 1989; Treves, 1996; Liu & Wang, 2001). Liu and Wang (2001) have shown that the latter approach gave qualitatively similar results to our model with W D W 1 and a D 1. Previous studies were thus limited to the case a D 1. Generally, increasing a from zero corresponds to increasing the degree of fatigue in the model and, consequently, its dependence on its past. The following sections examine how this parameter affects the correlation between consecutive ISIs and the response of the model to various stimuli. 3 Results 3.1 Constant and Step Current Stimulation. Throughout this section, we assume W D W1 and that I (t ) D m is constant. Two situations arise in response to such stimuli. When m tv < sr , the stimulus is subthreshold, the membrane potential v (t ) stabilizes at m tv , and no Žring occurs. Conversely, when m tv > sr , the model generates sustained Žring. The remainder of this section analyzes the corresponding discharge pattern. This is done through the construction of a map relating consecutive postdischarge threshold values. Let v ( u ) be the solution to equation 2.1 with v (0) D v0 , and let s ( u, S ) be the solution to equation 2.2 with s (0, S ) D S. These are given by v ( u ) D ( v 0 ¡ m tv ) e¡u/ tv C m tv

(3.1)

s (u, S ) D ( S ¡ sr ) e¡u/ ts C sr .

(3.2)

Let us assume that a discharge occurred at time tn and resulted in the postdischarge threshold s ( tnC ) ´ SnC . The next Žring takes place after a time interval D n C 1 ´ tn C 1 ¡ tn such that s (D n C 1 , SnC ) D v ( D n C 1 ) and s ( u, SnC ) > v ( u ) for all 0 · u < D n C 1 .

(3.3)

This Žring yields the postdischarge threshold SnCC 1 at time tnCC 1 : SnCC 1 D s 0 C as( D n C 1 , SnC ) D s 0 C av( D n C 1 ) C

D s 0 C a[(Sn ¡ sr ) e

(3.4) (3.5)

¡D n C 1 / ts

C sr ].

(3.6)

Memory and Adaptation in a Model Neuron

259

From these equations, the relation between the nth and n C 1th ISIs D n and D n C 1 is readily derived as: 8 [(v 0 ¡ m tv ) e ¡D nC 1 / tv C m tv ¡ sr ] > > < £eD n C 1 / ts C sr ¡ s 0 ¡ am tv D n D ¡tv ln > a(v 0 ¡ m tv ) > :

9 > > = > > ;

.

(3.7)

While equation 3.7 can be used to discuss the relation between successive ISIs, we prefer to focus on the relation between successive values of the postdischarge threshold. Indeed, for constant stimulation, from the ISI D n one can unambiguously derive SnC , the nth postdischarge threshold, and conversely, given SnC , there is a unique ISI D n , so that using either the ISIs or the postdischarge thresholds yields the same information about the dynamics of the model. However, for the class of stimuli treated in section 3.2, this is not necessarily so; different values of Sn may yield the same D n . This in turn makes it more difŽcult to obtain the relation between D n and D n C 1 . For this reason, the following paragraphs describe the relation between SnC and SnCC 1 . Equation 3.3 uniquely deŽnes D n C 1 D D n C 1 ( SnC ) as a function of SnC , so that we can rewrite equation 3.4 as SnCC 1 D s0 C as( D n C 1 ( SnC ) , SnC ) ´ F ( SnC ) .

(3.8)

This relation implies that the postdischarge threshold SnC after the nth Žring entirely determines the next postdischarge Žring SnCC 1 . In other words, the dynamics of the LIF with threshold fatigue receiving a constant stimulation is determined by the iterates of the map F. This map is deŽned on the interval [s 0 C asr , C 1) . In general, D (S ) , and consequently F ( S ) cannot be explicitly written in closed form because equation 3.3 cannot be solved analytically for arbitrary ts . However, the geometrical properties of F that determine the behavior of the sequence fSnC g can be determined without such knowledge. For Žxed u, the function s (u, S ) is monotonic increasing, so that if S > S0 , then s ( u, S) cannot intersect the voltage prior to s ( u, S0 ) . In other words, S > S0 implies that D (S ) > D ( S0 ) , which means that the larger the postdischarge threshold is, the longer the following interspike interval. Given that v0 · 0 < m tv (the postdischarge potential is reset below the resting voltage) and that v ( u ) is monotonic increasing, we have that D (S ) > D (S0 ) implies v (D ( S ) ) > v (D ( S0 ) ) . This relation together with equation 3.5 and a > 0 yields that F ( S ) > F ( S0 ) whenever S > S0 , that is, F is monotonic increasing. This, combined with the observation that F (s 0 C asr ) > s0 C asr , and F (S ) ! s0 C am tv as S ! C 1, ensures that for any S1C 2 [s0 C asr , C 1) , the sequence SnC is either increasing or decreasing and converges to some value in [s0 C asr , s0 C am tv ]. Finally, all sequences converge to the same value, that is, F has a unique Žxed point, because F is concave. This property comes from the fact

260

M. Chacron, K. Pakdaman, and A. Longtin

that S ! s (u, S ) is contracting, that is, |s(u, S ) ¡ s (u, S0 ) | D exp(¡u / ts ) |S ¡ S0 | < |S ¡ S0 | for u > 0, and v ( u ) is concave, so that F0 , the derivative of F is monotonic decreasing. In summary, we have established that the dynamics of the modiŽed LIF with constant current is governed by SnCC 1 D F ( SnC ) ,

(3.9)

where F, which maps [s 0 C asr , C 1) onto [s 0 C asr , s0 C am tv ) , is a concave monotonic increasing function with a unique Žxed point that we denote by S¤ . Thus, for all S1 < S¤ , the sequence SnC increases and converges to S¤ , and, conversely, for all S1C > S¤ , the sequence SnC decreases toward S¤ . The Žrst consequence of the above result is that in response to a constant stimulation, the modiŽed LIF stabilizes at a periodic Žring with constant interspike intervals D ¤ given by µ

D¤

D tv ln

¶ a(v 0 ¡ m tv ) . S¤ ¡ s0 ¡ am tv

(3.10)

We remark that since D ( S ) is independent of a, the map F deŽned in equation 3.8 is an increasing function of a. A consequence of this observation is that the value of equilibrium postdischarge S¤ increases with a. Given that D ¤ D D ( S¤ ) , the Žring period also increases with a. In other words, the stationary Žring slows down as the factor representing threshold fatigue increases. We now study how D ¤ varies with the input current m . Although it is not possible in general to solve equation 3.7 to Žnd D ¤ as a function of m , it is possible to solve for m as a function of D ¤ . We get m (D ¤ ) D

¤ ¤ ¤ ¤ ¡s 0 C sr ¡ sr eD / ts C v 0eD / ts e ¡D / tv ¡ av0 e¡D / tv . ¤ ¤ ¤ ¤ tv (a ¡ eD / ts C eD / ts e¡D / tv ¡ ae ¡D / tv )

(3.11)

The derivative of m with respect to D ¤ is always negative under the assumptions v 0 · 0 < sr · s 0, a > 0, and D ¤ ¸ ts ln(a). We will now show that the ISI D ¤ can never be less than ts ln(a) . If a · 1, this condition is trivially satisŽed as ts ln(a) · 0 and D ¤ > 0. We thus concentrate on the case a > 1. The denominator of equation 3.11 is zero when D ¤ D ts ln(a) , so the function m ( D ¤ ) has a pole at this point. Since the function m ( D ¤ ) is monotonically decreasing and continuously differentiable on the range (ts ln(a) , C 1) and limD ¤ !C 1 m (D ¤ ) D sr / tv > 0, we have that limD ¤ !ts ln(a) m ( D ¤ ) D C 1. Thus, as we increase m from sr / tv to C 1, the ISI D ¤ decreases from 1 to ts ln(a) . The ISI D ¤ can thus never be lesser than ts ln(a) if a > 1. This has important implications for the model dynamics. In particular, it implies that the stationary ISI D ¤ remains greater than ts ln(a) as we increase m to arbitrarily large values.

Memory and Adaptation in a Model Neuron

261

Moreover, the fact that m ( D ¤ ) is a decreasing function of D ¤ implies that the Žring frequency 1 / D ¤ is always a monotonically increasing function of m . However, if a > 1, the Žring frequency will saturate to a Žnite value given by (ts ln(a) ) ¡1 . As mentioned above, this feature is absent from the LIF model (a D 0, sr D s 0 ) where the Žring rate diverges as a function of the input current. This type of behavior is commonly seen in experimental recordings and is usually associated with absolute refractoriness. Note, however, that the saturation in Žring rate does not imply that an absolute refractory period is present in the model. Indeed, the model can give rise to an ISI smaller than ts ln(a) following a sufŽciently high current pulse or step (see Figure 2C). Thus, we show that even in the absence of an absolute refractory period in the classical sense, threshold fatigue can give rise to a saturation in the Žring frequency under constant current input. This has important implications; it may be that the saturation in Žring rate seen in experimental recordings under constant depolarizing current is due to threshold fatigue rather than the absolute refractory period of the neuron. A modiŽcation dm of the value of m leads to a transient modiŽcation of the discharge rate of the model. Notably, when dm > 0, the sequence Sn increases from the previous value of S¤ toward the new value, resulting in a progressive lengthening of intervals to the new stationary ISI. Note that this ISI will be shorter than the old one. If initially m tv < sr is subthreshold and the new value is suprathreshold, the transient regime in which the ISIs are shorter than in the stationary regime corresponds to neuronal adaptation, that is, a transient frequency increase at the onset of stimulation. The above results hold for all a > 0. However, some distinctions exist between the different values of a that clarify the role of this parameter. One point is that for a · 1, the map F is contracting, that is, F0 (S ) < 1 for all S, but this is not necessarily so for a > 1, where the map may be expanding in some intervals of S. This means that the effects such as discharge rate adaptation tend to be more pronounced in maps with large a than in those with small a. Figure 2A shows the response of the model to a step current going from the subthreshold regime to the suprathreshold regime. Note that the ISIs progressively increase to the new equilibrium value as is seen in experimental recordings (Adrian & Zotterman, 1926). Figure 2B shows the response of the model when the value of the current before the step was set to be in the suprathreshold regime with all other parameters unchanged. Figure 2C shows the response of the model to the same step as in Figure 2B but with a increased. Note the faster rate of adaptation. The rate at which the model adapts to a step current can thus be varied by changing the parameter a. 3.2 Gaussian White Noise Stimulation. Throughout this section, we assume W D W1 and that I ( t) D m C sj ( t) where m is constant and j is white gaussian noise with unit intensity. In this situation, the voltage and threshold of the model are stochastic processes that depend on the particular noise

262

M. Chacron, K. Pakdaman, and A. Longtin

Figure 2: Response of the model to a step increase in current. (A) The current goes from subthreshold (m D 0.1) to suprathreshold (m D 0.9) at t D 50. Note the adaptation in the threshold and the progressive lengthening of ISIs to the new equilibrium value. Parameter values used were ts D 2, tv D 1, a D 1, sr D 0.2, s0 D 0.1, v 0 D 0. (B) The current goes from m D 0.4 to m D 0.9 with all other parameters unchanged. (C) Illustration of the effects of increased a. m goes from 0.4 to 0.9 but a D 10 with all other parameters unchanged. Note the increased rate of adaptation.

realization. The interspike intervals are deŽned as the Žrst passage times (FPTs) of the voltage through the threshold. When a D 0, that is, in the absence of threshold fatigue, the ISIs are independent and identically distributed random variables. We denote by g ( t | s0 ) the probability density function (pdf) of these variables, that is, the function g ( t | s0 ) is the FPT pdf of the Ornstein-Uhlenbeck process g, dg D ¡g / tv C m C sj ( t) with g(0) D g 0, dt

(3.12)

Memory and Adaptation in a Model Neuron

263

through the threshold s ( t) D ( s0 ¡sr ) exp(¡t / ts ) C sr . The FPT pdf g contains all information concerning the point process formed by the discharge times of the model. When a > 0, the description of the discharge times needs to take into account the variations in the postdischarge time threshold. In the same way as for constant stimulation, the key point to the description of the behavior of the model is in establishing the relation between consecutive postdischarge thresholds. In this case, given that the stimulation is noise, the description involves the construction of a Markov chain rather than a map. This is detailed in the following. The conditional pdf of SnCC 1 given SnC can be written as µ ¶ ts a(SnC ¡ sr ) C | Sn , P 1 ( u | Sn ) D g ts ln u ¡ s 0 ¡ asr u ¡ s0 ¡ asr C

(3.13)

where as before g represents the FPT pdf of the Ornstein-Uhlenbeck process, equation 3.12, through the threshold s (t, SnC ) D ( SnC ¡ sr ) exp(¡t / ts ) C sr . The pdf P 1 deŽnes the Žrst-order transition probabilities of an irreducible Markov chain. We denote by h¤ ( S ) the stationary distribution of this chain— in other words, in the long run, the probability for the postdischarge threshold to take a value within (S, S C dS ) is given by h¤ (S ) dS. The characteristics of the point process formed by the discharge times of the model are then deŽned in terms of this stationary distribution together with g and P 1 . For instance, the pdf of ISI distribution g¤ (t ) is given by Z ¤( ) (3.14) g t D g ( t | S ) h¤ (S ) dS. We are interested in the serial correlation coefŽcient of ISIs deŽned by rp D

hD n D n C p i ¡ hD n i2 , hD 2n i ¡ hD n i2

(3.15)

where hD n D n C p i, hD 2n i, and hD n i are the expectations of D n D n C p , D 2n , and D n , respectively. R For n large, the last two quantities are determined by hD n i D tg¤ ( t) dt R and hD 2n i D t2 g¤ ( t) dt. For the Žrst term—the expectation of the product of two intervals—we Žrst consider the case where n is large and p D 1. Given SnC , the postdischarge threshold after the nth discharge, the ISI between the nth and n C 1th Žrings, that is, D n , is distributed according to g (t | SnC ) . The knowledge of SnC and D n uniquely determines SnCC 1 as SnCC 1 D (SnC ¡ sr ) e¡D n / ts C sr D f (SnC , D n ).

(3.16)

This knowledge in turn yields that the ISI between the n C 1th and n C 2th Žrings, D n C 1 , has pdf g ( t | SnCC 1 ) D g ( t | f ( SnC , D n ) ). Combining these with

264

M. Chacron, K. Pakdaman, and A. Longtin

the fact that for n large, SnC is distributed according to h¤ yields Z Z Z hD n D n C 1 i D

S

D D0

DD 0 £ g[D 0 | f (S, D )] £ g[D | S] £ h¤ (S ) dD 0 dD dS.

(3.17)

In a similar way, we derive Z Z Z Z hD n D n C p i D

S

D S0 D 0

DD 0 £g[D 0 | S0 ]P p¡1 [S0 | f ( S, D ) ]£g[D | S] £ h¤ ( S ) dD 0 dS0 dD dS,

(3.18)

where Pp¡1 is the p ¡ 1th transition pdf, R that is, P 0 (u | S ) D d ( u ¡ S ) the Dirac function at S, and P kC 1 ( u | S ) D S0 P 1 ( u | S0 ) P k (S0 | S ) dS0 . In the absence of threshold fatigue, when a D 0, the postdischarge threshold is a Žxed value s 0, so that P 1 ( u | S ) D d (u ¡ s0 ) , h¤ ( S ) D d (S ¡ s0 ), and f ( S, D ) D s 0. Substituting these into equation 3.18 yields that for p ¸ 1, hD n D n C p i D hD n ihD n C p i and consequently that rp D 0. The analysis of the previous paragraphs establishes that the response to gaussian white noise of the modiŽed LIF with threshold fatigue is not described by a renewal process when a > 0. This is in contrast with the standard LIF whose response to white gaussian noise can be described by a renewal process. It also revealed that the modiŽed model with noisy forcing was characterized by a Markov chain relating consecutive postdischarge thresholds. Finally, it showed that the dependence of a postdischarge threshold on the previous value induces correlation between ISIs. Once it is theoretically established that ISIs of the modiŽed model are correlated with one another, we examine the nature of this dependence. For this purpose, rather than using the integral expressions obtained previously, it is appropriate to use numerical simulations of the model. Indeed, the integrals require the computation of the FPT pdf of the Ornstein-Uhlenbeck process through an exponential boundary. Given that no general analytical expression is available for this quantity, derivation of the correlation from the integrals can be computationally more demanding than estimating the same quantities from simulations. Figure 3A shows the ISI distribution g¤ ( t) obtained when the system is driven by gaussian white noise for a D 1, and Figure 3B shows the ISI correlation coefŽcients rp . The estimated r 1 is negative. Serial correlation coefŽcients at higher lags are close to zero, illustrating the rapid return of the system to equilibrium. The fact that r 1 is negative and that it is the only coefŽcient substantially different from zero implies that ISIs shorter (longer) than average will be followed by ISIs longer (shorter) than average. Figure 3C shows the ISI distribution obtained for a D 4. We can see that increasing a increases the mean ISI. Also, the magnitude of the coefŽcient r1

Memory and Adaptation in a Model Neuron

265

Figure 3: (A) ISI distribution obtained for a D 1 in the presence of gaussian white noise of standard deviation 0.1. (B) Correlation coefŽcients rj as a function of lag. Note that only r1 D ¡0.38 is negative and that all coefŽcients are zero for higher lags. (C) ISI distribution obtained for a D 4. (D) Correlation coefŽcients. Note that r1 D ¡0.48 is lower than for a D 1. Other parameter values were ts D 8, tv D 1, m D 1, sr D 0, s0 D 1.

increases with a, which is consistent with the predictions that more memory will give rise to more pronounced ISI correlations. The magnitude of the serial correlation coefŽcient at lag one depends on the slope of the map S ! F (S ) and on the strength of the noise. For high noise intensity, it is expected that the noise will wash out the correlations induced by the map. We thus expect an increase in r1 as a function of noise intensity. This is shown in Figure 4, where noise activates the deterministic properties of the map. The value of r1 has already been linked to the detectability of weak signals in neurons as well as their information transfer capabilities (Chacron et al., 2001a).

266

M. Chacron, K. Pakdaman, and A. Longtin

Figure 4: r1 as a function of the noise standard deviation. r1 exhibits a minimum for the noise intensity around 0.2. It is at this noise level that the noise is most effective at perturbing the map without itself destroying the ISI correlations. Parameter values were the same as in Figure 3 with a D 1. Twenty thousand ISIs were used in each case.

To understand how the deterministic properties of the map can give rise to ISI correlations in the presence of noise, we study the response of the model to perturbations without noise (s D 0). We assume that the stationary regime has been reached, that the model Žres periodically, and that a single sufŽciently large pulsatile stimulation is delivered a time lapse h < D after a discharge, to make the neuron Žre ahead of time. This results in a shortened ISI with length h , and consequently the postdischarge threshold S (h C ) is larger than S¤ . Following the perturbation, the sequence Sn progressively decreases toward S¤ . Given that the interspike interval D ( S) is a monotonic increasing function of S, the sudden increase in S at the time of the perturbation and its progressive decrease indicate that the shortened interval is followed by one that is longer than the period and that subsequent intervals progressively decrease toward the stationary value. If the system returns quasi-instantaneously to equilibrium, then only r 1 will be signiŽcantly negative. However, if the system takes a long time to return to equilibrium after the perturbation, then coefŽcients at higher lags can also be negative. We concentrate on the regime at which

Memory and Adaptation in a Model Neuron

267

the voltage has almost reached its asymptotic value m tv before a Žring: this occurs when ts À tv . We start from equation 3.3 in which we set v (D n C 1 ) D m tv : S (D n C 1 , SnC ) D m tv .

(3.19)

The stationary threshold value is then S¤ D s0 C am tv , and equation 3.10 gives us the stationary ISI D ¤ :

D¤

D ts ln

s 0 C am tv ¡ sr . m tv ¡ sr

(3.20)

Let us suppose that a voltage perturbation advances or delays an action potential, thus causing an ISI h that can be smaller or greater than D ¤ . Then the new value of the threshold immediately after the action potential is given by µ ³ ´¶ ¡h . Sh D s0 C a sr C ( s0 C am tv ¡ sr ) exp ts C

(3.21)

The next ISI, in the absence of further perturbations, will then be given by » hnext D ts ln

¼ ShC ¡ sr . m tv ¡ sr

(3.22)

Equations 3.21 and 3.22 give us the relation between hnext and h: ( hnext D ts ln

s0 ¡ sr C a[sr C ( s0 C am tv ¡ sr ) exp(¡ ths ) ] m tv ¡ sr

) .

(3.23)

The derivative of the map hnext D f (h ) is given by f 0 (h ) D

¡a(s 0 C am tv ¡ sr ) s0 ¡ sr C a[sr C ( s0 C am tv ¡ sr ) exp(¡ ths ) ]

(3.24)

and is negative for s0 > sr > 0 and am tv > 0. We now consider the effects of a on the ISI hnext . If a D 0, we have » ¼ s ¡s D ¤ D hnext D ts ln 0 r , (3.25) m ¡ sr that is, hnext is independent of h since we have f 0 (h ) D 0. This is because the system returns instantaneously to equilibrium and there is no memory extending beyond one ISI. We now consider the effects of a > 0. hnext increases with decreasing h. Thus, a perturbation that causes h larger (smaller)

268

M. Chacron, K. Pakdaman, and A. Longtin

Figure 5: Voltage (black solid line) and threshold (gray solid line) time series. A perturbation in voltage was applied at t D 1098. The perturbation caused an action potential to occur earlier than expected. As such, the ISIh was shorter than D ¤ , and the value of the threshold immediately after that action potential, shC , was higher than the threshold equilibrium value S¤ . Consequently, the threshold took a longer time to decay to m , and the next ISI hnext was longer than D ¤ . Parameter values were the same as in Figure 4.

than D ¤ will make hnext smaller (larger) than D ¤ . Furthermore, increasing a increases hnext. We illustrate this with numerical examples. Figure 5 shows a time series in which a perturbation in the voltage was applied at t D 1099. The perturbation increased voltage and caused an action potential to occur prematurely. The ISI h is thus shorter than D ¤ and consequently ShC (see equation 3.21) is greater than S¤ . Since the threshold starts from a higher value, it will take a longer time to decay to the value m tv . As such, the next ISI hnext will be greater than D ¤ . Similarly, had h been longer than D ¤ , shC would have been smaller than S¤ , and consequently hnext would have been smaller than D ¤ . The actual map hnext D f (h ) is shown in Figure 6A and has a negative slope as expected. Thus, the deterministic properties of the map lead to a perturbed ISI longer (shorter) than average being followed by an ISI shorter (longer) than average. Figure 6B illustrates the dependence of hnext on a for different values of h. As discussed, hnext increases with a. The rate at which hnext increases is, however, greater for low h than for higher h . This shows how the determin-

Memory and Adaptation in a Model Neuron

269

Figure 6: (A) ISI hnext as a function of h (solid black line). The slope of the map is negative as expected (see the text). (B) ISI hnext as a function of a for different values of h . Increasing a will increase hnext . Parameters have the same value as in Figure 5 except a.

istic properties of the map in response to perturbations can give rise to ISI correlations in the presence of noise. 3.3 Sinusoidal Stimulation. In this section, we consider the case where the stimulation is sinusoidal I ( t) D m C a sin(vt) , with a ¸ 0. We show that the response of the LIF with threshold fatigue to such inputs is described by iterates of an annulus map. Such maps can exhibit complex dynamics that can even be chaotic. Through numerical simulations, we illustrate the occurrence of such regimes in the periodically forced LIF with threshold fatigue. Assuming a Žring took place at time tn , we denote by v (t, tn ) and s (t, tn , SnC ) the voltage and threshold at time t, given the initial conditions v (tn , tn ) D v 0 and s (tn , tn , SnC ) D SnC . Provided t ¸ tn and no discharge takes place on

270

M. Chacron, K. Pakdaman, and A. Longtin

[tn , t], we have v (t, tn ) D m tv (1¡exp[(tn ¡t) / tv ]) C

atv [sin(wt) ¡wtv cos(wt) ] 1 C w2 tv2

atv exp[(tn ¡ t) / tv ] [sin(wtn ) ¡ wtv cos(wtn ) ] 1 C w2 tv2 ³ ´ tn ¡ t . s (t, tn , SnC ) D sr C ( SnC ¡ sr ) exp ts ¡

(3.26) (3.27)

p If m tv C atv / 1 C v2 tv2 · sr , then v ( t, tn ) < s (t, tn , SnC ) for all t ¸ tn (i.e., no substained Žring occurs), and, conversely, if q m tv C atv / 1 C v2 tv2 > sr ,

(3.28)

then there exists some Žnite tn C 1 , with tn C 1 > tn such that v ( tn C 1 , tn ) D s ( tn C 1 , tn , SnC ). In other words, the inequality 3.28 is a necessary and sufŽcient condition for sustained Žring of the model. When this inequality holds, the LIF with threshold fatigue will generate an inŽnite sequence of Žring times fti g1 i D 1 with tn ! 1 as n ! 1. We will assume that this is the case throughout the remainder of this section. The condition for sustained Žring is independent of the initial voltage and threshold values; however, the sequence of discharge times depends on these quantities. In the following, we describe the relation between successive discharge times and postdischarge thresholds through the construction of an annulus map. The Žring time tn C 1 is given by def

tn C 1 D F1 ( tn , SnC ) D infft: t > tn , v ( t, tn ) D s ( t, tn , SnC ) g,

(3.29)

and the threshold SnCC 1 immediately after the Žring is given by def

SnCC 1 D F2 ( tn , SnC ) D s0 C W ( sr C (SnC ¡ sr ) e

tn C 1 ¡tn ts

, a).

(3.30)

We deŽne the discharge map as F: ( tn , SnC ) ! ( tn C 1 D F1 ( tn , SnC ) , SnCC 1 D F2 ( tnp, SnC ) ) on [0, 1) £ [s0 C W ( sr , a) , s0 C W (VM , a)], where VM D m tv C atv / 1 C v2 tv2 . For neural computation, it is of interest to know where spikes occur in relation to periodic forcing (see HopŽeld, 1995). Given that the input current I is T-periodic with T D 2p / v, we thus deŽne the Žring phase w n 2 [0, 2p ) as » µ ¶¼ tn w n D w tn ¡ Tint , T

where the int[.] denote the integer part.

(3.31)

Memory and Adaptation in a Model Neuron

271

We can thus associate to F a map f D ( f1 , f2 ) on the annulus [0, 2p ) £ [s 0 C W ( sr , a) , s0 C W ( VM , a) ] as ³ ´ 2p T w n , SnC modulo T (3.32) f1 (w n , SnC ) D F1 2p T ³ ´ T C C w n , Sn . (3.33) f2 (w n , Sn ) D F2 2p The behavior of the periodically forced LIF with threshold fatigue is completely characterized by the iterates of the discharge map F and its associated annulus map f , in the sense that given the initial discharge time t1 and the corresponding postdischarge threshold S1C , the following discharge times and postdischarge thresholds are determined iteratively as ( tn C 1 , SnCC 1 ) D F (tn , SnC ) . We will concentrate on the annulus map f and the sequence of Žring phases and postdischarge thresholds (w i , SiC ) 1 i D 1 . When a D 0, the postdischarge threshold is always set to s0, so that the annulus map reduces to a map on the circle of radius s0 . Furthermore, when sr D s 0, that is, when dealing with the standard LIF without threshold fatigue, this circle map, when restricted to its range, is orientation preserving (Pakdaman, 2001). In such a situation, the system can mainly display phase locking and quasiperiodic behavior, as well as strange nonchaotic dynamics on a parameter set of zero measure. It cannot produce chaos (Coombes, 1999). However, when a > 0, no such restrictions hold. Although we did not observe chaotic behavior with W D W1 for the region of parameter space considered in this article, it is possible to observe sensitivity to initial conditions and chaotic dynamics when a nonlinear form is taken for W. We observed chaotic behavior when we chose W (s, a) D W2 (s, a) ´ exp(as) ¡1 (data not shown). Through systematic numerical simulations, we have observed that such dynamics can be easily observed when a is large and that the sensitivity to initial conditions is associated with a positive leading Lyapunov exponent. The details of this analysis will be described in another study (Chacron, Pakdaman, & Longtin, 2002). 4 Discussion In this study, we have shown that a simple single neuron model that accounts for the memory seen in several classes of neurons has surprisingly rich dynamical behavior. Previous studies involved either a D 0 (Geisler & Goldberg, 1966; Kretzberg, Egelhaaf, & Warcheza, 2001) or a D 1 (Geisler & Goldberg, 1966; Holden, 1976; Vibert et al., 1994; Pakdaman & Vibert, 1995; Pakdaman et al., 1996; Chacron et al., 2000, 2001a, 2001b; Liu & Wang, 2001). We instead varied a 2 [0, 1) continuously and studied its effects. Different values for the parameter a (measuring the amount of memory in the system) gave rise to qualitatively different regimes. The rate and degree of adaptation to a step in current were shown to vary with a. Adaptation is commonly seen in biological neurons; we have found that this interesting

272

M. Chacron, K. Pakdaman, and A. Longtin

feature could be reproduced by our model and that the rate of adaptation depended on the amount of memory. Another common feature of biological neurons is the saturation in Žring rate as the input current is increased. Our model reproduces this feature when a > 1, in contrast with the LIF neuron for which the Žring rate diverges as a function of the input current. In experimental recordings, this saturation is usually thought to be associated with an absolute refractory period or network effects. Our results show that an absolute refractory period is not necessary to obtain such an effect. In fact, this prediction could be veriŽed experimentally in vitro if the Žringrate saturation due to adaptation is lesser than the Žring-rate saturation due to the absolute refractory period. Under the inuence of perturbations and noise, we have shown that the model could give rise to negative ISI interval correlations. The value and rate of decay of these correlations depend on the parameter a and on the noise strength and can be varied to give rise to very different regimes. Expressions were derived for the ISI correlation coefŽcients. The model was also shown to give rise to ISI correlations similar to those seen in experimental data (Longtin & Racicot, 1998; Chacron et al., 2000). Further, the value of the correlation coefŽcient at lag 1 was shown to increase as a function of noise intensity. These correlations were shown to increase the detectability of weak signals in neurons (Chacron et al., 2001a). There thus might be an optimal noise intensity at which signal detection is maximal. ISI correlations have been observed in experimental data taken from neurons in different sensory systems. The advantages of a negative correlation coefŽcient at lag 1 for stimulus detection through long-term spike train regularization as well as stimulus encoding have been outlined in another study (Chacron et al., 2001a). Moreover, a similar Žring mechanism has already been used successfully to model electroreceptors of weakly electric Žsh that display this negative serial correlation coefŽcient at lag 1 (Longtin & Racicot, 1998; Chacron et al., 2000, 2001a, 2001b) and could potentially be used to model other neurons that also display negative ISI correlations (Kufer et al., 1957; Goldberg et al., 1964; Geisler & Goldberg, 1966). In fact, models similar to our own have already been used to model several classes of neurons in the visual system (Keat, Reinagel, Reid, & Meister, 2001), cortical neurons (Liu & Wang, 2001), and electroreceptor afferents (Brandman & Nelson, 2002). Results similar to our own can be obtained by having adaptation in the reset value of the membrane potential. Geisler and Goldberg (1966) have shown that an adapting reset could give rise to ISI correlations. However, as mentioned, there is experimental evidence for the threshold to Žring being dependent on the spiking history (Azouz & Gray, 1999), while there is, to our knowledge, no such evidence for the reset value. We thus believe a dynamic threshold to be more physiologically realistic than a dynamic reset.

Memory and Adaptation in a Model Neuron

273

Although our model contains two timescales, it cannot reproduce the bursting dynamics seen in many neurons with the parameter range considered in this study. This is because we considered only threshold fatigue after an action potential and not facilitation. Our model is thus different from the integrate-and-Žre-or-burst model considered by Coombes, Owen, & Smith (2001), which also has two timescales. It has, however, been shown that the addition of a facilitation current to our model with a D 1 could give rise to bursting dynamics (Chacron et al., 2001b). The response of the model to perturbations and noise will carry over in the presence of sinusoidal forcing (Chacron et al., 2000). However, it is then impossible to obtain analytical explicit expressions for the ISI under voltage perturbations. Furthermore, under sinusoidal forcing, we have shown that our model could be described by a map deŽned on an annulus. While this may seem a simple extension of the many studies that have described the dynamics of sinusoidally forced standard LIFs by circle maps (Rescigno et al., 1970; Keener et al., 1981; Coombes & Bressloff, 1999; Pakdaman, 2001), it has major implications in terms of the dynamics of the system. Indeed, the circle map associated with the periodically forced standard LIF is oneto-one and orientation preserving, so that it produces only one of the three following regimes: phase locked, quasi-periodic, or strange nonchaotic behavior (Keener, 1980). For Žxed parameters, it can produce neither a mix of these nor chaos. In contrast, the annulus map associated with the LIF model with threshold fatigue is not constrained by such limitations and may very well lead to more complex dynamics (Le Calvez, 2000), in agreement with our numerical investigations (details will be described in a future work). Also, introducing threshold fatigue as in our model yields an adapting sequence of ISIs in the presence of periodic forcing. In contrast, for a step increase in bias current, models in which periodic forcing is taken into account by sinusoidally modulating the threshold or the voltage reset value see an abrupt change in ISI. It should be noted that the annulus map is dissipative and not conservative in general. This system is thus different from the so-called standard map studied by various authors (e.g., Meiss, 1992). Our results suggest that neurons with strong adaptation are more prone to displaying complex dynamics than those that either do not adapt or adapt more slowly. Such sensitivity to initial conditions may confer some advantages, as chaotic systems have the potential to transmit much more information about time-varying stimuli than nonchaotic ones, depending on the Lyapunov exponents of the system (Abarbanel, 1996). Further studies are needed to investigate such possible functional roles of chaos in signal processing in nervous systems. In this study, we have considered how the addition of threshold fatigue affected the spike train of a single LIF neuron. However, it is clear that adaptation and ISI correlations could also be due to network or synaptic dynamics. To expand on this point, we now discuss the possible biological mechanisms that the threshold fatigue could model.

274

M. Chacron, K. Pakdaman, and A. Longtin

Cumulative inactivation of sodium channels (Mickus, Jung, & Spruston, 1999) could, for example, give rise to neural adaptation and ISI correlations. However, a fast spike-activated slowly inactivating negative current could also give rise to similar effects: a likely candidate would be members of the KV family of potassium currents (Wang, Gan, Forsythe, & Kaczmarek, 1998). Another candidate could be a calcium-activated potassium current such as Iahp . This current was shown to be present in cortical neurons (Madison & Nicoll, 1984). Liu and Wang (2001) have shown that a model with an Iahp current was qualitatively equivalent to our threshold fatigue model with W D W1 (i.e., a linear threshold reset) and a D 1. It is further known that most ionic currents have voltage-dependent conductances. The parameter s0 represents the amount of reset that is voltage independent. The parameter a in our model could then represent the degree of dependence on the past Žring time in the ionic channel that the dynamic threshold models. Increasing a would thus lead to a more history-dependent conductance. Long-term depression at a synapse could also give rise to relative refractoriness (Hausser & Roth, 1997). In fact, a depressing synapse produces negative spike train correlations (Goldman, Maldonado, & Abbott, 2002). Our model thus bears some similarity with models of short-term plasticity (Fuhrman, Segev, Markram, & Tsodyks, 2002; Goldman et al., 2002). Furthermore, it is known that many synapses are voltage dependent. The parameter a in our model could then represent the voltage dependence of synaptic depression. It should, however, be noted that the recovery time constant of the neurotransmitter at typical synapses is usually in the range of hundreds, if not thousands, of milliseconds (von Gernsdoff, Schneggenburger, Weis, & Neher, 1997). Although neural adaptation usually occurs on much shorter timescales, our model could in principle still be used to account for such phenomena if the threshold time constant is sufŽciently long. Moreover, it has been observed that neural adaptation was in many ways similar to recurrent inhibition (Ermentrout, Pascal, & Gutkin, 2001). Our single-neuron model could therefore describe network effects. It has thus been shown that threshold fatigue can be similar to recurrent inhibition, synaptic depression, or an intrinsic Iahp current responsible for adaptation. Acknowledgments We thank C. Laing, B. Doiron, J. Benda, B. Lindner, and L. Maler for useful discussions. This research was supported by NSERC (M. J. C. and A. L.). References Abarbanel, H. D. I. (1996). Analysis of observed chaotic data. New York: SpringerVerlag.

Memory and Adaptation in a Model Neuron

275

Abbott, L. F., & Dayan, P. (1999). The effect of correlated variability on the accuracy of a population code. Neural Comp., 11, 91–101. Adrian, E. D., & Zotterman, Y. (1926) The impulse produced by sensory nerve endings. Part 2: The response of a single end-organ. J. Physiol. (Lond.), 61, 151–171. Alstrøm, P., & Levinsen, M. T. (1988). Phase-locking structure of “integrate-andŽre” models with threshold modulation. Phys. Lett. A, 128, 187–192. Azouz, R., & Gray, C. M. (1999). Cellular mechanisms contributing to response variability of cortical neurons in vivo. J. Neurosci., 19, 2209–2223. Brandman, R., & Nelson, M. E. (2002). A simple model of long-term spike train regularization. Neural Comp., 14, 1507–1544. Chacron, M. J., Longtin, A., & Maler, L. (2001a).Negative interspike interval correlations increase the neuronal capacity for encoding time dependent stimuli. J. Neurosci., 21, 5328–5343. Chacron, M. J., Longtin, A., & Maler, L. (2001b). Simple models of bursting and non-bursting P-type electroreceptors. Neurocomputing, 38, 129–139. Chacron, M. J., Longtin, A., St.-Hilaire, M., & Maler, L. (2000). Suprathreshold stochastic Žring dynamics with memory in P-type electroreceptors. Phys. Rev. Lett., 85, 1576–1579. Chacron, M. J., Pakdaman, K., & Longtin, A. (2002). Chaotic Žring in the sinusoidally forced leaky integrate and Žre model with threshold fatigue. Unpublished manuscript. Coombes, S. (1999). Lyapunov exponents and mode-locked solutions for integrate-and-Žre dynamical systems. Phys. Lett. A., 255, 49–57. Coombes, S., & Bressloff, P. C. (1999). Mode locking and Arnold tongues in integrate-and-Žre neural oscillators. Phys. Rev. E., 60, 2086–2096. Coombes, S., Owen, M. R., & Smith, G. D. (2001). Mode locking in a periodically forced integrate-and-Žre-or-burst neuron model. Phys. Rev. E, 64, 041914. Ermentrout, B., Pascal, M., & Gutkin, B. (2001). The effects of spike frequency adaptation and negative feedback on the synchronization of neural oscillators. Neural Comp., 13, 1285–1310. Fuhrmann, G., Segev, I., Markram, H., & Tsodyks, M. (2002). Coding of temporal information by activity-dependent synapses. J. Neurophysiol., 87, 140–148. Geisler, C. D., & Goldberg, J. M. (1966). A stochastic model of the repetitive activity of neurons. Biophys. J., 7, 53–69. Gerstner, W., & van Hemmen, J. L. (1992). Associative memory in a network of “spiking” neurons. Network, 3, 139–164. Glass, L., & Mackey, M. C. (1979). A simple model for phase locking of biological oscillators. J. Math. Biol., 7, 339–352. Goldberg, J. M., Adrian, H. O., & Smith, F. D. (1964). Response of neurons of the superior olivary complex of the cat to acoustic stimuli of long duration. J. Neurophysiol., 27, 706–749. Goldman, M. S., Maldonado, P., & Abbott, L. F. (2002). Redundancy reduction and sustained Žring with stochastic depressing synapses. J. Neurosci., 22, 584–591.

276

M. Chacron, K. Pakdaman, and A. Longtin

Hausser, M., & Roth, A. (1997). Dendritic and somatic glutamate receptor channels in rat cerebellar Purkinje cells. J. Physiol. (Lond.), 501, 77–95. Hodgkin, A. L., & Huxley, A. F. (1952). A quantitative description of membrane current and its application to conduction and excitation in nerve. J. Physiol. (Lond.), 117, 500–544. Holden, A. V. (1976). Models of the stochastic activity of neurones. New York: Springer-Verlag. HopŽeld, J. J. (1995). Pattern recognition computation using action potential timing for stimulus representation. Nature, 376, 33–36. Keat, J., Reinagel, P., Reid, R. C., & Meister, M. (2001). Predicting every spike: A model for the response of visual neurons. Neuron, 30, 803– 817. Keener, J. P. (1980). Chaotic behavior in piecewise continuous difference equations. Trans. Amer. Math. Soc., 261, 589–604. Keener, J. P., Hoppensteadt, F. C., & Rinzel, J. (1981). Integrate-and-Žre models of nerve membrane response to oscillatory input. SIAM J. of Appl. Math., 41, 816–823. Kreztberg, J., Egelhaaf, M., & Warzecha, A. K. (2001). Membrane potential uctuations determine the precision of spike timing and synchronous activity: A model study. J. Comput. Neurosci., 10, 79–97. Kufer, S. W., Fitzhugh, R., & Barlow, H. B. (1957). Maintained activity in the cat’s retina in light and darkness. J. Gen. Physiol., 40, 683–702. Lansky, P., & Sato, S. (1999). The stochastic diffusion models of nerve membrane depolarization and interspike interval generation. J. Peripher. Nerv. Syst., 4, 27–42. Le Calvez, P. (2000) Dynamical properties of diffeomorphisms of the annulus and of the torus. Providence, RI: American Mathematical Society. Liu, Y. H., & Wang, X. J. (2001). Spike frequency adaptation of a generalized leaky integrate-and-Žre neuron. J. Comput. Neurosci., 10, 25–45. Longtin, A., & Racicot, D. M. (1998). Spike train patterning and forecastability. Biosystems, 40, 111–118. MacGregor, R. J., & Lewis, E. R. (1977). Neural modeling. New York: Plenum Press. Madison, D. V., & Nicoll, R. A. (1984). Control of the repetitive discharge of rat CA1 pyramidal neurones in vitro. J. Physiol., 354, 319–331. Mickus, T., Jung, H. Y., & Spruston, N. (1999). Properties of slow cumulative sodium channel inactivation in rat hippocampal CA1 pyramidal neurons. Biophys. J., 76, 846–860. Meiss, J. D. (1992). Symplectic maps, variational principles, and transport. Rev. Mod. Phys., 64, 795–848. Neiman, A., & Russell, D. F. (2001). Stochastic biperiodic oscillations in the electroreceptors of paddleŽsh. Phys. Rev. Lett., 86, 3443–3446. Pakdaman, K. (2001). The periodically forced leaky integrate-and-Žre model. Phys. Rev. E., 63, 1907–1912. Pakdaman, K., Tanabe, S., & Shimokawa, T. (2001). Coherence resonance and discharge time reliability in neurons and neuronal models. Neural Networks, 14, 895–905.

Memory and Adaptation in a Model Neuron

277

Pakdaman, K., & Vibert, J. F. (1995). Modeling excitatory networks. Journal of Sice, 34, 788–793. Pakdaman, K., Vibert, J. F., Boussard, E., & Azmy, N. (1996). Single neuron with recurrent excitation: Effect of the transmission delay. Neural Networks, 9, 797– 818. Panzeri, S., Petersen, R. S., Schultz, S. R., Lebedev, M., & Diamond, M. E. (2001). The role of spike timing in the coding of stimulus location in rat somatosensory cortex. Neuron, 29, 769–777. Reiss, R. F. (1962). A theory and simulation of rhythmic behavior due to reciprocal inhibition in small nerve nets. Proceedings of the 1962 AFIPS Spring Joint Computer Conference, 21, 171–194. Rescigno, A., Stein, R. B., Purple, R. L., and Popele, R. E. (1970).A neuronal model for the discharge patterns produced by cyclic inputs. Bull. Math. Biophys., 32, 337–353. Ricciardi, L. M., Di Crescenzo, D., Giorno, V., & Nobile, A. G. (1999). An outline of theoretical and algorithmic approaches to Žrst passage time problems with applications to biological modelling. Mathematica Japonica, 50, 247–322. Sch¨afer, K., Braun, H. A., Peters, C., & Bretschneider, F. (1995). Periodic Žring pattern in afferent discharges from electroreceptor organs of catŽsh. Eur. J. Physiol., 429, 378–385. Segundo, J. P., Perkel, D. H., Wyman, H., Hegstad, H., & Moore, G. P. (1968). Input output relations in computer simulated nerve cells. Kybernetik, 4, 157– 175. Stein, R. B. (1965). A theoretical analysis of neuronal variability. Biophys. J., 5, 173–194. Teich, M. C., & Lowen, S. B. (1994). Fractal patterns in auditory nerve spike trains. IEEE Eng. Med. Biol. Mag., 13, 197–202. Tiesinga, P. H. E., Fellous, J. M., JosÂe, J. V., & Sejnowski, T. J. (2002). Information transfer in entrained cortical neurons. Network, 13, 41–66. Treves, A. (1996). Mean-Želd analysis of neuronal spike dynamics. Network, 4, 259–284. Tuckwell, H. C. (1988). Introduction to theoreticalneurobiology:I. Cambridge: Cambridge University Press. Vibert, J. F., Pakdaman, K., & Azmy, N. (1994). Inter-neural delay modiŽcation synchronizes biologically plausible neural networks. Neural Networks, 7, 589– 607. von Gernsdoff, H., Schneggenburger, R., Weis, S., & Neher, E. (1997). Presynaptic depression at a calyx synapse: The small contribution of metabotropic glutamate receptors. J. Neurosci., 17, 8137–8146. Wang, L. Y., Gan, L., Forsythe, I. D., & Kaczmarek, L. K. (1998). Contribution of the Kv3.1 potassium channel to high-frequency Žring in mouse auditory neurones. J. Physiol. Lond., 509, 183–194. Weiss, T. F. (1966). A model of the peripheral auditory system. Kybernetik, 3, 153–175. Wehmeier, W., Dong, D., Koch, C., & van Essen, D. (1989). Modeling the mamalian visual system. In C. Koch & I. Segev (Eds.), Methods in neuronal modeling. Cambridge, MA: MIT Press.

278

M. Chacron, K. Pakdaman, and A. Longtin

Wilson, D. M., & Waldron, I. (1968). Models for the generation of the motor output pattern in ying locusts. Proceedings IEEE, 56, 1058–1064. Yamamoto, A., & Nakahama, H. (1983). Stochastic properties in somatosensory cortex and mesencephalic reticular formation during sleep-waking states. J. Neurophysiol., 49, 1182–1198. Received April 9, 2002; accepted July 8, 2002.

LETTER

Communicated by Roderick V. Jensen

Reliability of Spike Timing Is a General Property of Spiking Model Neurons Romain Brette [email protected] Centre de MathÂematiques et de Leurs Applications,Ecole Normale SupÂerieurede Cachan, 94230 Cachan, France, and INSERM U483, UniversitÂe Pierre et Marie Curie, 75005 Paris, France

Emmanuel Guigon INSERM U483, UniversitÂe Pierre et Marie Curie, 75005 Paris, France

The responses of neurons to time-varying injected currents are reproducible on a trial-by-trial basis in vitro, but when a constant current is injected, small variances in interspike intervals across trials add up, eventually leading to a high variance in spike timing. It is unclear whether this difference is due to the nature of the input currents or the intrinsic properties of the neurons. Neuron responses can fail to be reproducible in two ways: dynamical noise can accumulate over time and lead to a desynchronization over trials, or several stable responses can exist, depending on the initial condition. Here we show, through simulations and theoretical considerations, that for a general class of spiking neuron models, which includes, in particular, the leaky integrate-and-Ž re model as well as nonlinear spiking models, aperiodic currents, contrary to periodic currents, induce reproducible responses, which are stable under noise, change in initial conditions and deterministic perturbations of the input. We provide a theoretical explanation for aperiodic currents that cross the threshold. 1 Introduction The responses of neurons to dynamic stimuli have been shown to exhibit high reliability in vitro (Mainen & Sejnowski, 1995; Hunter, Milton, Thomas, & Cowan, 1998; Fellous et al., 2001; Beierholm, Nielsen, Ryge, Alstrom, & Kiehn, 2001) and in vivo (Berry, Warland, & Meister, 1997; Nowak, SanchezVives, & McCormick, 1997; Reich, Victor, Knight, Ozaki, & Kaplan, 1997; Berry & Meister, 1998; Buracas, Zador, DeWeese, & Albright, 1998; Bair & Koch, 1996). In this case, spike timing is reproducible on a trial-by-trial basis up to a precision of 1 ms or less, even a long time after stimulus onset (1 s in Mainen & Sejnowski, 1995). Consequently, in a variety of neurons and for time-varying stimuli, spike times convey more information about the stimulus than spike count alone does (Victor & Purpura, 1996; Neural Computation 15, 279–308 (2003)

° c 2002 Massachusetts Institute of Technology

280

R. Brette and E. Guigon

Figure 1: Three cases regarding the reproducibility of neuron responses to a given stimulus. (A) Small variances in ISIs accumulate, so that after some time, spikes are not synchronized between trials. (B) Spike timing is reliable even in the long run but depends on the initial state of the neuron—here, two stable solutions coexist. (C) Spike timing is reliable; noise does not accumulate over time.

de Ruyter van Steveninck, Lewen, Strong, Koberle, & Bialek, 1997; Buracas et al., 1998; Reich, Mechler, Purpura, & Victor, 2000; Reinagel & Reid, 2000). Rate-modulated Poisson processes do not account for these properties (Victor & Purpura, 1996; Baddeley et al., 1997; de Ruyter van Steveninck et al., 1997; Berry et al., 1997; Nowak et al., 1997; Reich, Victor, & Knight, 1998; Kara, Reinagel, & Reid, 2000; Reich et al., 2000). On the other hand, stationary stimuli elicit spike trains with very coarse precision (Mainen & Sejnowski, 1995; Berry et al., 1997; de Ruyter van Steveninck et al., 1997; Buracas et al., 1998). Mainen and Sejnowski (1995) injected a constant current into a neuron and compared the output spike trains over repeated trials. At the beginning, spikes occurred at the same time in all trials, but then small variances in interspike intervals (ISIs) began to accumulate, so that spikes eventually were totally desynchronized over trials. It is not clear why this does not happen with time-varying currents. What happens when the responses of a neuron to a given stimulus are compared over repeated trials, knowing that each trial is different due to intrinsic noise? Three cases may occur, as illustrated in Figure 1: Case 1: Small variances in ISIs accumulate, so that eventually spikes tend to desynchronize over trials.

Reliability of Spike Timing

281

Case 2: Spike times are reproducible in the long run but depend on the initial state of the neuron (e.g., on the membrane potential at the onset of the stimulus); that is, several stable responses exist. Case 3: Spike times are reproducible in the long run and do not depend on the initial state of the neuron. Experimentally, case 1 has been shown to occur for constant input currents, whereas time-varying stimuli induce reproducible responses (Mainen & Sejnowski, 1995), although the experimental procedures used in these studies do not allow us to distinguish between cases 2 and 3. Only in the last case can the responses be considered truly reproducible. In this article, we show that for a general class of spiking neuron models, which includes, in particular, the leaky integrate-and-Žre model (Lapicque, 1907; Knight, 1972) as well as nonlinear spiking models, all three cases can occur if the input current is periodic, while aperiodic currents induce reproducible responses. In addition to numerical simulations, we put forth a theoretical explanation of this property for aperiodic currents that oscillate around threshold. The conditions required for our explanation are not fulŽlled by the nonleaky integrate-and-Žre model—also called perfect integrator—which is never reliable. 2 Materials and Methods 2.1 What Is Reliability? A neuron is said to be reliable when its responses to a given stimulus are reproducible. Thus, reliability is a priori stimulus dependent. The experimental results of Mainen and Sejnowski (1995) show that with a constant current, the responses of a neuron in different trials are initially synchronous, but then small variances in ISIs add up and the spikes become desynchronized. If the intrinsic noise of neurons were lower, it would simply take the spikes longer to desynchronize, but in the end, the result would be the same. However, with nonconstant currents resembling synaptic inputs (low-pass Žltered gaussian noise), desynchronization does not happen. In both cases, intrinsic noise is low at the timescale of a single ISI, but it eventually leads to a desynchronization only in the case of a constant current. Thus, we propose the following deŽnition of reproducibility: responses to a stimulus are considered reproducible if the spike time jitter remains of the order of the intrinsic noise, however long we wait, or, equivalently, that any given precision can be reached if the level of noise is low enough. This deŽnition allows us to distinguish the two cases mentioned above. However, neuron responses can also fail to be reproducible if spike timing depends on the initial condition, that is, the membrane potential at stimulus onset. Therefore, we shall consider in the deŽnition the asymptotic spike time jitter induced not only by intrinsic noise but also by the dispersion of initial conditions.

282

R. Brette and E. Guigon

Our aim is to show that depending on the stimulus, all of the three cases mentioned in section 1 can occur. Furthermore, we shall show that a broad class of models has reproducible responses to aperiodic stimuli. We pay particular attention to leaky models—in a general sense, as deŽned below. However, we also introduce and study other types of models—the perfect integrator and nonleaky models—to show that the nonreliability of the perfect integrator is the exception rather than the rule. 2.2 Models. 2.2.1 The General Model. We consider spiking neuron models, where the membrane potential V evolves according to the following one-dimensional differential equation, dV D f ( V, t) , dt

(2.1)

with the spike modeled as follows: when the potential reaches a threshold Vt , it is reset to Vr . This general deŽnition includes many different models, such as the leaky integrator (Lapicque, 1907; Knight, 1972) and the perfect integrator (Knight, 1972). The equation needs not be linear. Such models are not always reliable, as the example of the perfect integrator shows. However we shall show that a very broad class of models of this kind exhibits reproducible responses. 2.2.2 Leaky Models. One example of such a model is the leaky integrator (Lapicque, 1907; Knight, 1972), where the membrane potential V evolves according to the following equation: t

dV D ¡V C RI ( t) , dt

(2.2)

where t is the membrane time constant, R is the resistance, and I ( t) the input current. We generalize the notion of leak by deŽning a leaky model as a model of type 2.1 satisfying the following condition: @f @V

< 0.

This includes the case when t and R vary in time and also includes nonlinear equations. Basic mathematical properties of leaky models are summarized in the article appendix. In our simulations, we used a leaky integrator with t D 33 ms, R D 200 MV, Vt D 15 mV, and Vr D ¡5 mV, and a nonlinear leaky model deŽned by t

dV D ¡aV3 C RI ( t) , dt

(2.3)

Reliability of Spike Timing

283

with the same values for t , R, Vt , and Vr , and a D 4444 V¡2 (this is for scaling reasons). This corresponds to a neuron model with rectiŽcation, that is, with a voltage-dependent leak. We chose these parameter values so as to obtain Žgures of the same magnitude as in experimental results. However, the actual value of parameters has no inuence on the results, since two sets of parameters actually deŽne the same model (change of variables). 2.2.3 Nonleaky Models. We shall show that the leak is not necessary to obtain reliable spike timing. We simulated an example of a nonleaky model, described by the following equation, t

dV D VI ( t) C k, dt

(2.4)

where I (t) is the input current, with Vt D 1, Vr D 0, t D 33 ms, and k D 1. This model is leaky only if I ( t) < 0 for all t. Otherwise, the solutions of equation 2.4 diverge exponentially in all intervals where I ( t) > 0. However, we shall see that it does not prevent the model from having reliable spike timing. Equation 2.4 does not correspond to a neuron model; we chose it only to illustrate our explanation of reliability. 2.2.4 Perfect Integrator. As an example of a nonreliable model, we have taken the common nonleaky integrate-and-Žre model, t

dV D f ( t) , dt

(2.5)

where f (t ) is an input function. We used t D 12.5 ms (in order to have comparable timescales), Vt D 1, and Vr D 0. We shall see in the simulations that this model is never reliable. Mathematically, the unreliability of this model results from the fact that the distance between two solutions is constant modulo 1, and thus dynamical noise always accumulates (Knight, 1972). 2.3 Stimuli. We tested the reproducibility of model responses to various periodic and aperiodic currents. We also assessed the inuence of current parameters, such as amplitude and mean, on spike timing. For this purpose, we used the following generic method: we deŽne a basis function B (t ) as one with values between ¡1 and 1, such as a sinusoidal wave, and then derive a family of currents by a parameterized afŽne change of variables, for example, I ( t) D Ith C 0.5 C pB ( t) with p 2 [0, 1], where Ith is the input current threshold (the minimum constant current eliciting a response). In this example, we obtain a family of currents with a Žxed DC part and whose amplitude varies from 0 to 1.

284

R. Brette and E. Guigon

Figure 2: Construction of a 2D spike density Žgure. (A) Basis function. A family of currents is deŽned with a parameter controlling the current mean. (B) Trials for a given parameter. Every short vertical line represents a spike. (C) PSTH computed from 2000 trials with the parameter used in B. (D) The PSTH is converted into gray rasters, with the higher the Žring rate, the darker the lines. (E) Gray rasters for all parameters are collected and arranged vertically.

In most simulations, we chose the change of variables so that the current is always above threshold for p 2 [0, 0.5] and oscillates around it for p 2 [0.5, 1]. Subsequently, we want to display the reliability of model responses for all the parameters of a given family of currents on a single Žgure. How this is done is illustrated by Figure 2. For a given basis function (see Figure 2A), we compute the outputs of a large number of trials for each parameter value (see Figure 2B) and then compute a poststimulus time histogram (PSTH) using small time bins (see Figure 2C), which we convert into a line of gray rasters (see Figure 2D). Arranging these lines vertically, we get a two-dimensional spike density Žgure (see Figure 2E), where histograms for all parameter values between 0 and 1 are represented simultaneously, with parameter on

Reliability of Spike Timing

285

the vertical axis and time on the horizontal axis. This allows us to visualize not only the reliability of spike timing for every parameter value, but also the effect of parameters on spike timing and reliability. Thus, we assess the stability of spike timing under noise as well as under deterministic perturbations. For periodic currents, we used sine waves, and for aperiodic currents, we used two types of basis functions: (1) a triangle-shaped function, with the slope of each line chosen according to a random uniform distribution between two values, and (2) a noisier function: a low-pass Žltered gaussian noise that we distorted so that its magnitude distribution over time was uniform between ¡1 and 1 (see, e.g., Figure 4.2.2). This was done to distinguish between the input current always above threshold from the input current oscillating around it. The speciŽc shape of input currents did not inuence the results. 2.4 Measure of Precision. Precision of spike timing was assessed from PSTHs. Most previous measures relied on the identiŽcation of “events” (Mainen & Sejnowski, 1995), deŽned as peaks in the PSTH, whose width is an estimate of the precision. However, assessing the precision only in events might lead to an overestimate, as observed in Tiesinga, Fellous, & Sejnowski (2002a), since the periods when the PSTH is not peaked enough are discarded. Other measures are based on the variance of the PSTH about the mean output frequency (Hunter et al., 1998), but this measure is misleading when the output frequency varies over time. The measure we propose here is based on the entropy of distributions (Borst & Theunissen, 1999). We cannot directly compute the entropy from a PSTH, because this is not a normalized probability distribution. We choose the successive ISIs of one trial to divide the PSTH in successive windows. The models we use have the property that for every other trial, the neuron spikes exactly once in each of these windows. Equivalently, we can split the PSTH in successive windows containing 2000 spikes (the number of trials). We can evaluate the precision of spike timing in any window from the PSTH using the following formula, X HD¡ p (i) log2 p (i) , i

where p ( i) is the number of spikes in time bin i divided by the number of trials. H is the entropy of the distribution of spike times in the window and represents the number of bits required on average to express in which time bin a neuron spikes. One advantage over event-based measures is that if there are m stable solutions, each PSTH window contains m peaks, which gives a higher entropy. However, this quantity depends on the size of the time bin used for the PSTH. We can translate it into a time measure that does not depend on the size of the time bin in the following way. An entropy of H bits is the entropy of a uniform distribution over 2H time bins, that is, over

286

R. Brette and E. Guigon

an interval of size dt2H , where dt is the length of the time bin, which gives the following formula: Á dt exp ¡

X

! p ( i) log p (i) .

(2.6)

i

As dt tends to 0, this quantity tends to ³ Z ´ exp ¡ p ( t) log p ( t) dt , where p ( t) represents the spike time density. Thus, if dt is small enough, the precision measure given by formula 2.6 does not depend on the length of the time bin. As an example, this measure gives a precision of D for a uniform distribution over an interval [t ¡ D / 2, t C D / 2] and a precision of 4s for a gaussian distribution with standard deviation s. We used formula 2.6 to compute the precision in each window and averaged over all successive windows, discarding the Žrst 2.5 s of the 10 s runs (to take the time of convergence into account). In the results, we shall say that neuron responses have a high precision if their measure of precision given by formula 2.6 is small. 2.5 Details of Simulations. Input currents were sampled at a rate depending on the required resolution—2000 Hz in most simulations with the leaky integrator. Equations for the linear leaky integrator and the perfect integrator were solved by exact integration. The equations of the two other models—the nonlinear and the nonleaky ones—were integrated by the Euler method (these were simulated mostly for illustration, and only on a short timescale). Spike timing was computed by bisection with a precision of 1 m s using the following observation: over a single current sampling step (1 / 2000 s), equation 2.1 reduces to an autonomous scalar equation because the input current is constant, which means that V ( t) is monotonous over this duration, and the time when the potential hits threshold can then be computed unambiguously and quickly by bisection. A small noise was added to the potential at each time step of integration. The type of noise used is irrelevant because it tends to a gaussian noise as the time step tends to 0. Therefore, for computational ease, we used a uniform noise with an amplitude ranging from 2.5 mV.s¡1/ 2 to 3.5 mV.s ¡1 / 2 . Reliability of spike timing was assessed by 2000 repeated trials for every tested model and input current over an extended period of time—10 s for the leaky integrator—and PSTHs were computed with a 0.5 ms time bin. Doing many trials avoided the problem of smoothing PSTHs.

Reliability of Spike Timing

287

3 Theoretical Predictions 3.1 Periodic Currents. Responses to periodic currents highlight the two difŽculties that may prevent the neuron responses from being reproducible: (1) a small noise may accumulate over time, or (2) several stable responses may coexist. Because periodically driven leaky models are related to the dynamical theory of homeomorphisms of the circle (Denjoy, 1932; Coddington & Levinson, 1955; Arnold, 1961; Herman, 1977; Keener, 1980; Veerman, 1989), their mathematical properties are well known (Knight, 1972; Keener, Hoppensteadt, & Rinzel, 1981). However, the reliability of periodically driven neuron models is a recent issue (Tiesinga, 2002; Tiesinga, Fellous, & Sejnowski, 2002b). We describe the available theoretical results and explain how they relate to our problem. The dynamics of leaky models depend on the value of the rotation number, deŽned as the ratio of the input frequency to the output spike rate: • It is rational if and only if the output pattern of spikes is (asymptotically) periodic, the period being a multiple of the input period. When this pattern is stable under perturbations, this is called phase locking, which has also been studied experimentally (Rescigno, Stein, Purple, & Poppele, 1970; Ascoli, Barbi, Chillemi, & Petracchi, 1977; Guttman, Feldman, & Jakobsson, 1980; Koppl, 1997). There is p : q phase locking when a stable pattern of p spikes is produced every q periods of the input. Thus when the model is phase-locked, a small noise does not accumulate over time. However, several stable responses may coexist. As noted in Tiesinga (2002), if a p : q phase-locked solution is shifted by a multiple of the input period, a new solution is obtained, so that at least q distinct stable solutions exist. Thus, regarding reliability, cases 2 and 3 can appear. • When it is irrational, under some regularity conditions, the dynamics of the model are topologically equivalent to the dynamics of a model with constant input, which means that a small noise always accumulates over time, however small it may be. Thus, case 1 occurs. When the input current is parameterized, for example, by its frequency and mean, the rotation number is a continuous function of the parameters, which implies that cases 1, 2, and 3 occur in any area of the parameter space where the rotation number is not constant. Thus, there is no systematic relationship between current parameters and reliability. However, while the set of parameters where case 1 occurs has positive measure if the current is always above threshold (Herman, 1977), it has measure zero, and thus is not observable, if the current oscillates around threshold (Keener, 1980; see Veerman, 1989, for a rigorous mathematical proof). To sum up, cases 2 and 3 occur in any area where the current oscillates around threshold; cases 1,

288

R. Brette and E. Guigon

Figure 3: A current below threshold results in an interval where no spiking is possible. Arrows represent whether the vector Želd points up or down at threshold.

2, and 3 occur in any area where the current is above threshold (unless the rotation number is constant). The (asymptotic) spike time jitter is an increasing function of the level of intrinsic noise, which tends to 0 as the noise variance tends to 0 when the model is phase-locked and to a positive number when it is not. Quantitatively, this function depends on various detailed aspects of the dynamics, such as the Lyapunov exponent (Tiesinga, 2002). 3.2 Aperiodic Currents. In the case of aperiodic currents, such as Žltered gaussian noise, we will see in the numerical results that neither case 1 nor case 2 arises. Other studies suggest that the responses are reliable when the current is above threshold (Pakdaman & Tanabe, 2001). Here, we explain why this should also be so when the input current oscillates around threshold. Our explanation relies on the construction of a set of possible spike times, which is done with no noise on the dynamics. In the appendix, we prove that this set is stable under noise and deterministic perturbations. Let t1 be a time when the input current goes below threshold. A solution arriving at threshold at time t1 cannot cross it and will hit threshold again only later, at time t2 , as shown in Figure 3. The area deŽned by this solution and the line V D Vt cannot be reached by any other solution, since trajectories cannot cross. Therefore, no spike can occur between t1 and t2 , which happens every time the current goes below threshold, and so such time intervals can be removed from the possible spike times. The area deŽned by the solutions starting at reset between t1 and t2 cannot be reached by any solution starting before t1 , since such a solution would spike between t1 and

Reliability of Spike Timing

289

Figure 4: Construction of unreachable time intervals. As in Figure 3, time is on the horizontal axis and membrane potential on the vertical axis. The color areas are unreachable by any solution. Thus, the sample solution, shown as a dashed line, always remains in the white area. The simulation was 500 ms long with a leaky integrator.

t2 . The spike times of these forbidden solutions deŽne a whole sequence of unreachable spike time intervals. Therefore, every time the input current goes below threshold, a whole sequence of intervals is removed from the possible spike times, as shown in Figure 4. We can see that the Žrst time the current goes below threshold, a red area that no solution can reach is created near threshold. The solutions starting from the resulting forbidden time interval are in red. The next time the current goes below threshold occurs outside the red area and creates a new sequence of yellow stripes. The third time occurs inside the red area, and the result is displayed in blue. Thus, a solution starting at time 0 always remains in the white area, as shown by the dashed black line. One can see that the white area gradually gets smaller and smaller. Based on the above considerations, we predict that when the current oscillates around threshold, the set of possible spike times eventually gets very narrow, so that spike timing should be reproducible in the long run, even if the potential is not Žxed at reset at stimulus onset. For the general model given by equation 2.1, “the current goes below threshold” means that the vector Želd points downward at threshold, that is, f (Vt , t) < 0. In the construction described above, we implicitly assumed that a solution is at reset potential at time t only when it spikes at time t, which is not necessarily so in the general case of equation 2.1. For example, in Figure 4, one might imagine a solution starting at time 0, going below reset, and then going up into the Žrst red stripe. This cannot happen if we assume f (Vr , t) > 0 for all t. Another sufŽcient condition to avoid this problem is to assume that the model is leaky, as discussed in the appendix. None of these two conditions is fulŽlled by the perfect integrator. Indeed, in this model, when the vector Želd points downward at threshold, it also does so at reset. The numerical simulations show that this model is never reliable. We briey describe the numerical construction of the set of reachable spike times, as illustrated in Figure 4. First, Žnd the Žrst time t1 when the input current goes below threshold. This is the starting point of the Žrst

290

R. Brette and E. Guigon

forbidden interval. Then run the solution starting from threshold at time t1 and stop when it hits threshold again at time t2 . The interval I D [t1 , t2 ] is the Žrst forbidden interval, where no spike can occur. Then run the solution starting from reset at time t1 (resp. t2 ) and stop when it hits threshold at time t3 (resp. t4 ), so as to obtain the next forbidden interval [t3 , t4 ]. By repeating the process, we get a whole sequence of intervals where no spike can occur. In the remaining set of times, the current goes again below threshold. So Žnd the Žrst time when it does so after t2 , and repeat the procedure. Thus, when the algorithm ends, we get the remaining set of reachable spike times, and we shall see that this set eventually gets very small. Since we chose to display the reliability of model responses for all parameters of a given family of currents (see Figure 5A) on a single Žgure, we computed the set of reachable spike times for each parameter (see Figures 5B and 5C) and displayed it in a two-dimensional white and black Žgure, with time on the horizontal axis and parameter on the vertical axis (see Figure 5D). Black dots represent reachable spike times. The explanation above does not apply to the case when the input current is always above threshold. However, in our numerical results, responses seem to be reproducible in both cases, although the required intrinsic noise for reproducibility is signiŽcantly lower than for currents crossing the threshold. Our explanation does not rule out the possibility of multiple stable solutions, but this does not seem to happen in our simulations. 4 Numerical Results 4.1 Periodic Currents. The results regarding periodic stimuli are displayed in Figure 6, which illustrates the theoretical results. A leaky integrator is driven by a 20 Hz sinusoidal current (see Figure 6A), whose mean is a decreasing linear function of the parameter. For parameters p < 0.5, the current is always above threshold, whereas for p > 0.5, the current oscillates around threshold. The spike density Žgure (see Figure 6B) was computed without noise, with random initial potential for every trial, so that phase locking appears clearly. Adding dynamical noise results only in blurring this Žgure; it has no qualitative inuence on the results. The model was run for 5 s. When phase locking occurs, all the stable solutions can be observed in the last 250 ms of the runs (see Figure 6B). Phase locking appears for a given parameter value when the density line for this value consists of a Žnite number of black dots only, that is, when the corresponding PSTH consists of a Žnite number of isolated peaks. Conversely, there is no phase locking when the density line consists of gray shades, that is, when the corresponding PSTH is a smooth distribution. As expected, responses are always phase-locked for p > 0.5, whereas there are not always for p < 0.5. Mean Žring rate and precision were computed for every parameter value with noise-free dynamics, as seen in Figure 6C (solid lines), where phase locking corresponds to plateaus in the Žring rate and to drops in precision

Reliability of Spike Timing

291

Figure 5: Construction of the set of possible spike times for all parameters of a given family. (A) Basis function. (B) For a given parameter, we compute the times when no spike may occur (dark areas are not reachable). (C) This is converted into a line of white and black rasters, with black ones corresponding to possible spike times. (D) Collecting all these lines for all parameters and arranging them vertically, we get a two-dimensional white and black Žgure of possible spike times, with time on the horizontal axis and parameter on the vertical axis.

measure. Precision was also computed with a small noise on the dynamics (1 mV.s ¡1 / 2 ), which results in a smoothed version of the noise-free precision plot, where only 1:1 phase locking appears clearly (see Figure 6C, dashed line). There is no trivial relationship between precision and parameter. The precision is better for p > 0.5, where the model is always phase-locked (the current oscillates around threshold), than for p < 0.5. The best precision is obtained for 1:1 phase locking, as observed in Hunter et al. (1998). We showed the results for three parameter values (see Figures 6D–6F), which illustrate the three possible cases of reliability. The simulations were done with a small dynamical noise. Case 1: Noise accumulates. For p D 0, responses were not reproducible. After 5 s, the PSTH was at, and individual trials were not synchro-

292

R. Brette and E. Guigon

Reliability of Spike Timing

293

nized (see Figure 6D). The model behaves in the same way as with a constant current. Case 2: Multiple stable solutions. For p D 1, the responses are phaselocked, as shown in the spike density Žgure (see Figure 6B). The spike rate is 13.3 Hz (see Figure 6C), which is two-thirds the input frequency. Thus, there is 2:3 phase locking; the model neuron spikes twice every three periods, as shown in the individual trials (see Figure 6F). The density Žgure (see Figure 6B) and the PSTH (see Figure 6F) indicate six spikes every three periods, which means there are three stable solutions, resulting also in a higher sensitivity to noise (see Figure 6C, dashed line). Case 3: One stable solution. For p D 0.5, the spike rate is 20 Hz (see Figure 6C): the model is 1:1 phase-locked. There is only one stable solution, as seen in the individual trials (see Figure 6E). Thus, responses are truly reproducible, and precision is very high (see Figure 6D). In these simulations, case 3 occurs in only about 30% of the parameters range (the 1:1 phase-locking area). 4.2 Aperiodic Currents. 4.2.1 Noise Does Not Accumulate over Time. In Figure 7, we assessed the reliability of a noisy leaky integrator in response to a random triangle wave, whose mean is constant and amplitude varies linearly with the parameter (the current was constant for p D 0). For all trials, membrane potential was at rest at stimulus onset; thus, we do not distinguish between cases 2 (several stable solutions) and 3 (one stable solution). The input current is always above threshold for p < 0.5 and oscillates around threshold for p > 0.5. Figure 6: Facing page. Reliability of a leaky integrator driven by a periodic input current. The model is a linear leaky integrator. (A) Input current is a 20 Hz sine wave I ( t) D 85 C 40(1 ¡ p ) C 30sin(40p t) pA, where p is the parameter. The results are shown from 4.75 s after the onset of the stimulus. (B) The model was run 2000 times for every parameter value (400 values between 0 and 1), with initial potential drawn uniformly between threshold and reset, but no noise on the dynamics, so that phase locking appears clearly. (Adding noise results in a blurred version of this Žgure.) The results are displayed as a spike density Žgure. (C) Firing rate was computed as a function of the parameter, with no noise on the dynamics. Precision was computed with no noise on the dynamics (solid line) and with a noise of 1 mV.s¡1 / 2 (dashed line). Phase locking appears for parameter values below 0.5 as drops in precision. (D–F) For three different parameter values (0, 0.5, 1), we show the results of 10 trials of the model with noise on the dynamics (2.5 mV.s¡1 / 2 ) and random initial potential, and also show the PSTH, computed from 2000 trials. Note the different scales for Žring rate in these graphs.

294

R. Brette and E. Guigon

Trials were 10 s long. The spike density Žgure (see Figure 7B) shows that when the input current oscillated around threshold (p > 0.5), spikes were synchronized in the long run, and the actual spike times corresponded with our computational prediction of possible spike times (see Figure 7C). Tight synchronization can be clearly seen in the 10 individual trials computed for parameter p D 0.8 in Figure 7E, and the PSTH is very peaked. Thus, noise did not accumulate (case 2 or 3). Spiking seems less precise when the input current is always above threshold (p < 0.5), as shown in the spike density Žgure (see Figure 7B) and the precision plot (see Figure 7F, solid line). Individual trials for parameter p D 0.1 (see Figure 7D) show synchronized spikes at the beginning of the simulation, but not in the last 500 ms of the 10 s runs. This is reected by the PSTH as a steady decrease in the height of the peaks during the Žrst 500 ms and as a at distribution during the last 500 ms. However, we also computed the precision of long runs (200 s) with a lower noise on the dynamics (0.7 mV.s ¡1/ 2 ) (see Figure 7F, dashed line). This duration was enough for noise to accumulate in the case of a constant current (p D 0), but it did not lead to a desynchronization of trials for parameters above 0.2. Thus, it seems that in contrast with periodic currents, responses to aperiodic currents are also reproducible (case 2 or 3) when the current is above threshold, as suggested by other studies (Jensen, 1998; Pakdaman & Tanabe, 2001), though the level of tolerable noise is lower than when the current oscillates around threshold. The type of current used (e.g., gaussian instead of triangle wave) did not alter the results. 4.2.2 There Is Only One Stable Solution. In Figure 8, we assessed the reliability of a noisy leaky integrator in response to a random triangle wave, Figure 7: Facing page. Reliability of a leaky integrator with initial potential at rest. The model is a linear leaky integrator. (A) Input current is I ( t) D 150 C 150pB(t) pA, where p is the parameter and B ( t) is a triangular wave taking values between ¡1 and 1, with rising and falling time drawn uniformly between 10 ms and 50 ms. We show the results for the Žrst and last 500 ms of the 10 s simulations. (B) The model was run 2000 times for every parameter value (400 values between 0 and 1), with membrane potential initially at rest value, and noise on the dynamics (3.5 mV.s¡1/ 2 ). The results are displayed as a spike density Žgure. (C) The set of possible spike times was computed for every parameter value. (D, E) For parameter values p D 0.1 (E) and p D 0.8 (F), 10 individual trials were extracted and displayed, along with the PSTH computed from 2000 trials. (F) Mean Žring rate does not vary much with the parameter. The precision was computed as a function of the parameter from the last 7.5 s of 10 s0 runs with a noise of 3.5 mV.s¡1/ 2 on the dynamics (solid line, 400 parameter values), and from the last 10 s of 200 s0 runs with a noise 0.7 mV.s¡1 / 2 (dashed line, 10 parameter values).

Reliability of Spike Timing

295

296

R. Brette and E. Guigon

whose mean is constant and amplitude varies linearly with the parameter. For every trial, membrane potential was drawn at random uniformly between rest and threshold, so that we could distinguish between cases 2 (several stable solutions) and 3 (one stable solution). When the input current oscillates around threshold (p > 0.5), we can see from the spike density Žgure (see Figure 8B) that spike timing becomes precise after some time. In the individual trials for parameter p D 0.95 (see Figure 8E), spikes initially occur at different times, depending on the membrane potential at onset, and eventually they become synchronized over trials, whatever the initial potential, which corresponds to case 3, and was predicted by the computation of possible spike times (see Figure 8C). The results are shown only for the Žrst 500 ms, but the behavior after 9.5 s is similar to the one shown in Figure 7. When the input current is always above threshold (p < 0.5), responses are not reproducible with this magnitude of intrinsic noise, as the results of individual trials for p D 0.1 show (see Figure 8D). However, we found that with a lower intrinsic noise, responses were reproducible in the long run for small parameter values even after 200 s (precision of less than 6 ms for p > 0.2). 4.2.3 Spike Times Are Stable Under Deterministic Perturbations. Do spike times in the long run change when the input current is increased? The spike times of a neuron driven by a constant current are not stable under deterministic perturbations: if the (constant) ISI is T and a slight decrease of the current makes it T C d, then the difference between the times of the nth spike is nd, leading to a desynchronization of the two patterns of spikes after enough time. In Figure 9, we simulated a noisy leaky integrator driven by a random triangle wave, whose mean and amplitude increased linearly with the parameter. The current oscillated around threshold for every parameter value. For every trial, membrane potential was drawn at random uniformly between rest and threshold. The mean Žring rate, computed over 10 s, increased from 22 to 35 Hz with the parameter (see Figure 9D), but spike times did not vary signiŽcantly with the parameter, even after 750 ms (see Figures 9B and 9C). Figure 8: Facing page. Reliability of a leaky integrator with random initial potential. The model is a linear leaky integrator. (A) Input current is I ( t ) D 150 C 150pB(t) pA, where p is the parameter and B ( t) is a normalized Žltered gaussian noise with time constant 10 ms. (B) The model was run 2000 times for every parameter value (400 values between 0 and 1), with initial potential drawn uniformly between rest and threshold, and noise on the dynamics (2.5 mV.s¡1 / 2 ). The results are displayed for the Žrst 500 ms as a spike density Žgure. (C) The set of possible spike times was computed for every parameter value. (D, E) For parameter values p D 0.1 (D) and p D 0.95 (E), 10 individual trials were extracted and displayed, along with the PSTH computed from 2000 trials.

Reliability of Spike Timing

297

298

R. Brette and E. Guigon

Thus, an increase in Žring rate creates additional spikes rather than a shift of all spikes. Precision was almost constant for all parameter values—around 3 ms (see Figure 9D). Thus, when the input current oscillates around threshold, responses are stable under signiŽcantly large deterministic perturbations. 4.2.4 Nonlinear and Nonleaky Models Are Also Reliable. In Figure 10, we assessed the reliability of a noisy leaky nonlinear model driven by equation 2.3 (see Figures 10B and 10C) and a noisy nonleaky model described by equation 2.4 (see Figures 10D and 10E) driven by a gaussian input current whose mean is constant and amplitude varies linearly with the parameter. For every trial, membrane potential was drawn at random uniformly between rest and threshold. Again, responses were reproducible (case 3) when the input current oscillated around threshold (p > 0.5). 4.2.5 The Perfect Integrator Is Never Reliable. As a counterexample, we simulated a noisy perfect integrator (Knight, 1972; see Figure 11), driven by a gaussian current whose mean is constant and amplitude varies linearly with the parameter. For all trials, membrane potential was at rest at stimulus onset. The hypotheses required for our theoretical explanation are not fulŽlled by this model. We can see in Figure 11B that whether the input current is above threshold (p < 0.5) or oscillates around it (p > 0.5), the noise accumulates and leads to a complete desynchronization of spikes over trials (case 1). Figure 11C shows that the set of possible spike times does not get smaller over time, and Figure 11D that the precision is always above 13 ms, which is half the average ISI. 5 Discussion We identiŽed two ways in which neuron responses can fail to be reproducible: dynamical noise can accumulate over time and lead to a desynchronization over trials, or several stable responses can exist, depending on the initial condition. These two cases can occur when the model neuron is driven by a periodic current, but the former can occur only if the current is above threshold (Keener et al., 1981). In contrast, we showed that responses of a general class of spiking neuron models to aperiodic currents oscillating around threshold are reproducible: spike timing in the long run is stable under noise and deterministic perturbation and does not depend on the initial condition. We provided a theoretical explanation based on a geometrical construction that shows that the set of possible spike times becomes smaller and smaller over time. Our numerical results suggest that responses to aperiodic currents above threshold are also reproducible, which conŽrms other studies (Pakdaman & Tanabe, 2001), but the level of tolerable noise seems lower in this case.

Reliability of Spike Timing

299

Figure 9: Stability of spike timing under deterministic perturbations. The model is a linear leaky integrator. (A) Input current is I ( t) D 112.5 C (75 C 37.5p) B ( t ) C 37.5p pA, where p is the parameter and B ( t ) is a triangular wave taking values between ¡1 and 1, with rising and falling time drawn uniformly between 10 ms and 50 ms. Thus, the mean input current increases with the parameter from 112.5 pA to 150 pA. (B) The model was run 2000 times for every parameter value (400 values between 0 and 1), with initial potential drawn uniformly between rest and threshold and noise on the dynamics (2.5 mV.s¡1 / 2 ). The results are displayed for the Žrst 750 ms as a spike density Žgure. (C) The set of possible spike times was computed for every parameter value. (D) Mean Žring rate and precision were computed from 10 s simulations.

300

R. Brette and E. Guigon

Figure 10: Reliability of a nonlinear and a nonleaky model. We assessed the reliability of two models that differ from the linear leaky integrator: a leaky nonlinear model driven by equation 2.3 (B, C) and a nonleaky model driven by equation 2.4 (D, E). (A) Input current is I ( t) D 150 C 150pB(t) pA for the leaky nonlinear model and I ( t) D 0.5 C 3pB(t) for the nonleaky model (unit is irrelevant), where p is the parameter and B ( t) is a normalized Žltered gaussian noise with time constant 10 ms. (B) The nonlinear leaky model was run 2000 times for every parameter value (400 values between 0 and 1), with initial potential drawn uniformly between rest and threshold, and noise on the dynamics (2.5 mV.s¡1/ 2 for the leaky model). The results are displayed for the Žrst 500 ms as a spike density Žgure. (C) The set of possible spike times was computed for every parameter value. (D, E) The same simulations were done for the nonleaky model, with noise on the dynamics 0.1 V.s¡1/ 2 .

Reliability of Spike Timing

301

Figure 11: The perfect integrator is not reliable. The model is a perfect integrator driven by equation 2.5. (A) Input current is I ( t) D .5 C pB ( t) (units are not signiŽcant here), where p is the parameter and B (t ) is a a normalized Žltered gaussian noise with time constant 10 ms. We show the results for the Žrst and last 500 ms of the 10 s simulations. (B) The model was run 2000 times for every parameter value (400values between 0 and 1), with membrane potential initially at rest value and noise on the dynamics (0.27 V.s¡1 / 2 ). The results displayed are displayed as a spike density Žgure. (C) The set of possible spike times was computed for every parameter value. (D) Mean Žring rate is almost constant, around 40 Hz, while precision is always above 13 ms, which is very high, knowing that this is about half the mean ISI.

302

R. Brette and E. Guigon

We showed that for aperiodic currents that cross the threshold, noise does not accumulate over time. In other words, in this case, the asymptotic spike time jitter is a function of the level of intrinsic noise that tends to 0 as the noise variance tends to 0, which is not generally so for periodic currents. This result is not a consequence of particular factors such as the leak, the amount of uctuations in the input current, or its frequency spectrum. However, quantitatively, the ratio of the asymptotic spike time jitter to the noise variance depends on various characteristics of the input current (and, more generally, of the differential equation; Hunter et al., 1998; Fellous et al., 2001; Beierholm et al., 2001; Tiesinga, 2002; Tiesinga et al., 2002b). Such a relationship is relevant only when responses are reproducible in the way deŽned above. In particular, it would be illusory to search for a formula describing the precision as a function of a parameter of the input in the case of periodic currents above threshold. Indeed, when varying the parameter, the model alternates between phase locking, where precision is close to 0, and structural unstability, where spike timing is imprecise even with an arbitrarily small intrinsic noise (besides being dependent on the initial condition). However, when the model is phase-locked, the (quantitative) precision depends on dynamical properties such as the Lyapunov exponent and the distance to the boundary of the Arnold tongue (Tiesinga, 2002). Appendix: Stability of the Set of Possible Spike Times A leaky model is described by a one-dimensional differential equation, dV D f ( V, t) dt

(A.1)

where V is the membrane potential, with the spike modeled as follows: when the potential reaches a threshold Vt , it is reset to Vr . The leak is described by the following condition on f : @f @V

< 0.

This inequality implies that the distance between any two solutions of equation A.1 (without spike) always decreases and tends to zero—hence, the term leak. Since we are interested in the sequence of spike times, it is natural to introduce the map Q: R ! R that gives the time of the Žrst spike following a given spike time, that is, Q ( t) is the time when a run starting from reset potential at time t hits threshold. We shall call this map the spike map. Thus, the sequence of spike times of a run with Žrst spike at time t is just t, Q (t) , Q2 (t) , . . . The spike map was Žrst introduced in Rescigno et al. (1970) in the case when the input current is periodic and studied further in Knight (1972) and Keener et al. (1981).

Reliability of Spike Timing

303

One can easily show that for every t, f (Vt , Q ( t) ) ¸ 0, which means there can be no spike if the vector Želd points downward at threshold, that is, in the case of the linear leaky integrator, if the input current is below the current threshold. This result is the Žrst cornerstone of the construction of the set of possible spike times, and it holds also for nonleaky spiking models, including the perfect integrator. It allows us to remove intervals from the set of possible spike times, as described in section 3.2. In the construction of this set, we also remove the image of these intervals by iterates of Q. However, this is correct only if Q is injective. For leaky models, one can show that Q is strictly increasing on its range (see Keener et al., 1981, for the case of periodic currents). This is also true if f ( Vr , t) > 0 for all t, which is the case of the nonleaky models we simulated. For periodic currents, that is, in the general case, when f is periodic with respect to t, the fact that Q is strictly increasing on its range means it is a lift of an orientation-preserving circle map. If Q is surjective, that is, if the current is always above threshold, then Q is a lift of a homeomorphism of the circle. The dynamics of both noninvertible orientation-preserving circle maps and homeomorphisms of the circle are well known, as we described in section 3.1. Our aim is to prove that the set of possible spike times constructed in section 3.2 is stable under noise and deterministic perturbations. We assume that the duration of ISIs is bounded, so that there is an M > 0 such that Q ( t) ¡ t < M for all t. For mathematical convenience, we consider here that a run may hit threshold without spiking, as long as it does not cross it. Consider runs deŽned on R (not only R C ), with inŽnitely many spikes on both R C and R ¡ . Note R ½ R the set of spike times of all these runs, that is, the set of possible spike times constructed from t D ¡1. Now consider a perturbation of this system deŽned in the following way: take 2 > 0, representing the amount of perturbation; the spike following a spike at time tn occurs at time tn C 1 such that |tn C 1 ¡ Q ( tn ) | · 2 . This accounts for an uncertainty that may be random or deterministic. Note R2 the set of spike times of all possible perturbed runs for a given 2 > 0. We shall prove the following stability theorem: Theorem 1. For any compact set K such that K \ R D ;, there is an 2 > 0 such that K \ R2 D ;. In other words, any compact set, such as an interval, that is not reachable under the original deterministic dynamical system is not reachable either if a small perturbation is added on the dynamics. Therefore, adding a small noise to the dynamics should have a small inuence on the set of spike times. In the same way, a slight change of the vector Želd should cause a small perturbation of spike times.

304

R. Brette and E. Guigon

First, let us formalize the considerations above. Considering that a run may hit threshold without spiking, as long as it does not cross it, means in the present case that if the spike map is discontinuous at t, then the spike following a spike at time t may occur at either time Q ( t¡ ) or Q ( t C ). Now choose 2 > 0 as the level of uncertainty; if there is a spike at time t, then the next spike will occur within distance 2 of either Q ( t¡ ) or Q (t C ) , that is, the set of possible times for next spike is [Q ( t¡ ) ¡ 2 , Q (t¡ ) C 2 ] [ [Q ( t C ) ¡ 2 , Q ( t C ) C 2 ]. Formally, this framework may be expressed in terms of a setvalued dynamical system. We refer the reader to Aubin and Frankowska (1990) for more details about set-valued analysis. A sequence ( tn ) of spike times agreeing with our considerations is one that satisŽes tn C 1 2 F2 ( tn ) ,

(A.2)

where F2 ( t) D [Q (t¡ ) ¡ 2 , Q ( t¡ ) C 2 ] [ [Q ( t C ) ¡ 2 , Q ( t C ) C 2 ]. Thus, F2 is a set-valued map that we write F2 : R ; R, as in Aubin and Frankowska (1990). We introduce a few deŽnitions from set-valued analysis: DeŽnition 1. • The graph of a set-valued map F: X ; Y is the set of points ( x, y ) 2 X £ Y such that y 2 F ( x) .

• The inverse of a set-valued map F: X ; Y is the set-valued map F ¡1 : Y ; X deŽned by x 2 F ¡1 ( y ) if and only if y 2 F ( x) . • If A ½ R, then [ F (A) D F ( t). t2A

• If F: R ; R and G: R ; R are two set-valued maps, then the composition G ± F: R ; R is the set-valued map deŽned by G ± F ( t) D G ( F (t ) ) . • The notation Fn stands for the n-fold composition F ± F ± ¢ ¢ ¢ ± F.

• The intersection of F: R ; R and G: R ; R is the set-valued map F \ G: R ; R deŽned by F \ G ( t) D F ( t) \ G ( t) . Thus, F2 n ( t) is the set of possible times for the nth spike following a spike at time t, given an uncertainty 2 . The set of possible times for the nth spike of any run is F2 n ( R ). Consider a run deŽned on R, with inŽnitely many spikes on R ¡ . Every spike of this run must occur within the set F2 n (R ) , for any integer n. Thus, the spike times are contained in the set \ R (F2 ) D F2 n ( R ) . n> 0

Conversely, for any time t 2 R ( F2 ) , we can Žnd a sequence of spike times satisfying equation A.2. Thus, R (F2 ) is the set of spike times of all such runs.

Reliability of Spike Timing

305

In order to prove theorem 1, we need another deŽnition: DeŽnition 2. Let F: X ; Y be a set-valued map. F is said to be compact on compact sets (CC) if for any sequence xn 2 X converging to x, any sequence yn 2 F ( xn ) has a cluster point y 2 F (x ) . Equivalently, F is CC if its graph is closed and it maps bounded sets to bounded sets. This is a generalization of continuity for set-valued maps. One may note that when F is a single-valued map, it is equivalent to continuity. Let us state a few basic results regarding CC maps: Proposition 1. 1. If F: X ; Y and G: Y ; Z is CC, then so is G ± F.

2. If Fl : X ; Y, l 2 I, is a family of CC set-valued maps, then the intersection \ Fl l2I

is also CC. 3. Let I £X ; Y

(l, x ) 7! Ft ( x )

a CC set-valued map, and let K ½ X be a compact set. Then l 7! Fl ( K) is CC. Theorem 1 follows from the following lemma: Lemma 1.

The set-valued map 2 2 R C 7! F2 n (R ) \ K is CC.

Proof of Theorem 1. It follows from lemma 1 and proposition 1 that the set-valued map 2 7! R ( F2 ) \ K is CC; hence, its domain is closed, that is, the 6 ; is closed. Therefore, if R ( F 0 ) \ K D ;, then set of 2 such that R ( F2 ) \ K D for 2 small enough, R (F2 ) \ K D ;. Proof of Lemma 1. Note that F2 n ( R ) \ K D F2 n ( F2 ¡n ( K ) ). It follows from proposition 1 that (2 , t) 7! F2 n ( t) is CC. Therefore, it is sufŽcient to prove that (2 , t) 7! F2 ¡n ( t) is CC, and since its graph is closed, we need to prove only that it maps bounded sets to bounded sets. That is, we must prove that

306

R. Brette and E. Guigon

for any A > 0 and any compact interval I, the set [ 2 ·A

¡n (I) F2 ¡n (I ) D FA

is bounded. Since the function t 7! Q ( t) ¡ t is bounded, we have lim Qn ( t) D ¡1.

t!¡1

Therefore, Qn maps an unbounded set of R ¡ to an unbounded set. Hence, ¡n (I ) must be bounded. FA Acknowledgments We thank K. Pakdaman and J.-P. Aubin for many helpful discussions and Christine Lienhart and Marc Maier for revising our English. References Arnold, V. I. (1961). Small denominators. I. Mapping the circle onto itself. Izv. Akad. Nauk. SSSR Ser. Mat., 25, 21–86. Ascoli, C., Barbi, M., Chillemi, S., & Petracchi, D. (1977). Phase-locked responses in the Limulus lateral eye: Theoretical and experimental investigation. Biophys. J., 19(3), 219–240. Aubin, J. P., & Frankowska, H. (1990). Set-valued analysis. Boston: Birkh¨auser. Baddeley, R., Abbott, L., Booth, M., Sengpiel, F., Freeman, T., Wakeman, E., & Rolls, E. (1997).Responses of neurons in primary and inferior temporal visual cortices to natural scenes. Proc. R. Soc. Lond. B Biol. Sci., 264(1389), 1775– 1783. Bair, W., & Koch, C. (1996). Temporal precision of spike trains in extrastriate cortex of the behaving macaque monkey. Neural Comp., 8(6), 1185–1202. Beierholm, U., Nielsen, C., Ryge, J., Alstrom, P., & Kiehn, O. (2001). Characterization of reliability of spike timing in spinal interneurons during oscillating inputs. J. Neurophysiol., 86(4), 1858–1868. Berry, M., & Meister, M. (1998). Refractoriness and neural precision. J. Neurosci., 18(6), 2200–2211. Berry, M., Warland, D., & Meister, M. (1997). The structure and precision of retinal spike trains. Proc. Natl. Acad. Sci. USA, 94(10), 5411–5416. Borst, A., & Theunissen, F. (1999). Information theory and neural coding. Nat. Neurosci., 2, 947–957. Buracas, G., Zador, A., DeWeese, M., & Albright, T. (1998). EfŽcient discrimination of temporal patterns by motion-sensitive neurons in primate visual cortex. Neuron, 20(5), 959–969. Coddington, E., & Levinson, N. (1955). Theory of ordinary differential equations. New York: McGraw-Hill.

Reliability of Spike Timing

307

de Ruyter van Steveninck, R., Lewen, G., Strong, S., Koberle, R., & Bialek, W. (1997). Reproducibility and variability in neural spike trains. Science, 275, 1805–1808. Denjoy, A. (1932). Sur les courbes dÂeŽnies par les e quations diffÂerentielles a` la surface du tore. J. Math. Pures. Appl., 9(11), 333–375. Fellous, J. M., Houweling, A., Modi, R., Pao, R., Tiesinga, P., & Sejnowski, T. (2001).Frequency dependence of spike timing reliability in cortical pyramidal cells and interneurons. J. Neurophysiol., 85, 1782–1787. Guttman, R., Feldman, L., & Jakobsson, E. (1980). Frequency entrainment of squid axon membrane. J. Membrane Biol., 56, 9–18. Herman, M. (1977). Mesure de Lebesgue et nombre de rotation. Berlin: SpringerVerlag. Hunter, J., Milton, J., Thomas, P., & Cowan, J. (1998). Resonance effect for neural spike time reliability. J. Neurophysiol., 80(3), 1427–1438. Jensen, R. (1998). Synchronization of randomly driven nonlinear oscillators. Phys. Rev. E, 58(6), R6907–R6910. Kara, P., Reinagel, P., & Reid, R. (2000). Low response variability in simultaneously recorded retinal, thalamic, and cortical neurons. Neuron, 27(3), 635–646. Keener, J. (1980). Chaotic behavior in piecewise continuous difference equations. Trans. Amer. Math. Soc., 261(2), 589–604. Keener, J., Hoppensteadt, F., & Rinzel, J. (1981). Integrate-and-Žre models of nerve membrane response to oscillatory input. SIAM J. Appl. Math., 41, 503– 517. Knight, B. (1972). Dynamics of encoding in a population of neurons. J. Gen. Physiol., 59, 734–766. Koppl, C. (1997). Phase locking to high frequencies in the auditory nerve and cochlear nucleus magnocellularis of the barn owl, Tyto alba. J. Neurosci., 17(9), 3312–3321. Lapicque, L. (1907).Recherches quantitatives sur l’excitation e lectrique des nerfs traitÂee comme une polarisation. J. Physiol. Pathol. Gen., 9, 620–635. Mainen, Z., & Sejnowski, T. (1995). Reliability of spike timing in neocortical neurons. Science, 268, 1503–1506. Nowak, L., Sanchez-Vives, M., & McCormick, D. (1997). Inuence of low and high frequency inputs on spike timing in visual cortical neurons. Cereb. Cortex, 7(6), 487–501. Pakdaman, K., & Tanabe, S. (2001). Random dynamics of the Hodgkin-Huxley neuron model. Phys. Rev. E, 64(5Pt1), 050902. Reich, D., Mechler, F., Purpura, K., & Victor, J. (2000). Interspike intervals, receptive Želds, and information encoding in primary visual cortex. J. Neurosci., 20(5), 1964–1974. Reich, D., Victor, J., & Knight, B. (1998). The power ratio and the interval map: Spiking models and extracellular recordings. J. Neurosci., 18, 10090–10104. Reich, D., Victor, J., Knight, B., Ozaki, T., & Kaplan, E. (1997).Response variability and timing precision of neuronal spike trains in vivo. J. Neurophysiol., 77(5), 2836–2841. Reinagel, P., & Reid, R. (2000). Temporal coding of visual information in the thalamus. J. Neurosci., 20(14), 5392–5400.

308

R. Brette and E. Guigon

Rescigno, A., Stein, R., Purple, R., & Poppele, R. (1970). A neuronal model for the discharge patterns produced by cyclic inputs. Bull. Math. Biophys., 32, 337–353. Tiesinga, P. (2002). Precision and reliability of periodically and quasiperiodically driven integrate-and-Žre neurons. Phys. Rev. E, 65, 041913. Tiesinga, P., Fellous, J. M., & Sejnowski, T. (2002a). Attractor reliability reveals deterministic structure in neuronal spike trains. Neural Comput., 14(7), 1629– 1650. Tiesinga, P., Fellous, J. M., & Sejnowski, T. (2002b). Spike-time reliability of periodically driven integrate-and-Žre neurons. Neurocomputing, 44–46: 195–200. Veerman, J. (1989). Irrational rotation numbers. Nonlinearity, 2(3), 419–428. Victor, J., & Purpura, K. (1996). Nature and precision of temporal coding in visual cortex: A metric-space analysis. J. Neurophysiol., 76(2), 1310–1326.

Received March 15, 2002; accepted July 5, 2002.

LETTER

Communicated by Terrence Sejnowski

The Dynamic Neural Filter: A Binary Model of Spatiotemporal Coding Brigitte Quenet [email protected] Laboratoire d’Electronique, Ecole Superieure de Physique et Chimie Industrielles, Paris 75005, France David Horn [email protected] School of Physics and Astronomy, Raymond and Beverly Sackler Faculty of Exact Sciences, Tel Aviv University, Tel Aviv 69978, Israel We describe and discuss the properties of a binary neural network that can serve as a dynamic neural Žlter (DNF), which maps regions of input space into spatiotemporal sequences of neuronal activity. Both deterministic and stochastic dynamics are studied, allowing the investigation of the stability of spatiotemporal sequences under noisy conditions. We deŽne a measure of the coding capacity of a DNF and develop an algorithm for constructing a DNF that can serve as a source of given codes. On the basis of this algorithm, we suggest using a minimal DNF capable of generating observed sequences as a measure of complexity of spatiotemporal data. This measure is applied to experimental observations in the locust olfactory system, whose reverberating local Želd potential provides a natural temporal scale allowing the use of a binary DNF. For random synaptic matrices, a DNF can generate very large cycles, thus becoming an efŽcient tool for producing spatiotemporal codes. The latter can be stabilized by applying to the parameters of the DNF a learning algorithm with suitable margins. 1 Introduction

Three types of neural network paradigms (Hertz, Krogh, & Palmer, 1991; Peretto, 1992) were developed two decades ago. Two of them were based on supervised learning, in order to construct feedback attractor networks (HopŽeld, 1982) for associative memory and feedforward multilayer perceptrons (Rumelhart, Hinton, & Williams, 1986) for functional representation. The third used unsupervised learning to produce self-organized maps (Kohonen, 1982). These three conventional paradigms of neural computation were formulated in terms of mathematical neurons that are very different from biological reality, and it is still unclear how, if at all, biological neural Neural Computation 15, 309–329 (2003)

° c 2002 Massachusetts Institute of Technology

310

B. Quenet and D. Horn

networks perform any of them. Over the past decade, interest in theoretical neuroscience (Dayan & Abbott, 2001) has shifted to analyzing neural networks of spiking neurons with dynamical synapses in the hope of getting closer to the natural roles and purposes of the relevant biological networks. We will argue that it is nevertheless timely to consider yet another model of binary neurons with discrete temporal dynamics and show its applicability to a biological system. An interesting issue in neuroscience is the question of spatiotemporal coding. Its existence has been demonstrated (Wehr & Laurent, 1996) in the locust olfactory system, where the spatiotemporal behavior of projection neurons encodes the odor presented to its receptor neurons. This transformation from odor input to spatiotemporal activity occurs in the antennal lobe, that is, the Žrst module of the olfactory system. This system may therefore be regarded as a dynamic neural Žlter that turns spatial information distributed over its many glomeruli, fed by the receptor neurons, to speciŽc spatiotemporal outputs. Although this is a complicated biological system, it has an important simplifying feature that allows it to be represented by a simpliŽed model of mathematical neurons: the fact that the activity of the projection neurons is limited to temporal bins deŽned by an oscillatory local Želd potential. Hence, a model with binary neurons obeying HopŽeld-Little dynamics (Peretto, 1992) can provide a valid Žrst-order approximation of the spatiotemporal behavior of that system. We study a recurrent model of this kind using an asymmetric coupling matrix that allows for the generation of large temporal sequences. The novelty of our approach is that we use this model as a dynamic neural Žlter (DNF), relating input space to spatiotemporal behavior of the recurrent network. As such, it does not correspond to any of the three paradigms noted above, it does not come to rest at Žxed points, and it is not necessarily based on supervised learning. We have presented this model in a previous work (Quenet, Horn, Dreyfus, & Dubois, 2001) and demonstrated how it can be used to generate the spatiotemporal behavior of projection neurons observed by Wehr and Laurent (1996). Here we discuss several fundamental issues concerning this model, such as its stability and coding capacity. Moreover, we provide an algorithm for constructing a DNF that can generate a given set of spatiotemporal data. This algorithm is used to deŽne and characterize the complexity of the data set of Wehr and Laurent (1996). It should be emphasized that the latter is used only as a relevant example, and we do not claim that the DNF is a realistic representation of the biological reality. Nonetheless, as in the conventional three paradigms, it can be used as a schematic model that will subserve further detailed modeling of biological systems. 2 The Model 2.1 Short Biological Motivation. The antennal lobe of insects serves as the Žrst stage of olfactory computation, using as input signals of chemical

The Dynamic Neural Filter

311

receptors and transforming them into outputs of projection neurons (PN) to the next stage of the olfactory tract (the mushroom body). In the locust, there are about 800 excitatory PN cells (Laurent and Naraghi, 1994) and a slightly smaller number of inhibitory local interneurons (LN) that form dendrodendritic interactions with the PNs and with themselves, and receive receptor information as well. The output of this system is that of PNs only; hence, we will consider our model as a rough sketch of PNs, with interactions that should be regarded as effective interactions mediated by the LNs. The antennal lobe exhibits a reverberating Želd potential (LFP) at a frequency of about 20 Hz. This LFP reverberates for a few cycles (up to tens of them) and then quiets down until it begins to repeat such behavior. (See Laurent et al., 2001, for details.) The PNs Žre within the up-phase of the LFP; hence the latter may be regarded as deŽning an effective clock, with time bins of, for example, 30 ms of activity and 10 ms of rest. This motivates us to use a simpliŽed model with discrete temporal dynamics. 2.2 The Recurrent Network. In our model, a binary neuron i, representing a PN, may either Žre, ni D 1, or be quiescent, ni D 0, in a given temporal bin of the LFP. There are N neurons in the model obeying the following HopŽeld-Little dynamics,

0 ni (t C 1) D H (hi ( t C 1) ) D H @

1 X j

wij nj ( t) C Ri ¡ hi A ,

(2.1)

where wij is the synaptic coupling matrix, Ri is the external constant input (specifying odor activation), and hi is the threshold. H is the Heaviside step function taking the values 0 for nonpositive arguments and 1 for positive ones. This model can be readily generalized to account for the presence of noise by replacing the deterministic rule by the stochastic one, Prob[ni ( t C 1) D 1] D

1 . 1 C e¡hi ( t C 1) /2

(2.2)

where 2 is the noise parameter. We will study the stability of results of the deterministic dynamics to the noise introduced by the stochastic dynamics. 2.3 Deterministic Dynamics. In order to analyze the one-step dynamics of equation 2.1, we deŽne a Lyapunov function L that describes just this onestep, in the sense that, considering all dynamical states at time t C 1, it will be minimal for the state that is a solution to these dynamics. Let us deŽne the initial and Žnal states,

nIi D ni (t )

nFi D ni ( t C 1) ,

(2.3)

312

B. Quenet and D. Horn

relevant to the one-step of equation 2.1. Clearly, I and F are just two of the 2N states that the system of neuron possesses. We can prove that L JI D ¡

X ij

J wij ni njI ¡

X i

J ni ( Ri ¡ hi )

(2.4)

obtains its unique minimum on the state J D F. Consider 0 1 X JI J J ki D ¡(2ni ¡ 1) hIi D ¡(2ni ¡ 1) @ wij njI C Ri ¡ hi A .

(2.5)

j

For J D F, this quantity is negative for every i, following the deterministic dynamics. For any other J, there will be elements i for which the sign will P JI JI ip. Hence, i ki will be minimal for J D F. We note in equation 2.5 that ki J JI contains terms that are dependent on ni , to be denoted by 2li where JI li

D

¡niJ

0 1 X @ wij njI C Ri ¡ hi A ,

(2.6)

j

J

and terms that are independent of ni . Since the latter are common for all J, P JI the minimum of i ki will be reached for the same J as the minimum of P JI L JI D i li , the Lyapunov function of equation 2.4. 2.4 Stochastic Dynamics. The one-step Lyapunov function, equation 2.4, plays an important role in the stochastic dynamics.1 As we will see, the probability of obtaining a state J after starting from a state I can be written in terms of this function:

e¡L /2 P ( J | I ) D P ¡LKI 2 . / Ke JI

(2.7)

To prove this result, we start from the probability of obtaining the state F, that is, the one following from the deterministic dynamics. It is straightforward to show that P (nFi | I ) D

1 FI 1 C eki /2

I

(2.8)

1 It was applied to a model with symmetric synaptic couplings by Quenet, Cerny, Dreyfus, and Lutz (1997).

The Dynamic Neural Filter

313

hence, P (F | I ) D P i P ( nFi | I ) .

(2.9)

Any other state J differs from the state F by some ips of neuronal states, J for example, nm D 1 ¡ nFm . In such a case, P ( J | I ) will be the probability of obtaining the corrupted state F, having the wrong digit in its mth place, FI which is P ( F | I ) e km /2 . But at this location m, kFI m D

1 FI JI JI ( k ¡ km ) D lFI m ¡ lm . 2 m

(2.10)

JI Since for all other neurons lFI i D li , it follows that FI

P ( J | I ) D P (F | I ) e ( L

¡L JI ) /2

.

(2.11)

This will be true for any state regardless of how many ips occur because of their independent nature. Hence, equation 2.7 is proved. The stochastic dynamics of equations 2.1 and 2.2 lead to a homogeneous and irreducible Markov process; the transition matrix is time invariant, and all elements of the transition matrix, T JI D P ( J | I ) ,

(2.12)

are strictly positive. Such a Markov process has a stationary probability distribution, p ( I ), that is, an eigenvector of the transition matrix with the highest eigenvalue, 1: X (2.13) T JI p ( I ) D p ( J ) . I

From this stationary probability distribution, obtained asymptotically in the evolution of the system, one can calculate the asymptotic probabilities of the transitions p ( I ! J ) : p (I ! J ) D P ( J | I ) p (I ) .

(2.14)

The stationary Markov process has an entropy rate (Cover & Thomas, 1991), X HD¡ (2.15) p (I ) T JI log2 T JI , IJ

that serves as a lower bound on the expected dimension of any binary code of the Markov process. Since in our case all states are described by vectors of length N, it follows that

H · N.

(2.16)

The upper limit is approached when the noise parameter 2 is high and all transition matrix elements tend to be equal.

314

B. Quenet and D. Horn

3 Numerical Examples

We propose to view the model described in the previous section as a Žlterrelating input space Ri to spatiotemporal sequences of the recurrent network. In this section, we will demonstrate numerically how the DNF acts. For simplicity, we choose both wij and Ri to have positive and negative integer values, while Žxing hi D 12 . This means that Ri will now effectively represent both input and threshold of neuron i. 3.1 Mapping of Input Space. Once the matrix w is given, there is a

restricted range of interest for R: ¡

X j

wij H (wij ) · Ri · ¡

X j

wij H (¡wij ) C 1.

(3.1)

Outside this range the dynamics becomes trivial because the input determines the neuronal values directly. As a Žrst example of deterministic dynamics, we study an N D 2 system because it can be easily mapped in an exhaustive manner. We choose the synaptic matrix ³ ´ 1 2 wD ¡2 ¡1 and use an initial null state ni (0) D 0. The dynamics generate 14 different sequences, including the four (2N ) Žxed points, several two-cycles, and one four-cycle. In Figure 1, we display the coding zones, that is, the ranges of R corresponding to a single temporal sequence. The overall range of R space is chosen to be somewhat larger than the range of equation 3.1. In its center, we Žnd that many different sequences are produced (small coding zones), while in the periphery, a single sequence dominates. Figure 2 shows the length of each sequence on the same R plane. We deŽne it as the length of the sequence the model generates before one of its states is being repeated. This deŽnition of length takes into account both the order of the cycle and the transition time it takes to reach it. The interesting (i.e., long) sequences appear near the center of R space. The periphery is dominated by Žxed points and two-cycles. Building on this experience, we next study an N D 5 system with the synaptic matrix 0 1 0 ¡2 ¡5 ¡3 0 B 6 2 8 ¡14 0C B C 1 1 0 ¡2 1C wDB B C. @¡4 6 1 1 3A 4 ¡1 2 ¡4 0

Label of the spatio-temporal sequence

The Dynamic Neural Filter

315

14 12 10 8 6 4 2 0 10 5 Input R2

0 -5

-6

-4

-2

0

2

4

6-

Input R1

Figure 1: Ranges of input space in a two-neuron problem that lead to identical spatiotemporal behavior, deŽned as coding zones. The label of each code, or spatiotemporal behavior, varies from 1 to 14 in this problem.

Now much larger sequences can be generated, and we would like to see the ensuing mapping of input space and test the stability of the sequences to noise. In order to be able to view the results, we choose R3 R4 R5 to lie in the center of their expected range, as speciŽed by equation 3.1, and study the system in the R1 R2 plane, limited to their relevant ranges. First, let us view, in Figure 3, the coding zones of different sequences in R space. Their average size is 10; that is, there is an order of magnitude of information compression from R space to the spatiotemporal sequences. The lengths of the latter are displayed in Figure 4. They include many cycles of lengths 4, 5, and 6. The coding zones are the analogs of basins of attraction, which signify domains of initial conditions that lead to the same attractor. Here, we stay with a Žxed initial condition (the null state) and search for domains in input space that lead to the same temporal sequence (transition into a cycle). Note the large number of different sequences obtained in this problem. It is exponentially larger than the number of Žxed points that we are accustomed to in attractor neural networks (describable by the same dynamics with symmetric synaptic matrices). We estimate, on the basis of simulations, that the total number of sequences in the Žve-dimensional R space is of the order of 3800. In the two-dimensional section of R space shown in Figure 3, we Žnd 38 different sequences, whose lengths are displayed in Figure 4.

316

B. Quenet and D. Horn

Sequence length

4

3

2

1 10 5 R2

0 -5

-8

-6

0

-2

-4

2

4

6

R1

Figure 2: Sequence length of the N D 2 problem mapped on the two-dimensional input space. Table 1: Six Neighboring Sequences in a System with Five Neurons, Displayed over Seven Time Steps. R2

DE

DH7

tD1

2

3

4

5

6

7

¡15 ¡12 ¡8 ¡3 2 8

0 2 4 5 6 7

0 1 2 10 11 7

17 17 17 17 25 25

22 22 22 30 30 30

6 14 14 16 16 16

8 8 16 3 3 11

3 3 3 17 17 3

17 17 17 30 30 17

22 22 22 16 16 30

3.2 Distances Between Codes. The mapping of R space into spatiotemporal codes can also serve as deŽning a distance between different codes. It is thus interesting to compare different sequences generated over a neighborhood of R space, as seen in Table 1. All of these sequences can be read off as we move along the R2 axis at Žxed R1 D 4. The sequences are ordered according to increasing value of R2 , where we indicate the central value of the coding zone, and the states for each time bin are written in the biP nary representation 1 C 5iD1 ni 25¡i . The columns DE and DH7 describe two distances, deŽned below, from the Žrst sequence.

Label of the spatio-temporal sequence

The Dynamic Neural Filter

317

40 35 30 25 20 15 10 5 0 20 10 R2

0 -10 -20

0

2

4

6 R1

8

10

12

Figure 3: Coding zones of spatiotemporal sequences of an N D 5 problem, plotted on a plane deŽned by two of the input variables. Other inputs are held constant at their central values.

As one moves along from the Žrst sequence to the second, and then to the third, only one state changes over the observed range of T D 7. Moreover, this change corresponds to the ipping of just one neuron (n2 at t D 3 between the Žrst and second sequence). Proximity in R may therefore be related to proximity in the identity of states appearing in two sequences, or proximity in the sense of Hamming distance between all neurons at all time steps. Thus, we deŽne three distances between sequences: DE is the “edit distance,” deŽned as the number of insertions and deletions needed to change one sequence into another, limiting oneself to the natural length of each sequence, that is, until one of the states reappears. DH is the Hamming difference between the two spatiotemporal patterns of neural activities. Clearly this grows with the length of the sequence. In the example of Table 1, where we limit ourselves to seven time steps, we list the corresponding seven-step Hamming distance, DH7 . Finally, we may deŽne DR , the Euclidean distance between the centers of the coding zones in R space. Table 1 lists the values of DE and DH7 distances from the Žrst sequence in this table. Using 18 sequences of this problem, including the six shown in Table 1, we Žnd a high correlation (0.87) between DE and DR and a lower cor-

318

B. Quenet and D. Horn

Sequence length

7 6 5 4 3 2 1

20 10

15 R2

0

10 -10

5 -20

0

R1

Figure 4: Sequence length of the 38 sequences mapped out in Figure 3.

relation (0.56) between DH7 and DR . The latter signiŽes the fact that whereas the Hamming distance correlates well with R distance for some sequences, there are cases where several neurons ip as one moves from one code to the next. In Table 1, this occurs between the third and the fourth sequences and between the Žfth and the sixth ones. Note, however, that in the fourth sequence, the state 30 replaces two states, 22 and 14, of the third sequence. Therefore, DE between the two will be only 3. This explains in what sense the two sequences should be regarded as close to one another, as also borne out by the short distance in R. These results can also be studied from the viewpoint of the individual neurons. Since the Žrst line has the lowest value of R2 , none of its states involves an active neuron number 2, that is, all have n2 D 0. Moving up to R2 D ¡12, n2 is activated in state 14. At the next stage, it is activated in state 16. Further increases of R2 lead to activation of neuron number 2 in the states 30 and 25. Thus, we observe a gradual increase in the number of states that contain an active neuron number 2. Since the volume of state-space is relatively moderate, 32, we Žnd that in spite of the change in the identity of the state, for example, a change of 6 to 14 in step t D 3 between the Žrst and the second row, the next state remains 8, and the rest of the sequence is reiterated. This leads to the good correlation with DE . We believe that this

The Dynamic Neural Filter

319

property may be useful in DNF applications to the study of problems such as gene sequences, where low N models may apply. The correlation between R distance and edit distance may be lost at large N. The reason is that the volume of state-space grows like 2N . Hence, once a change occurs in one state, the next transition may point to another state in this large space, and the ensuing sequence may change completely. Numerical trials in an N D 50 model have shown this to be the case. Thus, the correlation of nearby sequences, as deŽned by proximity in R space, may be lost in large neural networks. 3.3 Stability Under Stochastic Dynamics. Next, let us ask how robust these sequences are against noise. We choose 2 D 0.5, which is where considerable effects may be expected, since this is the lowest absolute value that the potentials hi can have. As an example, let us look at the probability of a correct four-step sequence,

P4 D P ( F4 | F3 ) P ( F3 | F2 ) P (F2 | F1 ) P ( F1 | 1),

(3.2)

where 1 designates the initial null state. This is the product of the probabilities of obtaining the correct Žrst four states of the appropriate deterministic sequence at a given point in R space. The results for the problem at hand are plotted in Figure 5 on the same grid of R space as in the two previous Žgures. We see that a few sequences are relatively stable, but others have a low probability of being correctly recovered during the Žrst four steps of this Markov process. To exemplify the origins of instability we compare two six-cycles, at locations (10, ¡10) and (10, 15) of Figures 4 and 5. In Figure 6 we plot the histograms of the hi values (over all neurons for the Žrst four steps of the dynamics) for these two cases. Clearly, at 2 D 12 , the occupancy of the bins of h D § 12 will determine the instability following from equation 2.2. Since the sequence at (10, ¡10) has three elements in these bins, it is reduced to P4 D 0.36. All other sequences in Figure 5 have three elements in these bins or more. The sequence at (10, 15) has Žve elements in these bins, as can be seen in Figure 6; hence, its retrieval probability is reduced to P4 D 0.18. Let us remark at this point on the Markov chain properties discussed in section 2.4 and, in particular, the stationary probability distribution p ( I) of equation 2.13. Using the R-space point (10, ¡10) as an example, we Žnd that the following states are the most probable ones: 17, 22, 30, 32. Their respective probabilities are 0.106, 0.175, 0.173, 0.169. In the problem at hand, we start with state 1 as the initial condition, looking for the sequence 1 ! 17 ! 22 ! 30 ! 32. The relevant transition probabilities for the terms in equation 3.2 are 0.98 £ 0.95 £ 0.73 £ 0.53, leading to P4 D 0.36. Obviously, the major reduction of the probability occurs at the Žrst two steps, from state 1 to 17 and from 17 to 22. This is also where the three appearances of

320

B. Quenet and D. Horn

0.4 0.35 0.3 P4

0.25 0.2 0.15 0.1 0.05 0 20 10 R2

0 -10 -20

0

2

4

6

8

10

12

R1

Figure 5: Probability of recovering correctly the Žrst four time steps in the sequences of Figures 3 and 4 when stochastic dynamics is being used with 2 D 0.5.

h D § 12 occur: two of them affect the transition of 1 to 17, and one occurs in the transition from 17 to 22. 3.4 Entropy of the Markov Chains. Finally, we show in Figure 7 an analysis of the entropy of the different Markov chains of this system. This is done for 2 D 0.5 and can be compared with the P4 results in Figure 5. The Markov chains with low entropy are the ordered ones, leading to a relatively high probability of correct retrieval of the Žrst four states. The chains with high entropy are the disordered ones, leading to a low probability of correct retrieval. 4 The Inverse Problem

The previous section exhibited examples of what may be called the direct DNF problem: given a matrix w and a set of R values, solve the dynamics of equation 2.1, and generate the resulting sequence. We now wish to pose an inverse problem: given a set of several sequences, Žnd a corresponding DNF, that is, values of w and R that can produce this set. There are two parts to this question. First, does a solution exist? Second,

The Dynamic Neural Filter

321

Figure 6: Histograms of hi values of two six-cycles in the N D 5 problem. Table 2: Six Spatiotemporal Sequences DeŽned for Four Time Steps in a System with Four Neurons. t/ k

1

2

3

4

5

6

1 2 3 4

1100 1110 1101 0001

1000 1100 1101 0001

1110 1111 0111 0 01 1

1000 1 01 0 0110 0111

1 01 1 1000 1110 1111

1000 1110 0111 0 0 01

if the problem is soluble, Žnd an example. This may be expanded into a search for all possible solutions, which lies outside the scope of this work. 4.1 The Existence Problem. Suppose a given data set comprises K different spatiotemporal sequences of length T each, presented as activities of N neurons. An example is shown in Table 2 for K D 6, T D 4 in a system with N D 4 neurons. We wish to Žnd out if there exists a DNF such that this set of sequences is elicited by K different inputs Rik, k D 1, . . . , K.

322

B. Quenet and D. Horn

1.8 1.6

Entropy

1.4 1.2 1 0.8 0.6 0.4 0.2 20 10 R2

0 -10 -20

0

2

4

6

8

10

12

R1

Figure 7: Entropy of the Markov chains corresponding to the sequences of Figures 3 and 4 when stochastic dynamics is being used with 2 D 0.5.

The neuronal values may be labeled nik ( t) , t D 1, . . . , T. Each time step, for each sequence, deŽnes a state of the system | k, ti D fnik ( t)giD 1,...,N .

(4.1)

Twenty-four such states can be seen in Table 2, some of them identical, yet no identical states appear in a single sequence. Each state |k, ti follows from the previous state |k, t ¡ 1i through deterministic dynamics. The initial state |k, 0i is chosen as the null state nik D 0 for all i D 1, . . . , N and k D 1, . . . , K. Consider a single neuron i in this system. We may then reduce the question posed here into a perceptron problem for this neuron. The neuron is given KT initial vectors (counting states from t D 0 to t D T ¡ 1) and its required output values nik ( t) (from t D 1 to t D T for all k D 1, . . . , K). It has to perform calculation 2.1 given a different input Rik for each of the K sequences. To take account of the K different inputs (biases), we extend the vector states of length N to new vectors of length N C K through the concatenation | k, t) D |k, ti £ fda,kgaD 1,...,K ,

(4.2)

where da,k is the Kronecker delta function obtaining the value 1 for a D k and 0 otherwise. The extension by K bias axes allows us to represent the

The Dynamic Neural Filter

323

deterministic dynamics, for each neuron i, as KT perceptron inequalities in an N C K dimensional system. These can be strictly met (Hertz et al., 1991) for N obeying

A:

KT · N C K

or

K (T ¡ 1) · N.

(4.3)

In the large N limit, one can apply the Cover result (Cover, 1965), saying that a solution may be found for

B:

KT · 2(N C K )

or

1 K (T ¡ 2) · N. 2

(4.4)

For the example of Table 1, the strict condition A implies N ¸ 18, and condition B leads to N ¸ 6. This could mean one should extend the system shown in Table 2 by at least two, if not more, neurons. Nonetheless, one can obtain a solution for N D 4. This is due to the fact that the states in this table were not chosen in a random and independent fashion. 2 Note also the example discussed in section 3.1. Thirty-three of the 38 sequences in Figure 4 have length 4 or more. Counting only the Žrst four states, we Žnd 22 different sequences, many more than conditions A and B would suggest. This is so because we did not ask in section 3.1 for the matrix that can produce an arbitrary choice of states appearing in K sequences of length 4, but counted how many different sequences were produced by a given matrix. These sequences turned out to be quite similar to one another, as described in section 3.2. Thus, conditions A and B should be regarded as sufŽciency conditions, assuring us A that a solution must exist and B that it may be found for lower N. With more effort, one can try to solve the inverse problem, that is, Žnd an explicit matrix w , for yet smaller N. 4.2 Perceptron Learning. From our approach to the existence problem, it should be quite evident that if a solution exists, it may be obtained using a straightforward extension of the Rosenblatt algorithm (Rosenblatt, 1962; Hertz et al., 1991). We describe it in this section, realizing explictly the abstract vectors in N C K dimensions described in the previous section. Let us start by deŽning, for each neuron i, an N C K–dimensional vector of perceptron weights w E i,

(w E i ) j D wij

for j D 1, . . . , N

(w E i ) N C k D Rik ¡ hi

for k D 1, . . . , K.

(4.5)

The sequence states of interest will be represented by vectors xE i, k ( t) , for neuron i, living in the same N C K–dimensional space but carrying further 2

The rationale for the choice of states in Table 2 is discussed in section 5.3.

324

B. Quenet and D. Horn

indices of k and t as follows: ( xE i,k (t) ) j D njk ( t) (2nik ( t C 1) ¡ 1)

for j D 1, . . . , N, t D 0, . . . , T ¡ 1

(xE i,k ( t) ) N C a D dak (2nik (t C 1) ¡ 1)

(4.6)

for a D 1, . . . , K.

In other words, the vectors xE i, k ( t) represent the states weighted by a positive or negative sign depending on whether the target neuron i at the next time step will be 1 or 0, respectively. With these deŽnitions, all constraints of the inverse problem can be written as KT perceptron inequalities: Ei ¢ x E i,k ( t) > 0 for all k D 1, . . . , K, t D 0, 1, . . . , T ¡ 1. w

(4.7)

We propose to use the perceptron learning rule (Hertz et al., 1991),

D wE i

E i, k ( t) H (¡w E i ¢ xE i,k ( t) ) , D gx

(4.8)

while iterating the system, time and again, over all KT states. The Heaviside function guarantees that w E i gets modiŽed at a given iteration by the vectors xE i, k (t ) that do not satisfy the inequality. This algorithm converges (Hertz et al., 1991) if the system of inequalities is soluble. Moreover, one can use the same algorithm to require stability under stochastic dynamics by insisting on a margin M in the perceptron inequalities: Ei ¢ x E i,k ( t) > M. w

(4.9)

This leads to the learning rule:

D wE i

E i, k ( t) H (¡w E i ¢ xE i,k ( t) C M ) . D gx

(4.10)

In section 3, where we used integer values of w and R and kept all h D 12 , we worked with an implicit margin of size 12 . This is the reason that testing the system with stochastic dynamics of 2 D 12 , we found considerable effects. Insisting on larger M, if the data allow it, will guarantee stability for larger ranges of 2 . Given a situation of the type encountered in Figure 5, one may want to improve the robustness of a possible code, such as the one located at (10,15) in the R plane. This can be tried by applying the algorithm presented by the previous equation for a suitably chosen M. The size of M is, of course, limited by the data. (For a general discussion of optimal margin selection, see Vapnik, 1995.) If the algorithms are implemented with g D 1, they lead to integer values of w and R ¡h. An arbitrary choice of g < 1 can lead to continuous h values. It is then advisable to keep a Žnite M to ensure some robustness to noise. Finally, we wish to comment that one may apply in a similar manner the Ho-Kashyap procedure (Duda, Hart, & Stork, 2001), indicating, if it does not converge, that a given set of data is not linearly separable.

The Dynamic Neural Filter

325

5 Capacity and Complexity 5.1 Capacity of Codes of Given Length. The DNF produces sets of spatiotemporal codes. Its capacity could be measured by the number of codes it can produce. This is a very large number and may not be very meaningful if we consider an application with stochastic dynamics. Moreover, what would be really interesting is a measure of capacity of random codes, expected to be very distant from one another. Clearly, two codes that are close will not constrain the system. If one is learned, some other close-by codes will be automatically produced for nearby R values. Suppose we look for the number of codes of length T such that the same state does not appear twice, either within the same code or among the different codes. This would be the case if the states are chosen randomly from all 2N possibilities of an N-neuron system. Following the reasoning of section 4.1, this number can be expected to reach

CT ¼

2N T ¡2

(5.1)

if criterion B is applied. CT may be viewed as a capacity measure of a DNF. Note that the last equation leads to CT D 1 for T D 2N C 2. This means that a DNF of order N may be expected to accommodate a cycle of length 2N C 2. The latter may then be used to construct the DNF. 5.2 Random Synaptic Weights. There exist synaptic matrices that do not have the ability to generate large cycles. It is well known that symmetric matrices w lead to Žxed points or two-cycles and antisymmetric matrices may lead up to four-cycles only (Peretto, 1992). In general, however, a system of N neurons may generate large cycles, limited by the total number of states, 2N . Gutfreund, Reger, and Young (1988) have shown that for Ri D 0, large cycles are generated when values of wij are chosen randomly from a gaussian P distribution centered at zero and thresholds hi D 12 j wij ¼ 0.3 This was further studied numerically by Nutzel ¨ (1991), who pointedPout that large

cycles can be obtained for some range of the asymmetry a D Pij ij

wij wji wij wij

around

the point 0 of random asymmetry.4 The average size of the cycles grows as an exponential in N, with an exponent that increases as the asymmetry 3 Related questions within the framework of Ising spin systems have been studied by Crisanti and Sompolinsky (1988). 4 This deŽnition of asymmetry differs from the parameter used by Gutfreund et al. (1988) and Nutzel ¨ (1991), but it shares the relevant characteristics of varying between 1 and ¡1, with the extremes characterizing the symmetric and antisymmetric cases. The completely asymmetric case corresponds to a D 0. The example of the Žve-dimensional matrix in section 3 has a D ¡0.4067.

326

B. Quenet and D. Horn

Table 3: Binary Odor Coding. t/ k

1

2

3

4

5

6

1 2 3 4

11 11 11 00

10 11 11 00

11 11 01 00

10 10 01 01

10 10 11 11

10 11 01 00

approaches zero. A system with a similar structure but slightly different dynamics was studied by McGuire, Littlewort, and Rafelski (1991). Working with nonzero R values in our DNF, we noted that the interesting large cycles occur around the center of the relevant range of R. This is where the average effect of the inputs cancels the average effect of the synaptic weights, in agreement with the observations of Gutfreund et al. (1988). Thus, we expect, for any given N, that matrices w that have large capacities CT for large T have a random-like distribution of synaptic values. We are, however, content even with cycles of order N, as is the case of our examples in section 3. Note that in the locust antennal lobe (see section 2.1), hundreds of neurons are participating in the real biological system. Hence, it is quite reasonable for it not to display cyclic behavior over the few LFP bins over which it is measured. 5.3 An Example: The Wehr-Laurent Experiment. We return now to the example that motivated our investigation. Wehr and Laurent (1996) display in Figure 3 of their article the response of two speciŽc neurons to nine mixtures of odors. SigniŽcant results were obtained during the Žrst four reverberations of the local Želd potential. They can be presented as six different binary codings, shown in Table 3, three of which appeared twice in the nine odor mixtures. Each of the six columns speciŽes the state of activity of the observed pair of neurons for one of the six spatiotemporal codes. Looking at the Žrst column, it is clear that it cannot be produced by a DNF, where states at time t are determined by states at t ¡ 1, unless it possesses at least two hidden neurons. Hence, we conclude that this should be treated as a T D 4, K D 6 problem. Looking for a system with C4 ¼ 6, we know we can do it with N D 18 (using condition A), but may expect to be able to do it with N D 6 (using condition B) or less. It turns out we can do it with N D 4. A set of states of four neurons, which accommodate the two observed neurons of Table 3, is given in Table 2. The latter was constructed by choosing states of hidden neurons by trial and error until all perceptron inequalities were satisŽed, ensuring the existence of DNFs with matrices w and inputs Rk that can generate this table. Thus, we are able to implement the binary odor coding of Wehr and Laurent (1996) in an N D 4 model.

The Dynamic Neural Filter

327

5.4 Minimal DNF as Measure of Complexity. The example of the WehrLaurent experiment raises the possibility of using the DNF model to deŽne the complexity of a spatiotemporal data set. This degree of complexity can be characterized by the minimal number of neurons N needed to accommodate the data within a DNF. From the discussion in the previous paragraph we conclude that the Wehr-Laurent problem of Table 3 has DNF complexity of degree 4.5 6 Discussion

The problem of odor spatiotemporal encoding was recently reviewed in detail by Laurent et al. (2001). As a theoretical model, these authors suggest “winnerless competition,” a concept that was expanded in Rabinovich et al. (2000, 2001). Basing their intuition on Lotka-Volterra equations they point out that for suitably chosen parameters, there exist heteroclinic orbits connecting all N attractors of the N-dimensional system, which are very sensitive to external inputs. They studied systems of FitzHugh-Nagumo neurons that exhibit spatiotemporal sensitivity to external inputs as a neural implementation, but made no attempt to Žt a particular data set such as the one of Wehr and Laurent (1996). In comparison, we use a binary model with discrete dynamics, but nevertheless we claim that it is relevant to the observed data. The reason is that the periodic LFP in the antennal lobe provides a temporal scale that accounts for speciŽc time bins, and allows discrete coding, as observed by Wehr and Laurent (1996). With a DNF model that Žts these data, one can envisage a model of spiking neurons capable of mimicking the experimental results (Quenet, Dubois, Sirapian, Dreyfus, & Horn, in press). This can be done by imposing overall periodic inhibition that allows for action potentials during periodic time windows and by Žtting synaptic delay parameters to match the same windows. Recently Friedrich & Laurent (2001) have observed spatiotemporal odor representations in the olfactory bulb of zebraŽsh. This system also has a reverberating local Želd potential and mitral cells that Žre in coincidence with it. The authors have studied the response of this system to similar odors, characterized by small changes in molecular structures of the relevant chemicals. One of their interesting results is that the correlation between temporal patterns of similar odors over the mitral cells reduces with time (between the Žrst and second 500 ms after odor presentation). This contradicts the expectation that short distances in R space lead to high correlations between the spatiotemporal sequences. The reason is the large value of N in

5 Obviously this should be regarded as a mathematical statement and may have nothing to do with the biological reality. In the antennal lobe, one Žnds excitation of hundreds of PNs during each odor presentation.

328

B. Quenet and D. Horn

this system. Since the volume of state-space is 2N , once a change in a sequence occurs, it may lead to a new sequence uncorrelated with the old one. Our model can be compared with recent work by Maass, Natschl a¨ ger, and Markram (in press), who propose a recurrent neural network that performs what they call “liquid computation.” Their network is composed of spiking neurons and is structured in two stages: (1) a Žlter transforming spatiotemporal input into spatiotemporal behavior of the network and (2) the activation of a readout map. Thus, this network projects spatiotemporal input into an output representation that is spatial in nature. Although these authors put special emphasis on real-time learning based on perturbations, the common feature of their approach and ours is the use of a Žlter projecting from an input to an output space. Whereas in our model the input space is spatial and the output space is spatiotemporal, their model goes in the opposite direction. However, in general, both models can connect spatiotemporal spaces to one another, and both use neural networks as Žlters, performing computations that are different from the conventional major paradigms. Our model is based on binary mathematical neurons, whereas Maass et al. (in press) deal with more realistic neural behavior. Nevertheless, the advantage of our DNF is that its structure can be made explicit, thus allowing full understanding of its operation and capabilities. Among other things, we can use this understanding to solve an inverse problem: construct a DNF that produces a given repertoire of spatiotemporal data. Acknowledgments

We thank G. Dror for advice and assistance. References Cover, T. M. (1965). Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition. IEEE Trans. Elect. Comp., 14, 326–334. Cover, T. M., & Thomas, J. A. (1991). Elements of information theory. New York: Wiley. Crisanti, A., & Sompolinsky, H. (1988). Dynamics of spin systems with randomly asymmetric bonds: Ising systems and Glauber dynamics. Phys. Rev. A, 37, 4865–4874. Dayan, P., & Abbott, L. F. (2001). Theoretical neuroscience. Cambridge, MA: MIT Press. Duda, R. O., Hart, P. E., & Stork, D. G. (2001). Pattern classiŽcation. New York: Wiley. Friedrich, R. W., & Laurent, G. (2001). Dynamic optimization of odor representations by slow temporal patterning of mitral cell activity. Science, 291, 889–894.

The Dynamic Neural Filter

329

Gutfreund, H., Reger, D. J., & Young, A. P. (1988). The nature of attractors in an asymmetric spin glass with deterministic dynamics. J. Phys. A: Math. Gen., 21, 2775–2797. Hertz, J., Krogh, A., & Palmer, R. G. (1991). Introduction to the theory of neural computation. Reading, MA: Addison-Wesley. HopŽeld, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proc. Nat. Acad. Sci. USA, 79, 2554–2558. Kohonen, T. (1982). Self-organized formation of topologically correct feature maps. Biol. Cyb., 43, 59–69. Laurent, G., & Naraghi, M. (1994). Odorant-induced oscillations in the mushroom bodies of the locust. J. Neurosciences, 14, 2993–3004. Laurent, G., Stopfer, M., Friedrich, R. W., Rabinovich, M. I., Volkovskii, A., & Abarbanel, H. D. I. (2001). Odor encoding as an active dynamical process: Experiments, computation and theory. Ann. Rev. Neurosci., 24, 263–297. Maass, W., Natschl¨ager, T., & Markram, H. (in press). Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural Computation. McGuire, P. C., Littlewort, G. C., & Rafelski, J. (1991). Brainwashing random asymmetric “neural” networks. Physics Letters A, 160, 255–260. Nutzel, ¨ K. (1991). The length of attractors in asymmetric random neural networks with deterministic dynamics. J. Phys. A: Math. Gen., 24, L151–L157. Peretto, P. (1992). An introduction to the modeling of neural networks. Cambridge: Cambridge University Press. Quenet, B., Cerny, V., Dreyfus, G., & Lutz, A. (1997). A dynamic model of key feature extraction: The example of olfaction. Unpublished manuscript, ESPCI. Quenet, B., Dubois, R., Sirapian, S., Dreyfus, G., & Horn, D. (in press). Two step modeling of spatiotemporal olfactory data: From binary to Hodgkin-Huxley neurons. Biosystems. Quenet, B., Horn, D., Dreyfus, G., & Dubois, R. (2001). Temporal coding in an olfactory oscillatory model. Neurocomputing, 38–40, 831–836. Rabinovich, M. I., Huerta, R., Volkovskii, A., Abarbanel, H. D. I., Stopfer, M., & Laurent, G. (2000). Dynamical coding of sensory information with competitive networks. J. Physiol. (Paris), 94, 465–471. Rabinovich, M. I., Volkovskii, A., Lecanda, P., Huerta, R., Abarbanel, H. D. I., & Laurent, G. (2001). Dynamical encoding by networks of competing neuron groups: Winnerless competition. Phys. Rev. Lett., 87, 068102. Rosenblatt, F. (1962). Principles of neurodynamics. New York: Spartan. Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning internal representation by back-propagating errors. Nature, 323, 533–536. Vapnik, V. (1995). The nature of statistical learning theory. New York: SpringerVerlag. Wehr, M., & Laurent, G. (1996). Odor encoding by temporal sequences of Žring in oscillating neural assemblies. Nature, 384, 162–166. Received November 20, 2001; accepted July 30, 2002.

LETTER

Communicated by Kwabena Boahen

Modeling Short-Term Synaptic Depression in Silicon Malte Boegerhausen [email protected] Pascal Suter [email protected] Shih-Chii Liu [email protected] Institute of Neuroinformatics, University of Zurich and ETH Zurich, Winterthurerstrasse 190, CH-8057 Zurich, Switzerland We describe a model of short-term synaptic depression that is derived from a circuit implementation. The dynamics of this circuit model is similar to the dynamics of some theoretical models of short-term depression except that the recovery dynamics of the variable describing the depression is nonlinear and it also depends on the presynaptic frequency. The equations describing the steady-state and transient responses of this synaptic model are compared to the experimental results obtained from a fabricated silicon network consisting of leaky integrate-and-Žre neurons and different types of short-term dynamic synapses. We also show experimental data demonstrating the possible computational roles of depression. One possible role of a depressing synapse is that the input can quickly bring the neuron up to threshold when the membrane potential is close to the resting potential. 1 Introduction

Short-term synaptic dynamics has been observed in different parts of the cortical system (Stratford, Tarczy-Hornoch, Martin, Bannister, & Jack, 1998; Varela et al., 1997; Markram, Wang, & Tsodyks, 1998; Reyes et al., 1998). The functionality of the short-term dynamics has been implicated in various cortical models (Senn, Segev, & Tsodyks, 1998; Chance, Nelson, & Abbott, 1998; Matveev & Wang, 2000). The properties of a network with dynamic synapses have been investigated by Tsodyks, Pawelzik, and Markram (1998), and Maass and Zador (1999) and the use of such a network in time series processing by Maass and Zador (1999) and Natschlager, Maass, and Zador (2001). The introduction of these dynamic synapses into hardware implementations of recurrent neuronal networks allows a wide range of operating regimes, especially in the case of time-varying inputs. In this work, we describe a model that was derived from a circuit implementation of short-term depression and facilitation. The dynamics of this Neural Computation 15, 331–348 (2003)

° c 2002 Massachusetts Institute of Technology

332

M. Boegerhausen, P. Suter, and S. Liu

circuit, initially proposed by Rasche and Hahnloser (2001), was not analyzed in their work. Our work is presented in two parts. We Žrst describe the model of this synaptic circuit and compare the differential equations describing the depression with the equations of one of the theoretical models frequently used in network simulations (Abbott, Sen, Varela, & Nelson, 1997; Varela et al., 1997). We analyze the transient and steady-state responses of this synaptic model in response to inputs of different statistical distributions. Next, we compare the theoretical results with experimental responses obtained from a silicon network of leaky integrate-and-Žre neurons together with different short-term dynamic synapses. We also show experimental data from the chip that demonstrate the possible computational roles of depression. From these experiments, we postulate that one possible role of depression is to bring the neuron’s response up to threshold quickly if the membrane potential of the neuron was close to the resting potential. We also mapped a proposed cortical model of direction selectivity that uses depressing synapses onto this chip. The results are qualitatively similar to the results obtained in the original work (Chance et al., 1998). The similarity of the circuit responses to the responses from Abbott and colleagues’ model means that we can use these VLSI networks of integrateand-Žre (IF) neurons as an alternative to computer simulations of dynamical networks composed of large numbers of integrate-and-Žre (IF) neurons using synapses with different time constants. The outputs of such networks can also be used as an interface with wetware. An infrastructure for a reprogrammable, reconŽgurable, multichip neuronal system is being developed along with a user-deŽned interface so that the system is easily accessible to a naive user. 2 Comparisons Between Models of Depression

Two theoretical models describing synaptic depression and facilitation used frequently in network simulations are the phenomenological model of Tsodyks and Markram (1997) and the model from Abbott et al. (1997). Because these models are equivalent if the inactivation synaptic time constant in Tsodyks and Markram’s model is small when compared with the recovery time constant of the depression, we compare the circuit model only with the model from Abbott and colleagues. Here, we describe only the circuit model for synaptic depression. The equivalent model for facilitation is described elsewhere (Liu, in press). 2.1 Theoretical Model for Depression Model. In the model from Abbott et al. (1997), the synaptic strength is described by gD (t) , where D is a variable between 0 and 1 that describes the amount of depression (D D 1 means no depression) and g is the maximum synaptic strength. The recovery

Modeling Short-Term Synaptic Depression in Silicon

333

dynamics of D is td

dD D 1 ¡ D, dt

(2.1)

where td is the recovery time of the depression. The update equation for D right after a spike at time t D tsp is C ¡ ) D dD ( tsp ), D ( tsp

(2.2)

where d (d < 1) is the amount by which D is decreased right after the spike and tsp is the time of the spike. The average steady-state value of depression for a regular spike train with a rate r is Dss D

1 ¡ e¡1/ ( rtd ) . 1 ¡ de ¡1 / (rtd )

(2.3)

2.2 Circuit Model of Depressing Synapse. In this model of synaptic depression derived from the circuit implementation, the equation that describes the recovery dynamics of the depressing variable, D, is nonlinear. This nonlinearity comes about because there is no compact circuitry to implement a linear resistor whose value can be adjusted easily after fabrication. A transistor can act as a linear resistor as long as the terminal voltages satisfy certain criteria. Additional circuitry would be needed to satisfy these criteria for a wide range of voltages, and it would increase the size of the circuit. One alternative is to replace the exponential dynamics in equation 2.1 with diode dynamics, which is easily obtained with a single diode-connected transistor. The current through the diode has a nonlinear decay instead of an exponential decay in the case of a linear resistor. Because of the different dynamics, the equation describing the recovery of D (derived from the circuit in the region where a transistor operates in the subthreshold region or the current is exponential in the gate voltage of the transistor) can be formulated as

dD 1k D M (1 ¡ D / ) , dt

(2.4)

where 1 / M is the equivalent of td in equation 2.1 and k (a transistor parameter) is less than 1. The maximum value of D is 1. The update equation remains as before: C ¡ ) D dD ( tsp ). D ( tsp

(2.5)

2.2.1 Circuit. Equations 2.4 and 2.5 are derived from the circuit in Figure 1. The operation of this circuit is described in the caption, and the detailed analysis leading to the differential equations for D is described in the

334

M. Boegerhausen, P. Suter, and S. Liu

Figure 1: Schematic for a depressing synapse circuit and responses to a regular input spike train. (a) Depressing synapse circuit. The voltage Va determines the synaptic conductance g, while the synaptic term gD or Isyn is exponential in the voltage, Vx . The subcircuit consisting of transistors, M1 , M2 , and M3 controls the dynamics of Isyn. The presynaptic input goes to the gate terminal of M3 , which acts like a switch. When there is a presynaptic spike, a quantity of charge (determined by Vd ) is removed from the node Vx . In between spikes, Vx recovers to the voltage, Va , through the diode-connected transistor, M1 . When there is no spike, Vx is around Va . When the presynaptic input comes from a regular spike train, Vx decreases with each spike and recovers between spikes. It reaches a steady-state value, as shown in b. During the spike, transistor M4 turns on, and the synaptic weight current Isyn charges up the membrane potential of the neuron through the current-mirror circuit consisting of M6 , M7 , and the capacitor C2 . We can convert the Isyn current source into a synaptic current Id with some gain and a “time constant” by adjusting the voltage Vgain . The decay dynamics of Id is given by Id ( t) D

Isyn (t D tsp ) 1C

AIsyn (tD tsp ) QT

k (V

where QT D C2 UT and A D e

) dd ¡Vgain / UT .

In a

normal synapse circuit (i.e., without short-term dynamics), Vx is controlled by an external bias voltage. (b) Input spike train at a frequency of 20 Hz (bottom curve) and corresponding response Vx (top curve) of the circuit for Vd D 0.3 V. The diode-connected transistor M1 has nonlinear dynamics. This dynamics is discussed in the text. The recovery time of the variable D varies depending on the distance of the present value of Vx from Va . The recovery rate of D increases for a larger difference between Vx and Va .

appendix. The voltage Vx in Figure 1 codes for gD. The conductance g is set by Va , while the dynamics of D is set by both Vd and Va . The time taken for the present value of Vx to return to the value of Va is determined by the current dynamics of the diode-connected transistor M1 and Va . As shown in the appendix, the recovery time constant (1 / M) of D is set by Va .

Modeling Short-Term Synaptic Depression in Silicon

335

The synaptic weight is described by the current, Isyn in Figure 1a, Isyn ( t) D Ion ek Vx / UT D gD ( t) , where g D Ion ek Va / UT is the synaptic strength, D ( t) is k ( Vdd ¡Vx ) / UT

Irf D Iop e

(2.6) k (V

e

dd ¡Va ) / UT Iop , Irf ( t)

and

. The recovery time constant (1 / M) of D is set by Va

Iopk ¡(1¡k ) ( V ¡Va ) / UT dd ). QT e

(M D The synaptic current, Id to the neuron, is then a current source Isyn , which lasts for the duration of the pulse width of the presynaptic spike. However, we can set a longer time constant for the synaptic current through Vgain . The equation describing this dependence (that is, the equation of the current in a current-mirror circuit) is given in the caption of Figure 1. It is difŽcult to compute a closed-form solution for equation 2.4 for any value of k (a transistor parameter which is less than 1). This value also changes under different operating conditions and between transistors fabricated in different processes. Hence, we solve for D (t) in the case of k D 0.5 given that the last spike occurred at t D t0 1 : dD / dt D M (1 ¡ D2 ) ) D ( t) D D

D (t 0 ) C D ( t0 ) e2Mt ¡ 1 C e2Mt ¡D(t0 ) C D ( t0 ) e2Mt C 1 C e2Mt

D ( t0 ) cosh ( Mt ) C sinh ( Mt ) . cosh ( Mt ) C D ( t0 ) sinh ( Mt )

(2.7)

When D is far from its recovered value of 1, we can approximate its recovery dynamics by dD / dt D M (irrespective of k ), and solving for D (t ), we get D ( t) D Mt C D (t 0 ) . In this regime, D ( t) follows a linear trajectory. Note that the same is true of equation 2.1 when t ¿ td . 3 Neuron Circuit

The dynamics of the neuron circuit of Figure 2 is similar to that of a leaky IF neuron with a constant leak. The circuit was previously described in Indiveri (2000) and Liu et al. (2001). It is a modiŽed version of previous designs (Mead, 1989; Van Schaik, 2001) and also includes the circuitry that models Žring-rate adaptation (Boahen, 1997a, 1997b) frequently seen in pyramidal 1

Note that if k =1, the equation reduces to equation 2.1.

336

M. Boegerhausen, P. Suter, and S. Liu C1

Vb V thresh Id V leak

Ileak Cm

Vo

Vm M5

V pw

M6

Vo

Vo

Vm

M1

Vo

M2

Vca

M4

M3

V refr

Ipw

Ca

Vt

Spike adaptation circuitry

Figure 2: Schematic of the leaky integrate-and-Žre neuron circuit. In this Žgure, Vrefr, sets the refractory period, Vthresh sets the threshold voltage, Vpw sets the pulse width of the spike, Vleak sets the leak current Ileak, and VCa and Vt set the output spike adaptation dynamics.

cells in the cortex. The equation describing the depolarization of the soma potential Vm is Cm

dV m (t) D i ( t ) ¡ Ileak ¡ Iahp , dt

Vm (t ) < Vthresh ,

(3.1)

where i ( t) is the synaptic current to the soma, Ileak is the leakage current, and Iahp is the after-hyperpolarization potassium (K) current, which causes the adaptation in the Žring rate of the cells. The synaptic current can be set to a point current source, that is, i ( t) D Idd ( t ¡tjk ) , or it can have a “time constant” and gain that is determined by the current-mirror circuit consisting of M6 , M7 , and C2 in Figure 1. The time constant and gain are controlled by the ( tD tsp ) I voltage Vgain . The decay dynamics of Id is of the form Id ( t) D syn AIsyn (tD tsp ) , kV

dd

¡V

1C

QT

gain

UT where QT D C2 UT and A D e . The operation of the circuit is as follows. When Vm (t ) increases above Vthresh at ts D t (ts is the time of the output spike), it increases by a step increment determined by the amplitude of the spike and the capacitive divider set by C1 and Cm . The output Vo becomes active at this time and turns on the discharging current path through transistors M5 and M6 . The time during which Vo remains high, TP , depends on the time taken for Vm to discharge below Vthresh . In this design, the pulse width TP is determined by the rate at which Vm is discharged, which in turn depends on the difference between the input current Id , the leak current Ileak, and the current Ipw . In other designs, Vm is reset immediately below Vthresh when Vo becomes active because either the input current is blocked from charging the membrane or

Modeling Short-Term Synaptic Depression in Silicon

337

the current Ipw is much larger than the input current. The refractory period, TR , is determined by Vrefr , which keeps Vo high so that Id cannot charge up the membrane. The spike output is taken from the node VNo . We ignore the Iahp current in determining the Žring rate dependence on the input current. The time needed for the neuron to charge up to threshold V

with the input current is TI D Cm i ( t)thresh . The spiking rate is given by ¡I rD

leak

1 . TI C TP C TR

4 Comparison Between Models

We compare the two models by looking at how D changes in response to a Poisson-distributed train whose frequency varied from 40 Hz to 1 Hz, as shown in Figure 3. We used a simple linear differential equation to describe the dynamics of the membrane potential Vm , tm

dV m (t) D Ri ( t) ¡ Vm ( t) , dt

where tm is the membrane time constant and i ( t) is the synaptic current. We ran an optimization algorithm on the parameters in the two models so that the least-square error between the excitatory postsynaptic potential (EPSP) outputs of both models was at a minimum. In this case, the EPSP responses were identical (see Figure 3b) and the corresponding D values (see Figure 3c) were almost identical except in the region where D was close to the maximum value. We performed the same comparison with Tsodyks and Markram’s model, and the results were similar. Hence, the circuit model can be used to describe short-term synaptic depression in a network simulation. However, the nonlinear recovery dynamics of the circuit model leads a different functional dependence of the average steady-state EPSP on the frequency of a regular input spike train. 5 Transient Response

The data in the Žgures in the remainder of this article are obtained from a fabricated silicon network of aVLSI IF neurons of the type described in section 3 together with different types of synapses. We Žrst measured the transient response of the neuron when stimulated by a 10 Hz spike train through the depressing synapse. We tuned the parameters of the synapse and the leak current so that the membrane potential did not build up to threshold. These data are shown in Figure 4a for a regular train and in Figure 4b for a Poisson-distributed train. The measured EPSP amplitude, D Vm can be Žtted with equation 2.6,

D Vm ( t)

D

dt d (Isyn (t) ¡ Ileak ) D t ( gD ( t) ¡ Ileak ) , Cm Cm

(5.1)

338

M. Boegerhausen, P. Suter, and S. Liu

Figure 3: Comparison between the outputs of the two models of depression. An optimization algorithm was used to determine the parameters of the models so that the least-square error between the EPSPs from the two models was at a minimum. The corresponding D distribution is shown in c. (a) Poissondistributed input with an initial frequency of 40 Hz and an end frequency of 1 Hz. (b) The EPSP responses of both models were identical. (c) The D values were almost identical except in the region when D is close to 1. Parameters used in the simulations: tm D 20 ms, d D 0.6, k D 0.7, M D 2.20 s ¡1 .

where Cm is the membrane capacitance, Ileak is the leak current, dt is the pulse width of the presynaptic spike, and D ( t) is computed using equation 2.7. Equation 5.1 Žts the measured data well (see Figure 4). 6 Steady-State Response

The equation describing the dependence of the steady-state values of D on the presynaptic frequency can easily be determined in the case of a regular spiking input of rate r by using equations 2.5 and 2.7. From these equations, we get p (¡1 C d) (1 C e2M/ r ) C 4d(¡1 C e2M/ r ) 2 C (1¡ d ) 2 (1 C e2M/ r ) 2 . (6.1) Dss D 2d(¡1 C e2M/ r )

Modeling Short-Term Synaptic Depression in Silicon

339

Figure 4: Transient responses to a (a) 10 Hz regular spike train and a (b) 10 Hz Poisson-distributed train. The input is the bottom curve of each plot. In a, the amplitude of the EPSP decreases with each incoming input spike, clearly showing the effect of synaptic depression. In b, the EPSP amplitude depends on the occurrence of the previous spike. The asterisks are the Žts of the circuit model to the peak value of each EPSP. The Žts give a d value of 0.79.

340

M. Boegerhausen, P. Suter, and S. Liu

If we use the reduced dynamics expression dD / dt D M, we replace equation 2.7 with equation 3.1 and obtain a simpler expression for Dss : Dss D

M . (1 ¡ d ) r

(6.2)

This equation shows that the steady-state D, and, hence, the steady-state EPSP amplitude, is inversely dependent on the presynaptic rate r. The form of the curve is similar to the results obtained by Abbott et al. (1997), where the data can be Žtted with equation 2.3. From the chip, we measured the steady-state EPSP amplitudes using the two-input spike distributions changing over a frequency range of 3 Hz to 50 Hz in steps of 1 Hz. Each frequency interval lasted 15 s, and the EPSP amplitude was averaged in the last 5 s to obtain the steady-state value. Four separate trials were performed; the resulting mean and the variance of the measurements are shown in Figure 5 for both the regular train and the Poisson-distributed train. We Žtted the experimental data using equations 5.1 and 6.1. The parameters from the Žts to the data in Figure 5a were then used to generate the Žtted curve to the data from the Poissondistributed train. The values from the Žts give recovery time constants from 1 to 3 s and Dss values varying between 0.02 and 0.04. 7 Role of Synaptic Depression

Different computational roles have been proposed for networks that incorporate synaptic depression. In this section, we describe some measurements that illustrate the postulated roles of depression. 7.1 Gain Control. Depressing synapses have been implicated in cortical gain control (Abbott et al., 1997). A depressing synapse acts like a transient detector to changes in frequency (or a Žrst derivative Žlter) similar to the way in which the photoreceptor responds with a higher gain to contrast and is less responsive to background illumination. A synapse with short-term depression responds equally to equal percentage rate changes in its input on different Žring rates. We demonstrate the gain control mechanism of short-term depression by measuring the neuron’s response to step changes in input frequency from 10 Hz to 20 Hz to 40 Hz. Each step change is the same rate change in input frequency. These results are shown in Figure 7.1a for a regular train and in Figure 6b for a Poisson-distributed train. Each frequency epoch lasted 3 s, so the synaptic strength should have reached steady state before the next step increase in input frequency. For both Žgures in Figure 7.1, the top curve shows the response of the neuron when stimulated by the input (bottom curve) through a depressing synapse (top curve) and a nondepressing synapse (middle curve). We tuned the parameters of

Modeling Short-Term Synaptic Depression in Silicon

341

Figure 5: Dependence of the steady-state EPSP amplitude on the input frequency for different values of depression. The data were measured from the fabricated circuit. The solid lines are Žts using equations 5.1 and 6.1. The parameters for the Žts are determined using the data in a. (a) Steady-state EPSP amplitude versus frequency for a regular train. From the retrieved values of d (between 0.2 and 0.6), we Žnd that Dss varies between 0.02 to 0.04. (b) Steadystate EPSP amplitude versus frequency for a Poisson-distributed train.

342

M. Boegerhausen, P. Suter, and S. Liu

both types of synapses in Figure 6b so that the steady-state Žring rate of the neuron was similar during the 20 Hz input period. Figure 6a shows clearly the transient increase in the Žring rate of a neuron when stimulated through a depressing synapse right after each step increase in input frequency and the subsequent adaptation of its Žring rate to a steady-state value. The steady-state Žring rate of the neuron with a depressing synapse is less dependent on the absolute input frequency when compared to the Žring rate of the neuron when stimulated through the nondepressing synapse. In the latter case, the Žring rate of the neuron is approximately linear in the input rate. The data shown in Figure 6a were obtained using a Poisson-distributed input. One obvious difference in the responses between the depressing and nondepressing synapse is that the neuron quickly reached threshold for a 10 Hz input for the depressing-synapse case, while the neuron remained subthreshold in the nondepressing synapse case until the input increased to 20 Hz. This suggests that a depressing synapse can be used to drive a neuron quickly to threshold when the membrane potential of the neuron is far from the threshold. 7.2 Phase Advance. Another feature of the depressing synapse is the phase advance of the membrane response of the neuron. This advance increases when the depression increases (see Figure 7). This feature was exploited in a model that described the direction-selective responses of visual cortical neurons (Chance et al., 1998). In this model, the neuron was driven by the output of a set of lateral geniculate nucleus (LGN) cells through depressing synapses and through an adjacent spatially shifted set of LGN cells through nondepressing synapses. We have attempted the same experiment using a neuron on our chip by using spikes recorded from an LGN

Figure 6: Facing page. Response of neuron to a step change in input frequency (bottom curve) when stimulated through a depressing synapse (top curve) and a nondepressing synapse (middle curve). The neuron was stimulated for three frequency intervals (10 Hz to 20 Hz to 40 Hz) lasting 3 s each. (a) Response of neuron using a regular spiking input. The steady-state Žring rate of the neuron increased almost linearly with the input frequency when stimulated through the nondepressing synapse. In the depressing synapse curve, there is a transient increase in the neuron’s Žring rate before the rate adapted to steady state. The steady-state output is less dependent on the absolute input frequency. (b) Response of neuron using a Poisson-distributed input. The parameters for both types of synapses were tuned so that the steady-state Žring rates were about the same at the end of each frequency interval for both synapses. Notice that during the 10 Hz interval, the neuron quickly built up to threshold if it was stimulated through the depressing synapse, but it stayed subthreshold when it was driven through to the nondepressing synapse until the input frequency reached 20 Hz.

Modeling Short-Term Synaptic Depression in Silicon

343

344

M. Boegerhausen, P. Suter, and S. Liu

Figure 7: Phase advance in the membrane response for different levels of depression in the synapse. The neuron was driven through the synapse by a Poisson-distributed input depicted at the bottom of the Žgure. The neuron shows the largest phase advance when the synapse has the largest depression value (Vd D 0.3V has the largest d value in the plot).

cell in the cat visual cortex during stimulation with a drifting sinusoidal grating (courtesy of K. Martin). An example of the direction-selective response is shown in Figure 8 (Liu, 2001). The direction-selective results were qualitatively similar to the data in Chance et al. (1998). 8 Conclusion

We described a model of synaptic depression derived from a circuit implementation. This circuit model has nonlinear recovery dynamics for the depression variable in contrast to current theoretical models of dynamic synapses. However, it gives qualitatively similar results when compared to the model of Abbott and colleagues (1997). Measured data from a chip with aVLSI IF neurons and dynamic synapses show that this network can be used to simulate the responses of dynamic networks with short-term dynamical synapses. Experimental results suggest that depressing synapses can be used to drive a neuron quickly up to threshold if its membrane potential is at the resting potential. The silicon networks provide an alternative to computer simulation of spike-based processing models with different time constant synapses because they run in real time and the computational time does not scale with the size of the neuronal network.

Modeling Short-Term Synaptic Depression in Silicon

345

Figure 8: Response to a 1 Hz drifting sinusoidal grating. The input spikes from the LGN cell are depicted at the bottom of the curve. The top curve shows the response of the neuron when the grating drifted in the preferred direction. The sharp excursions at the top of the potential are the output spikes of the neuron. The middle curve shows the response to the stimulus in the null direction. The membrane potential did not build up to threshold.

Appendix

The current equation for a p-type Želd-effect transistor (pFET) in subthreshold where the bulk of the transistor is at the voltage Vdd is Iop ek ( Vdd ¡Vg ) / UT (e ¡(Vdd ¡Vs ) / UT ¡ e ( Vdd ¡Vd ) / UT ) , where Vg is the voltage at the gate, and Vd and Vs are the voltages at the drain and source terminals of a transistor; k describes the efŽciency of the gate in controlling the current in the channel and is less than 1 for a transistor operating in subthreshold. The voltages in the exponent are normalized in terms of the thermal voltage, UT . The differential equation for the current in a diode-capacitor circuit (e.g., M1 and C in Figure 1) when there is no input current is given by Iop ek ( Vdd ¡Vx ) / UT ( e¡(Vdd ¡Va ) / UT ¡ e ¡(Vdd ¡Vx ) / UT ) D C

dV x . dt

(A.1)

346

M. Boegerhausen, P. Suter, and S. Liu

By introducing the current variable, Irf D Iop ek ( Vdd ¡Vx ) / UT , we solve for the differential, dI rf : dI rf D ¡

k UT

Irf dV x ) dV x D ¡

UT dI rf . k Irf

By changing variables from Vx to Irf in equation A.1, we get, Á

0

!

1 d k Irf

QT

"

) D dt @e¡(Vdd ¡Va / UT

k (V

By substituting D D

e

dd

¡Va ) / UT

Irf

Iop

Iop ¡ Irf

#1/k 1 A.

(A.2)

into equation A.2, we get

Iopk ¡(1¡k ) ( V ¡Va ) / UT dD dd (1 ¡ D1/k ) D M (1 ¡ D1 /k ) , D e dt QT

(A.3)

where M D QT e¡(1¡k ) ( Vdd ¡Va ) / UT . In steady state, D D 1. The update dynamics for D can be inferred from the dynamics of Ir , which are described by Iopk

Ir ( tnC ) D (1 C a) Ir ( tn¡ ) ,

(A.4)

where tnC is the time right after the nth spike and tn¡ is the time right before the nth spike. The a factor is determined by a voltage parameter and the pulse width of the presynaptic spike. The corresponding equation for D is D ( tnC ) D dD ( tn¡ ) , where d is

1 1C a

(A.5)

in equation 2.5. 2

Acknowledgments

This work was supported in part by the Swiss National Foundation Research SPP grant. We also acknowledge Kevan Martin, Pamela Baker, and Ora Ohana for many discussions on dynamic synapses. 2 In equation A.5, for d D 1, a has to be 0. But a is never zero due to leakage currents. To compute d, we Žrst solve for 1 C a D e (k Idt )/ QT , where I is set by the voltage parameter Vd , dt is the pulse width of the input spike, and k is a transistor parameter. The smallest I is the leakage current, so for this value, we approximate a D (k Ir0dt ) / QT .

Modeling Short-Term Synaptic Depression in Silicon

347

References Abbott, L., Sen, K., Varela, J., & Nelson, S. (1997).Synaptic depression and cortical gain control. Science, 275, 220–223. Boahen, K. A. (1997a). The retinomorphic approach: Pixel-parallel adaptive ampliŽcation, Žltering, and quantization. Analog Integrated Circuits and Signal Processing, 13, 53–68. Boahen, K. A. (1997b). Retinomorphic vision systems: Reverse engineering the vertebrate retina. Unpublished doctoral dissertation, California Institute of Technology, Pasadena, CA. Chance, F., Nelson, S., & Abbott, L. (1998). Synaptic depression and the temporal response characteristics of V1 cells. Journal of Neuroscience, 18, 4785–4799. Indiveri, G. (2000). Modeling selective attention using a neuromorphic aVLSI device. Neural Computation, 12, 2857–2880. Liu, S.-C. (2001, Oct. 19). Simple cortical modeling with aVLSI spiking neurons and dynamic synapses. Paper presented at the ZNZ Symposium, University of Zurich. Liu, S.-C. (in press). Analog VLSI circuits for short-term dynamic synapses. EURASIP Journal on Applied Signal Processing: Special Issue. Liu, S.-C., Kramer, J., Indiveri, G., Delbruck, ¨ T., Burg, T., & Douglas, R. (2001). Orientation-selective aVLSI spiking neurons. Neural Networks: Special Issue, 14(6/7), 629–643. Maass, W., & Zador, A. (1999). Computing and learning with dynamic synapses. In W. Maass & C. M. Bishop (Eds.), Pulsed neural networks (pp. 157–178). Cambridge, MA: MIT Press. Markram, H., Wang, Y., & Tsodyks, M. (1998). Differential signaling via the same axon of neocortical pyramidal neurons. Proc. Natl. Acad. Sci. USA, 95, 5323–5328. Matveev, V., & Wang, X. (2000). Differential short-term synaptic plasticity and transmission of complex spike trains: To depress or to facilitate? Cerebral Cortex, 10, 1143–1153. Mead, C. (1989). Analog VLSI and neural systems. Reading, MA: Addison-Wesley. Natschlager, T., Maass, W., & Zador, A. (2001). EfŽcient temporal processing with biologically realistic dynamic synapses. Network, 12, 75–87. Rasche, C., & Hahnloser, R. (2001). Silicon synaptic depression. Biological Cybernetics, 84, 57–62. Reyes, A., Lujan, R., Rozov, A., Burnashev, N., Somogyi, P., & Sakmann, B. (1998). Target-cell-speciŽc facilitation and depression in neocortical circuits. Nature Neuroscience, 1, 279–285. Senn, W., Segev, I., & Tsodyks, M. (1998). Reading neuronal synchrony with depressing synapses. Neural Computation, 10, 815–819. Stratford, K., Tarczy-Hornoch, K., Martin, K., Bannister, N., & Jack, J. (1998). Excitatory synaptic inputs to spiny stellate cells in cat visual cortex. Nature, 382, 258–261. Tsodyks, M., & Markram, H. (1997). The neural code between neocortical pyramidal neurons depends on neurotransmitter release probability. Proc. Natl. Acad. Sci. USA, 94(2), 719–723.

348

M. Boegerhausen, P. Suter, and S. Liu

Tsodyks, M., Pawelzik, K., & Markram, H. (1998). Neural networks with dynamic synapses. Neural Computation, 10, 821–835. Van Schaik, A. (2001). Building blocks for electronic spiking neural networks. Neural Networks, 14(6/7), 617–628. Varela, J., Sen, K., Gibson, J., Fost, J., Abbott, L., and Nelson, S. (1997). A quantitative description of short-term plasticity at excitatory synapses in layer 2/3 of rat primary visual cortex. Journal of Neuroscience, 17, 7926–7940. Received June 14, 2002; accepted July 24, 2002.

LETTER

Communicated by Hagai Attias

Dictionary Learning Algorithms for Sparse Representation Kenneth Kreutz-Delgado [email protected] Joseph F. Murray [email protected] Bhaskar D. Rao [email protected] Electrical and Computer Engineering, Jacobs School of Engineering, University of California, San Diego, La Jolla, California 92093-0407, U.S.A. Kjersti Engan [email protected] Stavanger University College, School of Science and Technology Ullandhaug, N-4091 Stavanger, Norway Te-Won Lee [email protected] Terrence J. Sejnowski [email protected] Howard Hughes Medical Institute, Computational Neurobiology Laboratory, Salk Institute, La Jolla, California 92037, U.S.A. Algorithms for data-driven learning of domain-speciŽc overcomplete dictionaries are developed to obtain maximum likelihood and maximum a posteriori dictionary estimates based on the use of Bayesian models with concave/Schur-concave (CSC) negative log priors. Such priors are appropriate for obtaining sparse representations of environmental signals within an appropriately chosen (environmentally matched) dictionary. The elements of the dictionary can be interpreted as concepts, features, or words capable of succinct expression of events encountered in the environment (the source of the measured signals). This is a generalization of vector quantization in that one is interested in a description involving a few dictionary entries (the proverbial “25 words or less”), but not necessarily as succinct as one entry. To learn an environmentally adapted dictionary capable of concise expression of signals generated by the environment, we develop algorithms that iterate between a representative set of sparse representations found by variants of FOCUSS and an update of the dictionary using these sparse representations. Experiments were performed using synthetic data and natural images. For complete dictionaries, we demonstrate that our algorithms have imNeural Computation 15, 349–396 (2003)

° c 2002 Massachusetts Institute of Technology

350

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

proved performance over other independent component analysis (ICA) methods, measured in terms of signal-to-noise ratios of separated sources. In the overcomplete case, we show that the true underlying dictionary and sparse sources can be accurately recovered. In tests with natural images, learned overcomplete dictionaries are shown to have higher coding efŽciency than complete dictionaries; that is, images encoded with an overcomplete dictionary have both higher compression (fewer bits per pixel) and higher accuracy (lower mean square error). 1 Introduction

FOCUSS, which stands for FOCal Underdetermined System Solver, is an algorithm designed to obtain suboptimally (and, at times, maximally) sparse solutions to the following m £ n, underdetermined linear inverse problem1 (Gorodnitsky, George, & Rao, 1995; Rao & Gorodnitsky, 1997; Gorodnitsky & Rao, 1997; Adler, Rao, & Kreutz-Delgado, 1996; Rao & Kreutz-Delgado, 1997; Rao, 1997, 1998), y D Ax,

(1.1)

for known A. The sparsity of a vector is the number of zero-valued elements (Donoho, 1994), and is related to the diversity, the number of nonzero elements, sparsity D #fx[i] D 0g 6 0g diversity D #fx[i] D

diversity D n ¡ sparsity. Since our initial investigations into the properties of FOCUSS as an algorithm for providing sparse solutions to linear inverse problems in relatively noise-free environments (Gorodnitsky et al., 1995; Rao, 1997; Rao & Gorodnitsky, 1997; Gorodnitsky & Rao, 1997; Adler et al., 1996; Rao & Kreutz-Delgado, 1997), we now better understand the behavior of FOCUSS in noisy environments (Rao & Kreutz-Delgado, 1998a, 1998b) and as an interior point-like optimization algorithm for optimizing concave functionals subject to linear constraints (Rao & Kreutz-Delgado, 1999; Kreutz-Delgado & Rao, 1997, 1998a, 1998b, 1998c, 1999; Kreutz-Delgado, Rao, Engan, Lee, & Sejnowski, 1999a; Engan, Rao, & Kreutz-Delgado, 2000; Rao, Engan, Cotter, & Kreutz-Delgado, 2002). In this article, we consider the use of the FOCUSS algorithm in the case where the matrix A is unknown and must be learned. Toward this end, we Žrst briey discuss how the use of concave (and 1

For notational simplicity, we consider the real case only.

Dictionary Learning Algorithms for Sparse Representation

351

Schur concave) functionals enforces sparse solutions to equation 1.1. We also discuss the choice of the matrix, A, in equation 1.1 and its relationship to the set of signal vectors y for which we hope to obtain sparse representations. Finally, we present algorithms capable of learning an environmentally adapted dictionary, A, given a sufŽciently large and statistically representative sample of signal vectors, y, building on ideas originally presented in Kreutz-Delgado, Rao, Engan, Lee, and Sejnowski (1999b), Kreutz-Delgado, Rao, and Engan (1999), and Engan, Rao, and Kreutz-Delgado (1999). We refer to the columns of the full row-rank m £ n matrix A, A D [a1 , . . . , an ] 2 Rm£n ,

n À m,

(1.2)

as a dictionary, and they are assumed to be a set of vectors capable of providing a highly succinct representation for most (and, ideally, all) statistically representative signal vectors y 2 Rm . Note that with the assumption that rank(A) D m, every vector y has a representation; the question at hand is whether this representation is likely to be sparse. We call the statistical generating mechanism for signals, y, the environment and a dictionary, A, within which such signals can be sparsely represented an environmentally adapted dictionary. Environmentally generated signals typically have signiŽcant statistical structure and can be represented by a set of basis vectors spanning a lowerdimensional submanifold of meaningful signals (Field, 1994; Ruderman, 1994). These environmentally meaningful representation vectors can be obtained by maximizing the mutual information between the set of these vectors (the dictionary) and the signals generated by the environment (Comon, 1994; Bell & Sejnowski, 1995; Deco & Obradovic, 1996; Olshausen & Field, 1996; Zhu, Wu, & Mumford, 1997; Wang, Lee, & Juang, 1997). This procedure can be viewed as a natural generalization of independent component analysis (ICA) (Comon, 1994; Deco & Obradovic, 1996). As initially developed, this procedure usually results in obtaining a minimal spanning set of linearly independent vectors (i.e., a true basis). More recently, the desirability of obtaining “overcomplete” sets of vectors (or “dictionaries”) has been noted (Olshausen & Field, 1996; Lewicki & Sejnowski, 2000; Coifman & Wickerhauser, 1992; Mallat & Zhang, 1993; Donoho, 1994; Rao & KreutzDelgado, 1997). For example, projecting measured noisy signals onto the signal submanifold spanned by a set of dictionary vectors results in noise reduction and data compression (Donoho, 1994, 1995). These dictionaries can be structured as a set of bases from which a single basis is to be selected to represent the measured signal(s) of interest (Coifman & Wickerhauser, 1992) or as a single, overcomplete set of individual vectors from within which a vector, y, is to be sparsely represented (Mallat & Zhang, 1993; Olshausen & Field, 1996; Lewicki & Sejnowski, 2000; Rao & Kreutz-Delgado, 1997). The problem of determining a representation from a full row-rank overcomplete dictionary, A D [a1 , . . . , an ], n À m, for a speciŽc signal mea-

352

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

surement, y, is equivalent to solving an underdetermined inverse problem, Ax D y, which is nonuniquely solvable for any y. The standard least-squares solution to this problem has the (at times) undesirable feature of involving all the dictionary vectors in the solution2 (the “spurious artifact” problem) and does not generally allow for the extraction of a categorically or physically meaningful solution. That is, it is not generally the case that a least-squares solution yields a concise representation allowing for a precise semantic meaning.3 If the dictionary is large and rich enough in representational power, a measured signal can be matched to a very few (perhaps even just one) dictionary words. In this manner, we can obtain concise semantic content about objects or situations encountered in natural environments (Field, 1994). Thus, there has been signiŽcant interest in Žnding sparse solutions, x (solutions having a minimum number of nonzero elements), to the signal representation problem. Interestingly, matching a speciŽc signal to a sparse set of dictionary words or vectors can be related to entropy minimization as a means of elucidating statistical structure (Watanabe, 1981). Finding a sparse representation (based on the use of a “few” code or dictionary words) can also be viewed as a generalization of vector quantization where a match to a single “code vector” (word) is always sought (taking “code book” = “dictionary”).4 Indeed, we can refer to a sparse solution, x, as a sparse coding of the signal instantiation, y. 1.1 Stochastic Models. It is well known (Basilevsky, 1994) that the sto-

chastic generative model (1.3)

y D Ax C º, m

can be used to develop algorithms enabling coding of y 2 R via solving the inverse problem for a sparse solution x 2 Rn for the undercomplete (n < m) and complete (n D m) cases. In recent years, there has been a great deal of interest in obtaining sparse codings of y with this procedure for the overcomplete (n > m) case (Mallat & Zhang, 1993; Field, 1994). In our earlier work, we have shown that given an overcomplete dictionary, A (with the columns of A comprising the dictionary vectors), a maximum a posteriori (MAP) estimate of the source vector, x, will yield a sparse coding of y in the low-noise limit if the negative log prior, ¡ log(P(x) ) , is concave/Schurconcave (CSC) (Rao, 1998; Kreutz-Delgado & Rao, 1999), as discussed below. 2 This fact comes as no surprise when the solution is interpreted within a Bayesian framework using a gaussian (maximum entropy) prior. 3 Taking “semantic” here to mean categorically or physically interpretable. 4 For example, n D 100 corresponds to 100 features encoded via vector quantization (“one column = one concept”). features using up to four ³ ´ If³we are ´ allowed ³ ´ to³represent ´ 100 100 100 100 C C C D 4,087,975concepts showing columns, we can encode 1 2 3 4

a combinatorial boost in expressive power.

Dictionary Learning Algorithms for Sparse Representation

353

For P (x ) factorizable into a product of marginal probabilities, the resulting code is also known to provide an independent component analysis (ICA) representation of y. More generally, a CSC prior results in a sparse representation even in the nonfactorizable case (with x then forming a dependent component analysis, or DCA, representation). Given independently and identically distributed (i.i.d.) data, Y D YN D ( y1 , . . . , yN ) , assumed to be generated by the model 1.3, a maximum likelibML , of the unknown (but nonrandom) dictionary A can be hood estimate, A determined as (Olshausen & Field, 1996; Lewicki & Sejnowski, 2000) bML D arg max P (YI A ) . A A

This requires integrating out the unobservable i.i.d. source vectors, X D XN D ( x1 , . . . , xN ) , in order to compute P ( YI A ) from the (assumed) known probabilities P ( x) and P (º). In essence, X is formally treated as a set of nuisance parameters that in principle can be removed by integration. However, because the prior P ( x ) is generally taken to be supergaussian, this integration is intractable or computationally unreasonable. Thus, approximations to this integration are performed that result in an approximation to P (YI A ), which is then maximized with respect to Y. A new, better approximation to the integration can then be made, and this process is iterated until the estimate of the dictionary A has (hopefully) converged (Olshausen & Field, 1996). We refer to the resulting estimate as an approximate maxibAML ). mum likelihood (AML) estimate of the dictionary A (denoted here by A No formal proof of the convergence of this algorithm to the true maxibML, has been given in the prior literature, but mum likelihood estimate, A it appears to perform well in various test cases (Olshausen & Field, 1996). Below, we discuss the problem of dictionary learning within the framework of our recently developed log-prior model-based sparse source vector learning approach that for a known overcomplete dictionary can be used to obtain sparse codes (Rao, 1998; Kreutz-Delgado & Rao, 1997, 1998b, 1998c, 1999; Rao & Kreutz-Delgado, 1999). Such sparse codes can be found using FOCUSS, an afŽne scaling transformation (AST)–like iterative algorithm that Žnds a sparse locally optimal MAP estimate of the source vector x for an observation y. Using these results, we can develop dictionary learning algorithms within the AML framework and for obtaining a MAP-like bMAP , of the (now assumed random) dictionary, A, assuming in estimate, A the latter case that the dictionary belongs to a compact submanifold corresponding to unit Frobenius norm. Under certain conditions, convergence to a local minimum of a MAP-loss function that combines functions of the discrepancy e D ( y ¡ Ax ) and the degree of sparsity in x can be rigorously proved. 1.2 Related Work. Previous work includes efforts to solve equation 1.3 in the overcomplete case within the maximum likelihood (ML) framework.

354

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

An algorithm for Žnding sparse codes was developed in Olshausen and Field (1997) and tested on small patches of natural images, resulting in Gabor-like receptive Želds. In Lewicki and Sejnowski (2000) another ML algorithm is presented, which uses the Laplacian prior to enforce sparsity. The values of the elements of x are found with a modiŽed conjugate gradient optimization (which has a rather complicated implementation) as opposed to the standard ICA (square mixing matrix) case where the coefŽcients are found by inverting the A matrix. The difŽculty that arises when using ML is that Žnding the estimate of the dictionary A requires integrating over all possible values of the joint density P ( y, xI A ) as a function of x. In Olshausen and Field (1997), this is handled by assuming the prior density of x is a delta function, while in Lewicki and Sejnowski (2000), it is approximated by a gaussian. The Žxed-point FastICA (Hyv¨arinen, Cristescu, & Oja, 1999) has also been extended to generate overcomplete representations. The FastICA algorithm can Žnd the basis functions (columns of the dictionary A) one at a time by imposing a quasi-orthogonality condition and can be thought of as a “greedy” algorithm. It also can be run in parallel, meaning that all columns of A are updated together. Other methods to solve equation 1.3 in the overcomplete case have been developed using a combination of the expectation-maximization (EM) algorithm and variational approximation techniques. Independent factor analysis (Attias, 1999) uses a mixture of gaussians to approximate the prior density of the sources, which avoids the difŽculty of integrating out the parameters X and allows different sources to have different densities. In another method (Girolami, 2001), the source priors are assumed to be supergaussian (heavy-tailed), and a variational lower bound is developed that is used in the EM estimation of the parameters A and X. It is noted in Girolami (2001) that the mixtures used in independent factor analysis are more general than may be needed for the sparse overcomplete case, and they can be computationally expensive as the dimension of the data vector and number of mixtures increases. In Zibulevsky and Pearlmutter (2001), the blind source separation problem is formulated in terms of a sparse source underlying each unmixed signal. These sparse sources are expanded into the unmixed signal with a predeŽned wavelet dictionary, which may be overcomplete. The unmixed signals are linearly combined by a different mixing matrix to create the observed sensor signals. The method is shown to give better separation performance than ICA techniques. The use of learned dictionaries (instead of being chosen a priori) is suggested.

2 FOCUSS: Sparse Solutions for Known Dictionaries 2.1 Known Dictionary Model. A Bayesian interpretation is obtained from the generative signal model, equation 1.3, by assuming that x has the

Dictionary Learning Algorithms for Sparse Representation

355

parameterized (generally nongaussian) probability density function (pdf), Z Pp ( x) D Zp¡1 e ¡c p dp ( x ) ,

Zp D

e ¡c p dp ( x ) dx,

(2.1)

with parameter vector p. Similarly, the noise º is assumed to have a parameterized (possibly nongaussian) density P q (º) of the same form as equation 2.1 with parameter vector q. It is assumed that x and º have zero means and that their densities obey the property d ( x ) D d (| x| ) , for | ¢ | deŽned component-wise. This is equivalent to assuming that the densities are symmetric with respect to sign changes in the components x, x[i] à ¡x[i] and therefore that the skews of these densities are zero. We also assume that d (0) D 0. With a slight abuse of notation, we allow the differing subscripts q and p to indicate that dq and dp may be functionally different as well as parametrically different. We refer to densities like equation 2.1 for suitable additional constraints on dp (x ) , as hypergeneralized gaussian distributions (Kreutz-Delgado & Rao, 1999; Kreutz-Delgado et al., 1999). If we treat A, p, and q as known parameters, then x and y are jointly distributed as P (x, y ) D P ( x, yI p, q, A ) . Bayes’ rule yields P (x | yI p, A ) D

1 1 P ( y | xI p, A ) ¢ P ( xI p, A ) D Pq ( y ¡ Ax ) ¢ Pp ( x ) (2.2) b b Z

b D P ( y ) D P ( yI p, q, A ) D

P ( y | x) ¢ Pp ( x) dx.

(2.3)

Usually the dependence on p and q is notationally suppressed, for example, b D P ( yI A) . Given an observation, y, maximizing equation 2.2 with respect to x yields a MAP estimate b x. This ideally results in a sparse coding of the observation, a requirement that places functional constraints on the pdfs, and particularly on dp . Note that b is independent of x and can be ignored when optimizing equation 2.2 with respect to the unknown source vector x. The MAP estimate equivalently is obtained from minimizing the negative logarithm of P ( x | y ), which is b x D arg min dq ( y ¡ Ax) C ldp ( x) , x

(2.4)

where l D c p /c q and dq ( y¡Ax ) D dq ( Ax¡y ) by our assumption of symmetry. The quantity l1 is interpretable as a signal-to-noise ratio (SNR). Furthermore, assuming that both dq and dp are concave/Schur–concave (CSC) as deŽned in section 2.4, then the term dq ( y ¡ Ax) in equation 2.4 will encourage sparse

356

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

residuals, e D y ¡ Ab x, while the term dp ( x ) encourages sparse source vector estimates, b x. A given value of l then determines a trade-off between residual and source vector sparseness. This most general formulation will not be used here. Although we are interested in obtaining sparse source vector estimates, we will not enforce sparsity on the residuals but instead, to simplify the development, will assume the q D 2 i.i.d. gaussian measurement noise case (º gaussian with known covariance s 2 ¢ I), which corresponds to taking, bx ) D c q dq ( y ¡ Ab

1 bxk2 . ky ¡ Ab 2s 2

(2.5)

In this case, problem 2.4 becomes 1 b x D arg min ky ¡ Axk2 C ldp ( x ). x 2

(2.6)

In either case, we note that l ! 0 as c p ! 0 which (consistent with the generative model, 1.3) we refer to as the low noise limit. Because the mapping A is assumed to be onto, in the low-noise limit, the optimization, equation 2.4, is equivalent to the linearly constrained problem, b x D arg min dp (x )

subject to

Ax D y.

(2.7)

In the low-noise limit, no sparseness constraint need be placed on the residuals, e D y ¡ Ab x, which are assumed to be zero. It is evident that the structure of dp (¢) is critical for obtaining a sparse coding, b x, of the observation y (Kreutz-Delgado & Rao, 1997; Rao & Kreutz-Delgado, 1999). Throughtout this article, the quantity dp ( x ) is always assumed to be CSC (enforcing sparse solutions to the inverse problem 1.3). As noted, and as will be evident during the development of dictionary learning algorithms below, we do not impose a sparsity constraint on the residuals; instead, the measurement noise º will be assumed to be gaussian (q D 2). 2.2 Independent Component Analysis and Sparsity Inducing Priors.

An important class of densities is given by the generalized gaussians for which p

dp ( x ) D kxkp D

n X

|x[k]| p ,

(2.8)

kD 1

for p > 0 (Kassam, 1982). This is a special case of the larger `p class (the p-class) of functions, which allows p to be negative in value (Rao & KreutzDelgado, 1999; Kreutz-Delgado & Rao, 1997). Note that this function has the special property of separability, dp ( x ) D

n X kD 1

dp ( x[k]) ,

Dictionary Learning Algorithms for Sparse Representation

357

which corresponds to factorizability of the density Pp ( x ) , Pp ( x) D

n Y

Pp ( x[k]) ,

kD 1

and hence to independence of the components of x. The assumption of independent components allows the problem of solving the generative model, equation 1.3, for x to be interpreted as an ICA problem (Comon, 1994; Pham, 1996; Olshausen & Field, 1996; Roberts, 1998). It is of interest, then, to consider the development of a large class of parameterizable separable functions dp ( x) consistent with the ICA assumption (Rao & Kreutz-Delgado, 1999; KreutzDelgado & Rao, 1997). Given such a class, it is natural to examine the issue of Žnding a best Žt within this class to the “true” underlying prior density of x. This is a problem of parametric density estimation of the true prior, where one attempts to Žnd an optimal choice of the model density Pp ( x ) by an optimization over the parameters p that deŽne the choice of a prior from within the class. This is, in general, a difŽcult problem, which may require the use of Monte Carlo, evolutionary programming, or stochastic search techniques. Can the belief that supergaussian priors, Pp ( x ), are appropriate for Žnding sparse solutions to equation 1.3 (Field, 1994; Olshausen & Field, 1996) be clariŽed or made rigorous? It is well known that the generalized gaussian distribution arising from the use of equation 2.8 yields supergaussian distributions (positive kurtosis) for p < 2 and subgaussian (negative kurtosis) for p > 2. However, one can argue (see section 2.5 below) that the condition for obtaining sparse solutions in the low-noise limit is the stronger requirement that p · 1, in which case the separable function dp (x ) is CSC. This indicates that supergaussianness (positive kurtosis) alone is necessary but not sufŽcient for inducing sparse solutions. Rather, sufŽciency is given by the requirement that ¡ log Pp ( x ) ¼ dp ( x) be CSC. We have seen that the function dp (x ) has an interpretation as a (negative logarithm of) a Bayesian prior or as a penalty function enforcing sparsity in equation 2.4, where dp ( x ) should serve as a “relaxed counting function” on the nonzero elements of x. Our perspective emphasizes that dp ( x ) serves both of these goals simultaneously. Thus, good regularizing functions, dp ( x ), should be exibly parameterizable so that Pp ( x) can be optimized over the parameter vector p to provide a good parametric Žt to the underlying environmental pdf, and the functions should also have analytical properties consistent with the goal of enforcing sparse solutions. Such properties are discussed in the next section. 2.3 Majorization and Schur-Concavity. In this section, we discuss functions that are both concave and Schur-concave (CSC functions; Marshall & Olkin, 1979). We will call functions, dp (¢) , which are CSC, diversity functions, anticoncentration functions or antisparsity functions. The larger the value of the

358

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

CSC function dp (x ) , the more diverse (i.e., the less concentrated or sparse) the elements of the vector x are. Thus, minimizing dp ( x ) with respect to x results in less diverse (more concentrated or sparse) vectors x. 2.3.1 Schur-Concave Functions. A measure of the sparsity of the elements of a solution vector x (or the lack thereof, which we refer to as the diversity of x) is given by a partial ordering on vectors known as the Lorentz order. For any vector in the positive orthant, x 2 RnC , deŽne the decreasing rearrangement . x D ( xb1c , . . . , xbnc ) , xb1c ¸ ¢ ¢ ¢ ¸ xbnc ¸ 0 and the partial sums (Marshall & Olkin, 1979; Wickerhauser, 1994), Sx [k] D

k X iD 1

xbnc ,

k D 1, . . . , n.

We say that y majorizes x, y  x, iff for k D 1, . . . , n, S y [k] ¸ Sx [k]I

S y [n] D Sx [n],

and the vector y is said to be more concentrated, or less diverse, than x. This partial order deŽned by majorization then deŽnes the Lorentz order. We are interested in scalar-valued functions of x that are consistent with majorization. Such functions are known as Schur-concave functions, d (¢) : RnC ! R. They are deŽned to be precisely the class of functions consistent with the Lorentz order, yÂx

)

d ( y ) < d ( x ).

In words, if y is less diverse than x (according to the Lorentz order) then d ( y) is less than d ( x) for d (¢) Schur-concave. We assume that Schur-concavity is a necessary condition for d (¢) to be a good measure of diversity (antisparsity). 2.3.2 Concavity Yields Sparse Solutions. Recall that a function d (¢) is concave on the positive orthant RnC iff (Rockafellar, 1970) d ( (1 ¡ c ) x C c y ) ¸ (1 ¡ c ) d (x ) C c d ( y ) , 8x, y 2 RnC , 8c , 0 · c · 1. In addition, a scalar function is said to be permutation invariant if its value is independent of rearrangements of its components. An important fact is that for permutation invariant functions, concavity is a sufŽcient condition for Schur-concavity: Concavity C permutation invariance ) Schur-concavity. Now consider the low-noise sparse inverse problem, 2.7. It is well known that subject to linear constraints, a concave function on RnC takes its minima

Dictionary Learning Algorithms for Sparse Representation

359

on the boundary of RnC (Rockafellar, 1970), and as a consequence these minima are therefore sparse. We take concavity to be a sufŽcient condition for a permutation invariant d (¢) to be a measure of diversity and obtain sparsity as constrained minima of d (¢) . More generally, a diversity measure should be somewhere between Schur-concave and concave. In this spirit, one can deŽne almost concave functions (Kreutz-Delgado & Rao, 1997), which are Schur-concave and (locally) concave in all n directions but one, which also are good measures of diversity. 2.3.3 Separability, Schur-Concavity, and ICA. The simplest way to ensure that d ( x ) be permutation invariant (a necessary condition for Schurconcavity) is to use functions that are separable. Recall that separability of dp ( x) corresponds to factorizability of Pp (x ) . Thus, separability of d ( x ) corresponds to the assumption of independent components of x under the model 1.3. We see that from a Bayesian perspective, separability of d ( x) corresponds to a generative model for y that assumes a source, x, with independent components. With this assumption, we are working within the framework of ICA (Nadal & Parga, 1994; Pham, 1996; Roberts, 1998). We have developed effective algorithms for solving the optimization problem 2.7 for sparse solutions when dp (x ) is separable and concave (Kreutz-Delgado & Rao, 1997; Rao & Kreutz-Delgado, 1999). It is now evident that relaxing the restriction of separability generalizes the generative model to the case where the source vector, x, has dependent components. We can reasonably call an approach based on a nonseparable diversity measure d ( x) a dependent component analysis (DCA). Unless care is taken, this relaxation can signiŽcantly complicate the analysis and development of optimization algorithms. However, one can solve the lownoise DCA problem, at least in principle, provided appropriate choices of nonseparable diversity functions are made. 2.4 Supergaussian Priors and Sparse Coding. The P-class of diversity measures for 0 < p · 1 result in sparse solutions to the low-noise coding problem, 2.7. These separable and concave (and thus Schur-concave) diversity measures correspond to supergaussian priors, consistent with the folk theorem that supergaussian priors are sparsity-enforcing priors. However, taking 1 · p < 2 results in supergaussian priors that are not sparsity enforcing. Taking p to be between 1 and 2 yields a dp ( x) that is convex and therefore not concave. This is consistent with the well-known fact that for this range of p, the pth-root of dp ( x ) is a norm. Minimizing dp (x ) in this case drives x toward the origin, favoring concentrated rather than sparse solutions. We see that if a sparse coding is to be found based on obtaining a MAP estimate to the low-noise generative model, 1.3, then, in a sense, supergaussianness is a necessary but not sufŽcient condition for a prior to be sparsity enforcing. A sufŽcient condition for obtaining a sparse MAP coding is that the negative log prior be CSC.

360

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

2.5 The FOCUSS Algorithm. Locally optimal solutions to the knowndictionary sparse inverse problems in gaussian noise, equations 2.6 and 2.7, are given by the FOCUSS algorithm. This is an afŽne-scaling transformation (AST)-like (interior point) algorithm originally proposed for the low-noise case 2.7 in Rao and Kreutz-Delgado (1997, 1999) and Kreutz-Delgado and Rao (1997); and extended by regularization to the nontrivial noise case, equation 2.6, in Rao and Kreutz-Delgado (1998a), Engan et al. (2000), and Rao et al. (2002). In these references, it is shown that the FOCUSS algorithm has excellent behavior for concave functions (which includes the the CSC concentration functions) dp (¢) . For such functions, FOCUSS quickly converges to a local minimum, yielding a sparse solution to problems 2.7 and 2.6. One can quickly motivate the development of the FOCUSS algorithm appropriate for solving the optimization problem 2.6 by considering the problem of obtaining the stationary points of the objective function. These are given as solutions, x¤ , to

AT ( Ax¤ ¡ y ) C lrx dp ( x¤ ) D 0.

(2.9)

In general, equation 2.9 is nonlinear and cannot be explicitly solved for a solution x¤ . However, we proceed by assuming the existence of a gradient factorization,

rx dp (x ) D a(x) P ( x ) x,

(2.10)

where a(x) is a positive scalar function and P (x ) is symmetric, positivedeŽnite, and diagonal. As discussed in Kreutz-Delgado and Rao (1997, 1998c) and Rao and Kreutz-Delgado (1999), this assumption is generally true for CSC sparsity functions dp (¢) and is key to understanding FOCUSS as a sparsity-inducing interior-point (AST-like) optimization algorithm.5 With the gradient factorization 2.10, the stationary points of equation 2.9 are readily shown to be solutions to the (equally nonlinear and implicit) system, x¤ D ( AT A C b ( x¤ )P ( x¤ ) ) ¡1 AT y D P

¡1

¤

(x ) A

T (b ( ¤ )

x I C AP

¡1

(2.11) ¤

T ) ¡1

(x ) A

y,

(2.12)

where b ( x ) D la(x) and the second equation follows from identity A.18. Although equation 2.12 is also not generally solvable in closed form, it does 5 This interpretation, which is not elaborated on here, follows from deŽning a diagonal positive-deŽnite afŽne scaling transformation matrix W (x) by the relation

P (x) D W ¡2 (x).

Dictionary Learning Algorithms for Sparse Representation

361

suggest the following relaxation algorithm, b x Ã

P ¡1 (b x ) AT (b (b x ) I C AP ¡1 (b x) AT ) ¡1 y,

(2.13)

which is to be repeatedly reiterated until convergence. Taking b ´ 0 in equation 2.13 yields the FOCUSS algorithm proved in Kreutz-Delgado and Rao (1997, 1998c) and Rao and Kreutz-Delgado (1999) to converge to a sparse solution of equation 2.7 for CSC sparsity functions 6 0 yields the regularized FOCUSS algorithm that will dp (¢). The case b D converge to a sparse solution of equation 2.6 (Rao, 1998; Engan et al., 2000; Rao et al., 2002). More computationally robust variants of equation 2.13 are discussed elsewhere (Gorodnitsky & Rao, 1997; Rao & Kreutz-Delgado, 1998a). Note that for the general regularized FOCUSS algorithm, 2.13, we have b (b x) D la(b x) , where l is the regularization parameter in equation 2.4. The function b ( x) is usually generalized to be a function of b x, y and the iteration number. Methods for choosing l include the quality-of-Žt criteria, the sparsity critera, and the L-curve (Engan, 2000; Engan et al., 2000; Rao et al., 2002). The quality-of-Žt criterion attempts to minimize the residual error y ¡ Ax (Rao, 1997), which can be shown to converge to a sparse solution (Rao & Kreutz-Delgado, 1999). The sparsity criterion requires that a certain number of elements of each x k be nonzero. The L-curve method adjusts l to optimize the trade-off between the residual and sparsity of x. The plot of dp (x ) versus dq ( y ¡ Ax) has an L shape, the corner of which provides the best trade-off. The corner of the L-curve is the point of maximum curvature and can be found by a one-dimensional maximization of the curvature function (Hansen & O’Leary, 1993). A hybrid approach known as the modiŽed L-curve method combines the Lcurve method on a linear scale and the quality-of-Žt criterion, which is used to place limits on the range of l that can be chosen by the L-curve (Engan, 2000). The modiŽed L-curve method was shown to have good performance, but it requires a one-dimensional numerical optimization step for each xk at each iteration, which can be computationally expensive for large vectors. 3 Dictionary Learning 3.1 Unknown, Nonrandom Dictionaries. The MLE framework treats parameters to be estimated as unknown but deterministic (nonrandom). In this spirit, we take the dictionary, A, to be the set of unknown but deterministic parameters to be estimated from the observation set Y D YN . In particular, bML is found from maximizing given YN , the maximum likelihood estimate A the likelihood function L ( A | YN ) D P (YN I A) . Under the assumption that

362

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

the observations are i.i.d., this corresponds to the optimization bML D arg max A A

N Y kD 1

P ( y kI A ) ,

Z P ( yk I A ) D

(3.1)

Z P ( y k, xI A) dx D

Z

P ( y k | xI A ) ¢ Pp ( x ) dx

Pq ( yk ¡ Ax ) ¢ Pp (x ) dx.

D

(3.2)

DeŽning the sample average of a function f ( y ) over the sample set YN D ( y1 , . . . , yN ) by h f ( y) iN D

N 1 X f ( yk ) , N kD 1

the optimization 3.1 can be equivalently written as bML D arg min ¡hlog(P ( yI A) ) iN . A A

(3.3)

Note that P ( y kI A ) is equal to the normalization factor b already encountered, but now with the dependence of b on A and the particular sample, y k, made explicit. The integration in equation 3.2 in general is intractable, and various approximations have been proposed to obtain an approximate bAML (Olshausen & Field, 1996; Lewicki & maximum likelihood estimate, A Sejnowski, 2000). In particular, the following approximation has been proposed (Olshausen & Field, 1996), b) ) , Pp (x ) ¼ d (x ¡ b xk ( A

(3.4)

where b) D arg max P ( yk , xI A b) , b xk ( A x

(3.5)

b for A. This approximation for k D 1, . . . , N, assuming a current estimate, A, corresponds to assuming that the source vector x k for which y k D Ax k is b) . With this approximation, the optimization 3.3 known and equal to b xk ( A becomes bAML D arg minhdq ( y ¡ Ab A x) N , A

(3.6)

Dictionary Learning Algorithms for Sparse Representation

363

which is an optimization over the sample average h¢iN of the functional 2.4 encountered earlier. Updating our estimate for the dictionary, bà A bAML , A

(3.7)

bAML has converged, hopefully we can iterate the procedure (3.5)–(3.6) until A bML D A bML (YN ) as the maximum likelihood (at least in the limit of large N) to A bML ( YN ) has well-known desirable asymptotic properties in the estimate A limit N ! 1. Performing the optimization in equation 3.6 for the q D 2 i.i.d. gaussian measurement noise case (º gaussian with known covariance s 2 ¢ I) corresponds to taking bb dq ( y ¡ A x) D

1 bb ky ¡ A xk2 , 2s 2

(3.8)

in equation 3.6. In appendix A, it is shown that we can readily obtain the unique batch solution, bAML D SyxO S ¡1 , A Ox xO

SyxO

D

N 1 X ykb xTk , N kD 1

(3.9)

SxO xO

D

N 1 X b x kb xTk . N kD 1

(3.10)

Appendix A derives the maximum likelihood estimate of A for the ideal case of known source vectors X D ( x1 , . . . , xN ) , ¡1 Known source vector case: AML D Syx Sxx ,

which is, of course, actually not computable since the actual source vectors are assumed to be hidden. As an alternative to using the explicit solution 3.9, which requires an often prohibitive n £ n inversion, we can obtain AAML iteratively by gradient descent on equations 3.6 and 3.8, N X bAML Ã A bAML ¡ m 1 A e kb xTk , N kD 1

bAMLb ek D A x k ¡ y k,

(3.11)

k D 1, . . . , N,

for an appropriate choice of the (possibly adaptive) positive step-size pabAML D A. b rameter m . The iteration 3.11 can be initialized as A A general iterative dictionary learning procedure is obtained by nesting the iteration 3.11 entirely within the iteration deŽned by repeatedly solvbAML , of the dictionary becomes ing equation 3.5 every time a new estimate, A

364

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

available. However, performing the optimization required in equation 3.5 is generally nontrivial (Olshausen & Field, 1996; Lewicki & Sejnowski, 2000). Recently, we have shown how the use of the FOCUSS algorithm results in an effective algorithm for performing the optimization required in equation 3.5 for the case when º is gaussian (Rao & Kreutz-Delgado, 1998a; Engan et al., 1999). This approach solves equation 3.5 using the afŽne-scaling transformation (AST)-like algorithms recently proposed for the low-noise case (Rao & Kreutz-Delgado, 1997, 1999; Kreutz-Delgado & Rao, 1997) and extended by regularization to the nontrivial noise case (Rao & Kreutz-Delgado, 1998a; Engan et al., 1999). As discussed in section 2.5, for the current dictionary b a solution to the optimization problem, 3.5, is provided by the estimate, A, repeated iteration, bT (b (b b ¡1 (b bT ) ¡1 y k, b x k à P ¡1 (b xk ) A x k ) I C AP xk ) A

(3.12)

k D 1, . . . , N, with P ( x ) deŽned as in equation 3.18, given below. This is the regularized FOCUSS algorithm (Rao, 1998; Engan et al., 1999), which has an interpretation as an AST-like concave function minimization algorithm. The proposed dictionary learning algorithm alternates between iteration 3.12 and iteration 3.11 (or the direct batch solution given by equation 3.9 if the inversion is tractable). Extensive simulations show the ability of the AST-based algorithm to recover an unknown 20 £ 30 dictionary matrix A completely (Engan et al., 1999). 3.2 Unknown, Random Dictionaries. We now generalize to the case where the dictionary A and the source vector set X D XN D ( x1 , . . . , xN ) are jointly random and unknown. We add the requirement that the dictionary is known to obey the constraint

A 2 A D compact submanifold of Rm£n . A compact submanifold of Rm£n is necessarily closed and bounded. On the constraint submanifold, the dictionary A has the prior probability density function P ( A) , which in the sequel we assume has the simple (uniform on A) form, ( A 2 A) ,

P ( A ) D cX

(3.13)

where X (¢) is the indicator function and c is a positive constant chosen to ensure that Z P (A) D P ( A ) dA D 1. A

The dictionary A and the elements of the set X are also all assumed to be mutually independent, P ( A, X ) D P ( A) P ( X ) D P ( A) Pp ( x1 ) , . . . , Pp ( xN ).

Dictionary Learning Algorithms for Sparse Representation

365

With the set of i.i.d. noise vectors, (º1 , . . . , ºN ) also taken to be jointly random with, and independent of, A and X, the observation set Y D YN D ( y1 , . . . , yN ) is assumed to be generated by the model 1.3. With these assumptions, we have P ( A, X | Y) D P (Y | A, X ) P ( A, X ) / P (Y ) D cX

(3.14)

( A 2 A ) P ( Y | A, X ) P ( X ) / P ( Y)

D

N cX ( A 2 A ) Y P ( y k | A, x k ) Pp ( xk ) P (Y ) kD 1

D

N cX ( A 2 A ) Y Pq ( y ¡ Ax k ) Pp ( x k ) , P (Y ) kD 1

using the facts that the observations are conditionally independent and P ( y k | A, X ) D P ( yk | A, xk ) . The jointly MAP estimates bMAP , X bMAP ) D ( A bMAP , b (A x1, MAP , . . . , b xN, MAP ) are found by maximizing a posteriori probability density P ( A, X | Y) simultaneously with respect to A 2 A and X. This is equivalent to minimizing the negative logarithm of P ( A, X | Y ), yielding the optimization problem, bMAP , X bMAP ) D arg min hdq ( y ¡ Ax) C ldp ( x) iN . (A A2A,X

(3.15)

Note that this is a joint minimization of the sample average of the functional 2.4, and as such is a natural generalization of the single (with respect to the set of source vectors) optimization previously encountered in equation 3.6. By Žnding joint MAP estimates of A and X, we obtain a problem that is much more tractable than the one of Žnding the single MAP estimate of A (which involves maximizing the marginal posterior density P ( A | Y )). The requirement that A 2 A, where A is a compact and hence bounded subset of Rm£n , is sufŽcient for the optimization problem 3.15 to avoid the degenerate solution,6 for k D 1, . . . , N, yk D Axk, with kAk ! 1 and kx kk ! 0.

(3.16)

This solution is possible for unbounded A because y D Ax is almost always solvable for x since learned overcomplete A’s are (generically) onto, and for any solution pair ( A, x) the pair ( a1 A, ax) is also a solution. This fact shows that the inverse problem of Žnding a solution pair ( A, x) is generally ill posed 6

kAk is any suitable matrix norm on A.

366

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

unless A is constrained to be bounded (as we have explicitly done here) or the cost functional is chosen to ensure that bounded A’s are learned (e.g., by adding a term monotonic in the matrix norm kAk to the cost function in equation 3.15). A variety of choices for the compact set A are available. Obviously, since different choices of A correspond to different a priori assumptions on the set of admissible matrices, A, the choice of this set can be expected to affect the performance of the resulting dictionary learning algorithm. We will consider two relatively simple forms for A. 3.3 Unit Frobenius–Norm Dictionary Prior. For the i.i.d. q D 2 gaussian measurement noise case of equation 3.8, algorithms that provably converge (in the low step-size limit) to a local minimum of equation 3.15 can be readily developed for the very simple choice,

A F D fA | kAkF D 1g ½ Rm£n ,

(3.17)

where kAkF denotes the Frobenius norm of the matrix A, kAk2F D tr(AT A ) , trace ( AT A ) , and it is assumed that the prior P ( A) is uniformly distributed on AF as per condition 3.13. As discussed in appendix A, AF is simply connected, and there exists a path in AF between any two matrices in AF. Following the gradient factorization procedure (Kreutz-Delgado & Rao, 1997; Rao & Kreutz-Delgado, 1999), we factor the gradient of d ( x) as

r d (x ) D a(x) P ( x ) x,

a(x) > 0,

(3.18)

where it is assumed that P (x ) is diagonal and positive deŽnite for all nonzero p x. For example, in the case where d ( x ) D kxkp , P ¡1 ( x ) D diag(|x[i]| 2¡p ).

(3.19)

Factorizations for other diversity measures d (x ) are given in Kreutz-Delgado and Rao (1997). We also deŽne b ( x) D la(x). As derived and proved in appendix A, a learning law that provably converges to a minimum of equation 3.15 on the manifold 3.17 is then given by d bT A b C b (b bT y kg, b x k D ¡V kf( A xk ) P (b x k ) )b xk ¡ A dt d b b ¡ tr( A bTd A b) A b) , m > 0, A D ¡m (d A dt

(3.20)

Dictionary Learning Algorithms for Sparse Representation

367

b is initialized to kAk b F D 1, V k are n £ n positive for k D 1, . . . , N, where A b deŽnite matrices, and the “error” d A is b D he(b dA x)b xT iN D

N 1 X e (b xk )b xTk , N kD 1

bb e (b xk ) D A x k ¡ y k,

(3.21)

which can be rewritten in the perhaps a more illuminating form (cf. equations 3.9 and 3.10), bD A bSxO xO ¡ SyxO . dA

(3.22)

A formal convergence proof of equation 3.20 is given in appendix A, where it is also shown that the right-hand side of the second equation in 3.20 correb onto the tangent space of A F , thereby sponds to projecting the error term d A b lies in the tangent space. Convergence of ensuring that the derivative of A the algorithm to a local optimum of equation 3.15 is formally proved by interpreting the loss functional as a Lyapunov function whose time derivabX b) is guaranteed tive along the trajectories of the adapted parameters ( A, to be negative deŽnite by the choice of parameter time derivatives shown in equation 3.20. As a consequence of the La Salle invariance principle, the loss functional will decrease in value, and the parameters will converge to the largest invariant set for which the time derivative of the loss functional is identically zero (Khalil, 1996). b and the vectors b Equation 3.20 is a set of coupled (between A x k) nonlinear differential equations that correspond to simultaneous, parallel updatb and b ing of the estimates A x k. This should be compared to the alternated separate (nonparallel) update rules 3.11 and 3.12 used in the AML algorithm described in section 3.1. Note also that (except for the trace term) the right-hand side of the dictionary learning update in equation 3.20 is of the same form as for the AML update law given in equation 3.11 (see also the discretized version of equation 3.20 given in equation 3.28 below). The key difference is the additional trace term in equation 3.20. This difference corresponds to a projection of the update onto the tangent space of the manifold 3.17, thereby ensuring a unit Frobenius norm (and hence boundedness) of the dictionary estimate at all times and avoiding the ill-posedness problem indicated in equation 3.16. It is also of interest to note that choosing V k to be the positive-deŽnite matrix, bT A b C b (b Vk D g k ( A x k ) P (b x k ) ) ¡1 , gk > 0,

(3.23)

in equation 3.20, followed by some matrix manipulations (see equation A.18 in appendix A), yields the alternative algorithm, d bT (b (b b ¡1 (b bT ) ¡1 y kg b xk D ¡gkfb x k ¡ P ¡1 (b xk ) A xk ) I C AP xk ) A dt

(3.24)

368

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

with g k > 0. In any event (regardless of the speciŽc choice of the positive deŽnite matrices V k as shown in appendix A), the proposed algorithm outb) , which satisŽes the implicit and lined here converges to a solution (b x k, A nonlinear relationships, bT (b (b b ¡1 (b bT ) ¡1 y, b x k D P ¡1 (b xk ) A x k ) I C AP xk ) A b D S T ( SxO xO ¡ cI ) ¡1 2 AF , A yxO

(3.25)

bTd A b) . for scalar c D tr( A To implement the algorithm 3.20 (or the variant using equation 3.24) in discrete time, a Žrst-order forward difference approximation at time t D tl can be used, b d x k (tl C1 ) ¡ b x k ( tl ) b x k ( tl ) ¼ dt tl C 1 ¡ tl

,

b x k[l C 1] ¡ b x k[l]

Dl

.

(3.26)

Applied to equation 3.24, this yields b [l] x k[l C 1] D (1 ¡ m l )b xk [l] C m lb xFOCUSS k

bT (b (b b ¡1 (b bT ) ¡1 y k b [l] D P ¡1 (b xFOCUSS xk ) A xk ) I C AP xk ) A k m l D g kD l ¸ 0.

(3.27)

b Similarly, discretizing the A-update equation yields the A-learning rule, equation 3.28, given below. More generally, taking m l to have a value between zero and one, 0 · m l · 1 yields an updated value b xk [l C 1], which is [l]. a linear interpolation between the previous value b x k[l] and b xFOCUSS k When implemented in discrete time, and setting m l D 1, the resulting Bayesian learning algorithm has the form of a combined iteration where we loop over the operations, bT (b (b b ¡1 (b bT ) ¡1 y k, b x k à P ¡1 (b xk ) A x k ) I C AP xk ) A k D 1, . . . , N

and

bà A b ¡ c (d A b ¡ tr( A bTd A b) A b) A

c > 0.

(3.28)

We call this FOCUSS-based, Frobenius-normalized dictionary-learning algorithm the FOCUSS-FDL algorithm. Again, this merged procedure should be compared to the separate iterations involved in the maximum likelihood b given by approach given in equations 3.11 and 3.12. Equation 3.28, with d A equation 3.21, corresponds to performing a Žnite step-size gradient descent

Dictionary Learning Algorithms for Sparse Representation

369

on the manifold AF . This projection in equation 3.28 of the dictionary update onto the tangent plane of AF (see the discussion in appendix A) ensures the well-behavedness of the MAP algorithm.7 The speciŽc step-size choice m l D 1, which results in the Žrst equation in equation 3.28, is discussed at length for the low-noise case in Rao and Kreutz-Delgado (1999). 3.4 Column-Normalized Dictionary Prior. Although mathematically very tractable, the unit-Frobenius norm prior, equation 3.17, appears to be somewhat too loose, judging from simulation results given below. In simulations with the Frobenius norm constraint AF , some columns of A can tend toward zero, a phenomenon that occurs more often in highly overcomplete A. This problem can be understood by remembering that we are using the 6 0 diversity measure, which penalizes columns associated with dp ( x) , p D terms in x with large magnitudes. If a column ai has a small relative magnitude, the weight of its coefŽcient x[i] can be large, and it will be penalized more than a column with a larger norm. This leads to certain columns being underused, which is especially problematic in the overcomplete case. An alternative, and more restrictive, form of the constraint set A is obtained by enforcing the requirement that the columns ai of A each be normalized (with respect to the Euclidean 2-norm) to the same constant value (Murray & Kreutz-Delgado, 2001). This constraint can be justiŽed by noting that Ax can be written as the nonunique weighted sum of the columns ai ,

Ax D

n X iD 1

ai x[i] D

n ³ ´ X ai i D1

ai

(ai x[i]) D A0 x0 ,

for any ai > 0, i D 1, . . . , n, showing that there is a column-wise ambiguity that remains even after the overall unit-Frobenius norm normalization has occurred, as one can now Frobenius-normalize the new matrix A0 . Therefore, consider the set of matrices on which has been imposed the column-wise constraint that » ­ ¼ ­ 2 1 T ­ AC D A ­ kai k D ai ai D , i D 1, . . . , n . (3.29) n The set AC is an mn ¡ n D n ( m ¡1) –dimensional submanifold of Rm£n . Note that every column of a matrix in A C has been normalized to the value p1n . 7 Because of the discrete-time approximation in equation 3.28,and even more generally because of numerical round-off effects in equation 3.20, a renormalization,

bà A b/ kAkF , A is usually performed at regular intervals.

370

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

In fact, any constant value for the column normalization can be used (including the unit normalization), but, as shown in appendix B, the particular normalization of p1n results in AC being a proper submanifold of the mn ¡ 1 dimensional unit Frobenius manifold AF ,

A C ½ AF , indicating that a tighter constraint on the matrix A is being imposed. Again, it is assumed that the prior P ( A ) is uniformly distributed on AC in the manner of equation 3.13. As shown in appendix B, AC is simply connected. A learning algorithm is derived for the constraint AC in appendix B, following much the same approach as in appendix A. Because the derivation of the b x k update to Žnd sparse solutions does not depend on the form of b update in algorithm 3.28 needs to be modiŽed. the constraint A, only the A Each column ai is now updated independently (see equation B.17), ai à ai ¡ c ( I ¡b aib aTi )d ai i D 1, . . . , n,

(3.30)

b in equation 3.21. We call the resulting where dai is the ith column of d A column-normalized dictionary-learning algorithm the FOCUSS-CNDL algorithm. The implementation details of the FOCUSS-CNDL algorithm are presented in section 4.2. 4 Algorithm Implementation

The dictionary learning algorithms derived above are an extension of the FOCUSS algorithm used for obtaining sparse solutions to the linear inverse problem y D Ax to the case where dictionary learning is now required. We refer to these algorithms generally as FOCUSS-DL algorithms, with the unit Frobenius-norm prior-based algorithm denoted by FOCUSS-FDL and the column-normalized prior-base algorithm by FOCUSS-CNDL. In this section, the algorithms are stated in the forms implemented in the experimental tests, where it is shown that the column normalization-based algorithm achieves higher performance in the overcomplete dictionary case. 4.1 Unit Frobenius-Norm Dictionary Learning Algorithm. We now summarize the FOCUSS-FDL algorithm derived in section 3.3. For each of the data vectors yk, we update the sparse source vectors x k using the FOCUSS algorithm:

P ¡1 (b x k ) D diag(|b x k[i]| 2¡p )

bT (lkI C AP b ¡1 (b bT ) ¡1 y k b x k à P ¡1 (b xk ) A xk ) A

(FOCUSS),

(4.1)

Dictionary Learning Algorithms for Sparse Representation

371

where lk D b (b xk ) is the regularization parameter. After updating the N b is reestimated, source vectors x k, k D 1, . . . , n, the dictionary A

SyxO

D

N 1 X y kb xTk N kD 1

SxO xO

D

N 1 X b x kb xTk N kD 1

bD A bSxO dA O x ¡ S yxO bà A b ¡ c (d A b ¡ tr( A bTd A b) A b) , A

c > 0,

(4.2)

where c controls the learning rate. For the experiments in section 5, the data block size is N D 100. During each iteration all training vectors are updated using equation 4.1, with a corresponding number of dictionary updates b it is renormalized using equation 4.2. After each update of the dictionary A, b to have unit Frobenius norm, k AkF D 1. The learning algorithm is a combined iteration, meaning that the FOCUSS algorithm is allowed to run for only one iteration (not until full convergence) before the A update step. This means that during early iterations, the b x k are in general not sparse. To facilitate learning A, the covariances S yOx and SxO xO are calculated with sparsiŽed b x k that have all but the rQ largest elements set to zero. The value of rQ is usually set to the largest desired number of nonzero elements, but this choice does not appear to be critical. The regularization parameter lk is taken to be a monotonically increasing function of the iteration number, lk D lmax tanh (10¡3 ¢ ( iter ¡ 1500) ) C 1) .

(4.3)

While this choice of l k does not have the optimality properties of the modiŽed L-curve method (see section 2.5), it does not require a one-dimensional optimization for each b x k and so is much less computationally expensive. This is further discussed below. 4.2 Column Normalized Dictionary Learning Algorithm. The improved version of the algorithm called FOCUSS-CNDL, which provides increased accuracy especially in the overcomplete case, was proposed in Murray and Kreutz-Delgado (2001). The three key improvements are colb an efŽcient way of adjusting umn normalization that restricts the learned A, the regularization parameter l k, and reinitialization to escape from local optima. The column-normalized learning algorithm discussed in section 3.4 and derived in appendix B is used. Because the b x k update does not depend on the constraint set A, the FOCUSS algorithm in equation 4.1 is used to

372

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

update N vectors, as discussed in section 4.1. After every N source vectors are updated, each column of the dictionary is then updated as ai à ai ¡ c ( I ¡ b aib aTi )d ai i D 1, . . . , n,

(4.4)

b which is found using equation 4.2. After where dai are the columns of d A, updating each ai , it is renormalized to kai k2 D 1 / n by ai à p

ai , nkai k

(4.5)

b F D 1 as shown in section B.1. which also ensures that kAk The regularization parameter lk may be set independently for each vector in the training set, and a number of methods have been suggested, including quality of Žt (which requires a certain level of reconstruction accuracy), sparsity (requiring a certain number of nonzero elements), and the L-curve which attempts to Žnd an optimal trade-off (Engan, 2000). The L-curve method works well, but it requires solving a one-dimensional optimization for each l k, which becomes computationally expensive for large problems. Alternatively, we use a heuristic method that allows the trade-off between error and sparsity to be tuned for each application, while letting each training vector y k have its own regularization parameter lk to improve the quality of the solution, Á l k D lmax

! bx kk kyk ¡ Ab 1¡ , ky kk

l k, lmax > 0.

(4.6)

For data vectors that are represented accurately, lk will be large, driving the algorithm to Žnd sparser solutions. If the SNR can be estimated, we can set lmax D (SNR) ¡1 . The optimization problem, 3.15, is concave when p · 1, so there will be multiple local minima. The FOCUSS algorithm is guaranteed to converge only to one of these local minima, but in some cases, it is possible to determine when that has happened by noticing if the sparsity is too low. Periodically (after a large number of iterations), the sparsity of the solutions b xk is checked, and if found too low, b x k is reinitialized randomly. The algorithm is also sensitive to initial conditions, and prior information may be incorporated into the initialization to help convergence to the global solution. 5 Experimental Results

Experiments were performed using complete dictionaries ( n D m) and overcomplete dictionaries ( n > m ) on both synthetically generated data and

Dictionary Learning Algorithms for Sparse Representation

373

natural images. Performance was measured in a number of ways. With synthetic data, performance measures include the SNR of the recovered sources x k compared to the true generating source and comparing the learned dictionary with the true dictionary. For images of natural scenes, the true underlying sources are not known, so the accuracy and efŽciency of the image coding are found. 5.1 Complete Dictionaries:Comparison with ICA. To test the FOCUSSFDL and FOCUSS-CNDL algorithms, simulated data were created following the method of Engan et al. (1999) and Engan (2000). The dictionary A of size 20 £20 was created by drawing each element aij from a normal distribution with m D 0, s 2 D 1 (written as N (0, 1) ) followed by a normalization to ensure that kAkF D 1. Sparse source vectors x k, k D 1, . . . , 1000 were created with r D 4 nonzero elements, where the r nonzero locations are selected at random (uniformly) from the 20 possible locations. The magnitudes of each nonzero element were also drawn from N (0, 1) and limited so that |x kl | > 0.1. The input data yk were generated using y D Ax (no noise was added). For the Žrst iteration of the algorithm, the columns of the initializabinit, were taken to be the Žrst n D 20 training vectors yk. tion estimate, A The initial xk estimates were then set to the pseudoinverse solution b xk D bT ( A binit A bT ) ¡1 y k. The constant parameters of the algorithm were set as A init init follows: p D 1.0, c D 1.0, and lmax D 2 £ 10¡3 (low noise, assumed SNR ¼ 27 dB). The algorithms were run for 200 iterations through the entire b was updated after updating 100 data data set, and during each iteration, A vectors b x k. To measure performance, the SNR between the recovered sources b x k and the the true sources x k were calculated. Each element xk [i] for Žxed i was considered as a time-series vector with 1000 elements, and SNRi for each was found using

³ SNRi D 10 log10

kx k[i]k2 kx k[i] ¡ b xk[i]k2

´ .

(5.1)

The Žnal SNR is found by averaging SNRi over the i D 1, . . . , 20 vectors and 20 trials of the algorithms. Because the dictionary A is learned only to within a scaling factor and column permutations, the learned sources must be matched with corresponding true sources and scaled to unit norm before the SNR calculation is done. The FOCUSS-FDL and FOCUSS-CNDL algorithms were compared with extended ICA (Lee, Girolami, & Sejnowski, 1999) and Fastica 8 (Hyv¨arinen 8 Matlab and C versions of extended ICA can be found on-line at http://www.cnl.salk. edu/»tewon/ICA/code.html. Matlab code for FastICA can be found at: http://www.cis. hut.Ž/projects/ica/fastica/.

374

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

Figure 1: Comparison between FOCUSS-FDL, FOCUSS-CNDL, extended ICA, and FastICA on synthetically generated data with a complete dictionary A, size 20 £ 20. The SNR was computed between the recovered sources b xk and the true sources xk . The mean SNR and standard deviation were computed over 20 trials.

et al., 1999). Figure 1 shows the SNR for the tested algorithms. The SNR for FOCUSS-FDL is 27.7 dB, which is a 4.7 dB improvement over extended ICA, and for FOCUSS-CNDL, the SNR is 28.3 dB. The average run time for FOCUSS-FDL/CNDL was 4.3 minutes, for FastICA 0.10 minute, and for extended ICA 0.19 minute on a 1.0 GHz Pentium III Xeon computer. 5.2 Overcomplete Dictionaries. To test the ability to recover the true A and x k solutions in the overcomplete case, dictionaries of size 20 £ 30 and 64 £ 128 were generated. Diversity r was set to Žxed values (4 and 7) and uniformly randomly (5–10 and 10–15). The elements of A and the sources x k were created as in section 5.1. The parameters were set as follows: p D 1.0, c D 1.0, lmax D 2 £ 10¡3 . The algorithms were run for 500 iterations through the entire data set, and b was updated after updating 100 data vectors b during each iteration, A xk . As a measure of performance, we Žnd the number of columns of A that were matched during learning. Because A can be learned only to within column permutations and sign and scale changes, the columns are normalized b is rearranged columnwise so that b so that kb ai k D kaj k D 1, and A aj is given the index of the closest match in A (in the minimum two-norm sense). A match is counted if

1 ¡ |aTib ai | < 0.01.

(5.2)

Dictionary Learning Algorithms for Sparse Representation

375

Table 1: Synthetic Data Results. Learned A Columns Algorithm

Size of A

Diversity, r

FOCUSS-FDL FOCUSS-CNDL FOCUSS-CNDL FOCUSS-CNDL FOCUSS-FDL FOCUSS-CNDL

20 £ 30 20 £ 30 64 £ 128 64 £ 128 64 £ 128 64 £ 128

7 7 7 5–10 10–15 10–15

Average SD 25.3 28.9 125.3 126.3 102.8 127.4

3.4 1.6 2.1 1.3 4.5 1.3

Learned x

%

Average

SD

%

84.2 96.2 97.9 98.6 80.3 99.5

675.9 846.8 9414.0 9505.5 4009.6 9463.4

141.0 97.6 406.5 263.8 499.6 330.3

67.6 84.7 94.1 95.1 40.1 94.6

Similarly, the number of matching b xk are counted (after rearranging the b elements in accordance with the indices of the rearranged A), 1 ¡ |xTib xi | < 0.05.

(5.3)

If the data are generated by an A that is not column normalized, other measures of performance need to be used to compare x k and b x k. The performance is summarized in Table 1, which compares the FOCUSSFDL with the column-normalized algorithm (FOCUSS-CNDL). For the 20 £ 30 dictionary, 1000 training vectors were used, and for the 64£128 dictionary 10,000 were used. Results are averaged over four or more trials. For the 64 £ 128 matrix and r D 10–15, FOCUSS-CNDL is able to recover 99.5% (127.4/128) of the columns of A and 94.6% (9463/10,000) of the solutions x k to within the tolerance given above. This shows a clear improvement over FOCUSS-FDL, which learns only 80.3% of the A columns and 40.1% of the solutions x k. Learning curves for one of the trials of this experiment (see Figure 2) show that most of the columns of A are learned quickly within the Žrst 100 iterations, and that the diversity of the solutions drops to the desired level. Figure 2b shows that it takes somewhat longer to learn the xk correctly and that reinitialization of the low sparsity solutions (at iterations 175 and 350) helps to learn additional solutions. Figure 2c shows the diversity at each iteration, measured as the average number of elements of each b x k that are larger than 1 £ 10¡4 . 5.3 Image Data Experiments. Previous work has shown that learned basis functions can be used to code data more efŽciently than traditional Fourier or wavelet bases (Lewicki & Olshausen, 1999). The algorithm for Žnding overcomplete bases in Lewicki and Olshausen (1999) is also designed to solve problem 1.1 but differs from our method in a number of ways, including using only the Laplacian prior (p D 1), and using conjugate gradient optimization for Žnding sparse solutions (whereas we use the FOCUSS algorithm). It is widely believed that overcomplete representations

376

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

Figure 2: Performance of the FOCUSS-CNDL algorithm with overcomplete dictionary A, size 64 £ 128. (a) Number of correctly learned columns of A at each iteration. (b) Number of sources xk learned. (c) Average diversity (n-sparsity) of the x k . The spikes in b and c indicate where some solutions b x k were reinitialized because they were not sparse enough.

Dictionary Learning Algorithms for Sparse Representation

377

are more efŽcient than complete bases, but in Lewicki and Olshausen (1999), the overcomplete code was less efŽcient for image data (measured in bits per pixel entropy), and it was suggested that different priors could be used to improve the efŽciency. Here, we show that our algorithm is able to learn more efŽcient overcomplete codes for priors with p < 1. The training data consisted of 10,000 8 £ 8 image patches drawn at random from black and white images of natural scenes. The parameter p was varied from 0.5–1.0, and the FOCUSS-CNDL algorithm was trained for 150 iterations. The complete dictionary (64 £ 64) was compared with the 2£ overcomplete dictionary (64 £ 128). Other parameters were set: c D 0.01, lmax D 2 £ 10¡3 . The coding efŽciency was measured using the entropy (bits per pixel) method described in Lewicki and Olshausen (1999). Figure 3 plots the entropy versus reconstruction error (root mean square error, RMSE) and shows that when p < 0.9, the entropy is less for the overcomplete representation at the same RMSE. An example of coding an entire image is shown in Figure 4. The original test image (see Figure 4a) of size 256 £ 256 was encoded using the learned dictionaries. Patches from the test image were not used during training. Table 2 gives results for low- and high-compression cases. In both cases, coding with the overcomplete dictionary (64£128) gives higher compression (lower bits per pixel) and lower error (RMSE). For the high-compression case (see

Figure 3: Comparing the coding efŽciency of complete and 2£ overcomplete representations on 8 £8 pixel patches drawn from natural images. The points on the curve are the results from different values of p, at the bottom right, p D 1.0, and at the top left, p D 0.5. For smaller p, the overcomplete case is more efŽcient at the same level of reconstruction error (RMSE).

378

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

Figure 4: Image compression using complete and overcomplete dictionaries. Coding with an overcomplete dictionary is more efŽcient (fewer bits per pixel) and more accurate (lower RMSE). (a) Original image of size 256 £ 256 pixels. (b) Compressed with 64 £ 64 complete dictionary to 0.826 bits per pixel at RMSE = 0.329. (c) Compressed with 64 £ 128 overcomplete dictionary to 0.777 bits per pixel at RMSE = 0.328.

Dictionary Learning Algorithms for Sparse Representation

379

Table 2: Image Compression Results. Dictionary Size 64 £ 64 64 £ 128 64 £ 64 64 £ 128

p

Compression (bits/pixel)

RMSE

0.5 0.6 0.5 0.6

2.450 2.410 0.826 0.777

0.148 0.141 0.329 0.328

Average Diversity 17.3 15.4 4.5 4.0

Figures 4b and 4c), the 64 £ 128 overcomplete dictionary gives compression of 0.777 bits per pixel at error 0.328, compared to the 64 £ 64 complete dictionary at 0.826 bits per pixel at error 0.329. The amount of compression was selected by adjusting lmax (the upper limit of the regularization parameter). For high compression, lmax D 0.02, and for low compression, lmax D 0.002. 6 Discussion and Conclusions

We have applied a variety of tools and perspectives (including ideas drawn from Bayesian estimation theory, nonlinear regularized optimization, and the theory of majorization and convex analysis) to the problem of developing algorithms capable of simultaneously learning overcomplete dictionaries and solving sparse source-vector inverse problems. The test experiment described in section 5.2 is a difŽcult problem designed to determine if the proposed learning algorithm can solve for the known true solutions for A and the spare source vectors xk to the underdetermined inverse problem y k D Axk . Such testing, which does not appear to be regularly done in the literature, shows how well an algorithm can extract stable and categorically meaningful solutions from synthetic data. The ability to perform well on such test inverse problems would appear to be at least a necessary condition for an algorithm to be trustworthy in domains where a physically or biologically meaningful sparse solution is sought, such as occurs in biomedical imaging, geophysical seismic sounding, and multitarget tracking. The experimental results presented in section 5.2 show that the FOCUSSDL algorithms can recover the dictionary and the sparse sources vectors. This is particularly gratifying when one considers that little or no optimization of the algorithm parameters has been done. Furthermore, the convergence proofs given in the appendixes show convergence only to local optima, whereas one expects that there will be many local optima in the cost function because of the concave prior and the generally multimodal form of the cost function. One should note that algorithm 3.28 was constructed precisely with the goal of solving inverse problems of the type considered here, and therefore

380

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

one must be careful when comparing the results given here with other algorithms reported in the literature. For instance, the mixture-of-gaussians prior used in Attias (1999) does not necessarily enforce sparsity. While other algorithms in the literature might perform well on this test experiment, to the best of our knowledge, possible comparably performing algorithms such as Attias (1999), Girolami (2001), Hyv¨arinen et al. (1999), and Lewicki and Olshausen (1999) have not been tested on large, overcomplete matrices to determine their accuracy in recovering A, and so any comparison along these lines would be premature. In section 5.1, the FOCUSS-DL algorithms were compared to the well-known extended ICA and FastICA algorithms in a more conventional test with complete dictionaries. Performance was measured in terms of the accuracy (SNR) of the recovered sources x k, and both FOCUSS-DL algorithms were found to have signiŽcantly better performance (albeit with longer run times). We have also shown that the FOCUSS-CNDL algorithm can learn an overcomplete representation, which can encode natural images more efŽciently than complete bases learned from data (which in turn are more efŽcient than standard nonadaptive bases, such as Fourier or wavelet bases; Lewicki & Olshausen, 1999). Studies of the human visual cortex have shown a higher degree of overrepresentation of the fovea compared to the other mammals, which suggests an interesting connection between overcomplete representations and visual acuity and recognition abilities (Popovic & Sjostrand, ¨ 2001). Because the coupled dictionary learning and sparse-inverse solving algorithms are merged and run in parallel, it should be possible to run the algorithms in real time to track dictionary evolution in quasistationary environments once the algorithm has essentially converged. One way to do this would be to constantly present randomly encountered new signals, y k, to the algorithm at each iteration instead of the original training set. One also has to ensure that the dictionary learning algorithm is sensitive to the new data so that dictionary tracking can occur. This would be done by an appropriate adaptive Žltering of the current dictionary estimate driven by the new-data derived corrections, similar to techniques used in the adaptive Žltering literature (Kalouptsidis & Theodoridis, 1993).

Appendix A: The Frobenius-Normalized Prior Learning Algorithm

Here we provide a derivation of the algorithm 3.20–3.21 and prove that it converges to a local minimum of equation 3.15 on the manifold AF D fA | kAkF D 1g ½ Rm£n deŽned in equation 3.17. Although we focus on the development of the learning algorithm on AF , the derivations in sections A.2 and A.3, and the beginning of subsection are done for a general constraint manifold A.

Dictionary Learning Algorithms for Sparse Representation

381

A.1 Admissible Matrix Derivatives.

A.1.1 The Constraint Manifold AF . In order to determine the structural d form of admissible derivatives, AP D dt A for matrices belonging to AF ,9 it is useful to view A F as embedded in the Žnite-dimensional Hilbert space of matrices, Rm£n , with inner product hA, Bi D tr(AT B ) D tr(BT A ) D tr(ABT ) D tr(BAT ) . The corresponding matrix norm is the Frobenius norm, p p kAk D kAkF D tr AT A D tr AAT . We will call this space the Frobenius space and the associated inner product the Frobenius inner product. It is useful to note the isometry, A 2 Rm£n () A D vec ( A ) 2 Rmn , where A is the mn-vector formed by stacking the columns of A. Henceforth, bold type represents the stacked version of a matrix (e.g., B D vec (B ) ). The stacked vector A belongs to the standard Hilbert space Rmn , which we shall henceforth refer to as the stacked space. This space has the standard Euclidean inner product and norm, p hA , B i D A T B , kA k D A T A . It is straightforward to show that hA, Bi D hA , B i and

kAk D kA k.

In particular, we have A 2 AF () kAk D kA k D 1. Thus, the manifold in equation 3.17 corresponds to the (mn¡1)–dimensional unit sphere in the stacked space, Rmn (which, with a slight abuse of notation, we will continue to denote by A F ). It is evident that AF is simply connected so that a path exists between any two elements of A F and, in particular, a path exists between any initial value for a dictionary, Ainit 2 AF , used to initialize a learning algorithm, and a desired target value, AŽnal 2 AF .10 9 Equivalently, we want to determine the structure of elements AP of the tangent space, TAF , to the smooth manifold AF at the point A. 10 For example, for 0 · t · 1, take the t–parameterized path,

A (t) D

(1 ¡ t)A init C tA Žnal . k(1 ¡ t)A init C tA Žnal k

382

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

A.1.2 Derivatives on AF : The Tangent Space T AF . Determining the form of admissible derivatives on equation 3.17 is equivalent to determining the form of admissible derivatives on the unit Rmn –sphere. On the unit sphere, we have the well-known fact that A 2 AF H)

d P D 0 H) A P ? A. kA k2 D 2A T A dt

P is A P D ¤ Q , where Q is arbitrary and This shows that the general form of A ³ D

¤

I¡

AA T

kA k2

´ D ( I ¡ AA T )

(A.1)

is the stacked space projection operator onto the tangent space of the unit Rmn –sphere at the point A (note that we used the fact that kAk D 1). The projection operator ¤ is necessarily idempotent, ¤ D ¤ 2 . ¤ is also self-adjoint, ¤ D ¤ ¤ , where the adjoint operator ¤ ¤ is deŽned by the requirement that h¤

¤

Q 1 , Q 2 i D hQ 1 , ¤ Q 2 i,

for all Q 1 , Q 2 2 Rmn ,

showing that ¤ is an orthogonal projection operator. In this case, ¤ ¤ D ¤ T , so that self-adjointness corresponds to ¤ being symmetric. One can readily show that an idempotent, self-adjoint operator is nonnegative, which in this case corresponds to the symmetric, idempotent operator ¤ being a positive semideŽnite matrix. This projection can be easily rewritten in the Frobenius space, P D ¤ Q D Q ¡ hA , Q iA () AP D L Q D Q ¡ hA, QiA A D Q ¡ tr(AT Q) A.

(A.2)

Of course, this result can be derived directly in the Frobenius space using the fact that A 2 AF H)

d P D 2 tr(AT AP ) D 0, kAk2 D 2hA, Ai dt

from which it is directly evident that AP 2 T A F at A 2 A F

,

P D tr AT AP D 0, hA, Ai

(A.3)

and therefore AP must be of the form11 tr(AT Q ) AP D L Q D Q ¡ A D Q ¡ tr(AT Q ) A. tr(AT A ) 11

It must be the case that L D I ¡

notation (Dirac, 1958).

| AihA| kAk 2

(A.4)

D I ¡ |AihA|, using the physicist’s “bra-ket”

Dictionary Learning Algorithms for Sparse Representation

383

One can verify that L is idempotent and self-adjoint and is therefore a nonnegative, orthogonal projection operator. It is the orthogonal projection operator from Rm£n onto the tangent space T AF . In the stacked space (with some additional abuse of notation), we represent the quadratic form for positive semideŽnite symmetric matrices W as kA k2W D A T WA . Note that this deŽnes a weighted norm if, and only if, W is positive deŽnite, which might not be the case by deŽnition. In particular, when W D ¤ , the quadratic form kA k2 is only positive semideŽnite. Finally, note from ¤ equation A.4 that 8A 2 A F , L Q D 0 () Q D cA, with c D tr(AT Q) .

(A.5)

A.2 Minimizing the Loss Function over a General Manifold A. Con-

sider the Lyapunov function, VN (X, A ) D hdq ( y ¡ Ax ) C ldp (x ) iN ,

A 2 A,

(A.6)

where A is some arbitrary but otherwise appropriately deŽned constraint manifold associated with the prior, equation 3.13. Note that this is precisely the loss function to be minimized in equation 3.15. If we can determine smooth parameter trajectories (i.e., a parameter-vector adaptation P AP ) such that along these trajectories VP ( X, A) · 0, then as a conserule) ( X, quence of the La Salle invariance principle (Khalil, 1996), the parameter values will converge to the largest invariant set (of the adaptation rule viewed as a nonlinear dynamical system) contained in the set C D f(X, A ) | VP N (X, A ) ´ 0 and A 2 A g.

(A.7)

The set C contains the local minima of VN . With some additional technical assumptions (generally dependent on the choice of adaptation rule), the elements of C will contain only local minima of VN . Assuming the i.i.d. q D 2 gaussian measurement noise case of equation 3.8, 12 the loss (Lyapunov) function to be minimized is then ½ ¾ 1 kAx ¡ yk2 C ldp ( x) , A 2 A, (A.8) VN (X, A ) D 2 N which is essentially the loss function to be minimized in equation 3.15. 13 12 Note that in the appendix, unlike the notation used in equations 3.8 et seq., the “hat” notation has been dropped. Nonetheless, it should be understood that the quantities A and X are unknown parameters to be optimized over in equation 3.15, while the measured signal vectors Y are known. 13 The factor 12 is added for notational convenience and does not materially affect the derivation.

384

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

Suppose for the moment, as in equations 3.4 through 3.9, that X is assumed to be known, and note that then (ignoring constant terms depending on X and Y) VN can be rewritten as VN ( A) D htr(Ax ¡ y ) ( Ax ¡ y ) T iN

D htr(AxxT AT ) ¡ 2 tr(AxyT ) C tr(yyT ) iN D tr A Sxx AT ¡ 2 tr A Sxy

VN ( A) D trfA Sxx AT ¡ 2A Sxyg, T deŽned as in equation 3.10. Using standard results for Sxx and S xy D Syx from matrix calculus (Dhrymes, 1984), we can show that VN ( A ) is minimized by the solution 3.9. This is done by setting

@ @A

b) D 0, VN ( A

and using the identities (valid for W symmetric) @ @A

tr AWAT D 2AW and

@ @A

tr AB D BT .

This yields (assuming that Sxx is invertible), @ @A

b) D 2A bSxx ¡ 2 S T D 2( A bSxx ¡ Syx ) D 0, VN ( A xy b D SyxO S ¡1 , )A xO xO

which is equation 3.9, as claimed. For Sxx nonsingular, the solution is unique and globally optimal. This is, of course, a well-known result. Now return to the general case, equation A.8, where both X and A are unknown. For the data indexed by k D 1, . . . , N, deŽne the quantities d k D dp ( xk ) , e ( x) D Ax ¡ y, and e k D Ax k ¡ yk. The loss function and its time derivative can be written, ½ VN ( X, A) D

1 T e ( x ) e (x ) C ldp (x ) 2

¾ N

P N VP N ( X, A) D heT ( x ) eP (x ) C lr T dp ( x ) xi

P ) C lr T dp (x ) xi P N. D heT ( x ) ( AxP C Ax

Then, to determine an appropriate adaptation rule, note that VP N D T1 C T2 ,

(A.9)

Dictionary Learning Algorithms for Sparse Representation

385

where P ND T1 D h(eT ( x ) A C lr T dp ( x ) ) xi

N 1 X ( eT A C lr T d k ) xP k N kD 1 k

(A.10)

and N X P ND 1 P k. T2 D heT ( x ) Axi eT Ax N kD 1 k

(A.11)

Enforcing the separate conditions T1 · 0 and T2 · 0,

(A.12)

(as well as the additional condition that A 2 A) will be sufŽcient to ensure that VP N · 0 on A. In this case, the solution-containing set C of equation A.7 is given by C D f(X, A ) | T1 (X, A ) ´ 0, T2 ( X, A) ´ 0 and A 2 Ag.

(A.13)

Note that if A is known and Žxed, then T 2 ´ 0, and only the Žrst condition of equation A.12 (which enforces learning of the source vectors, x k) is of concern. Contrawise, if source vectors x k, which ensure that e ( x k ) D 0 are Žxed and known, then T1 ´ 0, and the second condition of equation A.12 (which enforces learning of the dictionary matrix, A) is at issue. A.3 Obtaining the x k Solutions with the FOCUSS Algorithm. We now develop the gradient factorization-based derivation of the FOCUSS algorithm, which provides estimates of x k while satisfying the Žrst convergence condition of equation A.12. The constraint manifold A is still assumed to be arbitrary. To enforce the condition T1 · 0 and derive the Žrst adaptation rule given in equation 3.20, we note that we can factor r d k D r d ( x k ) as (Kreutz-Delgado & Rao, 1997; Rao & Kreutz-Delgado, 1999)

r d k D akP kx k with ak D axk > 0 and P xk D P k positive deŽnite and diagonal for all nonzero x k. Then, deŽning b k D lak > 0 and selecting an arbitrary set of (adaptable) symmetric positive-deŽnite matrices V k, we choose the learning rule xP k D ¡V kfAT ek C lr d kg D ¡Vkf(AT A C b kP k ) x k ¡ AT y kg, k D 1, . . . , N,

(A.14)

386

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

which is the adaptation rule for the state estimates x k D b xk given in the Žrst line of equation 3.20. With this choice, we obtain T 1 D ¡hkAT e (x ) C lr d ( x) k2V iN D ¡

N 1 X kAT ek C lr d kk2Vk N kD 1

D ¡

N 1 X k(AT A C b kP k ) xk ¡ AT y kk2Vk · 0, N kD 1

(A.15)

as desired. Assuming convergence to the set A.13 (which will be seen to be the case after we show below how to ensure that we also have T2 · 0), we will asymptotically obtain (reintroducing the “hat” notation to now denote converged parameter estimates) bT A b C b kP k )b bT y kk2 ´ 0, k D 1, . . . , N, k( A xk ¡ A Vk which is equivalent to bT A b C b kP k ) ¡1 A bT y k, k D 1, . . . , N, b xk D ( A

(A.16)

at convergence. This is also equivalent to the condition given in the Žrst line of equation 3.25, as shown below. Exploiting the fact that V k in equation A.14 are arbitrary (subject to the symmetry and positive-deŽniteness constraint), let us make the speciŽc choice shown in equation 3.23, V k D gk ( AT A C b kP k ) ¡1 , g k > 0, k D 1, . . . , N.

(A.17)

Also note the (trivial) identity, ( AT A C bP ) P ¡1 AT D AT ( AP ¡1 AT C bI) , which can be recast nontrivially as ( AT A C bP ) ¡1 AT D P ¡1 AT (b I C AP ¡1 AT ) ¡1 .

(A.18)

With equations A.17 and A.18, the learning rule, A.14, can be recast as T (b C T ) ¡1 xP k D ¡gkfx k ¡ P ¡1 AP ¡1 ykg, k D 1, . . . , N, kI k A k A

(A.19)

which is the alternative learning algorithm, 3.24. At convergence (when T1 ´ 0) we have the condition shown in the Žrst line of equation 3.25, bT (b kI C AP b ¡1 A bT ) ¡1 y k. b x k D P ¡1 A

(A.20)

Dictionary Learning Algorithms for Sparse Representation

387

This also follows from the convergence condition A.16 and the identity A.18, showing that the result, A.20, is independent of the speciŽc choice of V k > 0. Note from equations A.14 and A.15 that T1 ´ 0 also results in xP k ´ 0 for k D 1, . . . , N, so that we will have converged to constant values, b x k, which satisfy equation A.20. For the case of known, Žxed A, the learning rule derived here will converge to sparse solutions, x k, and when discretized as in section 3.3, yields equation 3.27, which is the known dictionary FOCUSS algorithm (Rao & Kreutz-Delgado, 1998a, 1999). Note that the derivation of the sparse source-vector learning algorithm here, which enforces the condition T1 · 0, is entirely independent of any constraints placed on A (such as, for example, the unit Frobenius-norm and column-norm constraints considered in this article) or of the form of the A-learning rule. Thus alternative choices of constraints placed on A, as considered in Murray and Kreutz-Delgado (2001) and described in appendix B, will not change the form of the x k-learning rule derived here. Of course, because the x k learning rule is strongly coupled to the A-learning rule, algorithmic performance and speed of convergence may well be highly sensitive to conditions placed on A and the speciŽc A learning algorithm used. A.4 Learning the Dictionary A.

A.4.1 General Results. We now turn to the enforcement of the second convergence condition, T2 · 0 and the development of the dictionary adaptation rule shown in equation 3.20. First, as in equation 3.21, we deŽne the error term dA as dA D he(x) xT iN D

N 1 X e ( xk ) xTk D A Sxx ¡ Syx , N kD 1

(A.21)

using the fact that e ( x) D Ax ¡ y. Then, from equation A.11, we have P N D htr(xe T ( x ) AP ) iN D tr(hxe T ( x ) iN AP ) T2 D heT ( x ) Axi P. D tr(dAT AP ) D d A T A

(A.22)

So far, these steps are independent of any speciŽc constraints that may be placed on A, other than that the manifold A be smooth and compact. With A constrained to lie on a speciŽed smooth, compact manifold, to ensure correct learning behavior, it is sufŽcient to impose the constraint that AP lies in the tangent space to the manifold and the condition that T2 · 0. A.4.2 Learning on the Unit Frobenius Sphere, AF . To ensure that T2 · 0 and that AP is in the tangent space of the unit sphere in the Frobenius space Rmn , we take P D ¡m ¤ d A () AP D ¡m LdA D ¡m (d A ¡ tr(ATdA) A ) , m > 0, A

(A.23)

388

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

which is the adaptation rule given in equation 3.20. With this choice, and using the positive semideŽniteness of ¤ , we have T 2 D ¡m kd A k2¤ · 0,

as required. Note that at convergence, the condition T2 ´ 0, yields AP ´ 0, so that we will have converged to constant values for the dictionary elements, and b D L (A bSxO xO ¡ S yxO ) H) d A b D (A bSxO xO ¡ SyxO ) D cA, b 0 D Ld A

(A.24)

bTd A b) . Thus, the steady-state solution is from equation A.5, where c D tr( A b D S y, xO ( SxO xO ¡ cI ) ¡1 2 AF . A

(A.25)

Note that equations A.20 and A.25 are the steady-state values given earlier in equation 3.25. Appendix B: Convergence of the Column–Normalized Learning Algorithm

The derivation and proof of convergence of the column-normalized learning algorithm, applicable to learning members of AC , is accomplished by appropriately modifying key steps of the development given in appendix A. As in appendix A, the standard Euclidean norm and inner product apply to column vectors, while the Frobenius norm and inner product apply to m £ n matrix elements of Rm£n . B.1 Admissible Matrix Derivatives.

B.1.1 The Constraint Manifold AC . Let ei D (0 ¢ ¢ ¢ 0 1 0 ¢ ¢ ¢ 0) T 2 Rn be the canonical unit vector whose components are all zero except for the value 1 in the ith location. Then n X ID ei eTi and ai D Aei . iD 1

Note that kai k2 D aTi ai D ( Aei ) T Aei D eTi AT Aei D tr ei eTi AT A D tr MTi A D hMi , Ai,

where

Mi , Aei eTi D ai eTi D [0 0 ¢ ¢ ¢ 0 ai 0 ¢ ¢ ¢ 0] 2 Rm£n .

(B.1)

Note that only the ith column of Mi is nonzero and is equal to ai D ith column of A. We therefore have that AD

n X i D1

Mi .

(B.2)

Dictionary Learning Algorithms for Sparse Representation

389

Also, for i, j D 1, . . . , n, hMi , Mj i D tr MTi Mj D tr ei eTi AT Aej ejT D tr eTi AT Aej ejT ei D kai k2di, j ,

(B.3)

where di, j is the Kronecker delta. Note, in particular, that kai k D kMi k. Let A 2 AC , where AC is the set of column-normalized matrices, as deŽned by equation 3.29. AC is an mn¡n D n ( m¡1 )–dimensional submanifold of the Frobenius space Rm£n as each of the n columns of A is normalized as kai k2 D kM i k2 ´

1 , n

and kAk2 D tr AT A D tr

n X iD 1

MTi

n X jD 1

Mj D

n X iD 1

kai k2 D

n D 1, n

using linearity of the trace function and property, B.3. It is evident, therefore, that

AC ½ AF , for AF deŽned by equation 3.17. We can readily show that AC is simply connected, so that a continuous path exists between any matrix A D Ainit 2 AC to any other columnnormalized matrix A0 D AFinal 2 AC . Indeed, let A and A0 be such that A D [a1 , . . . , an ], A0 D [a01 , . . . , a0n ], p kai k D kaj0 k D 1 / n, 8i, j D 1, . . . , n. There is obviously a continuous path entirely in A C from A 2 A C to the intermediate matrix [a01 , a2 , . . . , an ] 2 A C . Similarly, there is a continuous path entirely in AC from [a01 , a2 , . . . , an ] to [a01 , a02 , . . . , an ], and so on to [a01 , . . . , a0n ] D A0 . To summarize, AC is a simply connected ( nm ¡ n ) –dimensional submanifold of the simply connected ( nm ¡ 1) –dimensional manifold AF and they are both submanifolds of the (nm ) –dimensional Frobenius space Rm£n . B.1.2 Derivatives on AC : The Tangent Space T AC . i D 1, . . . , n deŽne b ai D

p ai D nai , kai k

kb ai k D 1,

and p bi D Mi D nMi D b M ai eTi , kMi k

bi k D 1. kM

For convenience, for

390

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

Note that equation B.3 yields bi , M bj i D tr M bT M bj D di, j . hM i

(B.4)

For any A 2 AC we have for each i D 1, . . . , n, d d d P i kai k2 D aTi ai D eTi AT Aei D 2eTi AT Ae dt dt dt T P T P D 2 tr ei eT i A A D 2 tr Mi A,

0D

or P D tr MT AP D 0 for A 2 AC ) hMi , Ai i

i D 1, . . . , n.

(B.5)

In fact, AP 2 T A C at A 2 AC

,

bi , Ai P D tr M bT AP D 0 hM i for

i D 1, . . . , n.

(B.6)

Note from equations B.2 and B.5 that * + n X P P hA, Ai D Mi , A D 0, iD 1

showing that (see equation A.3) T AC ½ T AF , as expected from the fact that AC ½ A F . An important and readily proven fact is that for each i D 1, . . . , n, bT AP D 0 tr M i

,

AP D Pi Qi

(B.7)

for some matrix Qi and projection operator, Pi , deŽned by b i hM bi , Qi D Q ¡ M bi tr M bT Q. Pi Q , Q ¡ M i

(B.8)

From equation B.4, it can be shown that the projection operators commute, P i Pj D Pj P i ,

6 iD j,

i, j D 1, . . . , n,

(B.9)

and are idempotent, P 2i D Pi ,

i D 1, . . . , n.

(B.10)

Indeed, it is straightforward to show from equation B.4 that for all Q, b i hM bi , Qi ¡ M bj h M bj , Qi, P i Pj Q D Pj Pi Q D Q ¡ M

for all

6 iD j, (B.11)

Dictionary Learning Algorithms for Sparse Representation

391

and b i hM bi , Qi D Pi Q, P2i Q D Q ¡ M

i D 1, . . . , n.

(B.12)

It can also be shown that hPi Q1 , Q2 i D hQ1 , Pi Q2 i,

showing that Pi is self–adjoint, Pi D P¤i . DeŽne the operator, P D P 1 , . . . , Pn .

(B.13)

Note that because of the commutativity of the Pi , the order of multiplication on the right-hand side of equation B.11 is immaterial, and it is easily determined that P is idempotent, P2 D P. By induction on equation B.11, it is readily shown that b1 h M b1 , Qi ¡¢ ¢ ¢ ¡ M b n hM bn , Qi D Q ¡ PQ D Q ¡ M

n X i D1

b i hM bi , Qi. M

(B.14)

Either from the self-adjointness and idempotency of each Pi , or directly from equation B.14, it can be shown that P itself is self–adjoint, P D P¤ , hPQ1 , Q2 i D hQ1 , PQ2 i.

Thus, P is the orthogonal projection operator from Rm£n onto T AC . As a consequence, we have the key result that AP 2 T AC at A

AP D PQ,

,

(B.15)

for some matrix Q 2 Rm£n . This follows from the fact that given Q, then the right-hand side of equation B.6 is true for AP D PQ. On the other hand, P we can take Q D AP if the right-hand side of equation B.6 is true for A, in equation B.15. Note that for the T AF –projection operator L given by equation A.2, we have that P D L P D PL , consistent with the fact that T AC ½ T AF . Let qi , i D 1, . . . , n be the columns of Q D [q1 , . . . , qn ]. The operation Q0 D Pj Q, corresponds to q0i D qi ,

6 iD j,

and

qj0 D (I ¡b ajb ajT ) qj ,

while the operation Q0 D PQ, corresponds to q0i D (I ¡ b aib aTi ) qi ,

i D 1, . . . , n.

392

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

B.2 Learning on the Manifold AC . The development of equations A.6 through A.22 is independent of the precise nature of the constraint manifold A and can be applied here to the speciŽc case of A D A C . To ensure that T2 · 0 in equation A.22 and that AP 2 T AC , we can use the learning rule, Á ! n X P b b (B.16) A D ¡m PdA D ¡m dA ¡ Mi tr MidA , i D1

for m > 0 and dA given by equation A.21. With the ith column of dA denoted by dai , this corresponds to the learning rule, aP i D ¡m ( I ¡b aib aTi )d ai ,

i D 1, . . . , n.

(B.17)

With rule B.16, we have P D ¡m hdA, PdAi D ¡m hdA, P2dAi T 2 D hdA, Ai 2 D ¡m hPdA, PdAi D ¡m kPdAk · 0,

where we have explicitly shown that the idempotency and self-adjointness of P correspond to its being a nonnegative operator. Thus, we will have convergence to the largest invariant set for which AP ´ 0, which from equation B.16 is equivalent to the set for which PdA D dA ¡

n X iD 1

bi h M bi , dAi ´ 0. M

(B.18)

This, in turn, is equivalent to dA D

n X iD 1

bi h M bi , dAi D M

n X i D1

nMi hMi , dAi D

n X

ci M i ,

(B.19)

iD 1

with ci , nhMi , dAi D naTidai ,

i D 1, . . . , n.

An equivalent statement to equation B.19 is dai D ci ai ,

i D 1, . . . , n.

DeŽning the diagonal matrix C D diag[ci , . . . , cn ] and recalling the deŽnitions A.21 and B.1, we obtain from equation B.19 that A Sxx ¡ Syx D dA D AC,

which can be solved as

A D Syx ( Sxx ¡ C) ¡1 .

(B.20)

This is the general form of the solution found by the AC –learning algorithm.

Dictionary Learning Algorithms for Sparse Representation

393

Acknowledgments

This research was partially supported by NSF grant CCR-9902961. K. K.-D. also acknowledges the support provided by the Computational Neurobiology Laboratory, Salk Institute, during a 1998–1999 sabbatical visit. References Adler, J., Rao, B., & Kreutz-Delgado, K. (1996). Comparison of basis selection methods. In Proceedings of the 30th Asilomar Conference on Signals, Systems, and Computers. PaciŽc Grove, CA: IEEE. Attias, H. (1999). Independent factor analysis. Neural Computation, 11(4), 803– 851. Basilevsky, A. (1994). Statistical factor analysis and related methods: Theory and applications. New York: Wiley. Bell, A., & Sejnowski, T. (1995). An information-maximization approach to blind separation and blind deconvolution. Neural Computation, 7, 1129–1159. Coifman, R., & Wickerhauser, M. (1992).Entropy-based algorithms for best basis selection. IEEE Transactions on Information Theory, IT-38(2), 713–718. Comon, P. (1994). Independent component analysis: A new concept? Signal Processing, 36, 287–314. Deco, G., & Obradovic, D. (1996). An information-theoreticapproach to neural computing. Berlin: Springer-Verlag. Dhrymes, P. (1984). Mathematics for econometrics (2nd ed.). New York: SpringerVerlag. Dirac, P. A. M. (1958). The principles of quantum mechanics. Oxford: Clarendon Press. Donoho, D. (1994). On minimum entropy segmentation. In C. Chui, L. Montefusco, & L. Puccio (Eds.), Wavelets:Theory, algorithms,and applications(pp. 233– 269). San Diego, CA: Academic Press. Donoho, D. (1995). De-noising by soft-thresholding. IEEE Trans. on Information Theory, 41, 613–627. Engan, K. (2000). Frame based signal representation and compression. Unpublished doctoral dissertation, Stavanger Universty College, Norway. Engan, K., Rao, B., & Kreutz-Delgado, K. (1999). Frame design using FOCUSS with method of optimal directions (MOD). In Proc. NORSIG-99. Tronheim, Norway: IEEE. Engan, K., Rao, B. D., & Kreutz-Delgado, K. (2000). Regularized FOCUSS for subset selection in noise. In Proc. NORSIG 2000, Sweden, June, 2000 (pp. 247– 250). Kolmarden, Sweden: IEEE. Field, D. (1994). What is the goal of sensory coding. Neural Computation, 6, 559– 599. Girolami, M. (2001). A variational method for learning sparse and overcomplete representations. Neural Computation, 13, 2517–2532. Gorodnitsky, I., George, J., & Rao, B. (1995). Neuromagnetic source imaging with FOCUSS: A recursive weighted minimum norm algorithm. Journal of Electroencephalographyand Clinical Neurophysiology, 95(4), 231–251.

394

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

Gorodnitsky, I., & Rao, B. (1997). Sparse signal reconstruction from limited data using FOCUSS: A re-weighted minimum norm algorithm. IEEE Trans. Signal Processing, 45(3), 600–616. Hansen, P. C., & O’Leary, D. P. (1993). The use of the L-curve in the regularization of discrete ill-posed problems. SIAM J. Sct. Comput., 14, 1487–1503. Hyva¨ rinen, A., Cristescu, R., & Oja, E. (1999). A fast algorithm for estimating overcomplete ICA bases for image windows. In Proc. Int. Joint Conf. on Neural Networks (IJCNN’99) (pp. 894–899). Washington, DC. Kalouptsidis, T., & Theodoridis, S. (1993). Adaptive system identiŽcation and signal processing algorithms. Upper Saddle River, NJ: Prentice Hall. Kassam, S. (1982). Signal detection in non-gaussian noise. New York: SpringerVerlag. Khalil, H. (1996). Nonlinear systems (2nd ed). Upper Saddle River, NJ: Prentice Hall. Kreutz-Delgado, K., & Rao, B. (1997). A general approach to sparse basis selection: Majorization, concavity, and afŽne Scaling (Full Report with Proofs) (Center for Information Engineering Report No. UCSD-CIE-97-7-1). San Diego: University of California. Available on-line at: http://dsp.ucsd.edu/»kreutz. Kreutz-Delgado, K., & Rao, B. (1998a). A new algorithm and entropy-like measures for sparse coding. In Proc. 5th Joint Symp. Neural Computation. La Jolla, CA. Kreutz-Delgado, K., & Rao, B. (1998b). Gradient factorization-based algorithm for best-basis selection. In Proceedings of the 8th IEEE Digital Signal Processing Workshop. New York: IEEE. Kreutz-Delgado, K., & Rao, B. (1998c). Measures and algorithms for best basis selection. In Proceedings of the 1998 ICASSP. New York: IEEE. Kreutz-Delgado, K., & Rao, B. (1999). Sparse basis selection, ICA, and majorization: Towards a uniŽed perspective. In Proc. 1999 ICASSP. Phoenix, AZ. Kreutz-Delgado, K., Rao, B., & Engan, K. (1999). Novel algorithms for learning overcomplete Dictionaries. In Proc. 1999 Asilomar Conference. PaciŽc Grove, CA: IEEE. Kreutz-Delgado, K., Rao, B., Engan, K., Lee, T.-W., & Sejnowski, T. (1999a). Convex/Schur–convex (CSC) log-priors and sparse coding. In Proc. 6th Joint Symposium on Neural Computation. Pasadena, CA. Kreutz-Delgado, K., Rao, B., Engan, K., Lee, T.-W., & Sejnowski, T. (1999b).Learning overcomplete dictionaries: Convex/Schur-convex (CSC) log-priors, factorial codes, and independent/dependent component analysis (I/DCA). In Proc. 6th Joint Symposium on Neural Computation. Pasadena, CA. Lee, T.-W., Girolami, M., & Sejnowski, T. (1999). Independent component analysis using an extended infomax algorithm for mixed sub-gaussian and supergaussian sources. Neural Computation, 11(2), 417–441. Lewicki, M. S., & Olshausen, B. A. (1999). A probabilistic framework for the adaptation and comparison of image codes. J. Opt. Soc. Am. A, 16(7), 1587– 1601. Lewicki, M. S., & Sejnowski, T. J. (2000). Learning overcomplete representations. Neural Computation, 12(2), 337–365. Mallat, S., & Zhang, Z. (1993). Matching pursuits with time-frequency dictionaries. IEEE Trans. ASSP, 41(12), 3397–3415.

Dictionary Learning Algorithms for Sparse Representation

395

Marshall, A., & Olkin, I. (1979). Inequalities: Theory of majorization and its applications. San Diego, CA: Academic Press. Murray, J. F., & Kreutz-Delgado, K. (2001).An improved FOCUSS-based learning algorithm for solving sparse linear inverse problems. In Conference Record of the 35th Asilomar Conference on Signals, Systems and Computers. PaciŽc Grove, CA: IEEE. Nadal, J.-P., & Parga, N. (1994). Nonlinear neurons in the low noise limit: A factorial code maximizes information transfer. Network, 5, 565–581. Olshausen, B., & Field, D. (1996). Emergence of simple-cell receptive Želd properties by learning a sparse code for natural images. Nature, 381, 607–609. Olshausen, B., & Field, D. (1997). Sparse coding with an overcomplete basis set: A strategy employed by V1? Vision Research, 37, 3311–3325. Pham, D. (1996). Blind separation of instantaneous mixture of sources via an independent component analysis. IEEE Trans. Signal Processing, 44(11), 2768– 2779. Popovic, Z., & Sj¨ostrand, J. (2001). Resolution, separation of retinal ganglion cells, and cortical magniŽcation in humans. Vision Research, 41, 1313–1319. Rao, B. (1997). Analysis and extensions of the FOCUSS algorithm. In Proceedings of the 1996 Asilomar Conference on Circuits, Systems, and Computers (Vol. 2, pp. 1218–1223). New York: IEEE. Rao, B. (1998). Signal processing with the sparseness constraint. In Proc. 1998 ICASSP. New York: IEEE. Rao, B. D., Engan, K., Cotter, S. F., & Kreutz-Delgado, K. (2002). Subset selection in noise based on diversity measure minimization. Manuscript submitted for publication. Rao, B., & Gorodnitsky, I. (1997). AfŽne scaling transformation based methods for computing low complexity sparse solutions. In Proc. 1996 ICASSP (Vol. 2, pp. 1783–1786). New York: IEEE. Rao, B., & Kreutz-Delgado, K. (1997). Deriving algorithms for computing sparse solutions to linear inverse problems. In J. Neyman (Ed.), Proceedingsof the 1997 Asilomar Conference on Circuits, Systems, and Computers. New York: IEEE. Rao, B., & Kreutz-Delgado, K. (1998a). Basis selection in the presence of noise. In Proceedings of the 1998 Asilomar Conference. New York: IEEE. Rao, B., & Kreutz-Delgado, K. (1998b). Sparse solutions to linear inverse problems with multiple measurement vectors. In Proceedings of the 8th IEEE Digital Signal Processing Workshop. New York: IEEE. Rao, B., & Kreutz-Delgado, K. (1999). An afŽne scaling methodology for best basis selection. IEEE Trans. Signal Processing, 1, 187–202. Roberts, S. (1998). Independent component analysis: Source assessment and separation, a Bayesian approach. IEE (Brit.) Proc. Vis. Image Signal Processing, 145(3), 149–154. Rockafellar, R. (1970). Convex analysis. Princeton, NJ: Princeton University Press. Ruderman, D. (1994). The statistics of natural images. Network: Computation in Neural Systems, 5, 517–548. Wang, K., Lee, C., & Juang, B. (1997). Selective feature extraction via signal decomposition. IEEE Signal Processing Letters, 4(1), 8–11. Watanabe, S. (1981). Pattern recognition as a quest for minimum entropy. Pattern Recognition, 13(5), 381–387.

396

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

Wickerhauser, M. (1994).Adaptedwavelet analysis from theoryto software. Wellesley, MA: A. K. Peters. Zhu, S., Wu, Y., & Mumford, D. (1997). Minimax entropy principle and its application to texture modeling. Neural Computation, 9, 1627–1660. Zibulevsky, M., & Pearlmutter, B. A. (2001). Blind source separation by sparse decomposition in a signal dictionary. Neural Computation, 13(4), 863–882. Received August 6, 2001; accepted June 4, 2002.

LETTER

Communicated by Christian Wehrhahn

Spatiochromatic Receptive Field Properties Derived from Information-Theoretic Analyses of Cone Mosaic Responses to Natural Scenes Eizaburo Doi [email protected] Institute for Neural Computation, University of California, San Diego, La Jolla, CA 92093, U.S.A., and Graduate School of Informatics, Kyoto University, Kyoto 606-8501, Japan Toshio Inui [email protected] Graduate School of Informatics, Kyoto University, Kyoto 606-8501, Japan Te-Won Lee [email protected] Institute for Neural Computation, University of California, San Diego, La Jolla, CA 92093, U.S.A., and Howard Hughes Medical Institute, Computational Neurobiology Laboratory, Salk Institute, La Jolla, CA 92037, U.S.A. Thomas Wachtler [email protected] Neurobiology and Biophysics, Institute of Biology III, Albert-Ludwigs-Universit¨at, 79104 Freiburg, Germany Terrence J. Sejnowski [email protected] Institute for Neural Computation, University of California, San Diego, La Jolla, CA 92093, U.S.A., Howard Hughes Medical Institute, Computational Neurobiology Laboratory, Salk Institute, La Jolla, CA 92037, U.S.A., and Department of Biology, University of California, San Diego, La Jolla, CA 92093, U.S.A. Neurons in the early stages of processing in the primate visual system efŽciently encode natural scenes. In previous studies of the chromatic properties of natural images, the inputs were sampled on a regular array, with complete color information at every location. However, in the retina cone photoreceptors with different spectral sensitivities are arranged in a mosaic. We used an unsupervised neural network model to analyze the statistical structure of retinal cone mosaic responses to calibrated color natural images. The second-order statistical dependencies derived from the covariance matrix of the sensory signals were removed in the Žrst stage Neural Computation 15, 397–417 (2003)

° c 2002 Massachusetts Institute of Technology

398

E. Doi, T. Inui, T. Lee, T. Wachtler, and T. Sejnowski

of processing. These decorrelating Žlters were similar to type I receptive Želds in parvo- or konio-cellular LGN in both spatial and chromatic characteristics. In the subsequent stage, the decorrelated signals were linearly transformed to make the output as statistically independent as possible, using independent component analysis. The independent component Žlters showed luminance selectivity with simple-cell-like receptive Želds, or had strong color selectivity with large, often double-opponent, receptive Želds, both of which were found in the primary visual cortex (V1). These results show that the “form” and “color” channels of the early visual system can be derived from the statistics of sensory signals. 1 Introduction

New algorithms have recently been developed to test the hypothesis that sensory signals are transformed in the early stages of sensory processing to reduce the redundancy of the inputs (Barlow, 1961; Field, 1994). The spatial properties of ganglion cells in the retina and thalamocortical neurons in the lateral geniculate nucleus (LGN) are consistent with algorithms that use second-order statistics to decorrelate natural scenes (Atick, 1992; Atick & Redlich, 1993) and the localized and oriented simple cells in the primary visual cortex (V1) reduce higher-order statistics (Olshausen & Field, 1996; Bell & Sejnowski, 1997; van Hateren & van der Schaaf, 1998). These studies used gray-scale natural images. Natural color images have also been analyzed using RGB inputs (Tailor, Finkel, & Buchsbaum, 2000; Hoyer & Hyvarinen, 2000) and LMS inputs with the spectral sensitivities of the human L-, M-, and S-cone photoreceptors (Wachtler, Lee, & Sejnowski, 2001). In these studies, the images were sampled on a regular grid, with the color at each location represented as a vector of three elements. In the retina cone photoreceptors are arranged in a mosaic, with only one cone at each location. Thus, chromatic information in the retina is extracted by comparing the responses of different cone types at different locations, which requires spatial interaction. This is not the case for RGB or LMS color images, where the information for chromaticity is fully available at each pixel. Here, we consider a model that incorporates the cone mosaic found in the trichromatic foveal region of primates. We adopt a hierarchical model that consists of a decorrelating stage corresponding to LGN and a subsequent independent component analysis (ICA) stage corresponding to V1 (Bell & Sejnowski, 1997). Applying biological and computational constraints at each level of processing allows a better comparison to the properties of neurons in the early visual system. 2 Methods 2.1 Cone Mosaic Responses. A small cone mosaic patch was used to model the trichromatic foveal region of the primate retina (see Figure 1). To

Information-Theoretic Analyses of Cone Mosaic Responses

399

obtain estimates of the cone mosaic responses to natural scenes, we took natural images using a 3CCD camera and converted them into LMS images (see the appendix for details). By scanning the data set randomly with the cone mosaic, 507,904 samples were generated. 2.2 Model.

2.2.1 Computational Goal. cessing are linear,

We assume that the early stages of visual pro-

u D Wx ,

(2.1)

where x D ( x1 , . . . , xn ) T are the input signals to the visual system (namely, the cone mosaic responses), W D [w 1 , . . . , w n ]T is a square matrix consisting of receptive Želds w i , and u D ( u1 , . . . , un ) T is the output, corresponding to the neural activities (Olshausen & Field, 1996; Bell & Sejnowski, 1997). The computational goal of the system is assumed to reduce the redundancies or statistical dependencies among the output elements. This coding scheme is known as minimum entropy coding or factorial coding (Atick, 1992; Field, 1994) and can be implemented by ICA (Bell & Sejnowski, 1995; Lee, 1998; Girolami, 1999; Hyvarinen, Karhunen, & Oja, 2001). A linear generative model for the input data x is given by x D As D s1 a 1 C ¢ ¢ ¢ C sn a n ,

(2.2)

where A D [a1 , . . . , a n ] is a set of basis functions and s1 , . . . , sn are their coefŽcients, called sources or causes. Given sources s that are statistically independent of each other, equation 2.2 assumes that the input x is generated by a linear mixture of statistically independent causes s . The goal of ICA is to Žnd a matrix W that unmixes x so that u D s, namely, W D A ¡1 (up to permutation and scaling). The assumption of statistical independence among sources s and the general evidence of high redundancy in the sensory input x mean that the basis functions a i introduce statistical dependency in the input. Equation 2.2 means that when a source si is activated (i.e., taking nonzero value), more than one sensor will concurrently have nonzero values according to the pattern of a i , which leads to statistical dependency in the sensory signals x. 2.2.2 Hierarchical Implementation. We followed the approach of Bell and Sejnowski (1997) in decomposing the linear transformation W into decorrelation and ICA, W I W Z (see Figure 2): the Žrst operator W Z decorrelates the input x such that the covariance matrix of its output u Z satisŽes hu Z u Z T i D h( W Z x ) ( W Z x ) T i D I,

(2.3)

400

E. Doi, T. Inui, T. Lee, T. Wachtler, and T. Sejnowski

Information-Theoretic Analyses of Cone Mosaic Responses

401

Figure 2: Three-stage model of the early visual system. This is three-layer feedforward neural network. The Žrst stage corresponds to the photoreceptor layer in the retina, as shown in Figure 1. The subsequent stages are assumed to reduce redundancy in the sensory input in a sequential manner.

where I is the identity matrix. This decorrelation normalizing the output variance is also known as the whitening transformation or zero-phase Žlters (Bell & Sejnowski, 1997). The second operator W I makes the decorrelated signals W Z x to be statistically as independent as possible (ICA for whitened data). This decomposition is justiŽed as follows. First, the overlap of cone

Figure 1: Facing page. Properties of the input. (a) Cone mosaic model. This is based on the anatomical data from old world monkeys. It consists of 217 photoreceptors of L- (red), M- (green), and S-cone (blue). L- and M-cones are arranged randomly (Mollon & Bowmaker, 1992), whereas S-cones are arranged regularly (de Monasterio, McCrane, Newlander, & Schein, 1985; Szel, Diamantstein, & Rohlich, 1988). The cone ratio is L : M D 1 : 1 (Mollon & Bowmaker, 1992; Baylor et al., 1987), and S-cone occupies about 3% (de Monasterio et al., 1985; Mollon & Bowmaker, 1992). In this model L : M : S D 112 : 98 : 7. (b) Spectral sensitivities of cone photoreceptors. Here we used those of human cone photoreceptors (Stockman & Sharpe, 2000). (c–e) An example of cone mosaic output compared with RGB and gray-scale natural images. (c) RGB image. (d) Gray-scale image. M-cone responses are shown for reference. Larger responses correspond to brighter pixels. (e) Cone mosaic responses. The red hexagon shows the window of the cone mosaic shown in a; thus, the image inside is the response pattern of the cone mosaic. This image was prepared by concatenating several response patches of the same cone mosaic. The pixels are arranged in the hexagonal grid instead of a square grid to mimic the closely packed structure of the retinal cone mosaic as shown in a. They are generated by averaging four pixels of square-grid images with one pixel offset between each row. (f) Examples of natural images. Sixty-two natural images were used.

402

E. Doi, T. Inui, T. Lee, T. Wachtler, and T. Sejnowski

spectral sensitivities (especially between L- and M-cones) produces a strong correlation between elements of the input, as is the case of gray-scale natural images. This correlation could be removed as a Žrst step toward a statistically independent representation. This operation restricts the solution of the next stage W I to be in the orthogonal matrix manifold. Second, spatial properties of receptive Želds of retinal ganglion cells and LGN cells have been suggested to be explained by decorrelation, using gray-scale data sets (Atick, 1992; Atick & Redlich, 1993; Bell & Sejnowski, 1997). 2.2.3 Learning Algorithms. W Z is a whitening matrix that satisŽes equation 2.3 with an additional degree of freedom in the choice of whitening matrix: if W Z is a whitening matrix, UW Z is also a whitening matrix for arbitrary orthogonal matrices U . Here we assume that the whitening matrix is symmetric, W Z T D W Z , which makes it unique. For gray-scale natural images, this whitening yields a set of zero-phase Žlters, which atten the spatial-frequency (1/f) spectrum of shift-invariant data and show concentric center-surround receptive Želd organization (Atick, 1992; Atick & Redlich, 1993). Such a matrix can be derived by matrix square root of the inverse of the input covariance matrix (Golub & Loan, 1996; Bell & Sejnowski, 1997) or using a more biologically plausible learning algorithm (Atick & Redlich, 1993). Since both are equivalent, we used the former method for convenience. The ICA algorithm we used can be derived using information maximization or maximum likelihood estimation (Bell & Sejnowski, 1995; Cardoso, 1997) with the natural gradient (Amari, Cichocki, & Yang, 1996),

D W I / [I ¡ Q ( u ) u ] W I ,

(2.4)

( )

@ log p u where Q (u ) D ¡ @u , p ( u ) D P i p (ui ) is the probability density of u , and D W I is the change of weights computed iteratively until it converges to zero. Note that it requires a density model of p ( ui ) . We use the extended infomax algorithm for ICA that assumes different p (ui ) ’s for supergaussian and subgaussian density (Lee, Girolami, & Sejnowski, 1999). Thus, the results do not require the coefŽcients to have sparse distributions, unlike some previous methods (Olshausen & Field, 1996; Bell & Sejnowski, 1997).

3 Results 3.1 Decorrelating Stage. All units in the decorrelating stage showed concentric center-surround receptive Želd organization with the center region driven by a single cone (see Figure 3). Receptive Želds of type I neurons of parvo- or konio-cellular LGN (pLGN/kLGN) of the macaque have these properties (Lennie & D’Zmura, 1988; in their review, as in this study, the properties of retinal ganglion cells were not distinguished from those of LGN cells). They can be further classiŽed on the basis of the center-driving

Information-Theoretic Analyses of Cone Mosaic Responses

403

Figure 3: Receptive Želds of the decorrelating stage. Each panel shows a weighting pattern (receptive Želd organization) from cone mosaic to a certain decorrelating unit. Here we show two examples for each L-, M-, and S-center type. Bright and dark circles show positive and negative weights; red, green, and blue colors correspond to weights of L-, M-, and S-cone; and the circle size indicates the amplitude of the weight.

cone type, leading to three types of units that we refer to as L-center, Mcenter, and S-center. All units were classiŽed as color selective unit using the criterion in Hanazawa, Komatsu, and Murakami (2000): the Color Selectivity Index (CSI) greater than 0.5 (see Figure 4b). 1 This matches the lack of luminance or achromatic units in pLGN/kLGN (Lennie & D’Zmura, 1988; Hanazawa et al., 2000). Figure 4a illustrates color selectivity proŽles in comparison with those of pLGN/kLGN neurons, showing agreement between the model and the physiological data. Figure 4c is the histogram of their color tuning directions, segregated in three clusters as in the physiological data. The color tuning directions of the L-center and M-center types were almost completely opposite (¡6.3 § 13.5 and 180.8 § 1.9, respectively). This is also conŽrmed in Figure 4a, where contours of the responses were aligned in the same orientation but in the reverse order. This indicates that both the L- and M-center types were responsible for the same r-g channel, 1 (Color Selectivity Index) = [(max response) ¡ (min response)]/[max abs(response)]. In the physiological study of Hanazawa et al. (2000), the response is deŽned by [mean discharge ratio] ¡ [baseline activity]. Therefore, negative values are possible.

404

E. Doi, T. Inui, T. Lee, T. Wachtler, and T. Sejnowski

Information-Theoretic Analyses of Cone Mosaic Responses

405

while the S-center type was responsible for the y-b channel, in the color opponent stage. Regarding L-center and M-center type, there was no signiŽcant conetype speciŽcity in the receptive Želd organization. In fact, they showed close similarity to the whitening Žlters derived from a single cone-type cone mosaic (here we used M-cone, and as a result the data are gray-scale): w Z T w Zgray / kw Z kkw Zgray k D 0.995 § 0.005. This suggests that their receptive Želd organization is determined mainly by spatial factor, not by cone-type dependent chromatic factor. This also means that the signiŽcant color selectivity of L- and M-center type is mainly due to the weighting ratio between the center and the surround, itself mainly due to the spatial factor, not the cone-type speciŽc, antagonistic wiring. These results would be reasonable since the majority of the input signals come from L- and M-cones, of which spectral sensitivities are highly overlapping. Therefore, the statistics of cone mosaic data are close to those of gray-scale (only M-cone) data. S-center type shows S-cone speciŽcity with larger receptive Želds than Land M-center types (see Figure 3). This is consistent with the large dendritic arborization of the small bistratiŽed retinal ganglion cells, which are distinct from midget and parasol ganglion cells and belong to the konio-cellular (Scone) pathway (Dacey, 2000). Our results show that the S-cone weights in the surround are inhibitory. This is reasonable since the difference of correlated signals is oriented along the uncorrelated axis. The larger size of S-center

Figure 4: Facing page. Chromatic properties of the decorrelating stage. (a) Color selectivity. Model data (upper row) were derived from the same units illustrated in the lower row of Figure 3. Physiological data (lower row) are of pLGN/kLGN neurons. This diagram shows the responses of a certain unit to a set of color stimuli distributed in both MacLeod-Boynton (MB) and CIE 1931 xy (CIE-xy) chromaticity coordinates. For illustration purposes, a CIE-xy chromaticity diagram was used. Open and Žlled circles show positive and negative responses, respectively; their diameters indicate response amplitudes; C corresponds to the chromaticity of the color stimuli. It is important to note that all color stimuli were adjusted to have the same luminance (isoluminant stimuli), and thus any difference among the responses of a certain unit can be attributed to its color selectivity. Contours of the responses are also plotted: from thick to thin, 80%, 60%, 40%, and 20% of [maximum response] ¡ [minimum response]. Axes of color orientations deŽned in MB chromaticity coordinates are shown as well (0, 90, 180, 270 degrees). For the analysis of second-layer units, the stimuli were a uniform pattern to mimic diffuse light in the physiological experiment. (b) Distribution of color selectivity index. (c) Distribution of color tuning directions. This direction is deŽned by the maximum slope of the responses. Physiological data are of pLGN/kLGN. Color of the histogram is identical to that of b. All physiological data are from Hanazawa, Komatsu, and Murakami (2000).

406

E. Doi, T. Inui, T. Lee, T. Wachtler, and T. Sejnowski

type receptive Želds should be due to the sparse arrangement of S-cone in the cone mosaic, since in the case of L- and M-center type, the cones next to the center respond in a highly correlated way to the center cone. While the S-cone antagonism between the center and the surround is reasonable from the redundancy reduction point of view, physiological studies showed only the spatially coextensive yellow-blue cells in retinal ganglion cells (Dacey, 2000). 3.2 ICA Stage. In the ICA stage, two main types of receptive Želds emerged (see Figures 5 and 6): the majority (94.9%) of luminance type (achromatic, color nonselective) and the others (4.6%) of color-selective type. Note that this analysis was based on W , not W I . The luminance unit was similar to those derived using gray-scale samples, showing simple-cell-like receptive Želd with high spatial-frequency selectivity (see Figure 5a; Bell & Sejnowski, 1997; van Hateren & van der Schaaf, 1998). This suggests that the luminance units would represent spatial information (without color). Responses to isoluminant color stimuli are almost constant (see Figure 6a), and the color selectivity index was less than 0.5 for almost all luminance units (0.28 § 0.14; see Figure 6b), showing low color selectivity. The low value of this index means that the spectral selectivity was almost identical to the luminosity function Vl ,2 consistent with physiological data from simple cells in V1 (Lennie & D’Zmura, 1988; Lennie, Krauskopf, & Sclar, 1990). The color-selective units were characterized by cone-type speciŽc antagonistic regions in their receptive Želd, and classiŽed into two subtypes: Y/B type of yellow-blue selectivity and R/G type of red-green selectivity. The

Figure 5: Facing page. Receptive Želds of the ICA stage. (a) Receptive Želd organization. Here, each panel shows a weighting pattern from cone mosaic to a certain ICA unit. Two examples for each luminance, Y/B, and R/G type are shown. The optimal bar stimulus for the units is shown just below each receptive Želd. The optimal bar is deŽned by the bar that maximally matches the weighting pattern. A set of bars was prepared with 13 width, 1 » 3 length, and 22 orientation for arbitrary location. To Žnd the color-opponent region for color-selective type, weighting patterns were modiŽed as follows: C L C M ¡ S for Y/B type; C L ¡ M for R/G type (S-cone is neglected). (b) Basis function. These are the dual vectors of Žlters shown in a. 2

Due to the use of isoluminant color stimuli, the responses of luminosity function to the color stimuli are identical. The reverse is also true: any spectral sensitivity is written as the sum of luminosity function and R its residual, S (l) R D aVl (l) C D (l), R and the response to an isoluminant stimulus i(l) is S(l)i(l)dl D a Vl (l)i(l)dl C D (l)i (l)dl, where R R R a ´ S(l)i(l)dl / Vl (l)i(l)dl and a Vl (l)dl is constant by deŽnition. If the left side is constant over any isoluminant color stimuli, D (l) D 0; thus, S(l) D aVl (l).

Information-Theoretic Analyses of Cone Mosaic Responses

407

408

E. Doi, T. Inui, T. Lee, T. Wachtler, and T. Sejnowski

Figure 6: Chromatic properties of the ICA stage. (a) Color selectivity, derived from the units shown in the middle row of Figure 5a. For physiological data, see Figure 3A in Hanazawa et al. (2000) for R/G type; data corresponding to luminance or Y/B unit are not illustrated there, although they were reported. (b) Distribution of color selectivity index. (c) Distribution of color tuning direction. Here, those units of which CSI is greater than 0.5 are included in the analysis in accordance with the physiological study. Color of the histogram is identical to that of b. All physiological data are taken from Hanazawa et al. (2000).

Information-Theoretic Analyses of Cone Mosaic Responses

409

color selectivity index was much larger than 0.5 (larger than 1) for all colorselective units (1.43 § 0.16 for Y/B, 1.70 § 0.01 for R/G; see also Figure 6b), indicating that they are highly color selective. Note that CSI > 1 indicates the existence of color opponency, which yields positive response to one color and negative response to another. These color-selective subtypes may correspond to the two color channels suggested in psychophysical studies (Lennie & D’Zmura, 1988). However, it does not match the physiological studies that showed distributed tuning over the color axis (see the right panel in Figure 6c; see also Figure 2 in De Valois, Cottaris, Elfar, Mahon, & Wilson, 2000). Most color-selective units showed orientation selectivity. Although the orientation selectivity of color-selective cells in V1 is controversial (Michael, 1978; Lennie et al., 1990; Conway, 2001; Johnson, Hawken, & Shapley, 2001), our results suggest that they may be expected in redundancy reduction of cone mosaic responses. In addition to luminance units and color-selective units, we obtained a basis function showing a uniform pattern over the cone mosaic, referred to as the DC unit. Its receptive Želd organization was affected by the boundary effect. In a previous study, this component was excluded from the analysis by subtracting the average for each image patch (Hoyer & Hyvarinen, 2000). Color-selective units had a larger receptive Želd compared to luminance units: the size of the optimal bar is 4.42 § 3.41 (cone size) for luminance units and 36.30 § 6.73 and 74.62 § 25.52 for Y/B and R/G units. This is consistent with the low spatial-frequency selectivity of the color mechanism (Mullen, 1985), as expected, since the color information is derived from the comparison between at least two or three types of cones, but there is only a single cone at each retinal location. Thus, chromatic information requires averaging over several cones, leading to lower spatial-frequency selectivity. Note also that the percentage of the color-selective units was signiŽcantly lower than that of luminance units, indicating that the dimensionality assigned for chromatic information is much less than that for luminance information. Our analysis thus far of properties of receptive Želds w i suggests that ICA yields two types of units: luminance units responsible for encoding spatial information without color and color-selective units responsible for color information without Žne spatial information. We can rewrite the datagenerative model of the cone mosaic responses (see equation 2.2) as xD

X i2Form

ui a i C

X

uj aj ,

(3.1)

j2Color

where we assumed that Form consists of luminance and DC units and Color consists of color-selective units. Recall that the response ui represents the presence of the basis function a i in the input x. Some examples of the basis function are shown in Figure 5b. Regarding luminance units, the basis functions show cone-type irrespective edge pattern, exactly as the case of grayscale images. Note that even the locations of S-cone are Žlled in as if they

410

E. Doi, T. Inui, T. Lee, T. Wachtler, and T. Sejnowski

were part of the edge surface, while S-cone scarcely contributes their receptive Želd organization. Regarding color-selective units, the basis functions show cone-type speciŽc patterns, that is, color information. These suggest that the statistically independent patterns in the cone mosaic responses are a large number of cone-type unspeciŽc edge patterns and a small number of cone-type speciŽc edge patterns, along with a spatially uniform pattern for each type. In Figure 7, we visualize these different types of information by separate images (Color is divided into R/G and Y/B for convenience). It is clear that the form information is extracted by luminance and DC units (see Figure 7c), and color information is extracted by Y/B (see Figure 7d) and R/G units (see Figure 7e) from the cone mosaic responses (see Figure 7b). In this study, all resulting independent components had supergaussian distributions. When histogram equalization was used as a model for cone nonlinearity instead of the physiological model (see the appendix), one subgaussian component emerged whose basis function showed a DC component. This is consistent with other studies using gray-scale images (van Hateren & van der Schaaf, 1998) or LMS images (Wachtler et al., 2001), both of which applied the logarithm function as the cone nonlinearity. 3.3 PCA Comparison. The cone mosaic responses were also analyzed by principal component analysis (PCA) to compare the results with those derived from ICA. As shown in Figure 8, PCA yielded nonlocal Žlters as in the case of gray-scale images (Olshausen & Field, 1996; Bell & Sejnowski, 1997). There were only two components that showed cone-type speciŽcity; both showed S-cone speciŽcity. This means that the spatial variation (variance)

Figure 7: Facing page. Illustration of information content in Luminance, Y/B, or R/G channel. (a) RGB image. (b) Cone mosaic responses. This is the same as Figure 1e but for another natural scene. (c) Reconstruction from luminance and DC units. For each small patch of cone mosaic frame, the image xO was P reconstructed by xO D i ui a i , where i indicates luminance and DC units. These patches were tiled over the whole image. (d) Reconstruction from Y/B units. As in c above, but i indicates Y/B units. We also postprocessed the image to represent it using pseudocolor: L- and M-cones are regarded as contributing yellow direction (i.e., R- and G-pixels with like sign; B-pixel with opposite sign); S-cone is regarded as blue direction (B-pixel with like sign; R- and G-pixels with opposite sign). To avoid dark pixels, with color hard to see, the base color is set to be gray. (e) Reconstruction from R/G units. As in d, except that i indicates R/G units. In postprocessing, L- and S-cones are regarded as contributing red direction (i.e., R-pixel with like sign, G-pixel with opposite sign); M-cone is regarded as green direction (G-pixel with like sign; R-pixel with opposite sign). Any B-pixel is set as 0. As in d, the base color is set gray. (f) Superimposing luminance, R/G, and Y/B information above to see form and color information simultaneously.

Information-Theoretic Analyses of Cone Mosaic Responses

411

common with all cone-type dominates the chromatic variation speciŽc for each cone type (especially for L- and M-cone type). When LMS natural images are analyzed by PCA, Y/B and R/G components were as common as black-white components (Ruderman, Cronin, & Chiao, 1998). This demonstrates that it is not straightforward for the visual system to extract color information from the cone mosaic responses.

412

E. Doi, T. Inui, T. Lee, T. Wachtler, and T. Sejnowski

Figure 8: Principal components of cone mosaic responses. Here we show the 1st, 5th, 11th, 18th, 25th, and 120th principal component, as indicated for each panel. Here, cone type is not indicated by color, but otherwise these are the same illustrations as Figures 3 and 5a. Cone-type speciŽcity can be seen in only two units that show S-cone speciŽcity; the eighteenth component is one of them. The Žrst component is exactly the same as the DC type basis function in the ICA stage.

4 Discussion 4.1 Multiplexing and Separating Stages. It has been suggested that in the pLGN/kLGN, luminance information is conveyed with red-green information in the r-g opponent channel and that these “multiplexed” signals are “separated” into red-green and luminance information in the following stages, possibly in V1 (Ingling & Martinez-Uriegas, 1983; Lennie & D’Zmura, 1988; Lennie et al., 1990). Our results reproduce this two-stage model. We emphasize that this is an emergent property derived from redundancy reduction and not put in by hand as in the previous model studies (Billock, 1991; De Valois & De Valois, 1993; Kingdom & Mullen, 1995). These results suggest mechanisms by which the visual system could reduce redundancy in a sequence of stages. 4.2 Learning Visual Information Processing. It is possible that the color selectivity properties of neurons in the visual system are the end result of

Information-Theoretic Analyses of Cone Mosaic Responses

413

evolution, and during development they are implemented without visual experience. All of the properties of the cells in our model were obtained with unsupervised algorithms. This suggests that some of the properties of cells in both pLGN/kLGN and V1 could be acquired by unsupervised learning during development from visual input. Other properties of cortical neurons, such as orientation and direction selectivity, depend on the visual experience during development (Blakemore & Cooper, 1970; Blakemore & van Sluyters, 1975). Studies of natural scene statistics also support this idea (Olshausen & Field, 1996; Bell & Sejnowski, 1997; van Hateren & van der Schaaf, 1998). There are several psychophysical studies investigating the plasticity of color vision during development (Teller, 1998; Knoblauch, VitalDurand, & Barbur, 2001); however, the physical basis of plasticity in color vision has not yet been examined. Our results suggest that color selectivity, like orientation selectivity, may depend on the statistics of sensory signals. They also suggest that in general, the visual coding of attributes such as form and color (Livingstone & Hubel, 1988) can be understood as a result of redundancy reduction. 4.3 Relation to Previous Studies. Other studies applying ICA to RGB images (Tailor et al., 2000; Hoyer & Hyvarinen, 2000) used spectral sensitivities unlike those found in the human retina, which precludes direct comparisons to physiological or psychophysical data. We used spectral sensitivities of human cone photoreceptors, enabling us to obtain results under more realistic conditions. A similar approach was used earlier (Wachtler et al., 2001; Lee, Wachtler, & Sejnowski, 2002), but without sampling of a trichromatic cone mosaic. We examined the additional constraint that spatial separability of chromatic signals imposes on the human visual system and found that in contrast with the results of previous studies, (1) the cone-type speciŽcities for color vision can be learned under the realistic and difŽcult condition due to the cone mosaic sampling; (2) the spectral sensitivity of simple-cell-like units is along the luminance axis, not the black-white axis; (3) the percentage of luminance (simple-cell like) units is large ( > 90%), not low (about 1/3); and (4) the luminance units show higher spatial-frequency selectivity than color-selective units. 5 Conclusion

The goal of this study was to analyze the cone mosaic responses to natural scenes and to Žnd an efŽcient coding of spatiochromatic signals in the visual system. Our results suggest that a hierarchical redundancy reduction model explains the spatiochromatic properties of neurons in the early stages of processing in the visual system. Although our model used more realistic inputs than previous approaches (Tailor, et al., 2000; Hoyer & Hyvarinen, 2000; Wachtler et al., 2001), it still assumes a linear transformation. Thus, the current model cannot capture

414

E. Doi, T. Inui, T. Lee, T. Wachtler, and T. Sejnowski

complex properties such as nonlinear combination of cone signals (De Valois et al., 2000) or contextual interactions (Wachtler, Sejnowski, & Albright, 1999). Our model nevertheless approximates the early processing stages of the visual system. The similarities between our results and the known properties of the visual system further strengthen the hypothesis that redundancy reduction is an organizing principle in the visual system. Appendix: Preparation of Cone Responses to Natural Scenes

Images of 62 natural scenes was obtained using 3CCD digital still camera system (HC-2500, FujiŽlm, Japan). Each RGB image (1000 £ 1280 pixels, 10 bits for each R, G, B) were corrected for gamma calibration, effect of iris size, and exposure time. These linear RGB data, of which spectral sensitivities were also measured, were transformed into LMS cone responses using a 3£3 matrix. This matrix was derived by minimizing square errors for 170 surface reectance functions (Vrhel, Gershon, & Iwan, 1994) rendered by four kinds of CIE daylight functions (D50, D55, D65, and D75). The resulting errors were 0.23%, 0.14%, and 0.04% for L-, M-, and S-cone, respectively. Its approximation accuracy was also evaluated by a different set of reectance functions (ColorCheker, Macbeth, U.S.A.), and the estimation errors were reasonably small as 0.39%, 0.23%, and 0.03% for L-, M-, and S-cone, respectively. These linear cone response data were further transformed with an empirical cone nonlinear function (Baylor, Nunn, & Schnapf, 1987) given by rnl D 1 ¡ exp(¡k ¢ rl ) ,

(A.1)

where rl is linear cone responses, rnl is nonlinearly transformed cone responses, the range of which is [0,1], and the parameter k is determined so that the median of rl is 0.5 for each scene and for each cone type. We did not apply any preprocessing of the data such as low-pass Žltering (Olshausen & Field, 1996) or dimensionality reduction using PCA (van Hateren & van der Schaaf, 1998). Acknowledgments

E.D. was supported by JSPS Research Fellowships for Young Scientists (no. 200103140), T.I. was supported by Grant-in-Aid for ScientiŽc Research (A)(2) from MEXT (no. 11309007), and T.J.S. was supported by the Howard Hughes Medical Institute. We thank Akitoshi Hanazawa and Hidehiko Komatsu for allowing us to use their Žgures and for their helpful comments, Toshikazu Wada for help with the camera calibration, Kyoto Botanical Gardens for assistance with natural image collection, and Greg Horwitz for his helpful comments on the manuscript. Materials such as color natural images and corresponding cone responses, and the results of all units of decorrelation, ICA, and PCA are available on-line at http:/ / inc2.ucsd.edu/»edoi/pub/.

Information-Theoretic Analyses of Cone Mosaic Responses

415

References Amari, S., Cichocki, A., & Yang, H. (1996). A new learning algorithm for blind signal separation. In D. Touretzky, M. Mozer, & M. Hasselmo (Eds.), Advances in neural information processing systems, 8, (pp. 757–763). Cambridge, MA: MIT Press. Atick, J. J. (1992). Could information theory provide an ecological theory of sensory processing? Network, 3, 213–251. Atick, J. J., & Redlich, A. N. (1993). Convergent algorithm for sensory receptive Želd development. Neural Computation, 5, 45–60. Barlow, H. B. (1961). Possible principles underlying the transformation of sensory messages. In W. A. Rosenblith (Ed.), Sensory communication (pp. 217– 234). Cambridge, MA: MIT Press. Baylor, D. A., Nunn, B. J., & Schnapf, J. L. (1987). Spectral sensitivity of cones of the monkey macaca fascicularis. Journal of Physiology, 390, 145–160. Bell, A. J., & Sejnowski, T. J. (1995). An information-maximization approach to blind separation and blind deconvolution. Neural Computation, 7, 1129– 1159. Bell, A. J., & Sejnowski, T. J. (1997). The independent components of natural scenes are edge Žlters. Vision Research, 37, 3327–3338. Billock, V. A. (1991). The relationship between simple and double-opponent cells. Vision Research, 31, 33–42. Blakemore, C., & Cooper, G. F. (1970). Development of the brain depends on the visual environment. Nature, 228, 477–478. Blakemore, C., & van Sluyters, R. C. (1975). Innate and environmental factors in the development of the kitten’s visual cortex. Journal of Physiology, 248, 663–716. Cardoso, J. F. (1997). Infomax and maximum likelihood for source separation. IEEE letters on Signal Processing, 4, 112–114. Conway, B. R. (2001). Spatial structure of cone inputs to color cells in alert macaque primary visual cortex (v-1). Journal of Neuroscience, 21, 2768–2783. Dacey, D. M. (2000). Parallel pathways for spectral coding in primate retina. Annual Review of Neuroscience, 23, 743–775. de Monasterio, F. M., McCrane, E. P., Newlander, J. K., & Schein, S. J. (1985). Density proŽle of blue-sensitive cones along the horizontal meridian of macaque retina. Investigative Ophthalmology and Visual Science, 26, 289–302. De Valois, R. L., Cottaris, N. P., Elfar, S. D., Mahon, L. E., & Wilson, J. A. (2000). Some transformations of color information from lateral geniculate nucleus to striate cortex. PNAS, 97, 4997–5002. De Valois, R. L., & De Valois, K. K. (1993). A multi-stage color model. Vision Research, 33, 1053–1065. Field, D. J. (1994). What is the goal of sensory coding? Neural Computation, 6, 559–601. Girolami, M. (1999). Self-organizing neural networks: Independent component analysis and blind source separation. New York: Springer-Verlag. Golub, G. H., & Loan, C. F. (1996). Matrix computation (3rd ed.). Baltimore: Johns Hopkins University Press.

416

E. Doi, T. Inui, T. Lee, T. Wachtler, and T. Sejnowski

Hanazawa, A., Komatsu, H., & Murakami, I. (2000). Neural selectivity for hue and saturation of colour in the primary visual cortex of the monkey. European Journal of Neuroscience, 12, 1753–1763. Hoyer, P. O., & Hyvarinen, A. (2000). Independent component analysis applied to feature extraction from color and stereo images. Network, 11, 191–210. Hyvarinen, A., Karhunen, J., & Oja, E. (2001). Independent component analysis. New York: Wiley. Ingling, C. R. Jr., & Martinez-Uriegas, E. (1983). The relationship between spectral sensitivity and spatial sensitivity for the primate r-g X-channel. Vision Research, 23, 1495–1500. Johnson, E. N., Hawken, M. J., & Shapley, R. (2001). The spatial transformation of color in the primary visual cortex of the macaque monkey. Nature Neuroscience, 4, 409–416. Kingdom, F. A. A., & Mullen, K. T. (1995). Separating colour and luminance information in the visual system. Spatial Vision, 9, 191–219. Knoblauch, K., Vital-Durand, F., & Barbur, J. L. (2001). Variation of chromatic sensitivity across the life span. Vision Research, 41, 23–36. Lee, T.-W. (1998). Independent component analysis:Theory and applications. Norwell, MA: Kluwer. Lee, T.-W., Girolami, M., & Sejnowski, T. J. (1999). Independent component analysis using an extended infomax algorithm for mixed subgaussian and supergaussian sources. Neural Computation, 11, 417–441. Lee, T.-W., Wachtler, T., & Sejnowski, T. J. (2002). Color opponency is an efŽcient representation of spectral properties in natural scenes. Vision Research, 42, 2095-2103. Lennie, P., & D’Zmura, M. (1988). Mechanisms of color vision. CRC Critical Reviews on Neurobiology, 3, 333–400. Lennie, P., Krauskopf, J., & Sclar, G. (1990). Chromatic mechanisms in striate cortex of macaque. Journal of Neuroscience, 10, 649–669. Livingstone, M., & Hubel, D. (1988). Segregation of form, color, movement, and depth: Anatomy, physiology, and perception. Science, 240, 740–749. Michael, C. R. (1978). Color vision mechanisms in monkey striate cortex: Simple cells with dual opponent-color receptive Želds. Journal of Neurophysiology,41, 1233–1249. Mollon, J. D., & Bowmaker, J. K. (1992). The spatial arrangement of cones in the primate fovea. Nature, 360, 677–679. Mullen, K. T. (1985). The contrast sensitivity of human color vision to red-green and blue-yellow chromatic gratings. Journal of Physiology, 359, 381–400. Olshausen, B. A., & Field, D. J. (1996). Emergence of simple-cell receptive Želd properties by learning a sparse code for natural images. Nature, 381, 607–609. Ruderman, D. L., Cronin, T. W., & Chiao, C.-C. (1998). Statistics of cone responses to natural images: Implications for visual coding. Journal of the Optical Society of America A, 15, 2036–2045. Stockman, A., & Sharpe, L. T. (2000). Spectral sensitivities of the middle- and long-wavelength sensitive cone derived from measurements in observers of known genotype. Vision Research, 40, 1711–1737.

Information-Theoretic Analyses of Cone Mosaic Responses

417

Szel, A., Diamantstein, T., & Rohlich, P. (1988). IdentiŽcation of the blue-sensitive cones in the mammalian retina by anti-visual pigment antibody. Journal of Comparative Neurology, 273, 593–602. Tailor, D. R., Finkel, L. H., & Buchsbaum, G. (2000). Color-opponent receptive Želds derived from independent component analysis of natural images. Vision Research, 40, 2671–2676. Teller, D. Y. (1998). Spatial and temporal aspects of infant color vision. Vision Research, 38, 3275–3282. van Hateren, J. H., & van der Schaaf, A. (1998). Independent component Žlters of natural images compared with simple cells in primary visual cortex. Proceedings of Royal Society of London B, 265, 359–366. Vrhel, M. J., Gershon, R., & Iwan, L. S. (1994). Measurement and analysis of object reectance spectra. Color Research and Application, 19, 4–19. Wachtler, T., Lee, T.-W., & Sejnowski, T. J. (2001). Chromatic structure of natural scenes. Journal of Optical Society of America A, 18, 65–77. Wachtler, T., Sejnowski, T. J., & Albright, T. D. (1999). Interactions between stimulus and background chromaticities in macaque primary visual cortex. Investigative Ophthalmology and Visual Science, 40, 641. Received April 23, 2002; accepted July 23, 2002.

LETTER

Communicated by Erkki Oja

Linear Geometric ICA: Fundamentals and Algorithms Fabian J. Theis [email protected] Institute of Biophysics, University of Regensburg, Germany Andreas Jung [email protected] Institute for Theoretical Physics, University of Regensburg, Germany Carlos G. Puntonet [email protected] Elmar W. Lang elmar.lang@biologie,uni-regensburg.de Department of Architecture and Computer Technology, University of Granada, Spain Geometric algorithms for linear independent component analysis (ICA) have recently received some attention due to their pictorial description and their relative ease of implementation. The geometric approach to ICA was proposed Žrst by Puntonet and Prieto (1995). We will reconsider geometric ICA in a theoretic framework showing that Žxed points of geometric ICA fulŽll a geometric convergence condition (GCC), which the mixed images of the unit vectors satisfy too. This leads to a conjecture claiming that in the nongaussian unimodal symmetric case, there is only one stable Žxed point, implying the uniqueness of geometric ICA after convergence. Guided by the principles of ordinary geometric ICA, we then present a new approach to linear geometric ICA based on histograms observing a considerable improvement in separation quality of different distributions and a sizable reduction in computational cost, by a factor of 100, compared to the ordinary geometric approach. Furthermore, we explore the accuracy of the algorithm depending on the number of samples and the choice of the mixing matrix, and compare geometric algorithms with classical ICA algorithms, namely, Extended Infomax and FastICA. Finally, we discuss the problem of high-dimensional data sets within the realm of geometrical ICA algorithms. 1 Introduction

Given a random vector, independent component analysis (ICA) tries to Žnd its statistically independent components. This idea can also be used to solve the blind source separation (BSS) problem, which is, given only Neural Computation 15, 419–439 (2003)

° c 2002 Massachusetts Institute of Technology

420

F. Theis, A. Jung, C. Puntonet, and E. Lang

the mixtures of some underlying independent source signals, to separate the mixed signals—henceforth called sensor signals—thus recovering the original sources. In contrast to correlation-based transformations such as principal component analysis (PCA), ICA renders the output signals as statistically independent as possible by evaluating higher-order statistics. The idea of ICA was Žrst expressed by Jutten, HÂerault, Comon, and Sorouchiary (1991), while the term ICA was later coined by Comon (1994). However, the Želd became popular only with the seminal paper by Bell and Sejnowski (1995), who elaborated on the Infomax principle Žrst advocated by Linsker (1989, 1992). Many other ICA algorithms have been proposed, with the FastICA algorithm (Hyv¨arinen, 1999) being the most efŽcient among them. Recently, geometric ICA algorithms have received further attention due to their relative ease of implementation (Puntonet & Prieto, 1995). They have been applied successfully to the analysis of real-world biomedical data (Bauer, Puntonet, Rodriguez-Alvarez, & Lang, 2000) and have been extended as well to nonlinear ICA problems (Puntonet, Alvarez, Prieto, & Prieto, 1999). 2 Basics

In linear BSS, a random vector X: V ! Rn called sensor signal is given; it originates from an unknown independent random vector S: V ! Rn , which will be denoted as source signal, via a mixing process with an unknown mixing matrix A 2 Gl(n), that is, X D A ± S. Note that we assume as many sensors as sources. Here, V denotes a Žxed probability space and Gl(n) :D 6 fW 2 Mat(n £ nI R ) | det(W) D 0g the general linear group in Rn . Only the sensor signal is observable, and the task is to recover A and thereby S D A ¡1 ± X. Uniqueness of the solutions can be acertained if we allow at most one of the source variables Si :D p i ± S, where p i : Rn ! R denotes the projection on the ith coordinate, to be gaussian. Then any solution to the BSS problem, that is, any B 2 Gl(n) such that B ± X is independent, is equivalent to A ¡1 , where equivalent means that B can be written as B D LPA ¡1 with an invertible diagonal matrix (scaling matrix) L 2 Gl(n) and an invertible matrix with unit vectors in each row (permutation matrix) P 2 Gl(n) (Comon, 1994). Vice versa, any matrix B that is equivalent to A¡1 solves the BSS problem, since the transformed mutual information is invariant under scaling and permutation of coordinates. 3 Geometric Considerations

The basic idea of the geometric separation method lies in the fact that in the source space fs1 , . . . , sl g ½ Rn , where si represent a Žxed number of samples of the source vector S with zero mean, the data clusters along the axes of the coordinate system are transformed by A into data clusters along

Linear Geometric ICA

421

a2

a1

x

2

j=0

x1 Figure 1: A two-dimensional scatter plot of a mixture of two Laplacian signals with identical variance. The signals have been mixed by a matrix A mapping the unit vectors onto vectors inclined with angle ai to the x1 -axis. The ragged line along the circle is the histogram of the observations after projection onto the circle plotted in polar coordinates. Dashed lines show borders of the receptive Želds—the borders of the circle sections that lie closest to the angles ai or ¡ai , respectively.

transformed coordinate axes through the origin. The detection of these n new axes allows determining a demixing matrix B that is equivalent to A ¡1 (see Figure 1). We now consider the learning process to be terminated and describe precisely how to recover the matrix A then—after the axes, which span the observation space, have been extracted from the data successfully. Let 6 ig L :D f(x1 , . . . , xn ) 2 Rn | 9i xi > 0, xj D 0 for all j D

422

F. Theis, A. Jung, C. Puntonet, and E. Lang

be the set of positive coordinate axes, and denote with L 0 :D AL the image of this set under the transformation A. Note that due to A being bijective, L 0 intersects the unit (n ¡ 1) -sphere, Sn¡1 :D fx 2 Rn | |x| D 1g, in exactly n distinct points fp1 , . . . , pn g and that those pi ’s form a basis of Rn . DeŽne the matrix Mp1 ,...,pn 2 Gl(n) to be the linear mapping of ei onto pi for i D 1, . . . , n, that is, Mp1 ,...,pn D ( p1 | ¢ ¢ ¢ |pn ). This matrix thus effects the linear coordinate change from the standard coordinates ( ei ) to the new basis ( pi ). Note that for this coordinate transformation, the following lemma holds: For any permutation s 2 Sn , the two matrices Mp1 ,...,pn and Mps (1) ,...,ps(n) are equivalent.

Lemma 1.

Now we can state the following theorem: Theorem 1 (Uniqueness of the Geometric Method).

The matrix Mp1 ,...,pn is

equivalent to A. By construction of Mp1 ,...,pn , we have Mp1 ,...,pn ( ei ) D pi D f ( ei ) D so there exists a li 2 Rnf0g such that Mp1 ,...,pn (ei ) D li Aei . DeŽning L such that L ( ei ) :D li ei yields an invertible diagonal matrix L 2 Gl(n) such that Mp1 ,...,pn D LA. This shows the claim. Proof. Aei |Aei | ,

Corollary 1.

The matrix Mp¡1 solves the BSS problem. 1 ,...,pn

4 The Ordinary Geometric Algorithm

For now, we will restrict ourselves to the two-dimensional case for simplicity; extensions to higher dimensions are discussed in section 11. Let S: V ¡! R2 be an independent two-dimensional Lebesgue-continuous random vector describing the source pattern distribution; its density function is denoted by r: R2 ¡! R. As S is independent, r factorizes r ( x, y) D r1 ( x)r 2 ( y ) , with ri : R ¡! R denoting the corresponding marginal source density functions. As a further simpliŽcation, we will assume the source variables Si to have zero mean E (S ) D 0 and to be distributed symmetrically, that is, r i ( x) D r i (¡x ) for x 2 R and i D 1, . . . , n. To ensure stability of the geometric algorithm, we have to assume that the signal distributions are nongaussian and unimodal. In practice, these restrictions are often met at least approximately.

Linear Geometric ICA

423

As above, let X denote the sensor signal vector and A the invertible mixing matrix such that X D A ± S. Without loss of generality, assume that A is of the form Á ! cos a1 cos a2 AD , sin a1 sin a2 where ai 2 [0, p ) denote two angles. The ordinary geometric learning algorithm (Puntonet & Prieto, 1995) for symmetric distributions in its simplest form then is as follows: Pick four starting vectors w1 , w01 , w2 , and w02 on S1 such that wi and w0i are opposite each other (i.e., wi D ¡w0i for i D 1, 2) and w1 and w2 are linearly independent vectors in R2 . Usually one takes the unit vectors w1 D e1 and w2 D e2 . Furthermore, Žx P a learning rate g: N P¡! R with (Cottrell, Fort, & Pag`es, 1994) g(t) > 0, n2N g (n ) D 1 and n2N g (n ) 2 < 1. Then iterate the following step until an appropriate abort condition has been met: Choose a sample x ( t) 2 R2 according to the distribution of X. If x ( t) D 0, pick a new one. Note that this case happens with probability zero since the probability density function (pdf) r X of X is assumed to be continuous. x ( t) Project x ( t) onto the unit sphere to get y ( t) :D |x(t) | . Let i be in f1, 2g such that 0 wi or wi is the vector closest to y with respect to a Euclidean metric. Then update wi (t) according to the following update rule, ³ ´ y ( t) ¡ wi ( t) wi ( t C 1) :D pr wi ( t) C g ( t) , |y(t) ¡ wi ( t) | where pr: R2 nf0g ¡! S1 represents the projection onto the unit sphere, and w0i ( t C 1) :D ¡wi (t C 1). The other two w’s are not moved in this iteration. In Figures 2 and 3, the learning algorithm has been visualized on the sphere and after the projection onto [0, p ) . This weight update rule resembles unsupervised competitive learning rules used in many clustering algorithms like k-means, vector quantization, or Kohonen’s self-organizing maps, but with the modiŽcations that the step size along the direction of a sample does not depend on distance and the learning process takes place on S1 , not in R. 5 Formal Model of the Geometric Algorithm

Now we present a formal theoretical framework for geometric ICA that will be used in the next section to formulate a proper convergence condition. First, we show using the symmetry of S that it is in fact not necessary to have two vectors wi and w0i moving around on the same axis. Indeed, we should not speak of vectors but of lines in R2 , so the wi ’s would be in

424

F. Theis, A. Jung, C. Puntonet, and E. Lang

w2(0)

*

*

w2(too)

* w (t ) *w (0)

x2

1 oo

1

x1 Figure 2: Visualization of the geometric algorithm with starting points w1 (0) and w2 (0) and end points w1 (1) and w2 (1) . Dash-dotted lines mark receptive Želd borders.

the real projective space RP1 D S1 / », where » identiŽes antipodal points. This is the manifold of all one-dimensional subvector spaces of R2 . A metric is deŽned by setting d ([x], [y]) :D minf|x ¡ y|, |x C y|g for [x], [y] 2 RP1 . Alternatively, one can picture the w’s in S1C :D S1 \ f(x1 , x2 ) 2 R2 | x2 ¸ 0g / », where » identiŽes the two points (1, 0) and (¡1, 0) . Let f : S1 ¡! S1C represent the canonical projection. Furthermore, it is useful to introduce polar coordinates Q: S1C ¡! [0, p ) on S1C with the stratiŽcation Q0 : R ¡! S1C such that Q0 ± Q D id, where id denotes the identity. Let  :D Q ± Q0 : R ¡! [0, p ) be the modulo p map. We are interested in the projected random sensor signal vector pr ±X: V ¡! S1 , so after cutting open the circle S1 and identiŽcation of opposite points, we want to approximate the transformed

Linear Geometric ICA

425

0.01

Probability­ Density [1/Deg]

0.008

* w (t i

oo

)

0.006

0.004

50%

*

0.002

0

w (t) i

w (0) i

ai 0

10

20

50% 30

40 50 60 Angle in F [Deg]

70

80

* 90

i

Figure 3: Plot of the density r Y of a mixture of two Laplacian signals with identical variance. The weight adaptation by the geometric algorithm is also visualized. See also Figure 2.

random variable Y :D Q ± f ± pr ±X: V ¡! [0, p ) in a suitable manner. Note that using the symmetry of r, the density function r Y of the transformed sensor signal Y can be calculated from the density rX of the original sensor signal X by Z 1 rY (Q ) D 2|det A| ¡1 r ( A ¡1 ( r cos Q, r sin Q ) > ) r dr. 0

Then the geometric learning algorithm induces the following discrete Markov process W (t ) : V ¡! R2 , deŽned recursively by W (0) D ( w1 , w2 ) and W ( t C 1) D  2 ( W (t) C g ( t) q ( ( Y (t ) , Y (t) ) ¡ W (t ) ) ) , where ( (sgn (x ) , 0) |y| ¸ |x| q (x, y ) :D (0, sgn ( y) ) |x| > |y| and Y (0) , Y (1) , . . . is a sequence of independent and identically distributed random variables V ¡! R with the same distribution as Y. These random

426

F. Theis, A. Jung, C. Puntonet, and E. Lang

variables will be needed to represent the independence of the successive sampling experiments. Note that the modulo p map  guarantees that W (t C 1) 2 [0, p ) . Indeed, this is just winner-takes-all learning with a signum function in R, but taking into account the fact that we have to stay in [0, p ) . Note that the metric used here is the planar metric, which obviously is equivalent to the metric on S1C induced by the Euclidean metric on S1 ½ R2 . We furthermore can assume that after enough iterations, there is one point a 2 S1 that will not be transversed anymore, and without loss of generality, we assume a to be 0 (otherwise, cut S1 open at a and project along this resulting arc), so that the above algorithm simpliŽes to the planar case with the recursion rule W ( t C 1) D W ( t) C g (t )q ( (Y ( t) , Y ( t) ) ¡ W ( t) ). Without the sign function and the additional fact that the probability distribution of Y is log concave, it has been shown (Cottrell & Fort, 1987; Ritter & Schulten, 1988; Benaim, Fort, & Pag e` s, 1998) that the process W ( t) converges to a unique constant Žxed-point process W ´ w 2 R2 such that Z wi Z b2 ( Fi ) r Y (Q ) dQ D r Y (Q ) dQ b 1 ( Fi )

wi

for i D 1, 2, where Fi :D F ( wi ) :D fQ 2 [0, p ) |  (| Q ¡ wi | ) ·  (| Q ¡ wj |) 6 for all j D ig with Fi D  ([b1 (Fi ) , b2 (Fi ) ]) denotes the receptive Želd of wi and bj ( Fi ) designate the receptive Želd borders. However, it is not clear how to generalize the proof to the geometric case, especially because we do not have (and do not want) log concavity of Y, as this would lead to a unique Žxed point. Therefore, we will assume convergence in a sense stated in the following section. 6 Limit Points of the Geometric Algorithm

In this section, we study the end points of geometric ICA, so we will assume that the algorithm has already converged. The idea is to formulate a condition that the end points will have to satisfy and to show that the solutions are among them. DeŽnition 1 (Geometric Convergence Condition). Two angles l1 , l2 2 [0, p ) satisfy the geometric convergence condition (GCC) if they are the medians of Y restricted to their receptive Želds, that is, if li is the median of r Y | F ( li ) .

O ´ ( wO 1 , wO 2 ) 2 R2 is called Žxed A constant random vector W O (t) ) ) D 0. point of geometric ICA in the expectation if E (q ( Y ¡ W DeŽnition 2.

Hence, the expectation of a Markov process W ( t) starting at a Žxed point of geometric ICA will not be changed by the geometric update rule because E (W ( t C 1) ) D E ( W ( t) ) C g (t) E (q ( Y ( t) ¡ W (t) ) ) D E ( W (t ) ) .

Linear Geometric ICA

427

Given that the geometric algorithm converges to a constant random vector W (1) ´ ( w1 (1) , w2 (1) ) , then W (1) is a Žxed point of geometric ICA in the expectation, if and only if the wi (1) satisfy the GCC. Theorem 2.

Assume W (1) to be a Žxed point of geometric ICA in the expectation. Without loss of generality, let [b 1 , b2 ] be the receptive Želd of w1 (1) such that bi 2 [0, p ). Since W (1) is a Žxed point of geometric ICA in the expectation, we have E ( Â[b1 ,b2 ] ( Y ( t) ) sgn ( Y (t) ¡ w1 (1) ) ) D 0, where Â[b1 ,b2 ] denotes the characteristic function of that interval. But this means R w1 (1) R R (1) (¡1)rY (Q ) dQ C wb2 (1) 1r Y (Q ) dQ D 0 and therefore bw1 r Y (Q ) dQ D b1 1 1 R b2 ( ) (1) r Q dQ, so w1 satisŽes GCC. The same calculation for w2 (1) w1 (1) Y shows one direction of the claim. The other direction follows by simply reading the above proof backward, which completes the proof. Proof.

As before, let pi :D Aei be the transformed unit vectors, and let qi :D Q ± f ± pr(pi ) 2 [0, p ) be the corresponding angles for i D 1, 2. Theorem 3.

The transformed angles qi satisfy GCC.

Because of the symmetry of the claim, it is enough to show that q1 satisŽes GCC. Without loss of generality, let 0 < a1 < a2 < p using the symmetry of r. Then, due to construction, qi D ai . Let b1 :D a1 C2 a2 ¡ p2 and b2 :D b1 C p2 . Then the receptive Želd of q1 can be written (modulo p ) as F ( q1 ) D [b1 , b2 ]. Therefore, we have to show that q1 D a1 is the median of Ra Rb r Y restricted to F ( q1 ) , which means b11 rY (Q ) dQ D a12 rY (Q ) dQ. We will reduce this to the orthogonal standard case A D id by transforming the integral as follows: Z a1 Z a1 Z 1 r Y (Q ) dQ D 2|det A| ¡1 dQ r dr r ( A¡1 ( r cos Q , r sin Q ) > ) Proof.

b1

Z ¡1 D 2|det A|

b1

0

dx dy r ( A¡1 ( x, y) > ) , K

where K :D f(x, y ) 2 R2 | b1 · arctan(y / x ) · a1 g denotes the cone of opening angle a1 ¡ b1R starting from angle b1 . Using the transformation R a formula, we continue b11 rY (Q ) dQ D 2 A¡1 ( K ) dx dy r ( x, y ) . Now note that the transformed cone A¡1 (K ) is a cone ending at the x-axis of opening angle p / 4, because A is linear; therefore, we are left with the following integral: Z a1 Z 1 Z 0 Z 1 Z x r Y (Q ) dQ D 2 dx dy r (x, y ) D 2 dx dy r ( x, ¡y) b1

0

Z D 2

1 0

Z

¡x

0

Z

x

dx 0

dy r ( x, y ) D

b2

a1

0

rY (Q ) dQ,

428

F. Theis, A. Jung, C. Puntonet, and E. Lang

where we have used the same calculation for [a1 , b 2 ] as for [b1 , a1 ] at the last step. This completes the proof of the theorem. Combining both theorems, we have therefore shown: Let W be the set of Žxed points of geometric ICA in the expectation. ¡1 Then there exists ( wO 1 , wO 2 ) 2 W such that Mw O 1 , wO 2 solves the BSS problem. The stable Žxed points in W can be found by the geometric ICA algorithm. Theorem 4.

Furthermore, we believe that in the special case of unimodal, symmetric, and nongaussian signals, the set W consists of only two elements: a stable and an unstable Žxed point, where the stable Žxed point will be found by the algorithm: Assume that the sources Si are unimodal, symmetric, and nongaussian. Then there are only two Žxed points of geometric ICA in the expectation. Conjecture 1.

We can prove this conjecture for the special case of two sources with identical distributions that are nicely super- or subgaussian in the sense that r Y has only four extremal points. Theorem 5. Assume that the sources Si are unimodal and symmetric and that r1 D r2 . Assume that rY | [0, p ) with A D id has exactly two local maxima and two local minima Then there exist only two Žxed points of geometric ICA in the expectation.

The same calculation as in the proof of theorem 3 shows that we can assume A D id without loss of generality. Then by theorem 2 and by r1 D r2 , the two pairs f0, p2 g and f p4 , 3p 4 g satisfy GCC. We have to show that no other pair fulŽlls this condition. ) First, note that the symmetry of r1 and r2 shows that r Y ( np 2 ¡Q D np ) ( C rY ( np Q for 2 and Q 2 , and r r induces even r n Z R D 1 2 Y 2 4 ¡ ). C Q ) D rY ( np Q Due to assumption, r has only two maxima and two Y 4 minima in [0, p ) ; the above equations then show that those have to be at f0, p4 , p2 3p 4 g. np 6 | [b 1 , b1 C p2 ] Now we claim that for b1 D 4 , n 2 Z, the median of rY p does not equal b1 C 4 . Note that this shows theorem 5. For the proof of the claim, consider the smallest integer p 2 Z such that C p4 . Let b 2 :D b1 C p2 and a :D b1 C p4 . We have c 1 :D pp 2 > b 1 , and set c 2 :D c 1 to show that the median of r Y | [b1 , b 2 ] does not equal a. A visualization of these relations and the following deŽnitions is given in Figure 4. Using the symmetry noted above, we have for d1 :D c 1 C (c 1 ¡ b1 ) D Rc Rd 6 2c 1 ¡ b1 : C1 :D b11 r Y D c 11 rY . Note that d1 D a, or else pp D 2c 1 D Proof.

Linear Geometric ICA

429

Figure 4: Visualization of the proof of theorem 5. a, bi , c i , and di are deŽned in the text.

a C b1 D 2b 1 C p2 , which contradicts the assumption to b 1 . Without loss of generality, let a > d1 ; the other case can be proven similarly. Setting d2 :D c 2 C (c 2 ¡ a) D 2c 1 ¡ a, again symmetry shows that C2 :D R c2 R d2 Ra R b2 a r Y D c 2 rY and C3 :D d1 rY D d2 rY I the second equation follows because c 2 ¡ d1 D b2 ¡ c 2 and c 2 ¡ a D d2 ¡ c 2 . Now assume in contradiction to R aour claim R b that a is the median of rY | [b1 , b2 ]. Then we have 2C1 C C3 D b r Y D a 2 r Y D 2C2 C C3 and therefore 1 C1 D C3 . As shown above, r Y (c 1 ) and r Y (c 2 ) are the only extremal points of r Y | [b1 , b2 ]. Without loss of generality, let r Y (c 1 ) be R c a maximum; then r Y (c 2 ) has to be a minimum. But this means that C1 D b11 rY > (c 1 ¡ b1 )r Y (a) ¸ R c2 a r Y D C 2 which contradicts the above. This completes the proof of the theorem. Conjecture 1 states that there are only two Žxed points of geometric ICA. In fact, we claim that of those two, only one Žxed point is stable in the sense that slight perturbations of the initial conditions preserve the convergence. Then, depending on the kurtosis of the sources, either the stable (supergaussian case) or the unstable (subgaussian case) Žxed point represents the image of the unit vectors. This is stated in the following conjecture. Assume that the sources Si are unimodal, symmetric, and nongaussian. Then by conjecture 1, there are only two Žxed points (wO 1 , wO 2 ) and Conjecture 2.

430

F. Theis, A. Jung, C. Puntonet, and E. Lang

(wQ 1 , wQ 2 ) of geometric ICA in the expectation. We claim: i. There is only one stable Žxed point (wO 1 , wO 2 ) . ¡1 ii. If the sources are supergaussian, Mw O , wO solves the BSS problem. 1

iii. If the sources are subgaussian,

¡1 Mw Q 1 , wQ 2

2

solves the BSS problem.

7 Update Rules Without Sign Functions

We have shown that the geometric update step requires the signum function as follows: wi (t C 1) D wi (t) C g(t) sgn ( y ( t) ¡ wi ( t) ) . Then (normally), the wi converge to the medians in their receptive Želd. Note that the medians do not have to coincide with any maxima of the sensor signal density distribution on the sphere, as shown in Figure 5. Therefore, in general, any algorithm searching for the maxima of the distribution (Prieto, Prieto, Puntonet, Canas, & Martin-Smith, 1999) will not end at the medians, which are the correct images of the unit vectors under the given mixing transformation. Only given special restrictions of the sources (same supergaussian distribution of each component, as, for example, speech signals), the medians correspond to the maxima, and a maximum searching algorithm will converge to the correct Žxed points of geometric ICA. 8 Histogram-Based Algorithm: FastGeo

So far in geometric ICA, mostly on-line algorithms with competitive learning rules as in section 4 have been used (Puntonet & Prieto, 1995). As shown above, they search for points satisfying the GCC. In the following, we will establish a new geometric algorithm based on the GCC alone. For this, let n D 2 and Á cos(a1 ) A :D sin(a1 )

! cos(a2 ) . sin(a2 )

(8.1)

Theorem 3 shows that the vectors (cos(ai ) , sin(ai ) ) > satisfy the GCC. Therefore, the vectors wi will converge to the medians in their receptive Želds. This enables us to compute these positions directly using a search on the histogram of Y (see Figure 6), which reduces the computation time by a factor of about 100 or more. In the FastGeo algorithm, we scan through the different receptive Želds and test GCC. In practice, this means discretizing the distribution fY of Y using a given bin size b > 0 and then testing the p / b different receptive Želds. The algorithm will be formulated more precisely in the following.

Linear Geometric ICA

431

0.014

Probability­ Density [1/Deg]

0.012 0.01 0.008 0.006 0.004 0.002

a 0

0

a

1

45

2

90 Angle [Deg]

135

180

Figure 5: Projected density distribution rY of a mixture of two Laplacian signals with different variances, with the mixture matrix mapping the unit vectors ei to (cos ai , sin ai ) for i D 1, 2. Dark line = theoretical density function; gray line = histogram of a mixture of 10,000 samples.

For simplicity, let us assume that the cumulative distribution FY of Y is invertible; this means that FY is nowhere constant. DeŽne a function m : [0, p )

Q

¡! R p² l1 (Q ) C l2 ( Q ) ± 7¡! ¡ QC , 2 2

(8.2)

where ³ li (Q ) :D FY¡1

FY (Q C i p2 ) C FY (Q C ( i ¡ 1) p2 ) 2

´ (8.3)

is the median of Y | [Q C ( i ¡ 1) p2 , Q C i p2 ] in [Q C (i ¡ 1) p2 , Q C i p2 ] for i D 1, 2. Lemma 2.

Let Q be a zero of m in [0, p ) . Then the li (Q ) satisfy the GCC.

432

F. Theis, A. Jung, C. Puntonet, and E. Lang

0.01 F2

F1

Probability­ Density [1/Deg]

0.008

0.006

0.004

0.002

0

a1 0

a2 45

a1 +180°

90

135 180 225 Angle [Deg]

a2 +180° 270

315

360

Figure 6: Probability density function fY of Y from Figure 1 with the mixing angles ai and their receptive Želds Fi for i D 1, 2. ( ) ( ) ( ) ( ) By deŽnition, [ l1 Q C2 l2 Q ¡ p2 , l1 Q C2 l2 Q ] is the receptive Želd of l1 (Q ) . Since m (Q ) D 0, the starting point of the above interval is Q, because Q D l1 (Q ) C l 2 (Q ) ¡ p2 . Hence, we have shown that the receptive Želd of l1 (Q ) is 2 [Q, Q C p2 ], and by construction l1 (Q ) is the median of Y restricted to the above interval. The claim for l2 (Q ) then follows.

Proof.

Algorithm 1 (FastGeo).

Find the zeros of m .

m always has at least two zeros that represent the stable and the unstable Žxed point of the ordinary geometric algorithm. In practice, we extract the Žxed point, which then gives the proper demixing matrix A ¡1 by picking Q0 such that fY ( l1 (Q0 ) ) C fY ( l2 (Q0 ) ) is maximal. For unimodal and supergaussian source distributions, conjecture 2 claims that this results in a stable Žxed point. For subgaussian sources, choosing Q0 with fY ( l1 (Q0 ) ) C fY ( l2 (Q0 ) ) being minimal induces the corresponding demixing matrix. Hence, one advantage of this histogram-based algorithm is that without any modiŽcations, we can solve the ICA problem for subgaussian signals too. Furthermore, the

Linear Geometric ICA

433

sophisticated parameter choice of the ordinary geometric algorithm is not necessary any more; only one parameter, the bin size, has to be chosen. In practice, one sometimes notices that due to the discretization of the distribution, the approximated distribution has a rather noisy shape on small scales. This results from m having zeros being split up into multiple zeros close together. Therefore, a useful improvement of convergence can be established by smoothing this distribution with a kernel function with sufŽciently small halfwidth. This smoothing should be performed preferably during the discretization process of the original distribution. 9 Accuracy

In this section, we consider the dependence of the FastGeo ICA algorithm on the number of samples after the bin size b has been Žxed. As seen in the previous section, the accuracy of the histogram-based algorithm then depends only on the distribution of the samples X (resp. Y); that is, we can estimate the error made by approximating the mixing matrix A by a Žnite number of samples. In the following, we present some results of test runs made with this algorithm. When choosing two arbitrary angles ai 2 [0, p ) , i D 1, 2 for the mixing matrix A, we deŽne a as the distance between these two angles modulo p2 . This will give us an angle in the range between 0 and p2 . In Figure 7, the |a ¡arecovered |

relative accuracy of the recovered angles Daa D i ia is given as a function of the angle a for a Žxed number of samples. Obviously, the resulting graph is reasonably constant over a wide range of a, demonstrating that the estimate of the ai with respect to a is robust over a wide range of a (a > 10± ) and gets only slightly worse for small values of a. Note that the distortions around the origin are due to the Žnite bin size; increasing the number of bins increases the accuracy for small a’s, but also the computational effort. To investigate the relation between the error D a and the number of samples, we examine for different a the dependence of the standard deviation of D a on the number of samples (see Table 1), where we chose the following mixing matrix A: Á A :D

1 0.5

! 0.5 . 1

(9.1)

The table entry gives the standard deviation of the nondiagonal terms after normalizing each column of the mixing matrix, so that the diagonal elements are unity. For comparison, we also calculated the performance index E1 (Amari, Cichocki, & Yang, 1996).

434

F. Theis, A. Jung, C. Puntonet, and E. Lang

0.1

Mean Standard deviation 95% confidence interval

[°]

0.08 0.06 0.04 0.02 0

0

10

20

30

40

50

60

70

80

90

[°] Figure 7: Mixture of 1,000samples of two Laplacian source signals with identical variances. Plotted are the mean, standard deviation, and 95% conŽdence interval of D a/ a versus a calculated from 100 runs for each angle a. Table 1: Standard Deviations of the Nondiagonal Terms and the Performance Index E1 with Different Number of Samples. Number of Samples

Standard Deviation

Index E1

1000 10,000 100,000

0.033 0.013 0.007

0.18 0.07 0.038

10 Examples

In this section, we compare geometric ICA algorithms with other ICA algorithms: the Extended Infomax algorithm (Lee, Girolami, & Sejnowski, 1999), which is based on the classical ICA algorithm (Bell & Sejnowski, 1995), and the FastICA algorithm (Hyv¨arinen & Oja, 1997). As geometric algorithms, we use the ordinary geometric algorithm from section 4 (Puntonet & Prieto, 1995) and the FastGeo algorithm from above. Calculations were made on a PIII-850 PC using Matlab 6.0. In our Žrst example, we consider a mixture of two Laplacian signals. The results of the different algorithms are shown in Table 2. For each algorithm,

Linear Geometric ICA

435

Table 2: Comparison of Time per Run and Cross-talking Error of ICA Algorithms for a Random Mixture of two Laplacian Signals. Algorithm

Elapsed Time [s]

Index E1

Extended Infomax FastICA (pow3 D default) FastICA (tanh) FastICA (gauss) Ordinary Geometric FastGeo

11.1 0.068 0.11 0.12 > 60 0.84

0.072§ 0.002 0.076§ 0.004 0.052§ 0.001 0.048§ 0.001 0.18 § 0.10 0.110§ 0.071

Note: Means and standard deviations were taken over 1000 runs (100 runs for Extended Infomax and Ordinary Geo) with 10,000 samples and uniformly distributed mixing matrix elements.

Table 3: Comparison of Time per Run and Cross-Talking Error of ICA Algorithms for a Random Mixture of Two Sound Signals with 22,000 Samples. Algorithm

Elapsed Time [s]

Index E1

Extended Infomax FastICA (pow3 D default) FastICA (tanh) FastICA (tanh) Ordinary Geometric FastGeo

41.2 0.14 0.24 0.26 > 60 0.89

0.058§ 0.002 0.050§ 0.005 0.022§ 0.001 0.019§ 0.001 0.49 § 0.29 0.136§ 0.087

Note: Means and standard deviations were taken over 1000 runs (100 runs for Extended Infomax and Ordinary Geo) with uniformly distributed mixing matrix elements.

we measure the mean elapsed CPU time per run and the mean cross-talking error E1 with its standard deviation. Both the Extended Infomax and the FastICA algorithm perform best in terms of accuracy, and in terms of computational speed, FastICA lives up to its name, being followed by FastGeo, then InfoMax, and then the ordinary geometric algorithm, always by one scale difference. The last algorithm lacks accuracy and also shows some convergence problems, whereas FastGeo lies between the geometric algorithm and FastICA and Infomax regarding accuracy. The second example deals with real-world data: two audio signals (one speech and one music signal; see Table 3). The results are similar to the Laplacian toy example. FastICA outperforms the other algorithms in terms of speed. The accuracy of Extended Infomax and FastICA is comparable, and FastGeo is slightly (factor 4) worse but faster than the Extended Infomax. The ordinary geometric algorithm is both slower and less accurate, mainly because of convergence problems.

436

F. Theis, A. Jung, C. Puntonet, and E. Lang

11 Higher Dimensions

So far we have explicitly considered two-dimensional data sets only. In realworld problems, however, the sensor signals are usually high-dimensional data sets, for example, EEG data with 21 dimensions. Therefore, it would be satisfactory to generalize geometric algorithms to higher dimensions. The ordinary geometric algorithm can be easily translated to higher-dimensional cases, but one faces serious problems in the explicit calculations. In order to approximate higher-dimensional pdfs, it becomes necessary to have an exponentially growing number of samples available, as will be shown. The number of samples in a ball Bd¡1 of radius q on the unit sphere d¡1 ½ Rd divided by the number of samples on the whole Sd¡1 can be S calculated as follows if we assume a uniformly distributed random vector. Let Bd :D fx 2 Rd | |x| · 1g and Sd¡1 :D fx 2 Rd | |x| D 1g. The volume of Bd can be calculated by vol (Bd ) D

d

p2 ( d2 )!

D cd . It follows for d > 3: d¡1 )q d¡1

n vol(B Number of samples in ball vol(S d¡1 ) D n n

·

q d¡1 cd¡1 q d¡1 d . D p cd C 1

Obviously, the number of samples in the ball decreases by q d¡1 d if q < 1, which is the interesting case. To have the same accuracy when estimating the medians, the decrease must be compensated by an exponential growth in the number of samples. For three dimensions, we have found a good approximate of the demixing matrix by using 100,000 samples. In four dimensions, the reconstructed mixing matrix could not be reconstructed correctly, even with larger numbers of samples. A different approach for higher dimensions has been taken by Bauer et ) al. (2000), where A has been calculated using d ( d¡1 projections of X from 2 2 d R onto R along the different coordinate axes and reconstructed the multidimensional matrix from the two-dimensional solutions. However, this approach works satisfactorily only if the mixing matrix A is close to the unit matrix up to permutation and scaling. 12 Conclusion

The geometric ICA algorithm has been studied in a concise theoretical framework. The Žxed points of the geometric ICA learning algorithm have been examined in detail. We have introduced a GCC, which has to be fulŽlled by the Žxed points of the learning algorithm. We further showed that it is also fulŽlled by the mixed unit vectors spanning the sensor signal space. Hence, geometric ICA can solve the BSS problem. We then gave two conjectures for the unimodal case where the Žxed-point property is expected to be very rigid.

Linear Geometric ICA

437

We then presented a new algorithm for linear geometric ICA (FastGeo) based on histograms that is both robust and computationally much more efŽcient than the ordinary geometric ICA algorithm. The accuracy of the algorithm concerning the estimation of the relevant medians of the underlying data distributions, when varying both the mixing matrices and the sample numbers, has been explored quantitatively, showing a rather robust performance of the algorithm. When comparing FastGeo with classical ICA algorithms and the ordinary geometric one, we noticed that FastGeo performs only slightly worse than the classical ones in terms of accuracy and better than the ordinary geometric one. In terms of speed, FastGeo falls between FastICA and the Extended Infomax and is much faster than the Žrst geometric approaches, which also suffer from severe convergence problems. Furthermore, the fact that geometric algorithms and especially FastGeo are very easy to implement makes FastGeo a good choice even in comparison with the classical ICA algorithms in practical two-dimensional applications. We also considered the problem of high-dimensional data sets with respect to the geometrical algorithms and discussed how projections to lowdimensional subspaces could solve this problem for a special class of mixing matrices. Simulations with nonsymmetrical and nonunimodal distributions have shown promising results so far, indicating that the new algorithm will perform well with almost any distribution. This is the subject of ongoing research in our group. In future work, besides treating nonsymmetric sources S, the two conjectures will have to be proven in full, as well as the Kohonen proof of convergence to be translated into the above model. In addition, the histogrambased algorithm could be extended to the nonlinear case similar to Puntonet et al. (1999), using multiple centered spheres for projection on the surface on which the projected data histograms could then be evaluated. Finally, we are experimenting with the FastGeo algorithm for the overcomplete case, where we are currently able to detect three or more sources in only two mixtures. This can be useful for higher-dimensional cases as in section 11.

Acknowledgments

This research was supported by the Deutsche Forschungsgemeinschaft (DFG) in the Graduiertenkolleg “Nonlinearity and Nonequilibrium in Condensed Matter.” We thank Christoph Bauer and Tobias Westenhuber for suggestions and comments on the ordinary geometric algorithm and Michaela Lautenschlager and Stefanie Ulsamer for their helpful comments in trying to prove convergence.

438

F. Theis, A. Jung, C. Puntonet, and E. Lang

References Amari, S., Cichocki, A., & Yang, H. (1996). A new learning algorithm for blind signal separation. In D. Touretzky, M. Mozer, & M. Hasselmo (Eds.), Advances in neural information processing systems, 8 (pp. 757–763). Cambridge, MA: MIT Press. Bauer, C., Puntonet, C., Rodriguez-Alvarez, M., & Lang, E. (2000). Separation of EEG signals with geometric procedures. In C. Fyfe (Ed.), Engineering of intelligent systems (Proc. EIS’2000) (pp. 104–108). Millet, Alberta, Canada: ICSC Press. Bell, A. J., & Sejnowski, T. J. (1995). An information-maximization approach to blind separation and blind deconvolution. Neural Computation, 7, 1129–1159. Benaim, M., Fort, J.-C., & PagÂes, G. (1998). Convergence of the one-dimensional Kohonen algorithm. Adv. Appl. Prob., 30, 850–869. Comon, P. (1994). Independent component analysis—a new concept? Signal Processing, 36, 287–314. Cottrell, M., & Fort, J.-C. (1987). Etude d’un processus d’autoorganisation. Annales de l’Institut Henri PoincarÂe, 23(1), 1–20. (in French) Cottrell, M., Fort, J. C., & Pag`es, G. (1994). Two or three things that we know about the Kohonen algorithm. In M. Verleysen (Ed.), Proc. ESANN’94, European Symp. on ArtiŽcial Neural Networks (pp. 235–244).Brussels: D Facto Conference Services. Hyva¨ rinen, A. (1999). Fast and robust Žxed-point algorithms for independent component analysis. IEEE Transactions on Neural Networks, 10(3), 626–634. Hyva¨ rinen, A., & Oja, E. (1997). A fast Žxed-point algorithm for independent component analysis. Neural Computation, 9, 1483–1492. Jutten, C., HÂerault, J., Comon, P., & Sorouchiary, E. (1991). Blind separation of sources, parts I, II, and III. Signal Processing, 24, 1–29. Lee, T.-W., Girolami, M., & Sejnowski, T. (1999). Independent component analysis using an extended infomax algorithm for mixed sub-gaussian and supergaussian sources. Neural Computation, 11, 417–441. Linsker, R. (1989). An application of the principle of maximum information preservation to linear systems. In D. Touretzky (Ed.), Advances in neural information processing systems, 1. San Mateo, CA: Morgan Kaufmann. Linsker, R. (1992). Local synaptic learning rules sufŽce to maximize mutual information in a linear network. Neural Computation, 4, 691–702. Prieto, A., Prieto, B., Puntonet, C., Canas, A., & Martin-Smith, P. (1999). Geometric separation of linear mixtures of sources: Application to speech signals. In J. F. Cardoso, Ch. Jutten, & Ph. Loubaton (Eds.), Independent component analysis and signal separation (Proc. ICA’99) (pp. 295–300). Gi`eres, France: Imp. de Ecureuils. Puntonet, C., Alvarez, M., Prieto, A., & Prieto, B. (1999). Separation of speech signals for nonlinear mixtures. Lecture Notes in Computer Science, 1607, 665– 673. Puntonet, C., & Prieto, A. (1995). An adaptive geometrical procedure for blind separation of sources. Neural Processing Letters, 2.

Linear Geometric ICA

439

Ritter, H., & Schulten, K. (1988). Convergence porperties of Kohonen’s topology conserving maps: Fluctuations, stability, and dimension selection. Biological Cybernetics, 60, 59–71. Received December 14, 2001; accepted July 3, 2002.

LETTER

Communicated by Geoffrey Hinton

Equivalence of Backpropagation and Contrastive Hebbian Learning in a Layered Network Xiaohui Xie [email protected] Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, U.S.A. H. Sebastian Seung [email protected] Howard Hughes Medical Institute and Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, U.S.A. Backpropagation and contrastive Hebbian learning are two methods of training networks with hidden neurons. Backpropagation computes an error signal for the output neurons and spreads it over the hidden neurons. Contrastive Hebbian learning involves clamping the output neurons at desired values and letting the effect spread through feedback connections over the entire network. To investigate the relationship between these two forms of learning, we consider a special case in which they are identical: a multilayer perceptron with linear output units, to which weak feedback connections have been added. In this case, the change in network state caused by clamping the output neurons turns out to be the same as the error signal spread by backpropagation, except for a scalar prefactor. This suggests that the functionality of backpropagation can be realized alternatively by a Hebbian-type learning algorithm, which is suitable for implementation in biological networks. 1 Introduction

Backpropagation and contrastive Hebbian learning (CHL) are two supervised learning algorithms for training networks with hidden neurons. They are of interest, because they are generally applicable to wide classes of network architectures. In backpropagation (Rumelhart, Hinton, & Williams, 1986b, 1986a), an error signal for the output neurons is computed and propagated back into the hidden neurons through a separate teacher network. Synaptic weights are updated based on the product between the error signal and network activities. CHL updates the synaptic weights based on the steady states of neurons in two different phases: one with the output neurons clamped to the desired values and the other with the output neurons free (Movellan, 1990; Baldi & Pineda, 1991). Clamping the output neurons Neural Computation 15, 441–454 (2003)

° c 2002 Massachusetts Institute of Technology

442

X. Xie and H. Seung

causes the hidden neurons to change their activities, and this change constitutes the basis for the CHL update rule. CHL was originally formulated for the Boltzmann machine (Ackley, Hinton, & Sejnowski, 1985) and was extended later to deterministic networks (Peterson & Anderson, 1987; Hinton, 1989), in which case it can be interpreted as a mean-Želd approximation of the Boltzmann machine learning algorithm. However, this interpretation is not necessary, and CHL can be formulated purely for deterministic networks (Movellan, 1990; Baldi & Pineda, 1991). Compared to backpropagation, CHL appears to be quite different. Backpropagation is typically implemented in feedforward networks, whereas CHL is implemented in networks with feedback. Backpropagation is an algorithm driven by error, whereas CHL is a Hebbian-type algorithm, with update rules based on the correlation of pre- and postsynaptic activities. CHL has been shown to be equivalent to backpropagation for networks with only a single layer (Movellan, 1990), and there has been some other work to relate CHL to the general framework of backpropagation (HopŽeld, 1987; O’Reilly, 1996; Hinton & McClelland, 1988). However, a direct link between them for networks with hidden neurons has been lacking. To investigate the relationship between these two algorithms, we consider a special network for which CHL and backpropagation are equivalent. This is a multilayer perceptron to which weak feedback connections have been added and with output neurons that are linear. The equivalence holds because in CHL, clamping the output neurons at their desired values causes the hidden neurons to change their activities, and this change turns out to be equal to the error signal spread by backpropagation, except for a scalar factor. 2 The Learning Algorithms

In this section, we describe the backpropagation and CHL algorithms. Backpropagation is in the standard form, implemented in a multilayer perceptron (Rumelhart et al., 1986b). CHL is formulated in a layered network with feedback connections between neighboring layers of neurons. It is an extension of the typical CHL algorithm formulated for recurrent symmetric networks (Movellan, 1990). 2.1 Backpropagation. Consider a multilayer perceptron with L C 1 layers of neurons and L layers of synaptic weights (see Figure 1A). The activities of the kth layer of neurons are denoted by the vector xk, their biases by the vector b k, and the synaptic connections from layer k ¡ 1 to layer k by the matrix W k . All neurons in the kth layer are assumed to have the same transfer function fk, but this transfer function may vary from layer to layer. In particular, we will be interested in the case where fL is linear, though the other transfer functions may be nonlinear. In the basic deŽnition, fk acts on a scalar and returns a scalar. However, we will generally use fk to act on a

Backpropagation and Contrastive Hebbian Learning

443

Figure 1: Diagram on the network structures of the (A) multilayer perceptron and the (B) layered network with feedback connections. Layer 0 is the input, layer L is the output, and the others are hidden layers. The forward connections are the same for both networks. In B, there exist feedback connections between neighboring layers of neurons.

vector, in which case it returns a vector, operating component by component by fk. fk0 is the derivative of fk with respect to its argument. Similar to fk, when fk0 acts on a vector, it returns a vector as well, operated by fk0 in each component. We assume that fk is monotonically increasing. Backpropagation learning is implemented by repeating the following steps for each example in a training set of input-output pairs: 1. In the forward pass, x k D fk (W kxk¡1 C bk )

(2.1)

is evaluated for k D 1 to L, thereby mapping the input x0 to the output xL .

444

X. Xie and H. Seung

2. The desired output d of the network, provided by some teacher, is compared with the actual output xL to compute an error signal, yL D D L ( d ¡ xL ) .

(2.2)

The matrix D k ´ diagffk0 (W kxk¡1 C bk )g is deŽned by placing the components of the vector fk0 ( W kx k¡1 C b k ) in the diagonal entries of a matrix. 3. The error signal is propagated backward from the output layer by evaluating y k¡1 D Dk¡1 W Tk y k

(2.3)

for k D L to 2.

4. The weight update

D Wk

D gy k xTk¡1

(2.4)

is made for k D 1 to L, where g > 0 is a parameter controlling the learning rate. 2.2 Contrastive Hebbian Learning. To formulate CHL, we consider a modiŽed network in which, in addition to the feedforward connections from layer k ¡ 1 to layer k, there are also feedback connections between neighboring layers (see Figure 1B). The feedback connections are assumed to be symmetric with the feedforward connections, except that they are multiplied by a positive factor c . In other words, the matrix c W Tk contains the feedback connections from layer k back to layer k ¡ 1. CHL is implemented by repeating the following steps for each example of the training set:

1. The input layer x 0 is held Žxed, and the dynamical equations dx k ) C x k D f k ( W k x k¡1 C c W T kC 1 x kC 1 C b k dt

(2.5)

for k D 1 to L are run until convergence to a Žxed point. The case k D L is deŽned by setting xL C 1 D 0 and WL C 1 D 0. Convergence to a Žxed point is guaranteed under rather general conditions, to be shown later. This is called the free state of the network and is denoted by xL k for the kth layer neurons. 2. The anti-Hebbian update,

D Wk

D ¡gc k¡L xL k xL Tk¡1 ,

is made for k D 1, . . . , L.

(2.6)

Backpropagation and Contrastive Hebbian Learning

445

3. The output layer xL is clamped at the desired value d, and the dynamical equation 2.5 for k D 1 to L ¡ 1 is run until convergence to a Žxed point. This is called the clamped state and is denoted by xO k for the kth layer neurons. 4. The Hebbian update,

D Wk D gc

k¡L

xO kxO Tk¡1 ,

(2.7)

is made for k D 1, . . . , L. Alternatively, the weight updates could be combined and made after both clamped and free states are computed,

D W k D gc

k¡L ( O

x kxO Tk¡1 ¡ xL kxL Tk¡1 ).

(2.8)

This form is the one used in our analysis. This version of CHL should look familiar to anyone who knows the conventional version, implemented in symmetric networks. It will be derived in section 4, but Žrst we prove its equivalence to the backpropagation algorithm. 3 Equivalence in the Limit of Weak Feedback

Next, we prove that CHL in equation 2.8 is equivalent to the backpropagation algorithm in equation 2.4, provided that the feedback is sufŽciently weak and the output neurons are linear. In notation, x k, xO k, and xL k represent the kth layer activities of the feedforward network, the clamped state, and the free state, respectively. We consider the case of weak feedback connections, c ¿ 1, and use ¼ to mean that terms of higher order in c have been neglected and » to denote the order. The proof consists of the following four steps: 1. Show that the difference between the feedforward and free states is of order c in each component, d xL k ´ xL k ¡ x k » c ,

(3.1)

for all k D 1, . . . , L.

2. Show that in the limit of weak feedback, the difference between the clamped and free states satisŽes the following iterative relationship, dx k ´ xO k ¡ xL k D c DkW TkC 1dx kC 1 C O (c for k D 1, . . . , L ¡ 1, and dxL D d ¡ xL L .

L¡k C 1 )

,

(3.2)

446

X. Xie and H. Seung

3. Show that if the output neurons are linear, dxk is related to the error signal in backpropagation through dx k D c

L¡k

y k C O (c

L¡k C 1 )

.

(3.3)

4. Show that the CHL update can be approximated by

D Wk

D gy k xTk¡1 C O (c ) .

(3.4)

In the CHL algorithm, clamping the output layer causes changes in the output neurons to spread backward to the hidden layers because of the feedback connections. Hence, the new clamped state differs from the free state over the entire network, including the hidden neurons. Equation 3.2 states that dx k decays exponentially with distance from the output layer of the network. This is because the feedback is weak, so that dxk is reduced from dxkC 1 by a factor of c . Remarkably, as indicated in equation 3.3, the difference between the clamped and free states is equivalent to the error signal y k computed in the backward pass of backpropagation, except for a factor of c L¡k, when the output neurons are linear. Moreover, this factor annihilates the factor ofc k¡L in the CHL rule of equation 2.8, resulting in the update rule equation 3.4. 3.1 Proof. To prove the Žrst step, we start from the steady-state equation of the free phase,

xL k D fk ( W kxL k¡1 C bk C c W TkC 1 xL kC 1 ) ,

(3.5)

for k D 1, . . . , L ¡ 1. Subtracting this equation from equation 2.1 and performing Taylor expansion, we derive d xL k ´ xL k ¡ x k

D fk ( W k xL k¡1 C bk C c

(3.6) W TkC 1 xL kC 1 )

¡ fk ( W k xk¡1 C bk )

(3.7)

D D k W kd xL k¡1 C c D k W kTC 1 xL k C 1 C O (kW kd xL k¡1 C c W kTC 1 xL k C 1 k2 ) (3.8)

for all hidden layers, and d xL L D DL WLd xL L¡1 C O (kWLd xL L¡1 k2 ) for the output layer. In equation 3.7, the expansion is done around W kx k¡1 C bk. Since the zeroth layer is Žxed with the input, d xL 0 D 0, under the above iterative relationships, we must have d xL k » c in each component, that is, d xL k is in the order of c , for all k D 1, . . . , L. To prove equation 3.2, we compare the Žxed-point equations of the clamped and free states, f ¡1 ( xL k ) D W kxL k¡1 C bk C c W TkC 1 xL kC 1

(3.9)

f ¡1 ( xO k ) D W kxO k¡1 C bk C c W TkC 1 xO kC 1 ,

(3.10)

Backpropagation and Contrastive Hebbian Learning

447

for k D 1, . . . , L ¡ 1, where f ¡1 is the inverse function of f in each component. Subtract them, and perform Taylor expansion around xL k. Recall the deŽnition of dx k ´ xL k ¡ xO k . We have W kdxk¡1 C c W kTC 1dx kC 1 D f ¡1 (xO k ) ¡ f ¡1 ( xL k ) D Jkdx k C O (kd x k k2 ) ,

(3.11) (3.12)

where the matrix Jk ´ diagf@f ¡1 ( xL k ) / @xL kg. Since xL k ¡ xk » c , to the leading order inc , matrix Jk can be approximated by J k ¼ diagf@f ¡1 ( x k ) / @x kg D D ¡1 , k (c C and Jkdx k D D ¡1 dx O kdx k). Substituting this back to equation 3.12, k k k we get dx k D Dk (W kdx k¡1 C c W TkC 1dxkC 1 ) C O (c kdx kk) C O (kd x kk2 ) .

(3.13)

Let us start from k D 1. Since the input is Žxed (dx 0 D 0), we have dx1 D c D1 W 2Tdx2 C O (c kdx1 k) , and therefore, dx1 D c D1 W 2Tdx2 C O (c 2 kdx2 k). Next, we check for k D 2, in which case dx2 D D2 W2dx1 C c D2 W3Tdx3 C O (c kdx2 k) . Substituting dx1 , we have dx2 D c D2 W3Tdx3 C O (c kdx2 k), and therefore, dx2 D c D2 W3Tdx3 C O (c 2 kdx3 k). Following this iteratively, we have dx k D c D kW TkC 1dxkC 1 C O (c 2 kdx kC 1 k),

(3.14)

for all k D 1, . . . , L ¡ 1. Notice that dxL D d ¡ xL L D d ¡ xL C O (c ) is of order 1. Hence, dxL¡1 D c DL¡1 WLTdxL C O (c 2 ) , and iteratively, we have dx k D c D kW TkC 1dxkC 1 C O (c

L¡k C 1 )

,

(3.15)

for k D 1, . . . , L¡1. Therefore, equation 3.2 is proved. This equation indicates that dx k » c dx kC 1 and dx k » c L¡k in each component. If the output neurons are linear, then yL D dxL C O (c ) . Consequently, dxk D c L¡ky k C O (c L¡kC 1 ) for all k D 1, . . . , L. Finally, the weight update rule of CHL follows:

D Wk

D gc k¡L ( xO k xO Tk¡1 ¡ xL k xL Tk¡1 )

(3.16)

D gc k¡Ldx k xL Tk¡1 C gc k¡L xL kdxTk¡1 C gc k¡Ldx kdxTk¡1

(3.17)

D gc k¡Ldx k xL Tk¡1 C O (c )

(3.18)

D gy k xTk¡1 C O (c ).

(3.19)

The last approximation is made because xL k¡1 ¡ x k¡1 » c . This result shows that the CHL algorithm in the layered network with linear output neurons is identical to the backpropagation as c ! 0.

448

X. Xie and H. Seung

For nonlinear output neurons, dxL of CHL is different from yL in equation 2.2 computed in backpropagation. However, they are within 90 degrees if viewed as vectors. Moreover, if the activation function for the output neurons is taken to be the sigmoidal function, that is, fL ( z ) D 1 / (1 C exp(¡z) ) , the CHL algorithm is equivalent to backpropagation based on the cost function, Remark.

¡ dT log(xL ) ¡ (1 ¡ d ) T log(1 ¡ xL ) ,

(3.20)

since in this case, yL D d ¡ xL . In the above, function log acts on a vector and returns a vector as well. 4 Contrastive Function

The CHL algorithm stated in section 2.2 can be shown to perform gradient descent on a contrastive function that is deŽned as the difference of the network’s Lyapunov functions between clamped and free states (Movellan, 1990; Baldi & Pineda, 1991). Suppose E ( x) is a Lyapunov function of the dynamics in equation 2.5. Construct the contrastive function C (W ) ´ E ( xO ) ¡ E ( xL ) , where xO and xL are steady states of the whole network in the clamped and free phase, respectively, and W ´ fW1 , . . . , W L g. For simplicity, let us Žrst assume that E ( x) has a unique global minimum in the range of x and no local minima. According to the deŽnition of Lyapunov functions, xL is the global minimum O but under the extra constraints that the output neurons are of E and so is x, clamped at d. Therefore, C ( W ) D E ( xO ) ¡ E ( xL ) ¸ 0 and achieves zero if and O that is, when the output neurons reach the desired values. only if xL D x, Performing gradient descent on C ( W ) leads to the CHL algorithm. On the other hand, if E ( x) does not have a unique minimum, xL and xO may be only local minima. However, the above discussion still holds, provided that xO is in the basin of attraction of xL under the free phase dynamics. This imposes some constraints on how to reset the initial state of the network after each phase. One strategy is to let the clamped phase settle to the steady state Žrst and then run the free phase without resetting hidden neurons. This will guarantee that C (W ) is always nonnegative and constitutes a proper cost function. Next, we introduce a Lyapunov function for the network dynamics in equation 2.5, E (x ) D

L X kD 1

c

k¡L [1T

FN k ( x k ) ¡ xTk W kx k¡1 ¡ bTk x k],

(4.1)

where function FN k is deŽned so that FN0k ( x ) D fk¡1 (x ) . x ´ fx1 , . . . , xL g represents the states of all layers of the network. Equation 4.1 is extended from

Backpropagation and Contrastive Hebbian Learning

449

Lyapunov functions previously introduced for recurrent networks (HopŽeld, 1984; Cohen & Grossberg, 1983). For E ( x ) to be a Lyapunov function, it must be nonincreasing under the dynamics equation 2.5. This can be shown by EP D D D

´T L ³ X @E xk

kD 1 L X

c

xP k

k¡L [f ¡1 ( ) xk k

kD 1 L X kD 1

¡c

(4.2)

¡ W kx k¡1 ¡ c W TkC 1 x kC 1 ¡ bk ]T xP k

k¡L [f ¡1 ( ) xk k

¡ fk¡1 ( xP k C x k ) ]T [x k ¡ (xP k C x k ) ]

· 0,

(4.3)

(4.4) (4.5)

where the last inequality holds because fk is monotonic as we have assumed. Therefore, E (x ) is nonincreasing following the dynamics and stationary if and only if at the Žxed points. Furthermore, with appropriately chosen fk, such as sigmoid functions, E ( x) is also bounded below, in which case E ( x) is a Lyapunov function. Given the Lyapunov function, we can form the contrastive function C ( W ) and derive the gradient-descent algorithm on C accordingly. The derivative of E (xO ) with respect to W k is X @E @xO k dE ( xO ) @E C D dW k @W k @xO k @W k k D

(4.6)

@E

(4.7)

@W k

D ¡c k¡L xO k xO Tk¡1 ,

(4.8)

where the second equality holds because @E / @xO k D 0 for all k at the steady states. Similarly, we derive dE ( xL ) D ¡c dW k

k¡L L

x kxL Tk¡1 .

(4.9)

Combining equations 4.8 and 4.9, we Žnd the derivative of C ( W ) with respect to W k shall read dC dE ( xO ) dE ( xL ) ¡ D D ¡c dW k dW k dW k

k¡L (

xO k xO Tk¡1 ¡ xL kxL Tk¡1 ).

(4.10)

With a suitable learning rate, gradient descent on C ( W ) leads to the CHL algorithm in equation 2.8.

450

X. Xie and H. Seung

5 Equivalence of Cost Functions

In section 3, we proved that the CHL algorithm in the layered network with linear output neurons is equivalent to backpropagation in the weak feedback limit. Since both algorithms perform gradient descent on some cost function, the equivalence in the update rule implies that their cost functions should be equal, up to a multiplicative or an additive constant difference. Next, we demonstrate this directly by comparing the cost functions of these two algorithms. The backpropagation learning algorithm is gradient descent on the squared difference, kd ¡ xL k2 / 2, between the desired and actual outputs of the network. For the CHL algorithm, the cost function is the difference of Lyapunov functions between the clamped and free states, as shown in the previous section. After reordering, it can be written as CD

L X

c

k¡L

kD 1

[1T ( FN k ( xO k ) ¡ FN k (xL k ) ) ¡ dxTk (W kxL k¡1 C bk ) ¡ dxTk¡1 W kT xO k]. (5.1)

Recall that dx k » c L¡k. Therefore, the dxk term above multiplied by the factor c k¡L is of order 1, whereas the dx k¡1 multiplied by the same factor is of order c , and thus can be neglected in the leading-order approximation. After this, we get CD

L X

c

k¡L [1T (

kD 1

FN k ( xO k ) ¡ FN k (xL k ) ) ¡ dxTk (W kxL k¡1 C bk ) ] C O (c ) .

(5.2)

If the output neurons are linear (fL ( x) D x), then FNL ( x) D xT x / 2 and WL xL L¡1 C bL D xL L . Substituting them into C and separating terms of the output and hidden layers, we derive CD

1 T ( xO xO L ¡ xL TL xL L ¡ dxTL xL L ) 2 L C

L¡1 X kD 1

c

dxTk [fk¡1 ( xL k ) ¡ W kxL k¡1 ¡ bk ] C O (c )

k¡L

1 D kd ¡ xL k2 C O (c ) , 2

(5.3)

where the second term with the sum vanishes because of the Žxed-point equations. In conclusion, to the leading order in c , the contrastive function in CHL is equal to the squared error cost function of backpropagation. The demonstration on the equality of the cost functions provides another perspective on the equivalence between these two forms of learning algorithms.

Backpropagation and Contrastive Hebbian Learning

451

So far, we have always assumed that the output neurons are linear. If this is not true, how different is the cost function of CHL from that of backpropagation? Repeating the above derivation, we get the cost function of CHL for nonlinear output neurons, C D 1T FNL ( xO L ) ¡ 1T FNL ( xL L ) ¡ dxTL fL¡1 (xL L ) C O (c )

¡1 TN D ¡1T FNL ( xL ) ¡ dxT L fL ( xL ) C 1 FL ( d ) C O (c ).

(5.4) (5.5)

With sigmoidal-type activation function for output neurons, FNL ( z ) D x log(z) C (1 ¡ z ) log(1 ¡ z) . Substituting this into equation 5.5, we Žnd C D ¡dT log(xL ) ¡ (1 ¡ d) T log(1 ¡ xL ) C 1T FNL ( d) C O (c ) ,

(5.6)

which is the same as the cost function in equation 3.20 in the small c limit, except a constant difference. 6 Simulation

In this section, we use the backpropagation and the CHL algorithm to train a 784-10-10 three-layer network to perform handwritten digit recognition. The data we use are abridged from the MNIST database containing 5000 training examples and 1000 testing examples. We use the sigmoidal function fk ( x ) D 1 / (1 C exp(¡x) ) for the hidden and output layers. The backpropagation algorithm is based on the cost function in equation 3.20. We simulate the CHL algorithm in the layered network with three different feedback strengths: c D 0.05, 0.1, and 0.5. The label of an input example is determined by the index of the largest output. After each epoch of on-line training, the classiŽcation and squared error for both training and testing examples are computed and plotted in Figure 2. The classiŽcation error is deŽned as the percentage of examples classiŽed incorrectly. The squared error is deŽned as the mean squared difference between the actual and desired output for all training or testing examples. The desired output is 1 on the output neuron whose index is the same as the label and zero on the output neurons otherwise. The learning curve of CHL algorithm is very similar to those of backpropagation for c D 0.05 and 0.1, and it deviates from the learning curve of backpropagation for c D 0.5 (see Figure 2). In the simulation, we Žnd the overall simulation time of the CHL algorithm is not signiŽcantly longer than that of the backpropagation. This is because the layered network tends to converge to a steady state fast in the case of weak feedback. 7 Discussion

We have shown that backpropagation in multilayer perceptrons can be equivalently implemented by the CHL algorithm if weak feedback is added.

452

X. Xie and H. Seung

Figure 2: Comparison of performance between the backpropagation and the CHL algorithm. The two algorithms are used to train a 784-10-10 three-layer network to perform handwritten digit recognition. The CHL algorithm is used to train the three-layer network with feedback c D 0.05, 0.1, and 0.5 added. (A, B) ClassiŽcation and squared errors for training examples. (C, D) Test examples.

This is demonstrated from two perspectives: evaluating the two algorithms directly and comparing their cost functions. The essence behind this equivalence is that CHL effectively extracts the error signal of backpropagation from the difference between the clamped and free steady states. The equivalence between CHL and backpropagation in layered networks holds in the limit of weak feedback, which is true mathematically. This, however, does not imply that in engineering problem solving, we should substitute CHL for backpropagation to train neural networks. This is because in networks with many hidden layers, the difference between the clamped and free states in the Žrst few layers would become very small in the limit of weak feedback, and therefore CHL will not be robust against noise during training. In practice, when CHL is used for training the layered networks with many hidden layers, the feedback strength should not be chosen to be too small, in which case the approximation of CHL to backpropagation algorithm will be inaccurate.

Backpropagation and Contrastive Hebbian Learning

453

The investigation on the relationship between backpropagation and CHL is motivated by research looking for biologically plausible learning algorithms. It is believed by many that backpropagation is not biologically realistic. However, in an interesting study on coordinate transform in posterior parietal cortex of monkeys, Zipser and Anderson (1988) show that hidden neurons in a network model trained by backpropagation share very similar properties to real neurons recorded from that area. This work prompted the search for a learning algorithm, which has similar functionality as backpropagation (Crick, 1989; Mazzoni, Andersen, & Jordan, 1991) and at the same time is biologically plausible. CHL is a Hebbian-type learning algorithm, relying on only pre- and postsynaptic activities. The implementation of backpropagation equivalently by CHL suggests that CHL could be a possible solution to this problem. Mazzoni et al. (1991) also proposed a biologically plausible learning rule as an alternative to backpropagation. Their algorithm is a reinforcementtype learning algorithm, which is usually slow, has large variance, and depends on global signals. In contrast, the CHL algorithm is a deterministic algorithm, which could be much faster than reinforcement learning algorithms. However, a disadvantage of CHL is its dependence on special network structures, such as the layered network in our case. Whether either algorithm is used by biological systems is an important question that needs further investigation in experiments and theory. Acknowledgments

We acknowledge helpful discussions with J. J. HopŽeld and S. Roweis. We thank J. Movellan for suggesting the error function for nonlinear output neurons. References Ackley, D. H., Hinton, G. E., & Sejnowski, T. J. (1985). A learning algorithm for Boltzmann Machines. Cognitive Science, 9, 147–169. Baldi, P., & Pineda, F. (1991). Contrastive learning and neural oscillator. Neural Computation, 3, 526–545. Cohen, M. A., & Grossberg, S. (1983). Absolute stability of global pattern formation and parallel memory storage by competitive neural networks. IEEE Trans. on Systems, Man, and Cybernetics, 13, 815–826. Crick, F. (1989). The recent excitement about neural networks. Nature, 337, 129–132. Hinton, G. E. (1989). Deterministic Boltzmann learning performs steepest descent in weight-space. Neural Computation, 1, 143–150. Hinton, G. E., & McClelland, J. (1988). Learning representations by recirculation. In D. Z. Anderson (Ed.), Neural Information Processing Systems. New York: American Institute of Physics.

454

X. Xie and H. Seung

HopŽeld, J. J. (1984). Neurons with graded response have collective computational properties like those of two-state neurons. Proc. Natl. Acad. Sci. USA, 81, 3088–3092. HopŽeld, J. J. (1987). Learning algorithms and probability distributions in feedforward and feed-back networks. Proc. Natl. Acad. Sci. USA, 84, 8429–8433. Mazzoni, P., Andersen, R. A., & Jordan, M. I. (1991). A more biologically plausible learning rule for neural networks. Proc. Natl. Acad. Sci. USA, 88, 4433–4437. Movellan, J. (1990). Contrastive Hebbian learning in the continuous HopŽeld model. In D. Touretzky, J. Elman, T. Sejnowski, & G. Hinton (Eds.), Proceedings of the 1990 Connectionist Models Summer School (pp. 10–17). San Mateo, CA: Morgan Kaufmann. O’Reilly, R. (1996). Biologically plausible error-driven learning using local activation differences: The generalized recirculation algorithm. Neural Computation, 8, 895–938. Peterson, C., & Anderson, J. (1987). A mean Želd theory learning algorithm for neural networks. Complex Systems, 1, 995–1019. Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986a). Learning internal representations by error propagation. In D. E. Rumelhart, & J. L. McClelland (Eds.), Parallel distributed processing: Explorations in the microstructure of cognition. Vol. 1: Foundations (pp. 318–362). Cambridge, MA: MIT Press. Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986b). Learning representations by back–propagating errors. Nature, 323, 533–536. Zipser, D., & Andersen, R. A. (1988). A back-propagation programmed network that simulates response properties of a subset of posterior parietal neurons. Nature (London), 331, 679–684.

Received January 25, 2002; accepted August 1, 2002.

LETTER

Communicated by Maxwell Stinchcombe

Indexed Families of Functionals and Gaussian Radial Basis Functions Irwin W. Sandberg [email protected] Department of Electrical and Computer Engineering, University of Texas at Austin, Austin, TX 78712, U.S.A. We report on results concerning the capabilities of gaussian radial basis function networks in the setting of inner product spaces that need not be Žnite dimensional. SpeciŽcally, we show that important indexed families of functionals can be uniformly approximated, with the approximation uniform also with respect to the index. Applications are described concerning the classiŽcation of signals and the synthesis of reconŽgurable classiŽers. 1 Introduction

Radial basis functions are of interest in connection with a variety of approximation problems in the neural networks area and in other areas as well. Much is understood about the properties of these functions (Park & Sandberg, 1991, 1993; Hartman, Keeler, & Kowalski, 1990; Lo, 1972; Parzen, 1962). It is known (Park & Sandberg, 1993), for example, that arbitrarily good approximation in L1 ( Rn ) of a general f 2 L1 ( Rn ) is possible using uniform smoothing factors and radial basis functions generated in a certain natural way from a single g in L1 ( Rn ) if and only if g has a nonzero integral. As another example, Hartman et al. (1990) proved that gaussian radial basis functions can uniformly approximate arbitrarily well any continuous real functional deŽned on a compact convex subset of Rn . In Sandberg (2001), an approximation result is proved concerning gaussian radial basis functions in a general inner product space, and two applications are given. In particular, it is shown that gaussian radial basis functions deŽned on Rn can in fact uniformly approximate arbitrarily well over all of Rn any continuous real functional f on Rn that meets the condition that lim | f ( x ) | D 0.

kxk!1

This generalizes the result in Hartman et al. (1990) because, by the LebesgueUrysohn extension theorem (Stone, 1962), sometimes attributed to Tietze, any continuous real functional deŽned on a bounded closed subset of Rn Neural Computation 15, 455–468 (2003)

° c 2002 Massachusetts Institute of Technology

456

I. Sandberg

Figure 1: ClassiŽcation network.

can be extended so that it is deŽned and continuous on all of Rn and meets the above condition.1 The second application concerns the problem of classifying signals, which is of interest in several application areas (e.g., signal detection, computerassisted medical diagnosis, automatic target identiŽcation). Typically, we are given a Žnite number m of pairwise disjoint sets C1 , . . . , Cm of signals, and we would like to synthesize a system that maps the elements of each Cj into a real number aj so that the numbers a1 , . . . , am are distinct. Sandberg (1994) (see also Sandberg et al., 2001) showed that the structure displayed in Figure 1 can perform this classiŽcation for any prescribed distinct a1 , . . . , am , assuming only that the Cj are compact subsets of a real normed linear space. In the Žgure, x is the signal to be classiŽed, y1 , . . . , yk are functionals that can be taken to be linear,2 h denotes a continuous memoryless nonlinear mapping that, for example, can be implemented by a neural network having one hidden nonlinear layer, the blocklabeled Q is a quantizer that for each j maps numbers in the interval (aj ¡0.25r , aj C 0.25r) into aj , where r D miniD6 j |ai ¡aj |, and w is the output of the classiŽer.3 Sandberg (2001) showed that a similar classiŽcation problem can be solved in a very different way using just a generalized gaussian radial basis function network and a quantizer of the type in Figure 1. The network structure considered can also classify signals x whose critical property is that the norm of x is sufŽciently large.

1

For example, let a continuous f0 deŽned on a bounded closed subset A of Rn be given, and let A be contained in an open ball B centered at the origin of Rn . Then, since the complement C of B with respect to Rn is closed (and thus A [ C is closed), by the Lebesgue-Urysohn extension theorem there is a continuous extension f of f0 deŽned on Rn such that f (x) D 0, x 2 C. 2 Examples are given in Sandberg (1994) and Sandberg et al. (2001). 3 The proof given in Sandberg (1994) and Sandberg et al. (2001) expands on a brief remark in Sandberg (1991) that any continuous real functional on a compact subset of a real normed linear space can be uniformly approximated arbitrarily well using only a feedforward neural network with a linear-functional input layer and one memoryless hidden nonlinear (e.g., sigmoidal) layer.

Families of Functionals

457

This article too is concerned with the capabilities of gaussian radial basis functions. Here we show that important indexed families f f (a, ¢) : a 2 Ag of functionals of the type addressed in Sandberg (2001) can be uniformly approximated, with the approximation uniform also with respect to the index. More speciŽcally, we consider a setting in which A denotes, for example, the index set [0, 1], and X2 (we avoid the use of X or X1 because they are used differently in the following section) is taken to be Rn or a certain more general type of subset of an inner product space S2 that may be, for instance, a function space. We show that large families of continuous real functionals f on A £ X2 can be uniformly approximated arbitrarily well using gaussian radial basis functions, in the sense that given 2 > 0, there are a positive integer q, real functions c1 , . . . , cq deŽned on A, positive numbers b1 , . . . , bq , and elements u1 , . . . , uq of a certain subset of S2 such that ­ ­ q ­ ­ X ­ 2 ­ <2 (1.1) f (a, x) ¡ ck ( a) expf¡b kkx ¡ u kk g­ ­ ­ ­ kD 1

for all ( a, x ) 2 A £ X2 , in which k¢k is the norm in S2 . We show also that the coefŽcients ck (a ) can be taken to be given by ck ( a) D ak expf¡b k |a ¡ a k | 2 g,

(1.2)

in which a1 , . . . , aq are real numbers and a1 , . . . , aq belong to A. In particular, for X2 D S2 D Rn , approximation in the sense of inequality 1.1 holds whenever f is continuous on A £ X2 and lim | f ( a, x ) | D 0

kxk!1

uniformly in a (meaning that for each c > 0, there is a j > 0 such that | f ( a, x ) | < c for kxk > j and all a 2 A.). Such f ’s arise naturally. For example, let p be an arbitrary positive integer, and let f1 , f2 , . . . , fp be p continuous real functionals on Rn with the property that lim | fq ( x ) | D 0

kxk!1

for each q. Let I denote f1, 2, . . . , pg, and let A be any bounded closed real interval containing I. There are many functions f from A £ Rn to R such that f (q, x ) D fq ( x ) for all (q, x ) 2 I £ Rn . The collection of these functions is of interest with regard to the possibility of synthesizing an easily reconŽgurable network that can approximate any one of the p functionals, with “easily reconŽgurable” meaning essentially that one simply changes the point a at which the coefŽcients in equation 1.1 are evaluated. An example of such an f : A £ Rn ! R is given by the Lagrange-type interpolation relation, f ( a, x ) D

p X qD 1

c q ( a ) fq ( x ) ,

(1.3)

458

I. Sandberg

in which c q (a ) D

P `2I¡fqg ( a ¡ `) . P `2I¡fqg ( q ¡ `)

This f satisŽes the two conditions mentioned above for approximation in the sense of equation 1.1. To see this, note that since each c q is continuous on A and each fj is continuous on Rn , f is continuous on A £ Rn ,4 and that given c > 0, we may choose j so that for all j, max max |c q ( a) | ¢ | fj ( x ) | < c / p, kxk > j, q

a

in which case we have | f ( a, x ) | ·

p X q D1

|c q (a ) fq ( x ) | · max max |c q ( a) | q

a

p X

| fj (x ) | < c

jD 1

for kxk > j and all a 2 A. Motivated by the connection described in Sandberg (2001) between classiŽcation problems and the implementation of functionals as networks, this leads to the conclusion that there are classiŽers on Rn that are reconŽgurable in a simple manner. (See theorem 4 for an example of a precise statement along these lines, and see theorem 7 for a related proposition.) The results in section 2 show that this conclusion holds also for a large collection of classiŽcation problems in which the objects to be classiŽed are drawn from a function space, and a speciŽc result is given that addresses the case in which the objects are Lipschitz continuous functions deŽned on the n-dimensional interval [0, 1]n . There are applications of the material in section 2 to problems other than classiŽcation problems. In particular, because of the close relation between the problem of approximating shift-invariant nonlinear input-output maps, in which inputs and outputs depend on a time or space variable, and the problem of approximating functionals (see, for instance, Sandberg et al., 2001), the results in section 2 are of direct interest concerning the problem of approximating indexed families of input-output maps. In this connection, cases in which the elements of f f (a, ¢) : a 2 Ag are deŽned on a compact set are also addressed in section 2. The problem of approximating indexed families of maps arises also in connection with shift-varying nonlinear input-output maps, in which time or a space variable plays the role of the index (Sandberg et al., 2001; Sandberg, 1998). It comes up in addition in connection with studies of adaptive systems. 5 4 Regard eachc q and each fj as deŽned on A£Rn (with any of the usual product metrics; Sutherland, 1975), and use the proposition that the sum and product of two continuous R-valued functions on a metric space are continuous. 5 See, in particular, Lo and Bassu (2002) in which the focus is on approximation using sigmoidal functions and maps deŽned on compact subsets of Žnite-dimensional spaces.

Families of Functionals

459

2 Approximation of Indexed Families 2.1 Preliminaries. We Žrst describe a theorem in Sandberg (2001) that we will use. Let S be a real inner-product space6 (i.e., a real pre-Hilbert space) with inner product h¢, ¢i and norm k¢k derived in the usual way from h¢, ¢i. Let X be a metric space whose points are a subset of the points of S, and denote the metric in X by d. With V any convex subset of S such that fk¢ C vk: v 2 Vg separates the points of X (i.e., such that for x and y in X 6 6 with x D ky C vk), and with P any y, there is a v 2 V for which kx C vk D nonempty subset of (0, 1) that is closed under addition, let X 0 denote the set of functions g deŽned on X that have the representation

g ( x) D a expf¡bkx ¡ vk2 g

(2.1)

in which a 2 R, b 2 P, and v 2 V. Let X stand for the set of continuous functions f from X to the reals R with the property that for each 2 > 0, there is a compact subset X f,2 of X such that | f ( x ) ¡ f ( y) | < 2 for x, y 2/ X f,2 . And let X 1 denote the family of f ’s in X such that for each f and each 2 > 0, there is a compact subset X f,2 of X such that | f ( x ) | < 2 for x 2/ X f,2 . The result in Sandberg (2001) is the following: Assume that X is locally compact but not compact7 and that X is such that X 0 ½ X 1 . Then for each f 2 X 1 and each 2 > 0, there are a positive integer q, real numbers a1 , . . . , aq , numbers b 1 , . . . , bq belonging to P, and elements v1 , . . . , vq of V such that Theorem 1.

­ ­ q ­ ­ X ­ 2 ­ ak expf¡b kkx ¡ v kk g­ <2 f (x ) ¡ ­ ­ ­

(2.2)

kD 1

for all x 2 X. Because of, for example, the simple nature of the expression for the ck (a) in equation 1.2, the results in this article are of interest with regard to the approach discussed in Lo and Bassu (2002). For material related in a general sense concerning applications, see, for instance, Lainiotis (1976). In this context, see also material in the control systems literature concerning the topic of gain scheduling. 6 Theorems 1 and 2, which follow, hold also if all inner-product spaces considered are complex inner-product spaces. 7 X is locally compact if for each point x of X there is an open subset Ox of X such that x 2 Ox and the closure of Ox is compact. For example, Rn is locally compact.

460

I. Sandberg

6 Comments. Since the condition that kx ¡ vk D ky ¡ vk is equivalent to the

6 kxk2 ¡kyk2 , and assuming that x D 6 y in X implies condition that 2hx¡y, vi D 6 that x D y in S, we see that V can be taken to be, for instance, any convex subset of S that contains the points of an open ball in S. Of course, P can be taken to be (0, 1) or f1, 2, 3, . . .g. Since expf¡bkx ¡ vk2 g ! 0 as kxk ! 1 when b > 0, we have X 0 ½ X 1 when for each c > 0 there is a compact subset Xc of X such that kxk ¸ c for x 2 X with x 2/ Xc .

2.2 Product Spaces and Indexed Families. Let S1 be a real inner-product space with inner product h¢, ¢i1 and norm k¢k1 , and let S2 be a second real inner-product space with inner product h¢, ¢i2 and norm k¢k2 . The norms in S1 and S2 are derived from their respective inner products in the usual way. Take S (in section 2.1) to be the inner-product space S1 £ S2 with inner product h¢, ¢i given by

h(a1 , b1 ) ( a2 , b2 ) i D ha1 , a2 i1 C hb1 , b2 i2 . Let ( X1 , d1 ) and (X2 , d2 ) be two metric spaces whose points are subsets of the points of S1 and S2 , respectively. (The metrics d1 and d2 are not necessarily derived in the usual way from k¢k1 and k¢k2 .) Take X to be the metric space X1 £ X2 with the metric given by d[ (a1 , b1 ) , ( a2 , b2 )] D d1 (a1 , a2 ) C d2 ( b1 , b2 ) . Recall that V is a convex subset of S such that fk¢ C vk: v 2 Vg separates 6 the points of X (i.e., such that for x and y in X with x D y, there is a v 2 V 6 ky C vk), and that P is any nonempty subset of (0, 1) for which kx C vk D that is closed under addition. For v 2 S, we use v1 and v2 to denote the components of v belonging to S1 and S2 , respectively. One of our main results concerning the approximation of families of functionals is the following. Assume that ( X1 , d1 ) and ( X2 , d2 ) are locally compact but that (X1 , d1 ) or (X2 , d2 ) , or both, are not compact, and that X is such that X 0 ½ X 1 . Then for each f 2 X 1 and each 2 > 0, there are a positive integer q, real numbers a1 , . . . , aq , numbers b1 , . . . , bq belonging to P, and elements v1 , . . . , vq of V such that Theorem 2.

­ ­ q ­ ­ X ­ ­ 2 2 ak expf¡b kka ¡ v k1 k1 g expf¡b kkb ¡ vk2 k2 g­ <2 f ( a, b ) ¡ ­ ­ ­

(2.3)

kD 1

for all (a, b) 2 X.8 8 Any f 2 X can be similarly approximated by adding a real constant (that depends on f ) to the sum in inequality 2.3.

Families of Functionals

461

The proof follows from theorem 1 and the observation that for x D ( a, b ) 2 X, we have Proof.

expf¡b kkx ¡ v kk2 g D expf¡b kka ¡ v k1 k21 g expf¡b kkb ¡ v k2 k22 g. More speciŽcally, X is locally compact but not compact because ( X1 , d1 ) and ( X2 , d2 ) are locally compact and (X1 , d1 ) or ( X2 , d2 ) , or both, are not compact. The metric product X is locally compact (Mukherjea & Pothoven, 1978), and by the equivalence of compactness and sequential compactness in metric spaces, it easily follows that X is not compact.

In theorem 3, below, we refer to the following assumption. A.1: The metric d2 associated with X2 (recall that X2 is a subset of the points of a real innerproduct space S2 with inner product h¢, ¢i2 and norm k¢k2 ) is such that closed bounded subsets of (X2 , d2 ) are compact, ( X2 , d2 ) is unbounded, and such that for each c > 0, there is a compact subset Xc of ( X2 , d2 ) such that kxk2 ¸ c for x 2 X2 with x 2/ Xc . We give two examples. Assumption A.1 is met if X2 D S2 D Rn , in which Rn stands for the linear space of real n-vectors (as suggested earlier, n is an arbitrary positive integer), the inner product and corresponding norm k¢k2 on Rn are the usual ones, and the metric d2 associated with X2 is derived in the usual way from k¢k2 . For our other example, take S2 to be the inner-product space of real continuous functions deŽned on the n-dimensional interval [0, 1]n , with the inner product and corresponding norm on S2 speciŽed by Z hx, yi D

x (w ) y ( w ) dw. [0,1]n

With ` a positive number and with some norm on Rn , take X2 to be the set of all elements of S2 that are Lipschitz continuous with Lipschitz constant `, and let d2 be the usual uniform (i.e., max) metric. By a version of the Arzel`aAscoli theorem (Sutherland, 1975), closed bounded subsets of ( X2 , d2 ) are compact. In addition, (X2 , d2 ) is locally compact because the closure of any open ball in ( X2 , d2 ) is compact, and ( X2 , d2 ) is unbounded. By the equicontinuity of X2 , it follows that for each c > 0, there is a compact subset Xc of ( X2 , d2 ) such that kxk2 ¸ c for x 2 X2 with x 2/ Xc . More speciŽcally, let c be given and set d D sup a1 ,a2 2[0,1]n Ix2X2 |x(a1 ) ¡x(a2 ) |, which is Žnite. The subset Xc of (X2 , d2 ) given by fx 2 X2 : maxa |x(a) | · c C dg is compact, and we have Z

[0,1]n

|x(w) | 2 dw > c

2

for x 2 X2 with x 2/ Xc . This shows that A.1 is met.

462

I. Sandberg

With regard to slightly more general examples, we note that if A.1 is met, then A.1 is met also if ( X2 , d2 ) is replaced with ( XQ 2 , d2 ), where XQ 2 is any nonempty unbounded closed subset of ( X2 , d2 ) . Other examples can be given in which the elements of X2 are not necessarily continuous. Let A be a bounded closed subinterval of R. Suppose that A.1 holds, and let U be any convex subset of S2 such that fk¢ C uk2 : u 2 Ug separates the points of X2 . Let f be a continuous function from ( A, | ¢ |) £ ( X2 , d2 ) 9 to R such that for each c > 0, there is a compact subset Xc of (X2 , d2 ) for which Theorem 3.

6 Xc ) . | f ( a, x ) | < c ( a 2 A, x 2

(2.4)

Then for each 2 > 0 there are a positive integer q, real numbers a1 , . . . , aq , numbers b1 , . . . , bq belonging to P, elements a1 , . . . , aq of A, and elements u1 , . . . , uq of U such that ­ ­ q ­ ­ X ­ ­ ak expf¡b k |a ¡ a k | 2 g expf¡b kkx ¡ u kk22 g­ <2 f ( a, x ) ¡ ­ ­ ­

(2.5)

kD 1

for all (a, x) 2 A £ X2 . The theorem is a corollary of theorem 2 for S1 D R with the usual multiplication inner product, X1 D ( A, | ¢ |) , and the observation that A £ U is a convex subset of S D S1 £S2 for which fk¢ C vk: v 2 A £Ug separates the points of X. In particular, the space ( X2 , d2 ) is locally compact because each open ball in ( X2 , d2 ) is the interior of a compact subset of (X2 , d2 ) . (X2 , d2 ) , because it is unbounded, is thus locally compact but not compact, and of course A is compact and thus locally compact. We have f 2 X 1 because f is continuous, closed balls in ( A£X2 , d ) are compact (we show this below), and by condition 2.4 and the expression for d, in which maxa1 ,a2 2A |a1 ¡ a2 | < 1, for each 2 > 0 there is a closed ball in ( A £ X2 , d) outside of which | f (a, x) | < 2 . SpeciŽcally, let 2 be given, and select a compact subset X2 of ( X2 , d2 ) so 6 X2 and all a 2 A. Choose any ( a 0, x 0 ) 2 A £ X2 , that | f (a, x ) | < 2 for all x 2 and let r be the radius of a closed ball B 0 ( r) in ( X2 , d2 ) centered at x 0 such that B 0 (r ) contains the points of X2 . Observe that for ( a, x ) 2 A £ X2 , but outside the closed ball in ( A£X2 , d ) of radius r C maxa1 ,a2 2A |a1 ¡a2 | centered at ( a0 , x0 ) , we have x 2/ B 0 ( r) and thus x 2/ X2 . Now we make use of the hypothesis that for eachc > 0 there is a compact subset Xc of ( X2 , d2 ) such that kxk2 ¸ c for x 2 X2 with x 2/ Xc , and let Proof.

9 Here, as suggested, | ¢ | denotes the metric associated with the absolute-value norm. In accord with the material in section 2.2, the metric on (A, | ¢ | ) £ (X2 , d2 ) is the sum of the two metrics.

Families of Functionals

463

j > 0 be given. Select (a 0 , b0 ) 2 A £ X2 , and choose r > maxa2A |a ¡ a 0 | so that fx 2 Xj g ½ fb 2 X2 : d2 ( b, b0 ) · r ¡ max |a ¡ a 0 |g. a2A

Let B denote the closed ball in ( A £ X2 , d ) of radius r centered at ( a 0, b0 ). Then for ( a, b) 2 A £ X2 with (a, b) 2/ B, we have |a ¡ a0 | C d2 ( b, b0 ) > r and thus d2 ( b, b 0 ) > r ¡ maxa2A |a ¡ a 0 |, implying that b 2/ Xj . Therefore, for ( a, b) 2 A £ X2 with ( a, b) 2/ B, we have 1

k(a, b) k D (kak21 C kbk22 ) 2 ¸ kbk2 ¸ j, showing that X 0 ½ X 1 . Finally, let C denote a closed ball in (or any closed bounded subset of) ( A £ X2 , d) , and let f(a k, bk ) g1 be a sequence in C. By the compactness of A kD 0 and the compactness of closed bounded subsets of ( X2 , d2 ), there is a point ( a1 , b1 ) 2 A £ X2 and a subsequence f(a 0k, b0k ) g1 of f(ak, bk ) g1 such that kD 0 kD 0 |a 0k ¡ a1 | ! 0 as k ! 1 and d2 ( b0k, b1 ) ! 0 as k ! 1, and thus such that d[ ( a0k, b 0k ) , (a1 , b1 ) ] ! 0 as k ! 1. Since C is closed, (a1 , b1 ) 2 C. Thus, by the equivalence of compactness and sequential compactness in metric spaces, C is compact.10 In the case of either of the two examples given concerning the hypothesis A.1, U can be taken to be X2 , or even any subset of X2 that contains an open ball in ( X2 , d2 ) (see the comments following theorem 1). In theorem 3, a key assumption is that f is continuous on the product space ( A, |¢| ) £(X2 , d2 ). It is not enough that f be continuous separately in each of its two variables. A sufŽcient condition that is often useful is given in the appendix. Our next result concerns classiŽcation. Let U be any convex subset of S2 such that fk¢ C uk2 : u 2 Ug separates the points of X2 , suppose that A.1 holds, let d2 be derived in the usual way from a norm k¢kd2 , and let p be a positive integer. For each ` 2 f1, . . . , pg, let C1 ( `) , . . . , Cm (`) ( `) be pairwise disjoint compact subsets of ( X2 , d2 ) , in which m (`) is a positive integer, and let C0 ( `) D fx 2 X2 : kxkd2 ¸ j`g where j` > maxjD6 0 supfkxkd2 : x 2 Cj ( `) g. Also for each ` 2 f1, . . . , pg, let a0 ( `) , a1 ( `) , . . . , am ( `) (`) be distinct real numbers with a0 ( `) D 0 and, with r` D miniD6 j |ai ( `) ¡ aj (`) |, let Q`: [j ( aj (`) ¡ 0.25r `, aj ( `) C 0.25r `) ! R be speciŽed by Q (a ) ` D aj ( `) for a 2 ( aj ( `) ¡ 0.25r`, aj ( `) C 0.25r`) and each j. Then there are a positive integer Theorem 4.

10 Theorem 3 can be easily extended to cover the case in which A is instead a bounded closed n-dimensional subinterval of Rn , in which n is an arbitrary positive integer. In that case, | ¢ | in the theorem denotes the euclidean norm on Rn , and A D A1 £ ¢ ¢ ¢ £ An , in which the Aj are closed bounded subintervals of R. The proof is essentially the same.

464

I. Sandberg

q, real numbers a1 , . . . , aq , numbers b1 , . . . , bq belonging to P, elements a1 , . . . , aq of [1, p], and elements u1 , . . . , uq of U such that, for each ` 2 f1, . . . , pg, Á

q X

Q` k D1

! ak expf¡b k | ` ¡ ak | g expf¡b kkx 2

¡ u kk22 g

D aj ( `)

for x 2 Cj ( `) with j D 0, 1, . . . , m (`) . ( ) For each ` 2 f1, . . . , pg, let g`: [jmD 0` Cj ( `) ! R be deŽned by g` ( x ) D aj ( `) for x 2 Cj ( `) and each j. Observe that each [jmD 0 Cj ( `) is closed in (X2 , d2 ) , and that each g` is continuous (note that the distance between pairs of the Cj ( `) is positive). Using the Lebesgue-Urysohn extension theorem (Stone, 1962), let f` be a continuous extension of g` to (X2 , d2 ) for each `. Note that for every `, f`( x ) D 0 for kxkd2 ¸ j`. Proceeding as indicated in section 1, and with A D [1, p], construct a continuous function f from ( A, | ¢ |) £ ( X2 , d2 ) to R such that f (l, x ) D f` (x ) for each `, and f (a, x ) D 0 for a 2 A and x outside the compact set fx 2 X2 : kxkd2 · max` j`g. Thus, by theorem 3, for each 2 > 0 there are a positive integer q, real numbers a1 , . . . , aq , numbers b1 , . . . , bq belonging to P, elements a1 , . . . , aq of A, and elements u1 , . . . , uq of U such that ­ ­ q ­ ­ X ­ ­ 2 2 ak expf¡b k |a ¡ a k | g expf¡b kkx ¡ u kk2 g­ <2 (2.6) f ( a, x ) ¡ ­ ­ ­

Proof.

kD 1

for all ( a, x ) 2 A £ X2 . Since we may select 2 to be less than 0.25 min` r`, we have proved the theorem. Referring to the hypotheses of theorem 4, both examples given earlier (just before theorem 3) concerning A.1 are ones in which d2 is derived in the usual way from a norm. The interpretation of theorem 4 is that for each ` 2 f1, . . . , pg, the classiŽcation problem addressed by the theorem can be solved using a structure consisting of only a generalized RBF network followed by a quantizer, and that q and the ak, b k, a k, as well as the u k do not depend on `. Even for p D 1, this family of classiŽcation problems is more general than the one studied in Sandberg (1994) and Sandberg et al. (2001) in that here we also consider the problem of classifying signals x whose critical property is that the norm of x is sufŽciently large. This additional degree of generality is interesting from the viewpoint of understanding the capabilities of the structure considered, but it may not be of much signiŽcance from a practical viewpoint. A result similar to theorem 4, in which the additional class of signals is not considered and whose proof is along the same lines but is more direct, is given in the next section. 2.3 Related Results. The following theorem concerns cases in which (X1 , d1 ) and (X2 , d2 ) are compact.

Families of Functionals

465

Assume that ( X1 , d1 ) and ( X2 , d2 ) are compact. Then for each continuous f : X ! R, and each 2 > 0, there are a positive integer q, real numbers a1 , . . . , aq , numbers b1 , . . . , b q belonging to P, and elements v1 , . . . , vq of V such that ­ ­ q ­ ­ X ­ 2 2 ­ ak expf¡b kka ¡ vk1 k1 g expf¡b kkb ¡ v k2 k2 g­ <2 (2.7) f (a, b) ¡ ­ ­ ­ Theorem 5.

kD 1

for all ( a, b) 2 X. This theorem is a direct application of the following lemma given in Sandberg (2001). 11 Assume that X (of section 2.1) is compact, and let f : X ! R be continuous. Then for each 2 > 0, there are a positive integer q, real numbers a1 , . . . , aq , numbers b1 , . . . , b q belonging to P, and elements v1 , . . . , vq of V such that ­ ­ q ­ ­ X ­ ­ ak expf¡b kkx ¡ v kk2 g­ <2 (2.8) f (x ) ¡ ­ ­ ­ Lemma 1.

kD 1

for all x 2 X. Theorem 6, which follows, is a corollary of theorem 5 in which attention is focused on the approximation of a family of functionals indexed by a single parameter and deŽned on a compact set. Let A be a bounded closed subinterval of R. Suppose that (X2 , d2 ) is compact, and let U be any convex subset of S2 such that fk¢ C uk2 : u 2 Ug separates Theorem 6.

11 The lemma is an extension of the result in Hartman et al. (1990) concerning approximation on compact convex subsets of Rn . Theorems along the lines of theorem 5, involving, for example, sigmoidal functions or more general ridge functions instead of gaussian functions, can be proved using a theorem due to Dieudonne (see Stone, 1962), which asserts that if Y is the cartesian product of compact topological spaces Yc , c 2 C, then any continuous real functional on Y can be uniformly approximated by Žnite sums of Žnite products of continuous functions of one variable on Y. It is not difŽcult to check that for C a Žnite set and for each c , these continuous functions of one variable on Yc can be assumed to belong to any family of R-valued functions on Yc that is dense, in the sense of uniform approximation, in the set of continuous R-valued functions on Yc . However, this approach does not quite lead to theorem 5 in which the same constant bk appears in both exponential factors in the kth term in the sum in inequality 2.7. It is not difŽcult to see that theorem 5 can be generalized directly in that the exponential factors in inequality 2.7 can be replaced with other continuous functions drawn from corresponding dense sets. Also, in the light of a well-known one-point compactiŽcation proposition and a related continuous function extension idea (Stone, 1962) involving locally compact spaces, similar remarks hold in the context of theorem 2. In fact, the proof of theorem 1 given in Sandberg (2001) indirectly makes use of this material in Stone (1962).

466

I. Sandberg

the points of X2 . Let f be a continuous function from ( A, | ¢|) £ (X2 , d2 ) 12 to R. Then for each 2 > 0, there are a positive integer q, real numbers a1 , . . . , aq , numbers b1 , . . . , bq belonging to P, elements a1 , . . . , aq of A, and elements u1 , . . . , uq of U such that ­ ­ q ­ ­ X ­( ­ 2 2 ak expf¡b k |a ¡ ak | g expf¡b kkb ¡ u kk2 g­ <2 (2.9) f a, b ) ¡ ­ ­ ­ kD 1

for all (a, b) 2 A £ X2 . Theorem 7 is a version of theorem 4 whose proof is along the same lines but is more direct. Let U be a convex subset of S2 such that fk¢ C uk2 : u 2 Ug separates the points of X2 . For each ` 2 f1, . . . , pg, let C1 ( `) , . . . , Cm ( `) ( `) be pairwise disjoint compact subsets of ( X2 , d2 ) , in which m ( `) is a positive integer. Also for each ` 2 f1, . . . , pg, let a1 ( `) , . . . , am ( `) ( `) be distinct real numbers and, with r` D miniD6 j |ai (`) ¡ aj (`) |, let Q`: [j (aj ( `) ¡ 0.25r `, aj (`) C 0.25r `) ! R be speciŽed by Q (a ) ` D aj ( `) for a 2 ( aj ( `) ¡ 0.25r `, aj ( `) C 0.25r`) and each j. Then there are a positive integer q, real numbers a1 , . . . , aq , numbers b1 , . . . , bq belonging to P, elements a1 , . . . , aq of [1, p], and elements u1 , . . . , uq of U such that for each ` 2 f1, . . . , pg, Theorem 7.

Á Q`

q X k D1

! ak expf¡b k | ` ¡ ak | g expf¡b kkx 2

¡ u kk22 g

D aj ( `)

for x 2 Cj ( `) with j D 1, . . . , m ( `) . The proof is similar to the proof of theorem 4, so we give only an outline of it. ( ) For each ` 2 f1, . . . , pg, let g`: [jmD 1` Cj (`) ! R be deŽned by g` (x ) D Proof.

( )

m` aj ( `) for x 2 Cj ( `) and each j. Set D D [` [jD 1 Cj (`) . Observe that each [jmD 1 Cj (`) is closed in the metric space ( D, d2 ) , which is compact, and that each g` is continuous (note that the distance between pairs of the Cj ( `) is positive). Using the Lebesgue-Urysohn extension theorem, let f` be a continuous extension of g` to ( D, d2 ) for each `, and with A D [1, p], construct a continuous function f from ( A, | ¢ |) £ ( D, d2 ) to R such that f ( l, x) D f` ( x) for each `. The remainder of the proof makes use of theorem 6 instead of theorem 3 and uses the compactness of D.

12 Again, in accord with the material in section 2.2, the metric on (A, | ¢ |) £ (X2 , d2 ) is the sum of the two metrics.

Families of Functionals

467

For examples of spaces ( X2 , d2 ) of interest, see the examples concerning hypothesis A.1 in section 2.2. Appendix: On Continuity on Product Spaces

Here we record some observations concerning conditions for the continuity of a function deŽned on a product space. Let ( Y1 , d1 ) , ( Y2 , d2 ), and ( Y3 , d3 ) be three metric spaces, and with d deŽned on the product set Y :D (Y1 £ Y2 ) by d[ ( a1 , b1 ) , (a2 , b2 ) ] D d1 (a1 , a2 ) C d2 ( b1 , b2 ) (here d1 , d2 , and d are not necessarily the metrics of section 2.2), let f be a map from the metric space ( Y, d) into ( Y3 , d3 ) . Suppose that for each a 2 Y1 , the map f ( a, ¢) : (Y2 , d2 ) ! ( Y3 , d3 ) is continuous. Suppose also that f is continuous on (Y1 , d1 ) locally uniformly with respect to ( Y2 , d2 ) , in the sense that given ( a 0, b0 ) 2 Y1 £ Y2 and 2 > 0, there are positive constants d1 and d2 such that d3 [ f ( a, b) , f ( a0 , b )] < 2 for d1 ( a, a0 ) < d1 and d2 (b, b0 ) < d2 . Under these conditions, f is continuous on ( Y, d) . To see this, let ( a 0, b0 ) 2 Y1 £ Y2 and 2 1 > 0 be given. Choose d > 0 so that d3 [ f ( a 0, b) , f ( a0 , b0 )] < 2 1 / 2 for d2 ( b, b0 ) < d, and so that d3 [ f ( a, b ) , f ( a 0, b) ] < 2 1 / 2 for d1 ( a, a 0 ) < d and d2 ( b, b0 ) < d. Then observe that for d[ ( a, b) , ( a 0, b0 ) ] < d, we have d3 [ f ( a, b) , f ( a0 , b 0 ) ] · d3 [ f ( a, b) , f ( a0 , b )] C d3 [ f ( a0 , b ) , f (a 0, b0 ) ] < 2 1 , showing that f is continuous. (A simple example of an f that meets the conditions given here is f deŽned by equation 1.3.) We note also that if f is continuous on ( Y, d) and ( Y, d ) is compact, then f is uniformly continuous on (Y, d ) and therefore is both continuous separately in its second variable and continuous on ( Y1 , d1 ) locally uniformly with respect to (Y2 , d2 ) . References Hartman, E. J., Keeler, J. D., & Kowalski, J. M. (1990). Layered neural networks with gaussian hidden units as universal approximators. Neural Computation, 2, 210–215. Lainiotis, D. (1976). Partitioning: A uniform framework for adaptive systems (parts I and II). Proceedings of the IEEE, 64, 1126–1197. Lo, J. T. (1972). Finite dimensional sensor orbits and optimal nonlinear Žltering. IEEE Transactions on Information Theory, IT-18-15, 583–588. Lo, J. T., & Bassu, B. (2002). Adaptive multilayer perceptrons with long- and short-term memories. IEEE Transactions on Neural Networks, 13, 22–33. Mukherjea, A., & Pothoven, K. (1978). Real and functional analysis. New York: Plenum Press.

468

I. Sandberg

Park, J., & Sandberg, I. W. (1991). Universal approximation using radial basis function networks. Neural Computation, 3, 246–257. Park, J., & Sandberg, I. W. (1993). Approximation and radial-basis function networks. Neural Computation, 5, 305–316. Parzen, E. (1962). On estimation of a probability density function and mode. Annals of Mathematical Statistics, 33, 1065–1076. Sandberg, I. W. (1991). Structure theorems for nonlinear systems. Multidimensional Systems and Signal Processing, 2, 267–286. (Errata in vol. 3, no. 1, p. 101, 1992.) Sandberg, I. W. (1994). General structures for classiŽcation. IEEE Transactions on Circuits and Systems-I, 41, 372–376. Sandberg, I. W. (1998).Notes on uniform approximation of time-varying systems on Žnite time intervals. IEEE Transactions on Circuits and Systems-I, 45, 863– 865. Sandberg, I. W. (2001). Gaussian radial basis functions and inner-product spaces. Circuits, Systems, and Signal Processing, 20, 635–642. (Errata in vol. 21, no. 1, p. 123, 2002.) Sandberg, I. W., Lo, J. T., Francourt, C., Principe, J., Katagiri, S., & Haykin, S. (2001). Nonlinear dynamical systems: Feedforward neural network perspectives. New York: Wiley. Stone, M. H. (1962). A generalized Weierstrass approximation theorem. In R. C. Buck (Ed.), Studies in modern analysis. Englewood Cliffs, NJ: Prentice Hall. Sutherland, W. A. (1975). Introduction to metric and topological spaces. Oxford: Clarendon Press.

Received April 17, 2002; accepted July 17, 2002.

LETTER

Communicated by John Platt

EfŽcient Greedy Learning of Gaussian Mixture Models J. J. Verbeek [email protected] N. Vlassis [email protected] B. Krose ¨ [email protected] Informatics Institute, University of Amsterdam, 1098 SJ Amsterdam, The Netherlands This article concerns the greedy learning of gaussian mixtures. In the greedy approach, mixture components are inserted into the mixture one after the other. We propose a heuristic for searching for the optimal component to insert. In a randomized manner, a set of candidate new components is generated. For each of these candidates, we Žnd the locally optimal new component and insert it into the existing mixture. The resulting algorithm resolves the sensitivity to initialization of state-of-the-art methods, like expectation maximization, and has running time linear in the number of data points and quadratic in the (Žnal) number of mixture components. Due to its greedy nature, the algorithm can be particularly useful when the optimal number of mixture components is un known. Experimental results comparing the proposed algorithm to other methods on density estimation and texture segmentation are provided. 1 Introduction

This article concerns the learning (Žtting the parameters) of a mixture of gaussian distributions (McLachlan & Peel, 2000). Mixture models form an expressive class of models for density estimation. Applications in a wide range of Želds have emerged, used for density estimation in unsupervised problems, clustering purposes, estimating class-conditional densities in supervised learning settings, situations where the data are partially supervised, and situations where some observations have missing values. The most popular algorithm to learn mixture models is the expectationmaximization (EM) algorithm (Dempster, Laird, & Rubin, 1977). For a given Žnite data set X n of n observations and an initial mixture f 0 , the algorithm provides a means to generate a sequence of mixture models f f i g with nondecreasing log likelihood on X n . The EM algorithm is known to converge to a locally optimal solution. However, convergence to a globally optimal solution is not guaranteed. The log likelihood of the given data set under the found mixture distribution is highly dependent on the initial mixture f 0. Neural Computation 15, 469–485 (2003)

° c 2002 Massachusetts Institute of Technology

470

J. J. Verbeek, N. Vlassis, and B. Krose ¨

The standard procedure to overcome the high dependence on initialization is to start the EM algorithm for several random initializations and use the mixture yielding maximum likelihood on the data. In this article, we present a method to learn Žnite gaussian mixtures, based on the approximation result of Li and Barron (2000), which states that we can quickly approximate any target density with a Žnite mixture model, as compared to the best we can do with any (possibly nonŽnite) mixture from the same component class. This result also holds if we learn the mixture models in a greedy manner: start with one component and optimally add new components one after the other. However, locating the optimal new component boils down to Žnding the global maximum of a log-likelihood surface. Li (1999) proposed to grid the parameter space to locate the global maximum, but this is infeasible when learning mixtures in high-dimensional spaces. Here, we propose a search heuristic to locate the global maximum. Our search procedure selects the best member from a set of candidate new components that is dynamic in the sense that it depends on the mixture learned so far. Our search procedure for a new component has running time O (n ) . For learning a k component mixture, this results in a run time of O ( nk2 ) if the complete mixture is updated with EM after each insertion. If the mixture is not updated between the component insertions (the approximation result still holds in this case), the run time would be O ( nk). The article is organized as follows. In section 2, we recapitulate the definition and EM learning of gaussian mixtures. Section 3 discusses greedy learning of gaussian mixtures. Our component insertion procedure is discussed in section 4, the core of the article. In section 5, we present experimental results on two tasks: modeling of artiŽcially generated data drawn from several types of gaussian mixtures and texture segmentation. The experiments compare the performance of the new algorithm with the performance of several other methods to learn gaussian mixtures. Section 6 presents conclusions and a discussion. 2 Gaussian Mixtures and the EM Algorithm

A gaussian mixture (GM) is deŽned as a convex combination of gaussian densities. A gaussian density in a d-dimensional space, characterized by its mean m 2 Rd and d £ d covariance matrix C , is deŽned as w (x I h ) D (2p ) ¡d/ 2 det( C ) ¡1/ 2 exp (¡( x ¡ m ) > C ¡1 ( x ¡ m ) / 2),

(2.1)

where h denotes the parameters m and C . A k component GM is then deŽned as fk ( x) D and for

k X jD 1

p j w ( xI h j ) ,

with

j 2 f1, . . . , kg: p j ¸ 0.

k X jD 1

pj D 1 (2.2)

EfŽcient Greedy Learning of Gaussian Mixture Models

471

The p j are called the mixing weights and w (x I h j ) the components of the mixture. As compared to nonparametric density estimation, mixture density estimation (or semi-parametric density estimation) allows for efŽcient evaluation of the density, since only relatively few density functions have to be evaluated. Furthermore, by using few mixture components, a smoothness constraint on the resulting density is enforced, allowing for robust density estimates. The EM algorithm (Dempster et al., 1977) enables us to update the parameters of a given k-component mixture with respect to a data set X n D fx1 , . . . , x n g with all x i 2 Rd , such that the likelihood of X n is never smaller under the new mixture. The updates for a GM can be accomplished by iterative application of the following equations for all components j 2 f1, . . . , kg: P ( j | x i ) :D p j w ( x i I hj ) / fk (x i ) , p j :D m j :D C j :D

n X

(2.3)

P ( j | xi ) / n,

(2.4)

P ( j | xi ) x i / (np j ) ,

(2.5)

P ( j | xi ) ( x i ¡ mj ) (x i ¡ mj ) > / (np j ) .

(2.6)

iD 1 n X iD 1 n X iD 1

The EM algorithm is not guaranteed to lead us to the best solution, where “best” means the solution yielding maximal likelihood on X n among all maxima of the likelihood. (We implicitly assume that the likelihood is bounded by restricting the parameter space, and hence the maximum likelihood estimator is known to exist; (Lindsay, 1983.) The good thing is that if we are close to the global optimum of the parameter space, then it is very likely that by using EM, we obtain the globally optimal solution. 3 Greedy Learning of Gaussian Mixtures

In this section, we discuss the greedy approach to learning GMs. The basic idea is simple. Instead of starting with a (random) conŽguration of all components and improving on this conŽguration with EM, we build the mixture component-wise. We start with the optimal one-component mixture, whose parameters are trivially computed. Then we start repeating two steps until a stopping criterion is met: (1) insert a new component, and (2) apply EM until convergence. The stopping criterion can implement the choice for a prespeciŽed number of components, or it can be any model complexity selection criterion (like minimum description length, Akaike information criterion, or cross validation).

472

J. J. Verbeek, N. Vlassis, and B. Krose ¨

3.1 Motivation. The intuitive motivation for the greedy approach is that the optimal component insertion problem involves a factor k fewer parameters than the original problem of Žnding a near-optimal start conŽguration. We therefore expect it to be an easier problem. Of course, the intuitive assumption is that we can Žnd the optimal (with respect to log likelihood) ( k C 1) -component mixture by local search if we start the search from the mixture obtained by inserting optimally a new component in the optimal k-component mixture. We recently applied a similar strategy to the k-means clustering problem (Likas, Vlassis, & Verbeek, in press) and obtained encouraging results. Two recent theoretical results provide complementary motivation. The Kullback-Leibler (KL) divergence between two densities f and g is deŽned as

Z D ( f k g) D

V

f ( x ) log [ f ( x ) / g ( x )] dx,

(3.1)

where V is the domain of the densities f and g (see Cover & Thomas, 1991, for details). Li and Barron (2000) showed that for an arbitrary probability density function f , there exists a sequence f fi g of Žnite mixtures such that Pk ( ) ( ) ( ) fk ( x ) D i D 1 p i w xIRhi achieves KL divergence D f k f k · D f k gP C c / k for every gP D w ( xI h ) P (dh ) . Hence, the difference in KL divergence achievable by k-component mixtures and the KL divergence achievable by any (possibly nonŽnite) mixture from the same family of components tends to zero with a rate of c / k, where c is a constant dependent not on k but only on the component family. Furthermore, Li and Barron (2000) showed that this bound is achievable by employing the greedy scheme discussed in section 3.2. This tells us that we can quickly approximate any density by the greedy procedure. Therefore, we might expect the results of the greedy procedure—as compared to methods that initialize all at once—to differ more when Žtting mixtures with many components. The sequence of mixtures generated by the greedy learning method can conveniently be used to guide a model selection process in the case of an unknown number of components. Recently, a result of “almost” concavity of the log likelihood of a data set under the maximum likelihood k-component mixture, as a function of the number of components k, was presented (Cadez & Smyth, 2000). The result states that the Žrst-order Taylor approximation of the log likelihood of the maximum likelihood models as a function of k is concave under very general conditions. Hence, if we use a penalized loglikelihood model selection criterion based on a penalty term that is concave or linear in k, then the penalized log likelihood is almost concave. This implies that if the “almost” concave turns out to be concave, then there is only one peak in the penalized log likelihood, and hence this peak can be relatively easily identiŽed with a greedy construction method.

EfŽcient Greedy Learning of Gaussian Mixture Models

473

3.2 A General Scheme for Greedy Learning of Mixtures. Let L ( X n , f k ) P n i D 1 log f k ( xi ) (we will just write L k if no confusion arises) denote the log likelihood of the data set X n under the k-component mixture fk. The greedy learning procedure above can be summarized as follows: D

1. Compute the optimal (yielding maximal log likelihood) one-component mixture f1 . Set k :D 1. 2. Find the optimal new component w ( x Ih ¤ ) and corresponding mixing weight a¤ , fh ¤ , a¤ g D arg max fh,ag

n X iD 1

log [(1 ¡ a) fk (x i ) C aw (x i I h ) ],

(3.2)

while keeping fk Žxed. 3. Set fkC 1 ( x ) :D (1 ¡ a¤ ) fk ( x ) C a¤ w ( x Ih ¤ ) and k :D k C 1. 4. Optional: Update fk using EM until convergence.

5. If a stopping criterion is met, then quit; else go to step 2. Clearly, the crucial step is the component insertion (step 2), which will be the topic of the next section. Let us make a few remarks before we turn to that section. First, the stopping criterion in step 5 can be used to force the algorithm to Žnd a mixture of a prespeciŽed number of components. Step 5 may also implement any kind of model selection criterion. Second, note that step 4 may also be implemented by algorithms other than EM. Furthermore, step 4 is not needed to obtain the approximation result of Li and Barron (2000). Third, note the similarity to the vertex direction method (VDM) (Bohning, ¨ 1995; Lindsay, 1983; Wu, 1978) to learn GMs. Indeed, VDM may be regarded as a speciŽc choice for implementing the search needed for step 2, as we explain in the discussion in Section 6. 4 EfŽcient Search for New Components

Suppose we have obtained a k-component mixture fk , and the quest is to Žnd the component characterized by equation 3.2. It is easily shown that if we Žx fk and h , then L kC 1 is concave as a function of a only, allowing efŽcient optimization. For example, Vlassis and Likas (2002) used a secondorder Taylor approximation. However, L kC 1 as a function of h can have multiple maxima. Hence, we have to perform a global search among the new components in order to identify the optimum since no constructive method to Žnd the global maximum is known. In our new algorithm, the global search for the optimal new component is achieved by starting km partial EM searches from initial parameter pairs f(hi , ai ) g, 1 · i · km. By partial EM search, we mean that we Žx fk and optimize L kC 1 over h and a only. The use of partial EM searches is dictated

474

J. J. Verbeek, N. Vlassis, and B. Krose ¨

by the general scheme above, for which the results of Li and Barron (2000) hold. One may wonder why we would use such partial searches, since we might as well optimize over all parameters of the resulting ( k C 1) -component mixture. The answer is that if we updated all parameters, then the number of computations needed in each of the km individual searches would be O ( nk) instead of O ( n ). Each partial search starts with a different initial conŽguration. After these multiple partial searches, we end up with km candidate new components together with their mixing weights. We pick the candidate component w ( xI hO ) that maximizes the likelihood when mixed into the previous mixture by a factor a, O as in equation 3.2. Then, in step 3 of the general algorithm, in place O a). of the (unknown) optimal parameter pair (h ¤ , a¤ ), we use (h, O Note that a simple uniform quantization of the parameter space, as implemented for VDM for mixtures of univariate gaussian distributions in DerSimonian (1986), is not feasible in general since the number of quantization levels grows exponentially with the dimension of the parameter space. In the following section, we discuss step 2 as proposed in Vlassis and Likas (2002) and its drawbacks. In section 4.2, we present our improved search procedure. 4.1 Every Data Point Generates a Candidate. Vlassis and Likas (2002) proposed using n candidate components. (Henceforth, we refer to this method as VL.) Every data point is the mean of a corresponding candidate. All candidates have the same covariance matrix s 2 I, where s is taken at a theoretically justiŽable value. For each candidate component, the mixing weight a is set to the mixing weight maximizing the second-order Taylor approximation of L kC 1 around a D 1 / 2. The candidate yielding the highest log likelihood when inserted in the existing mixture fk is selected. The selected component is then updated using partial EM steps before it is actually inserted into fk to give fkC 1 . Note that for every component insertion that is performed, all point candidates are considered. There are two main drawbacks to this method. First, using n candidates in each step results in a time complexity O ( n2 ) for the search, which is unacceptable in most applications. O ( n2 ) computations are needed since the likelihood of every data point under every candidate component has to be evaluated. Second, by using Žxed small (referring to the size of the determinant of the covariance matrix) candidate components, the method keeps inserting small components in high-density areas, while in low-density areas, the density is modeled poorly. We observed this experimentally. Larger components that give greater improvement of the mixture are not among the candidates, nor are they found by the EM procedure following the component insertion. Working on improvements of the algorithm in Vlassis and Likas (2002), we noticed that both better and faster performance could be achieved by using a smaller set of candidates tailored to the existing mixture. In the

EfŽcient Greedy Learning of Gaussian Mixture Models

475

next section, we discuss how we can generate the candidates in a data- and mixture-dependent manner. 4.2 Generating Few Good Candidates. Based on our experience with the above method, we propose a new search heuristic for Žnding the optimal new mixture component. Two observations motivate the new search method:

1. The size (i.e., the determinant of the covariance matrix) of the new component should in general be smaller than the size of the existing components of the mixture. 2. As the existing mixture fk contains more components, the search for the optimal new component should become more thorough. In our search strategy, we account for both observations by (1) setting the size of the initial candidates to a value related to (and in general smaller than) the size of the components in the existing mixture and (2) increasing the number of candidates linearly with k. For each insertion problem, our method constructs a Žxed number of candidates per existing mixture component. Based on the posterior distributions, we partition the data set X n in k disjoint subsets Ai D fx 2 X n : i D arg maxj fP( j | x ) gg. If one is using EM for step 4 (of the general scheme given above), then the posteriors P (i | x ) are available directly since we have already computed them for the EM updates of the k-component mixture. For each set Ai (i D 1, . . . , k), we construct m candidate components. In our experiments we used m D 10 candidates, but more can be used, trading run time for better performance. In fact, as in Smola and Sch¨olkopf (2000), we can compute conŽdence bounds of the type: “With probability at least 1 ¡d the best split among m uniformly randomly selected splits is among the best 2 fraction of all splits, if m ¸ dlogd / log (1 ¡ 2 )e.” To generate new candidates from Ai , we pick uniformly randomly two data points x l and x r in Ai . Then we partition Ai into two disjoint subsets Ail and Air . For elements of Ail , the point x l is closer than x r and vice versa for Air . The mean and covariance of the sets Ail and Air are used as parameters for two candidate components. The initial mixing weights for candidates generated from Ai are set to p i / 2. To obtain the next two candidates, we draw new x l and x r , until the desired number of candidates is reached. The initial candidates can be replaced easily by candidates that yield higher likelihood when mixed into the existing mixture. To obtain these better candidates, we perform the partial EM searches, starting from the initial candidates. Each iteration of the km partial updates takes O ( mnk) computations, since we have to evaluate the likelihood of each datum under each of the mk candidate components. In the following section, we discuss how we can reduce the amount of computations needed by a factor k, resulting in only O ( mn) computations to perform one iteration of the partial updates.

476

J. J. Verbeek, N. Vlassis, and B. Krose ¨

Figure 1: The construction of a four-component mixture distribution.

We stop the partial updates if the change in log likelihood of the resulting ( k C 1) -component mixtures drops below some threshold or if some maximal number of iterations is reached. After these partial updates, we set the new component w ( x I hkC 1 ) as the candidate that maximizes the log likelihood when mixed into the existing mixture with its corresponding mixing weight. An example is given in Figure 1, which depicts the evolution of a solution for artiŽcially generated data. The mixtures f1 , . . . , f4 are depicted; each component is shown as an ellipse that has the eigenvectors of the covariance matrix as axes and radii of twice the square root of the corresponding eigenvalue. 4.3 Speeding up the Partial EM Searches. In order to achieve O ( mn) time complexity for the partial EM searches initiated at the O ( km ) initial candidates, we use a lower bound on the true log likelihood instead of the true log likelihood itself. The lower bound is obtained by assuming that the support (nonzero density points in X n ) for a candidate component w ( x , m kC 1 , C kC 1 ) originating from existing component i is fully contained in Ai . The lower bound, termed negative free energy (Neal & Hinton, 1998), is detailed in the appendix. Optimizing the lower bound allows us to base the

EfŽcient Greedy Learning of Gaussian Mixture Models

477

partial updates for a candidate from Ai purely on the data in Ai . The update equations are: aw ( xj I m kC 1 , C kC 1 ) (1 ¡ a) fk ( xj ) C aw ( xj I m kC 1 , C kC 1 ) P j2Ai p ( k C 1 | xj ) xj D P ( ) j2Ai p k C 1 | xj P ( )( )( )> j2Ai p k C 1 | xj xj ¡ m k C 1 xj ¡ m k C 1 P D j2Ai p ( k C 1 | xj ) P j2Ai p ( k C 1 | xj )

p ( k C 1 | xj ) D m kC 1 C kC 1

aD

N

(4.1) (4.2)

(4.3)

(4.4)

4.4 Time Complexity Analysis. The total run time of the algorithm is O ( k2 n ) (or O ( kmn) if k < m). This is due to the updates of the mixtures fi , which cost O ( ni ) computations each if we use EM for these updates. P Therefore, summing over all mixtures, we get O ( ikD 1 ni ) 2 O (nk2 ) . This is a factor k slower than the standard EM procedure. The run times of our experiments conŽrm this analysis; the new method performed on average about k / 2 times slower than standard EM. However, if we use EM to learn a sequence of mixtures consisting of 1, . . . , k components, then the run time would also be O ( nk2 ) . If we do not use the speed-up, the run time would become O ( k2 mn) since in that case, the component allocation step for the i C 1st mixture component would cost O ( imn) . This is a factor m more than the preceding EM steps; hence, the total run time goes up by a factor m. 5 Experimental Results

In this section, we present results of two experiments that compare the results obtained with k-means initialized EM, the method of VL discussed in section 4.1, split-and-merge EM (SMEM; Ueda, Nakano, Ghahramani, & Hinton, 2000), and the new greedy approach presented here. The SMEM algorithm checks after convergence of EM if the log likelihood can be improved by splitting one mixture component in two and merging two other mixture components. To initialize SMEM, we also used the k-means initialization, which makes use of the results of applying the k-means or generalized Lloyd algorithm to the data (Gersho & Gray, 1992). The means are initialized as the centroids of the Voronoi regions (VR) found and the covariance matrices as the covariance matrix corresponding to the data in the VRs. The mixing weights were initialized as the proportion of the data present in each VR. The centroids of the k-means algorithm were initialized as a uniformly random selected subset of the data.

478

J. J. Verbeek, N. Vlassis, and B. Krose ¨

5.1 ArtiŽcial Data. In this experiment, we generated data sets of 400 points in Rd with d 2 f2, 5g. The data were drawn from a gaussian mixture of k 2 f4, 6, 8, 10g components. The separation of the components was chosen: c 2 f1, 2, 3, 4g. A separation of c means (Dasgupta, 1999)

8iD6 j : k¹i ¡ ¹j k2 ¸ c ¢ maxftrace( C i ) , trace( Cj ) g. fi, jg

(5.1)

For each mixture conŽguration, we generated 50 data sets. We allowed a maximum eccentricity (i.e., the largest singular value of the covariance matrix over the smallest) of 15 for each component. Also, for each generated mixture, we generated a test set of 200 points not presented to the mixture learning algorithm. In the comparison below, we evaluate the difference in log likelihood of the test sets under the mixtures provided by the different methods. In Table 1, we show experimental results comparing our greedy approach to VL, SMEM, and several runs of normal EM initialized with k-means. The average of the difference in log likelihood on the test set is given for different characteristics of the generating mixture. The results show that the greedy algorithm performs comparably to SMEM and generally outperforms VL. Both greedy methods and SMEM outperform the k-means initialization by far. 5.2 Texture Segmentation. In this experiment, the task is to cluster a set of 16 £ 16 D 256 pixel image patches. The patches are extracted from 256£256 Brodatz texture images; four of these textures are shown in Figure 2. Because different patches from the same texture roughly differ from each other only in a limited number of degrees of freedom (translation, rotation, scaling, and lightness), patches from the same texture are assumed to form clusters in the 256-dimensional patch space. We conducted experiments where the number of textures from which patches were extracted ranged from k D f2, . . . , 6g. For each value of k, we created 100 data sets of 500k patches each by picking up 500 patches at random from each of k randomly selected textures. The experiment consisted of learning a gaussian mixture model with as many components as different textures involved. We compared our greedy approach to the same methods as in the previous experiment. To speed up the experiment, we projected the data into a linear subspace with principal component analysis. We used a 50-dimensional subspace or a lower than 50-dimensional space if that could capture at least 80% of the data variance. Once the mixture model was learned, we segmented the patches by assigning each patch to the mixture component with the highest posterior probability for this patch. To evaluate the quality of the different segmentations discovered by the different learning methods, we considered how informative the found segmentations are. We measure how predictable the texture label of a patch is given the cluster of the

EfŽcient Greedy Learning of Gaussian Mixture Models

479

Table 1: Detailed Exposition of Log-Likelihood Differences. Vlassis and Likas

dD2

cD1

2

3

4

dD5

cD1

2

3

4

kD4 6 8 10

0.03 0.01 0.00 0.01

0.00 0.03 0.04 0.06

0.01 0.05 0.06 0.07

0.01 0.02 0.03 0.03

kD4 6 8 10

0.05 0.04 0.03 0.04

0.02 0.04 0.07 0.05

¡0.01 0.13 0.13 0.11

0.04 0.13 0.04 0.16

Best of k runs of k-means initialization EM

dD2

cD1

2

3

4

dD5

cD1

2

3

4

kD4 6 8 10

0.03 0.03 0.02 0.06

0.14 0.16 0.23 0.27

0.13 0.34 0.38 0.45

0.43 0.52 0.55 0.62

kD4 6 8 10

0.02 0.09 0.12 0.13

0.28 0.35 0.45 0.49

0.36 0.48 0.75 0.75

0.37 0.54 0.82 0.83

dD2

cD1

2

3

4

dD5

cD1

2

3

4

kD4 6 8 10

¡0.01 ¡0.02 ¡0.02 ¡0.00

0.00 0.00 0.00 0.00

0.00 0.02 0.00 0.03

0.00 0.02 0.03 0.04

kD4 6 8 10

¡0.02 ¡0.06 ¡0.05 ¡0.01

0.01 ¡0.01 ¡0.04 0.04

0.00 ¡0.01 0.00 0.01

0.00 0.00 0.02 0.05

SMEM

Distance to generating mixture

dD2

cD1

2

3

4

dD5

cD1

2

3

4

kD4 6 8 10

0.04 0.07 0.10 0.13

0.03 0.06 0.07 0.12

0.03 0.05 0.07 0.10

0.02 0.04 0.09 0.12

kD4 6 8 10

0.16 0.28 0.45 0.58

0.13 0.22 0.33 0.50

0.14 0.19 0.32 0.45

0.11 0.18 0.42 0.51

Notes: The upper three tables give the log likelihood of a method subtracted from the log likelihood obtained using our greedy approach. The bottom table gives averages of the log likelihood under the generating mixture minus the log likelihood given by our method.

class. This can be done using the conditional entropy (Cover & Thomas, 1991) of the texture label given the cluster label. In Table 2, the average conditional entropies (over 50 experiments) are given. The results of the greedy method and SMEM are roughly comparable, although the greedy approach was much faster. Both provide generally more informative segmentations than using the k-means initialization. Note that VL fails to produce informative clusters in this experiment, probably because the high dimensionality of the data renders spherical candidate components rather unŽt.

480

J. J. Verbeek, N. Vlassis, and B. Krose ¨

Figure 2: Several Brodatz textures. The white squares indicate the patch size.

Table 2: Average Conditional Entropy for Different Values of k. k Greedy k-means SMEM VL Uniform

2

3

4

5

6

0.20

0.34 0.46

0.48 0.74

0.53

0.61

0.22

0.36

1.52 1.59

2.00 2

0.80 0.56 2.29 2.32

0.94 0.68 2.58 2.58

0.19

0.21 0.93 1

Notes: The bottom line gives the conditional entropy if the clusters are totally uninformative. Boldface is used to identify the method with minimum conditional entropy for each k.

EfŽcient Greedy Learning of Gaussian Mixture Models

481

6 Discussion and Conclusion 6.1 Discussion. Both VDM (Bohning, ¨ 1995; Lindsay, 1983; Wu, 1978) and the proposed method are instantiations of the more general greedy scheme given in section 3.2 (although VDM skips step 4 of the scheme). VDM makes use of the directional derivative D fk (w ) of the log likelihood for the current mixture fk, where

D fk (w ) D lim [L ( X n , (1 ¡ a) fk C aw ) ¡ L (X n , fk ) ] / a. a!0

VDM proceeds by picking the w ¤ that maximizes D fk (w ) and inserting this in the mixture with a factor a¤ such that a¤ D arg maxa fL ( X n , (1 ¡a) fk C aw ¤ g. Using the directional derivative at the current mixture can be seen as an instantiation of step 2 of the general scheme. The optimization over both w and a is replaced by an optimization over D fk (w ) (typically implemented by gridding the parameter space) and then an optimization over a. Note that by moving in the direction of maximum, D fk (w ) does not guarantee that we move in the direction of maximum improvement of log likelihood if we optimize over a subsequently. Recently, several novel methods to learn mixtures (of gaussian densities) were proposed (Dasgupta, 1999; Figueiredo & Jain, 2002; Ueda et al., 2000). The Žrst tries to overcome the difŽculties of learning mixtures of gaussian densities in high-dimensional spaces. By projecting the data to a lowerdimensional subspace (which is chosen uniformly randomly), Žnding the means in that space, and then projecting them back, the problems of high dimensionality are reduced. The last two methods try to overcome the dependence of EM on the initial conŽguration as does our method. In Ueda et al. (2000), split-and-merge operations are applied to local optima solutions found by applying EM. These operations constitute jumps in the parameter space that allow the algorithm to jump from a local optimum to a better region in the parameter space. By then reapplying EM, a better (hopefully global) optimum is found. The latter method and the greedy approach are complementary; they can be combined. The split-and-merge operations can be incorporated in the greedy algorithm by checking after each component insertion whether a component i can be removed such that the resulting log likelihood is greater than the likelihood before insertion. If so, the component i is removed and the algorithm continues. If not, the algorithm proceeds as usual. However, in experimental studies on artiŽcially generated data, we found that this combination gave little improvement over the individual methods. An important beneŽt of our new method over Ueda et al. (2000) is that the new algorithm produces a sequence of mixtures that can be used to perform model complexity selection as the mixtures are learned. For example, a kurtosis-based selection criterion, like the one in Vlassis and Likas (1999), can be used here. We also note that our algorithm executes faster

482

J. J. Verbeek, N. Vlassis, and B. Krose ¨

than SMEM, which has a run time of O ( nk3 ) . Figueiredo and Jain (2002) proposed starting with a large number of mixture components and successively annihilating components with small mixing weights. This approach can be characterized as pruning a given mixture, whereas our approach can be characterized as growing a mixture. As compared to both these methods, ours does not need an initialization scheme, since the optimal one-component mixture can be found analytically. Also, Bayesian methods have been used to learn gaussian mixtures. For example, in Richardson and Green (1997), a reversible-jump Markov chain Monte Carlo (MCMC) method is proposed. There, the MCMC allows jumps between parameter spaces of different dimensionality (i.e., parameter spaces for mixtures consisting of differing number of components). However, the experimental results that Richardson and Green (1997) reported indicate that such sampling methods are rather slow compared to constructive maximum likelihood algorithms. It is reported that about 160 sweeps per second are performed on a SUN Sparc 4 workstation. The experiments involve 200,000 sweeps, resulting in about 20 minutes of run time. Although the 200,000 sweeps are not needed for reliable results, this contrasts sharply with the 2.8 seconds and 5.7 seconds run time (allowing respectively about 480 and 960 sweeps) of the standard EM and our greedy EM in a similar experimental setting also executed on a SUN Sparc 4 workstation. 6.2 Conclusion. We proposed an efŽcient greedy method to learn mixtures of gaussian densities that have run time O ( k2 n C kmn) for n data points, a Žnal number of k mixture components, and m candidates per component. The experiments we conducted have shown that our greedy algorithm generally outperforms the k-means initialized EM. In experiments on artiŽcial data, we observed that SMEM performs in general comparable to the new method in terms of quality of the solutions found. The results on natural data were comparable to those obtained with SMEM if SMEM was initialized with the k-means algorithm. An important beneŽt of the greedy method, compared to both SMEM and normal EM, is the production of a sequence of mixtures, which obviates the need for initialization and facilitates model selection. As compared to VL, the O ( n2 k2 ) time complexity has been reduced by a factor n; the somewhat arbitrary choice of spherical candidate components with Žxed variance has been replaced by a search for candidate components that depends on the data and the current mixture; and experiments suggest that if the methods yield different performance, then the new method generally outperforms the old one. Appendix: A Lower Bound for Fast Partial EM

In the insertion step, we have a two-component mixture problem. The Žrst component is the mixture fk, which we keep Žxed in the partial EM steps.

EfŽcient Greedy Learning of Gaussian Mixture Models

483

The second component is the new component w , which has a mixing weight a. The initial k C 1 component mixture is given by fk C 1 D aw C (1 ¡ a) fk . Using the maximization-maximization view on EM (Neal & Hinton, 1998), we can deŽne a lower bound F on the actual log likelihood: X X log fkC 1 ( x i ) ¸ F ( Q, h ) D log fkC 1 ( x i ) ¡ DKL ( Qi k p (¢ | x i ) ) , (A.1) i

i

where DKL denotes Kullback-Leibler divergence and p (¢ | x i ) denotes the posterior probability on the mixture of components. The distributions Qi are the responsibilities used in EM. We write qi for the responsibility of the new component and (1 ¡qi ) for the responsibility of fk for data point x i . The above lower bound can be made to match the log likelihood by setting the responsibilities equal to the posteriors p ( s | x i ) . However, in our case, when considering a candidate based on component 6 Aj : qi D 0. It is easy j, we Žx the Qi for all points outside Aj such that 8xi 2 to see from equation A.1 that the Qi for which x i 2 Aj should match the posterior probability on the new component in order to maximize F. Note that the more the assumption that data points outside Aj have zero posterior on the new component is true, the tighter the lower bound becomes. If the data outside Aj indeed have zero posterior on the new component, the lower bound equals the log likelihood after the partial E-step. For the M-step, we use the alternative decomposition of F, X EQi log p ( x i , s) C H ( Qi ) , (A.2) F (Q, h ) D i

where we used the variable s to range over the two mixture components, fk and w . Note that the entropies H ( Qi ) remain Žxed in the M-step since they do not depend on h. The Žrst term can be written as: X (A.3) qi [log w ( x i ) C log a] C (1 ¡ qi ) [log fk ( xi ) C log (1 ¡ a) ] i

D

X i2Aj

qi [log w ( xi ) C log a] C

X i

(1 ¡ qi ) [log fk ( x i ) C log (1 ¡ a) ]. (A.4)

Note that in equation A.4, the new mixture component w occurs only in terms for data in Aj . Setting the derivative of equation A.5 to zero with regard to the parameters of the new component (mixing weight a, mean m and covariance matrix C ) gives the update equations in section 4.3. Note that from equation A.4, F can also be evaluated using only the points in Aj if we stored the complete data log likelihood under fk in advance. Acknowledgments

We thank Aristidis Likas for useful discussions and a referee for pointing out the conŽdence bounds used in Smola and Scholkopf ¨ (2000). This research

484

J. J. Verbeek, N. Vlassis, and B. Krose ¨

is supported by the Technology Foundation STW (project AIF4997) applied science division of NWO, and the technology program of the Dutch Ministry of Economic Affairs. Software implementing the algorithm in MATLAB is available from J. J. V. by e-mail.

References Bohning, ¨ D. (1995). A review of reliable maximum likelihood algorithms for semiparametric mixture models. J. Statist. Plann. Inference, 47, 5–28. Cadez, I. V., & Smyth, P. (2000). On model selection and concavity for Žnite mixture models. In Proc. of Int. Symp. on Information Theory (ISIT). Available on-line at: http://www.ics.uci.edu / »icadez / publications.html. Cover, T., & Thomas, J. (1991). Elements of information theory. New York: Wiley. Dasgupta, S. (1999). Learning mixtures of gaussians. In Proc. IEEE Symp. on Foundations of Computer Science. New York. Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, 39(1), 1–38. DerSimonian, R. (1986). Maximum likelihood estimation of a mixing distribution. J. Roy. Statist. Soc. C, 35, 302–309. Figueiredo, M. A. T., & Jain, A. K. (2002). Unsupervised learning of Žnite mixture models. IEEE Transactions on Pattern Analysis and Machine Intelligence,24, 381– 396. Gersho, A., & Gray, R. M. (1992). Vector quantization and signal compression. Norwell, MA: Kluwer. Li, J. Q. (1999). Estimation of mixture models. Unpublished doctoral dissertation, Yale University. Li, J. Q., & Barron, A. R. (2000). Mixture density estimation. In S. A. Solla, T. K. Leen, & K.-R. Muller ¨ (Eds.), Advances in neural information processing systems, 12. Cambridge, MA: MIT Press. Likas, A., Vlassis, N., & Verbeek, J. J. (in press). The global k-means clustering algorithm. Pattern Recognition. Lindsay, B. G. (1983). The geometry of mixture likelihoods: A general theory. Ann. Statist., 11(1), 86–94. McLachlan, G. J., & Peel, D. (2000). Finite mixture models. New York: Wiley. Neal, R. M., & Hinton, G. E. (1998). A view of the EM algorithm that justiŽes incremental, sparse, and other variants. In M. Jordan (Ed.), Learning in graphical models (pp. 355–368). Norwood, MA: Kluwer. Richardson, S., & Green, P. J. (1997). On Bayesian analysis of mixtures with an unknown number of components. J. Roy. Statist. Soc. B, 59(4), 731–792. Smola, A. J., & Sch¨olkopf, B. (2000). Sparse greedy matrix approximation for machine learning. In Proc. of Int. Conf. on Machine Learning (pp. 911–918). San Mateo, CA: Morgan Kaufmann. Ueda, N., Nakano, R., Ghahramani, Z., & Hinton, G. E. (2000). SMEM algorithm for mixture models. Neural Computation, 12, 2109–2128.

EfŽcient Greedy Learning of Gaussian Mixture Models

485

Vlassis, N., & Likas, A. (1999). A kurtosis-based dynamic approach to gaussian mixture modeling. IEEE Trans. Systems, Man, and Cybernetics, Part A, 29, 393–399. Vlassis, N., & Likas, A. (2002). A greedy EM algorithm for gaussian mixture learning. Neural Processing Letters, 15(1), 77–87. Wu, C. (1978). Some algorithmic aspects of the theory of optimal designs. Annals of Statistics, 6, 1286–1301. Received November 13, 2001; accepted June 27, 2002.

LETTER

Communicated by John Platt

SMO Algorithm for Least-Squares SVM Formulations S.S. Keerthi [email protected] Department of Mechanical Engineering, National University of Singapore, Singapore 117576 S.K. Shevade [email protected] Genome Institute of Singapore, National University of Singapore, Singapore 117528 This article extends the well-known SMO algorithm of support vector machines (SVMs) to least-squares SVM formulations that include LS-SVM classiŽcation, kernel ridge regression, and a particular form of regularized kernel Fisher discriminant. The algorithm is shown to be asymptotically convergent. It is also extremely easy to implement. Com putational experiments show that the algorithm is fast and scales efŽciently (quadratically) as a function of the number of examples. 1 Introduction

This article concerns least-squares formulations of support vector machines (SVMs). These formulations, which provide interesting and useful alternatives to SVMs, have been proposed for both regression and classiŽcation. Saunders, Gammerman, and Vovk (1998) proposed the kernel version of ridge regression and empirically demonstrated its usefulness. Suykens and Vandewalle (1999) gave an interesting and useful extension of the idea to classiŽcation and called their method least-squares SVM (LS-SVM). A careful empirical study by Van Gestel et al. (2000) has shown that LS-SVM is comparable to SVM in terms of generalization performance. Recently, Van Gestel et al. (2002) also established the equivalence of LS-SVM with a particular form of regularized kernel Fisher discriminant (KFD) method (Mika, Ratsch, ¨ & Muller, ¨ 2001). While SVMs are characterized by convex quadratic programming problems involving inequality constraints, kernel ridge regression and LS-SVMs are formulated using equality constraints only. Hence, their solution is obtained by solving a system of linear equations (of the order of the number of training examples) involving the dense kernel matrix. To tackle large-scale problems, Suykens, Lukas, Van Dooren, De Moor, and Vandewalle (1999) and Van Gestel et al. (2000) effectively employed the conjugate-gradient method. In this article, we propose a new iterative algorithm for these probNeural Computation 15, 487–507 (2003)

° c 2002 Massachusetts Institute of Technology

488

S.S. Keerthi and S.K. Shevade

lems. It is based on the solution of the Wolfe dual problem using ideas similar to those of the SMO (sequential minimal optimization) algorithm for SVMs (Platt, 1998; Keerthi, Shevade, Bhattacharyya, & Murthy, 2001). The algorithm is extremely easy to implement and scales efŽciently (quadratically) as a function of the number of examples. The scaling is much better than that of the CG algorithm. The article is organized as follows. In section 2, we formulate the problem and develop the Wolfe dual. Optimality conditions for the dual are derived in section 3. A useful condition for terminating numerical algorithms solving the least-squares formulations is also given. These conditions form the basis for the SMO algorithm developed in section 4, where we also prove that the algorithm is asymptotically convergent. In section 5, we discuss the conjugate gradient method of Suykens et al. (1999) and show how the termination criterion derived in section 3 can be easily applied to this method. Computational experiments describing the effectiveness of the algorithm are given in section 6. Section 7 briey discusses the solution of KFD. Section 8 extends the SMO algorithm to least-square formulations without the threshold parameter, followed by concluding remarks in section 9. The appendix gives a pseudocode for the SMO algorithm. 2 Problem Formulation

Throughout, we will use x to denote the input vector of the classiŽcationregression problem and z to denote the feature space vector related to x by the transformation, z D Q ( x) . As in all other kernel designs, we do not assume Q to be known; all computations will be done using only the kernel function, k ( x, xO ) D Q ( x ) ¢ Q ( xO ) ,

(2.1)

where “¢” denotes inner product in the z space. Let f(xi , yi )gm i D 1 denote the training set, where xi is the ith input pattern and yi is the corresponding target value. For classiŽcation problems (two classes), yi takes only two possible values (say, 1 if xi is in class 1 and ¡1 if xi is in class 2), while for regression, yi takes any real value. Let zi D Q ( xi ) . In kernel ridge regression1 and classiŽcation by LS-SVMs, the following optimization problem is solved: min E s.t.

D

1 2 C C 2 kwk 2

P

2 i ji

yi ¡ ( w ¢ zi ¡ b) D ji 8i.

(2.2)

1 Saunders et al. (1998) leave out b from their formulation. This actually leads to a simpler problem. We will discuss its solution in section 8.

SMO Algorithm for Least-Squares SVM Formulation

489

DeŽne Á

! w p I Cj

wQ D

Á zQ i D

zi

!

p1 ei C

8i,

(2.3)

where j is an m-dimensional vector containing the ji ’s and, for every i, ei is the m-dimensional vector in which the ith component is 1 and all other components are 0. DeŽne kQ ( xi , xj ) D zQ i ¢ zQj . By equations 2.1 and 2.3, 1 kQ ( xi , xj ) D k ( xi , xj ) C dij , C

(2.4)

where dij D 1 if i D j and 0 otherwise. It is easy to see that equation 2.2 transforms to min E D

1 kwk Q 2 s.t. yi ¡ ( wQ ¢ zQ i ¡ b ) D 0 8i. 2

(2.5)

The Lagrangian for this problem is LD

X 1 kwk Q 2C ai [yi ¡ ( wQ ¢ zQi ¡ b) ]. 2 i

(2.6)

The KKT optimality conditions are given by

rwQ L D wQ ¡ @L @b

D

X i

X i

ai zQ i D 0

(2.7)

ai D 0.

(2.8)

Note that because equation 2.5 has only equality constraints, the ai ’s are not constrained to be nonnegative. Using equation 2.7, wQ can be expressed as a function of the ai ’s: X ai zQ i . (2.9) wQ (a) D i

Let us now apply Wolfe duality theory to equation 2.5. The Wolfe dual corresponds to the maximization of L subject to equations 2.7 and 2.8, with Q b, and ai ’s as variables. Using equations 2.8 and 2.9, we can simplify the w, Wolfe dual as X 1 max f (a) D ¡ kwQ (a) k2 C ai yi 2 i

s.t.

X i

ai D 0.

(2.10)

490

S.S. Keerthi and S.K. Shevade

P P Note that kwk Q 2 D i j ai aj kQ ( xi , xj ) . Hence, equation 2.10 is a simple convex quadratic programming problem involving a single equality constraint and no nonnegativity constraints. Once the ai ’s are obtained by solving equation 2.10, the primal variables, w and ji ’s, can be determined using equations 2.9 and 2.3. The determination of b will be addressed in the next section. 3 Optimality Conditions for the Dual

To derive proper stopping conditions for algorithms that solve the dual and also determine the threshold parameter b, it is important to write down the optimality conditions for the dual. The Lagrangian for the problem in equation 2.10 is X X 1 ai yi C b ai . LN D ¡ kwQ (a) k2 C 2 i i

(3.1)

DeŽne Fi D ¡

@f @ai

X

Q (a) ¢ zQi ¡ yi D Dw

j

aj kQ ( xi , xj ) ¡ yi ,

(3.2)

where f is deŽned in equation 2.10. The KKT conditions for the dual problem are @LN

@ai

D (b ¡ Fi ) D 0 8 i.

(3.3)

DeŽne bup D max Fi

i up D arg max Fi

blow D min Fi

i low D arg min Fi .

i

i

i

and

i

(3.4)

Then optimality conditions will hold at a given a iff blow D bup.

(3.5)

In equations 3.2 and 3.4, note that Fi , bup, i up, blow , and i low are all functions of a. The functional dependencies have not been noted to avoid notational clutter. Using equations 3.3, 3.2, and 2.5, it is easy to see the close relationship between the threshold parameter b in the primal problem and the multiplier, b. In particular, at optimality, b and b are identical. Therefore, in the rest of the article, b and b will denote the same quantity.

SMO Algorithm for Least-Squares SVM Formulation

491

We will say that an index pair (i, j) deŽnes a violation at a if 6 Fj . Fi D

(3.6)

Thus, optimality conditions will hold at a iff there does not exist any index pair ( i, j) that deŽnes a violation. Suppose ( i, j) satisŽes equation 3.6 at some a. Then it is possible P to achieve an increase in f (while maintaining the equality constraint, ak D 0) by adjusting ai and aj only. To see this, let us deŽne the following: a Q i ( t) D ai ¡ tI

a Q j ( t) D aj C tI

and

6 i, j a Q k ( t) D ak 8k D

(3.7)

and w ( t) D f ( a(t) Q ).

(3.8)

Then it is easy to verify that w 0 ( t) D Fi ¡ Fj ,

(3.9)

6 where Fi and Fj are evaluated at a(t) 0 at Q . Since by equation 3.6, Fi ¡ Fj D t D 0, an increase in w is possible by choosing t suitably away from 0. Since in numerical solution, it takes too much computing time to compute the exact optimal solution, there is a need to deŽne approximate stopping conditions that guarantee a speciŽed closeness to the solution of the primal problem, equation 2.2. We consider a stopping condition based on duality gaps. This has been used effectively by Smola and Sch¨olkopf (1998) in SVM regression. P Suppose a is any vector that satisŽes i ai D 0, b is any real number, P w D i ai zi , and

ji D yi ¡ (w ¢ zi ¡ b ) D b C

ai ¡ Fi , C

(3.10)

where Fi is as deŽned by equation 3.2. Note that ( w, b, j ) satisŽes the constraints of the primal, equation 2.2, and a satisŽes the constraints of the dual, equation 2.10. Let E denote the primal cost corresponding to ( w, b, j ) , E? be the optimal primal cost of equation 2.2, f denote the dual objective function value given by a, and f ? be the optimal dual objective function value. By Wolfe duality (Fletcher, 1987), we have: E ¸ E ? D f ? ¸ f.

(3.11)

Although E ?( D f ?) is unknown, equation 3.11 allows us to obtain guaranteed bounds using duality gap (a computable quantity) deŽned by ¶ Xµ ± ai ² C 2 C ji . ai Fi ¡ (3.12) Dgap D E ¡ f D 2C 2 i

492

S.S. Keerthi and S.K. Shevade

Dgap is a nonnegative quantity that becomes zero iff optimality is reached. Thus, if far g is a sequence that converges to a?, the solution of equation 2.10, then the continuity of the Dgap function implies that fDgapr g goes to zero, where Dgapr is the value of Dgap for ar . Also, for any nontrivial problem, E? D f ? > 0. Thus, it is appropriate to choose 2 > 0 and use the stopping criterion, Dgap · 2 f.

(3.13)

More important, if equation 3.13 is satisŽed, then we can obtain a guaranteed bound for the primal problem using equation 3.11: E ¡ E? · Dgap · 2 f · 2 E.

(3.14)

Thus, if 2 D 10¡p and equation 3.13 is satisŽed, then the closeness of E to E? has more than p decimal digits of accuracy. These ideas are used in interior point methods of linear programming (Vanderbei, 1994) and have also been employed in SVMs (Smola & Sch¨olkopf, 1998). In our implementation of the SMO algorithm derived in the next section, we employ equation 3.13 as the stopping criterion. In section 5, we explain how to use equation 3.13 to efŽciently stop the conjugate gradient algorithm for LS-SVM proposed by Suykens et al. (1999). At a given a, we need a method for determining a good value of b for use in the above calculations as well as for use in the classiŽer function deŽned by o ( x) D w ¢ Q ( x) ¡ b D

X i

ai k ( x, xi ) ¡ b.

(3.15)

One possibility is to take bD

blow C bup , 2

(3.16)

where bup and blow are as deŽned in equation 3.4. An alternative method is to choose bPto minimize Dgap after Žxing w at w (a) . This is equivalent to minimizing i ji2 , which can be done in closed form to get bD

1 X± ai ² . Fi ¡ m i C

(3.17)

4 SMO Algorithm

In this section, we give the SMO algorithm for solving equation 2.10. A basic step consists of starting with a point a and optimizing only two variables,

SMO Algorithm for Least-Squares SVM Formulation

493

ai and aj , to form the new point anew . Consider equations 3.7 and 3.8. Given equation 3.9, the natural choice is i D i up and j D i low so as to make |w 0 (0) | as large as possible. Using the notations of section 3, we can write the optimization problem and the resulting solution as t? D arg max w ( t) t

and

anew D a(t Q ?) .

(4.1)

The function w ( t) is a simple quadratic function of t, w ( t) D w (0) C tw 0 (0) C

t2 00 w (0) , 2

(4.2)

where w 0 is given by equation 3.9, and w 00 (0) D g D 2 kQ ( xi , xj ) ¡ kQ ( xi , xi ) ¡ kQ (xj , xj ) .

(4.3)

Thus, equation 4.1 can be easily solved in closed form: t? D ¡

Fi ¡ Fj w 0 (0) . D¡ 00 w (0) g

(4.4)

The SMO algorithm can now be described. It is very much along the lines of the simpliŽed version of the SMO algorithm for SVMs given by Chang and Lin (2001). SMO Algorithm for Equation 2.10

1. Choose any a0 , and set r D 0.

2. If ar satisŽes equation 3.5, stop. If not, set a D ar , choose i D i up, j D i low, and solve equation 4.1. 3. Let ar C 1 D anew , r :D r C 1 and go back to step 2.

We now establish asymptotic convergence of the algorithm. The absence of hard boundaries in the optimization makes the proof of convergence much simpler than corresponding proofs for SMO algorithm for SVMs. Putting equation 4.4 in equation 4.2 and simplifying, we get f (ar C 1 ) ¡ f (ar ) D w ( t?) ¡ w (0) D ¡

(t?) 2 g. 2

(4.5)

p Using equation 2.4, we see that g < ¡2 / C. Also, |t?| D kar C 1 ¡ ar k / 2. Thus, equation 4.5 implies a sufŽcient increase in f in each basic step of the SMO algorithm: f (ar C 1 ) ¡ f (ar ) ¸

1 kar C 1 ¡ ar k2 . 2C

(4.6)

494

S.S. Keerthi and S.K. Shevade

Because of equation 2.4, ¡ f is a positive deŽnite quadratic form. Hence, the solution of equation 2.10 is unique. Call this point a?. We show the following result. Lemma 1.

far g converges to a?.

Proof. Since the algorithm increases f at each step and f is bounded above,

f f (ar ) g is a convergent sequence. By equation 4.6, we get that far C 1 ¡ ar g converges to 0. Because ¡ f is a positive deŽnite quadratic form, the set fa | f (a) ¸ f (a0g is a compact set. Since far g lies in this set, it is a bounded sequence and therefore has at least one limit point. Let far ( s) gs ¸ 0 denote a convergent subsequence and a N denote the limit point to which it converges. For any r ¸ 0, let i ( r) D i up (ar ) and j ( r) D i low (ar ), the two indices chosen for optimization at the rth step. Since w 0 ( t?) D 0 for t? given by equation 4.1, we get from equation 3.9 that Fi ( r ) (ar C 1 ) ¡ Fj ( r ) (ar C 1 ) D 0.

(4.7)

Since there is only a Žnite number of indices, there exists at least one pair (i1 , j1 ) such that i1 D i (r (s ) ) and j1 D j (r (s ) ) for inŽnitely many s. Let us restrict ourselves to this subsequence. To keep notations simple, we rename the subsequence and take that i1 D i (r (s ) ) D i up (ar ( s ) )

and

j1 D j (r (s ) ) D i low (ar ( s) )

8 s ¸ 0.

Since bup and blow are continuous functions of a, we also get bup (a) N ¡ blow ( a) N D lim [bup (ar ( s) ) ¡ blow (ar ( s) ) ] s!1

( ) ( ) D lim [Fi1 (ar s ) ¡ Fj1 (ar s ) ] s!1

(4.8)

D lim [P(s) C Q ( s ) C R ( s) ], s!1

where P ( s) D [Fi1 (ar ( s) ) ¡ Fi1 (ar ( s) C 1 ) ]

Q ( s) D [Fi1 (ar ( s) C 1 ) ¡ Fj1 (ar ( s) C 1 ) ]

(4.9)

( ) ( ) R ( s) D [Fj1 (ar s C 1 ) ¡ Fj1 (ar s ) ].

Since far ( s) C 1 ¡ ar ( s) g converges to 0, lims!1 P (s ) D 0 and lims!1 R ( s) D 0. By equation 4.7, Q ( s) D 0 8s. Thus, equation 4.8 yields bup (a) N ¡ blow ( a) N D 0. By equation 3.5, a N is a solution of equation 2.10, that is, aN D a?. Strict convexity of ¡ f implies that the sequence far g itself converges to a?. This completes the proof.

SMO Algorithm for Least-Squares SVM Formulation

495

The implementation of the SMO algorithm is extremely simple. A pseudocode is given in the appendix. For efŽciency, a cache is maintained for all Fi ’s. At each basic step, when only two indices, say, i1 and i2, change their old and aold to anew and anew , the values from ai1 Fi ’s can be updated as i2 i1 i2 new old Q new old Q Fnew D Fold i i C (ai1 ¡ ai1 ) k ( xi1 , xi ) C (ai2 ¡ ai2 ) k ( xi2 , xi ) 8i.

(4.10)

Also, the updating of the dual objective function can be written, using equations 4.4 and 4.5, as f new D f old ¡

g ?2 (t ) , 2

(4.11)

new ¡ aold ) old ¡ anew ) . (ai1 where t? D (ai2 i2 D i1 The algorithm can be initialized with a D 0. For this choice, Fi D ¡yi for all i and f (a) D 0. When the regularization parameter C is tuned (say, using k-fold cross validation) we usually come across a situation in which the solution of equation 2.10 is available for some C and 2.10 needs to be solved again for dC where d > 0. In such a situation, we can do alpha seeding (DeCoste & Wagstaff, 2000), that is, use the solution at C to initialize the variables for solving the problem at dC. We give the details below. Suppose a is the solution at C, and f , Fi 8 i are available. To start the algorithm for dC, we can use the same a. For this choice, the Fi for dC can be initialized using

F i :D F i ¡

(d ¡ 1) ai 8i. dC

(4.12)

After the Fi are computed, f can be obtained using f D

1X ai ( yi ¡ Fi ). 2 i

(4.13)

An alternative is to initialize with da for solving the problem at dC. Update formulas similar to equations 4.12 and 4.13 can be derived for this case too. Both initializations above work effectively and give similar performance. 5 Conjugate Gradient Algorithm

Suykens et al. (1999) devised an efŽcient method for obtaining the solution of LS-SVM using the conjugate gradient technique (Golub & Van Loan, 1996). This method is well suited for problems with a large number of examples. Let KQ be the m £m matrix whose (i, j)th element is kQ (xi , xj ). The steps of their method are as follows.2 2 The notations and description of the method given here are different from those given by Suykens et al. (1999).

496

S.S. Keerthi and S.K. Shevade

CG Method

1. Solve Q De Kg

and

Q Dy Kº

(5.1)

for g and º using the conjugate gradient method (twice), where e is a vector with all elements equal to 1 and y is a vector whose ith component is yi . 2. Compute b D ¡g ¢ y /g ¢ e, and set a D º C bg. Although the conjugate gradient algorithm solves a system of m linear equations in m steps, for efŽciency, it is important to stop the algorithm much earlier than m steps. Hamers, Suykens, and De Moor (2001) suggest stopping the algorithm when one of three criteria is satisŽed; for details, see their report. These criteria are not directly related to the solution of the problem in equation 2.2 and hence cannot be used efŽciently to stop the algorithm with a prescribed guarantee on the primal solution, such as satisfying equation 3.14. Below we explain how Dgap can be computed easily in the CG method so that equation 3.14 can be satisŽed by using equation 3.13. Although the linear systems in equation 5.1 are decoupled, for efŽcient use of kernel computations, they are solved together. At any stage of application of the conjugate gradient method to solve them, g and º, which approxQ imately satisfy equation 5.1, are available. The residue vectors, rg D e ¡ Kg Q and rº D y ¡ Kº, play a crucial role in the computations and are also available. Step 2 of the CG method can be used to construct approximate b and a. Using that a, it is possible to compute F (the vector containing the Fi ’s) as Q ¡ y D b ( e ¡ rg ) ¡ rº. F D Ka

(5.2)

Then Dgap can be obtained using equations 3.10 and 3.12. However, there is a difŽculty associated with the use of Dgap computed in this way for ensuring equation 3.14. Recall from section 3 that P to derive equation 3.14, an important requirement is that the constraint i ai D 0 has to be satisŽed. Since g and º are only approximate solutions of equation 5.1, this condition is usually not satisŽed. Fortunately, this issue is easily taken care of if we replace step 2 of the CG method with this alternate implementation: Compute b D ¡º ¢ e /g ¢ e and set a D º C bg.

(5.3)

What we have done is replace g ¢ y with º ¢ e in the calculation of b. Note that when g and º satisfy equation 5.1, we have g ¢ y D º ¢ e. Once step 2 is replaced by equation 5.3, application of equations 5.2, 3.10, 3.12, and 3.13 guarantees equation 3.14.

SMO Algorithm for Least-Squares SVM Formulation

497

Table 1: Computational Costs for SMO and CG Algorithms (a D 0 Initialization).

log 10 C ¡4 ¡3 ¡2 ¡1 0 1 2 3 4

Banana s 2 D 1.8221

Image s 2 D 2.7183

Splice s 2 D 29.9641

Waveform s 2 D 24.5325

SMO

CG

SMO

CG

SMO

CG

SMO

CG

0.90 0.83 0.53 0.50 0.70 3.53 31.58 320.42 3317.90

0.32 0.48 0.64 1.28 2.24 4.00 7.84 14.40 32.32

9.30 7.48 5.17 4.84 7.05 33.65 254.13 1908.62 11825.40

3.38 5.07 8.05 20.28 49.01 131.82 410.67 1275.95 4096.56

4.73 4.35 2.94 3.13 3.07 5.33 11.45 29.88 99.90

1.00 2.00 3.00 6.00 13.00 30.00 65.00 89.00 109.00

0.93 0.91 0.52 0.42 0.54 2.39 10.51 21.62 28.33

0.32 0.48 0.64 1.12 2.40 5.76 14.56 23.36 27.36

Note: Each unit corresponds to 106 kernel evaluations.

As in the SMO algorithm, alpha seeding can be done for the CG method also, as pointed out by Hamers et al. (2001). EfŽcient update formulas for rº, rg , and the dual objective function can be written down similar to equations 4.12 and 4.13.

6 Numerical Experiments on ClassiŽcation Problems

The SMO and CG methods were implemented in Fortran and executed on a Pentium IV 1.5 GHz machine running on Windows platform. The gaussian xk2 ) was used. For both methods we used the kernel K ( x, xN ) D exp(¡ kx¡N 2s 2 termination criterion in equation 3.13 with 2 D 10¡6 . We empirically compared the two methods in terms of computational cost, measured using the number of kernel evaluations since the bulk of the effort in both algorithms resides in computing kernels. Four benchmark data sets were used for this purpose: Banana, Image, Splice, and Waveform. Detailed information about these data sets can be found in Ra¨ tsch (1999). For each data set, we Žxed s at a suitable value that gives good generalization performance and varied C over a wide range. In the Žrst experiment, we tried the following nine C values: 10i , i D ¡4, ¡3, . . . , 3, 4. For each C, the methods were initialized with a D 0. The computational costs associated with the four data sets are given in Table 1 as functions of C. In the second experiment, we tested the effect of alpha seeding on the computational cost. C was sequentially varied from 2 ¡10 to 210 in multiples of 2. Thus, there were 21 C values. For C D 2 ¡10, the methods were ini-

498

S.S. Keerthi and S.K. Shevade

Table 2: Computational Costs for SMO and CG Algorithms with Alpha Seeding. Banana s 2 D 1.8221

Image s 2 D 2.7183

Splice s 2 D 29.9641

Waveform s 2 D 24.5325

log 2 C

SMO

CG

SMO

CG

SMO

CG

SMO

CG

¡10 ¡9 ¡8 ¡7 ¡6 ¡5 ¡4 ¡3 ¡2 ¡1 0 1 2 3 4 5 6 7 8 9 10

0.83 0.62 0.47 0.51 0.44 0.44 0.42 0.42 0.43 0.45 0.60 0.90 1.47 2.61 4.84 9.31 18.42 36.56 72.40 146.95 297.37

0.48 0.32 0.32 0.48 0.64 0.80 0.96 1.12 1.28 1.60 2.08 2.24 2.88 3.04 4.00 5.12 5.44 7.04 8.64 9.76 12.16

7.44 6.15 5.54 4.56 4.69 4.09 3.98 3.79 3.82 4.38 5.83 8.71 13.95 24.16 43.70 81.07 149.06 278.62 518.92 955.57 1717.04

5.07 3.38 5.07 5.07 6.76 10.14 13.52 18.59 21.97 28.73 38.87 54.08 72.67 98.02 138.58 197.73 265.33 378.56 549.25 763.88 1098.50

4.37 3.50 3.34 2.92 2.96 2.75 2.56 2.70 2.40 2.32 2.43 2.74 3.19 3.62 3.91 4.09 4.04 4.17 4.18 4.04 3.77

2.00 1.00 2.00 2.00 3.00 3.00 4.00 5.00 6.00 8.00 10.00 14.00 17.00 23.00 32.00 37.00 52.00 65.00 73.00 82.00 88.00

0.91 0.78 0.70 0.50 0.39 0.40 0.42 0.37 0.38 0.39 0.49 0.73 1.09 1.82 2.99 4.70 7.09 9.92 12.78 14.91 15.83

0.48 0.32 0.32 0.48 0.48 0.64 0.80 0.96 1.12 1.44 1.92 2.56 3.68 4.96 6.40 8.96 11.84 14.72 17.44 20.64 23.36

Note: Each unit corresponds to 106 kernel evaluations.

tialized with a D 0. For all other C values, alpha seeding was used. 3 The computational costs associated with the four data sets are given in Table 2 as functions of C. For the SMO algorithm, the increase in cost at large C values is sharp. See, for instance, the growth in cost of SMO after C D 102 for the Banana and Image data sets. In the case of the Image data set, while SMO was doing quite better than CG at low and medium C values, it became very expensive at large C values. In most practical problems, the optimal value of C where best generalization occurs (say, the value at which the k-fold cross-validation error is smallest) is usually not high. For instance, in the case of Image data set, Žve-fold cross-validation error was smallest around C D 24.16. Thus, when searching for the optimal C, if large C values can be carefully avoided, then the SMO algorithm will work very efŽciently. It is also worth noting that for both SMO and CG methods, the effectiveness of the alpha seeding diminishes at large C values. Since large C values correspond to large costs 3 In section 4 we mentioned two alpha seedings: (1) set a to the optimal a of the previous C value and (2) set a to d times the optimal a of the previous C value. Our experiment showed that both initializations give nearly the same cost.

SMO Algorithm for Least-Squares SVM Formulation

499

Table 3: Ratio of the Cross-Validation Cost of CG and the SMO Algorithm. Banana (s 2 D 1.8221)

Image (s 2 D 2.7183)

Splice (s 2 D 29.9641)

Waveform (s 2 D 24.5325)

1.89

2.62

5.77

1.34

for both methods, such values should be avoided (unless really needed) in the C tuning procedure to maintain efŽciency. On the Image and Splice data sets, SMO was quite faster than CG at the middle C values. Since the “optimal” C usually occurs in this range, this is an advantage in favor of the SMO method. On the Banana and Waveform data sets, the difference between the two methods was not much in this range of C values. When C is tuned using cross validation, we usually start with a low C value, say, 2 ¡10, increase C in steps (say, in multiples of 2), and stop when the cross-validation error clearly shows no further decrease. We compared the CG and the SMO methods with respect to the total cost associated with this process. To do this, we used the same four data sets of Table 2, while also Žxing s 2 at the values given in that table. Fivefold cross validation was employed. Table 3 gives the ratio of the computational efforts associated with the two methods. The SMO method is clearly faster in all cases, so the SMO algorithm proposed here can be viewed as an excellent alternative method for solving LS-SVMs. To see how the SMO algorithm for LS-SVM scales to large-size problems, we did a scaling experiment on the Adult and Web data sets by gradually increasing the training set size in eight steps and observing the training cost. (These data sets are available on-line from Platt’s SMO page: http://www.research.microsoft.com/˜jplatt/smo.html.) The Adult data set was trained with C D 1.0 and s 2 D 10, while the Web data set was trained with C D 5.0 and s 2 D 10. Figures 1 and 2 give plots of the computational costs of the two algorithms as functions of the problem size. The SMO algorithm for LS-SVM scales well on both data sets, with scaling exponents of 2.017 and 2.006, respectively, for the Adult and Web data sets. For the CG algorithm, the corresponding scaling exponents were 2.385 and 2.496. This is due to the fact that the number of conjugate gradient iterations needed to achieve equation 3.13 slowly rises as the number of examples increases. The results indicate that the SMO algorithm is better suited for large problems. Next, we briey compared the generalization performance of SVM and LS-SVM. Fivefold cross validation was used to tune the hyperparameters involved in the problem formulations (that is, C and s), and the test set error was obtained using the optimal hyperparameter values for each formulation. The search for optimal hyperparameters was done on a 20 £ 20 uniform grid in the (log C, log s) space. The fractions of test set errors (TErr) are given in Table 4. It is clear that the generalization capabilities of both

500

S.S. Keerthi and S.K. Shevade

1e+11

N u m b e r o f K e rn e l E v a lu a tio n s

CG SMO 1e+10

1e+09

1e+08

1e+07 1000

10000 Training set size

Figure 1: Variation of computational cost with training set size for the Adult data set. Table 4: Generalization Performances of LS-SVM and SVM on Benchmark Data Sets. Data Set

LS-SVM

SVM

Banana Image Splice Waveform

.1161 .0208 .1012 .1089

.1180 .0228 .0989 .1076

methods are comparable. This observation is consistent with that made by Van Gestel et al. (2000). 7 Solution of KFD

Mika, Ra¨ tsch, Weston, Sch¨olkopf, and Muller ¨ (1999) Žrst proposed the idea of (regularized) KFD. Mika et al. (2001) gave a quadratic programming

SMO Algorithm for Least-Squares SVM Formulation

501

1e+11

N u m b e r o f K e r n e l E v a lu a tio n s

CG SMO 1e+10

1e+09

1e+08

1000

10000 Training set size

Figure 2: Variation of computational cost with training set size for the Web data set.

formulation of KFD employing general regularizers. For the case of Kregularizer, that is, kwk2 , Van Gestel et al. (2002) established the equivalence 4 between LS-SVM and KFD. In this section, we give an alternative to their way of establishing the equivalence between the KFD and LS-SVM formulations. Our approach clearly brings out the relationship between the dual variables and the threshold parameter associated with the two formulations. The quadratic programming formulation of Mika et al. (2001) is: min s.t.

1 O 2 C C 2 kwk 2

P

O2 i ji

yi ¡ (wO ¢ zi ¡ bO ) D jOi 8i P P jOi D 0 and i2IC

(7.1) O i2I¡ ji D 0,

4 By equivalence, we mean that the normal directions of the optimal hyperplanes associated with the two formulations are the same.

502

S.S. Keerthi and S.K. Shevade

where I C and I¡ are the index sets corresponding to the two classes and, as usual, yi takes the value 1 if example i is in class 1 and ¡1 otherwise. The Lagrangian for this problem is 1 C X O2 X O 2C j C a O i [yi ¡ ( wO ¢ zi ¡ bO ) ¡ jOi ] LO D kwk 2 2 i i i X X C mC jOi C m ¡ jOi , i2IC

i2I¡

where the a O i ’s, m C and m ¡ are Lagrange multipliers. Setting the gradients O O bO and jOi ’s to zero and using them together with the of L with respect to w, equalities in equation 7.1 yields the following square system of equations O m C , and m ¡ : for solving the a O i ’s, b, P P i

P

aO i D 0,

j

Oj C Kij a

i2IC

1 a O C i

m D bO C Ci C yi 8 i

aO i D m C m C

and

P

i2I¡

aO i D m¡m ¡ ,

(7.2)

where m i is m C if yi D 1 and m ¡ if yi D ¡1; m C is the number of examples in class 1 (positive class); m ¡ is the number of examples in class 2 (negative class); and m D m C C m ¡. These equations imply that m C m C C m¡m ¡ D 0. Let us deŽne m using the equation m D m C m C D ¡m¡m ¡ .

(7.3)

If we deŽne aD 1C

m 2C

³

1 1 C m C m¡

´ and

bD

µ ³ ´¶ 1 O m 1 1 ¡ bC , 2C m C m ¡ a

(7.4)

then the right-hand side of each of the Žrst-level equations in equation 7.2, that is, bO C mCi C yi , can be rewritten as a ( b C yi ) . If we also deŽne ai D

a Oi a

8i,

(7.5)

it is easy to see that the ai and b are obtained by solving P j

Kij aj C C1 ai ¡ b D yi 8 i P i ai D 0.

(7.6)

Using the ai ’s, we can get m by solving X i2IC

µ ai D m / a D m /

m 1C 2C

³

1 1 C mC m¡

´¶ .

(7.7)

SMO Algorithm for Least-Squares SVM Formulation

503

O and the a Then equations 7.3, 7.4, and 7.5 yield m C , m ¡ , a, b, O i ’s. Since equation 7.6 is nothing but the solution of the instance of equation 2.2 corresponding to LS-SVM, it follows that the solution of equation 7.1 can be obtained by solving LS-SVM; this can be numerically done using the SMO algorithm of section 4. The two equality constraints in the last row of equation 7.1 contribute only to a scaling of w and a shift in b given by the LS-SVM solution. Thus, if b and bO associated with equations 2.2 and 7.1 are determined in a uniform way (say, using the class posterior probabilities obtained in a post-processing step; Platt, 1999), then the classiŽers obtained by using these two formulations are identical. 8 Primal Problem Without b

In kernel ridge regression (Saunders et al., 1998) and Bayesian designs (Van Gestel et al., 2002), it is usual to consider the primal problem, equation 2.2, P without the b parameter.5 When b is absent, the equality constraint i ai D 0 disappears from the dual, equation 2.10, and the optimality conditions for the dual problem become Fi D 0 8i.

(8.1)

Because of the absence of the equality constraint, the SMO algorithm can simply modify one ai at a time. If the indices are chosen sequentially for updating, then the SMO algorithm is precisely the SOR algorithm (Bertsekas, 1999; Hamers et al., 2001). The efŽciency of this SOR algorithm can be signiŽcantly improved if we modify it so that at any given instance, the index that gives the best increase in f value is chosen. This can be neatly done by maintaining a cache for the Fi ’s. Like the sequential SOR algorithm, the modiŽed algorithm also requires only O ( m ) computations and kernel evaluations in each step. However, the modiŽed algorithm requires many fewer steps to achieve the termination criterion. Let us now work out the details of the modiŽed algorithm. Suppose at a given a, an index i is considered for updating. DeŽne a Q i ( t) D ai C tI

6 iI a Q j ( t) D aj 8j D

and

w ( t) D f (a(t) Q ).

(8.2)

With t? and anew as deŽned in equation 4.1, it is easy to check that t? D

Fi kQ ( xi , xi )

and

f (anew ) ¡ f (a) D

F2i 2 kQ (xi , xi )

.

(8.3)

5 If b is present and the term c b2 / 2 is added to the cost function in equation 2.2, that is equivalent to a primal problem without b if the constant 1 /c is added to the kernel function k.

504

S.S. Keerthi and S.K. Shevade

Thus, at each step of the SMO algorithm, we can choose i that has the F2 largest value of Q i . Since a cache is maintained for the Fi ’s and kQ ( xi , xi ) , 2 k ( xi ,xi )

i D 1, . . . , m can be precomputed and stored, the above step can be efŽciently done. Convergence of the resulting SMO algorithm can be established along the lines given in section 4. The proofs actually become much simpler because of the simpler problem and algorithm associated with the absence of b. The solution of the problem in equation 2.10 without the equality conQ D y, so the solution can also be straint corresponds to the solution of Ka obtained directly using the conjugate gradient method. Duality gap ideas discussed in section 3 can be easily adapted (simply omit b in all calculations) to give efŽcient stopping criteria for both SMO and CG methods. 9 Conclusion

We have given a new algorithm for least-squares formulations of SVMs (LSSVMs and kernel ridge regression), proved its convergence, and discussed implementation aspects. Our algorithm also applies to a particular form of regularized kernel Fisher discriminant method. The algorithm is robust in the sense that on the many complex data sets we have tried, there was not a single case of failure. It is also fast and scales well to large-size problems. It provides a better alternative to the conjugate gradient algorithm developed by Hamers et al. (2001), especially for large problems. Appendix: Pseudocode for the SMO Algorithm

Notes for the Pseudocode 1. kernel(i,j) refers to kQ ( xi , xj ) given by equation 2.4. 2. The ai are stored in alpha array, while the Fi are stored in Fcache array. 3. 2 in equation 3.13 is referred to as epsilon . 10¡6 is a suitable value for it. 4. eps is used to check if sufŽcient change has occurred in each step. It can be set to 10¡12. 5. The function fabs returns the absolute value of a real number. 6. Any line in the pseudocode that begins with the character % is treated as a comment.

procedure takeStep(i1, i2) alph1 = alpha{i1}; alph2 = alpha{i2}; y1 = y{i1}; y2 = y{i2}; F1 = Fcache{i1}; F2 = Fcache{i2}; gamma = alph1 + alph2; k11 = kernel(i1,i1); k22 = kernel(i2,i2); k12 = kernel(i1,i2);

SMO Algorithm for Least-Squares SVM Formulation

505

eta = 2*k12-k11-k22; a2 = alph2 - (F1-F2)/eta; if (fabs(a2-alph2) < eps*(a2+alpha2+eps)) return 0; a1 = gamma-a2; alpha{i1} = a1; alpha{i2} = a2; Update the DualObjectiveFunction using (4.11). Update Fcache{i} for all i in I using (4.10). Compute (iÇ low, bÇ low) and (iÇ up, bÇ up) using (3.4). return 1; endprocedure procedure ComputeDualityGap() DualityGap = 0; set b using (3.16) or (3.17). loop over all i xi = b + (alpha{i}/C) - Fcache{i}; DualityGap += alpha{i}(Fcache{i}-(0.5*alpha{i}/C)) + C*xi*xi/2; return DualityGap; endprocedure procedure InitializeVariables(delta) if delta is non-zero % delta denotes the factor by which C is changed. for every i, reset Fcache{i} using (4.12). compute DualObjectiveFunction using (4.13). else initialize alpha{i}=0 for all i. set Fcache{i}=-y{i} for all i and DualObjectiveFunction to 0. Compute (iÇ low, bÇ low) and (iÇ up, bÇ up) using (3.4). endprocedure procedure SMO() % If solution is available at some value of C=C1 % and solution is to be % found at C=C2 then set delta := C2/C1. Otherwise, % set delta := 0. InitializeVariables(delta); numChanged = 1; DualityGap = ComputeDualityGap(); while (DualityGap > epsilon*DualObjectiveFunction && numChanged != 0) numChanged = takeStep(iÇ up, iÇ low); DualityGap = ComputeDualityGap(); % If the procedure stops with numChanged=0, it means that % epsilon is too small. endprocedure

506

S.S. Keerthi and S.K. Shevade

References Bertsekas, D. P. (1999). Nonlinear programming (2nd ed.). Belmont, MA: Athena ScientiŽc. Chang, C. C., & Lin, C. J. (2001). LIBSVM: A library for support vector machines. Available on-line at: http://www.csie.ntu.edu.tw/»cjlin/libsvm. DeCoste, D., & Wagstaff, K. (2000).Alpha seeding for support vector machines. Paper presented at the International Conference on Knowledge Discovery and Data Mining, Boston. Fletcher, R. (1987). Practical methods of optimizations (2nd ed.). New York: Wiley. Golub, G. H., & Van Loan, C. F. (1996). Matrix computations (3rd ed.). Baltimore: Johns Hopkins University Press. Hamers, B., Suykens, J., & De Moor, B. (2001). A comparison of iterative methods for least squares support vector machine classiŽers (Internal Rep. 01-110). Leuven, Belgium: ESAT-SISTA, K. U. Leuven. Keerthi, S. S., Shevade, S. K., Bhattacharyya, C., & Murthy, K. R. K. (2001). Improvements to Platt’s SMO algorithm for SVM design. Neural Computation, 13, 637–649. Mika, S., R¨atsch, G., & Muller, ¨ K. R. (2001). A mathematical programming approach to the kernel Fisher algorithm. In T. K. Leen, T. G. Dietterich, & V. Tresp (Eds.), Advances in neural informationprocessingsystems,13 (pp. 591–597).Cambridge, MA: MIT Press. Mika, S., R¨atsch, G., Weston, J., Sch¨olkopf, B., & Muller, ¨ K. R. (1999). Fisher discriminant analysis with kernels. In Y. H. Hu, J. Larsen, E. Wilson, & S. Douglas (Eds.), Neural networks for signal processing IX (pp. 41–48). New York: IEEE. Platt, J. (1998). Sequential minimal optimization: A fast algorithm for training support vector machines (Tech. Rep. MSR-TR-98-14). Redmond, WA: Microsoft Research. Platt, J. (1999). Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In A. Smola, B. Sch¨olkopf, & D. Schuurmans (Eds.), Advances in large margin classiŽers. Cambridge, MA: MIT Press. R¨atsch, G. (1999). [Benchmark repository.] Available on-line at: http://ida.Žrst. gmd.de/»raetsch/data/benchmarks.htm. Saunders, C., Gammerman, A., & Vovk, V. (1998). Ridge regression learning algorithm in dual variables. In Proceedingsof the Fifteenth InternationalConference on Machine Learning (ICML-98). Madison, WI. Smola, A., & Sch¨olkopf, B. (1998) A tutorial on support vector regression (NeuroColt2 Tech. Rep. NC2-TR-1998-030).Berlin: ESPRIT Working Group in Neural and Computational Learning II. Suykens, J., Lukas, L., Van Dooren, P., De Moor, B., & Vandewalle, J. (1999). Least squares support vector machine classiŽers: A large scale algorithm. In Proceedings of the European Conference on Circuit Theory and Design (ECCTD’99) (pp. 839–842). Stresa, Italy. Suykens, J., & Vandewalle, J. (1999). Least squares support vector machine classiŽers. Neural Processing Letters, 9, 293–300.

SMO Algorithm for Least-Squares SVM Formulation

507

Vanderbei, R. J. (1994). LOQO: An interior point code for quadratic programming (Tech. Rep. SOR-94-15). Princeton, NJ: Statistics and Operations Research, Princeton University. Van Gestel, T., Suykens, J., Baesens, B., Viaene, S., Vanthienen, J., Dedene, G., De Moor, B., & Vandewalle, J. (2000). Benchmarking least squares support vector machine classiŽers (Internal Rep. 00-37). Leuven, Belgium: ESAT-SISTA, K. U. Leuven. Van Gestel, T., Suykens, J., Lanckriet, G., Lambrechts, A., De Moor, B., & Vandewalle, J. (2002). A Bayesian framework for least squares support vector machine classiŽers. Neural Computation, 14, 1115–1147. Received May 9, 2002; accepted July 8, 2002.

ARTICLE

Communicated by Bard Ermentrout

Synchronization in Networks of Excitatory and Inhibitory Neurons with Sparse, Random Connectivity Christoph Borgers ¨ [email protected] Department of Mathematics, Tufts University, Medford, MA 02155, U.S.A.

Nancy Kopell [email protected] Department of Mathematics and Center for BioDynamics, Boston University, Boston, MA 02215, U.S.A.

In model networks of E-cells and I-cells (excitatory and inhibitory neurons, respectively), synchronous rhythmic spiking often comes about from the interplay between the two cell groups: the E-cells synchronize the I-cells and vice versa. Under ideal conditions—homogeneity in relevant network parameters and all-to-all connectivity, for instance—this mechanism can yield perfect synchronization. We Žnd that approximate, imperfect synchronization is possible even with very sparse, random connectivity. The crucial quantity is the expected number of inputs per cell. As long as it is large enough (more precisely, as long as the variance of the total number of synaptic inputs per cell is small enough), tight synchronization is possible. The desynchronizing effect of random connectivity can be reduced by strengthening the E!I synapses. More surprising, it cannot be reduced by strengthening the I!E synapses. However, the decay time constant of inhibition plays an important role. Faster decay yields tighter synchrony. In particular, in models in which the inhibitory synapses are assumed to be instantaneous, the effects of sparse, random connectivity cannot be seen. 1 Introduction In networks of E-cells (excitatory neurons) and I-cells (inhibitory neurons), synchronous, rhythmic spiking often results from the interplay between the two cell groups, with the E-cells synchronizing the I-cells and vice versa. This mechanism, called PING (pyramidal interneuronal network gamma; Whittington, Traub, Kopell, Ermentrout, & Buhl, 2000) or ° -II (Tiesinga, Fellous, Jos´e, & Sejnowski, 2001), has been observed in modeling studies, and there are reasons to believe that some experimentally observed gamma rhythms are in fact based on this mechanism (Traub, Jefferys, & Whittington, 1999; Whittington et al., 2000; Tiesinga et al., 2001). Neural Computation 15, 509–538 (2003)

c 2003 Massachusetts Institute of Technology °

510

C. Borgers ¨ and N. Kopell

A

B E­ cells

400

E­ cells

400 200

200 0 100

I­ cells

I­ cells

0 100 50 0 0

100 t

200

50 0 0

100 t

200

Figure 1: PING in E-I networks with (A) all-to-all connectivity and (B) sparse connectivity.

The conditions under which PING occurs are not completely understood, though we discuss them heuristically in section 6. We focus here on the effects of random connectivity on PING. This effect is illustrated by Figure 1. Details about this Žgure will be given later. For now, it sufŽces to say that Figure 1A shows the emergence of PING in a model E=I network with all-to-all connectivity and without any kind of heterogeneity in network parameters. The Žgure indicates spike times. Both cell groups (E and I) synchronize tightly. If the connectivity is made sparse and random, Figure 1A turns into Figure 1B. The two cell groups now Žre spike volleys of brief but positive duration. (The spike volleys of the I-cells are so brief that they appear to have zero duration in the plot.) The main goal of this article is to analyze the durations and shapes of these volleys and their dependence on network parameters. Synchronization in the presence of random connectivity has been studied previously for excitatory networks (Barkai, Kanter, & Sompolinsky, 1990), inhibitory networks (Brunel & Hakim, 1999; Wang & Buzs a´ ki, 1996), and E=I networks (Brunel, 2000; Bush & Sejnowski, 1996; Golomb & Hansel, 2000; Hansel & Mato, 2001; van Vreeswijk & Sompolinsky, 1996, 1998; Wang, Golomb, & Rinzel, 1995). This literature is aimed at understanding either the stability of the asynchronous state or transitions from asynchrony to rhythms and vice versa. We do not consider these issues here, but focus instead on a detailed understanding of the near-synchronous state. In section 2, we review the theta model (Ermentrout & Kopell, 1986; Gutkin & Ermentrout, 1998; Hoppensteadt & Izhikevich, 1997), an idealization of a large class of conductance-based neuronal models. Our arguments and simulations in this article are based on this model. We also introduce

Synchronization in Sparse, Random Networks

511

our model of synapses in section 2 and describe the connectivity of our model networks. In section 3, we present numerical experiments demonstrating that the desynchronizing effect of sparseness and randomness in the connectivity primarily originates from the variance in the number of inputs per cell, not from the randomness per se. The synchronization of a population of cells by an inhibitory input pulse is analyzed in section 4, Žrst assuming that all cells receive the same input pulse, and then, motivated by the result of section 3, assuming that different cells receive input pulses of different strengths. Similarly, the synchronization of a population of cells by an excitatory input pulse is analyzed in section 5, Žrst assuming that all cells receive the same input pulse, then assuming that different cells receive input pulses of different strengths. The results of sections 4 and 5 are combined in section 6 to analyze PING in E=I networks. In section 7, we summarize our results and put them into the context of other recent work on the same subject. 2 Review of Theta Neurons 2.1 Equation of a Single Theta Neuron. In the Hodgkin-Huxley model, a periodically spiking space-clamped neuron is represented by a point moving on a limit cycle in a four-dimensional phase space. Analogously, in the theta model (Ermentrout & Kopell, 1986; Gutkin & Ermentrout, 1998; Hoppensteadt & Izhikevich, 1997), a neuron is represented by a point P D .cos µ; sin µ / moving on the unit circle S1 . In the absence of synaptic coupling, the differential equation governing the motion is 1 dµ D .1 ¡ cos µ/ C I.1 C cos µ/: dt ¿

(2.1)

Here, I should be thought of as an input “current,” measured in radians per unit time. The time constant ¿ > 0 is needed to make equation 2.1 dimensionally correct; it is analogous to a membrane time constant. When I < 0, equation 2.1 has the two Žxed points: µ0§ D §2 arccos p

1 1 ¡ ¿I

:

µ0¡ is stable, and µ0C is unstable. For I < 0, the vector Želd on the circle is shown in the left panel of Figure 2A. If µ is perturbed slightly from µ0¡ , it returns to µ0¡ . However, if µ is raised beyond µ0C , a large excursion occurs, with the point P moving around the entire circle while µ increases to µ0¡ C2¼. The stable Žxed point µ0¡ is the analog of the stable equilibrium of a neuron. The unstable Žxed point µ0C is the analog of a spiking threshold. As I approaches 0, the Žxed points approach each other. A saddle-node bifurcation occurs when I D 0. The two Žxed points come together at µ D 0 (see the center panel of Figure 2A). When I > 0, dµ=dt > 0 for all t, so there is no Žxed point (see the right panel of Figure 2A).

512

C. Borgers ¨ and N. Kopell

A:

B:

sin q

1 0

­1 0

20

40

t

60

80

100

Figure 2: Theta model. (A) Vector Želd on the circle for I < 0, I D 0, and I > 0. (B) Sin µ as a function of time.

The transition from I < 0 to I > 0 is the analog of the transition from excitability to spiking in a neuron. Neuronal models are called of type I if this transition involves a saddle-node bifurcation on a limit cycle and of type II if it involves a subcritical Hopf bifurcation (Ermentrout, 1996; Gutkin & Ermentrout, 1998; Rinzel & Ermentrout, 1998). This classiŽcation goes back to Hodgkin (1948). Thus, the theta model is a type I neuronal model. It has been shown to be canonical, in the sense that other type I models can be reduced to it by coordinate transformations (Ermentrout & Kopell, 1986; Hoppensteadt & Izhikevich, 1997). If 0 < I ¿ 1=¿ , the motion is much slower near µ D 0, that is, near the “ghost” of the Žxed point annihilated in the saddle-node bifurcation, than elswhere. This is illustrated in Figure 2B, which shows sin µ as a function of t for ¿ D 1, I D 0:02. As the point P D .cos µ; sin µ/ moves slowly past .1; 0/, sin µ changes slowly. As P moves rapidly around the circle, sin µ rapidly rises to 1, then falls to ¡1, then returns to values slightly below 0. To some extent the graph of sin µ resembles the voltage trace of a spiking neuron.1 1 Some authors think of ¡ cos µ , not sin µ , as the “voltage-like” quantity in the theta model. Which of the two we call voltage-like is of no consequence for this article, and neither is voltage-like in any precise sense. However, we prefer to think of sin µ as the voltage-like quantity for the following aesthetic reason. Consider a theta neuron driven slightly above threshold. Near the ghost of the equilibrium point (µ D 0), sin µ is slowly increasing, while ¡ cos µ has a local minimum. On the other hand, for a Hodgkin-Huxleytype neuron of type I driven slightly above threshold, the membrane potential V is slowly

Synchronization in Sparse, Random Networks

513

When µ crosses .2l ¡ 1/¼ , l integer, with dµ=dt > 0, we say that the neuron spikes. If I > 0 and ¡¼ · µ1 · µ2 · ¼, the time it takes for µ to rise from µ1 to µ2 equals Z

µ2 µ1

dµ D .1 ¡ cos µ/=¿ C I.1 C cos µ/

r µ ¶ tan.µ =2/ µ2 ¿ arctan p : I ¿I µ1

(2.2)

Setting µ1 D ¡¼ and µ2 D ¼ in this formula, we Žnd that the period equals r

PD¼

¿ : I

(2.3)

We denote the time it takes for µ to rise from ¼=2 to 3¼=2 by W and call it the spike width. Applying formula 2.2 with .µ1 ; µ2 / D .¼=2; ¼ / and .µ1 ; µ2 / D .¡¼; ¡¼=2/ and adding the results, we Žnd µ ¶r 1 ¿ W D ¼ ¡ 2 arctan p : I ¿I

(2.4)

For physiological realism, we wish to ensure W=P ¿ 1. By equations 2.3 and 2.4, 2 1 W D 1 ¡ arctan p : P ¼ ¿I Therefore, W=P ¿ 1 means the same as ¿ I ¿ 1. Since arctan.1=²/ D ¼=2 ¡ ² C O.² 2 / as ² ! 0, equation 2.4 implies W ¼ 2¿ when ¿ I ¿ 1. Thus, in the parameter regime of interest to us, ¿ is approximately half the spike width. Motivated by this discussion and by the fact that spike widths in real neurons are on the order of milliseconds, we set ¿ D1 for the remainder of this article, think of time as measured in milliseconds, and always consider input currents I ¿ 1. 2.2 Synapses Between Theta Neurons. We model synapses by adding time-dependent input currents to equation 2.1. When the presynaptic neuron spikes, the postsynaptic neuron receives an input current (positive or negative, depending on whether the synapse is excitatory or inhibitory), which jumps to its maximum value instantaneously and then decays exponentially. This is the model of section 2.1.5 of Izhikevich (2000) (see also Ermentrout, 1996, for a discussion of synapses between theta neurons). increasing near the ghost of the equilibrium point. In this sense, V is more similar to sin µ than to ¡ cos µ .

514

C. Borgers ¨ and N. Kopell

Thus, a network of N coupled theta neurons is described by a system of differential equations of the form Á ! N X dµj D 1 ¡ cos µj C Ij C ®i gij sij .1 C cos µj /; 1 · j · N: dt iD1 Ij denotes the external input to the jth neuron, which can be positive or negative. The constant ®i equals C1 or ¡1, depending on whether neuron i is excitatory (E) or inhibitory (I). The constant gij ¸ 0 measures the strength of the synapse from neuron i to neuron j, and sij D sij .t/ is the synaptic gating variable associated with this synapse. The value of sij always lies between 0 and 1. It jumps to 1 when neuron i spikes. Between spikes of neuron i, it decays exponentially, following the differential equation dsij sij D¡ : dt ¿ij The decay time constants ¿ij are positive. The jumps of sij occurring when neuron i spikes cause difŽculties in the numerical simulation of the network and are not physiologically realistic. In our simulations, we therefore replace the jumps by rapid but smooth rises, letting sij be governed by a differential equation of the form dsij dt

D¡

sij ¿ij

C e¡´.1Ccos µi /

1 ¡ sij ¿R

;

with ¿R D 0:1 and ´ D 5. The term e¡´.1Ccos µi / .1 ¡ sij /=¿R is very close to zero unless µi ¼ .2l ¡ 1/¼ , l integer, and drives sij toward 1 rapidly when µi ¼ .2l ¡ 1/¼ . The parameter ¿R is reminiscent of a synaptic rise time. Figure 3 shows sij for ¿ij D 2 and 10, with the input of the presynaptic neuron chosen so that its period equals 25. 2.3 Sparse, Random E=I Networks. Throughout this article, the decay time constants ¿ij are assumed to depend on the type of i only (excitatory or inhibitory), so there are two distinct values of ¿ij , denoted ¿E and ¿I . We also assume that all excitatory neurons receive the same constant external input drive IE , and all inhibitory neurons receive the same constant external drive II . In later sections, we consider networks of coupled theta neurons, including NE D 4N=5 excitatory and NI D N=5 inhibitory neurons. These proportions are motivated by the fact that in large portions of the cortex, there are about four times more excitatory than inhibitory neurons (Braitenberg & Schuz, ¨ 1998). The strengths gij of the synapses are chosen at random. For a given network, they are chosen once and for all; they do not depend on time. To de-

Synchronization in Sparse, Random Networks

515

1

2:

0.5

s

t

0 0

20

40

20

40

60

80

100

60

80

100

1

10 :

0.5

s

t

0 0

t

Figure 3: Synaptic gating variable s as a function of time, with decay time constants ¿ D 2 and ¿ D 10.

scribe the choice of the gij , the following notation is useful. Let E µ f1; : : : ; Ng denote the set of all indices of excitatory neurons, and similarly I the set of all indices of inhibitory neurons. For i 2 E and j 2 I , we deŽne gij D

gEI wij ; pEI NE

(2.5)

with ( wij D

1 0

with probability pEI , otherwise,

(2.6)

where gEI ¸ 0 and pEI 2 .0; 1] are constants. For j 2 I , we deŽne wEj D

X

(2.7)

wij :

i2E

Note that wEj is a binomially distributed random variable. We also deŽne gEj D

X i2E

gij D

gEI wEj : pEI NE

(2.8)

516

C. Borgers ¨ and N. Kopell

From the formulas for the mean and standard deviation of binomially distributed random variables, we see that gEj has mean gEI and standard deviation s 1 ¡ pEI (2.9) ¾EI D gEI : pEI NE By the central limit theorem, gEj is approximately normally distributed if pEI NE is large. Assuming pEI ¿ 1, as is physiologically realistic (Braitenberg & Schuz, ¨ 1998), equation 2.9 shows that 1 ¾ EI ¼p : gEI pEI NE

(2.10)

The left-hand side of equation 2.10 is the coefŽcient of variation (the standard deviation divided by the mean) of gEj , j 2 I . The expression pEI NE appearing on the right-hand side is the expected number of excitatory inputs per I-cell. Formulas analogous to 2.5 through 2.10 apply to the I ! E, E ! E, and I ! I synapses. In particular, s

¾IE D gIE

1 ¡ pIE ; pIE NI

(2.11)

and therefore 1 ¾ IE ¼ p gIE pIE NI

(2.12)

for pIE ¿ 1. The left-hand side of equation 2.12 is the coefŽcient of variation of gIj , j 2 E . The expression pIE NI appearing on the right-hand side is the expected number of inhibitory inputs per E-cell. Several authors have pointed out that pEI NE and pIE NI are much more important than pEI , NE , pIE , and NI in isolation (Golomb & Hansel, 2000; Tiesinga, Fellous, Jos´e, & Sejnowski, 2002; Wang et al., 1995). 3 Loss of Tight Synchrony Is Attributable to Variance in the Number of Inputs per Cell Before considering networks with sparse, random connectivity, we consider one with all-to-all connectivity (pEI D pIE D 1:0). Figure 1A shows an example of a simulation with IE D 0:1; II D 0; gEI D gIE D 0:25; gEE D gII D 0; ¿E D 2; ¿I D 10:

(3.1)

Synchronization in Sparse, Random Networks

517

Figure 1A shows spike times, with the horizontal axis indicating time and the vertical axis cell index. Each of the two cell groups (E and I) synchronizes very rapidly, with the synchronous population spikes of the I-cells slightly lagging behind those of the E-cells. The synchronization mechanism seen in Figure 1A is PING, briey described in section 1 and discussed in more detail in section 6.1. We comment briey on our parameter choices. A neuron driven with p IE D 0:1 spikes periodically with an interspike interval equal to ¼= I ¼ 9:93. Since we think of time as measured in milliseconds, this corresponds to a frequency of .1000=9:93/ H ¼ 100 Hz. Thus, the E-cells are driven so hard that they would spike above gamma frequency if they were not subject to any inhibition. The I-cells are driven at threshold; they do not spike without additional excitatory input, but any excitatory input, regardless how weak, will make them spike. The values of gEI and gIE can be varied considerably without any qualitative change in Figure 1A. However, for small values of gEI (roughly < 0:1), two or more population spikes of the E-cells occur before the I-cells respond. For large values of gEI (roughly > 0:7), two or more population spikes of the I-cells occur in response to a population spike of the E-cells. For small values of gIE (roughly < 0:1), the E-cells are not synchronized. For large values of gIE , the rhythm is slow but not qualitatively different from that in Figure 1A. For simplicity, we assume here that there are no E ! E or I ! I synapses— gEE D gII D 0. In our experience, E ! E synapses (with a brief synaptic decay time such as ¿E D 2) do not affect PING rhythms much. However, I ! I synapses are crucial in some parameter regimes; this will be discussed in section 6.1. Our choices of ¿E D 2 and ¿I D 10 are motivated by the decay time constants of excitatory synapses involving AMPA receptors (approximately 2 ms) and inhibitory synapses involving GABAA receptors (approximately 10 ms); recall that we think of time as measured in milliseconds. When pEI and pIE are reduced from 1:0 to 0:5, Figure 1A turns into Figure 1B. The two cell groups now Žre spike volleys of positive durations. As stated in section 1, the main goal of this article is to understand the durations and shapes of these volleys. In the network underlying Figure 1B, each E-cell receives input from a random number of I-cells. The expected value of this number is 50. Similarly, each I-cell receives input from a random number of E-cells, with expectation 200. We now repeat the simulation of Figure 1B with a network in which the connectivity is still sparse and random, but the variance in the number of inputs per cell has been eliminated. That is, each E-cell receives input from a random set of exactly 50 I-cells, and each I-cell receives input from a random set of exactly 200 E-cells. The network is like that of Figure 1B in all other regards. The result is shown in Figure 4A; tight synchrony is restored. When the number of inputs per cell is Žxed, tight synchronization is possible even for much sparser networks. Figure 4B shows a simulation

518

C. Borgers ¨ and N. Kopell

A

B E­ cells

400

E­ cells

400 200

200 0 100

I­ cells

I­ cells

0 100 50 0 0

100 t

50 0 0

200

200

D 400

E­ cells

E­ cells

C

100 t

1000

200

I­ cells

0 100

I­ cells

0 400 200 0 0

100 t

200

50 0 0

100 t

200

Figure 4: Simulation of E-I networks with sparse, random connectivity but without variance in the number of inputs per cell. (A) Parameters as in Figure 1B, but with variance in numbers of inputs per cell eliminated. (B) Only four excitatory inputs per I-cell and one inhibitory input per E-cell. (C) Same as B in a larger network. (D) Same as B, but with only one excitatory input per I-cell.

in which each E-cell receives input from one I-cell, and each I-cell receives input from four E-cells. Although synchronization is achieved a little less rapidly than in Figure 4A, it becomes tight within a few oscillation periods. A four times larger network, again with one inhibitory input into each Ecell and four excitatory inputs into each I-cell, shows similar behavior (see Figure 4C).

Synchronization in Sparse, Random Networks

519

If the number of excitatory inputs per I-cell is reduced again, from four to one, there appears to be no more synchronization (see Figure 4D). Thus, there appears to be a minimum number of inputs per cell needed for synchronization—a “percolation threshold.” However, this number is very small in our simulations. In previous work on different models, similar thresholds have been found (Golomb & Hansel, 2000; Wang & Buzs a´ ki, 1996; Wang et al., 1995). In all three of these references, asynchrony was found to become unstable when the number of inputs per cell exceeded a threshold value independent of network size. A cell in a human brain typically receives input from thousands of other cells (Braitenberg & Schuz, ¨ 1998). Our numerical experiments suggest that with so many synapses, the impact of sparse, random connectivity on synchronization is attributable to the variance in the number of inputs per cell, not to the percolation threshold. We will assume this to be the case from now on. 4 Synchronization of a Population of Theta Neurons by a Single Strong Inhibitory Pulse 4.1 Synchronization by an Inhibitory Pulse of Uniform Strength. We consider a population of N identical, uncoupled neurons with common constant external drive above threshold, receiving a common inhibitory synaptic input pulse at time 0. Following section 2, we model this population by the equations dµj D .1 ¡ cos µj / C .I ¡ gs.t//.1 C cos µj /; dt

1 · j · N;

(4.1)

with I > 0; g > 0, and ( e¡t=¿I if t > 0 s.t/ D 0 if t · 0 with ¿I > 0. If the inhibitory pulse is strong, it brings the population close to synchrony. An example is shown in Figure 5A, which displays results of a simulation with N D 100;

I D 0:05;

g D 0:25;

¿I D 10:

(4.2)

The synchronization brought about by the inhibitory pulse at time 0 is immediate and nearly perfect. To understand the synchronization shown in Figure 5A, consider the initial value problem, dµ D 1 ¡ cos µ C .I ¡ ge¡t=¿I /.1 C cos µ/; dt µ.0/ D µ0 ;

t > 0;

(4.3) (4.4)

520

C. Borgers ¨ and N. Kopell

Figure 5: Synchronization by a single inhibitory pulse. (A) Inhibitory pulse of uniform strength. (B) Inhibitory pulse of nonuniform strength. (C) Phase portrait for equations 4.5 and 4.6 with I D 0:05, ¿I D 10, with the stable river indicated boldly. (D) Distribution of spikes within the Žrst volley following t D 0, in a simulation identical to that in B, but with 1000 neurons: predicted (solid line) and actual (bars).

with ¡¼ < µ0 < ¼ . We deŽne J.t/ D I ¡ ge¡t=¿I : Equations 4.3 and 4.4 can then be rewritten in the form dµ D 1 ¡ cos µ C J.1 C cos µ/ dt dJ J¡I D¡ dt ¿I µ .0/ D µ0

J.0/ D I ¡ g:

(4.5) (4.6) (4.7) (4.8)

Synchronization in Sparse, Random Networks

521

The phase portrait for the two-dimensional dynamical system 4.5 and 4.6 is shown in Figure 5C for I D 0:05 and ¿I D 10. The dashed line in Figure 5C is the nullcline dµ=dt D 0. The nullcline dJ=dt D 0 is the horizontal line J D I, the upper edge of the window shown in Figure 5C. The Žgure should be extended periodically in µ with period 2¼. The ow is upward, in the direction of increasing J. The most striking feature of Figure 5C is the existence of strongly attracting and strongly repelling trajectories. Trajectories of this kind exist in many systems of ordinary differential equations and are called rivers (Diener, 1985a, 1985b). The Žgure reveals a stable river, that is, a trajectory .µs ; Js / that is attracting in forward time, indicated as a bold line in Figure 5C, with .µs ; Js / ! .¡¼; ¡1/ as t ! ¡1, dµs =dt > 0, and Js .t/ D I ¡ ge ¡t=¿I

(4.9)

for all t. We denote by T the time when µs .T/ D ¼, and deŽne J¤ D Js .T/ 2 .0; I/:

(4.10)

Equations 4.9 and 4.10 imply T D ¿I ln g ¡ ¿I ln.I ¡ J¤ /:

(4.11)

Note that the phase portrait depends on the parameters I and ¿I but not on g. Therefore, the value of J¤ depends on I and ¿I but not on g. The synchronization seen in Figure 5A can be understood from Figure 5C in the following way. For g sufŽciently large (that is, J.0/ sufŽciently negative), and for µ0 sufŽciently far from ¼ , .µ .t/; J.t// is rapidly attracted to .µs .t/; Js .t//. At the time when µ D ¼, we therefore have J ¼ J¤ or t ¼ T. Thus, the Žrst spike after time zero occurs approximately at time T. For µ0 sufŽciently close to ¼ , µ.t/ quickly passes through ¼ and is then rapidly attracted to .µ s .t/ C 2¼; Js .t//. When µ.t/ reaches 3¼, J ¼ J¤ and therefore t ¼ T. Thus, a spike occurs soon after time zero, followed by a spike approximately at time T. Only for values of µ0 in a narrow transition regime is .µ .t/; J.t// attracted to neither .µ s .t/; Js .t// nor .µs .t/ C 2¼; Js .t//. Equation 4.11 gives the time of the Žrst approximately synchronous population spike. Since we have no explicit formula for J¤ , equation 4.11 is not an explicit formula for T. However, since J¤ does not depend on g, equation 4.11 does give the precise dependence of T on g. This dependence will be of primary interest to us in the remainder of section 4. For later reference, we note how equation 4.11 is modiŽed when the synchronizing inhibitory pulse arrives not at time 0 but at some time T0 : T D T0 C ¿I ln g ¡ ¿I ln.I ¡ J¤ /:

(4.12)

522

C. Borgers ¨ and N. Kopell

4.2 Approximate Synchronization by an Inhibitory Pulse of Nonuniform Strength. Motivated by section 3, we are interested in the effects of variable synaptic strengths. We therefore let the constant g in equation 4.1 depend on j: dµj

D .1 ¡ cos µj / C .I ¡ gj s.t//.1 C cos µj /; 1 · j · N: dt That is, different neurons receive inhibitory pulses of different strengths. We assume that the gj are independent, normally distributed random variables, with mean g > 0 and standard deviation ¾ g > 0. If g is large enough and ¾ g is small enough, the population is still synchronized approximately. This is illustrated in Figure 5B, which shows results of a simulation similar to that of Figure 5A, with N D 100;

I D 0:05;

g D 0:25;

¾ g D 0:025;

¿I D 10:

(4.13)

(Compare these parameters with those in equations 4.2.) Instead of the nearly synchronous population spikes of Figure 5A, we now see spike volleys of brief but positive durations. To analyze the durations of these spike volleys, let us consider the initial value problem 4.3 and 4.4 with a random g > 0. Let ½ g D ½g .° /, ° > 0 be the probability density of g, and let g > 0 and ¾ g > 0 be its mean and standard deviation. Let X D ln g:

(4.14)

Combining equations 4.11 and 4.14, T D ¿I X ¡ ¿I ln.I ¡ J¤ /:

(4.15)

The only random quantity on the right-hand side of equation 4.15 is X. We will discuss its distribution Žrst. Let ½X D ½X .» /, ¡1 < » < 1 be the probability density of X. For ¡1 < a < b < 1, Z

b a

½X .» / d» D P.X 2 .a; b// D P.ln g 2 .a; b// D P.g 2 .ea ; eb // Z D

Z

eb ea

½ g .° / d° D

b

e» ½ g .e» / d»:

a

Therefore, ½X .» / D e» ½g .e» /

(4.16)

for all » . For small ¾ g , the standard deviation of X is ¾X ¼ ln0 .g/¾ g D

¾g : g

(4.17)

Synchronization in Sparse, Random Networks

523

From equations 4.15 and 4.16, we see that the probability density function of T is ½T .t/ D

1 .tC¿I ln.I¡J¤ //=¿I ¤ e ½g .e.tC¿I ln.I¡J //=¿I /: ¿I

(4.18)

For later reference, we note how formula 4.18 changes if the approximately synchronizing inhibitory pulse arrives not at time 0 but at some time T0 : ½T .t/ D

1 .t¡T0 C¿I ln.I¡J¤ //=¿I ¤ e ½ g .e.t¡T0 C¿I ln.I¡J //=¿I /: ¿I

(4.19)

Using equations 4.15 and 4.17, we see that the standard deviation of T is ¾ T D ¿I ¾ X ¼ ¿I

¾g g

(4.20)

for small ¾ g . We think of ¾T as a measure of the duration of the spike volleys. Thus, the duration of the spike volleys is proportional to the product of ¿I , the decay time constant of inhibition, and the coefŽcient of variation ¾ g =g of g. To verify these results computationally, we return to the example of Figure 5B. Strictly speaking, the preceding discussion does not apply to this example, since g, which is assumed to be normally distributed, is not guaranteed to be positive. However, formulas 4.16, 4.18, and 4.20 are well deŽned if ½ g is a normal density. Since g D 0:25 and ¾ g D 0:025, the probability of g · 0 is extremely small. We therefore expect equations 4.16, 4.18, and 4.20 to hold with good accuracy. We deŽne T .j/ to be the time of the spike of the jth neuron within the Žrst nearly synchronous spike volley. We set

O TD

PN

jD1 T

N

.j/

and

¾O T D

v uP u N .T .j/ ¡ T/ O 2 t jD1 N¡1

:

(4.21)

Here and for the remainder of this article, hats indicate results obtained from numerical simulations. In the example of Figure 5B, we Žnd ¾O T ¼ 1:02: We see that ¾O T is indeed close to ¿I ¾ g =g, which equals 1:0 in this example. If we double ¿I in this experiment, ¾O T rises from 1:02 to 2:04, in agreement with equation 4.20. To verify equation 4.18 numerically, we must know the value of ¿I ln.I ¡ J ¤ /. Taking expectations on both sides of equation 4.15, we Žnd ¿I ln.I ¡ J ¤ / D ¿I E.X/ ¡ E.T/ D ¿I E.ln g/ ¡ E.T/:

524

C. Borgers ¨ and N. Kopell

For sufŽciently small ¾ g , this implies ¿I ln.I ¡ J¤ / ¼ ¿I ln g ¡ E.T/; suggesting the approximation O ¿I ln.I ¡ J¤ / ¼ ¿I ln g ¡ T:

(4.22)

Figure 5D shows the density ½T , as deŽned in equation 4.18, using the approximation 4.22, with ¿I D 10, and assuming that ½ g is a normal density with g D 0:25 and ¾ g D 0:025. The histogram in Figure 5D indicates the actual spike time density, determined from the numerical simulation. The agreement between the theoretical prediction and the actual spike time distribution is excellent. For later reference, we note how equation 4.22 changes when the inhibitory pulse arrives not at time 0 but at some time T0 . From equation 4.12, we then obtain O ¿I ln.I ¡ J¤ / ¼ ¿I ln g C T0 ¡ T:

(4.23)

5 Synchronization of a Population of Theta Neurons by a Single Strong Excitatory Pulse 5.1 The Synchronous Population Spike Triggered by an Excitatory Pulse of Uniform Strength. We next consider a population of N identical, uncoupled neurons with common constant external drive below or at threshold, receiving a common excitatory synaptic input pulse at time 0. We model this situation by the equations dµj D .1 ¡ cos µj / C .I C gs.t//.1 C cos µj /; dt

1 · j · N;

(5.1)

with I · 0, g > 0, and ( e¡t=¿E if t > 0 s.t/ D 0 if t · 0 with ¿E > 0. If the excitatory pulse is strong, it triggers a nearly synchronous population spike soon after time 0. To analyze this in more detail, we consider the initial value problem: dµ D 1 ¡ cos µ C .I C ge¡t=¿E /.1 C cos µ/; dt 1 µ .0/ D ¡2 arccos p : 1¡I

t > 0;

(5.2) (5.3)

Synchronization in Sparse, Random Networks

525

cell index

100

A:

50 0 ­ 10

0 t

10

B:

T

10 5 0 0

0.2

0.4

0.2

0.4

g

0.6

0.8

1

0.6

0.8

1

C:

¶ T/¶ g

0 ­5

­ 10 0

g

Figure 6: (A) Approximately synchronous population spike triggered by a single nonuniform excitatory pulse. (B) Time T between arrival of excitatory pulse and spike triggered by it, as a function of the strength g of the pulse, for .I; ¿E /D.0; 1/ (solid line), .0; 5/ (dashed line), .0; 2/ (dash-dotted line), .¡0:01; 2/ (circles). (C) @T=@g as a function of g, for the parameter values of B.

Recall from section 2 that for I < 0, the right-hand side of equation 5.3 represents the stable Žxed point of the equation dµ=dt D 1 ¡ cos µ C I.1 C cos µ /. Thus, we are considering the response of a neuron at rest to an excitatory synaptic pulse. We denote by T the Žrst time at which the neuron spikes, with T D 1 if there is no spike at all. We have no general analytic expression for T as a function of I, ¿E , and g. However, it is easy to see that for Žxed I and ¿E , T is a strictly decreasing function of g, with limg!1 T D 0 and limg!gc C T D 1 for some gc ¸ 0 (see Figure 6B). For ¿E D 1, T can be computed using formula 2.2, with I

526

C. Borgers ¨ and N. Kopell

replaced by I C g and ¿ D 1. The formula becomes particularly simple for I D 0; in that case, TD

¼ 1 p : 2 g

(5.4)

The approximation ¿E D 1 is accurate as long as e¡T=¿E ¼ 1, since then the exponential decay in equation 5.2 can be neglected over the time interval [0; T]. Since T ! 0 as g ! 1, this means that the assumption ¿E D 1 is accurate for sufŽciently large g. The assumption I D 0 is accurate when jIj=g is sufŽciently small. So this assumption too is accurate for sufŽciently large g. Figure 6B shows T as a function of g, for various values of I and ¿E , demonstrating that equation 5.4 approximates T reasonably well over a large range of parameter values. 5.2 The Approximately Synchronous Population Spike Triggered by an Excitatory Pulse of Nonuniform Strength. If the synaptic strength in equation 5.1 depends on j, that is, if different neurons receive excitatory pulses of different strengths, the equations are dµj

D .1 ¡ cos µj / C .I C gj s.t//.1 C cos µj /; 1 · j · N: dt Figure 6A shows that the resulting population spike is not perfectly synchronous. (But notice that Figure 6A shows a brief time window only; the synchronization is not perfect but fairly tight.) In the simulation underlying this Žgure, N D 100;

I D 0;

g D 0:25;

¾ g D 0:025;

¿E D 2:

(5.5)

To analyze the duration of the spike volley triggered by an excitatory pulse of nonuniform strength, we consider the initial value problem 5.2, 5.3 with a random g > gc . If ¾ g is small, the standard deviation of T is ­ ­ ­ @T ­ ­ (5.6) ¾T ¼ ­ ¾g : ­ @g ­ For ¿E D 1 and I D 0, @T ¼ D g¡3=2 4 @g

(5.7)

by equation 5.4. Figure 6C shows @ T=@g as a function of g, for various values of I and ¿E . The Žgure conŽrms that the right-hand side of equation 5.7 approximates @T=@g reasonably well for large values of g. Combining equations 5.6 and 5.7 yields ¾T ¼

¼ 1 ¾g p : 4 g g

(5.8)

Synchronization in Sparse, Random Networks

527

For illustration, we return to the example of Figure 6A (I D 0, ¿E D 2, g D 0:25, ¾ g D 0:025). We deŽne T .j/ to be the time of the spike of the jth neuron, and deŽne TO and ¾O as in equation 4.21. Numerically, we Žnd T

¾O T ¼ 0:270: To evaluate the right-hand side of equation 5.6, we approximate @T=@g numerically. We Žnd @T=@g ¼ ¡10:30 for the parameter values of Figure 6A. The approximation of equation 5.6, based solely on the assumption that ¾ g is so small that the relation between ¾T and ¾ g is approximately linear, proves fairly accurate here; it yields ¾T ¼ 0:256: The assumption ¿E D 1, which underlies equation 5.8, degrades the accuracy, but by less than a factor of two: ¾T ¼ 0:157: 6 The PING Synchronization Mechanism 6.1 PING in Fully Connected E-I Networks. We return to the example of Figure 1A. Each of the cell groups (E and I) synchronizes rapidly, with the population spikes of the inhibitory neurons slightly lagging behind those of the excitatory ones. We state, in a nonrigorous way, based on numerical experience and heuristics, conditions that are sufŽcient to induce Žring patterns as in Figure 1A: Condition 1: The E-cells receive external input signiŽcantly above their spiking threshold. Condition 2: The E ! I synapses are so strong and have so short a rise time that a surge in spiking of the E-cells quickly triggers a surge in spiking of the I-cells. Condition 3: The I-cells spike only in response to the E-cells. Condition 4: The I ! E synapses are so strong that a population spike of the I-cells approximately synchronizes the E-cells. If these four conditions are satisŽed, synchronous rhythmic spiking develops as follows (Whittington et al., 2000; Tiesinga et al., 2001). Initial activity in the E-cells triggers activity in the I-cells. This inhibits activity in the E-cells, thereby removing the drive to the I-cells. A period of low activity in both E- and I-cells results. When the inhibition wears off and the E-cells spike again, they are closer to synchrony than previously because of the mechanism described in section 4.1. The spiking of the E-cells causes spiking of the I-cells, closer to synchrony than previously because of the mechanism of section 5.1. The cycle now repeats.

528

C. Borgers ¨ and N. Kopell

A

B E­ cells

400

E­ cells

400 200

200 0 100

I­ cells

I­ cells

0 100 50 0 0

100 t

50 0 0

200

C E­ cells

E­ cells

400

200

200 0 100

I­ cells

0 100

I­ cells

200

D

400

50 0 0

100 t

100 t

200

50 0 0

100 t

200

Figure 7: Illustration of conditions 2–4 from section 6.1. (A) PING is lost when E ! I synapses become too weak. (B) PING is lost as a result of too much drive to the I-cells. (C) Rhythm is restored by adding I ! I synapses. (D) PING is lost when I ! E synapses become too weak.

Condition 1 is evidently needed to drive activity. We discuss conditions 2 through 4 in more detail and present numerical results illustrating what happens when they are violated. When condition 2 is violated, that is, when the E ! I synapses are weak, a pattern such as the one shown in Figure 7A often develops: the E-cells and the I-cells still synchronize, but the E-cells spike several times between population spikes of the I-cells. The parameters in Figure 7A are as in Figure 1A, except that gEI has been reduced from 0:25 to 0:05.

Synchronization in Sparse, Random Networks

529

Condition 3 is violated if the drive to the I-cells becomes too strong, but can be restored by introducing I ! I-synapses. We illustrate this with the following numerical experiment. In Figure 1A, II D 0. If we raise II to 0:05, the Žgure changes dramatically, as shown in Figure 7B. Here, II is strong enough to drive asynchronous activity in the I-cells that suppresses the Ecells altogether. Condition 3, and with it the rhythm, is restored by setting gII D 0:25 (see Figure 7C). We note that in the example of Figure 7C, the I-cells would spike synchronously even without the E-cells. Thus, the role of the I-I synapses is to synchronize the I-cells, replacing nearly constant inhibition by phasic inhibition, which allows the E-cells to Žre. One might therefore consider the rhythm in Figure 7C an “ING” or “° -I” rhythm (see section 7). The parameter regime investigated here is similar to that of Figure 7d of Tiesinga et al. (2001). There as here, asynchronous activity of the I-cells suppresses activity in the E-cells altogether, while synchronous activity of the I-cells permits Žring of the E-cells. However, in Figure 7C, the E-cells do play an important role in setting the frequency of the rhythm. In the absence of the E-cells, the rhythm would be much slower. Thus, the Žring of the I-cells in Figure 7C does come in response to the Žring of the E-cells, as in PING. Figure 7D shows what may happen when condition 4 is violated, that is, when the I ! E synapses are weak. The parameters in Figure 7D are as in Figure 1A, except that gIE has been reduced from 0:25 to 0:05. The inhibitory synapses no longer sufŽce to synchronize the E-cells. As a result, the I-cells, which were synchronized by the E-cells in Figure 1A, are no longer synchronized either. A mathematical examination of conditions 1 through 4 will be the subject of future publications. 6.2 PING in Sparsely, Randomly Connected E-I Networks. When pEI and pIE are reduced from 1:0 to 0:5, Figure 1A turns into Figure 1B. In analyzing the population spikes of the E-cells in Figure 1B, we make the simplifying assumption that the population spikes of the I-cells are perfectly synchronous. Since the I-cells are in fact fairly tightly synchronized in Figure 1B, this is a good approximation at least for the parameters used in Figure 1B. When the I-cells spike, all E-cells receive inhibitory pulses. However, different neurons receive inputs of different strengths because of the random connectivity. The resulting approximately synchronous spike volley of the E-cells can be analyzed using section 4.2. We focus on one particular spike volley of the E-cells, say, the Žrst one .j/ following t D 100 in Figure 1B. We deŽne TE to be the time of the spike of the jth E-cell during this volley. We deŽne v u P .j/ P .j/ u .T ¡ TO /2 T t j E E OT D j E and ¾O E D : E NE NE ¡ 1

530

C. Borgers ¨ and N. Kopell

From equations 2.11 and 4.20, we Žnd the prediction s ¾O E ¼ ¿I

1 ¡ pIE : pIE NI

(6.1)

For the Žrst spike volley of the E-cells following t D 100 in Figure 1B, we Žnd numerically ¾O E D 1:18: The prediction of equation 6.1 is remarkably accurate: s 1 ¡ pIE ¼ 1:22: ¿I pIE NI We expect the shape of the spike volley to be approximately described by equation 4.19. To evaluate equation 4.19, we must evaluate T0 ¡ ¿I ln.I ¡ J¤ /. (Recall that T0 is the time at which the synchronizing inhibitory pulse arrives, here the time of the inhibitory population spike immediately preceding the excitatory population spike under consideration.) Following equation 4.23, we use the approximation T 0 ¡ ¿I ln.I ¡ J¤ / ¼ TO E ¡ ¿I ln gIE : The predicted and actual spike time distributions are shown in Figure 8. The agreement is good. We now consider the Žrst spike volley of the I-cells following t D 100 in .j/

Figure 1B. We deŽne TI to be the time of the spike of the jth I-cell during this volley, and v u P .j/ P .j/ u .T ¡ TO /2 t j I I OT D j TI and O D ¾ : I I NI NI ¡ 1 From equations 2.9 and 5.8, we Žnd the prediction s ¼ 1 ¾O I ¼ p 4 gEI

1 ¡ pEI : pEI NE

(6.2)

For the Žrst spike volley of the I-cells following t D 100 in Figure 1B, we Žnd numerically ¾O I D 0:151 : The prediction of equation 6.2 is somewhat inaccurate, as was to be expected, because it is based on three rather substantial idealizations: ¿E D 1, perfect

Synchronization in Sparse, Random Networks

531

Figure 8: Distribution of spikes within the Žrst spike volley following t D 100 in Figure 1B: predicted (solid line) and actual (bars).

synchrony of the E-cells, and the assumption that the I-cells return to rest between the spike volleys of the E-cells (see equation 5.3). However, the discrepancy is still not greater than a factor of two: s ¼ 1 p 4 gEI

1 ¡ pEI ¼ 0:0785: pEI NE

We note that it would be possible to relax the assumption of perfect synchrony in the E-cells, since we have fairly precise information about the durations and even the shapes of the spike volleys of the E-cells. We do not pursue this here, since our goal here is qualitative insight, not precise quantitative information. 7 Discussion We have analyzed the effects of sparse, random connectivity on the PING synchronization mechanism. In particular, we have derived approximate formulas for the durations of the spike volleys, equations 6.1 and 6.2. To make these formulas as transparent as possible, let us use the approximations 1 ¡ pIE ¼ 1 and 1 ¡ pEI ¼ 1, and write M EI D pEI NE

532

C. Borgers ¨ and N. Kopell

for the expected number of excitatory inputs per inhibitory cell and MIE D pIE NI for the expected number of inhibitory inputs per excitatory cell. Equations 6.1 and 6.2 then become ¾O E ¼ ¿I p

1

(7.1)

M IE

and 1 ¼ p ¾O I ¼ p : 4 gEI MEI

(7.2)

An interesting feature of these formulas is their lack of symmetry. The time constant ¿I appears in equation 7.1, but the time constant ¿E does not appear in equation 7.2. Similarly, gEI appears in equation 7.2, but gIE does not appear in equation 7.1. If a theta neuron were driven with a constant drive equal to gEI , it would spike periodically with a period which we call PEI . From equation 2.3, ¼ P EI D p : gEI Using this in equation 7.2, we Žnd ¾O I ¼

1 PEI p : 4 M EI

(7.3)

Let P denote the period of the PING rhythm. In the simulations of this article, P EI ¿ ¿I < P;

M EI > MIE À 1:

(7.4)

The inequality MIE À 1 is certainly realistic (Braitenberg & Schuz, ¨ 1998). The inequality ¿I < P holds for a gamma rhythm if the inhibitory synapses are mediated by GABAA , since the typical period of gamma oscillations is about 25 ms and the decay time constant of GABA A synapses is about 10 ms. It appears that PING without PEI ¿ ¿I is not possible with realistic values of ¿I ; more will be said on this point. Combining 7.1, 7.3, and 7.4, we Žnd ¾O I ¿ ¾O E ¿ P: Thus, our theory predicts that for PING in realistic parameter regimes, sparse and random connectivity will not prevent either E- or I-cells from synchronizing fairly tightly, but the I-cells will synchronize much more tightly

Synchronization in Sparse, Random Networks

533

E­ cells

400 200

I­ cells

0 100 50 0 0

100 t

200

Figure 9: PING in an E-I network with sparse, random connectivity, with nearly instantaneous synapses.

than the E-cells. Our computational results are in agreement with these conclusions (see Figure 1B). It is possible to obtain PING without the condition PEI ¿ ¿I if ¿I is made unrealistically small. To compensate, one must then also make gIE unrealistically large. In fact, in many modeling studies, synapses have been assumed to act instantaneously, that is, with zero rise and decay times (Brunel, 2000; Brunel & Hakim, 1999; Izhikevich, 1999; Mirollo & Strogatz, 1990; Tiesinga & Sejnowski, 2001); this amounts to taking a limit in which simultaneously ¿I ! 0 and gIE ! 1. (In Brunel, 2000, and Brunel & Hakim, 1999, there is a delay between the spiking of the presynaptic neuron and its effect on the postsynaptic neuron; however, this delay is not relevant for this discussion.) In some regard, the behavior of instantaneous synapses is not very different from that of synapses with more realistic time courses. However, equations 7.1 and 7.2 predict that the effects of random connectivity will be reduced dramatically by making the synapses instantaneous. To verify this numerically, we repeat the simulation of Figure 1B, with ¿I reduced by a factor of 50 and gIE raised by a factor of 50. This closely mimics instantaneous synapses but is computationally simpler. The result is shown in Figure 9. As predicted, synchronization is much tighter than in Figure 1B, even though the randomness in the connectivity is the same in Figure 9 as in Figure 1B. The rhythm is also accelerated; this is a result of the reduction in ¿I .

534

C. Borgers ¨ and N. Kopell

Our theory for the spike volleys of the E-cells is precise enough to allow accurate predictions not just of the durations of the volleys but even of their shapes, as shown in Figure 8. A reŽnement of the theory could be obtained by taking into account the positive durations of the spike volleys of the I-cells, perhaps using ideas similar to those of section 3.2 of Tiesinga and Sejnowski (2001). (Recall that in section 4, the duration of the spike volleys of the I-cells was assumed to be negligible.) However, our numerical results indicate that in our parameter regime, neglecting the durations of the spike volleys of the I-cells gives an excellent approximation. Tiesinga et al. (2002) discuss a related problem, using numerical simulation primarily, and with an emphasis on information-theoretic ideas. They also present experimental results. For instance, Figure 8b of their article illustrates how the jitter in the spike times of a rat hippocampal neuron decreases when the quantity npre (our pIE NI ) increases, in rough qualitative agreement with equation 6.1. (Rough qualitative agreement is the best that can be expected here, in view of the idealized nature of our theory.) Our theory for the spike volleys of the I-cells is less accurate than that for the excitatory ones. However, since the synchronization of the I-cells is typically quite tight and brought about by a crude mechanism (a burst of excitation triggers an almost immediate, and therefore almost synchronous, response of the I-cells), a precise theory for the spike volleys of the I-cells is of less interest here. To create such a theory, we would need to study how the duration of an excitatory input spike volley is related to the duration of an output spike volley triggered by it. (Recall that in section 5, the duration of the spike volleys of the E-cells was assumed to be negligible.) This issue is centrally important in the study of synŽre chains (Diesmann, Gewaltig, & Aertsen, 1999). Figure 3c of Diesmann et al. (1999) shows that the strength of the input spike volley is crucial. Strong, loosely synchronous input spike volleys can trigger tightly synchronous output spike volleys. Using a different way of measuring synchrony, the relation between input synchronization and output synchronization was also studied by Burkitt and Clark (2001). Combining ideas from the preceding two paragraphs, a more accurate overall theory of PING may be created as follows. First, the approximate spike time distribution within the spike volleys of the E-cells, computed based on section 4, is taken into account in approximating the spike time distribution within the spike volleys of the I-cells. This yields a reŽnement of section 5, which in turn can be taken into account in approximating the spike time distribution within the spike volleys of the E-cells, leading to a reŽnement of section 4. Iterating this process, one may obtain increasingly accurate approximations to the spike time distributions within spike volleys. However, such an improved theory of PING is beyond the scope of this article. While working on this project, we carried out far more simulations than have been presented here. Our conclusions hold over a wide range of pa-

Synchronization in Sparse, Random Networks

535

E­ cells

400 200

I­ cells

0 100 50 0 0

100 t

200

Figure 10: ING in an E-I network with sparse, random connectivity.

rameters. For instance, E!E synapses with gEE D 0:25 have little effect on Figure 1B. Weak I!I synapses have little effect, except for the point made at the end of section 6.1: such synapses can make gamma rhythms possible in cases when the external drive to the I-cells is fairly strong. Strengthening I!I synapses often leads to a transition to a different synchronization mechanism, called ING by Whittington et al. (2000) and ° -I by Tiesinga et al. (2001). In ING, the I-cells are driven externally, not by the E-cells, and synchronize not only the E-cells but also themselves. For ING, the widths and shapes of the spike volleys of both E- and I-cells can be approximated using the ideas of section 4.1. Figure 10 shows a simulation with gIE D gII D 0:25;

gEI D gEE D 0;

IE D II D 0:1;

¿I D 10;

pIE D pII D 0:5:

As our theory would predict, the volleys of the E- and I-cells are now of equal duration. In future work, we will investigate ING in sparse, random networks in more detail, including in particular the effects of E ! I-synapses. Throughout this article, we have used the theta model. As remarked in section 2.1, the theta model is canonical for type I neuronal models in the sense that other type I models can be reduced to it by coordinate transformations (Ermentrout & Kopell, 1986; Hoppensteadt & Izhikevich, 1997). We therefore expect the picture to be qualitatively similar for all type I models,

536

C. Borgers ¨ and N. Kopell

even though the details of our calculations, and in particular the details of the centrally important Figure 5C, do depend on our choice of model. Whether and how our results generalize to type II models remains to be explored. It would also be interesting to explore the effects of spike adaptation on our analysis. Synchronization in sparsely, randomly connected networks with spike adapation has been studied by van Vreeswijk and Hansel (2001). They discuss synchronization of bursts (not individual spikes) via adaptation (not inhibition) and observe that strong I ! E synapses desynchronize bursts, in contrast with our regime, in which strong I ! E synapses synchronize spikes. The model and analysis are so different from ours that a detailed comparison would be a major endeavor, but the article certainly suggests studying effects of adaptation in our model in the future. We have analyzed synchronization by common input for the purpose of better understanding PING and ING. We remark, however, that synchronization by common input is also of neurobiological interest by itself (Usrey & Reid, 1999). We have shown that even very sparse input can synchronize and have analyzed the desynchronizing effect of heterogeneity in input strength. Acknowledgments We are grateful to Marcelo Camperi for stimulating discussions during the early stages of this work and to Steve Epstein for carefully reading the manuscript and providing helpful criticism. We thank the referees for interesting and useful comments. C. B. thanks the Center for BioDynamics at Boston University for a productive and pleasant visit during the spring of 2001, when this project was begun. His work was also supported by an equipment grant from Tufts University’s Marshall Fund for Biomedical Research. N. K. received support from NSF grant DMS-9706694 and NIH grant MH47150. References Barkai, E., Kanter, I., & Sompolinsky, H. (1990). Properties of sparsely connected excitatory neural networks. Phys. Rev. A, 41, 590–597. Braitenberg, V., & Schuz, ¨ A. (1998). Cortex: Statistics and geometry of neuronal connectivity (2nd ed.). New York: Springer-Verlag. Brunel, N. (2000). Dynamics of sparsely connected networks of excitatory and inhibitory spiking neurons. J. Comp. Neurosci., 8, 183–208. Brunel, N., & Hakim, V. (1999). Fast global oscillations in networks of integrateand-Žre neurons with low Žring rates. Neural Comp., 11, 1621–1671. Burkitt, A. N., & Clark, G. M. (2001). Synchronization of the neural response to noisy periodic synaptic input. Neural Computation, 13, 2639–2672.

Synchronization in Sparse, Random Networks

537

Bush, P., & Sejnowski, T. (1996). Inhibition synchronizes sparsely connected cortical neurons within and between columns in realistic network models. J. Comp. Neurosci., 3, 91–110. Diener, F. (1985a). Propri´et´es asymptotiques des euves. C. R. Acad. Sci. Paris, 302, 55–58. Diener, M. (1985b). D´etermination et existence des euves en dimension 2. C. R. Acad. Sci. Paris, 301, 899–902. Diesmann, M., Gewaltig, M.-O., & Aertsen, A. (1999). Stable propagation of synchronous spiking in cortical neural networks. Nature, 402, 529–533. Ermentrout, G. B. (1996). Type I membranes, phase resetting curves, and synchrony. Neural Comp., 8, 879–1001. Ermentrout, G. B., & Kopell, N. (1986). Parabolic bursting in an excitable system coupled with a slow oscillation. SIAM J. Appl. Math, 46, 233–253. Golomb, D., & Hansel, D. (2000). The number of synaptic inputs and the synchrony of large sparse neuronal networks. Neural Comp., 12, 1095–1139. Gutkin, B. S., & Ermentrout, G. B. (1998). Dynamics of membrane excitability determine interspike interval variability: A link between spike generation mechanisms and cortical spike train statistics. Neural Comp., 10, 1047–1065. Hansel, D., & Mato, G. (2001). Existence and stability of persistent states in large neuronal networks. Phys. Rev. Lett., 86, 4175–4178. Hodgkin, A. L. (1948). The local changes associated with repetitive action in a non-medullated axon. J. Physiol. (London), 107, 165–181. Hoppensteadt, F. C., & Izhikevich, E. M. (1997). Weakly connected neural networks. New York: Springer-Verlag. Izhikevich, E. M. (1999). Class 1 neural excitability, conventional synapses, weakly connected networks, and mathematical foundations of pulse-coupled models. IEEE Transactions on Neural Networks, 10, 499–507. Izhikevich, E. M. (2000). Neural excitability, spiking, and bursting. Int. J. Bifurcation and Chaos, 10, 1171–1266. Mirollo, R. E., & Strogatz, S. H. (1990). Synchronization of pulse-coupled biological oscillators. SIAM J. Appl. Math., 50, 1645–1662. Rinzel, J., & Ermentrout, G. B. (1998). Analysis of neural excitability and oscillations. In C. Koch & I. Segev (Eds.), Methods in neuronal modeling (pp. 251–292). Cambridge, MA: MIT Press. Tiesinga, P. H. E., Fellous, J.-M., Jos´e, J. V., & Sejnowski, T. J. (2001). Computational model of carbachol-induced delta, theta, and gamma oscillations in the hippocampus. Hippocampus, 11, 251–274. Tiesinga, P. H. E., Fellous, J.-M., Jos´e, J. V., & Sejnowski, T. J. (2002). Information transfer in entrained cortical neurons. Network: Comput. Neural Syst., 13, 41–66. Tiesinga, P. H. E., & Sejnowski, T. J. (2001). Precision of pulse-coupled networks of integrate-and-Žre neurons. Network: Comput. Neural Syst., 12, 215– 233. Traub, R. D., Jefferys, J. G. R., & Whittington, M. A. (1999). Fast oscillations in cortical circuits. Cambridge, MA: MIT Press. Usrey, W. M., & Reid, R. C. (1999). Synchronous activity in the visual system. Annual Review of Physiology, 61, 435–456.

538

C. Borgers ¨ and N. Kopell

van Vreeswijk, C., & Hansel, D. (2001). Patterns of synchrony in neural networks with spike adaptation. Neural Comp., 13, 959–992. van Vreeswijk, C. A., & Sompolinsky, H. (1996). Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science, 274, 1724–1726. van Vreeswijk, C. A., & Sompolinsky, H. (1998). Chaotic balanced state in a model of cortical circuits. Neural Comp., 10, 1321–1372. Wang, X.-J., & Buzs´aki, G. (1996). Gamma oscillation by synaptic inhibition in a hippocampal interneuronal network model. J. Neurosci., 16, 6402–6413. Wang, X.-J., Golomb, D., & Rinzel, J. (1995). Emergent spindle oscillations and intermittent burst Žring in a thalamic model: SpeciŽc neuronal mechanisms. Proc. Natl. Acad. Sci. USA, 2, 5577–5581. Whittington, M. A., Traub, R. D., Kopell, N., Ermentrout, B., & Buhl, E. H. (2000). Inhibition-based rhythms: Experimental and mathematical observations on network dynamics. Int. J. of Psychophysiol., 38, 315–336. Received February 11, 2002; accepted July 30, 2002.

NOTE

Communicated by Emilio Salinas

The Effects of Input Rate and Synchrony on a Coincidence Detector: Analytical Solution Shawn Mikula [email protected]

Ernst Niebur [email protected] Zanvyl Krieger Mind=Brain Institute and Department of Neuroscience, Johns Hopkins University, Baltimore, MD 21218, U.S.A.

We derive analytically the solution for the output rate of the ideal coincidence detector. The solution is for an arbitrary number of input spike trains with identical binomial count distributions (which includes Poisson statistics as a special case) and identical arbitrary pairwise crosscorrelations, from zero correlation (independent processes) to complete correlation (identical processes). 1 Introduction Individual neurons typically receive thousands of inputs that vary temporally with respect to both mean rate and cross-correlation. Much experimental evidence underscores the importance of mean input rate and cross-correlation as determinants of neural Žring, though the relative contributions of each are not so clear (Abeles, 1982, 1991; Softky & Koch, 1992, 1993; Shadlen & Newsome, 1994; Alonso, Usrey, & Reid, 1996; Decharms & Merzenich, 1996; Konig, ¨ Engel, & Singer, 1996; de Oliveira, Thiele, & Hoffmann, 1997; Riehle, Grun, ¨ Diesmann, & Aertsen, 1997; Steinmetz et al., 2000; Salinas & Sejnowski, 2000; Niebur, Hsiao, & Johnson, 2002; Williams & Stuart, 2002). Given the importance of rate and cross-correlation in neural coding, the determination of how inputs modulated with respect to both affect neural Žring assumes signiŽcance if the impact of these different neural codes is to be fully appreciated. Employing simple probabilistic and combinatorial methods, we derive the exact analytical solution for the output rate of a coincidence detector receiving an arbitrary number of inputs modulated with respect to both mean rate and cross-correlation 2 Binomial Spike Trains with SpeciŽc Cross-Correlation In this section, we introduce a systematic method for the generation of an arbitrary number of spike trains with speciŽed pair-wise mean crossNeural Computation 15, 539–547 (2003)

c 2003 Massachusetts Institute of Technology °

540

S. Mikula and E. Niebur

correlations and Žring rates. 1 Spikes are distributed according to binomial counting statistics in each spike train. Mean Žring rates and crosscorrelations are the same for all spike trains (or all pairs of spike trains, respectively), but they can be chosen independently of each other. The algorithm that we describe here is as suitable for the analytical computations in this report as for the implementation of such spike trains in numerical simulations. We emphasize that although we use this particular algorithm to illustrate the derivation of the output rate of the coincidence detector, our results are of general validity since the properties of the spike trains are entirely speciŽed by their correlation coefŽcient and the statistics of the individual spike trains. However, no higher-order effects (correlations of order 3 and higher; see, e.g., Bohte, Spekreijse, & Roelfsema, 2000) are included. First, we establish some notation. Let m be the number of input spike trains, each one having n time bins. All bins are of equal length 1t, chosen sufŽciently small so that each contains a maximum of one spike, that is, each bin is guaranteed to contain either one or zero spikes. The probability that a spike is found in any given time bin is p; no spike is therefore found with probability 1 ¡ p. Bins in any given spike train are independent, which implies that all the following analysis can be limited to a single time bin. Note that the physiologically important Poisson statistics are the special case of low rates and long spike trains. Generation of m spike trains with speciŽc correlation starts by generating m C 1 independent spike trains with the desired mean Žring rate (which is just p=1t). Let us assume we want a correlation coefŽcient q (with 0 · q · 1) between each and any pair of the m spike trains. We designate spike train number m C 1 as the reference spike train. In order to introduce the correlation coefŽcient q between spike trains 1; : : : ; m, we will switch, with p a probability q, the state of a time bin in each of these spike trains to that found in the reference spike train. This yields a mean correlation coefŽcient of q between any two of the spike trains 1; : : : ; m without changing the mean Žring rate.2 No further use is made of the reference spike train.

1 The method proposed here is not the only possible way to introduce correlations between spike trains. For instance, one could add spikes generated by a common Poisson process to each individual spike train (Stroeve & Gielen, 2001), or start from a common spike train and generate two different spike trains by removing spikes independently. The method presented here has the advantage of being straightforward and efŽcient and of generating spike trains with controlled rates and correlation coefŽcients, both determined in a direct way. 2 Since all spike trains, including the reference spike train, have the same mean Žring rate, changing the state of any bin to the state of any bin in another spike train does not change the probability of this bin’s containing a spike. Therefore, this manipulation does not change the mean Žring rate of the spike train.

Coincidence Detector: Analytical Solution

541

3 Coincidence Detection for Vanishing Input Correlation The coincidence detector is a computational unit that Žres if the number of input spikes received within a given time bin equals or exceeds the threshold, µ. It is instructive to Žrst derive a solution for independent inputs (q D 0) with mean rate p=1t. Using the binomial distribution and elementary combinatorics, we Žnd that the probability of obtaining exactly j coincident input spikes from a set of m input spike trains with probability p for a spike (to exist within each time bin) is

P.j/ D

³ ´ m j p .1 ¡ p/m¡j : j

(3.1)

Since we are interested only in cases where the coincidence detector receives at least µ coincident input spikes, the probability for the coincidence detector to produce an output spike is

Pout .p; m; µ I q D 0/ D

m ³ ´ X m jDµ

j

p j .1 ¡ p/m¡j :

(3.2)

Dividing equation 3.2 by 1t, we obtain the mean output rate of a coincidence detector in the case of vanishing input correlation:

Nout .p; m; µI q D 0/ D

m ³ ´ 1 X m j p .1 ¡ p/m¡j : 1t jDµ j

(3.3)

4 Finite Correlation Let us continue to the case of input spikes that vary with respect to both mean rate and cross-correlation. In the notation introduced in section 3, we have to compute Pout .p; m; µ; q/, the probability that the coincidence detector produces an output spike (in this time bin) when it receives m input spike trains with cross-correlation q. Conceptually, it is helpful to start with independent input spike trains, q D 0, and then apply the modiŽcation of the result as the procedure described in section 2 is applied to generate input spike trains with Žnite q. Thus, the generation of our correlated input spike trains has two parts: (1) the generation of independent spike trains and (2) the imposition of our cross-correlation procedure to introduce crosscorrelation between the input spike trains.

542

S. Mikula and E. Niebur

Without loss of generality, we treat Pout .p; m; µ; q/ as the following joint probability, again derived from the binomial distribution: m ³ ´ X m j P out .p; m; µ; q/ D p .1 ¡ p/m¡j C1 .j; q/ j jDµ C

µ¡1 ³ ´ X m jD0

j

p j .1 ¡ p/m¡j C2 .j; q/:

(4.1)

In this equation, both C1 .j; q/ and C2 .j; q/ are conditional probabilities; in addition to j and q, they depend on µ and m, but these arguments are suppressed to alleviate the notation. SpeciŽcally, C1 .j; q/ is the probability of obtaining at least µ coincident input spikes given that initially (i.e., before applying our cross-correlation procedure), there were j ¸ µ coincident input spikes. The factor C2 .j; q/ is the probability of obtaining at least µ coincident input spikes given that initially there were j < µ. From their deŽnition, it is obvious that C1 .j; 0/ ´ 1 and C2 .j; 0/ ´ 0, which will later be found to be the case explicitly (see equations 4.2 and 4.4). As it must be, equation 3.2 is thus just the special case of q D 0. Equation 4.1 expresses the joint probabilities for a two-part experiment: the generation of independent spike trains and the correlation procedure applied to these spike trains that is described in section 2. Let us Žrst consider C2 , the conditional probability for a suprathreshold event (that is, among the m input neurons, µ or more have a spike) occurring in the (whole) experiment given that initially (after completion of only the Žrst part) it was subthreshold. This can happen only if the reference spike train has a spike in the time bin under consideration since otherwise, a subthreshold event can never be converted to a suprathreshold event (if the state of any neuron will be switched, it will be switched to that of the reference spike train). This occurs with a probability p.3 Since by assumption j < µ neurons already have a spike (and their state will not be inuenced even if a switch should occur since the reference spike by assumption also has a spike), we have to compute the probability that a sufŽcient number of the remaining .m ¡ j/ neurons that do not spike before application of the correlation procedure will be switched. Since p all switches are independent and occur with a probability q, we can expect i switches among the available .m ¡ j/ neurons with a probability ¡m¡j¢p i p q .1 ¡ q/.m¡j/¡i . This gives rise to a suprathreshold event if i ¸ µ ¡ j i 3

Formally, the procedure could also be viewed as a three-part experiment: (1) the generation of spike trains, (2) the correlation procedure described in section 3 for any reference spike train, and (3) the drawing of either a spike or a nonspike in the reference spike train, with probabilities p and .1 ¡ p/, respectively. We have simpliŽed the analysis somewhat by taking only those parts in equations 4.2 and 4.3 that actually contribute to the output of the coincidence detector.

Coincidence Detector: Analytical Solution

543

since in this case, the total number of spikes is j C i ¸ µ . We therefore have to sum the probabilities for all cases with i ¸ µ ¡ j and obtain C2 .j; q/ in equation 4.1 as C2 .j; q/ D

³

m¡j X iDµ ¡j

´ m¡ j p i p q .1 ¡ q/m¡j¡i p: i

(4.2)

Now we turn to the derivation of C1 .j; q/ in equation 4.1, which is the probability that an initially (i.e., at q D 0) suprathreshold event remains suprathreshold after cross-correlation. This is more difŽcult to compute directly, and therefore we compute its complement, that is, one minus this probability. The latter is the probability of switching a suprathreshold event to a subthreshold event. Assuming now j ¸ µ, this probability is found by an argument completely analogous to that leading to equation 4.2 as ³´ j X j p i p q .1 ¡ q/ j¡i .1 ¡ p/: i iDj¡µ C1

(4.3)

It can be seen that this is the probability that i (with i > j ¡ µ) input spikes get switched to the value in the reference spike train during the generation of correlated spike trains. This expression also takes into account the requirement that the bin in the reference train contains no spike (from which the .1 ¡ p/ term results). Therefore, we have the following: C1 .j; q/ D 1 ¡

³´ j X j p i p q .1 ¡ q/ j¡i .1 ¡ p/: i iDj¡µC1

(4.4)

This completes the computation of equation 4.1 for Pout .q/. To compute the mean output rate of the coincidence detector Nout .p; q; µ/ as a function of input rate p=1t, cross-correlation q, and threshold µ, we collect terms and multiply equation 4.1 by the inverse of the time bin width, 1t, to yield the following: Nout .p; m; µ; q/ 2 0 1 ³´ j m ³ ´ X 1 X m j p p 4 D p j .1¡p/m¡j @1¡ qi .1¡ q/ j¡i .1¡p/A 1t jDµ j i iDj¡µ C1 C

3 ³ ´ m¡j p i p p j .1 ¡ p/m¡j q .1 ¡ q/m¡j¡i 5 : (4.5) j i iDµ¡j

µ¡1 ³ ´ X m jD0

m¡j X

We have thus obtained our main result: the exact analytical solution for the output rate of a coincidence detector receiving an arbitrary number of

544

S. Mikula and E. Niebur

inputs with identical binomial statistics and modulated with respect to both mean rate and cross-correlation. The validity of equation 4.5 is conŽrmed by simulation (not shown). 5 Example An illustration of equation 4.1 for a coincidence detector receiving m D 100 binomial input spike trains and with a threshold µ D 15 is shown in Figure 1a. For low values of cross-correlation, the output rate of the coincidence detector is sigmoidal as a function of the input rate, and it rapidly saturates as the input rate increases. For increasing cross-correlation, this behavior approaches more and more a linear increase with input rate, which is, of course, nothing but the solution in which each synchronous volley of incoming action potentials leads to exactly one output spike. In Figure 1b, we show a cut through Figure 1a for the case of constant probability p D 0:1, corresponding to a Žring rate of 50 Hz for 1t D 2 ms, which is physiologically realistic in many cases. For this low probability, the binomial distribution of input spike trains can be reasonably well approximated by a Poisson distribution. Bernander, Koch, and Usher (1994) emphasized that increasing synchrony does not necessarily lead to an increased output rate since synchronous volleys with more spikes than required to overcome the threshold are wasted. They demonstrated by way of simulation of different neuron models that the output Žring rate as a function of input correlation in this case takes on the shape of an inverted U. Figure 1b shows an example of this behavior in the case of our exact solution. While synchrony in this example always increases the efŽcacy of the incoming Žring rates beyond that obtained for independent spike trains, this increase reaches a maximum for small q, after which the output rate decreases again. For this value of p, the trivial solution in which one input volley generates exactly one output spike (i.e., output rate ´ p) is obtained for all q »> 0:1.

Figure 1: Facing page. The plots illustrate the main equation (4.1) derived in this article for a coincidence detector receiving m D 100 binomial input spike trains with a threshold µ D 15. (a) The probability for a coincidence detector to Žre an action potential per unit time (see equation 4.1). (b) An iso-frequency (p = .1) slice through the surface of a, showing the output probability of the coincidence detector as a function of input cross-correlation. At this relatively low input frequency (corresponding to 50 Hz for 1t D 2 ms), the input spike trains are approximately Poisson. Note the inverted U shape of the plot, whereby higher cross-correlated inputs result in a lower output due to inefŽcient allocation of input spikes.The horizontal dashed line shows p D 0:1.

Coincidence Detector: Analytical Solution

545

546

S. Mikula and E. Niebur

6 Discussion In this article, we have derived analytically the solution for the output Žring rate of an ideal coincidence detector as a function of the input rate and correlation between input spike trains. Our derivation was based on a simple model and it is important to make explicit the key assumptions of our model: (1) the bins in any given input spike train are independent, (2) the coincidence detector “neuron” integrates over a very short time window, and (3) there is no inhibition. The Žrst condition means that there might be additional effects that depend on the structure of the individual spike trains; the second condition makes this model memoryless, which serves to differentiate it from integrator models with long time constants; and the last condition means that the applicability of our model to neurons that receive strong, structured inhibitory input may be limited. All of these conditions are signiŽcant simpliŽcations; for instance, it is well known (for a recent example, see Tiesinga, Fellous, & Sejnowski, 2002) that neurons are not memoryless processes (assumption 1 for incoming spike trains and assumption 2 for the neural processing). We feel that these simpliŽcations are justiŽed in the interest of obtaining an analytical solution. Others have noted that output frequency varies with the correlation of input spike trains in a nontrivial and frequently nonmonotonic way. In particular, the existence of a maximum, as shown in Figure 1b, has been predicted by Bernander et al. (1994), based on simulations. We can compute the location of this maximum from our solution by differentiating equation 4.5 with respect to the cross-correlation q and setting the right-hand side equal to zero. The computation is straightforward but not particularly illuminating. Instead, we plotted iso-frequency curves through the surface given by equation 4.5 to obtain the position of this maximum (not shown). We observed that the prominence of this peak increases with increasing number of inputs and that it is conŽned to low input rates (the Poisson regime), disappearing altogether above a certain input rate where monotonically decreasing iso-frequency curves are obtained. This general behavior can be seen in Figure 1a. These observations lead us to infer that correlations modulate Žring rate best when the number of inputs is high and the input rates are low, which corresponds to the most neurally plausible scenario. Thus, our results may have relevance for synchrony-based models of neural computing. In sum, we believe our derivation will prove useful both as a theoretical example highlighting the power of combinatorial methods and for explaining the relative contributions of rate versus synchrony in neural coding. Acknowledgments This work was supported by the National Science Foundation under a CAREER grant to E.N. (grant 9876271).

Coincidence Detector: Analytical Solution

547

References Abeles, M. (1982). Local cortical circuits. New York: Springer-Verlag. Abeles, M. (1991). Corticonics—Neural circuits of the cerebral cortex. Cambridge: Cambridge University Press. Alonso, J. M., Usrey, W. M., & Reid, R. C. (1996). Precisely correlated Žring in cells of the lateral geniculate nucleus. Nature, 383 (6603), 815–819. Bernander, O., Koch, C., & Usher, M. (1994). The effect of synchronized inputs at the single neuron level. Neural Computation, 6(4), 622–641. Bohte, S. M., Spekreijse, H., & Roelfsema, P. R. (2000). The effects of pair-wise and higher-order correlations on the Žring rate of a postsynaptic neuron. Neural Computation, 12, 153–179. Decharms, R. C., & Merzenich, M. M. (1996). Primary cortical representation of sounds by the coordination of action potential timing. Nature, 381, 610–613. de Oliveira, S. C., Thiele, A., & Hoffmann, K. P. (1997). Synchronization of neuronal activity during stimulus expectation in a direction discrimination task. J. Neurosci., 17(23), 9248–9260. Konig, ¨ P., Engel, A. K., & Singer, W. (1996). Integrator or coincidence detector? The role of the cortical neuron revisited. Trends in Neurosciences, 719(4), 130– 137. Niebur, E., Hsiao, S. S., & Johnson, K. O. (2002). Synchrony: A neuronal mechanism for attentional selection? Current Opinion in Neurobiology,12(2), 190–194. Riehle, A., Grun, ¨ S., Diesmann, M., & Aertsen, A. (1997). Spike synchronization and rate modulation differentially involved in motor cortical function. Science, 278, 1950–1953. Salinas, E., & Sejnowski, T. J. (2000). Impact of correlated synaptic input on output Žring rate and variability in simple neuronal models. Journal of Neuroscience, 20(16), 6193–6209. Shadlen, M. N., & Newsome, W. T. (1994). Noise, neural codes and cortical organization. Current Opinion in Neurobiology, 4, 569–579. Softky, W., & Koch, C. (1992). Cortical cells should Žre regularly but do not. Neural Computation, 4(5), 643–646. Softky, W., & Koch, C. (1993). The highly irregular Žring of cortical cells is inconsistent with temporal integration of random EPSPs. J. Neurosci., 13(1), 334–350. Steinmetz, P. N., Roy, A., Fitzgerald, P., Hsiao, S. S., Johnson, K. O., & Niebur, E. (2000). Attention modulates synchronized neuronal Žring in primate somatosensory cortex. Nature, 404, 187–190. Stroeve, S., & Gielen, S. (2001). Correlation between uncoupled conductancebased integrate-and-Žre neurons due to common and synchronous presynaptic Žring. Neural Computation, 13, 2005–2029. Tiesinga, P. H. E., Fellous, J.-M., & Sejnowski, T. J. (2002). Attractor reliability reveals deterministic structure in neuronal spike trains. Neural Computation, 14, 1629–1650. Williams, S. R., & Stuart, G. J. (2002). Dependence of EPSP efŽcacy on synapse location in neocortical pyramidal neurons. Science, 295, 1907–1910. Received May 2, 2002; accepted July 30, 2002.

NOTE

Communicated by Arjen van Ooyen

A Theoretical Model of Axon Guidance by the Robo Code Geoffrey J. Goodhill [email protected] Department of Neuroscience, Georgetown University Medical Center, Washington, D.C. 20007, U.S.A.

After crossing the midline, different populations of commissural axons in Drosophila target speciŽc longitudinal pathways at different distances from the midline. It has recently been shown that this choice of lateral position is governed by the particular combination of Robo receptors expressed by these axons, presumably in response to a gradient of Slit released by the midline. Here we propose a simple theoretical model of this combinatorial coding scheme. The principal results of the model are that purely quantitative rather than qualitative differences between the different Robo receptors are sufŽcient to account for the effects observed following removal or ectopic expression of speciŽc Robo receptors, and that the steepness of the Slit gradient in vivo must exceed a certain minimum for the results observed experimentally to be consistent. 1 Introduction A crucial process in the construction of a nervous system is the establishment of appropriate connectivity. This consists of two stages: the guidance of axons to target regions largely independent of neural activity, followed by the activity-dependent reŽnement of synaptic connections. Much progress has recently been made in understanding the mechanisms of axon guidance in response to molecular cues in the developing nervous system (TessierLavigne & Goodman, 1996; Mueller, 1999; Wilkinson, 2001; Song & Poo, 2001). An emerging theme is that a limited number of evolutionarily conserved axon guidance molecules are reused in many different contexts. When it is necessary to generate several different outcomes in the same region of space and time, an obvious strategy is combinatorial coding. For instance, two guidance molecules could simultaneously attract three populations of axons to different targets if population 1 expressed only one type of receptor, population 2 expressed only the other type, and population 3 expressed both types. Since axons can also be attracted or repelled depending on their internal state (Song, Ming, & Poo, 1997), for N guidance molecules, the number of separate populations that could in principle simultaneously be guided to different locations is of order 3N . About 100 guidance molecules are now known, allowing many more potential outcomes than there are total Neural Computation 15, 549–564 (2003)

c 2003 Massachusetts Institute of Technology °

550

G. Goodhill

connections in any biological nervous system. Combinatorial coding is thus a very efŽcient coding strategy and is used by nervous systems in contexts ranging from axon guidance (Shirasaki & Pfaff, 2002) to information coding by spike trains (Diesman, Gewaltig, & Aertsen, 1999) and odor coding in the olfactory system (Malnic, Hirono, Sato, & Buck, 1999). A particularly well-studied model system in axon guidance is the growth of commissural axons to, across, and beyond the midline (Kaprielian, Imondi, & Runko, 2000). Commissural axons are initially attracted toward the midline (e.g., the insect nerve cord or the vertebrate spinal cord) by a gradient of Netrin released by midline glia (SeraŽni et al., 1994; Kennedy, SeraŽni, de la Torre, & Tessier-Lavigne, 1994; Mitchell et al., 1996). On reaching the midline, they lose their responsiveness to Netrin but become responsive instead to the repellent molecule Slit, which binds to the Robo family of receptors (Brose et al., 1999; Kidd, Bland, & Goodman, 1999). In Drosophila, this currently consists of the three members Robo, Robo2, and Robo3, each of which plays a particular role in enabling commissural axons to cross the midline and not then recross (Rajagopalan, Nicolas, Vivancos, Berger, & Dickson, 2000; Simpson, Kidd, Bland, & Goodman, 2000). After crossing, axons project to speciŽc lateral pathways before turning longitudinally. Recent results from Drosophila (Rajagopalan, Vivancos, Nicolas, & Dickson, 2000; Simpson, Bland, Fetter, & Goodman, 2000) have focused on three speciŽc longitudinal Fas-II positive tracts, referred to as medial, intermediate, and lateral. In this case, the choice of pathway depends on a combinatorial code of Robo receptors expressed by axons after they have crossed the midline—the so-called Robo code. Axons that normally turn in the medial tract express only Robo, axons that normally turn in the intermediate tract express Robo and Robo3, and axons that normally turn in the lateral tract express Robo, Robo2, and Robo3 (see Figure 1A). Although a gradient of Slit emanating from the midline has not yet been directly observed, it has been hypothesized that this gradient drives axons to speciŽc lateral positions based on their combinatorial code of Robo receptors (Rajagopalan, Vivancos, Nicolas, & Dickson, 2000; Simpson, Bland, Fetter, & Goodman, 2000). This hypothesis is supported by loss- and gain-of-function experiments (Rajagopalan, Vivancos, Nicolas, & Dickson, 2000; Simpson, Bland, Fetter, & Goodman, 2000). When particular Robo receptors are deleted, axons often project to a more medial fascicle than normal, whereas when particular Robo receptors are ectopically expressed, axons often project to a more lateral fascicle than normal. The shifts induced by these genetic perturbations are always discrete jumps between fascicles: these axons do not generally turn between fascicles. Both Rajagopalan, Vivancos, Nicolas, and Dickson (2000) and Simpson, Bland, Fetter, and Goodman, (2000) argue that the Robo code targets axons to a general region; then local cues take over to target a particular tract within that region. Although these different Robo receptors all mediate repulsion by Slit, the speciŽc signal transduction pathways they activate inside the growth

Axon Guidance by the Robo Code

A

551

B M

I

L S(x)

robo robo3 robo robo3 robo2

MIDLINE

robo

Slit gradient S(M) S(I) S(L) x0

M

d

M

d

d

I

d

L

X

Figure 1: (A) Schematic diagram of the Robo code. Three different populations of commissural axons (dashed lines) are shown projecting to three different lateral fascicles: M (medial), I (intermediate), and L (lateral). (B) Parameters of the model. x0 D distance of the medial fascicle from the midline, 2d D spacing of the fascicles. S(M), S(I), and S(L) refer to the Slit concentrations at the medial, intermediate, and lateral fascicles, respectively.

cone may be different (discussed in Rajagopalan, Vivancos, Nicolas, & Dickson, 2000). One possibility is that each type of Robo receptor signals qualitatively different information regarding levels of Slit, which would be a combinatorial code in the sense most analogous to the other examples cited above. However, an alternative possibility is that the differences between types of Robo receptors are simply quantitative; each generates the same type of repulsive signal to particular levels of Slit, differing only in degree. Roughly speaking, this is consistent with the experimental results that the more Robo receptors an axon expresses, the more repulsion and thus the further its lateral projection. By analogy with levels of measurement in statistics (Krzanowski, 1988), we refer to the Žrst possibility as a categorical model and the second as a ratio model. In the categorical model, only the categories of Robo receptors expressed by a given axon determine where it projects to, whereas in the ratio model, only the relative proportions matter. The categorical model is consistent with many of the experimental results. For instance, overexpressing Robo does not generally cause a lateral shift (Simpson, Bland, Fetter, & Goodman, 2000). However, a purely categorical model is insufŽcient to explain some results: for instance, axons that normally project laterally still do so in the robo=robo3 double mutant (Simpson, Bland, Fetter, & Goodman, 2000), and medially projecting axons that project to the intermediate zone when robo3 is added can be “supershifted” to the lateral zone by the addition of two copies of the robo3 gene (Rajagopalan, Vivancos, Nicolas, & Dickson, 2000). Results such as these led both Simpson, Bland, Fetter, and Goodman (2000) and Rajagopalan, Vi-

552

G. Goodhill

vancos, Nicolas, and Dickson (2000) to propose a mixed categorical-ratio model, where Robo acts as a category (only the presence of Robo is important, not its precise amount) and Robo2 and Robo3 act in a more graded way. To explore these coding issues in a more quantitative way, we have developed a simple mathematical model of the Robo code based on a pure ratio and a mixed categorical-ratio approach. We show that a pure ratio model could in fact be consistent with the data if the Slit gradient is steep enough. This leads to experimentally testable predictions regarding how the outcome of Robo overexpression experiments might depend on the steepness of the Slit gradient. In addition, for the mixed categorical-ratio model, we show that even this version is consistent with the data only when the Slit gradient is sufŽciently steep. 2 Mathematical Model 2.1 Form of the Slit Gradient. There is no direct evidence concerning the shape of the Slit gradient in vivo. Theoretically, one approach would be to model it as free diffusion from a line source in an inŽnite three-dimensional volume (Crank, 1975; Carslaw & Jaeger, 1959), analogous to a recent model of axon guidance by a target-derived diffusible factor (Goodhill, 1997, 1998). However, data from other systems suggest the diffusion might not be free (Hiramoto, Hiromi, Giniger, & Hotta, 2000). Given this uncertainty, it seems sensible to choose a very generic shape with a parameterized steepness for the Slit gradient. We therefore assume it to be a decaying exponential, S.x/ D Ce¡x=® , where C gives the concentration of Slit at the midline, x is distance from the midline, and ® gives the rate of decrease moving away from the midline. For small ®, Slit is tightly concentrated around the midline, whereas for larger ®, it decays more gradually. 2.2 Generation of Repulsion: Linear Model. In the linear version of the model, we assume that the amount of repulsion induced in growth cones by Slit is given by S.x/ £ R, where R is just a sum of the amount of repulsion transduced by each Robo receptor separately (Nakamoto et al., 1996; Goodhill & Richards, 1999). That is, R D r for axons normally projecting to the medial fascicle, R D r C r3 for intermediate axons, and R D r C r3 C r2 for lateral axons, where r, r3 , and r2 give the “net” amount of repulsion due to each Robo receptor (for the mixed categorical-ratio model, R is assumed to depend on only r2 and r3 ). The values of the r’s that are consistent with the data come out of the model. For experiments where a second copy of a gene is inserted, this net repulsion is assumed to be doubled. That is, the net repulsion is assumed to be equal to the product of the number of receptors times the strength of repulsive signaling through a single receptor of that type. For Robo overexpression experiments, where the degree of ampliŽcation of r is uncertain, we introduce a multiplication

Axon Guidance by the Robo Code

553

parameter n. The values of n that are consistent with the data emerge from the model. Each axon is assumed to be repelled by Slit until the amount of repulsion S.x/£R reaches a certain threshold T that is the same for all axons. Axons are then assumed to be attracted by local cues toward the nearest of the medial, intermediate, or lateral fascicles. The model presented here concerns only how axons target a particular general area based on their Robo code and does not explicitly address how local cues reŽne their positions within these general areas (we return to this in section 4). This means that the positions that axons reach deŽned purely by Slit repulsion can be anywhere in the region of a particular fascicle, provided they are nearer to that fascicle than neighboring fascicles. Fascicles are assumed to be evenly spaced, distance 2d apart, with the medial fascicle distance x0 from the midline (see Figure 1B). We also assume that for axons to project to the medial fascicle, their turning position must be more than x0 ¡ d from the midline, and for axons to project to the lateral fascicle, their turning position must be less than x0 C 5d from the midline. Based on data such as Figure 5A of Simpson, Bland, Fetter, and Goodman (2000), we chose x0 D 4, d D 1 (in arbitrary units), so that the three fascicles run at distances 4, 6, and 8 from the midline (absolute distances are not indicated in Simpson, Bland, Fetter, & Goodman, 2000, or Rajagopalan, Vivancos, Nicolas, & Dickson, 2000). These units set the scale for ®. The constraints imposed by the experimental data can now be expressed as a set of inequalities that must all be simultaneously satisŽed. As an example, consider the normal targeting of axons projecting to the intermediate fascicle. The equation for targeting to position x is S.x/.r C r3 / D T:

(2.1)

Substituting the exponential form for S gives e¡x=® .r C r3 / D T=C: The ratio T=C sets the absolute scale for the r’s. However, since we are interested only in the relative values of the r’s, we can without loss of generality set T=C D 1. This yields a value for x of x D ® log.r C r3 /:

(2.2)

This target position must be closer to the intermediate tract than either the medial or lateral tracts. Thus we require (cf. Figure 1) that x0 C d < ® log.r C r3 / < x0 C 3d or equivalently e

x0 Cd ®

< r C r3 < e

x0 C3d ®

:

554

G. Goodhill

Table 1 shows a set of inequalities that can be derived by this approach from the experimental data of Rajagopalan, Vivancos, Nicolas, and Dickson (2000) and Simpson, Bland, Fetter, and Goodman (2000) (see particularly the schematic Figure 4 in Simpson, Bland, Fetter, & Goodman, 2000). Cases labeled “crossing defect” are assumed to represent problems with crossing rather than lateral termination zone and are not considered further in our analysis. In case 9 in Table 1 (axons that normally project to the lateral fascicle in the robo2 mutant), the axons seem undecided between the intermediate and lateral fascicles. This was not included in our analysis, and we return to this issue in section 4. Not all of these inequalities are independent. For instance, if ® log.r3 / > x0 C d (index 5 in Table 1), then it is certainly true that ® log.r C r3 / > x0 C d (index 2 in Table 1), and the latter constraint is superuous. However, we explicitly include all the constraints in Table 1 since it makes it easier to see which piece of datum each is based on and what would happen if each were deleted or altered. 2.3 Generation of Repulsion: Nonlinear Model. A more realistic picture of binding kinetics than the linear version presented above is [RL] D

[R][L] ; [L] C Kd

where [R] is the receptor concentration, [L] is the ligand concentration, [RL] is the concentration of the product, and Kd is the dissociation constant (Gutfreund, 1995). When [L] ¿ Kd , this simpliŽes to the linear version (with Kd absorbed into the threshold T). The three Robo receptors have similar Kd s for binding Slit, in the range 10 to 40 nM (Simpson, Kidd, Bland, & Goodman, 2000), and we consider these as equal. The analog of equation 2.1 is then S.x/ .r C r3 / D T: S.x/ C Kd Substituting the exponential form for S.x/ and dividing by C gives e¡x=® .r C r3 / D T; C 1=Nc

e¡x=®

(2.3)

where cN D C=Kd , the normalized Slit concentration at the midline. Again, T simply sets the overall scale of the r’s, so we can set it to 1 without loss of generality. The analog to equation 2.2 is now x D ® log[Nc.r C r3 ¡ 1/]; and an analogous set of constraints to Table 1 can be generated (not shown).

WT WT WT r¡ r¡ r¡ r2 ¡ r2 ¡ r2 ¡ r3 ¡ r3 ¡ r3 ¡ r¡, r2 ¡ r¡, r2 ¡ r¡, r2 ¡ r¡, r3 ¡ r¡, r3 ¡ r¡, r3 ¡ r2 ¡, r3 ¡ r2 ¡, r3 ¡ r2 ¡, r3 ¡ nr nr nr r2 C r3 C r2 C, r3 C 2r3 C 2r2 C, r3 C

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29

Medial Intermediate Lateral Medial Intermediate Lateral Medial Intermediate Lateral Medial Intermediate Lateral Medial Intermediate Lateral Medial Intermediate Lateral Medial Intermediate Lateral Medial Intermediate Lateral Medial Medial Medial Medial Medial

Population

r2 r r r nr nr, r3 nr, r3 , r2 r, r2 r, r3 r, r3 , r2 r, 2r3 r, r3 , 2r2 < < < < < < < < < < < <

< < <

x0 ¡ d x0 ¡ d x0 C 3d

x0 C 3d x0 ¡ ¡d x0 ¡ ¡d x0 ¡ d x0 ¡ d x0 C d x0 C 3d x0 C 3d x0 C d x0 C 3d x0 C 3d x0 C 3d

< < < <

x0 C d x0 C 3d x0 ¡ d x0 C d

r3 r2 , r3 r r, r3 r, r3 r r r, r2 r3 r3

< < <

x0 ¡ d x0 C d x0 C 3d

r r, r3 r, r3 , r2

Robo Code ® log.r/ ® log.r C r3 / ® log.r C r2 C r3 / Crossing defect ® log.r3 / ® log.r2 C r3 / ®log.r/ ® log.r C r3 / See text ® log.r/ ® log.r/ ® log.r C r2 / Crossing defect Crossing defect Crossing defect Crossing defect Crossing defect ® log.r2 / ® log.r/ ® log.r/ ® log.r/ ® log.nr/ ® log.nr C r3 / ® log.nr C r3 C r2 / ® log.r C r2 / ® log.r C r3 / ® log.r C r3 C r2 / ® log.r C 2r3 / ® log.r C r3 C 2r2 /

Equation

< < < < < < < < < < < <

< < <

< < < <

< < <

x0 C 5d x0 C d x0 C d x0 C d x0 C d x0 C 3d x0 C 5d x0 C 5d x0 C 3d x0 C 5d x0 C 5d x0 C 5d

x0 C d x0 C d x0 C 5d

x0 C 3d x0 C 5d x0 C d x0 C 3d

x0 C d x0 C 3d x0 C 5d

Notes: r¡ indicates null mutant for Robo and so on; r2 C indicates ectopic expression for Robo2 and so on; 2r2 indicates two copies of Robo2. “Population” identiŽes the axons involved in each case by where they would normally project to in the wild type (WT).

Mutant

Index

Table 1: Complete Set of Constraints for the Linear Model Derived from Wild Type and Mutants.

Axon Guidance by the Robo Code 555

556

G. Goodhill

3 Results 3.1 Linear Model. In the mixed categorical-ratio model, it is assumed that only r2 and r3 act in a quantitative way: the set of constraints used is as in Table 1, except that r is ignored (i.e., set to zero). For instance, constraint 2 now becomes x0 C d < ® log.r3 / < x0 C 3d (or e.x0 Cd/=® < r3 < e.x0 C3d/=® ). Since there are now only two variables, r2 and r3 , each constraint can be represented as a straight line in the r2 , r3 plane. Figure 2 shows how the constraints conspire to limit the allowable combinations of r2 and r3 that satisfy the inequalities. This is illustrated for two gradient steepnesses, ® D 1 and ® D 2. It is immediately obvious from Figure 2 that for ® & 2, there can be no more solutions. For the pure ratio model, there are the additional parameters r and n, giving four dimensions in total, which is hard to represent graphically. To investigate which sets f®; r; r2 ; r3 g satisfy all the inequalities, an exhaustive search was performed over the parameter space ri 2 [0; rmax ] in increments rinc for various values of ®. For certain values of ®, we found many different combinations of r, r3 , and r2 , which satisfy the constraints. The precise number of solutions per value of ® depends on rinc , which was varied for different ®. The total number of solutions is not important here, providing solutions exist, but we ensured that there were always at least 100 so that averaging over them would be meaningful. Since only the ratios of the r’s are of interest, they were normalized by r3 . The results are shown in Table 2. Means and standard deviations for the r’s are quoted: the standard deviations give a measure of the size of the box in the four-dimensional space in which the solutions exist. Several trends are apparent: ² As for the mixed categorical-ratio model, there are no solutions for ® & 2. ² The values of the r’s (i.e., the relative strength of signaling of the different Robo receptors) are always ordered r2 > r3 > r. ² The ratios of the r values become more extreme as ® decreases (i.e., the gradient becomes steeper). For instance, for n D 2, the ratios r:r3 :r2 go from 0.4:1:2.2 for ® D 2:0 to 0.007:1:20.1 for ® D 0:5.

² For a very steep gradient (® D 0:5), values of n (the ampliŽcation factor for Robo overexpression experiments) up to 25 are allowable. As ® decreases further, even larger values of n are allowed (results not shown). As the gradient becomes shallower, n becomes more restricted, until Žnally for ® D 2, solutions exist only for n D 2. 3.2 Nonlinear Model. Comparing equations 2.2 and 2.3, it can be seen that the nonlinear model with S.x/ D e¡x=® is formally equivalent to a linear e¡x=® model with S.x/ D e¡x=® . Thus, we may think of the nonlinear version C1=Nc

Axon Guidance by the Robo Code

557 120

10000

100

8000

Robo2

Robo2

80 6000

4000

40

2000

0 0

60

20

2000

4000

6000

Robo3

8000

0 0

10000

20

40

20

40

20

40

20

40

60

80

100

120

60

80

100

120

60

80

100

120

60

80

100

120

Robo3

120

10000

100

8000

Robo2

Robo2

80 6000

4000

40

2000

0 0

60

20

2000

4000

6000

Robo3

8000

0 0

10000

Robo3

120

10000

100

8000

Robo2

Robo2

80 6000

4000

40

2000

0 0

60

20

2000

4000

6000

Robo3

8000

0 0

10000

Robo3

120

10000

100

8000

Robo2

Robo2

80 6000

4000

40

2000

0 0

60

20

2000

4000

6000

Robo3

8000

10000

0 0

Robo3

Figure 2: Interaction of constraints for the mixed categorical-ratio model (linear version). (Left) ® D 1:0. (Right) ® D 2:0 (x0 D 4, d D 1). The shaded box in each case represents the region of r2 –r3 space allowed by the constraints. First row: cases 2 and 12 from Table 1. Second row: case 28 is added. Third row: case 6 is added. Fourth row: case 29 is added. It can be seen that the allowable region of space shrinks as ® increases (the Slit gradient becomes shallower).

558

G. Goodhill

Table 2: Values of r, r2 , and r3 Obtained for the Pure Ratio Model (Linear Version). n

® D 0:5

® D 1:0

® D 2:0

2

0:0067 § 0:0038 20 § 11 0:0047 § 0:0026 20 § 11 0:0035 § 0:0018 20 § 11 0:0030 § 0:0015 20 § 11 0:0025 § 0:0011 20 § 11 0:0021 § 0:0008 20 § 11 0:0017 § 0:0007 20 § 11 0:001 § 0:0002 20 § 11 No solutions

0:065 § 0:026 3:3 § 1:2 0:048 § 0:015 3:3 § 1:2 0:039 § 0:010 3:3 § 1:2 0:035 § 0:007 3:3 § 1:2 0:031 § 0:005 3:3 § 1:2 No solutions

0:35 § 0:04 2:2 § 0:2 No solutions

No solutions

No solutions

No solutions

No solutions

No solutions

No solutions

3 4 5 6 7 10 25 30

No solutions No solutions No solutions No solutions

Notes: n is the increase in repulsion compared to wild type in Robo ectopic expression experiments; ® is the steepness of the Slit gradient. Shown are the mean and standard deviation of the values obtained for r (above) and r2 (below) normalized by r3 (i.e., r3 D 1).

as simply the linear model with a different Slit gradient. In comparing different gradients, the overall scaling is unimportant since in the model, we are interested only in the ratio of the r’s. Thus, signiŽcant differences are expected only between the results for the linear and nonlinear models when the effective gradients have signiŽcantly different shapes in the region between x D 4 and x D 8. For the range of ® considered above, this occurs only when cN À 1 (see Figure 3). Table 3 shows results for the pure ratio model for cN D 100, that is, a midline Slit concentration 100Kd (® D 1). This implies Slit concentrations at the medial, intermediate, and lateral fascicles of 65Kd , 20Kd , and 3Kd , respectively (650 nM, 200 nM, and 30 nM for Kd D 10 nM). There are slight differences from the linear model, but overall the general trends are the same. The numerical differences are smaller for smaller cN (data not shown). 4 Discussion The model presented here is a way of simplifying a varied and complex set of experimental results so as to attempt to extract certain key pieces of quantitative information about the system. A number of biologically relevant conclusions can be drawn. Perhaps the most important of these is that a pure ratio model can be consistent with the data, despite the fact

Axon Guidance by the Robo Code

559

A

B 1

1

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

0 0

2

4

6

8

10

0 0

2

4

6

8

10

Figure 3: Comparison of an exponential Slit gradient with the effective gradient shape in the nonlinear model. The horizontal axis is distance from the midline; the vertical axis is Slit concentration (arbitrary units). (A) From left to right, effective gradient in nonlinear model for cN D 1, exponential gradient, effective gradient in nonlinear model for cN D 10, 100. (B) Same gradients, but normalized so that the Slit concentration at distance 4 is always 1. From left to right, exponential gradient, effective gradient in nonlinear model for cN D 1 (indistinguishable from exponential), cN D 10, 100. Note that the effective gradient shape diverges signiŽcantly from an exponential only when cN À 1. For all curves, ® D 1.

that Simpson, Bland, Fetter, and Goodman (2000) and Rajagopalan, Vivancos, Nicolas, and Dickson (2000) did not generally Žnd lateral shifts when Robo was overexpressed. The model predicts that arbitrarily large degrees of overexpression of Robo may indeed not cause shifts in a ratio model, provided the Slit gradient is sufŽciently steep. The reason for this is actually quite intuitive. The ratio of Slit concentration at the intermediate compared ¡6=® to medial fascicles in the linear model is ee¡4=® D e¡2=® . As ® becomes smaller, this ratio becomes smaller; that is, for steeper gradients, there is a larger disparity between the concentrations at the intermediate and medial fascicles. Since in the linear model the repulsion due to Robo induced by Slit is given by r £ S.x/, a larger Slit disparity means that a larger increase in Robo expression is possible before a shift to the intermediate fascicle is induced. Thus, by precisely measuring the actual level of Robo in overexpression experiments where no shifts occur, it should be possible to predict based on the model a minimum steepness for the Slit gradient in vivo. Similarly, the pure ratio model predicts that there should be a certain level of overexpression of Robo at which lateral shifts can be made to occur. Interestingly, Simpson, Bland, Fetter, and Goodman (2000) did in fact Žnd one GAL4 line (15J2) where increased Robo can cause a lateral shift of the dMP2 and vMP2 axons. A possible explanation suggested by the model is that it is only in this line that Robo levels exceed that required for a lateral shift. A robust result of both the pure ratio and mixed categorical-ratio model is that the experimental data are consistent only when the Slit gradient is

560

G. Goodhill

Table 3: Values of r, r2 , and r3 Obtained for the Pure Ratio Model (Nonlinear Version). n

® D 0:5

® D 1:0

® D 2:0

2

0:0067 § 0:0038 20 § 11 0:0046 § 0:0025 20 § 11 0:0036 § 0:0019 20 § 11 0:0030 § 0:0015 20 § 11 0:0025 § 0:0012 20 § 11 0:0022 § 0:0010 20 § 11 0:0016 § 0:0007 20 § 11 0:00083 § 0:00016 20 § 11 0:00083 § 0:00016 20 § 11 No solutions

0:17 § 0:028 3:4 § 1:2 No solutions

No solutions

No solutions

No solutions

No solutions

No solutions

No solutions

No solutions

No solutions

No solutions

No solutions

No solutions

No solutions

No solutions

No solutions

No solutions

No solutions

No solutions

3 4 5 6 7 10 25 30 35

No solutions

Notes: n is the increase in repulsion compared to wild type in Robo ectopic expression experiments; ® is the steepness of the Slit gradient. Shown are the mean and standard deviation of the values obtained for r (above) and r2 (below) normalized by r3 (i.e., r3 D 1).

sufŽciently steep. For the linear model, the cut-off is about ® & 2. For the nonlinear model, the cut-off depends on cN, the midline concentration compared to Kd . When cN . 1, the cut-off is similar to the linear model; for higher values of cN, a slightly steeper gradient is required. According to the linear model, the shallowest allowable Slit gradient consistent with the data has a concentration at the medial fascicle (position x0 D 4) of approximately e¡4=2 D 0:14 times its concentration at the midline, and corresponding values 0:05 and 0:03 at the intermediate and lateral fascicles, respectively. The exact numbers are dependent on the assumption of an exponential gradient shape; while there is no direct experimental evidence for this shape, there is no direct evidence for any other shape either. The exponential function provides a convenient way of exploring gradients varying from very steep (small ®) to very shallow (large ®) and has been used to model the gradient in previous theoretical analyses of axon guidance such as Gierer (1987). When experimental data about Slit gradients in vivo become available, these could be used in the model instead. Although changing the gradient function will likely change the precise numerical results, it is unlikely to have a strong effect on the general conclusions of the model.

Axon Guidance by the Robo Code

561

What implications does this work have for our understanding of the coding strategies employed in axon guidance? In a strict sense, the pure ratio model is not a combinatorial code, since the receptors are interchangeable with appropriate scaling. In this case, why have three receptors when in principle one would do? One possibility is that more than one receptor type is already required by commissural axons for midline crossing (Rajagolapalan, Vivancos, Nicolas, & Dickson, 2000; Simpson, Bland, Fetter, & Goodman, 2000), and since this diversity is available, it is simply reused. However, an alternative possibility is that it might not be possible to robustly achieve the widely but precisely different levels of expression that would be required of a single receptor. From Table 2, for ® D 0:5, the relative levels of expression needed for the putative single receptor would be 0:0067 for axons projecting to the medial tract, 1:0067 for axons projecting to the intermediate tract, and 21:0067 for axons projecting to the lateral tract. These differ by a factor of about 3000. Since the number of receptors expressed on the surface of a growth cone is probably no more than about 10,000, the number of receptors present on the growth cones of medially projecting axons would need to be about 3. Even if it was possible to achieve this precisely, the response of these axons to a Slit gradient would then be overwhelmed by noise (Berg & Purcell, 1977; Goodhill & Urbach, 1999). While it is true that for the shallowest gradients allowable by the model, these demands are more reasonable (e.g., for ® D 2:0, the difference in expression required would be about a factor of ¼ 10), using multiple receptors with different strengths of signaling is clearly a more robust strategy overall. The model presented here does not consider the precise timing of robo expression, that is, the way the expression of each robo receptor varies as axons traverse the midline and then project laterally. It also does not consider how local cues reŽne axonal targeting within a general region. One hypothesis is that each speciŽc Fas II tract releases an attractive factor, and it is the balance of this attractive force with the Slit repulsion that guides axons to a speciŽc tract. To include such a mechanism in the model would, however, require introducing additional parameters whose values are not currently constrained by experimental data. Also, there is a general problem with this hypothesis in that such an attractive force would necessarily decay symmetrically on each side of the Fas II tract, whereas the Slit repulsion is asymmetric on each side of the tract. The current model instead remains agnostic about the nature of this local focusing mechanism. The model also does not take into account the fact that many of the mutant and misexpression results listed in Table 1 are in reality just statistical tendencies and do not occur in 100% of cases (this is particularly true in case 9). A possible extension of the model would include probabilistic rules, for instance, that there is noise in the measurement of the level of Slit present (Berg & Purcell, 1977; Goodhill & Urbach 1999). Although we have shown that qualitative differences between robo receptors are not required to account for the lateral targeting of commis-

562

G. Goodhill

sural axons in Drosophila, it does not rule out the possibility that qualitative differences play a role. There are several other indications that qualitative differences between Robo and Robo2=Robo3 do exist (Simpson, Kidd, Bland, & Goodman, 2000). Robo2 and Robo3 are positioned close by chromosomally and have closely related sequences. Their intracellular capabilities are different from Robo; for instance, unlike Robo, they lack an Enabled binding site. The different Robos certainly play qualitatively different roles in the midline crossing of commissural axons (Simpson, Kidd, Bland, & Goodman, 2000) and in the growth and guidance of motor neuron dendrites (Godenschwege et al., 2002). Rajagopalan, Vivancos, Nicolas, and Dickson (2000) report that experiments are now under way to generate chimeric Robo receptors, where, for example, the extracellular domain of Robo is coupled to the intracellular domain of Robo2. This will give information about the relative importance of the different signal transduction pathways that Robo and Robo2=Robo3 activate. The model we have presented so far combines extra- and intracellular effects into a single effective strength of signaling. An extension would be to split each r value into the product re £ ri , representing extra- and intracellular inuences, respectively. When available, the chimeric receptor data could then be used to constrain the additional three parameters and determine whether there are any sets of values that are still consistent with a purely quantitative model. Acknowledgments I thank the anonymous referees for many helpful suggestions for improving this article. References Berg, H. C., & Purcell, E. M. (1977). Physics of chemoreception. Biophysical Journal, 20, 193–219. Brose, K., Bland, K. S., Wang, K. H., Arnott, D., Henzel, W., Goodman, C. S., Tessier-Lavigne, M., & Kidd, T. (1999). Slit proteins bind robo receptors and have an evolutionarily conserved role in repulsive axon guidance. Cell, 96, 795–806. Carslaw, H. S., & Jaeger, J. C. (1959). Conduction of heat in solids (2nd ed.). Oxford: Clarendon Press. Crank, J. (1975). The mathematics of diffusion (2nd ed.). Oxford: Clarendon Press. Diesmann, M., Gewaltig, M. O., & Aertsen, A. (1999). Stable propagation of synchronous spiking in cortical neural networks. Nature, 402, 529–533. Gierer, A. (1987). Directional cues for growing axons forming the retinotectal projection. Development, 101, 479–489. Godenschwege, T. A., Simpson, J. H., Shan, Y., Bashaw, G. J., Goodman, C. S., & Murphey, R. K. (2002). Ectopic expression in the giant Žber system of

Axon Guidance by the Robo Code

563

Drosophila reveals distinct roles for Rounabout (Robo) Robo2, and Robo3 in dendritic guidance and synaptic connectivity. J. Neurosci., 22, 3117–3129. Goodhill, G. J. (1997). Diffusion in axon guidance. European Journal of Neuroscience, 9, 1414–1421. Goodhill, G. J. (1998). Mathematical guidance for axons. Trends in Neurosciences, 21, 226–231. Goodhill, G. J., & Richards, L. J. (1999). Retinotectal maps: Molecules, models, and misplaced data. Trends in Neurosciences, 22, 529–534. Goodhill, G. J., & Urbach, J. S. (1999). Theoretical analysis of gradient detection by growth cones. Journal of Neurobiology, 41, 230–241. Gutfreund, H. (1995). Kinetics for the life sciences. Cambridge: Cambridge University Press. Hiramoto, M., Hiromi, Y., Giniger, E., & Hotta, Y. (2000). The Drosophila Netrin receptor Frazzled guides axons by controlling Netrin distribution. Nature, 406, 886–889. Kaprielian, Z., Imondi, R., & Runko, E. (2000). Axon guidance at the midline of the developing CNS. Anat. Rec., 261, 176–197. Kennedy, T. E., SeraŽni, T., de la Torre, J. R., & Tessier-Lavigne, M. (1994). Netrins are diffusible chemotropic factors for commissural axons in the embryonic spinal cord. Cell, 78, 425–435. Kidd, T., Bland, K. S., & Goodman, C. S. (1999). Slit is the midline repellent for the robo receptor in Drosophila. Cell, 96, 785–794. Krzanowski, W. J. (1988). Principles of multivariate analysis: A user’s perspective. New York: Oxford University Press. Malnic, B., Hirono, J., Sato, T., & Buck, L. B. (1999). Combinatorial receptor codes for odors. Cell, 96, 713–723. Mitchell, K. J., Doyle, J. L., SeraŽni, T., Kennedy, T. E., Tessier-Lavigne, M., Goodman, C. S., & Dickson, B. J. (1996). Genetic analysis of Netrin genes in Drosophila: Netrins guide CNS commissural axons and peripheral motor axons. Neuron, 17, 203–215. Mueller, B. K. (1999). Growth cone guidance: First steps towards a deeper understanding. Annu. Rev. Neurosci., 22, 351–388. Nakamoto, M., Cheng H. J., Friedman, G. C., Mclaughlin, T., Hansen, M. J., Yoon, C. H., O’Leary, D. D. M., & Flanagan, J. G. (1996). Topographically speciŽc effects of ELF–1 on retinal axon guidance in–vitro and retinal axon mapping in–vivo. Cell, 86, 755–766. Rajagopalan, S., Nicolas, E., Vivancos, V., Berger, J., & Dickson, B. J. (2000). Crossing the midline: Roles and regulation of Robo receptors. Neuron, 28, 767–777. Rajagopalan, S., Vivancos, V., Nicolas, E., & Dickson, B. J. (2000). Selecting a longitudinal pathway: Robo receptors specify the lateral position of axons in the Drosophila CNS. Cell, 103, 1033–1045. SeraŽni, T., Kennedy, T. E., Galko, M. J., Mirzayan, C., Jessell, T. M., & TessierLavigne, M. (1994). The netrins deŽne a family of axon outgrowth–promoting proteins homologous to C. elegans UNC–6. Cell, 78, 409–424. Shirasaki, R., & Pfaff, S. L. (2002). Transcriptional codes and the control of neuronal identity. Annu. Rev. Neurosci., 25, 251–281.

564

G. Goodhill

Simpson, J. H., Kidd, T., Bland, K. S., & Goodman, C. S. (2000). Short-range and long-range guidance by slit and its Robo receptors: Robo and Robo2 play distinct roles in midline guidance. Neuron, 28, 753–766. Simpson, J. H., Bland, K. S., Fetter, R. D., & Goodman, C. S. (2000). Short range and long range guidance by Slit and its Robo receptors: A combinatorial code of Robo Receptors controls lateral position. Cell, 103, 1019–1032. Song, H., Ming, G. L., & Poo, M.-M. (1997). CAMP-induced switching in turning direction of nerve growth cones. Nature, 388, 275–279. Song, H., & Poo, M.-M. (2001). The cell biology of neuronal navigation. Nat. Cell. Biol., 3, E81–88. Tessier-Lavigne, M., & Goodman, C. S. (1996). The molecular biology of axon guidance. Science, 274, 1123–1133. Wilkinson, D. G. (2001). Multiple roles of Eph receptors and ephrins in neural development. Nat. Rev. Neurosci., 2, 155–164.

Received April 25, 2002; accepted August 5, 2002.

LETTER

Communicated by Misha Tsodyks

Spike-Driven Synaptic Dynamics Generating Working Memory States Daniel J. Amit [email protected]

Gianluigi Mongillo [email protected] Dipartimento di Fisica, Universita’ di Roma, La Sapienza, 00185 Rome, Italy, and Racah Institute of Physics, Hebrew University, Jerusalem, Israel

The collective behavior of a network, modeling a cortical module of spiking neurons connected by plastic synapses is studied. A detailed spikedriven synaptic dynamics is simulated in a large network of spiking neurons, implementing the full double dynamics of neurons and synapses. The repeated presentation of a set of external stimuli is shown to structure the network to the point of sustaining working memory (selective delay activity). When the synaptic dynamics is analyzed as a function of pre- and postsynaptic spike rates in functionally deŽned populations, it reveals a novel variation of the Hebbian plasticity paradigm: in any functional set of synapses between pairs of neurons (e.g., stimulated– stimulated, stimulated–delay, stimulated–spontaneous), there is a Žnite probability of potentiation as well as of depression. This leads to a saturation of potentiation or depression at the level of the ratio of the two probabilities. When one of the two probabilities is very high relative to the other, the familiar Hebbian mechanism is recovered. But where correlated working memory is formed, it prevents overlearning. Constraints relevant to the stability of the acquired synaptic structure and the regimes of global activity allowing for structuring are expressed in terms of the parameters describing the single-synapse dynamics. The synaptic dynamics is discussed in the light of experiments observing precise spike timing effects and related issues of biological plausibility. 1 Introduction Long-term modiŽcations of the synaptic efŽcacies are believed to affect information processing in the brain. The occurrence of such modiŽcations would be manifest as the appearance of new patterns of neural activity. In both IT and PF cortex of monkeys, trained to perform a delayed-matchto-sample (DMS) task, small neural assemblies have been found to exhibit selective, persistent enhanced spike rates during the delay interval between successive stimulations (Miyashita, 1988; Miller, Erickson, & DesNeural Computation 15, 565–596 (2003)

c 2003 Massachusetts Institute of Technology °

566

D. Amit and G. Mongillo

imone, 1996). This kind of activity is related to the ability of the monkey to hold an item in memory (Amit, 1995); indeed, it reects the last stimulus seen. It will be referred to as working memory (WM) activity. WM appears only after a substantial training stage, during which, presumably, the local synaptic structure is modiŽed by the incoming stimuli.1 Then new patterns of neural activity appear. Spiking neural network models indicate that the phenomenology observed in DMS tasks can be reproduced by coherent modiŽcations of the synaptic efŽcacies (Amit & Brunel, 1997a; Amit, 1998). SpeciŽcally, the mean excitatory synaptic efŽcacy among neurons belonging to the selective population must increase, while the efŽcacies in synaptic population connecting selective to other neurons must decrease. Hence, one expects that the neural activity evoked by presentation of stimuli during the learning stage produces such modiŽcations so that WM can appear. This raises the question of how the synaptic dynamics is related to the neural activity. A simulation like the one presented here has multiple utility. First, it calls into focus certain features of the single synapse that should be clariŽed in biological investigation. Second, it provides a more complete benchmark (relative to simulations with Žxed synapses), in the bridge between experiment and theory (Amit, 1998). And once the simulation clariŽes the essential features of the microscopic synaptic dynamics, simpliŽed (and hence faster) learning algorithms can be identiŽed, to be used in simulating complex behavioral situations. We Žrst review the experimental protocols used to induce long-term synaptic modiŽcations. Then we recapitulate a general model of spikedriven synaptic plasticity, proposed by Fusi, Annunziato, Badoni, Salamon, and Amit (2000) and discuss the extent to which this dynamics satisŽes general desiderata and concords with experimental Žndings. The plastic synapse is then embedded in a full-scale simulation of a large network of spiking neurons. The synaptic dynamics is propelled purely by the actual spikes emitted by the neurons, as a consequence of a preassigned protocol of stimulus presentation, mimicking those used in the DMS tasks. It is shown that WM is actually formed in the process and that its slow formation can be qualitatively understood. 2 Experimental Protocols for LTP and LTD Long-term potentiation (LTP) and long-term depression (LTD) are persistent changes in synaptic efŽcacy that are expected to be induced by patterns of pre- and postsynaptic activity. Hebbian tradition had these changes re-

1 There are zones in cortex where the synaptic structure giving rise to delay activity may be built in (Kritzer & Goldman-Rakic, 1995; Compte, Brunel, Goldman-Rakic, & Wang, 2000). Those are outside the scope of our discussion.

Spike-Driven Synaptic Dynamics Generating Working Memory States

567

lated to pre- and postsynaptic spike rates. A bridge between spikes and plasticity has been recently conŽrmed by detailed experimental studies. SpeciŽc, well-controlled stimulation protocols have been developed that reliably induce LTP and LTD in hippocampal and cortical synapses. However, certain questions remain open. Most important, we consider the passage from spike-induced synaptic plasticity to rate-dependent plasticity, so essential for the learning of working memory. The difŽculty in providing such a bridge, from spikes to rates, opens up questions concerning the dependence of the effects observed on the speciŽc details of the protocols used (see section 7.1). This calls for a closer identiŽcation of the neural activity parameters that control the synaptic transitions and determine the type of modiŽcation that occurs: LTP or LTD. Some protocols for the induction of LTP and LTD involve repetitive presynaptic stimulation by electrical pulses (Dudek & Bear, 1992). The induction consists of a Žxed number of several hundred (e.g., 600–900) pulses delivered at various frequencies. No experimental control is exercised on the postsynaptic activity, which is determined by the presynaptic activation. The experiments are carried out in slices from the CA1 region of the hippocampus, stimulating the Schaffer collateral afferents. Depending on the stimulation frequency, LTP or LTD is induced. The same protocols also induce reliably synaptic changes in the neocortex. High-frequency stimulation is particularly effective in inducing LTP. On the other hand, presynaptic activation at low frequencies is likely to trigger homosynaptic LTD. More precisely, no plasticity is observed at frequencies less than 0:1 Hz, LTD is observed using 1 Hz stimulation, and LTP is observed using stimulation frequencies greater than 10 Hz (Bear, 1996). This phenomenology is explained in the following way (Bear, 1996): Upand down-regulation of synaptic strength depends on the postsynaptic intracellular concentration of calcium, which in turn depends on the level of postsynaptic membrane potential. If the level of [Ca2C ] exceeds a threshold, the synapse tends to be potentiated. Otherwise, it tends to be depressed. High-frequency stimulation of a cell produces a higher level of postsynaptic depolarization, due to the fast temporal summation of excitatory postsynaptic potentials (EPSPs) and, hence, higher levels of [Ca2C ]. Stimulation at 10 Hz produces, on average, neither LTD nor LTP, which may indicate a critical level of postsynaptic [Ca2C ] corresponding to the presynaptic activation. This hypothesis seems corroborated by other experiments (Steele & Mauk, 1999) in which the level of recurrent inhibition during the stimulation is pharmacologically controlled by using either agonist or antagonist of the GABA receptor. When antagonist is applied, the threshold frequency for LTP is decreased; if agonist is present, it is increased. When GABA antagonist is present, the recurrent inhibition is less effective. Accordingly, the depolarization level allowing for LTP can be reached with lower frequencies. The reasoning is analogous in the presence of the GABA agonist.

568

D. Amit and G. Mongillo

More recently, other stimulation protocols have been developed to induce long-term synaptic modiŽcations in which postsynaptic activity also is experimentally controlled (Markram, Lubke, ¨ Frotscher, & Sakmann, 1997; Bi & Poo, 1998). In this experimental setup, both pre- and postsynaptic events are evoked by injecting current pulses into the cell body. Events are then coupled according to various protocols, varying either the temporal order between the pre- and postsynaptic events or the number and/or the frequency of pairing. The role of the number of pre- and postsynaptic spike pairings is studied in Markram et al. (1997). A postsynaptic spike is evoked several milliseconds after presynaptic emission. Such pairing is repeated 2, 5, and 10 times, and the sequence is repeated 10 times every 4 seconds. The postsynaptic spikes are evoked at 20 Hz. In all these cases, LTP is observed. Subsequently, the frequency dependence has been studied according to the following protocol: Žve presynaptic spikes are paired, as previously described, with postsynaptic spikes at various frequencies. Again, the pairing is repeated 10 times every 4 seconds. It was found that no potentiation occurs if the frequency is lower than 10 Hz. For the dependence on the relative timing between pre- and postsynaptic spikes, it is found that LTP occurs if the presynaptic spike precedes the postsynaptic one, while if the presynaptic spike follows the postsynaptic one, the synapse undergoes LTD. It is also noted that if the temporal interval between the two events is too large, no synaptic modiŽcations occur. Bi and Poo (1998) determined the exact temporal window in which the occurrence of both pre- and postsynaptic emission is effective to induce long-term synaptic modiŽcations. They coupled 60 presynaptic spikes delivered at 1 Hz with postsynaptic spikes by varying the temporal order of occurrence of the events (pre-post or post-pre) and the time difference between them. Synapses are strengthened if the presynaptic spikes precede postsynaptic spikes by less than 20 milliseconds and are weakened if instead presynaptic spikes follow postsynaptic spikes within the same interval. Markram et al. (1997) did not Žnd potentiation for presynaptic activation rates lower than 10 Hz, while Bi and Poo (1998) did Žnd both potentiation and depression at 1 Hz. The seemingly inconsistent behavior could be due to the fact that while Markram et al.’s experiment was carried out with cortical cells, Bi and Poo worked with hippocampal synapses. An alternative explanation is based on the difference in the number of pairings in the two cases. In other words, 5 pairings at 1 Hz do not provoke LTP, whereas 60 at the same frequency increase the synaptic efŽcacy. This fact could be understood in the following way: Each pairing of pre- and postsynaptic spikes tends to modify the synaptic efŽcacy, but a single event does not trigger long-term modiŽcation. The effect of a single pairing should therefore decay with time, and it is the accumulation of the effects of several pairings that is required to trigger a long-lasting modiŽcation. To provoke LTP or LTD, a speciŽc number of couplings must occur in a given time interval,

Spike-Driven Synaptic Dynamics Generating Working Memory States

569

so the individual effects are not totally forgotten. The number of pairings with respect to the time interval could in principle be experimentally determined, furnishing information about the effect of a single pairing and its decay time. Despite the somewhat contrasting conclusions reached using different protocols, as to the event that ultimately triggers the modiŽcations, other properties of synaptic plasticity are widely accepted. Experiments in vitro and in vivo have shown that a synapse is bidirectionally modiŽable (Dudek & Bear, 1993). In other words, the same synapse can undergo both LTP and LTD. Moreover, the same synapse once depressed can be newly potentiated by a suitable stimulation, and vice versa. However, the modiŽcation of the synaptic strength is prevented when NMDA receptors are blocked, regardless of the stimulation used (Dudek & Bear, 1992; Markram et al., 1997). An immediate action of the NMDA release is the opening of voltage-gated Ca2C channels and the consequent inux of the calcium ions. When this is prevented, the expression of the synaptic plasticity is impaired; no transitions are observed. This fact seems to support the model in which intracellular calcium concentration is a fundamental parameter controlling LTP and LTD. 3 From Experiments to the Model We now proceed to recapitulate the model of synaptic dynamics (Fusi et al., 2000) to be used as the learning element in the simulation. A plausible synaptic device would be able to maintain only a small number of discrete stable efŽcacies on long timescales. We choose the synapses to have two such states and to move between them stochastically. There is also some experimental evidence that synapses may in fact have only two stable values on a long timescale (Petersen, Malenka, Nicoll, & HopŽeld, 1998) . In section 7.1, we show that our synapse is not inconsistent with the wealth of Žndings concerning synaptic plasticity at the individual synaptic level described in section 2. We have not opted for a detailed agreement with all these Žndings because quite a few of them are still rather tentative and because our objective has been to have a reasonable synapse for a Žrst study of the formation of selective delay activity by the spikes produced in the course of the natural behavior of the neurons in a network. 3.1 A Model of Plastic Synapse. The plastic synapse is characterized by an internal analog variable X and a two-state value for its stable efŽcacy Jd ; Jp (depressed and potentiated, respectively). The stable efŽcacy is in turn determined by X (Fusi et al., 2000; Del Giudice & Mattia, 2001). When X is above a threshold µX , the synapse is in its potentiated state of efŽcacy Jp . Otherwise, the synapse is in its depressed state Jd .< Jp /. A transition occurs when X crosses the synaptic threshold: if X crosses from below to above, the result is LTP (Jd ! Jp ); if X crosses from above to below, the result is LTD (Jp ! Jd ). X is restricted to the interval (0 and 1), whose end points are

570

D. Amit and G. Mongillo

reecting barriers for the dynamics of X, and it obeys P D R.t/ C H.t/; X.t/

(3.1)

where R.t/ is a refresh term, responsible for long-term state preservation. It drives X toward 0.1/, depending on whether it is below (above) the synaptic threshold µX . This term mimics the biochemical mechanisms that keep synaptic efŽcacy stable in the absence of stimulation, against erasure by spontaneous activity. The dynamics of the drift we choose to be linear, R.t/ D ¡®2.¡X C µX / C ¯2.X ¡ µX /;

(3.2)

where 2.¢/ is the Heaviside function, 2.x/ D 1 for x > 0 and 0 otherwise. This bistability is analogous to that of a computer DRAM memory bit. H.t/ relates the synaptic dynamics to the pre- and postsynaptic neural activities and is responsible for synaptic transitions. The synaptic efŽcacy is modiŽable only when the synapse is activated by a presynaptic action potential. Hence, H.t/ should be different from zero only upon arrival of a presynaptic spike, which we express as X pre pre (3.3) H.t/ D F.tk /±.t ¡ tk /; k

that is, a sum over all presynaptic spike emission times. A priori, the time of the synaptic activation does not coincide with the presynaptic emission. In fact, once an action potential is generated in the soma, it travels along the axon, reaching the synaptic boutons (synaptic activations). Here, the spike provokes the release of the neurotransmitter, which in turn provokes the current inux into the postsynaptic cell. The time elapsed from the presynaptic spike emission to the postsynaptic variation of the depolarization is the synaptic delay and is of the order of a couple of milliseconds. However, most of this time is associated with postsynaptic ionic ow along dendrites. The axonal delay is very short, and the synaptic activation coincides with the presynaptic emission. Each spike induces a jump in X whose value is determined by F.¢/, which in turn depends on the state of the postsynaptic neuron at the time of the presynaptic emission. The complete speciŽcation of H.t/ requires the introduction of a neural model, which we assume to be that of an integrateand-Žre neuron (see section 4.1). One possibility is that F is determined by the intracellular concentration of calcium, as proposed by Bear (1996). F.¢/ would be positive when intracellular [Ca2C ] exceeds some critical level and negative if it is below another critical level. We model this complex mechanism taking F.¢/ to be a function of the postsynaptic depolarization.2 This 2 This assumption seems to be corroborated by recent experimental evidences (Sj¨ostr¨om, Turrigiano, & Nelson, 2001). See also section 7.1.

Spike-Driven Synaptic Dynamics Generating Working Memory States

571

may not be the Žnal answer, but it can be judged by its performance for the special protocols used in experiments on plasticity and with respect to the distributed learning dynamics it generates in a network. The synaptic dynamics is then related to the neural dynamics by postulating 8 if Vpost > µ1 > : 0 otherwise; with µ2 · µ1 < µ , where µ is the neural threshold for spike emission. Vpost is the membrane depolarization of the postsynaptic neuron. Examples of synaptic dynamics for LTP and LTD are depicted in Figures 1 and 2, respectively. 3.2 Transition Probabilities. The temporal evolution of H.t/, and hence the dynamics of the synaptic internal variable X.t/, could in principle be determined by specifying the pre- and postsynaptic neural activity. The situation we are interested in is neurons embedded in a large recurrent network. In this case, neural Žring is stochastic, principally due to the random pattern of connectivity (van Vreeswijk & Sompolinsky, 1996). As a consequence, LTP TRANSITION Pre­ synaptic spikes

NO TRANSITION Pre­ synaptic spikes

Synaptic internal variable X(t)

1 q

Synaptic internal variable X(t)

X

0

Post­ synaptic depolarization V(t)

Post­ synaptic depolarization V(t)

1 q 0

50

100 time (ms)

150

200

V

0

0

50

100 time (ms)

150

200

Figure 1: Synaptic dynamics. The time evolution of X.t/ (middle panel); following presynaptic emission (top), X.t/ is regulated up (down) if the postsynaptic depolarization V.t/ is greater (smaller) than µV .D µ1 ´ µ2 ) (bottom). In the time intervals between spikes, X.t/ drifts linearly up or down, according to equation 3.2 (see text for details). (Left) LTP. The synapse starts from its depressed value Jd , (since X < µX ); by the end of the interval, it is potentiated to Jp , (since X > µX ). (Right) No transition. At the end of the interval, X returns to its initial value. The evolution of V.t/ is that of an integrate-and-Žre neuron with a linear leakage current. For details see section 4.1. (Reproduced from Fusi et al., 2000.)

572

D. Amit and G. Mongillo

LTD TRANSITION Pre­ synaptic spikes

NO TRANSITION Pre­ synaptic spikes

Synaptic internal variable X(t)

1 q

X

0

Post­ synaptic depolarization V(t)

Synaptic internal variable X(t)

Post­ synaptic depolarization V(t)

1 q

V

0

50

100 time (ms)

150

200

0

0

50

100 time (ms)

150

200

Figure 2: Synaptic dynamics. (Left) LTD. A presynaptic burst allows X to cross the threshold µX from above, provoking downregulation of synaptic efŽcacy .Jp ! Jd /. (Right) No transition. Conventions as in Figure 1. (Reproduced from Fusi et al., 2000.)

long-term synaptic transitions are themselves stochastic, and it is possible to characterize them by a frequency-dependent transition probability. We deŽne PLTP .ºpre; ºpost ; T/ as the probability that a depressed synapse undergoes LTP during a time interval T of constant external afferents. ºpre and ºpost are, respectively, the mean emission rates of the pre- and postsynaptic neuron. The probability of LTD, PLTD .ºpre ; ºpost ; T/, is deŽned analogously. PLTP .ºpre; ºpost ; T/ and PLTD .ºpre ; ºpost ; T/ contain the information relevant for the unsupervised learning dynamics. They can be estimated by simulating the coupled neural and synaptic dynamics as described in Fusi et al. (2000). Once they are known, it becomes possible to determine the regimes of network activity of plasticity (i.e., signiŽcant modiŽcations of synaptic structure) and of overall structural stability (i.e., negligible modiŽcations). It also becomes possible to estimate the number of synaptic transitions in a given synaptic population per stimulus presentation or, equivalently, the probability of transition per presentation. When these probabilities are known, the structuring process can be described in a compact form, obtaining the fraction of potentiated synapses in each population as a function of the number of stimulus presentations. A simpliŽed classiŽcation of the synaptic dynamics can be obtained in terms of PT ´ PLTP C PLTD and PR ´ PLTP =PT (see also Fusi et al., 2000). PT is deŽned as the total transition probability, that is, the probability that a long-term transition occurs during T (either potentiation or depression), and P R is deŽned as the relative probability of LTP with respect to LTD. When PR » 1, LTP is likely to occur.

Spike-Driven Synaptic Dynamics Generating Working Memory States

573

Suppose a < µX and b < .1 ¡ µX /, so that several jumps are required to cross synaptic threshold. When the presynaptic neuron Žres, X is modiŽed according to the value of the postsynaptic depolarization. If a further spike does not arrive to push X toward the threshold, the synaptic internal state is reset by the refresh current, to either 0 or 1, depending on whether X is above or below its threshold (see equation 3.2). The time to forget completely a given jump in X is the ratio between the amplitude of the jump and the synaptic refresh current. In the synaptic device described here, we have a reset time for potentiation ¿p .D a=®/ and one for depression ¿d .D b=¯/. To simplify the reasoning, we suppose ¿p D ¿d . Transitions typically would be provoked by bursts of presynaptic spikes. A burst can cause a transition if the time intervals between successive spikes (ISI) are on average shorter than ¿p . In this case (the high presynaptic rate regime), a long-term synaptic transition, either up or down, can occur. When the mean ISI is greater than ¿p (the low presynaptic rate regime), the probability of a synaptic transition is negligible. Consequently, PT is zero if ºpre ¿ 1=¿p and is an increasing function of the presynaptic emission rates for ºpre > 1=¿p . The direction of a jump in the synaptic internal variable depends on the value of the postsynaptic depolarization (see equation 3.4). The relative frequency of upregulation of X (X ! X C a) with respect to that of downregulation (X ! X ¡ b) increases with postsynaptic emission rates. Indeed, to emit at high rates, the postsynaptic depolarization must often be near the threshold and, hence, V > µ1 . On the other hand, when the postsynaptic neuron is emitting at a low rate, its depolarization is uctuating around the rest potential, and hence below µ2 , so P R would be an increasing function of ºpost . We have argued that the synaptic device behaves in a Hebbian way. In other words, a synapse between two neurons, both emitting at a high rate, tends to be potentiated (PT > 0, PR » 1), while a synapse connecting a high-rate presynaptic neuron to a low-rate postsynaptic one tends to be depressed (PT > 0, PR » 0). In fact, as we shall see, a dynamic synapse naturally extends the plasticity scenario: synapses in neural populations where one would expect LTP may undergo LTD, and vice versa. This may be due to uctuations or to the width of the emission rate distribution in a functionally uniform neural population. When the average rates are high and close to each other in the pre- and postsynaptic cells or if there is a very large difference between the two, deviations are few, and Hebb is maintained to a very good accuracy. But if the average rates are closer, as would, for example, be the case when one population of neurons is stimulated and another is in enhanced delay activity (the substrate for the generation of contextcorrelations; Miyashita, 1988; Brunel, 1996), or of pair-associate representations (Sakay & Miyashita, 1991; Erickson & Desimone, 1999), LTP and LTD probabilities for the synapses connecting the two populations would both be signiŽcant. The one for LTP would be higher to allow the association.

574

D. Amit and G. Mongillo

The structuring in that population will not go on indeŽnitely (creating representation collapse), but will saturate at a level determined by the ratio of the two probabilities, a very welcome modiŽcation that will be explored in depth elsewhere (Mongillo & Amit, 2001a). 4 Simulations 4.1 Single Cell Model. As a model of the spiking neuron we use the linear integrate-and-Žre (LIF) model of Fusi and Mattia (1999) because it has been found to be consistent with the behavior of real cells in noisy conditions (Rauch, La Camera, Luscher, ¨ Senn, & Fusi, 2002) and also because of its analytical tractability. Despite its simplicity, this neural model is able to reproduce much of the phenomenology of networks of RC integrate-and-Žre neurons (Fusi & Mattia, 1999; Mongillo & Amit, 2001b). It is characterized by a Žring threshold µ, a (postspike) reset potential Vr , a constant leakage current ¯I , and a refractory period ¿arp . The evolution of the membrane depolarization below threshold is ( ¡¯I C I.t/ if V.t/ > 0 (4.1) VP D 0 if V.t/ D 0 and VP < 0: I.t/ is the afferent current charging the cell’s membrane. When V crosses the threshold, a spike is emitted, and the depolarization is reset to Vr and kept constant for ¿arp milliseconds. If I.t/ is stationary, gaussian, and independent at different times (delta correlated), the distribution of the depolarization P.V/ and the mean emission rate º are calculated to be (Fusi & Mattia, 1999) ± ²i º h ¹ 1 ¡ exp ¡2 2 .µ ¡ V/ for V 2 [0; µ] (4.2) P.V/ D ¹ ¾ µ ³ ´¶¡1 ¾ 2 2¹µ ¡ 2¹µ ¾2 ¡ 1 C (4.3) º ´ 8.¹; ¾ 2 / D ¿arp C e ; 2¹2 ¾ 2 for Vr D 0. The hypotheses on the statistics of the afferent current are quite well satisŽed in a large spiking network (Amit & Brunel, 1997b). This model does not include adaptation effects, essential for Žtting neural response dynamics of real cells (Rauch et al., 2002), limiting the increase in neural emission rates during sustained stimulation. In our simulations, this simpliŽcation is corrected by hand (see section 4.2.5). 4.2 The Learning Process. 4.2.1 The Network Architecture. The network (modeling a cortical module) is composed of NE pyramidal cells and NI interneurons. Each neuron receives, on average, CE synaptic contacts from excitatory and CI from inhibitory neurons inside the network and Cext excitatory synaptic contacts

Spike-Driven Synaptic Dynamics Generating Working Memory States

575

representing external afferents (Amit & Brunel, 1997b). The current resulting from the activation of the external synapses represents noise from the rest of the cortex and selective afferents due to stimuli. When the network is set up, the afferent presynaptic neighbors of a given neuron are selected as follows: for each (postsynaptic) neuron (excitatory or inhibitory), one selects the presynaptic neighbors, independently and at random, by a binary process in which a presynaptic excitatory (inhibitory) neuron is a neighbor with probability CE =NE (CI =NI ). The existing inhibitory as well as the excitatory synapses onto interneurons are assigned continuous values, each drawn from a gaussian distribution with its preassigned mean and variance. These synapses remain Žxed throughout the simulation. The plastic excitatory-excitatory synapses (prior to training) are distributed independently and randomly, with P .Jij .0/ D Jp / D Cp0 D 0:25. This is equivalent to a distribution for excitatory-excitatory synapses with mean JEE =µ D Cp0 Jp C .1 ¡ Cp0 /Jd D 0:022 and variance 12 JEE =µ 2 D Cp0 .1 ¡ Cp0 /.Jp2 C Jd2 / D 0:00063. Numerical values correspond to the parameters in Table 1. Furthermore, to each synapse is associated a transmission delay ±, which represents the time needed by the presynaptic spike to affect the postsynaptic depolarization. In our simulations, we chose ± D 1 ms for all the synapses. This is the unstructured state of the network, which is supposed to sustain spontaneous activity. The complete list of parameters used in the simulations is reported in Table 1. 4.2.2 The Dynamics. The simulation consists of a numerical integration of the discretized dynamical equations of both neurons and plastic synapses. The temporal step 1t is chosen to be shorter than the time between two successive afferent spikes. In our simulations, we take 1t D 0:05 ms. The initial distribution of depolarization in the network is set uniform. Spikes begin to be emitted due to external excitatory afferents. All neurons are receiving a nonselective external current Ii.ext/ D Jext ½.t/, where ½.t/ is a Poisson process with mean Cext ºext per unit time. ºext is taken of the order of the rate of excitatory neurons in the spontaneous state and is kept Žxed. At each 1t, the afferent external current is Jext ½.t/1t. The initial distribution of the neural depolarization is found not to be important. The network reaches its stationary state (depending on the synaptic matrix and level of external signal) within short relaxation times (see Figure 6 and the discussion in section 5.1). The network is allowed to evolve freely (i.e., without stimulus presentation) for about 100 ms, to reach stationary spontaneous activity, and then the training stage begins. The depolarization of all neurons is sequentially updated according to equation 4.1. If Vj .tC1t/ > µ, a spike is delivered to all postsynaptic neurons, and the depolarization is reset to Vj D Vr . The spike adds to the value of the depolarization of postsynaptic neuron i, at time t C 1t C ±, the value of the synaptic efŽcacy connecting the neuron j to i, if they are connected.

576

D. Amit and G. Mongillo Table 1: Parameters Used in the Simulations. Network parameters NE Number of pyramidal cells NI Number of interneurons CE Number of recurrent E connections per cell CI Number of recurrent I connections per cell Cext Number of connections from outside ºext [Hz] Spike rate at external synapses p Number of selective populations f Fraction of cells responding to a stimulus Single cell parameters E µ[µ ] Spike emission threshold 1 Vr =µ Postspike reset potential 0 ¯I =µ [ms¡1 ] Leakage current .011 ¿arp [ms] Absolute refractory period 2 Synaptic parameters Jext=µ EPSP produced by external afferent JEI =µ IPSP amplitude on pyramidal cells JIE =µ EPSP amplitude on interneurons JII =µ IPSP amplitude on interneurons Jp =µ EPSP produced by potentiated synapse Jd =µ EPSP produced by depressed synapse Cp0 Fraction of Jp in unstructured network ± [ms] Synaptic delay Synaptic dynamics parameters µX Threshold for synaptic transition µ1 =µ Threshold for upregulation of X µ2 =µ Threshold for downregulation of X ®[ms¡1 ] Drift toward 0 ¯[ms¡1 ] Drift toward 1 a Amplitude of up jump b Amplitude of down jump

5000 1250 380 120 380 5 5 0.1 I 1 0 .0113 2 .019 .063 .025 .055 .057 .01 .25 1 .374 .7 .5 .0067 .01 .17 .14

Notes: Units are given in brackets. Before structuring, the mean and variance of the excitatory efŽcacy in the excitatory population are given by JEE =µ D Cp0 Jp C.1¡ Cp0 /Jd D 0:022 and 12 JEE =µ 2 D Cp0 .1 ¡ Cp0 /.Jp2 C Jd2 / D 0:00063.

Moreover, if the emitting neuron is excitatory, all plastic synapses connecting it to other excitatory neurons are updated according to equations 3.1, 3.2, and 3.4. The level of the postsynaptic depolarization, determining the direction of the jump in X.t/ according to equation 3.4, is read at the time of the presynaptic emission (see section 3.1). If Xij .t C 1t/ (the internal variable of the plastic synapse connecting neuron j to i) crosses the threshold µX , the efŽcacy Jij .t C 1t/ is suitably modiŽed, producing either LTP or LTD. 4.2.3 Statistics of Stimuli. The p stimuli to be used in training are set up when the simulation is initialized and is kept Žxed. Each stimulus cor-

Spike-Driven Synaptic Dynamics Generating Working Memory States

577

responds to a pool of fN e , visually responsive excitatory neurons. f is the coding level of the stimuli and is chosen low ( f ¿ 1). These pools are selected nonoverlapping, that is, the neurons have a perfectly sharp tuning curve, and pf < 1. They are p consecutive groups of fNe neurons. Although this is not a realistic constraint, it is very useful in allowing the monitoring of the complex double dynamics by mean-Želd theory and in simplifying greatly the analysis of the learning dynamics. This fact, however, does not render the process trivial (see section 5.1). Moreover, for low memory loading levels, p ¿ 1=f , it is a good approximation to the case of unconstrained random choice of stimuli (Amit & Brunel, 1997b). At higher loading levels, one must confront the full issue of the network memory capacity, whose details are beyond this study. The presentation of a stimulus is expressed by an increase in the rates of the external afferents to the selective cells (the corresponding pool). The rate of spikes arriving at these neurons is increased by a factor ge > 1. ½k .t/ is still a Poisson process, but with mean ge Cext ºext dt, where the index k runs over the visually responsive population. External currents to the other excitatory neurons are unaltered. Similarly, the afferents to all interneurons increase their activation rates by a factor gi > 1 (Miller et al., 1996). Accordingly, the neurons selective to the stimulus presented emit at elevated rates. In other words, a stimulus elicits a visual response in the same subset of cells whenever it is presented. There is no noise in the process of presentation. In a more realistic situation, the subset of cells excited may be slightly different on different presentations of the same stimulus. Accordingly, we carried out simulations with a noisy version of the stimuli. In these simulations, each stimulus was considered a prototype of speciŽc f NE cells. In each presentation, a fraction 1 ¡ f1 of the prototype was excited, as well as f2 of all other .1 ¡ f /NE excitatory cells. f1 is the noise level, and f1 D f2 D 0 is the noiseless case. To have a constant (on average) number of stimulated cells in each presentation, we chose f2 so that f2 D f1 f=.1 ¡ f /: 4.2.4 Training Protocol. For the training protocol, the set of stimuli is repeatedly presented to the network. Each stimulus appears for Tstim milliseconds. Then it is removed, and following a delay period of Tdelay, during which none of the populations is stimulated, another stimulus is presented. The presentation sequence of the stimuli is either kept Žxed (1 ! ¢ ¢ ¢ ! p ! 1) or the sequence is generated by choosing each stimulus to be presented, independently and randomly, with probability 1=p. All along the simulation, during and between stimulus presentation, the neural and synaptic dynamics are free and are described by equations 3.4 and 4.1. 4.2.5 Control of Visual Response. As structuring takes place, the increasing average recurrent excitatory synaptic efŽcacy causes a signiŽcant in-

578

D. Amit and G. Mongillo

crease of neural emission rates, at parity of external signal (contrast, ge ). Such an increase is doubly undesirable—Žrst, because it is not observed experimentally (Erickson & Desimone, 1999), and second, and more relevant for the present purposes, because it distorts the learning process. As things stand, it seems to be an artifact of the simplicity of the single cell dynamics as well as of the synaptic transmission model rather than of the learning process. In a more realistic situation, this problem is resolved by the adaptation features of the neurons and of the synaptic transmission (Tsodyks & Markram, 1997). This tendency of increased visual response during training is partially balanced by the stimulation of the inhibitory neurons during stimulus presentation. The enhanced Žring of interneurons limits the visual response of the pyramidal cells by the hyperpolarizing currents. Moreover, it enhances the synaptic depression process between stimulated and unstimulated neurons because the stronger hyperpolarizing currents, arising from stimulated interneurons, decrease the mean depolarization level of neurons of the unstimulated population. This enhances the depression of synapses afferent on them. The stimulation of inhibitory neurons, and its effects, appears to be consistent with experimental Žndings (Steele & Mauk, 1999). However, the activity of the inhibitory population cannot grow beyond a certain level because the interneurons also inhibit each other. On the other hand, inhibitory contacts among interneurons cannot be weakened too much, or the emission rate of the inhibitory population becomes so high as to suppress the activity of the excitatory cells completely. When inhibition was no longer sufŽcient, we artiŽcially kept the rate of stimulated neurons approximately constant during the learning process. The emission rates during stimulus presentation are suggested by the properties of the single synapse (see section 3.2) and are chosen at the start of the simulation. When, as a consequence of synaptic structuring, these rates exceed the initial level by more than 15%, stimulus presentation is interrupted. The synaptic structuring reached is used to calculate, by mean-Želd analysis (see the appendix), a new level of contrast (ge ) for the stimuli, keeping gi Žxed. ge is calculated to produce the original rate of visual response, and the simulation resumes. 4.2.6 Observables Monitored. During the simulation, various observables related to the collective neural activity as well as the synaptic dynamics are sampled. To describe the evolution of the structuring among excitatory neurons in the network, we deŽne functional neural populations and functional synaptic populations. The neural populations are of excitatory neurons, such as the population corresponding to one of the p stimuli; when a particular stimulus is presented, the union of the p ¡ 1 populations corresponding to the other stimuli, not now recalled; and the population of background cells. In every phase of the simulation, the neurons in each of these populations receive the same average external input and see, on average, the same synaptic structure.

Spike-Driven Synaptic Dynamics Generating Working Memory States

579

The functional synaptic populations are the synapses connecting pairs of neurons within a functional neural population or between two different functional neural populations. Therefore, a synaptic population is fully deŽned once the functional properties of its pre- and postsynaptic neurons are speciŽed. The state of any synaptic population (its structuring) is quantiŽed by the fraction, Cp , of potentiated synapses in that population. In practice, we monitor only two structuring parameters, Cp . One of them is CpHH , between neurons in one of the p selective populations. It is one parameter and not p since we observe a very small variability of structuring corresponding to the different stimuli, and so we monitor their average. The other is CpHL , between neurons belonging to a selective population and those in the background or in other, unstimulated populations. Again, the average over p such populations is monitored. Recall that only synapses with high-rate presynaptic neurons have a signiŽcant transition probability. We do not distinguish between postsynaptic neurons in the background and neurons of nonselected populations because their rates are rather close. This is obvious before the network is signiŽcantly structured. It remains so after structuring due to the enhanced inhibitory activity. Most of the (nonstimulated) neurons emit at very low rates, and the distributions of frequencies in those populations are quite similar. This was checked by carrying out a set of simulations and comparing the structuring of the synaptic population between visually responsive and background neurons with the structuring between visually responsive and neurons responsive to a different stimulus. Moreover, we do not consider the structuring between background, or unselected, neurons and other populations, because the presynaptic rates are low, and so structuring is negligible, as conŽrmed by the simulations. Because the selective neural populations are nonoverlapping, the functional synaptic populations deŽned above are also nonoverlapping. This is true also for the synaptic populations between selected neurons and the rest, because even when the synapses of the two populations share postsynaptic neurons, the presynaptic neurons are different. The synaptic structuring reached in one synaptic population is not disrupted by the presentation of a different stimulus, and this fact simpliŽes the analysis of synaptic structuring. The level of synaptic structuring required for WM states may be estimated by mean-Želd analysis (see the appendix). When synaptic structuring reaches the level theoretically estimated, the training stage is interrupted. Each stimulus is presented for 150 ms and then removed and the network evolves for 1 second in the absence of further stimulations. If, following the removal, the network exhibits a WM state for all stimuli, the learning stage is terminated. Otherwise, it is resumed. If stimulus presentation continues after WM activity appeared, the structuring reaches an asymptotic level and further presentations do not affect the synaptic matrix. The asymptotic

580

D. Amit and G. Mongillo

structuring level expresses a detailed balance between potentiating and depressing processes (see section 6.1). 5 Results 5.1 The Structuring Process. The simulations presented here are for nonoverlapping stimuli. The parameters are given in Table 1. Even in this basic case, the process of synaptic structuring, and the consequent appearance of WM states, is not trivial. Indeed, various instabilities tend to appear during learning. The most common instabilities encountered in our simulations were oscillatory behavior, uncontrolled growth of the network global activity, or depression where one expects potentiation or vice versa. The essential underlying reason is that to reach stable, selective WM states, synapses must not only be potentiated among visually selective neurons, but this potentiation must be accompanied by adequate depression from selective to nonselective cells (Brunel, 2000). On the other hand, with the synaptic dynamics proposed here, in any population of synapses, one Žnds potentiations as well as depressions, as we explain below. Depressing synapses that should be potentiated can prevent the formation of WM states, because the fraction of potentiated synapses remains too low, at Žxed Jp and Jd . Often these effects appear when the level of visual response becomes too high. As a consequence, the rate of synaptic modiŽcations increases appreciably, and hence, the effects of unwanted synaptic transitions, occurring due to the wide distribution of rates inside the neural populations, are ampliŽed. Alternatively, the ratio between the probabilities of potentiation and depression could vary, again provoking unwanted synaptic modiŽcations. Another source of unwanted plasticity is the correlation between the stimulated and the unstimulated pyramidal cells. Due to the increased inhibitory activity, the activity in the background population is quite low. The spike emission of a neuron belonging to the background population is provoked principally by the spikes afferent from the visually responsive population. As a consequence, spike emission in a presynaptic neuron of the selective population consistently tends to precede the emission of the postsynaptic neuron if it belongs to the background population (Rubin, Lee, & Sompolinsky, 2001). Thus, despite the low emission rate, the postsynaptic depolarization could be found often at a high level when the synapse is activated by presynaptic emission. This could provoke LTP instead of the appropriate LTD. However, this effect is negligible when the network is sufŽciently large and the single EPSP is small with respect to the threshold for spike emission. Hence, in a way, the results we report represent a demonstration that in the rich and complex space of parameters of the synapses and the neurons exists a zone in which the fundamental structuring for working memory can take place. It would have been more satisfactory had the scenario been

Spike-Driven Synaptic Dynamics Generating Working Memory States

581

more robust, in the sense of self-organization, so that learning would induce neural activity conducive to structuring. But the complexity of the situation limits us to one step at a time. Figure 3 reports the average fraction of potentiated synapses in both the HH and HL synaptic populations as a function of the number of presentations per stimulus. Cp increases monotonically in the HH populations with stimulus presentation, while the fraction of potentiated synapses decreases with stimulus presentation (in the HL populations). Figure 4 reports the average number (averaged over stimuli) of synaptic potentiations and depressions per presentation in both the HH and HL synaptic populations. Note that there are depressions in the HH populations as well as potentiations in the HL populations, due to the wide distribution of rates in each

0.9 0.8

HH

0.7 0.6

C

p

0.5 0.4 0.3 0.2 HL

0.1 0 0

20

40 60 80 # of presentations/stimulus

100

120

Figure 3: Synaptic structuring: Cp in the two monitored synaptic populations as a function of the number of presentations (nonoverlapping populations). Horizontal line: The unstructured initial state (Cp0 D 0:25). CpHH (between visually responsive neurons) increases monotonically with stimulus presentation; CpHL (between visually responsive to nonresponsive neurons) decreases monotonically. The values of CpHH and CpHL are averaged over all functionally equivalent synaptic populations. The interpopulation variability is included in the circles representing the points. Synaptic structuring allowing for WM is reached after 105 presentations of each stimulus. See also Figure 7. The parameters are given in Table 1.

582

D. Amit and G. Mongillo

1500

400

1000

200

500

# of transitions

600

0 0

10 20 # of presentation

0 0

10 20 # of presentation

Figure 4: Number of synaptic transitions (stimulus average) per presentation versus presentation number for the Žrst 20 presentations. Squares represent number of potentiations and circles the number of depressions. (Left) HH population: number of potentiations greater than the number of depressions. (Right) HL population: number of depressions greater than the number of potentiations. The difference between the absolute numbers of transitions in the two Žgures is due to the greater number of synapses in the HL population. Parameters are as in Figure 3.

population (see Figure 5), which produces a Žnite probability that synapses within a selective population of neurons Žnd a high-rate presynaptic neuron and a low-rate postsynaptic neuron. Similarly, there is a Žnite probability that synapses between selective and nonselective populations Žnd two high-rate neurons. However, with the parameters of Table 1 and the relevant rates, the number of “correct” transitions is overwhelmingly greater than the number of “wrong” transitions. The outcome is an increase of CpHH and a strengthening of the mean synaptic efŽcacy among selective neurons and a monotonic decrease of CpHL and a depression of the mean synaptic efŽcacy from selective to nonselective neurons. In this situation, we recover the classical Hebb rule. As a control, we have veriŽed that similar results are obtained when the set of stimuli is noisy. We carried out simulations with f1 D 0:1 and f1 D 0:2. Recall that 1 ¡ f1 is the probability that a neuron of the prototype is effectively activated by its presentation in the presence of noise. The only noticeable, and expected, effect was that WM states require more presentations to appear: 140 to 150 presentations per stimulus instead of 100 to 110 required in the noiseless case. 5.1.1 Appearance of Working Memory. Working memory (selective delay activity) states appear for CpHH D 0:875 and CpHL D 0:0045. The appearance of WM states is related not only to the level of synaptic structuring, that is, the values of Cp in the various synaptic populations, but also to the other

Spike-Driven Synaptic Dynamics Generating Working Memory States 0.07

583

0.1 0.09

0.06

0.08 0.05

0.07 0.06

P(n)

0.04

0.05 0.03

0.04 0.03

0.02

0.02 0.01 0 20

0.01 40

60 n [Hz]

80

100

0 0

20 n [Hz]

40

Figure 5: Population distribution of spike rates during stimulation. (Left) Stimulated population. The mean emission rate is ºstim D 60 Hz. (Right) Unstimulated pyramidal cells; the mean emission rate is ºbg D 0:9 Hz. Despite the low mean emission rate, neurons are found with high rate. Note that the high-frequency tail of the background distribution overlaps with the stimulated distribution. This could impair the synaptic structuring. Recurrent inhibition is effective in reducing this overlap. See the text for details.

parameters of the network, principally the ratio Jp =Jd between potentiated and depressed synaptic efŽcacy. For example, decreasing Jp once the structuring level allowing for WM states is reached prevents the appearance of WM states. On the other hand, increasing Jp could destabilize spontaneous activity. All network parameters, as connectivity, mean synaptic efŽcacies, level of synaptic structuring for WM states, and so forth should be chosen to lie in a biologically plausible range. Starting from an unstructured synaptic matrix (Cp0 D 0:25), 100 to 150 presentations per stimulus are required to reach stable WM, that is, selective delay states. Figure 6 shows the neural activity in a visually responsive population before, during, and following the presentation of a stimulus at various level of synaptic structuring. What is plotted is the number of spikes emitted per neuron, in bins of 5 ms, averaged over the selective populations and normalized to Hz (dividing by the bin size). The transients into the various stationary states, spontaneous activity, WM, or stimulated state are

584

D. Amit and G. Mongillo

100

100

n [Hz]

A

B 50

50

0 0 ­

500

­ T [ms] 100

0 0 ­

­ T [ms]

500

n [Hz]

C 50

0 0­

­

500 T [ms]

1000

Figure 6: Selective population activity during trial at three structuring stages: population-averaged neural activity (in 5 ms bins) converted to Hz (see text), before, during, and following stimulation. Arrows indicate 150 ms stimulation interval. Note the short transients from one stationary state to another. Mean Želd predictions of mean emission rates, ºth , during stimulation (ST), spontaneous (SA), and working memory (WM) activity are compared with the results of the simulation. (A) Fifty-Žve presentations per stimulus; CpHH D 0:755; CpHL D 0:0245. ST: ºth D 51:5 Hz ºsim D 44:5 § 1:5 Hz; SA: ºth D 1:3 Hz ºsim D 1:21 § 0:21 Hz. (B) Seventy presentations per stimulus; CpHH D 0:8; CpHL D 0:016. ST: ºth D 51:5 Hz ºsim D 45:7 § 2:9 Hz; SA: ºth D 1:3 Hz ºsim D 1:17 § 0:18 Hz. Horizontal dashed line: Average emission rate during stimulus presentation and SA. In both A and B no WM: population returns to spontaneous state after removal of stimulus. (C) Appearance of WM state: 105 presentations per stimulus; CpHH D 0:875 and CpHL D 0:0045. WM: ºth D 27 Hz ºsim D 24:3 § 4:25 Hz. Horizontal dashed line: Average emission rate during WM activity. Large uctuations are due to strong Žnite-size effects. See text for details.

Spike-Driven Synaptic Dynamics Generating Working Memory States

585

short, on the order of 50 to 100 ms. In Figures 6A and 6B, the level of synaptic structuring is still too low, and the stimulated population returns to spontaneous activity when the stimulus is removed. In Figure 6C, one observes a WM state in which the cells of the selective populations emit at elevated rates (24.3 Hz) following the removal of the stimulus. Mean Želd predictions for the mean emission rates during spontaneous and WM activity (dashed horizontal lines) are in good agreement with the simulations. During WM activity, we observed very large uctuations of the average emission rate. This is due to Žnite size effects; to reduce the duration of the simulations with the double dynamics, the selective populations were taken relatively small and, given the low connectivity, a neuron receives only about 38 connections from other neurons in the same selective population (see Table 1). Due to the small number of connections from other selective cells and the stochastic nature of spike emission,pthe current afferent on a cell has large uctuations (of relative order 1= N, N is the number of pre-synaptic neurons) around its mean which produce uctuations in the emission rate. 6 Estimates of the Structuring Process 6.1 Population Dynamics of the Structuring Level. We deŽne qC as the probability that a depressed synapse undergoes LTP during one stimulus presentation and q¡ as the probability of depression of a potentiated synapse per presentation. It is expected that qC > q¡ in an HH synaptic population and vice versa in an HL population. The fraction of potentiated synapses follows a population evolution as a function of the number of stimulus presentations, Cp .n C 1/ D Cp .n/[1 ¡ q¡ .n/] C qC .n/[1 ¡ Cp .n/];

(6.1)

where Cp .n/ represents the fraction of potentiated synapses in a given synaptic population after n presentations of the same stimulus. The dependence of qC and q¡ on n takes into account the possibility that they could vary along the learning process. If both the frequencies of visual response and the statistics of the emission process do not vary appreciably during learning, qC and q¡ would remain approximately constant. In this case, for nonoverlapping populations, after a large number of presentations, (

HH HH CpHH D qHH C =.qC C q¡ /

CpHL

D

HL qHL C =.qC

C qHL ¡ /

for HH populations for HL populations:

(6.2)

Note that when the selective populations overlap, the structuring of a given population could change even if its preferred stimulus is not presented. In

586

D. Amit and G. Mongillo

this case, equation 6.1 becomes more complicated, and one must also take into account the structuring due to the presentation of different stimuli. We do not expand on this issue here. In our case (see, e.g., Figure 4), the number of wrong transitions—depressions instead of potentiations in the selective population and potentiation between selective and nonselective—is rather small. We neglect these transitions, and equation 6.1 becomes (

Cp .n C 1/ D Cp .n/ C qC .n/[1 ¡ Cp .n/] Cp .n C 1/ D Cp .n/[1 ¡ q¡ .n/]

for HH populations for HL populations:

(6.3)

The dashed curves in Figure 7 are a least-square Žt of Cp in the HH and HL populations measured in the simulation by the solution of equation 6.3 with constant qC and q¡ , during the Žrst 20 stimulations. The degree of agreement between the curves indicates that these probabilities are approximately constant during learning, partly because the rates under stimulation are kept approximately constant. 6.2 Population Transition Probabilities and Transition Numbers. The probability of an LTP transition in a homogeneous population of synapses, per stimulus presentation of duration T, could be estimated as follows.3 Let c be the probability of a synaptic contact from a neuron in the presynaptic population to a neuron in the postsynaptic one. The population of synapses connects neurons in two populations: one of Npre presynaptic neurons and one of Npost postsynaptic ones (the physical neurons in the two populations can, of course, be the same). The number of potentiations in such a synaptic population, per presentation of a stimulus, is a random variable, X Npot D cij ¢ °ij ; i;j

where the index j runs over the presynaptic neurons and i over the postsynaptic ones. If there exists a synapse between the two neurons, cij D 1, otherwise D 0. °ij D 1 if the synapse was potentiated during the stimulation, otherwise D 0. Hence, °ij D 1 with probability PLTP .ºi ; ºj ; T/ ¢ ±.Jij ¡ Jd /, where Jij is the synaptic efŽcacy before the stimulation and ±.¢/ is 1 if its argument is zero, and zero otherwise. The mean number of potentiations is Npot D

X i;j

cij ±.Jij ¡ Jd / ¢ PLTP .ºi ; ºj ; T/:

(6.4)

The sum on the right-hand side can be converted to a sum over pairs of activity .ºi ; ºj /. Let Ppost.pre/ .º/ be the probability of Žnding a neuron with 3

The reasoning is analogous for LTD.

Cp

Spike-Driven Synaptic Dynamics Generating Working Memory States

0.6

0.3

0.4

0.2

587

0.2 0.1 0 10 20 0 10 20 # of presentations/stimulus # of presentations/stimulus Figure 7: Population dynamics description of synaptic structuring during initial 20 presentations per stimulus. (Left) Average Cp in HH populations. (Right) Average Cp in HL populations. Dashed lines represent the solution of equation 6.3, with qC D 0:0233 and q¡ D 0:0426, constant (independent of n). Error bars are included in the circles. Horizontal lines: unstructured state of the network.

rate º in the post-(pre-)synaptic neural neural population, respectively. The probability of having a depressed synapse with a presynaptic neuron at rate ºj afferent on a postsynaptic neuron at rate ºi is c.1 ¡ Cp /Ppre .ºj /Ppost .ºi /. Hence, when the number of neurons is large, equation 6.4 becomes Z Z Npot ’ c.1¡Cp /NpreNpost dºi dºj PLTP .ºi ; ºj ; T/Ppre.ºj /Ppost .ºi /: (6.5) Correspondingly, the mean number of synaptic depressions is Z Z Ndep ’ cCp NpreNpost dºi dºj PLTD .ºi ; ºj ; T/Ppre.ºj /Ppost .ºi /:

(6.6)

If during learning, the integrals on the right-hand side of equations 6.5 and 6.6 do not vary signiŽcantly, they can be identiŽed, respectively, with qC and q¡ . In that case, it becomes straightforward to obtain equations 6.3 from equations 6.5 and 6.6. In order to compare the theoretical estimates with the values of qC and q¡ obtained from simulations, we have to estimate PLTP and PLTD over the range of frequencies observed in the simulation. For PLTP , the rate intervals of both pre- and postsynaptic activity cover the distribution in the stimulated population. Similarly, for PLTD , ºpre varies over the emission rates of stimulated neurons, while ºpost varies over the rates of unstimulated neurons. (See, e.g., Figure 5.) To obtain the currents giving rise to the observed emission rates, we proceeded as follows. In the stimulated population, the neurons operate in a signal-dominated regime (Fusi & Mattia, 1999), with small uctuations. We

588

D. Amit and G. Mongillo

assume that the variance of the current is constant within the population and is given by mean-Želd analysis (see the appendix). The mean current corresponding to a given frequency is simply obtained (for constant ¾ ) by inverting the transduction function, equation 4.3, which is a monotonic function of ¹. In the unstimulated population, the neurons operate in the noise-dominated regime (Fusi & Mattia, 1999); spike emission is due to sporadic large uctuations in the afferent current, which tends to hyperpolarize the membrane. Mean emission rates are very low, and the corresponding population distribution of rates is peaked around the mean-Želd result. We assume that all neurons in the population are in this state of activity. Mean Želd then provides the mean and variance of the currents. We proceed by simulating the coupled neural and synaptic dynamics, as described in Fusi et al. (2000), to obtain PLTP and PLTD . The extent to which the assumptions used are justiŽed in the real network can be judged by comparing the theoretical estimates, equations 6.5 and 6.6, and the probabilities sim observed during the simulation: qth C D 0:017 and qC D 0:023 § 0:005, while th sim for LTD we have q¡ D 0:038 and q¡ D 0:043 § 0:009. 6.3 Estimating Lifetime of Synaptic Structure. The synaptic structure may be affected by synaptic transitions provoked by spontaneous or selective delay activity. This may cause progressive erasure of the stored memories in the absence of stimulation. The mean lifetime of the acquired synaptic structure depends on the probability of having such synaptic transitions. After stable WM states appeared, a separate set of simulations was carried out in order to check the stability of the synaptic structure against both spontaneous and selective delay activity. First, we checked the stability of the synaptic structure when the network exhibits spontaneous activity. Simulations as long as 60 seconds were carried out, with no stimulation. There were no modiŽcations in synaptic matrix. Similarly, a WM state was elicited by presenting the corresponding stimulus for 150 ms, and the network evolved freely for 4 seconds. The time difference with respect to the case of the spontaneous activity is due to the shorter lifetime of the WM state as a consequence of Žnite-size effects. Again, there were no synaptic modiŽcations. The probability of a synaptic transition during spontaneous activity is very small, because the average rate of about 1 Hz ¿ 1=¿p . Thus, the number of repetitions needed to obtain a reasonable estimate of these probabilities becomes enormous. To obtain a rough estimate of q in spontaneous activity, we ran simulations to obtain the P LTP for neurons at a rate of 20 Hz and then rescaled this probability to rates of 1 Hz. To obtain the Žrst, we ran 106 repetitions of the single synapse simulation, of T D 400 ms each, with ºpre ´ ºpost D 20 Hz. No transition occurred. The default estimate of the transition probability is then P LTP .20; 20; T D 400/ » 10¡6 . In other words, the mean time one has to wait until the synapse makes a spontaneous transition (at 20 Hz) is about 4 days. Next we approximately rescale this probability to 1 Hz.

Spike-Driven Synaptic Dynamics Generating Working Memory States

589

For n up-jumps to provoke LTP within a time interval T 0 , one must have, na ¡ ¯T 0 ¸ µX ;

(6.7)

where a is the single up-jump amplitude and ¯ is the synaptic refresh current (see equations 3.2 and 3.4). For n Žxed, the longest time interval within which the presynaptic spikes must arrive and yet be able to produce LTP is Tmax .n/ D .na ¡ µX /=¯. For the parameters of Table 1, three spikes, all provoking up-jump, within 20 ms is the most probable burst that can produce LTP, considering the low emission rates of both pre- and postsynaptic neurons. Hence, to provoke the transition, such a burst must occur within the 400 milliseconds over which we have measured the transition probabilities above. During spontaneous activity, the statistics of the emission process of the neurons is well approximated by a Poisson process (Amit & Brunel, 1997b), then P3 .T 0 D 20I º D 20/ · 203 ¢ P3 .T 0 D 20I º D 1/; where Pn .T 0 I º/ is the probability that n presynaptic spikes occur within a time interval T 0 when the emission rate is º. Neglecting other factors, as the dependence of the probability of an up-jump on the postsynaptic emission rate, we obtain qSA .400/ D PLTP .1; 1; 400/ » 20¡3 PLTP .20; 20; 400/; leading to a mean lifetime of the order of 10 years. 7 Discussion The principal result of this study is a feasibility test. The spike-driven synaptic dynamics, introduced in Fusi et al. (2000), for a suitable choice of the parameters implements rate-dependent plasticity and exhibits both longterm potentiation and long-term homosynaptic depression under diverse experimental stimulation protocols. 4 It is not to be excluded that a synaptic device in natural conditions behaves more like the synapse discussed here than as in the special protocols in which precise timing is observed. Moreover, leaving from an unstructured synaptic state, the synaptic dynamics is able to drive the network into a structured state, sustaining selective delay activity, or working memory. The generated synaptic structure is robust against spontaneous and delay activity in the absence of stimulation. The question of stability of the acquired synaptic structure, and hence the related neural activity, is also addressed. The appearance of WM states, following the repetitive presentation of a set of stimuli, is not simply a direct consequence of the rate-dependent 4

This synaptic dynamics does not generate long-term heterosynaptic depression.

590

D. Amit and G. Mongillo

plasticity, implemented by the synaptic model. Several constraints must be met during the structuring stage: ² The effects of LTP should be adequately balanced by LTD to avoid instabilities along the learning process. This balance between the probabilities of LTP and LTD, imposes constraints on the synaptic and neural parameters. Further work is needed to relate such constraints to the neural and synaptic parameters in a simple form. ² The frequency distributions in the stimulated and unstimulated populations should not overlap signiŽcantly. Alternatively, the synapse should be highly sensitive to small variations of the frequencies in the overlap region (Del Giudice & Mattia, 2001). Indeed, this could be a source of unwanted plasticity that in turn impairs the structuring process. We have found that recurrent inhibition could play a signiŽcant role in separating the distributions, reducing the high-rate tail of the frequency distribution of the unstimulated populations. A signiŽcant by-product, mentioned here only briey, is the resulting extension of the Hebbian plasticity rule. A realistic synaptic model would generate both LTP as well as LTD in any population of synapses. In the particular situation (parameters) considered in detail here, the difference between “right” and “wrong” transitions is so large that the classical Hebbian picture ensues. But when one deals with the emergent coupling between two neural populations whose mean rates are not so different (as between stimulus driven cells and delay activity cells), the extension of the Hebbian scenario promises saturation in structuring, which is an essential ingredient in maintaining independent representations alongside WM. 7.1 Back to Timing-Dependent Plasticity. The sharp cutoff in the time difference between the arrival of pre- and postsynaptic spike, observed in experiment (Bi & Poo, 1998), could be a consequence of the experimental procedure. In the model presented here, the two thresholds (see equation 3.4) can be chosen to reproduce the results of Markram et al. (1997) when the two neurons connected by the synapse operate in deterministic conditions. This may, in fact, be the case in the experiments mentioned. In a deterministic situation, one can consider the two neurons stimulated by a constant noiseless current. The evolution of the depolarization is deterministic. Thus, there is a relation between the value of V at a given instant and the time elapsed from the emission of the last spike. If one chooses the two synaptic thresholds µ1 and µ2 of equation 3.4 to be µ1 D µ ¡ T1 ¹;

µ2 D T2 ¹;

(7.1)

where ¹ is the total current, the following picture emerges. Upon the emission of a presynaptic spike, the postsynaptic neuron has V < µ2 only if it

Spike-Driven Synaptic Dynamics Generating Working Memory States

591

emitted a spike at most T2 before, and the synaptic activation causes downregulation. Similarly, the postsynaptic neuron has V > µ1 only if the time of the presynaptic spike emission precedes the postsynaptic spike by at most T1 . It will then be followed by an upregulation of the synapse. If this is the case, one should conclude that up- and downregulation depend on the level of postsynaptic depolarization and the timing effect is only a consequence of the experimental setup, as argued above. Recent experimental Žndings seem to corroborate this hypothesis (Sj¨ostrom ¨ et al., 2001). Recently, Abbott and Song (1999) and Rubin et al. (2001) studied the behavior of a synaptic model into which the exact temporal relation, observed experimentally by Bi & Poo (1998), is built in. EfŽcacy is modiŽed according to the temporal interval 1t D tpost ¡ tpre , where tpre and tpost are the times of the pre- and postsynaptic spike emission. When 1t > 0, efŽcacy is upregulated, and the efŽcacy is downregulated for 1t < 0. Such a synaptic mechanism tends to potentiate synapses connecting correlated neurons, for example, a presynaptic neuron that consistently Žres before the postsynaptic one. On the other hand, presynaptic inputs that are not causally correlated with postsynaptic Žring are weakened. The overall effect is the convergence of the synaptic distribution to an asymptotic stationary distribution. Depending on the update rule, the equilibrium distribution can be either unimodal or bimodal (Rubin et al., 2001). However, in both cases, the asymptotic distribution is largely insensitive to the Žring rates of pre- and postsynaptic cells. Therefore, such a model of plastic synapse is unable to structure the synaptic matrix to sustain selective delay activity. 7.2 Open Issues. The collective behavior of coupled homogeneous neural and synaptic populations has been described in a compact form, in terms of probabilities of potentiation PLTP and depression P LTD , functions of the pre- and postsynaptic emission rates. Here, these functions were obtained from the detailed model of the plastic synapse. It is not clear whether such a description could be more general, so as to be qualitatively independent of the detailed neural and synaptic dynamics. A priori, this seems plausible as a general consequence of the stochasticity of the neural activity. If so, we could characterize the properties of P LTP and PLTD , allowing for successful structuring, regardless of the speciŽc model of the synapse. Of course, the construction of a detailed synaptic model, which matches both available experimental data and theoretical desiderata, should be the natural conclusion of such a study. The constraints on the transition probabilities become tighter when the issue of network capacity is considered. If stimuli are sparsely coded ( f ¿ 1) and the probabilities of transition are low (slow learning), a necessary condition to recover the optimal storage capacity is that the number of potentiations approximately balances the number of depressions in each presentation, requiring q¡ » fqC (Amit & Fusi, 1994; Brunel, Carusi, & Fusi,

592

D. Amit and G. Mongillo

1998). This ensures that the difference between Cp in HH populations and Cp in HL populations, reached asymptotically, is maximal, depending on the number of stimuli to be learned. However, the appearance of WM states is not guaranteed. It depends not only on the level of structuring but also on the other network parameters. It remains to be seen whether the network exhibits WM activity in a biologically plausible range of parameters. The behavior of the synaptic device is manipulable. By choosing suitably the parameters a, b, ®, ¯, and µX , the balance constraint may be satisŽed. Simulations should be carried out to check the behavior of large networks at high loading level. Those require very efŽcient algorithms, such as Mattia and Del Giudice (2000). But little is guaranteed; the results reported in Amit and Fusi (1994) and Brunel et al. (1998) are obtained under a simplifying hypothesis: the neurons are binary, and the existence of the attractors is checked by using a signal-to-noise ratio analysis. A further step toward the networks of spiking neurons is made by Herz and colleagues (see for a review Herz, 1996). They studied the capacity of network with analog neurons (no spiking) but with a symmetric connectivity matrix. The situation may be quite different when one deals with a recurrent network of spiking neurons, like that studied here. Appendix: Mean Field for Nonoverlapping Populations The mean-Želd analysis developed allows calculating the mean Žring frequencies in each of the populations in stationary (asynchronous) states of the network as a function of the instantaneous synaptic structure and the level of the external signal. The instantaneous synaptic structuring is described by the fraction of potentiated synapses in the various synaptic populations. Plastic synapses are divided by an incoming stimulus in three homogeneous populations, according to pre- and postsynaptic activities: ² High pre- and postsynaptic activity. It is expected that in this population, the fraction of potentiated synapses increases to CpHH .> Cp0 /. ² High presynaptic and low postsynaptic activity. The fraction of potentiated synapses decreases to CpHL .< Cp0 /. ² Low presynaptic activity. The probability of a synaptic transition is negligible; thus, the fraction of potentiated synapses is unaltered. Following the presentation of a stimulus that has been previously “learned,” the network divides in four functionally different populations of cells: cells belonging to the population activated by the stimulus (denoted sel); cells representing other stimuli, not activated (denoted +); cells not responsive to any stimulus (denoted bg); and interneurons (denoted I).

Spike-Driven Synaptic Dynamics Generating Working Memory States

593

The mean Žring rates in each in the four neural populations are given by mean-Želd equations (Amit & Brunel, 1997a), 2 2 ºsel D 8.¹sel ; ¾sel /; ºC D 8.¹C ; ¾C2 /; ºbg D 8.¹bg ; ¾bg /; ºI D 8.¹I ; ¾I2 /:

The statistics of the afferent currents is calculated as function of synaptic structuring. The recurrent currents are .e/ ¹sel D ¡¯I C CE [ f JC ºsel C f .p ¡ 1/J¡ ºC C .1 ¡ fp/J0 ºbg ] ¡ CI JEI ºI

¹C D ¡¯I.e/ CCE [ f J¡ ºsel C f [JC C .p ¡ 2/J¡ ]ºC C.1¡fp/J0 ºbg]¡CI JEI ºI .e/ ¹bg D ¡¯I C CE [ f J¡ ºsel C f .p ¡ 1/J¡ ºC C .1 ¡ fp/J0 ºbg ] ¡ CI JEI ºI

¹I D ¡¯I.i/ C CE JIE [ f ºsel C f .p ¡ 1/ºC C .1 ¡ fp/ºbg] ¡ CI JII ºI ;

(A.1)

where JC D CpHH Jp C .1 ¡ CpHH /Jd J¡ D CpHL Jp C .1 ¡ CpHL /Jd J0 D Cp0 Jp C .1 ¡ Cp0 /Jd :

(A.2)

The variances of the recurrent currents are 2 2 D CE [ f 1JC ºsel C f .p ¡ 1/1J¡ ºC C .1 ¡ fp/1J0 ºbg ] C CI JEI ¾sel ºI 2 ¾C2 D CE [ f 1J ¡ºsel C f [1JC C.p¡2/1J¡ ]ºC C.1¡fp/1J0 ºbg ]CCI JEI ºI 2 2 D CE [ f 1J¡ ºsel C f .p ¡ 1/1J¡ ºC C .1 ¡ fp/1J0 ºbg ] C CI JEI ¾bg ºI 2 [ f ºsel C f .p ¡ 1/ºC C .1 ¡ fp/ºbg] C CI JII2 ºI ¾I2 D CE JIE

where 1JC D CpHH Jp2 C .1 ¡ CpHH /Jd2 and analogous for 1J¡ , 1J0 . The external currents depend on the level of the external signal: .ext/

¹sel .ext/

¹C

2

.ext/ 2 D ge Cext Jext ºext; ¾sel D ge Cext Jext ºext 2

2

.ext/ 2 ´ ¹.ext/ D Cext Jext ºext ; ¾ C.ext/ ´ ¾bg D Cext Jext ºext bg 2

2 D gi Cext Jext ºext ; ¾I.ext/ D gi Cext Jext ¹.ext/ ºext : I

In a steady state, the density function of V will be also stationary. It is given by equation 4.2. Both the mean and the variance of the afferent

594

D. Amit and G. Mongillo

currents are a linear function of º; hence, P.V/ is parameterized only by the mean emission rates. In other words, there is a one-to-one correspondence between the emission rates and the distribution of the depolarization. Here we use the theory to determine (1) the range of CpHH and CpHL for which the network is able to sustain selective delay activity states and (2) the mean emission rate of stimulated populations as a function of the instantaneous level of synaptic structuring (3) and to compare simulation results in stationary states with theory. Acknowledgments We thank Stefano Fusi for having brought to our attention Sjostr ¨ om ¨ et al. (2001). This study was supported by a Center of Excellence grant, “Changing Your Mind,” from the Israel Science Foundation and by a Center of Excellence grant, “Statistical Mechanics and Complexity,” of the INFM, Rome. References Abbott, L. F., & Song, S. (1999). Temporally asymmetric Hebbian learning and neuronal response variability. In M. S. Kearns, S. A. Solla, & D. A. Cohn (Eds.), Advances in neural information processing systems, 11. Cambridge, MA: MIT Press. Amit, D. J. (1995). The Hebbian paradigm reintegrated: Local reverberations as internal representations. Behavioral and Brain Sciences, 18, 617–626. Amit, D. J. (1998). Simulation in neurobiology—Theory or experiment? TINS, 21, 231–237. Amit, D. J., & Brunel, N. (1997a). Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex. Cerebral Cortex, 7, 237–252. Amit, D. J., & Brunel, N. (1997b). Dynamics of a recurrent network of spiking neurons before and following learning. Network: Computation in Neural Systems, 8, 373–404. Amit, D. J., & Fusi, S. (1994).Learning in neural networks with material synapses. Neural Computation, 6, 957–982. Bear, M. F. (1996). A synaptic basis for memory storage in the cerebral cortex. Proc. Natl. Sci. USA, 93, 13453–13459. Bi, G. Q., & Poo, M. M. (1998). Synaptic modiŽcations in cultured hippocampal neurons: Dependence on spike timing, synaptic strength and cell type. Journal of Neuroscience, 18, 10464–10472. Brunel, N. (1996). Hebbian learning of context in recurrent neural networks. Neural Computation, 8, 1677–1710. Brunel, N. (2000). Persistent activity and the single cell f-I curve in a cortical network model. Network, 11, 261–280. Brunel, N., Carusi, F., & Fusi, S. (1998). Slow stochastic Hebbian learning of classes of stimuli in a recurrent neural network. Network, 9, 123–152.

Spike-Driven Synaptic Dynamics Generating Working Memory States

595

Compte, A., Brunel, N., Goldman-Rakic, P. S., & Wang, X. J. (2000). Synaptic mechanisms and neural dynamics underlying spatial working memory in a cortical network model. Cerebral Cortex, 10, 910–923. Del Giudice, P., & Mattia, M. (2001). Long and short-term synaptic plasticity and the formation of working memory: A case study. Neurocomputing, 38, 1175–1180. Dudek, S. M., & Bear, M. F. (1992). Homosynaptic long-term depression in area CA1 of hippocampus and effects of NMDA receptor blockade. Proc. Natl. Sci. USA, 89, 4363–4367. Dudek, S. M., & Bear, M. F. (1993).Bidirectional long-term modiŽcation of synaptic effectiveness in the adult and immature hippocampus. Journal of Neuroscience, 13, 2910–2918. Erickson, C. A., & Desimone, R. (1999).Responses of macaque perirhinal neurons during and after visual stimulus association learning. Journal of Neuroscience, 23, 10404–10416. Fusi, S., & Mattia, M. (1999). Collective behavior of networks with linear (VLSI) integrate-and-Žre neurons. Neural Computation, 11, 643–652. Fusi, S., Annunziato, M., Badoni, D., Salamon, A., & Amit, D. J. (2000). Spikedriven synaptic plasticity: Theory, simulation, VLSI implementation. Neural Computation, 12, 2227–2258. Herz, A. V. M. (1996). Global analysis of recurrent neural networks. In E. Domany, J. L. van Hemm, & K. Schulten (Eds.), Models of neural networks III. New York: Springer-Verlag. Kritzer, M. F., & Goldman-Rakic, P. S. (1995). Intrinsic circuit organization of the major layers and sublayers of the dorsolateral prefrontal cortex in the rhesus monkey. Journal of Computational Neurobiology, 359, 131–143. Markram, H., Lubke, ¨ J., Frotscher, M., & Sakmann, B. (1997). Regulation of synaptic efŽcacy by coincidence of postsynaptic APs and EPSPs. Science, 275, 213–215. Mattia, M., & Del Giudice, P. (2000). EfŽcient event-driven simulation of large networks of spiking neurons and dynamical synapses. Neural Computation, 12, 2305–2329. Miller, E. K., Erickson, C., & Desimone, R. (1996). Neural mechanism of working memory in prefrontal cortex of macaque. Journal of Neuroscience, 16, 5154– 5167. Miyashita, Y. (1988). Neural correlate of visual associate long-term memory in the primate temporal cortex. Nature, 335, 817–820. Mongillo, G., & Amit, D. J. (2001a). Spike driven synaptic dynamics—a selfsaturating Hebbian paradigm. Neural Plasticity, 8, 188. Mongillo, G., & Amit, D. J. (2001b). Oscillations and irregular emission in networks of linear spiking neurons. Journal of Computational Neuroscience, 11, 249–261. Petersen, C. C. H., Malenka, R. C., Nicoll, R. A., & HopŽeld, J. J. (1998). All-ornone potentiation at CA3-CA1 synapses. Proc. Natl. Sci. USA, 95, 4732–4737. Rauch, A., La Camera, G., Luscher, ¨ H. R., Senn, W., & Fusi, S. (2002). Neocortical pyramidal cells respond as integrate-and-Žre neurons to in vivo–like input currents. Manuscript submitted for publication.

596

D. Amit and G. Mongillo

Rubin, J., Lee, D. D., & Sompolinsky, H. (2001). Equilibrium properties of temporally asymmetric Hebbian plasticity. Physical Review Letters, 86, 364–366. Sakay, K., & Miyashita, Y. (1991). Neural organization for the long-term memory of paired associates. Nature, 354, 152–155. Sjostr¨ ¨ om, P. J., Turrigiano, G. G., & Nelson, S. B. (2001). Cooperativity and frequency dependence of spike timing–dependent plasticity in visual cortical layer-5 pairs. In Proceedings of the 31st SFN Meeting. San Diego. Steele, P. M., & Mauk, M. D. (1999). Inhibitory control of LTP and LTD: Stability of synapse strength. Journal of Neurophysiology, 81, 1559–1566. Tsodyks, M. V., & Markram, H. (1997). The neural code between neocortical pyramidal neurons depends on neurotransmitter release probability. Proc. Natl. Sci. USA, 94, 719–723. van Vreeswijk, C. A., & Sompolinsky, H. (1996). Chaos in neural networks with balanced excitatory and inhibitory activity. Science, 274, 1724–1726. Received December 13, 2001; accepted August 2, 2002.

LETTER

Communicated by David Horn

A Stochastic Method to Predict the Consequence of Arbitrary Forms of Spike-Timing-Dependent Plasticity Hideyuki Cˆateau [email protected] Core Research for the Evolutional Science and TechnologyProgram, JST, Tokyo 1948610, Japan

Tomoki Fukai [email protected] Department of Engineering, Tamagawa University and CREST, JST, Tokyo 1948610, Japan

Synapses in various neural preparations exhibit spike-timing-dependent plasticity (STDP) with a variety of learning window functions. The window functions determine the magnitude and the polarity of synaptic change according to the time difference of pre- and postsynaptic spikes. Numerical experiments revealed that STDP learning with a single-expo nential window function resulted in a bimodal distribution of synaptic conductances as a consequence of competition between synapses. A slightly modiŽed window function, however, resulted in a unimodal distribution rather than a bimodal distribution. Since various window functions have been observed in neural preparations, we develop a rigorous mathematical method to calculate the conductance distribution for any given window function. Our method is based on the Fokker-Planck equation to determine the conductance distribution and on the OrnsteinUhlenbeck process to characterize the membrane potential uctuations. Demonstrating that our method reproduces the known quantitative results of STDP learning, we apply the method to the type of STDP learning found recently in the CA1 region of the rat hippocampus. We Žnd that this learning can result in nearly optimized competition between synapses. Meanwhile, we Žnd that the type of STDP learning found in the cerebellum-like structure of electric Žsh can result in all-or-none synapses: either all the synaptic conductances are maximized, or none of them becomes signiŽcantly large. Our method also determines the window function that optimizes synaptic competition. 1 Introduction Plasticity that depends on the interval t D tpost ¡tpre between pre- and postsynaptic spikes has been observed in layer II/III or V pyramidal neurons, spiny stellate cells of neocortex, Xenopus tectum, excitatory and inhibitory cells of Neural Computation 15, 597–620 (2003)

c 2003 Massachusetts Institute of Technology °

598

H. Caˆ teau and T. Fukai

the hippocampus, and the cerebellum-like structure of electric Žsh (Abbott & Nelson, 2000; Levy & Steward, 1983; Gustafsson, Wigstrom, Abraham, & Huang, 1987; Debanne, Gahwiler, & Thompson, 1994, 1998; Bell, Han, Sugawara, & Grant, 1997; Markram, Lubke, Frotscher, & Sakmann, 1997; Magee & Johnston, 1997; Bi & Poo, 1998; Zhang, Tao, Holt, Harris, & Poo, 1998; Egger, Feldmeyer, & Sakmann, 1999; Feldman, 2000; Nishiyama, Hong, Mikoshiba, Poo, & Kato, 2000). This form of plasticity is termed spiketiming-dependent plasticity (STDP). The window function, for instance, ( AC e¡®C t .t ¸ 0/; (1.1) G.t/ D ¡A¡ e¡®¡ jtj .t < 0/; speciŽes that a synapse is potentiated by AC e¡®C t when a postsynaptic spike follows a presynaptic spike .tpost ¸ tpre/ and is depressed by ¡A¡ e¡®¡ jtj when a postsynaptic spike precedes a presynaptic spike .tpost ¸ tpre /. Using this window function, Song, Miller, and Abbott (2000) simulated STDP learning of excitatory synaptic conductances mediating a thousand inputs to one model neuron (a leaky integrator). They examined how excitatory synaptic conductances on a model neuron changed due to STDP learning evoked by Poissonian inputs under the condition that the total depression is slightly larger than the total potentiation (i.e., A¡ =®¡ > AC =®C ). As a result, they found that the Žnal distribution PŽn .w/ of synaptic conductances w displayed a bimodal distribution in which conductances were concentrated around either zero or the maximal value. Since synaptic conductances are divided into “winners” and “losers,” we regard this phenomenon as synaptic competition. On the other hand, van Rossum, Bi, and Turrigiano (2000) and Rubin, Lee, and Sompolinsky (2001) examined STDP learning using a single-exponential window function that employed a multiplicative update rule w ! w.1C1w/ instead of the additive update ruled w ! wC1w assumed in Song et al. (2000). Noting that this update rule is equivalent to replacing a window function G.t/ for wG.t/ (Kistler & van Hemmen, 2000; van Rossum et al., 2000; Rubin et al., 2001; Gutig, ¨ Aharono, Rotter, & Sompolinsky, 2002), they showed that multiplicative learning led to a gaussian conductance distribution, representing no synaptic competition. These theoretical studies of STDP learning were conducted using the Fokker-Planck equations (Risken, 1996), which generally describe how the distribution of stochastically changing variables (the conductance for STDP) evolves in time. To determine a critical coefŽcient of the Fokker-Planck equation, a drift term, van Rossum et al. (2000) simpliŽed the calculations by adopting a piecewise linear, neuronal response. Rubin et al. (2001) simpliŽed the calculations by assuming a reasonable functional form for the correlation between pre- and postsynaptic spikes. The Fokker-Planck equations thus approximated clariŽed the qualitative differences between additive and multiplicative STDP learning. Levy, Horn, Meilijson, and Ruppin (2001) modeled STDP learning by introducing a relaxation dynamics of w

Stochastic Method to Analyze STDP

599

in addition to the usual synaptic update. They constructed the corresponding Fokker-Planck equation and derived an exponential distribution in the no-relaxation limit. They also conducted simulations of mutually connected neurons in their STDP learning framework and obtained bimodal conductance distributions for such window functions that display a total depression smaller than a total potentiation. Similar numerical simulations of a network of Hodgkin-Huxley neurons were conducted without the synaptic relaxation dynamics (Kitano, Cateau, ˆ & Fukai, 2002). In the simulations, bimodal conductance distributions were obtained when the total depression is slightly larger than the total potentiation, as was the case in Song et al. (2000). In this study, we develop an analytical method to calculate the Žnal distribution PŽn .w/ for an arbitrary window function G.t/. Such a general framework is useful for analyzing the learning procedure for many different window functions observed in experiments (Abbott & Nelson, 2000). To describe the evolution of synaptic conductances, we adopt the Fokker-Planck equation (van Rossum et al., 2000; Rubin et al., 2001; Levy et al., 2001). To analytically evaluate the drift term approximately, we take the nonlinearity in neural spiking into account by using the Ornstein-Uhlenbeck process for the membrane uctuations (Tuckwell, 1988; Risken, 1996; Cateau ˆ & Fukai, 2001). With this careful evaluation of the drift term, which has not previously been attempted, we derive P Žn .w/ as an explicit function of model parameters, such as the size and the width of a window function and Žring threshold. Our method allows us to reproduce the previously known quantitative characteristics of STDP learning (Song et al., 2000), such as the linear relationship between pre- and postsynaptic Žring rates and unique dependencies of postsynaptic Žring rate and coefŽcient of variation on the amplitude ratio A¡ =AC . Our method also enables us to derive a corresponding formula for an arbitrary window function. Our method proves quite generally that synaptic competition occurs only when the total depression R 1is slightly larger than the total potentiation, that is, the total area S0 D ¡1 G.t/ dt is negative and small. This criterion is valid if the mean postsynaptic spike interval T is greater than the width of a window function (moderate Žring condition) and the synaptic update is additive. Note that since the criterion is derived for Poissonian input spike trains, it does not necessarily apply to network simulations. Our method also proves that synaptic conductances exhibit either gaussian distribution or a saturated distribution, in which conductances are concentrated around the maximal value, when the total depression is smaller than the total potentiation (i.e., S0 > 0). Thus, the total area S0 plays the most crucial role in determining PŽn .w/. If the moderate Žring condition is not RT obeyed, the total area should be replaced with the restricted area ¡T G.t/ dt in the above statements. To demonstrate the wide applicability of our method, we apply our method to two speciŽc window functions that are similar to those observed

600

H. Caˆ teau and T. Fukai

in the rat hippocampal CA1 slice preparation (Nishiyama et al., 2000) and in the cerebellum-like structure of the electric Žsh (Bell et al., 1997). Our method predicts that the CA1 type learning (i.e., the window function has depressing effects also in the causal domain, tpost > tpre ) results in almost maximal competition between synapses when the total depression is slightly larger than the total potentiation. Our method also predicts that the electric Žsh–type learning can induce an all-or-none development of synaptic conductances when the total depression is slightly smaller than the total potentiation. The results of our analyses can be summarized as the following general properties of additive STDP learning: ² The correlation between pre- and postsynaptic spikes increases only by such STDP learning that leads to efŽcient synaptic competition. ² A depression component in the late causal domain of the window function enhances synaptic competition. ² Synaptic competition is optimal when the causal part of a window function has the same shape as the nonaccidental correlation function of pre- and postsynaptic spikes, C.t/. 2 The Fokker-Planck Equation for the Conductance Distribution This section explains how the Fokker-Planck equation is used to elucidate the synaptic conductance distribution and how to derive the Fokker-Planck equation corresponding to any given STDP learning episode. This article focuses on additive STDP learning, although our formulation also applies to multiplicative STDP learning when an appropriate w-dependence to a window function is included. Throughout, we consider STDP learning of excitatory synapses assuming nonplastic inhibition. As discussed later, the extension of our formalism to STDP learning of inhibitory synapses is straightforward. We assume that a neuron receives synaptic inputs from thousands of excitatory presynaptic neurons. Accordingly, the neuron occasionally Žres a spike. The stochastic changes in thousands of synaptic conductances w are described by the following Fokker-Planck equation for the conductance distribution P.w; t/ (Tuckwell, 1988; Risken, 1996): 1 @2 @ P.w; t/ @ D¡ .A.w/P.w; t// C .B.w/P.w; t//: 2 @w2 @t @w

(2.1)

Here, time is measured by the membrane time constant (i.e., t RD time=¿m ). N The drift term R and the diffusion term are calculated as A.w/ D dr rT.w; r/ N N and B.w/ D dr r2 T.w; r/, respectively, from the transition probability T.w; r/ that the synaptic update w ! wCr occurs. As in Song et al. (2000), we assume that synaptic conductance is conŽned in 0 · w · wmax : if a synaptic update makes w go beyond these boundary values, w is reset to zero or wmax . This

Stochastic Method to Analyze STDP

601

is equivalent to solving equation 2.1 with a reecting boundary condition, that is, J.0; t/ D J.wmax ; t/ D 0; J.w; t/ D A.w/P.w; t/ ¡

1 @ .B.w/P.w; t//: 2 @w

(2.2)

Hereafter, we set wmax D 1 and measure the potential v in the unit of the maximal conductance. Since the transition amplitude r is related to the time interval t D tpost ¡tpre via a window function G.t/, A.w/ and B.w/ can be reexpressed as follows: Z A.w/ D

1 ¡1

Z dt G.t/T.w; t/ and B.w/ D

1

dt G.t/2 T.w; t/:

(2.3)

¡1

N Here, T.w; t/ D T.w; G.t//G0 .t/ is the probability that pre- and postsynaptic spikes are paired with the interval t. The Fokker-Planck equation depends on the shape of the window function G.t/, as indicated in equation 2.3. To deŽne STDP learning precisely, we must specify the rule by which each presynaptic spike is paired with individual spikes in a postsynaptic spike train. In Song et al. (2000), each presynaptic spike was paired with every postsynaptic spike, even in cases in which the postsynaptic spikes were separated by arbitrarily large intervals. In this article, however, to make analytical study more tractable, each presynaptic spike is paired only with the last and the next postsynaptic spikes. This rule is based on the assumption that a postsynaptic spike “resets” the internal state of a neuron almost completely. We must await future experiments to reveal which pairing rule is more relevant to actual synapses. We note that the two rules are essentially the same if the postsynaptic interspike interval (ISI) is larger than the width of the window function (the time difference for which G.t/ has a signiŽcant size). In this case, pairings considered beyond the postsynaptic ISI will contribute negligibly to each synaptic update. We call this condition the moderate Žring condition because the postsynaptic Žring rate cannot be too high for the condition to be met. We carry out all the calculations except some in appendix A under this moderate Žring condition. The activity regulation inherent in STDP learning (Song et al., 2000) tends to achieve a moderate postsynaptic Žring rate at the late stage of learning, validating our analysis of the steady state based on the moderate Žring condition. Now we describe T.w; t/: the probability of having pre- and postsynaptic spikes with interval t. We denote presynaptic (input) and postsynaptic (output) Žring rates as ¸in and ¸out , respectively. In the following argument, it is understood that a presynaptic spike occurs at t D 0, and a postsynaptic spike occurs at time t. In the acausal domain (t < 0), pre- and postsynaptic spikes can be regarded as independent events. Hence, T.w; t/ is equal to

602

H. Caˆ teau and T. Fukai

an accidental correlation resulting from the product of pre- and postsynaptic spikes. In the causal domain .t > 0/, in which pre- and postsynaptic spikes are causally related, the correlation acquires a small additional term proportional to C.t/ as follows (see appendix A for detailed calculations): ( T.w; t/ D

¸in ¸out ¸in ¸out .1 C wC.t//

t < 0; t ¸ 0:

(2.4)

The nonaccidental correlation C.t/ represents an effect that a single presynaptic spike at an arbitrarily chosen synapse advances a postsynaptic spike. We can calculate this effect by regarding other presynaptic spikes as background. Hence, we obtain C.t/ D ¡

1 ¸out

Z

µ

¡1

dv ’ 0.v/ f .v; µ; t/;

(2.5)

(see appendix A for the derivations). Here, ’.v/ is the probability of Žnding the membrane potential at v and f .v; µ; t/ is the Žrst passage time distribution—the probability of the membrane potential going from v to threshold µ in time t (Tuckwell, 1988; Risken, 1996). With an integrate-and-Žre neuron, the uctuations can be characterized by a stochastic process called the Ornstein-Uhlenbeck process—the Brownian motion with leak effects—which depends on the mean .m/ and the variance .s2 / of the total synaptic input to a neuron (Tuckwell, 1988). For the Ornstein-Uhlenbeck process, ’.v/ was analytically calculated (Brunel, Chance, Fourcaud, & Abbott, 2001) and an efŽcient algorithm for numerically calculating f .v; µ; t/ was developed (Plesser & Tanaka, 1997). Thus, C.t/ can be numerically evaluated as in Figure 1. Since a presynaptic spike at the referenced excitatory synapse always advances a postsynaptic spike, the probability of having a postsynaptic spike is enhanced in the neighborhood of t D 0.C.t/ > 0/; this enhancement must be compensated by the decreases in the probability for large t.C.t/ R 1 < 0/. This compensation is implied by the probability conservation 0 C.t/ dt D 0, which can be directly shown from equation 2.5. Note that jC.t/j ¿ 1 in almost every instance except in the neighborhood of t D 0, for C.t/ represents the impact of just a single presynaptic spike on the postsynaptic activity. From equations 2.3 and 2.4, we can express the drift and diffusion terms for an arbitrary window function as follows: Z A.w/=.¸in ¸out / D Z B.w/=.¸in ¸out / D

1 ¡1 1

¡1

Z dt G.t/ C w

1

Z dt G.t/2 C w

dt G.t/C.t/;

(2.6)

0

0

1

dt G.t/2 C.t/:

(2.7)

Stochastic Method to Analyze STDP

603

C(t)

0.2 0.15 0.1 0.05 1

2

3

4

5

time/t m Figure 1: The plot of the nonaccidental correlation C.t/ between pre- and postsynaptic spikes versus t D time=¿m . Given this function, the Fokker-Planck equation corresponding to any given STDP learning can be calculated. The p normalized form D 2 depends on only two parameters, x0 D C .t/ sC.t/= 0 p p 2.v0 ¡ m/=s; x1 D 2.µ ¡ m/=s, where v0 is the reset potential after Žring. We set x0 D ¡1, x1 D 1:5, and s D 7 here, which are typical values when synapse competition occurs.

Here, the higher-order terms in the w-expansion were dropped since C.t/ p is small, as explained above. (Since we can show that C.t/ » 1= t in the neighborhood of t D 0, the contributions of this region to the integrations are negligible). The stationary synaptic distribution PŽn .w/, which is reached by integrating the Fokker-Planck equation up to t D 1, can be obtained by setting the right-hand side of equation 2.1 equal to zero: @P.w; t/=@t D 0. Thus, from the above expressions, the Žnal synaptic distribution is given as PŽn .w/ D n0 .ew .1 C ¯w/¡1=¯ ¡wd /° ;

(2.8)

with Z w0 D ¡ Z ¯D Z ° D

1

dt G.t/

Z

¯

¡1

1

0

dt C.t/G.t/2

¯

0 1 0

dt C.t/G.t/

1

¯

Z

dt C.t/G.t/; 1

¡1

Z

1

dt G.t/2 ;

dt C.t/G.t/2 :

(2.9)

0

It is somewhat complicated to obtain P Žn .w/ for a given window function, since C.t/ depends on the mean m and the variance s2 of the membrane uctuations, whereas these quantities are given in terms of PŽn .w/

604

H. Caˆ teau and T. Fukai

(see appendix B for details). Therefore, we must solve the self-consistency equations given as m D °1 .m; s2 / and s2 D °2 .m; s2 /. 3 Analytical Expressions of the Synaptic Distributions for Single-Exponential Window Functions If the causal part of the window function is given by a single-exponential function, all the integrations deŽning the drift and diffusion terms in equation 2.9 reduce to the Laplace transform of C.t/, which can be expressed by R1 the Laplace transform fO.v; µ; ®/ D 0 dt e¡®t f .v; µ; t/ through equation 2.5. Unlike f .v; µ; t/ itself, fO.v; µ; ®/ has been analytically calculated (Tuckwell, 1988; Risken, 1996). Thus, the drift and diffusion terms can be analytically calculated as A¡ AC C C A.w/=.¸in ¸out / D ¡ ®¡ ®C A2¡ A2 C C C B.w/=.¸in ¸out / D 2®¡ 2®C

p p

2AC h.x1 ; ®C / w; s.®C C 1/

2A2C h.x1 ; 2®C / w; s.2®C C 1/

(3.1) (3.2)

R1 R1 2 2 where h.x1 ; ®/ D 0 dy y® e¡.y¡x1 / =2 = 0 dy y®¡1 e¡.y¡x1 / =2 (see appendix C). p This quantity depends on ® and x1 D 2.µ ¡ m/=s, which is a normalized distance between m and the threshold µ . The window function studied in Song et al. (2000) corresponds to ®C D ¿m =¿C D 1 and ®¡ D ¿m =¿¡ D 1, and in this case, the Žnal distribution of synapses takes a particularly simple form: P Žn .w/ D n0 exp.a.w ¡ w0 /2 /;

(3.3)

with 8 > :

2AC

A2C C A2¡

Á

! 2 2 2 µ ¡m e¡.µ¡m/ =s C p ; s2 s ¼ 1 C erf..µ ¡ m/=s/

aw0 D 4.A¡ ¡ AC /=.A2C C A2¡ /:

In equation 3.3, the exponent of PŽn .w/ was obtained up to O.w2 /. This expression is informative because the dependencies of the Žnal distribution on each model parameter are manifest. From equation 3.3, PŽn .w/ is expected to show two sharp peaks at w D 0 and w D 1 if 0 < w0 < 1. For the STDP window function adopted in Song et al., (2000), the temporal evolution of the conductance distribution starting with a gaussian distribution is shown in Figure 2 by numerically integrating the Fokker-Planck equation deŽned by equations 2.1, 3.1, and 3.2. As expected from analytical expression 3.3, the Žnal distribution of synapses exhibits two sharp peaks.

Stochastic Method to Analyze STDP

605

time/t m 105 103 101

8 6 4 2 0

0.5

1

w Figure 2: Temporal evolution of the conductance distribution obtained by numerical integration of the Fokker-Planck equation. The window function used here is the same as that used in Song et al. (2000, equation 1.1) with AC D 0:005, A¡ D 0:00525, and ®C D ®¡ D 1/. Synaptic conductances were initially distributed around value w D 0:8 with the standard deviation ¾ D 0:1 in a gaussian manner. The postsynaptic neuron model is deŽned by the membrane time constant ¿m , Žring threshold µ in units of maximal excitatory conductance, the reset potential v0 , R (ratio of the inhibitory synaptic conductance to the maximal excitatory conductance), the number of excitatory and inhibitory inputs .N; N 0 /, and their Žring rates .¸in ; ¸0in /. We used the setting, ¿m D 20 ms, µ D 20, v0 D 0, R D 1, and N D 2 £ 104 throughout this article. We set N 0 ¸0in D 80 and ¸in =¿m D 10 Hz to obtain the three-dimensional plot.

The previous numerical study (Song et al., 2000) showed that ¸out linearly depended on ¸in with a very gentle slope. It was also shown that ¸out is a monotonically decreasing concave curve as a function of A¡ =AC , and the coefŽcient of variation (CV) of postsynaptic spikes is a monotonically increasing convex curve approaching unity as a function of the same ratio. With the present method, ¸out and CV can be calculated as 8 Z .µ ¡m/=s p > > 2 ¡m/=s/¡ ¼ u..µ .1¡erf.y// exp.y2 / dy < 1 m=s Z m=s D p ¸out > > : ¼ .1¡erf.y// exp.y2 / dy .µ¡m/=s

.m < µ / (3.4) .µ · m/;

606

H. Caˆ teau and T. Fukai

with u.x/ D

Rx 0

exp.y2 / dy, and

q CV D

M2 ¡ M 21 =M1

(3.5)

R1 with Mj D 0 t j f .0; µ; t/ dt. We note that m and s2 depend on A¡ =AC , since they are determined by solving the self-consistency equation for each value of A¡ =AC (see the explanation following equation 2.9). Plots of ¸out and CV versus A¡ =AC shown in Figure 3 prove that the method successfully reproduces all of these unique parameter dependencies. 4 Conductance Distributions for Two Examples of STDP Learning Having validated our method, we now examine STDP learning that has not been examined in detail in the literature. 4.1 CA1-Type Window Function. We consider the window function 8 ¡® t 0 ¡® t
(4.1)

As with this, the analytical approach based on the Laplace transform is practically applicable to a wide class of window functions, since most of them can be expressed as linear combinations of exponential functions. As we noted in the previous section, m and s2 must be determined in a self-consistent manner. In Figure 4B, 1m D °1 .m; s2 / ¡ m is plotted against m with s2 Žxed. By substituting the unique value of m, at which this curve intersects with the m-axis, to the right-hand side of s2 D °2 .m; s2 /, we obtain the next candidate value for s2 . Then we again calculate a similar curve for this value for s2 and repeat the procedure. The values of m and s2 can be determined for m and s2 as the stable Žxed points of these operations.

Stochastic Method to Analyze STDP

607

A 16 lout/t (Hz) 14 12 10 8 6 4 2

lin/t (Hz) 15

B

50 40

20

25

30

35

lout/tm (Hz)

30 20 10 0 1.02

C 1 0.9

1.04

1.06

1.08

CV

0.8 0.7 0.6 1.02

A-/A+ 1.04

1.06

1.08

Figure 3: Consistency with previous numerical experiments. All the parameter values were the same as those used in Figure 2, except the values of ¸in . For each given value of ¸in in the range 10 Hz · ¸in =¿m · 40 Hz, m and s2 were evaluated by solving the self-consistency equations, and the Žnal distribution was determined using equation 2.8. (A) Firing rate ¸out of a postsynaptic neuron was determined and plotted against presynaptic Žring rate ¸in . (B, C) We set ¸in =¿m D 10 Hz, and for each value of A¡ =AC in the range 1:02 < A¡ =AC < 1:1, the Žnal distribution was determined. Postsynaptic Žring rate ¸out (B) and CV values of output Žring trains (C) were calculated and plotted against A¡ =AC . All of these properties coincided well with previous results from numerical experiments (Song et al., 2000).

608

H. Caˆ teau and T. Fukai

B

A 0.01

Dm 100

0.005

-4

4

2

-2

tim e/ t m

-0.005

50

6 -20

-10

m 10

20

30

-50 -100

C 8 6 4 2 0.2

0.4

0.6

0.8

1

w

Figure 4: CA1 type window function. A window function similar to that observed in area CA1 of the rat hippocampus (Nishiyama et al., 2000) was constructed as in A (see the text for its deŽnition), and the associated STDP learning was examined. (B) The 1m versus m plot for a Žxed value of s2 exhibited a unique, stable, Žxed point for m. (C) The Žnal distribution was determined as in equations 2.8 and 2.9 with ° D 206; ¯ D 0:207; w0 D 0:480. The dotted line representing drift term A.w/ versus w was superimposed on the Žnal distribution. A.w/ was scaled to facilitate visualization of the dotted line.

We found that the Žnal distribution is bimodal (see Figure 4C) and the ¸in dependence of ¸out is linear (not shown), as in the case of a single-exponential window function (see Figure 2). In fact, this is anticipated from the expression 4.1 of the drift term A.w/. In the present case, A.w/ is represented as a line that intersects the w-axis at w0 D 0:48 with a positive slope. Since the sign of the drift term represents the direction of the average conductance change at any particular w, w tends to increase on the right-hand side of the intersection point .w0 < w/, and w tends to decrease on the left-hand side .w < w0 /. Such a stochastic ow of w results in a bimodal distribution.

Stochastic Method to Analyze STDP

609

4.2 Electric Fish–Type Window Function. Next, we examine the window function » G.t/ D

0

A1 e¡®1 t ¡ A01 e¡®1 t ¢ ¢ ¢ t ¸ 0; ¡A2 e¡®2 jtj C A02 e¡®2 jtj ¢ ¢ ¢ t < 0;

with A1 D A01 D 0:005, ®1 D 4:5, ®10 D 1:5, A2 D A02 D 0:0026, ®2 D 2:25, and ®20 D 0:75 (see Figure 5A). The total depression component of this window funcition is slightly smaller than the total potentiation. We named it the electric Žsh type because of its similarity to the window function observed in electric Žsh (Bell et al., 1997). The drift term can be calculated analytically again. The electric Žsh–type window function exhibits potentiation in the acausal domain and depression in the causal domain, in contrast to the single exponential or the CA1-type window function. This reversal of the potentiation and depression roles transforms the stable Žxed point in the plot of 1m versus m (see Figure 4B) to an unstable Žxed point (see Figure 5B). This stability change makes electric Žsh–type STDP learning very different from that of the previous examples. To the right and left of the unstable Žxed point, two stable Žxed points appear. Hence, depending on the initial conductance distribution, m should be stabilized at either of these two Žxed points. We show examples for both cases. If synaptic conductances are initially concentrated around w0 D 0:2; m, eventually evolves into a Žxed point with a smaller value. In this case, the drift term represents a line intersecting the w-axis at w0 D 0:037 with a negative slope; w is thus distributed around this small value of w0 (see Figure 5C). On the other hand, if synaptic conductances are initially concentrated at w0 D 0:8; m evolves into a Žxed point with a larger value (m can be negative due to the inhibitory input). In this case, the drift term is always positive in 0 < w < 1, and therefore PŽn .w/ shows saturation at the maximal w (see Figure 5D). For the case shown in Figure 5C, the slope of the linear relation between ¸out and ¸in becomes steep, and the activity of postsynaptic neuron is not regulated moderately (see Figure 5.E). In numerical simulations corresponding to Figure 5C, the synaptic distribution actually obtained is seemingly different from what is shown here. As synapses are weakened, they are modiŽed only at very slow rates, since the postsynaptic neuron can respond only at very low Žring rates. Therefore, much time must elapse to obtain the conductance distribution predicted by the theory. More practically, the postsynaptic neuron ceases to Žre before the predicted distribution is reached. The existing experimental data on electric Žsh–type window functions are not precise enough to judge if the total depression is larger or smaller than the total potentiation. The above results of theoretical analyses show the qualitative differences between the two cases.

610

H. Caˆ teau and T. Fukai

5 Multiplicative STDP Learning For additive STDP learning, in which G.t/ is independent of w, the second term on the right-hand side of equation 2.6 is the only term linear in w. This term, although it is generally small, encodes the essential causal relationship between pre- and postsynaptic spikes. In contrast, in typical multiplicative STDP learning, in which the depression component .G.t/ < 0/ is multiplied by w and=or the potentiation component .G.t/ > 0/ is multiplied by .1 ¡ w/

B

A

Dm

0.003

60 40 20

0.002 -4

0.001

-2

4

2

-150 -100

time/t m

-0.001

-50

-20

50

100

-40 -60

-0.002

C

m

12 10 8 6 4 2 -0.2

0.2

-2

0.4

0.6

0.8

w

1

D

-7.8

-7.6

E 200

0.2

0.4

lout/t m (Hz)

150 100 50

lin/t m (Hz) 6

7

8

9

10

0.6

0.8

1

w

150

Stochastic Method to Analyze STDP

611

(Kistler & van Hemmen, 2000; van Rossum et al., 2000; Rubin et al., 2001), a large w-proportional term appears in the drift term from the Žrst term in the right-hand side of equation 2.6. The coefŽcient of this term—the Žrst term in equation 2.6—is generally large and always negative because w is multiplied by a negative number and 1 ¡ w is multiplied by a positive number. Therefore, the drift term will have a negative slope unless the w2 proportional term originating in the second term in equation 2.6 becomes exceptionally large for some reason. Consequently, the Žnal distribution quite generally becomes either a gaussian-like distribution .0 < w0 < 1/ or a saturated one (w0 < 0 or 1 < w0 ), where w0 is the intersection of the drift term with the w axis (see Figures 5C and 5D). Recently, it was demonstrated numerically that a gaussian-like distribution changes into a bimodal distribution when a continuous shift of the learning rule occurs (i.e., from a multiplicative rule to an additive rule; (G utig ¨ et al., 2002). 6 Crucial Determinant of the Final Conductance Distribution Let us consider the general condition for obtaining a bimodal, gaussian, or saturated conductance distribution. In order for the Žnal distribution to be bimodal, which leads to synaptic competition, the constant term of A.w/ in equation 2.6—the total area S0 encircled by the window function—must be negative, and the line representing A.w/ must intersect the w-axis at about w D 0:5 (see Figure 4) with a posture slope. For the latter condition to be satisŽed, the constant term and the coefŽcient of the w-proportional term in A.w/ must be comparable. However, this can be the case only when the net potentiation and depression effects represented by G.t/ are highly compensated to make the constant term small because C.t/ is, in general, small, as explained following Figure 5: Facing page. Electric Žsh–type window function. (A) A window function similar to that observed in the electric Žsh cerebellum-like structure (Bell et al., 1997) was constructed (see the text for its precise deŽnition), and associated STDP learning was examined using the same procedures and the same parameter values, with the exception N 0 ¸0in D 100 (the number times the rate of inhibitory inputs), used to construct Figures 3 and 4. (B) Unlike the previous cases, the 1m versus m plot for a Žxed value of s2 revealed one unstable and two stable Žxed points. As for s2 , a unique Žxed point was always found. (C) Small initial conductances (w is distributed around 0.2 with ¾ D 0:1) caused m to be Žxed at a small value, resulting in the concentration near w D 0 (° D 1360; ¯ D 0:441; w0 D 0:0377), whereas (D) large initial conductances (w is distributed around 0:8 with ¾ D 0:1) caused m to be Žxed at a large value, resulting in the concentration near w D 1.° D 1150; ¯ D 0:00254; w0 D ¡7:73/. (E) Despite the large difference between this type of learning and the singleexponential, or CA1-type learning, the ¸out versus ¸in plot is linear. A.w/ was scaled to facilitate visualization of the dotted line.

612

H. Caˆ teau and T. Fukai

equation 2.5. Meanwhile, if the total area S0 is positive, the Žnal distribution becomes either a gaussian or saturated distribution depending on whether the line representing A.w/ intersects the segment 0 < w < 1. These considerations show that the total area S0 is another crucial determinant of the Žnal conductance distribution. 7 General Properties of Additive STDP Learning Song et al. (2000) showed that STDP learning deŽned by their window function increased the correlation between pre- and postsynaptic spikes. Is the correlation always enhanced by STDP learning? In fact, we can derive the following general rule: The correlation between pre- and postsynaptic spikes increases when STDP learning is designed to lead to strong synaptic competition. R1 R1 When synaptic competition occurs, ¡1 G.t/ dt C w 0 G.t/C.t/ dt intersects near the middle point of the segment 0 < w < 1 with a positive slope. In fact, the upward ow of w to the right of the intersection and downward ow of w to the left of the intersection result in the synaptic competition. R1 Therefore, if synaptic competition is strong, 0 G.t/C.t/ dt must be a large, positive value. For this integral to be a large, positive value, G.t/ must be positive when C.t/ is positive and G.t/ must be negative when C.t/ is negative. This implies that the connections between positively correlated neuron pairs .C.t/ > 0/ are strengthened .G.t/ > 0/, and the connections between negatively correlated neuron pairs .C.t/ < 0/ are weakened .G.t/ < 0/. The overall correlation thereby increases, leading to the Žrst property, noted above. This property suggests that the correlation between pre- and postsynaptic spikes may not be enhanced for STDP learning that does not imply strong synaptic competition. The electric Žsh–type learning is a typical example of noncompetitive STDP learning. The correlation between pre- and postsynaptic spikes does not increase through this type of STDP learning. As shown in Figure 1, C.t/ changes its sign at The intersection t0 depends on two p some point t D t0 . p parameters x1 D 2.µ ¡m/=s and x0 D 2.v0 ¡m/=s ( v0 : the reset potential). When synaptic competition occurs, x1 and x0 , respectively, take positive and negative values of an order of unity. For such parameter values, we typically have t0 D time=¿m » D 1 (¿m : membrane time constant). In other words, C.t/ changes its sign at time » D ¿m D 10 » 20 ms. Therefore, a depression component .G.t/ < 0/ in the late causal domain .t0 < t/ contributes positively to synaptic competition, implying the second property: A depression component in the late causal domain .t0 < t/ of the window function enhances synaptic competition. This enhancement of synaptic competition is understood to result from the removal R 1 of less correlated synaptic connections. If we note that the slope 0 G.t/C.t/ dt measures the strength of synaptic R1 competition and that 0 G.t/C.t/ dt is maximized for G.t/ / C.t/, we Žnd

Stochastic Method to Analyze STDP

613

the third property: Synaptic competition is optimal when the causal part of a window function has the same shape as C.t/. 8 Discussion A recent experiment describing plasticity in the rat hippocampal slice preparation of CA1 (Nishiyama et al., 2000) revealed a window function whose causal part was similar to C.t/. Meanwhile, our analysis has revealed that the optimal window function for synaptic competition is proportional to C.t/. This suggests that the window function of the hippocampal CA1 pyramidal neurons may be designed to enhance synaptic competition in this area—that is, the competition between inputs from the CA3 neurons. This result raises the interesting possibility that memory traces loaded into CA3 are highly selected when they are passed to CA1 for further processing required for the consolidation in long-term memory storage. It is worth summarizing the differences between our method and previous ones. van Rossum et al. (2000) and Rubin et al. (2001) revealed the qualitative features of STDP learning with approximate drift and diffusion terms that were evaluated by assuming a piecewise linear neuron, a rectangular window function, simpliŽed forms of the correlation functions, and other mathematical simpliŽcations. We have taken the nonlinearity of neural Žring into account by using the Ornstein-Uhlenbeck process for membrane uctuations. The previous work stated that m and ¾ 2 are determined by solving the self-consistency equations. However, they did not need to solve these equations explicitly because they focused on the qualitative aspects of STDP learning. We have solved the self-consistent equation explicitly (see Figures 4B and 5B) and conducted quantitative analyses (see Figure 3). This article has also evaluated the Laplace transformed correlation to be » O C.®/ D .µ ¡ m/=s2 for the window function used in Song et al. (2000), which conŽrmed the assumption by Rubin et al. (2001) that this quantity should be a monotonically decreasing function of ¸out . Note that ¸out is roughly proportional to m. Levy et al. (2001) deŽned STDP learning as a stochastic process consisting of the usual synaptic update and the relaxation of synaptic conductance. Therefore, in the corresponding Fokker-Planck equation, the average change of w had an additional component arising from the relaxation. Their analysis of the Fokker-Planck equation yielded exponential distributions rather than bimodal ones in the no-relaxation limit. In their analysis, however, the average change of w originating from the correlation between preand postsynaptic spikes was not treated explicitly. The necessary condition for justifying this approximation is not obvious. Their deŽnition of STDP is by itself biologically interesting and may be tested by experiment. If the relaxation effect exists in STDP learning, it might dramatically inuence the distribution of synapses.

614

H. Caˆ teau and T. Fukai

Kempter, Gerstner, and van Hemmen (2001) derived what is conceptually similar to equation (2.5), which deŽnes C.t/, and discussed its approximate structure. A great advantage of our approach is that we can evaluate the equation exactly as in Figure 1 by using the Ornstein-Uhlenbeck process. From the obtained expression of equation 2.5, we derived the three general properties of STDP learning. p p O We can show that C.t/ ! 1= t for t ! 0 from C.®/ ! 1= ® for ® ! 1 (see appendix C), that is, C.t/ changes steeply around t D 0. Since several experimentally observed window functions display a steep change around t D 0 (Abbott & Nelson, 2000), our analysis predicts that any modulation of a spike-generation mechanism or backpropagation mechanism must have a signiŽcant impact on STDP learning. Under the moderate Žring condition assumed throughout this study, the acausal part of G.t/ appears in the Fokker-Planck equation only through R0 R1 the Žrst part of the total integral S0 D ¡1 G.t/ dt C 0 G.t/ dt. This implies that the shape of the acausal part is unimportant under this condition. However, when this condition is violated, S0 must be replaced by R1 R jtj 0 0 ¡1 G.t/.1 ¡ 0 dt f .v0 ; µ; t // dt (see appendix A). Therefore, the explicit shape of the acausal part of the window function becomes important. Since the second factor in the integrand approaches zero for jtj being much greater than the average interspike interval T D 1=¸out , the integral is nearly equal RT to ¡T G.t/ dt. Our method is applicable to STDP learning of inhibitory synapses provided that appropriate changes are made in the calculations. First, the selfconsistency equations must be changed to 8 Z 1 > 0 0 > < m D N¸in R ¡ N ¸in dw wP.w; t/; Z0 1 > > :s2 D N¸in R2 C N0 ¸0in dw w2 P.w; t/; 0

where R now represents the number of random inputs mediated by (nonplastic) excitatory synapses. Second, equation 2.4 must be replaced with ( ¸in ¸out t < 0; T.w; t/ D ¸in ¸out .1 ¡ wC.t// t ¸ 0; since timing of postsynaptic spikes is negatively correlated with that of presynaptic spikes mediated by inhibitory synapses. Here, we do not repeat the detailed calculations for STDP learning of inhibitory synapses. In summary, this study provided a powerful analytical tool to study how a group of synapses on a postsynaptic neuron changes according to STDP learning with an arbitrary window function. We have identiŽed the window function that optimizes competition between synapses. For a more complete understanding of STDP, it is important to clarify how STDP learning reorganizes a recurrent network of neurons in an activity-dependent way. Such

Stochastic Method to Analyze STDP

615

an issue has been studied numerically by several authors (Amemori & Ishii, 1998; Levy et al., 2001; Kitano et al., 2002). Besides an asynchronous activity at a moderate Žring rate, a synŽre chain-like activity (Abeles, 1991; Diesmann, Gewaltig, & Aertsen, 1999; Cˆateau & Fukai, 2001) was self-organized under a certain condition. How to apply our method to STDP learning in mutually connected neurons is open for further study. Appendix A N This appendix calculates T.w; t/ for negative and positive t. Suppose that f .v0 ; µ; t/ denotes the ISI distribution of a postsynaptic neuron with reset point v0 and Žring threshold µ. If a check mark is indicated on a time axis every time a postsynaptic neuron Žres, the lengths of the resulting time intervals on the time axis will be distributed according to f .v0 ; µ; t/. A.1 Acausal Domain. For t D tpost ¡ tpre < 0, pre- and postsynaptic spikes are not causally related. Therefore, T.w; t/ is estimated by supposing that a presynaptic spike occurs at any time point on the checked axis considered above with equal probability. SpeciŽcally, T.w; t/ is equal to the probability that an arbitrarily chosen point (presynaptic spike) on the checked axis is located at distance jtj from the last check mark (postsynaptic spike). Our pairing rule (see section 2) takes only the last check mark into account. Among all the intervals on the checked axis, only ones longer than jtj can accommodate a point representingR a presynaptic spike considered above. 1 Hence, we Žnd that T.w; t/=¸in / jtj dt 0 f .v0 ; µ; t0 /. Normalization of it for R0 a given presynaptic Žring event, ¡1 dt0 T.w; t/=¸in D 1, determines T.w; t/ R jtj R jtj as T.w; t/ D ¸in ¸out .1 ¡ 0 dt 0 f .v0 ; µ; t0 //. The term 0 dt0 f .v0 ; µ; t0 / signiŽes that the contribution from pairings beyond the postsynaptic ISI was removed. Under the moderate Žring condition, this term can be neglected, because when this term is large, G.t/ becomes small. A.2 Causal Domain. For t D tpost ¡tpre ¸ 0, pre- and postsynaptic spikes are causally related: a presynaptic spike advances the occurrence of a postsynaptic spike. Postsynaptic membrane potential v uctuates due to random excitatory and inhibitory inputs from thousands of presynaptic neurons. When a presynaptic neuron Žres a spike, membrane potential v of a postsynaptic neuron abruptly increases to v C w due to the synaptic input from a synapse with conductance w. This neuron elicits a spike at time t with a probability f .v C w; µ; t/. Summing this probability over all possible values of v with probability weight ’.v/ (the probability of Žnding membrane potential at v) results in the total probability of a postsynaptic spike occurring t after a R µ ¡w presynaptic spike, T.w; t/ D ¸in ¡1 dv ’.v/ f .v C w; µ; t/.

616

H. Caˆ teau and T. Fukai

A change of variable and an expansion of this in w up to the linear order result in Z T.w; t/=¸in » D

µ ¡1

Z dv ’.v/ f .v; µ; t/ ¡ w

µ

¡1

dv ’ 0.v/ f .v; µ; t/:

The second term on the right-hand side of this equation represents how much the probability of postsynaptic Žring increases due to one presynaptic input of size w. This probability increase resulting from only a single input is understood to be small if we consider that 10 to 100 presynaptic inputs are required to elicit neural Žring. In fact, the Laplace transform of the above expression calculated in equation 3.1 shows that the linear term in w is about 1=s times smaller than the constant term, where s (standard deviation of the membrane potential uctuations) is of the same order as µ (the number of excitatory postsynaptic potentials needed to Žre a neuron from the rest). Therefore, neglecting higher-order terms in w is a good approximation. The Žrst term on the right-hand side equals T.w D 0; t/, which is the Žring probability when no speciŽc input other than the background is speciŽed. Therefore, this Žrst term equals ¸out We note that ’.v/ and f .v; µ; t/ appearing in the above expression can be calculated analytically (see Brunel et al., 2001) and numerically (Plesser & Tanaka, 1997), respectively. Appendix B This appendix explains that membrane potential uctuations can be described by the Ornstein-Uhlenbeck process (Tuckwell, 1988; Risken, 1996). When a neuron receives random N excitatory inputs with their sizes distributed as P.w; t/.1 · w · wmax D 1/ and random N0 inhibitory inputs with their sizes being R in units of maximal excitatory conductance, at average rates of ¸in and ¸0in , respectively, membrane potential uctuations are characterized by the Ornstein-Uhlenbeck process deŽned by the stochastic differential equation, dv D ¡.v ¡ m/ dt C s dW. Here, m and s2 are written as 8 Z 1 > > D m N¸ dw wP.w; t/ ¡ RN0 ¸0in ; < in 0 Z 1 > > :s2 D N¸in dw w2 P.w; t/ C R2 N 0¸0in : 0

Since ’.v/ and f .v; µ; t/ depend on m and s2 , the drift and diffusion terms in the Fokker-Planck equation also depend on m and s2 (see appendix A), and, consequently, its solution P.w; t/ also depends on m and s2 . Therefore, the above equations imply self-consistency equations for m and s2 of the form m D °1 .m; s2 / and s2 D °2 .m; s2 /.

Stochastic Method to Analyze STDP

617

Appendix C This appendix shows how to calculate the Laplace transform of C.t/ analytically. It is known that ’.v/ and fO.v; µ; ®/ solve backward and forward Kolmogorov equations (Risken, 1996): ³ ´ @ s2 @ 2 ¡.v ¡ m/ C fO.v; µ; ®/ D ® fO.v; µ; ®/; 2 @v2 @v ³ ´ @ s2 @ 2 ..v ¡ m/¢/ C ’.v/ D ¡¸out ±.v ¡ v0 /: 2 @v2 @v This implies that F.w; ®/ D ential equation

Rµ

O

¡1 dv ’.v ¡ w/ f .v; µ; ®/ satisŽes

a linear differ-

» ¼ s2 O0 C D ¡ ¡ C .® w@=@w/F.w; ®/ .µ m/ f .µ ; µ; ®/ ’.µ ¡ w/ 2 ¡

s2 0 ’ .µ ¡ w/ ¡ ¸out fO0 .w C v0 ; µ; ®/: 2

This is solved to give F.w; ®/=¸out D

1 ¡ fO.v0 ; µ; ®/ fO0 .µ ; µ; ®/ ¡ fO0 .v0 ; µ; ®/ C w C ¢¢¢: ® ®C1

Under the moderate Žring condition, fO.v0 ; µ; ®/ ¿ 1 and fO0 .v0 ; µ; ®/ ¿ fO0 .µ ; µ; ®/, we have F.w; ®/=¸out D

1 fO0 .µ ; µ; ®/ C w: ® ®C1

Meanwhile, fO.v; µ; ®/ of the Ornstein-Uhlenbeck process deŽned by .m; s2 / is expressed using the corresponding function fO0 .x; x1 ; ®/ of the standard Ornstein-Uhlenbeck process deŽned by .m; s2 / D .0; 2/ as p p fO.v; µ; ®/ D fO0 .x D 2.v ¡ m/=s; x1 D 2.µ ¡ m/=s; ®/: p Correspondingly, we have fO0 .µ; µ; ®/ D 2=s fO00 .x1 ; x1 ; ®/, where fO0 .x; x1 ; ®/ is calculated as fO0 .x; x1 ; ®/ D Á.x; ®/=Á.x1 ; ®/; using a solution to .@ 2 =@x2 ¡ x@=@x ¡ ®/Á .x; ®/ D 0 with Á.0; ®/ D 1; Á.x; 0/ D 1:

(C.1)

618

H. Caˆ teau and T. Fukai

The solution can be obtained as Á .x; ®/ D

1 X 1 2n=2 0.®=2 C n=2/xn : 0.®=2/ nD0 n!

(C.2)

The series expansion in terms of the gamma function 0.x/ can be trans2 formed into an integral form to give Á .x; ®/ D ex =2 9.x; ®/=.0.®=2/2®=2¡1 /, R1 2 where 9.x; ®/ D 0 dy y®¡1 e¡.y¡x/ =2 . With the recursion formula derived p from equations C.1 and C.2, @ Á.x; ®/=@ x D 20..® C 1/=2/= 0.®=2/Á .x; ® C 1/, we obtain Z 1 Z ¯ 1 2 2 fO00 .x1 ; x1 ; ®/ D h.x1 ; ®/ D dy y® e¡.y¡x1 / =2 dy y®¡1 e¡.y¡x1 / =2 : 0

0

From this expression, we can show p fO0 .µ ; µ; ®/ ’ .µ ¡ m C .µ ¡ m/2 C 2s2 .® ¡ 1/=s2 for large ®. References Abbott, L. F., & Nelson, S .B. (2000). Synaptic plasticity: Taming the beast. Nature Neuroscience, 3, 1178–1183. Abeles, M. (1991). Corticonics: Neural circuits of the cerebral cortex Cambridge: Cambridge University Press. Amemori, K., & Ishii, S. (1998). Self-organizing network learning for submillisecond temporal coded information. In S. Usui & T. Omori (Eds.), Proceedings of the Fifth International Conference on Neural Information Processing (Vol. 3, pp. 1285–1288). Burke, VA: IOS Press. Bell, C. C., Han, V. G., Sugawara, Y., & Grant, K. (1997). Synaptic plasticity in a cerebellum-like structure depends on temporal order. Nature, 387, 278–281. Bi, G.-Q., & Poo, M.-M. (1998). Activity-induced synaptic modiŽcations in hippocampal culture, dependence on spike timing, synaptic strength and cell type. Journal of Neuroscience, 18, 10464–10472. Brunel, N., Chance, F. S., Fourcaud, N., & Abbott, L. F. (2001). Effects of synaptic noise and Žltering on the frequency response of spiking neurons. Physical Review Letters, 86, 2186–2189. Cˆateau, H., & Fukai, T. (2001). The Fokker-Planck approach to the pulse packet propagation in synŽre chain. Neural Networks, 14, 675–685. Debanne, D., Gahwiler, B. H., & Thompson, S. M. (1994). Asynchronous preand postsynaptic activity induces associative long-term depression in area CA1 of the rat hippocampus in vitro. Proceedings of National Academy of Science (USA), 91, 1148–1152. Debanne, D., Gahwiler, B. H., & Thompson, S. M. (1998). Long-term synaptic plasticity between pairs of individual CA3 pyramidal cells in rat hippocampal slice cultures. Journal of Physiology (London), 507, 237–247.

Stochastic Method to Analyze STDP

619

Diesmann, M, Gewaltig, M. O., & Aertsen, A. (1999). Stable propagation of synchronous spiking in cortical neural networks. Nature, 402, 529– 533. Egger, V., Feldmeyer, D., & Sakmann, B. (1999). Coincidence detection and efŽcacy changes in synaptic connections between spiny stellate neurons of the rat barrel cortex. Nature Neuroscience, 2, 1098–1105. Feldman, D. E. (2000). Timing-based LTP and LTD at vertical inputs to layer II/III pyramidal cells in rat barrel cortex. Neuron, 27, 45–56. Gustafsson, B., Wigstrom, H., Abraham, W. C., & Huang, Y.-Y. (1987). Longterm potentiation in the hippocampus using depolarizing current pulses as the conditioning stimulus to single volley synaptic potentials. Journal of Neuroscience, 7, 774–780. Gutig, ¨ R., Aharonov, R., Rotter, S., & Sompolinsky, H. (2002).Learning input correlations through non-linear temporallyasymmetricHebbian plasticity.Unpublished manuscript, Albert-Ludwigs-University. Kempter, R., Gerstner, W., & van Hemmen, J. L. (2001). Intrinsic stabilization of output rates by spike-based Hebbian learning. Neural Computation, 13, 2709–2741. Kistler, W. M., & van Hemmen, J. L. (2000). Modeling synaptic plasticity in conjunction with the timing of pre- and postsynaptic action potentials. Neural Computation, 12, 385–405. Kitano, K., Caˆ teau, H., & Fukai, T. (2002). Self-organization of memory activity through spike-timing-dependent plasticity, Neuro Report, 13, 795–798. Levy, N., Horn, D., Meilijson, I., & Ruppin, E. (2001). Distributed synchrony in a cell assembly of spiking neurons. Neural Networks, 14, 815–824. Levy, W. B., & Steward, D. (1983). Distributed synchrony in a cell assembly of spiking neurons: Temporal contiguity requirements for long-term associative potentiation/depression in the hippocampus. Neuroscience, 8, 791–797. Magee, J. C., & Johnston, D. (1997). A synaptically controlled, associative signal for Hebbian plasticity in hippocampal neurons. Science, 275, 209–213. Markram, H., Lubke, J., Frotscher, M., & Sakmann, B. (1997). Regulation of synaptic efŽcacy by coincidence of postsynaptic APs and EPSPs. Science, 275, 213–215. Nishiyama, M., Hong, K., Mikoshiba, K., Poo, M.-M., & Kato, K. (2000). Calcium stores regulate the polarity and input speciŽcity of synaptic modiŽcation. Nature, 408, 584–588. Plesser, H. E., & Tanaka, S. (1997). Stochastic resonance in a model neuron with reset. Physics Letters, A225, 228–237. Risken, H. (1996). The Fokker-Planck equation. New York: Springer-Verlag. Rubin, J., Lee, D. D., & Sompolinsky, H. (2001). Equilibrium properties of temporally asymmetric Hebbian plasticity. Physical Review Letters, 86, 364– 367. Song, S., Miller, K. D., & Abbott, L. F. (2000). Competitive Hebbian learning through spike-timing-dependent synaptic plasticity. Nature Neuroscience, 3, 919–926. Tuckwell, H. (1988). Introduction to theoretical neurobiology. Cambridge: Cambridge University Press.

620

H. Caˆ teau and T. Fukai

van Rossum, M. C. W, Bi, G. Q., & Turrigiano, G. G. (2000). Stable Hebbian learning from spike timing-dependent plasticity. Journal of Neuroscience, 20, 8812–8821. Zhang, L. I., Tao, H. W., Holt, C. E., Harris, W. A., & Poo, M.-M. (1998). A critical window for cooperation and competition among developing retinotectal synapses. Nature, 395, 37–44. Received January 25, 2002; accepted August 1, 2002.

LETTER

Communicated by Peter Latham

Permitted and Forbidden Sets in Symmetric Threshold-Linear Networks Richard H. R. Hahnloser [email protected]

H. Sebastian Seung [email protected] Howard Hughes Medical Institute, Department of Brain and Cognitive Sciences, MIT E25-210, Cambridge, MA 02139, U.S.A.

Jean-Jacques Slotine [email protected] Department of Mechanical Engineering and Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, U.S.A.

The richness and complexity of recurrent cortical circuits is an inexhaustible source of inspiration for thinking about high-level biological computation. In past theoretical studies, constraints on the synaptic connection patterns of threshold-linear networks were found that guaranteed bounded network dynamics, convergence to attractive Žxed points, and multistability, all fundamental aspects of cortical information processing. However, these conditions were only sufŽcient, and it remained unclear which were the minimal (necessary) conditions for convergence and multistability. We show that symmetric threshold-linear networks converge to a set of attractive Žxed points if and only if the network matrix is copositive. Furthermore, the set of attractive Žxed points is nonconnected (the network is multiattractive) if and only if the network matrix is not positive semideŽnite. There are permitted sets of neurons that can be coactive at a stable steady state and forbidden sets that cannot. Permitted sets are clustered in the sense that subsets of permitted sets are permitted and supersets of forbidden sets are forbidden. By viewing permitted sets as memories stored in the synaptic connections, we provide a formulation of long-term memory that is more general than the traditional perspective of Žxed-point attractor networks. There is a close correspondence between threshold-linear networks and networks deŽned by the generalized Lotka-Volterra equations.

Neural Computation 15, 621–638 (2003)

c 2003 Massachusetts Institute of Technology °

622

R. Hahnloser, H. Seung, and J. Slotine

1 Introduction Recurrent networks of neurons are believed to play an important role in the perceptual interpretation and memorization of sensory events. The dynamics of simpliŽed networks are often studied in a regime where as a response to constant sensory stimulation, neural activities converge to a steady state. Such networks can efŽciently implement complex nonlinear mappings between input stimulation and response activity. In a very general framework, we characterize the geometry of potentially multiple stable, steady responses and show that in addition to the dependence on synaptic connections, steady responses also depend on the inherent ambiguity of the sensory input. A Lyapunov function can be used to prove that a Žxed point of a set of differential equations is stable. By extension, for some differential equations that have many distinct Žxed points, Lyapunov-like functions can be constructed with multiple local minima, each corresponding to a stable Žxed point. These local minima were previously interpreted as memorized patterns that can be recalled by suitable initialization. Lyapunov theory applies mainly to symmetric networks in which neurons have monotonic activation functions (HopŽeld, 1984; Cohen & Grossberg, 1983). Here we show that the restriction of activation functions to threshold linear ones can yield new insights into the computational behavior of recurrent networks. The results we present elaborate on previous work (Hahnloser, Sarpeshkar, Mahowald, Douglas, & Seung, 2000; Hahnloser & Seung, 2001). We present three main theorems about the neural responses to constant inputs. The Žrst theorem provides necessary and sufŽcient conditions on the synaptic weight matrix for the existence of a set of globally attractive Žxed points. These conditions can be expressed in terms of copositivity, a concept from quadratic programming and linear complementarity theory. Alternatively, they can be expressed in terms of certain eigenvalues and eigenvectors of submatrices of the synaptic weight matrix, making a connection to linear systems theory. The theorem guarantees that the network will produce a steady-state response to any constant input. We regard this response as the computational output of the network, and its characterization is the topic of the second and third theorems. In the second theorem, we introduce the idea of permitted and forbidden sets. Under certain conditions on the synaptic weight matrix, we show that there exist sets of neurons that are “forbidden” by the recurrent synaptic connections from being coactivated at a stable steady state, no matter what input is applied. Other sets are “permitted,” in the sense that they can be stably coactivated for some input. Our main result is that if and only if a network possesses a forbidden set is it conditionally multiattractive, that is, there exists an input for which there is a nonconnected set of attractive Žxed points, each of which can be retrieved by suitable initial conditions.

Permitted and Forbidden Sets

623

Hence, we Žnd that forbidden sets and conditional multiattractiveness are inseparable concepts. The conceptual importance of permitted and forbidden sets suggests a new way of thinking about memory in neural networks. When an input is applied, the network must select a set of active neurons, and this selection is constrained to be one of the permitted sets. Therefore, the permitted sets can be regarded as memories stored in the synaptic connections. The memory storage in threshold-linear networks is limited by the topological properties of the memories. Our third theorem states that there are constraints on the groups of permitted and forbidden sets that can be stored by a network. No matter which learning algorithm is used to store memories, active neurons cannot arbitrarily be divided into permitted and forbidden sets, because subsets of permitted sets have to be permitted and supersets of forbidden sets have to be forbidden. In the appendix, we show that symmetric threshold-linear networks have the same set of attractive Žxed points as do networks deŽned by the generalized Lotka-Volterra equations. 2 Threshold-Linear Network Equations Our theory is applicable to the network dynamics, 2 3C X dxi C xi D 4bi C Wij xj 5 ; dt j

(2.1)

where [u]C D maxfu; 0g is a (half-wave) rectiŽcation nonlinearity and the synaptic weight matrix is symmetric, Wij D W ji . The dynamics can also be written in a more compact matrix vector form as xP C x D [b C Wx]C . The state of the network is x. An input to the network is an arbitrary vector b. An output of the network is a steady state x.xP D 0/ in response to b. The existence of outputs and their relationship to the input are determined by the synaptic weight matrix W. Threshold-linear units were introduced to neural modeling by Hartline and Ratliff (1958). Under certain circumstances, networks of conductance-based model neurons can be reduced to nonspiking dynamics like equation 2.1 (Ermentrout, 1994). In this context, the term in square brackets is interpreted as a spike rate, whereas xi is interpreted as a synaptic activation variable. DeŽnition 1. A vector v is said to be nonnegative, v ¸ 0, if all of its components are nonnegative. Similarly, a vector is said to be positive if all of its components are positive. The nonnegative orthant fv: v ¸ 0g is the set of all nonnegative vectors.

624

R. Hahnloser, H. Seung, and J. Slotine

It can be shown that any trajectory of equation 2.1 starting in the nonnegative orthant remains in the nonnegative orthant. Therefore, for simplicity, we will consider initial conditions that are conŽned to the nonnegative orthant. 3 Convergence to an Equilibrium The following theorem establishes necessary and sufŽcient conditions for a symmetric network of the form in equation 2.1 to converge always to a steady state. DeŽnition 2. Let A be an N £ N matrix, and let 6 D .i1 ; : : : ; ik / be a set of k ordered indices, 1 · i1 < i2 < ¢ ¢ ¢ < ik · N. The matrix A6 is said to be a k £ k 6 submatrix of A if A6 uv D Aiu jv for 8u; v 2 6. The matrix A can be constructed from A simply by removing from A all rows and columns not indexed by 6. Theorem 1. alent:

Assuming that W is symmetric, the following statements are equiv-

1. All positive eigenvectors of all submatrices of I¡W have positive eigenvalues. 2. The matrix I ¡ W is copositive; that is, xT .I ¡ W/x > 0 for all nonnegative x, except x D 0.

3. For all constant b and all initial conditions, the dynamics converge to an equilibrium point. Proof. ² Statement 2 follows from statement 1. We will assume that I ¡ W is not copositive and show that there exists a positive eigenvector of a submatrix of W with nonpositive eigenvalue. Let v¤ be the minimum of Q D vT .I ¡ W/v over nonnegative v on the unit sphere, v¤ D argminv¸0 G, where G D Q ¡ ®vT v (® is a Lagrange multiplier). DeŽne by 6 D .i1 ; : : : ; ik / the k positive components of v¤ , v¤i26 > 0. The gradient rG has to vanish in these components,P and so for all i 2 P 6, we have the Kuhn-Tucker conditions ®v¤i D v¤i ¡ j Wij vj¤ D v¤i ¡ j26 Wij vj¤ . Thus, the vector v¤i26 is a positive eigenvector of the matrix .I ¡ W/6 with eigenvalue ®. And because the minimum Q.v¤ / D ® is nonpositive by assumption, we Žnd that the eigenvalue of this positive eigenvector is nonpositive.

² Statement 3 follows from statement 2. We show that given the copositivity of I¡W, we can Žnd a Lyapunov-like function that is strictly decreasing under the dynamics in equation 2.1 and is zero only at steady states. By the copositivity of I ¡ W, the function L D 12 xT .I ¡ W/x ¡ bT x is lower bounded and radially unbounded in the nonnegative orthant, x ¸ 0. For all Žxed b, L is nonincreasing under the network dynamics:

Permitted and Forbidden Sets

625

LP D xP T .x ¡ Wx ¡ b/ D ¡.x ¡ [Wx C b]C /T .x ¡ Wx ¡ b/ D ¡.x ¡ yC /T .x ¡ y/ · 0, where y D Wx C b. This last inequality follows from the fact that all components of x and y contribute to the decrease of L: (1) if a component i satisŽes yi ¸ 0, we have that yC i D yi and so 2 ¡.xi ¡yC i /.xi ¡yi / D ¡.xi ¡yi / · 0; (2) if a component j satisŽes yj < 0, we have that ¡.xj ¡ yjC /T .xj ¡ yj / D ¡xj .xj ¡ yj / · 0 (because xj ¸ 0). All equilibria of L correspond to steady states of the dynamics of equation 2.1. From LP D ¡.x ¡ yC /T .x ¡ y/ D 0, it follows that for every component i, we have that either .xi ¡ yC i / D 0 or .xi ¡ yi / D 0. If P i D 0; if xi ¡ yi D 0, then we must have xi ¡ yC i D 0, then by deŽnition x P i D 0. that yi D yC i , which by positivity of xi implies x Hence, for all b and all initial conditions, the network converges to an equilibrium. ² Statement 1 follows from statement 2. By contradiction, assume there is a positive eigenvector v6 of a submatrix .I ¡ W/6 with eigenvalue ¸ · 0. DeŽne the vector v by vi D v6 i if i 2 6 and vi D 0 otherwise. Then vT .I ¡ W/v D ¸vT v · 0, and therefore I ¡ W is not copositive.

² Statement 2 follows from statement 3. By contradiction, assume that I ¡ W is not copositive. From the equivalence of statement 2 with statement 1, we know that the positive components 6 of the minimum v¤ D argminv¸0 G correspond to a positive eigenvector of the submatrix .I ¡ W/6 with nonpositive eigenvalue ® · 0. According to the Kuhn-Tucker conditions, minimization also implies that the gradient ¤ ¤ rG restricted to the vanishing P components P Z of v¤ (vi2Z D 0) has to be ¤ ¤ ¤ nonnegative: ¡®vi C vi ¡ j W ij vj D ¡ j Wij vj ¸ 0 for i 2 Z. Thus, by choosing inputs b6 D v¤6 and bZ D 0 as well as initial conditions x6 .0/ D v¤6 and xZ .0/ D 0, we have constructed a nonconverging trajectory: if ® < 0, this trajectory is given by x6 .t/ D .e¡®t C ®1 /v¤6 and xZ .t/ D 0; if ® D 0, it is given by x6 .t/ D tv¤6 and xZ .t/ D 0. The function L that was constructed for the proof of theorem 1 is locally a Lyapunov function in the sense that the stable Žxed points of the dynamics correspond to local minima of L. Conversely, we also have that all local minima of L correspond to stable Žxed points. We will again make use of L in the next section, where we characterize the geometry of attractive Žxed points. The fact that L is a quadratic function will be particularly useful. Theorem 1 states sufŽcient and necessary conditions that guarantee bounded dynamics, the absence of limitcycles, and the absence of chaos. The Žrst two conditions for convergence in theorem 1 are best appreciated by comparison with the conditions for convergence of a purely linear network constructed from equation 2.1 by dropping the rectiŽcation nonlinearity. Let us Žrst consider statement 1: in a linear network, all eigenvalues of I ¡ W have to be positive to ensure convergence. In theorem 1, only eigenvalues of nonnegative eigenvectors have to be positive to ensure convergence. Eigen-

626

R. Hahnloser, H. Seung, and J. Slotine

values of nonpositive eigenvectors may well be nonpositive, a fact that will turn out to be important for multistability, as we shall see. But all submatrices of I ¡ W must be considered in theorem 1, because different sets of feedback connections are active for different sets of neurons that are above threshold. Let us now consider statement 2: in a linear network, I ¡W would have to be positive deŽnite to ensure asymptotic stability, but because of the rectiŽcation, in theorem 1 positive deŽniteness is replaced by the weaker condition of copositivity. Although theorem 1 implies convergence to a Žxed point, it does not imply that there is only a single Žxed point. As we shall see, for example, there can be an attractive line of Žxed points (a line attractor) toward which the dynamics must converge. Nevertheless, theorem 1 excludes line attractor networks of the integrator type (Seung, 1996), that is, where there exists a positive eigenvector with zero eigenvalue, such as in xP C x D [x C b]C . In such networks, a positive input is indeŽnitely integrated, resulting in unbounded behavior. Lemma 1. For any nonnegative vector v ¸ 0, there exists an input b, such that v is a steady state of equation 2.1 with input b. Proof. Choose b D .I ¡ W/v. Lemma 1 states that any nonnegative vector can be realized as a Žxed point. Sometimes this Žxed point is stable, such as in symmetric and copositive networks in which only a single neuron is active. Indeed, the submatrix of I ¡ W corresponding to a single active neuron corresponds to a diagonal element, which according to statement 1 of theorem 1 must be positive. Hence, it is always possible to activate only a single neuron at an asymptotically stable Žxed point. However, as will become clear from theorem 2, not all nonnegative vectors can be realized as an asymptotically stable Žxed point. 4 Forbidden and Permitted Sets In this section, we present a geometrical description of the set of attractive Žxed points by introducing the notion of forbidden and permitted sets of neurons. Recall Žrst the classical deŽnition of stability in the sense of Lyapunov. DeŽnition 3. A steady state x is stable if for all balls B with x 2 B, there exists a ball A with x 2 A, such that for any initial condition x.0/ 2 A, we have that x.t/ 2 B for all times t ¸ 0. A steady state is asymptotically stable if it is stable and there is a ball of initial conditions that converge to it.

Permitted and Forbidden Sets

DeŽnition 4.

627

A neuron is said to be activated if its activity is nonzero.

DeŽnition 5. A set of neurons is permitted if the neurons can be coactivated at a stable steady state for some input b. A set of neurons is forbidden if it is not permitted, that is, if the neurons cannot be coactivated at a stable steady state no matter what the input b. According to these deŽnitions, permitted and forbidden sets are network characteristics that are independent of the input. Their signiŽcance can best be appreciated by comparison to linear systems. In a linear network, any set of neurons is permitted if and only if all eigenvalues of I ¡ W are nonnegative (the linear system is stable). This means that either all sets of neurons are permitted or none. For a threshold-linear network, this is not the case. It is possible that some sets are permitted, whereas others are forbidden. Whether a particular set is permitted depends on the eigenvalues of .I ¡ W/6 . For example, a given set 6 is forbidden means that .I ¡ W/6 must have at least one negative eigenvalue. And by copositivity of I ¡ W, we know that the corresponding eigenvector cannot be nonnegative and so must have both positive and negative components. DeŽnition 6. A threshold-linear network is said to be conditionally multiattractive if there exists an input b such that the set of stable equilibrium points is disconnected. Multiattractiveness is a statement about the topology of stable Žxed points. The word conditional refers to the fact that multiattractiveness exists only for certain inputs, but not for others. For example, consider a network and a particular input for which all stable Žxed points lie on a line. The network is not multiattractive for this input, because from any point of the line, any other point of the line can be reached by staying on the line only (i.e., a line is connected). However, it may well be that for another input, the network has two stable Žxed points that are separate from each other, which would characterize the network as conditionally multiattractive. To prove our theorems about conditional multiattractiveness and forbidden sets, we will need the following lemma about the minimization of convex functions over convex sets (Bertsekas, 1995). DeŽnition 7. A set C is convex if ®x C .1 ¡ ®/y 2 C for all x, y 2 C, and 0 · ® · 1 (if any line between two points remains in C). DeŽnition 8. A scalar function f is convex if f .®x C .1 ¡ ®/y/ · ®f .x/ C .1 ¡ ®/ f .y/ for all x, y 2 C, and 0 · ® · 1 (if any line between two points does not pass below the function f ).

628

R. Hahnloser, H. Seung, and J. Slotine

Lemma 2. (a) If C is a convex set (such as the nonnegative orthant) and f a convex function on C, then a local minimum of f is also a global minimum. (b) The set of global minima of f is convex (and thus connected). Proof. (a) Let x be a local minimum of f but not a global minimum. Then there exists a y such that f .y/ < f .x/. From the convexity of f , we have that f .®x C .1 ¡ ®/y/ · ®f .x/ C .1 ¡ ®/ f .y/ · f .x/ for every ® 2 [0; 1]. This contradicts the assumption that x is a local minimum. (b) Assume there exist two global minima x, y 2 C and ® 2 [0; 1] such that z D ®x C .1 ¡ ®/y is not a global minimum, f .z/ > f .x/ D f .y/. By convexity of f , we have that f .z/ · ®f .x/ C .1 ¡ ®/ f .y/ D f .x/, contradicting the assumption that z is not a global minimum. DeŽnition 9. A symmetric N £ N matrix A is positive semideŽnite if for all vectors x we have that xT Ax ¸ 0. If a matrix is positive semideŽnite, then all its eigenvalues are nonnegative. We will use that if I ¡ W is a positive semideŽnite matrix, then for all inputs b, the Lyapunov-like energy function L is convex (this follows from the fact that L is quadratic). We will also need the following interlacing theorem for eigenvalues of symmetric matrices. Lemma 3. Let A be a symmetric N £ N matrix with real eigenvalues ¸1 · ¸2 · ¢ ¢ ¢ · ¸N , and let A0 be a .N ¡ 1/ £ .N ¡ 1/ submatrix of A. Then the eigenvalues ´1 · ¢ ¢ ¢ · ´N¡1 of A0 interlace the eigenvalues of A: ¸1 · ´1 · ¸2 · ¢ ¢ ¢ · ´N¡1 · ¸N . Proof. The proof is given in Horn and Johnson (1985). As a corollary of lemma 3, if A has k negative eigenvalues, then A0 must have at least k ¡ 1 negative eigenvalues. A symmetric threshold-linear network has a unique stable, steady state if and only if I ¡ W is positive deŽnite (all eigenvalues are positive; Feng & Hadeler, 1996). As an extension, the following theorem contains necessary and sufŽcient conditions for symmetric threshold-linear networks to be conditionally multiattractive. Theorem 2. Assume that the matrix I ¡ W is symmetric and copositive. The following statements are equivalent: 1. The matrix I ¡ W is not positive semideŽnite. 2. There exists a forbidden set.

3. The network is conditionally multiattractive.

Permitted and Forbidden Sets

629

Proof. ² Statement 2 follows from statement 1. I ¡ W is not positive semideŽnite, and so I ¡ W has a negative eigenvalue. Consider a steady state with all neurons active. Because of this negative eigenvalue, the steady state cannot be stable, and so the set of all neurons is forbidden. ² Statement 1 follows from statement 2. Let 6 be the forbidden set. The smallest eigenvalue of .I ¡ W/6 must be negative. By the interlacing theorem, the smallest eigenvalue of I ¡ W must be negative as well and so I ¡ W is not positive semideŽnite.

² Statement 1 follows from statement 3. Suppose that statement 1 is false. Choose an input b, and let X be a nonconnected set of attractive Žxed points of the network. By Lyapunov stability, X corresponds to the set of local minima of L. Because I ¡ W is positive semideŽnite, L is a convex function. By lemma 2, the local minima of L in the nonnegative orthant are connected, contradicting our initial assumption that X is nonconnected. ² Statement 3 follows from statement 1. The plan is to construct an input for which the network is multiattractive. I¡W is not positive semideŽnite, and so must have at least one negative eigenvalue. If I ¡ W has more than one negative eigenvalue, then deŽne a sequence of subsets 6N ¾ 6N¡1 ¾ ¢ ¢ ¢ ¾ 6k D 6 by removing neurons one by one until only one of the k eigenvalues of .I ¡ W/6 is negative. By the corollary of the interlacing theorem, such a set 6 can always be found. The forbidden set 6 contains at least k ¸ 2 neurons because by copositivity, any single-neuron set is permitted. Denote the negative eigenvalue of .I ¡ W/6 by ¸, and let v be the corresponding eigenvector. Without loss of generality, assume that v D .vp ; vn / has q nonnegative and k ¡ q p negative components, vi D vi ¸ 0 for i D 1; : : : ; q and vni D vqCi < 0 ± ² v for i D 1; : : : ; k¡q. DeŽne a nonnegative vector y D kvppk2 ; ¡ kvvnnk2 that is orthogonal to v, and deŽne the input to the network as bi D yi for i 2 6 and bi D ¡K ¿ 0 for i 2 = 6. The value K has to be sufŽciently large to prevent neurons not belonging to 6 from being activated during the dynamics (how to construct K is shown after the next paragraph). DeŽne a hyperplane H passing through b, spanned by the k ¡ 1 stable eigenvectors orthogonal to v. The hyperplane divides N6 —the nonnegative orthant restricted to 6—into two disjoint regions. The function L restricted to this hyperplane is convex, so by lemma 2, it has a connected set of local minima that are also global minima. Choose an arbitrary minimum z D argminx2H L.x/, and deŽne two initial conditions of the dynamics, both contained in N6 but on opposite sides of the hyperplane, x1i D zi C ²vi , x2i D zi ¡ ²vi for i 2 6, and x1i D 0, x2i D 0 for i 2 = 6, with ² ¿ 1. Because L.x1 / D L.x2 / D L.z/ C ¸2 ² 2 < L.z/ and because L is nonincreasing along

630

R. Hahnloser, H. Seung, and J. Slotine

trajectories, the two trajectories will not cross the hyperplane, but will converge toward two sets of attractive Žxed points X1 and X2 on either side of the hyperplane. How to choose K: DeŽne a region C ½ N6 by C D fx ¸ 0 j x 2 N6 ; L.x/ < L.z/g. Because L is radially unbounded, C is contained in a ball of radius R centered at the origin. By choosing K D Rkwk P C 1 (where k:k denotes an arbitrary matrix norm), we guarantee that j wij xj < bi for all i 2 = 6 and all x 2 C, and thus neurons that are initially inactive remain inactive for all times. It remains to be shown that X1 and X2 are sets of Žxed points that are not connected, neither by a curve conŽned within N6 nor by a curve extending beyond N6 . First, X1 and X2 are not connected by a curve 01 within N6 because this curve would have to cross the hyperplane, implying that there exists a point u 2 H for which L.u/ < L.x1 / < L.z/, contradicting the assumption that z is a global minimum of L restricted to H. Second, X1 and X2 are not connected by a curve 02 extending beyond N6 , because the strongly inhibitory inputs prevent the existence of Žxed points in a small neighborhood around N6 . By contradiction, assume there exists a Žxed 1 point q 2 , = N6 on a curve c connecting X1 and X2 and satisfying kq¡pk < kwk where p 2 c and p 2 N6 . Because q 2 = N6 , at least one component i 2 = 6 of the steady state q mustPbe positive, qP i > 0. Thus, the total input to neuron i must be positive: 0 < j wij qj ¡ K D j wij .qj ¡ pj C pj / ¡ K · kwkkq ¡ pk C P j wij pj ¡ K · 1 C kwkR ¡ K D 0, a contradiction. In conclusion, we have constructed an input b for which there exist two nonconnected sets X1 and X2 of attractive Žxed points (see Figure 1). How does our new concept of multiattractiveness compare to the older concept of multistability? Given our deŽnition of multiattractiveness, multistability is a subtly more general concept: whereas a single line attractor is multistable (it has multiple stable Žxed points), it is not multiattractive. On the other hand, any multiattractive system is necessarily also multistable, implying that the difference between multistability and multiattractiveness is that the latter excludes single line (and surface) attractors. Previous results about stability of rectiŽed linear networks were concerned with either necessary and sufŽcient conditions for monostability (Feng & Hadeler, 1996; Hadeler & Kuhn, 1987) or sufŽcient conditions for multistability (Hahnloser, 1998; Wersing, Beyn, & Ritter, 2001). As an extension to these results, theorem 2 contains a sufŽcient and necessary condition for multiattractiveness: the existence of a forbidden set. To give an intuitive understanding of the equivalence of conditional multiattractiveness and forbidden sets, let us consider a one-dimensional system of the form xP D ¡ f .x/. If such R x a system has to have two stable Žxed points, then the function F.x/ D dy f .y/ must have two separate local minima. And by the smoothness of f , it must be that F has a local maximum between the two local minima, that is, the system must have an instability. A similar

Permitted and Forbidden Sets

q p

G2 i 0

631

NS

H

X1 u G1 X2

Figure 1: Proof of theorem 2. The nonnegative orthant N6 is two-dimensional and is drawn in the horizontal plane. The hyperplane H is a line dividing N6 into two disjoint regions (shaded areas). Because 6 is a forbidden set, the two sets of stable Žxed points X1 and X2 must lie on the boundary of N6 . They are not connected, neither by a trajectory 01 (straight line) passing inside N6 and intersecting H in u, nor by a trajectory 02 passing outside N6 (along axis i), passing by points p 2 N6 and q 2 = N6 .

principle also exists for two-dimensional systems, in which case f is a twodimensional vector Želd. The index theorem says that the contour integral of f along a closed curve C equals the sum of indices of the isolated Žxed points inside C (Strogatz, 1994). Maxima and minima have an index of 1, and saddles have an index of ¡1. Hence, if f is radially unbounded, then the index of a contour integral along C equals 1. And if C is to contain two minima, then it must also contain a saddle: 1 D 2 ¡ 1. Hence, two-dimensional systems are multistable only if they contain an instability. Unfortunately, there is no similarly general result for multistability in dynamical systems of more than two dimensions. But we show here that for rectiŽed linear networks, the intuitive generalization of the index theorem—the equivalence of multistability and instability—is true in arbitrary dimensions. Using theorem 2 and previous results, we can summarize all possible geometries of stable steady states in symmetric networks: ² I ¡ W is positive deŽnite: There exists a single globally attractive Žxed point. ² I ¡ W is copositive and positive semideŽnite but not positive deŽnite: Multiple steady states can exist in the form of a line (or surface) attractor. For example, the two-neuron network given by W D .0; ¡1I ¡1; 0/

632

R. Hahnloser, H. Seung, and J. Slotine

results in eigenvalues 0 and 2 of I ¡ W, and so the two-neuron set is permitted. Let b D .1I 1/; it follows that there is a line of attractive Žxed points given by x1 C x2 D 1 and x1 ¸ 0, x2 ¸ 0. The Žxed points are connected, and so the network is not multiattractive for this input. By theorem 2, the network is not conditionally multiattractive. Recall that no surface attractor of the integrator type is possible for copositive networks. ² I ¡ W is copositive but not positive semideŽnite. There may exist multiple, nonconnected sets of attractive Žxed points dependent on the input. These might either be separate points or nonintersecting line or surface attractors. The following theorem characterizes the relationship between forbidden and permitted sets. Theorem 3. Assume that the matrix I ¡ W is symmetric and copositive. Any subset of a permitted set is permitted. Any superset of a forbidden set is forbidden. Proof. Use the interlacing theorem. If the smallest eigenvalue of a symmetric matrix is nonnegative, then so are the smallest eigenvalues of all its submatrices. And if the smallest eigenvalue of a symmetric submatrix is negative, then so is the smallest eigenvalue of the original matrix. The hierarchical grouping of permitted sets holds only for symmetric networks and may be violated in nonsymmetric networks. Consider the two-neuron network deŽned by w D .2; ¡1I 4; ¡2/. The matrix I ¡ W has the unique eigenvalue 1, and so the two neurons form a permitted set. However, the Žrst diagonal element is ¡1, and so the Žrst neuron, as a subset of a permitted set, forms a forbidden set, in disagreement with theorem 3. What happens in this network is that due to the strong self-excitation, the activity of the Žrst neuron diverges until it necessarily activates the second neuron, the inhibition of which has a stabilizing effect. 5 An Example: The Ring Network Symmetric threshold-linear networks with local excitation and more global inhibition have been studied in the past as models for cortical processing (Ben-Yishai, Lev Bar-Or, & Sompolinsky, 1995; Douglas, Koch, Mahowald, Martin, & Suarez, 1995). Here we argue that invariances of neural responses to sensory stimulation in these models arise from the existence of forbidden sets. We also show that many of these networks exhibit spurious permitted sets, consisting of sets of noncontiguous neurons. Let the synaptic matrix of a 10-neuron ring network be translationally invariant. The connection between neurons i and j is given by Wij D ¡¯C®0 ±ij C®1 .±i;jC1 C±iC1;j /C®2 .±i;jC2 C±iC2;j /, where ¯ quantiŽes global inhi-

633

Permitted set number

Permitted and Forbidden Sets

N e u ro n n u m b e r

N e u ro n n u m b e r

Figure 2: (Left) Output of a ring network of 10 neurons to uniform input (random initial condition). (Right) The 9-parent permitted sets (x-axis: neuron number; y-axis: set number). White means that a neuron belongs to a set, and black means that it does not. Left-right and translation-symmetric parent-permitted sets of the ones shown have been excluded (they can be obtained by translation of the sets shown). The Žrst parent permitted set (Žrst row from the bottom) corresponds to the output on the left.

bition, ®0 self-excitation, ®1 Žrst-neighbor lateral excitation, and ®2 secondneighbor lateral excitation. In Figure 1, we numerically computed the permitted sets of this network, with the parameters taken from Hahnloser et al. (2000), that is, ®0 D 0, ®1 D 1:1, ®2 D 1, and ¯ D 0:55. The permitted sets were determined by diagonalizing the 210 submatrices of I ¡ W and classifying the eigenvalues. In agreement with theorem 1, we Žnd that all positive eigenvectors have positive eigenvalues, and so the network has a set of globally attractive Žxed points. (For a sufŽcient but not necessary criterion for stability of a similar ring network, see Wersing et al., 2001.) We classiŽed sets of neurons 6 into permitted and forbidden sets according to the eigenvalues of .I ¡ W/6 . In Figure 2, we depict the parent permitted sets (sets that have no permitted supersets). Consistent with the Žnding that such ring networks can explain contrast invariant tuning of V1 cells (BenYishai et al., 1995) and multiplicative response modulation of parietal cells (Salinas & Abbott, 1996), we Žnd that there are no permitted sets that consist of more than Žve contiguous active neurons. For example, as a model for parietal cells, any combination of positive visual and positive eye-position inputs retrieves a permitted set consisting of a group of contiguous neurons. Because the resulting activity pattern in the network is localized, the tuning widths of visual responses are limited, making the combined eye position and visual responses look invariant in shape. It can be shown that in order to retrieve one of the spurious permitted sets in Figure 2, some inputs must be negative, a rather irrelevant case, because cortical inputs are usually excitatory.

634

R. Hahnloser, H. Seung, and J. Slotine

6 Discussion In earlier network models of associative memory (HopŽeld, 1982; Hinton & Sejnowski, 1986), computational inputs were encoded as initial conditions to the neural dynamics and memories as attractive Žxed points. After applying an input in these models, the dynamics converge to a steady state, which represents the retrieved memory in response to that input. At the steady state, all information about the input is lost. In threshold-linear networks, the situation is different; the attractive Žxed points are input dependent, meaning that a small change in the input leads to a small change in the location of the retrieved steady state. Thus, in these networks, memories cannot be represented by steady states. We have shown that memories in threshold-linear networks can be deŽned in terms of permitted sets of neurons, for example, sets of neurons that can be stably coactivated at a steady state. This offers a new concept of pattern memorization that fulŽlls the criterion of input independence also for the case of threshold-linear networks. Although memories are input independent, the retrieval of a memory is strongly constrained by the input. A typical input will not allow for the retrieval of arbitrary stored permitted sets. This comes from the fact that the existence of nonconnected sets of stable Žxed points is dependent not just on whether there exists a forbidden set but also on the input (see theorem 2). Generally, multiattractiveness in the ring network is possible only when more than a single neuron is excited. Notice that threshold-linear networks can behave as traditional attractor networks when the inputs are represented as initial conditions of the dynamics. For example, by Žxing b D 1 and initializing a copositive network with some input, the permitted sets unequivocally determine the stable Žxed points. Thus, in this case, the notion of permitted sets is no different from Žxed-point attractors. However, the hierarchical grouping of permitted sets (see theorem 3) becomes irrelevant, since there can be only one attractive Žxed point per hierarchical group deŽned by a parent permitted set. We would like to discuss the limits within which we believe the network dynamics studied here are approximations of real biological neural networks. The fact that threshold-linear neurons are nonsaturating is a simpliŽcation of real neurons that loses its validity when interspike intervals are close to the refractory time of action potentials or when postsynaptic receptors become saturated. Unfortunately, for saturating nonlinearities, we are unable to derive any formal and input-independent results about the computations performed by a network. The reason is that the effect of a saturating nonlinearity is to induce an activity-dependent change in the slopes of transfer functions. As a result, depending on the network architecture, some eigenvalues may decrease, whereas others may increase as Žring rates change, making stability criteria subtly activity dependent and hard to characterize. The symmetry of connections is a simpliŽcation that

Permitted and Forbidden Sets

635

is valid on average when excitatory neurons are recurrently connected (as is common, e.g., in cortex; Douglas et al., 1995) and when inhibitory neurons are nonspeciŽc and fast acting (in which case, inhibitory Žring rates can be expressed in terms of excitatory Žring rates, then inhibitory neurons can be eliminated from the network, and one ends up with symmetric connections). Last but not least, the notion of forbidden sets does not have to be incompatible with the nonzero spontaneous (background) Žring rates of cortical neurons. We would like to argue that the spontaneous background Žring of cortical neurons may involve the rapid switching of permitted sets so that on a larger timescale, all neurons seem to be simultaneously active, whereas in reality they are not. The switching of permitted sets may be driven, for example, by the dynamics of long-range cortico-cortical inputs, which reect the behavioral state of an animal and its expectations about the sensory environment; switching may also involve the intrinsic noisiness of synapses (such as their release probability). The fact that permitted sets never have forbidden subsets implies a hierarchical organization of symmetric networks. As a model for perception, if neurons are feature detectors, then different parent permitted sets may represent different objects. Certain combinations of features will not be perceived as a recognizable object because the features are encoded by neurons forming a forbidden set. If not all but just a subset of the features of an object is present, such as may occur during occlusion, for example, then a subset of the neurons encoding the object may still be coactivated because they still form a permitted set. But how would a network encode, for example, the letter P as a separate object from the letter R and not as a subset thereof? Are symmetric networks not a suitable medium for encoding such object pairs? We do not believe so, because cortical feature detectors have receptive Želds comprising both excitatory and inhibitory subŽelds. In this way, they can signal the absence of a feature as well as they can signal its presence. For example, a neuron that responds to a dark vertical bar surrounded by white anks will respond to the letter P but not to the letter R, because the slanted bar in R excites the inhibitory subŽeld, causing the neuron to be silent. It is thus highly plausible that the encoding of the letter P is based on the activation of sensory neurons that will not be activated for the letter R and so the parent permitted sets of P and R will not be subsets of each other. The representational strategy of excitatory and inhibitory receptive subŽelds makes it possible in principle to store just about any collection of objects as an antichain of subsets, that is, as a collection of mutually exclusive sets. For a simple procedure on how to store a given family of possibly overlapping patterns as permitted sets, see Xiaohui, Hahnloser, and Seung (2002). Appendix: The Generalized Lotka-Volterra Equations In many previous studies, the following generalized Lotka-Volterra equations were used to model neuron dynamics, instead of the rectiŽed linear

636

R. Hahnloser, H. Seung, and J. Slotine

equation 2.1 (Fukai & Tanaka, 1997; Rabinovich et al., 2001): 0 1 X dxi D xi @¡xi C bi C Wij xj A : dt j

(A.1)

The generalized Lotka-Volterra equation A.1 has a similar property as the rectiŽed linear equation, which is that once initialized to nonnegative values, neural activities xi remain nonnegative for all future times. The reason for nonnegativity is that the nonnegative orthant cannot be left, because for all inputs b, we have that xP i D 0 whenever xi D 0. Here we show that given the same connection matrix W and input b, the rectiŽed linear dynamics of equation 2.1 and the generalized Lotka-Volterra dynamics of equation A.1 have the same set of stable Žxed points. This correspondence of attractors will follow from theorem 4, establishing that the quadratic function L that was a Lyapunov-like function for equation 2.1 is also a Lyapunov-like function for equation A.1. Theorem 4. If W is symmetric and I ¡ W copositive, then for all constant b and initial conditions, the generalized Lotka-Volterra equation A.1 converges to an equilibrium. Furthermore, for all b, the set of attractive Žxed points is identical to that of the threshold-linear equation 2.1. Proof. Because I ¡ W is copositive, L D 12 xT .I ¡ W/x ¡ bT x is bounded below. @L The dynamics of equation A.1 can be written as xP i D ¡xi @x . From this, it i ± ²2 @L P @L follows that L is nonincreasing along trajectories: LP D @x D ¡ x xi · i @xi i 0. Fixed points of 2 correspond to steady states of L and vice versa: from xP i D @L 0, it follows that either xi D 0 or @x D 0, and so LP D 0. Similarly, from LP D 0, it i follows that xP i D 0. Hence, for all b, the dynamics of equation A.2 converge to a steady state. Furthermore, because L is a Lyapunov-like function for both equations 2.1 and A.1, it follows that their sets of attractive Žxed points must be identical. The two networks in equations 2.1 and A.1 have the same Lyapunov-like function. It follows that theorems 2 and 3 about conditional multiattractiveness and the grouping of permitted sets also apply to the generalized Lotka-Volterra equations, and so make these networks practically identical. The only differences between them are their basins of attraction, their speed of convergence, and the pathological unstable Žxed points xi D 0 of the generalized Lotka-Volterra equation (recall that the origin is alway a Žxed point of equation A.1 because it is not possible to leave this Žxed point by the action of b, changing b). In practical applications, however, the origin is not a worrisome Žxed point. An approximation to a threshold-linear network was recently im-

Permitted and Forbidden Sets

637

plemented in terms of a silicon circuit using analog very large-scale integration (aVLSI) technology (Hahnloser et al., 2000). There, neural activities are represented by electrical currents. The equations describing the current dynamics were shown to be the generalized Lotka-Volterra equation A.1. The origin is not a pathological Žxed point in this physical system because thermal noise causes the dynamics to leave the repulsive origin for positive b. Acknowledgments We thank Misha Tsodyks for his critical reading of the manuscript. References Ben-Yishai, R., Lev Bar-Or, R., & Sompolinsky, H. (1995). Theory of orientation tuning in visual cortex. Proc. Natl. Acad. Sci. USA, 92, 3844–3848. Bertsekas, D. (1995). Nonlinear programming. Belmont, MA: Athena ScientiŽc. Cohen, M., & Grossberg, S. (1983). Absolute stability of global pattern formation and parallel memory storage by competitive neural networks. IEEE Transactions on Systems, Man and Cybernetics, 13, 288–307. Douglas, R., Koch, C., Mahowald, M., Martin, K., & Suarez, H. (1995). Recurrent excitation in neocortical circuits. Science, 269, 981–985. Ermentrout, B. (1994). Reduction of conductance-based models with slow synapses to neural nets. Neural Computation, 6, 679–695. Feng, J., & Hadeler, K. (1996). Qualitative behaviour of some simple networks. J. Phys. A, 29, 5019–5033. Fukai, T., & Tanaka, S. (1997). A simple neural network exhibiting selective activation of neuronal ensemblies: From winner-take-all to winners-shareall. Neural Computation, 9(1), 77–97. Hadeler, K., & Kuhn, D. (1987). Stationary states of the Hartline-Ratliff model. Biological Cybernetics, 56, 411–417. Hahnloser, R. H. (1998). About the piecewise analysis of networks of linear threshold neurons. Neural Networks, 11, 691–697. Hahnloser, R., Sarpeshkar, R., Mahowald, M., Douglas, R., & Seung, H. (2000). Digital selection and analog ampliŽcation coexist in a cortex-inspired silicon circuit. Nature, 405, 947–951. Hahnloser, R. H., & Seung, H. (2001). Permitted and forbidden sets in symmetric threshold linear networks. In Proceedings of NIPS2001—Neural Information Processing Systems: Natural and Synthetic (Vol. 13, pp. 217–223). Cambridge, MA: MIT Press. Hartline, H. K., & Ratliff, F. (1958). Spatial summation of inhibitory inuence in the eye of limulus and the mutual interaction of receptor units. J. Gen. Physiol., 41, 1049–1066. Hinton, G., & Sejnowski, T. (1986). Learning and relearning in Boltzmann machines. In D. Rumelhart & J. McClelland (Eds.), Parallel distributed processing (pp. 448–453). Cambridge, MA: MIT Press.

638

R. Hahnloser, H. Seung, and J. Slotine

HopŽeld, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. USA, 79(8), 2554–2558. HopŽeld, J. J. (1984). Neurons with graded response have collective properties like those of two-state neurons. Proc. Natl. Acad. Sci. USA, 81, 3088–3092. Horn, R., & Johnson, C. (1985). Matrix analysis. Cambridge: Cambridge University Press. Rabinovich, M., Volkovskii, A., Lecanda, P., Huerta, R., Abarbanel, H., & Laurent, G. (2001). Dynamical encoding by networks of competing neuron groups: Winnerless competition. Physical Review Letters, 87(6), 068102 (1–4). Salinas, E., & Abbott, L. (1996). A model of multiplicative neural responses in parietal cortex. Proc. Natl. Acad. Sci. USA, 93, 11956–11961. Seung, H. S. (1996). How the brain keeps the eyes still. Proc. Natl. Acad. Sci. USA, 93, 13339–13344. Strogatz, S. (1994). Nonlinear dynamics and chaos. Reading, MA: Addison-Wesley. Wersing, H., Beyn, W.-J., & Ritter, H. (2001). Dynamical stability conditions for recurrent neural networks with unsaturating piecewise linear transfer functions. Neural Computation, 13(8), 1811–1825. Xiaohui, X., Hahnloser, R. H., & Seung, H. S. (2002).Selectively grouping neurons in recurrent networks of lateral inhibition. Neural Computation, 14(11), 2627– 2646. Received April 29, 2002; accepted August 2, 2002.

LETTER

Communicated by Richard Hahnloser

Multistability Analysis for Recurrent Neural Networks with Unsaturating Piecewise Linear Transfer Functions Zhang Yi [email protected] College of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 610054, People’s Republic of China, and Department of Electrical and Computer Engineering, National University of Singapore, Singapore 117576

K. K. Tan [email protected]

T. H. Lee [email protected] Department of Electrical and Computer Engineering, National University of Singapore, Singapore 117576

Multistability is a property necessary in neural networks in order to enable certain applications (e.g., decision making), where monostable networks can be computationally restrictive. This article focuses on the analysis of multistability for a class of recurrent neural networks with unsaturating piecewise linear transfer functions. It deals fully with the three basic properties of a multistable network: boundedness, global attractivity, and complete convergence. This article makes the following contributions: conditions based on local inhibition are derived that guarantee boundedness of some multistable networks, conditions are established for global attractivity, bounds on global attractive sets are obtained, complete convergence conditions for the network are developed using novel energy-like functions, and simulation examples are employed to illustrate the theory thus developed. 1 Introduction The dynamical properties of recurrent neural networks play primarily important roles in their applications. The dynamical analysis of recurrent neural networks, especially convergence analysis, has attracted extensive interest and the attention of scientists. So far, many of the reported conditions for the convergence of neural networks are centered around monostability. However, monostable networks are computationally restrictive; they cannot deal with important neural computations (Hahnloser, 1998), such as those necessary in decision making , where multistable networks become necessary. Recently, there has been increasing interest in multistability analysis for Neural Computation 15, 639–662 (2003)

c 2003 Massachusetts Institute of Technology °

640

Z. Yi, K. Tan, and T. Lee

neural networks (see Hahnloser, 1998; Hahnloser, Sarpeshkar, Mahowald, Douglas, & Seung, 2000; Wersing, Beyn, & Ritter, 2001). Multistable networks are signiŽcantly different from monostable ones, and their analysis also merits different treatment. Monostable networks restrict each network to one equilibrium point, whereas a multistable network can have multiple equilibrium points. Conditions for monostability are usually not applicable to multistable networks. In our view, there are three basic properties of a multistable network. The Žrst property is boundedness. It is interesting to derive conditions that not only guarantee boundedness but also allow networks to possess more than one equilibrium point. The second property is that of attractivity. It is necessary to establish whether there exist compact sets into which all the trajectories of a network will be attracted and bounded indeŽnitely. Clearly, all the equilibrium points and limit circles (if any) must be located in the attracting compact sets. The third property relates to complete convergence. It is necessary to know if every trajectory of a network converges to the equilibrium set. In the dynamical analysis of recurrent neural networks, the activation functions are important factors that affect the dynamics of neural networks. Various activation functions have been used for neural networks. Recently, the studies reported in Ben-Yishai, Lev Bar-Or, and Sompolinsky, (1995), Douglas, Koch, Mahowald, Martin, & Suarez, (1995), Hahnloser (1998), Hahnloser et al. (2000), Wersing, Beyn, and Ritter (2001), and Wersing, Steil, and Ritter (2001) are focused on a class of recurrent neural networks with unsaturating linear threshold activation functions (LT networks). This class of neural networks has potential in many applications. Hahnloser et al. (2000) demonstrated an efŽcient silicon design for LT networks and discussed the coexistence of analog ampliŽcation and digital selection in networks circuits. This transfer function is also more appropriate for recurrent neural networks (Douglas et al., 1995). Since the unsaturating piecewise linear activation functions are unbounded, more complex dynamic properties may exist in the networks. As far as we know, there have been few results on this class of neural networks originating from a general dynamic analysis, although some boundedness conditions have been reported in Wersing, Beyn, and Ritter (2001). In this article, we study the recurrent neural networks described by the following nonlinear differential equation, xP i .t/ D ¡xi .t/ C

n X jD1

wij ¾ .xj .t// C hi ;

.i D 1; : : : ; n/;

(1.1)

or its equivalent vector form, P D ¡x.t/ C W¾ .x.t// C h; x.t/

for all t ¸ 0, where each xi denotes the activity of neuron i, W D .wij /n£n is a real n £ n matrix, each of its elements wij denotes the synaptic weights

Multistability of Recurrent Neural Networks

641

Figure 1: The activation function ¾ .s/.

and represents the strength of the synaptic connection from neuron j to neuron i, x D .x1 ; : : : ; xn /T 2 Rn , and h 2 Rn denotes external inputs. For any x 2 Rn , ¾ .x/ D .¾ .x1 /; : : : ; ¾ .xn //T , and the activation function ¾ is deŽned as follows: ¾ .s/ D maxf0; sg;

s 2 R:

The activation function ¾ is an unsaturating piecewise linear function, which is continuous, unbounded, and nondifferentiable. Figure 1 shows this function. We address the three dynamical properties for network 1.1. Conditions based on local inhibition will be derived to guarantee boundedness and also allow the network to have multiequilibrium points. Under such boundedness conditions, we will show that the network possesses compact sets that attract all the trajectories of the network. Explicit inequalities to calculate the global attractive sets will be given. In addition, if the weight matrix of the network possesses some kind of symmetry, we will show that network 1.1 has the property of complete convergence. Wersing, Beyn, and Ritter (2001) studied the boundedness of the following neural networks: P D ¡y.t/ C ¾ .Wy.t/ C h/ y.t/

(1.2)

for t ¸ 0. Based on local inhibition of the network, they established some interesting and elegant boundedness conditions. The network in equation 1.2 has some differences from the network described in equation 1.1. Dynamically, network 1.1 is more general than that of network 1.2. In fact, if we let x.t/ D Wy.t/ C h, then network 1.2 can be easily transformed to network 1.1. The networks have the following relationship, P D ¡y.t/ C ¾ .x.t//; y.t/

for t ¸ 0. For any T ¸ 0, we have Z

ky.t/k · ky.T/ke¡.t¡T/ C

t T

e¡.t¡s/ k¾ .x.s//k ds;

(1.3)

642

Z. Yi, K. Tan, and T. Lee

for t ¸ T. By equation 1.3, the boundedness, attractivity, and convergence of network 1.1 can be easily translated to network 1.2. However, network 1.1 cannot be transformed into 1.2 and retain the dynamic properties, except when matrix W is invertible (Wersing, Beyn, & Ritter, 2001). In fact, if W is not invertible, the equilibrium points of networks 1.1 and 1.2 do not have a oneto-one correspondence under the above transform. In many applications, it may not be reasonable to assume that the matrix W is invertible. Many neural systems exhibiting short-term memory are modeled by noninvertible networks, such as the oculomotor integrator (Seung, 1996) or the headdirection system (Zhang, 1996). In these networks, precise tuning of synaptic connections is necessary to result in line attractors along which the dynamics are marginally stable. In addition, the attractivity issue was not studied by Wersing, Beyn, and Ritter (2001). We believe that attractivity is an important property for multistable networks. The rest of this article is organized as follows. Preliminaries are given in section 2. Boundedness and attractivity analysis of network 1.1 are given in section 3. In section 4, we discuss the complete convergence of the network. Simulations and illustrative examples are given in section 5. Finally, section 6 presents the conclusions. 2 Preliminaries Since the activation function ¾ .¢/ is an important factor that affects the dynamics of the network, we begin with important properties that will be useful in the subsequent multistability analysis. Lemma 1.

Z

s¾ .s/ D ¾ 2 .s/ D 2

s

¾ .µ / dµ; 0

for any s 2 R. Proof. If s ¸ 0, then Z s Z s 2 ¾ .µ / dµ D 2 µ dµ D s2 D ¾ 2 .s/ D s¾ .s/: 0

0

If s < 0, then ¾ .µ / D 0 for all s · µ · 0. Thus, Z s 2 ¾ .µ / dµ D 0 D s¾ .s/ D ¾ 2 .s/: 0

Lemma 2. [¾ .s/ ¡ ¾ .µ /]2 · .s ¡ µ /[¾ .s/ ¡ ¾ .µ /] for all s; µ 2 R.

Multistability of Recurrent Neural Networks

643

Proof. For any s; µ 2 R, we have ¾ .s/ ¡ ¾ .µ / D ¾ .s ¡ µ C µ/ ¡ ¾ .µ / · ¾ .s ¡ µ/: Without loss of generality, we assume that s ¸ µ. Then ¾ .s/ ¸ ¾ .µ /, and so [¾ .s/ ¡ ¾ .µ /]2 · ¾ .s ¡ µ/[¾ .s/ ¡ ¾ .µ /] · .s ¡ µ /[¾ .s/ ¡ ¾ .µ /]: Denote in this article a norm in Rn by v u n uX kxk D t jxi j2 ; iD1

for any x 2 Rn . Then, from lemmas 1 and 2, it follows that n Z xi X xT ¾ .x/ D ¾ T .x/¾ .x/ D 2 ¾ .µ / dµ; iD1

0

and k¾ .x/ ¡ ¾ .y/k2 · .x ¡ y/T ¢ [¾ .x/ ¡ ¾ .y/]; for all x; y 2 Rn DeŽnition 1.

Network (1.1) is bounded if each of its trajectories is bounded.

DeŽnition 2. Let S be a compact subset of Rn . We denote the ² ¡ neighborhood of S by S² . The compact set S is called a global attractive set of network 1.1 if for any ² > 0, all trajectories of that network ultimately enter and remain in S² . Let f .t/ be a continuous function deŽned on [0; C1/; the upper limit of f .t/ is deŽned as " # lim sup f .t/ D lim

t!C1

t!C1

sup f .µ / : µ ¸t

Clearly, if f .t/ is upper-bounded, its upper limit exists. If limt!C1 sup f .t/ D » < C1, then for any ² > 0, there exists a T ¸ 0 such that f .t/ · » C ² for t ¸ T. If every trajectory x.t/ of network 1.1 satisŽes lim sup jxi .t/j · si < C1;

t!C1

644

Z. Yi, K. Tan, and T. Lee

then, clearly, S D fx j jxi j · si ; .i D 1; : : : ; n/g is a global attractive set of network 1.1. A vector x¤ 2 Rn is called an equilibrium point of network 1.1, if it satisŽes ¡x¤ C W¾ .x¤ / C h ´ 0:

We use Se to denote the equilibrium set of network 1.1. DeŽnition 3. Network 1.1 is said to be completely convergent if the equilibrium set Se is not empty and every trajectory x.t/ of that network converges to Se , that is, 1

kx.t/ ¡ x¤ k ! 0 dist.x.t/; Se / D min ¤ e x 2S

as t ! C1. Throughout this article for any constant c 2 R, we denote cC D max.0; c/. We also denote C T hC D .hC 1 ; : : : ; hn / :

For any matrix A D .aij /n£n , we denote AC D .aC ij /, where ( aii ; iD j aC ij D max.0; aij /; i 6D j: We will use ¸min .A/ and ¸max .A/ to denote the smallest and largest eigenvalues of A, respectively. 3 Boundedness and Global Attractivity In this section, conditions based on local inhibition will be derived that guarantee boundedness and allow network 1.1 to have the property of multistability. Moreover, explicit expressions will be provided, locating the compact sets that globally attract all the trajectories of network 1.1. Let us Žrst prove a lemma that will be useful in the proof of the main theorems. Lemma 3. Let M ¸ 0 and T ¸ 0 be constants. If xi .t/ · M; .i D 1; : : : ; n/ for t ¸ T, then we have jxi .t/j · jxi .T/je¡.t¡T/ C M for all t ¸ T.

n X jD1

jwij j C jhi j; .i D 1; : : : ; n/

Multistability of Recurrent Neural Networks

645

Proof. Since xi .t/ · M.i D 1; : : : ; n/ for t ¸ T, then by the deŽnition of the activation function ¾ .¢/, clearly it holds that ¾ .xi .t// · M for all t ¸ T. From network 1.1, we have 2 3 Z t n X xi .t/ D xi .T/e¡.t¡T/ C e¡.t¡s/ 4 wij ¾ .xj .s//Chi 5 ds; .i D 1; : : : ; n/; T

jD1

for t ¸ T. Then it follows that

n X

jxi .t/j · jxi .T/je¡.t¡T/ C M

jD1

jwij j C jhi j; .i D 1; : : : ; n/

for all t ¸ T. Lemma 3 shows that if a trajectory of network 1.1 is upper-bounded, then it is lower-bounded also. This property will help us to prove the following theorem. Theorem 1.

If there exist constants ®i > 0.i D 1; : : : ; n/ such that

1

°i D 1 ¡ wii ¡

n 1 X ®j wC ij > 0; ®i jD1;j6Di

.i D 1; : : : ; n/;

then network 1.1 is bounded. Moreover, 8 ­ 9 < ­ n = X ­ jxi j · 5 ¢ jwij j C jhi j; .i D 1; : : : ; n/ SD x ­ ­ : ­ ; jD1 is a global attractive set of network 1.1, where ( 5 D max f®j g ¢ max 1·j·n

1·j·n

hjC

) :

°j

Proof. Since ¾ .¢/ ¸ 0, it follows from network 1.1 that xP i .t/ · ¡xi .t/Cwii ¾ .xi .t//C

n X jD1;j6Di

for t ¸ 0. DeŽne vi .t/ D

xi .t/ ; .i D 1; : : : ; n/; ®i

C wC ij ¾ .xj .t//Chi ; .i D 1; : : : ; n/;

(3.1)

646

Z. Yi, K. Tan, and T. Lee

for t ¸ 0. Then, from equation 3.1, we have vP i .t/ · ¡vi .t/ C wii ¾ .vi .t// 0 1 n X 1 CA @ C ®j wC ; .i D 1; : : : ; n/ ij ¾ .vj .t// C hi ®i jD1;j6Di

(3.2)

for t ¸ 0. For any x.0/ 2 Rn , denote » ¼ » C¼ 1 hi D kx.0/k ¢ max C max C 1: m 1·i·n ®i 1·i·n ®i °i We will prove that vi .t/ < m; .i D 1; : : : ; n/

(3.3)

for all t ¸ 0. Otherwise, since vi .0/ < m.i D 1; : : : ; n/, there must exist some i and a time t1 > 0 such that vi .t1 / D m and ( < m; j D i; 0 · t < t1 vj .t/ · m; j 6D i; 0 · t · t1 : Thus, it must follow that vP i .t1 / ¸ 0. However, from equation 3.2, we have 0 1 n X 1 CA @ vP i .t1 / · ¡m C wii m C ®j wC ij m C hi ®i jD1;j6Di D ¡m ¢ °i C

hC i ®i

< 0: This is a contradiction that proves equation 3.3 to be true. From equation 3.3, it follows that 1

xi .t/ < max f®j g ¢ m D M; .i D 1; : : : ; n/ 1·j·n

for all t ¸ 0. By lemma 3, we have jxi .t/j · jxi .0/je¡t C M ¢ · jxi .0/j C M ¢

n X jD1

n X jD1

jwij j C jhi j

jwij j C jhi j; .i D 1; : : : ; n/

for t ¸ 0. This proves that network 1.1 is bounded.

Multistability of Recurrent Neural Networks

647

Next, we will prove the global attractivity under the conditions. Denote lim sup vi .t/ D »i ; .i D 1; : : : ; n/:

t!C1

Since each vi .t/ is upper-bounded, then »i < C1; .i D 1; : : : ; n/. We will prove that ( »i < max

hjC

) ; .i D 1; : : : ; n/:

°j

1·j·n

(3.4)

For convenience in the subsequent discussion and without loss of generality, we assume that »l D max1·j·n f»j g. If »l · 0, clearly equation 3.4 is true. In what follows, we consider the case that »l > 0. Suppose equation 3.4 is not true: ( »l > max

1·j·n

hjC °j

) :

(3.5)

We will prove that this assumption will lead to a contradiction that will then conŽrm that equation 3.4 is true. Clearly, we can choose a small ² > 0 such that ( ) 2².1 C jwjj j/ C hjC 1 D k: »l > max ²; 1·j·n °j For this choice of ², by the basic property of the upper limit, there exists a t2 ¸ 0 such that vj .t/ · »j C ² · »l C ²; .j D 1; : : : ; n/; for all t ¸ t2 . We claim that there exists a t3 ¸ t2 such that vP l .t/ < 0

(3.6)

for all t ¸ t3 . If equation 3.6 is not true, there must exist a t4 > t2 such that vP l .t4 / ¸ 0;

vl .t4 / ¸ »l ¡ ²:

However, from equation 3.2, we have vP l .t4 / · ¡.»l ¡ ²/ C wll »l C jwll j² C

n 1 X C ²/ C hC ®j wC lj ¾ .»j l ®l jD1;j6Di

648

Z. Yi, K. Tan, and T. Lee

· ¡.»l ¡ ²/ C wll »l C jwll j² C

n 1 X C ²/ C hC ®j wC lj .»l l ®l jD1;j6Di

· ¡°l »l C 2².1 C jwll j/ C hC l · ¡°l k C 2².1 C jwll j/ C hC l < 0; which is a contradiction, thus proving that equation 3.6 is true. Equation 3.6 shows that vl .t/ is monotonically decreasing. Thus, the limit of vl .t/ exists, that is, lim vl .t/ D lim sup vl .t/ D »l :

t!C1

t!C1

Then there exists a t5 ¸ t3 such that ( »l ¡ ² < vl .t/ < »l C ² vj .t/ < »j C ² < »l C ²; .j D 1; : : : ; n/ for all t ¸ t5 . From equation 3.2, it follows for t ¸ t5 that vP l .t/ · ¡.»l ¡ ²/ C wll »l C jwll j² C

n 1 X C ²/ C hC ®j wC lj ¾ .»l l ®l jD1;j6Di

· ¡°l »l C 2².1 C jwll j/ C hC l

· ¡°l k C 2².1 C jwll j/ C hC l : Thus, ] vl .t/ · vl .t5 / ¡ .t ¡ t5 /[°l k ¡ 2².1 C jwll j/ ¡ hC l ! ¡1;

as t ! C1:

This contradicts »l > 0. Therefore, equation 3.5 does not hold, and instead, equation 3.4 is true. By equation 3.4, we have ( C) hj 1 lim sup xi .t/ D ®i »i · max f®j g ¢ max D 5; .i D 1; : : : ; n/: t!C1 1·j·n 1·j·n °j

Multistability of Recurrent Neural Networks

649

For any ´ > 0, there exists a t6 ¸ 0 such that xi .t/ · 5 C ´;

.i D 1; : : : ; n/;

for t ¸ t6 . By lemma 3, it follows that jxi .t/j · jxi .t6 /je¡.t¡t6 / C .5 C ´/ ¢ for t ¸ t6 . Thus, we have lim sup jxi .t/j · .5 C ´/ ¢

t!C1

n X jD1

n X jD1

jwij j C jhi j

jwij j C jhi j:

Letting ´ ! 0, it follows that lim sup jxi .t/j · 5 ¢

t!C1

n X jD1

jwij j C jhi j; .i D 1; : : : ; n/:

This shows that S is a global attractive set of network 1.1. The local inhibition-based conditions of theorem 1 are identical to Wersing, Beyn, and Ritter (2001) for neural network 1.2. As far as we known, Wersing, Beyn, and Ritter (2001) are the Žrst authors to obtain conditions for boundedness of neural networks based on local inhibition. Such conditions are quite interesting and elegant. However, they did not study attractivity, which is an important dynamic property for multistability of recurrent neural networks. Theorem 2.

If µ

WT C W ÄDI¡ 2

¶C

is a positive deŽnite matrix, then network 1.1 is bounded. Moreover, » ­ ¼ p ­ khC k T W/C / ¢ kxk · C khk SD x ­ ¸ ..W max ­ ¸min .Ä/ is a global attractive set of the network 1.1. Proof. DeŽne a differentiable function n Z xi .t/ X V.t/ D ¾ .µ / dµ jD1

0

for t ¸ 0. Obviously, we have V.t/ ¸ 0 for all t ¸ 0.

650

Z. Yi, K. Tan, and T. Lee

Calculating the derivative of V.t/ along the trajectories of equation 1.1, we have P D ¡¾ T .x.t//xT .t/ C ¾ T .x.t//W¾ .x.t// C ¾ T .x.t//h V.t/ µ T ¶ W CW · ¾ T .x.t// ¡ I ¾ .x.t// C ¾ T .x.t//hC 2 Áµ ! ¶ TCW C W · ¾ T .x.t// ¡ I ¾ .x.t// C ¾ T .x.t//hC 2 · ¡¸min .Ä/ ¢ ¾ T .x.t//¾ .x.t// C ¾ T .x.t//hC ·¡

khC k2 ¸min .Ä/ T ¾ .x.t//¾ .x.t// C 2 2¸min .Ä/

for all t ¸ 0. By lemma 1, we have khC k2 2¸min .Ä/

P · ¡¸min .Ä/V.t/ C V.t/ for all t ¸ 0 and V.t/ · V.0/e

¡¸min .Ä/t

khC k2 C 2¸min .Ä/

· V.0/e¡¸min .Ä/t C

Z

t

e¡¸min .Ä/.t¡s/ ds

0

khC k2 2[¸min .Ä/]2

for all t ¸ 0. By lemma 1, we have n Z xi .0/ X kx.0/k2 xT .0/¾ .x.0// · V.0/ D ¾ .µ / dµ D 2 2 jD1 0 and k¾ .x.t//k2 · kx.0/k2 e¡¸min .Ä/t C

khC k2 1 D Á.t/ [¸min .Ä/]2

(3.7)

for all t ¸ 0. Moreover, we have lim

t!C1

p

Á.t/ D

khC k : ¸min .Ä/

(3.8)

From equation 1.1, we have Z x.t/ D x.0/e¡t C

t 0

e¡.t¡s/ [W¾ .x.s// C h] ds

(3.9)

Multistability of Recurrent Neural Networks

651

for t ¸ 0. Then, from equations 3.9 and 3.7, we have Z

kx.t/k · kx.0/ke¡t C Z · kx.0/ke¡t C Z · kx.0/ke¡t C · kx.0/ke¡t C ¡t

· kx.0/ke

C

t 0 t

p e¡.t¡s/ ¾ T .x.s//.W T W/¾ .x.s// ds C khk

t

p e¡.t¡s/ ¾ T .x.s//..W T W/C /¾ .x.s// ds C khk

0

0

p

e¡.t¡s/ kW¾ .x.s//k ds C khk

Z ¸max ..W T W/C / ¢

t 0

Z

p ¸max

..W T W/C /

¢

t 0

e¡.t¡s/ k¾ .x.s//k ds C khk p e¡.t¡s/ Á.s/ ds C khk

1

D 8.t/ for t ¸ 0. Since lim 8.t/ D

p

t!C1

D

p

Z ¸max ..W T W/C / lim

t

t!C1 0

¸max ..W T W/C / lim

p

t!C1

p e¡.t¡s/ Á.s/ ds C khk

Á.t/ C khk;

then, using equation 3.8, we have lim 8.t/ D

p

t!C1

¸max ..W T W/C / ¢

khC k C khk: ¸min .Ä/

Thus, it follows that lim sup kx.t/k ·

p

t!C1

¸max ..W T W/C / ¢

khC k C khk: ¸min .Ä/

This shows that S globally attracts network 1.1. In theorems 1 and 2, if hi · 0 .i D 1; : : : ; n/, the global attractive sets become S D fx j jxi j · jhi j; .i D 1; : : : ; n/g and S D fx j kxk · khkg in theorems 1 and 2, respectively. In particular, if hi D 0.i D 1; : : : ; n/, the attracting sets contain only the origin; that is, network 1.1 can have only the

652

Z. Yi, K. Tan, and T. Lee

origin as its unique equilibrium point and all trajectories of it converge to the origin. The conditions attached to the weight matrix of network 1.1 in theorems 1 and 2 are different. For example, consider the matrices 2 3 2 3 0 6 7 0 5 0:5 6 7 6 7 2 5: W 1 D 4¡4 0 55 ; W2 D 4¡7 0 ¡5 ¡3 0 0:5 ¡4 0 Matrix W1 satisŽes the conditions in theorem 1 but not theorem 2. And, matrix W2 satisŽes conditions in theorem 2 but not theorem 1. Hence, although both theorems relate to boundedness and global attractivity, they can be used complementarily in different situations. 4 Complete Convergence In this section, we study the complete convergence of network 1.1, assuming that the connection matrix W is symmetric in some sense. The general method to study complete convergence of neural networks is to construct suitable energy functions. However, for network 1.1, since the activation function ¾ .¢/ is not differentiable, it is difŽcult to construct energy functions such as those used in Chua and Yang (1988), Cohen and Grossberg (1983), HopŽeld (1984), Wersing, Beyn, and Ritter (2001), and Wu and Chua (1997). To counter this nondifferentiability problem, we Žrst associate network 1.1 with another differential equation. Through this equation, a novel energy-like function can be constructed and the conditions for complete convergence derived accordingly. For any x0 2 Rn , we will associate network 1.1 with the following equivalent differential equation instead, zP .t/ D ¡z.t/ C ¾ .Wz.t/ C h C .x0 ¡ h/e¡t /;

(4.1)

for t ¸ 0. Lemma 4. For any x0 2 Rn , the solution x.t/ of network 1.1 starting from x0 can be represented as x.t/ D Wz.t/ C h C .x0 ¡ h/e¡t ; where z.t/ is the solution of equation 4.1 starting from the origin. Proof. For any x0 2 Rn , let x.t/ be the solution of equation 1.1 starting from x0 . Consider the following differential equation, zP .t/ D ¡z.t/ C ¾ .x.t//; for all t ¸ 0.

(4.2)

Multistability of Recurrent Neural Networks

653

From equations 1.1 and 4.2, it follows that d[x.t/ ¡ h ¡ Wz.t/] D ¡[x.t/ ¡ h ¡ Wz.t/] dt for t ¸ 0. Solving this equation, we have x.t/ D .x0 ¡ h ¡ Wz.0//e¡t C Wz.t/ C h for t ¸ 0, where z.0/ 2 Rn is any initial vector of equation 4.2. Taking z.0/ D 0 in particular—that is, z.t/ is the solution of equation 4.2 starting from the origin—then x.t/ D .x0 ¡ h/e¡t C Wz.t/ C h

(4.3)

for t ¸ 0. The result now follows by substituting equation 4.3 into 4.2. Lemma 5. For any x0 2 Rn , the solution z.t/ of equation 4.1 starting from the origin satisŽes zi .t/ ¸ 0; .i D 1; : : : ; n/ for all t ¸ 0. Proof. Since z.0/ D 0, then from equation 4.1, we have 0 1 Z t n X zi .t/ D e¡.t¡s/ ¾ @ wij zj .s/ C hi C .xi0 ¡ hi /e¡s A ds 0

jD1

for t ¸ 0 and .i D 1; : : : ; n/. Since ¾ .¢/ ¸ 0 always, clearly, zi .t/ ¸ 0 for t ¸ 0. Lemma 6. Suppose there exists a diagonal positive deŽnite matrix D such that the matrix DW is symmetric. Given any x0 2 Rn , if the solution z.t/ of equation 4.1 starting from the origin is bounded, then lim zP .t/ D 0:

t!C1

Proof. Construct an energy-like function E.t/ D

1 T z .t/D.I ¡ W/z.t/ ¡ hT Dz.t/ 2 ³ ´ Z t ¡t ¡s T ¡ .x0 ¡ h/ D z.t/e C z.s/e ds 0

for all t ¸ 0. Clearly, E.0/ D 0.

(4.4)

654

Z. Yi, K. Tan, and T. Lee

By the assumption that z.t/ is bounded, it is easy to see that E.t/ is also bounded. By lemma 5, zi .t/ ¸ 0.i D 1; : : : ; n/ for all t ¸ 0; we have ¾ .z.t// D z.t/ for all t ¸ 0. Denote D D diag.d1 ; : : : ; dn /; then di > 0.i D 1; : : : ; n/. Let d D min1·i·n .di /. It follows from equations 4.4 and 4.1 and lemma 2 that P D [z.t/ ¡ ..x0 ¡ h/e¡t C Wz.t/ C h/]T D E.t/

£ [¡z.t/ C ¾ ..x0 ¡ h/e¡t C Wz.t/ C h/]

D [z.t/ ¡ ..x0 ¡ h/e¡t C Wz.t/ C h/]T D

£ [¡¾ .z.t// C ¾ ..x0 ¡ h/e¡t C Wz.t/ C h/]

· ¡d ¢ k¾ .z.t// ¡ ¾ ..x0 ¡ h/e¡t C Wz.t/ C h/k2 D ¡d ¢ kz.t/ ¡ ¾ ..x0 ¡ h/e¡t C Wz.t/ C h/k2 D ¡d ¢ kPz.t/k2

(4.5)

for all t ¸ 0. Thus, E.t/ is monotonically decreasing. Since E.t/ is bounded, there must exist a constant E0 such that lim E.t/ D E0 < ¡1:

t!C1

From equation 4.5, Z

C1 0

Z kPz.s/k2 ds D lim

t

t!C1 0

Z · lim

t!C1 0

D¡

t

kPz.s/k2 ds P ¡E.s/ ds d

1 lim E.t/ d t!C1

¡E0 d < C1: D

Since z.t/ is bounded, from equation 4.1 it follows that zP .t/ is bounded. Then z.t/ is uniformly continuous on [0; C1/. Again, from equation 4.1, it follows that kPz.t/k2 is also uniformly continuous on [0; C1/. Thus, it must hold that lim kPz.t/k2 D 0:

t!C1

Theorem 3. Suppose there exists a diagonal positive deŽnite matrix D such that DW is symmetric. If the network 1.1 is bounded, then the network is completely convergent.

Multistability of Recurrent Neural Networks

655

Proof. By lemma 4, for any x0 2 Rn , the solution of equation 1.1 starting from this point can be represented as x.t/ D .x0 ¡ h/e¡t C Wz.t/ C h

(4.6)

for t ¸ 0, where z.t/, which depends on x0 , is the solution of equation 4.1 starting from the origin. Let us Žrst show that z.t/ is bounded. By the assumption that x.t/ is bounded, there exists a constant K > 0 such that kx.t/k · K for all t ¸ 0. From equation 4.2, we have Z t z.t/ D e¡.t¡s/ ¾ .x.s// ds 0

for t ¸ 0. Thus,

Z

kz.t/k ·

t 0

Z ·

t

e¡.t¡s/ k¾ .x.s//k ds e¡.t¡s/ K ds

0

·K for t ¸ 0. This shows that z.t/ is bounded. Using lemma 6, we have limt!C1 zP .t/ D 0. Then, from equation 4.6, we have P D W lim zP .t/ D 0: lim x.t/

t!C1

t!C1

Since x.t/ is bounded, every subsequence of fx.t/g must contain convergent subsequence. Let fx.tm /g be any of such a convergent subsequence. There exists a x¤ 2 Rn such that lim x.tm / D x¤ :

tm !C1

Then, from equation 1.1, P m / D 0: ¡x¤ C W¾ .x¤ / C h D lim x.t tm !C1

Thus, x¤ 2 Se . This shows that Se is not empty, and any convergent subsequence of fx.t/g converges to a point of Se . Next, we use the method in Liang and Wang (2000) to prove lim dist.x.t/; Se / D 0:

t!C1

(4.7)

Suppose equation 4.7 is not true. Then there exists a constant ²0 > 0 such N Se / ¸ ²0 . From that for any T ¸ 0, there exists a tN ¸ T that satisŽes dist.x.t/;

656

Z. Yi, K. Tan, and T. Lee

this and by the boundedness property of x.t/, we can choose a convergent subsequence fx.tNm /g, where limtNm !C1 x.tNm / D x† 2 Se , such that dist.x.tNm /; Se / ¸ ²0 ; .m D 1; 2; : : :/: Letting tm ! C1, we have dist.x† ; Se / ¸ ²0 > 0; which contradicts dist.x† ; Se / D 0 since x† 2 Se . This proves equation 4.7 is true. Network 1.1 is thus completely convergent. 5 Simulation Examples In this section, we provide examples of simulation results to illustrate and verify the theory developed. 5.1 Example 1. Consider the two-dimensional neural network "

# " # " 0 xP 1 .t/ x1 .t/ D¡ C ¡0:6 xP 2 .t/ x2 .t/

for t ¸ 0. Clearly, " 0 WD ¡0:6

¡1 0:4

#" # " # 0:8 ¾ .x1 .t// C 0:48 ¾ .x2 .t//

(5.1)

# ¡1 : 0:4

Let us Žrst use theorem 1 to show the boundedness of this network. C Taking ®1 D ®2 D 1, since wC 12 D w21 D 0, it is easy to calculate that °1 D 1 > 0 and °2 D 0:6 > 0. Thus, the conditions of theorem 1 are fully satisŽed. Using theorem 1, the network is bounded, and there exists a compact set to attract the network. By simple calculations, we obtain the global attractive set (" # ­ ) x1 ­ ­ jx1 j · 1:6; jx2 j · 1:28 : SD ­ x2 ­ DeŽne a diagonal positive deŽnite matrix " # 1 0 DD : 0 53 Then we have " DW D

0 ¡1

¡1

¡ 23

# ;

Multistability of Recurrent Neural Networks

657

which is symmetric. By theorem 3, this network is completely convergent. Thus, every trajectory of equation 5.1 must converge to the equilibrium set. The equilibrium set of the network must be located in the attractive set S. In fact, it is not difŽcult to calculate out the equilibrium set of equation 5.1 as (" # ­ ) x1 ­ ­ e S D x1 C x2 D 0:8; x1 ¸ 0; x2 ¸ 0 : ­ x2 ­ Clearly, Se ½ S. The boundedness and global attractivity of equation 5.1 can also be checked by theorem 2. If we invoke theorem 2, we can calculate out another global attractive set: SN D fx j kxk · 2:6077g: This also shows that theorem 2 does not imply monostability. Figure 2 shows the simulation results for global attractivity and complete convergence of network 5.1 for 250 trajectories originating from randomly selected initial points. It can be observed that every trajectory converges to the equilibrium set. Clearly, this network has a line attractor Se . The squarecontained part in Figure 2 is the global attractive set S calculated by using theorem 1. The circle-contained part is the global attractive set SN calculated by using theorem 2. 5.2 Example 2. Consider the following three-dimensional neural network P D ¡x.t/ C W¾ .x.t// C h; x.t/

(5.2)

for t ¸ 0, where x D .x1 ; x2 ; x3 /T , h D .¡0:2; ¡0:8; 0:3/T and 2 3 0 1 0:5 6 7 2 5: W D 4¡3 0 0:5 ¡4 0 It is not difŽcult to show that there does not exist any diagonal matrix D such that DW becomes symmetric. Hence, we cannot use theorem 3 to ascertain whether the network is completely convergent. However, we can use theorem 2 to show the boundedness and global attractivity of the network. We have 2 3 1 0 ¡0:5 µ T ¶C W CW 6 7 D4 0 1 0 5: I¡ 2 ¡0:5 0 1

658

Z. Yi, K. Tan, and T. Lee

12 10 8 6

x2

4 2 0 ­2 ­4 ­6 ­ 10

­5

0

x1

5

10

15

Figure 2: Global attractivity and complete convergence of network 5.1.

Its eigenvalues are 0:5, 1:0, and 1:5, respectively. Thus, it is a positive deŽnite matrix. By theorem 2, the network is bounded, and a global attractive set can be located as S D fx j kxk · 3:3528g: Figure 3 shows the simulation results in three-dimensional space for 50 trajectories originating from randomly selected initial points. Figures 4, 5, and 6 show the projections of the trajectories onto the phase planes .x1 ; x2 /, .x1 ; x3 / and .x2 ; x3 /, respectively. The parts contained in the circles in Figures 4 through 6 are the projections of the global attractive set S onto the corresponding phase planes. The boundedness of this network cannot be checked by theorem 1. In fact, we have 2 3 1 ¡1 ¡0:5 6 7 1 ¡2 5 ; I ¡ WC D 4 0 ¡0:5 0 1 which has a negative eigenvalue that obviously violates the conditions of theorem 1.

Multistability of Recurrent Neural Networks

659

20 10 x3

0 ­ 10 ­ 20 20 0 ­ 20 ­ 40

x2

­ 20

10

0

­ 10

20

x1

Figure 3: Boundedness and global attractivity of network 5.2.

20 15 10

x2

5 0 ­5 ­ 10 ­ 15 ­ 20 ­ 20

­ 15

­ 10

­5

0 x1

5

10

15

Figure 4: Projection of trajectories of equation 5.2 on .x1 ; x2 / phase plane.

20

660

Z. Yi, K. Tan, and T. Lee

15 10 5

x3

0 ­5 ­ 10 ­ 15 ­ 20 ­ 25 ­ 20

­ 15

­ 10

­5

0 x2

5

10

15

20

Figure 5: Projection of trajectories of equation 5.2 on .x2 ; x3 / phase plane.

From the simulation results, it can be observed that this network seems to have monostability property; however, we are not able to prove this point. It is worthwhile noting that this network does not satisfy the existing stability conditions, for example, the global stability conditions recently reported in Forti (1994), Liang and Si (2001), Yi, Heng, and Fu (1999), Yi, Heng, and Leung (2001), and Yi, Heng, and Vadakkepat (2002). Apart from the presented simulations, we have extensively simulated the boundedness, global attractivity, and complete convergence of networks that belong to network model 1.1. All the results verify the theory developed. In fact, certain global attractive sets approach the critical bounds containing the equilibrium points. 6 Conclusion In this article, we have studied the multistability of recurrent neural networks with unsaturating piecewise linear transfer functions. We have addressed three basic dynamical problems for this class of networks: boundedness, global attractivity, and complete convergence. Local inhibition-based conditions for boundedness and global attractivity are established. Under

Multistability of Recurrent Neural Networks

661

20 15 10

x3

5 0 ­5 ­ 10 ­ 15 ­ 20 ­ 20

­ 15

­ 10

­5

x1

0

5

10

15

Figure 6: Projection trajectories of equation 5.2 on .x1 ; x3 / phase plane.

these conditions, the global attractive sets of the networks are obtained. With symmetry conditions attached to the networks’ weights matrix, the complete convergences are analyzed by showing that each trajectory converges to the equilibrium set. We believe that the complete convergence results of this article could be improved to show that each trajectory converges to an equilibrium point instead of the equilibrium set. However, we could not prove this, and it remains an interesting problem to study. Acknowledgments We thank the reviewers for their many helpful comments and suggestions to improve the quality and readability of this article. References Ben-Yishai, R., Lev Bar-Or, R., & Sompolinsky, H. (1995). Theory of orientation tuning in visual cortex. Proc. Natl. Acad. Sci. USA, 92, 3844–3848. Chua, L. O., & Yang, L. (1988). Cellular neural networks: Theory. IEEE Trans. Circuits Systems, 35, 1257–1272.

662

Z. Yi, K. Tan, and T. Lee

Cohen, M. A., & Grossberg, S. (1983). Absolute stability of global pattern formation and parallel memory storage by competitive neural networks. IEEE Trans. Systems, Man, Cybernetics, 13, 815–826. Douglas, R., Koch, C., Mahowald, M., Martin, K., & Suarez, H. (1995). Recurrent excitation in neocortical circuits. Science, 269, 981–985. Forti, M. (1994). On global asymptotic stability of a class of nonlinear systems arising in neural network theory. J. Differential Equations, 113, 246–264. Hahnloser, R. L. T. (1998). On the piecewise analysis of linear threshold neural networks. Neural Networks, 11, 691–697. Hahnloser, R., Sarpeshkar, R., Mahowald, M. A., Douglas, R. J. , & Seung, H. S. (2000). Digital selection and analogue ampliŽcation coexist in a cortexinspired silicon circuit. Nature, 405, 947–951. HopŽeld, J. J. (1984). Neurons with graded response have collective computational properties like those of two-state neurons. Proc. Natl. Acad. Sci. USA, 81, 3088–3092. Liang, X. B., & Si, J. (2001). Global exponential stability of neural networks with globally Lipschitz continuous activations and its applications to linear variational inequality problem. IEEE Trans. Neural Networks, 12, 349–359. Liang, X. B., & Wang, J. (2000). A recurrent neural network for nonlinear optimization with a continuously differentiable objective function and bound constraints. IEEE Trans. Neural Networks, 11, 1251–1262. Seung, H. S. (1996). How the brain keeps the eyes still. Proc. Natl. Acad. Sci. USA, 93, 13339–13344. Wersing, H., Beyn, W. J., & Ritter, H. (2001). Dynamical stability conditions for recurrent neural networks with unsaturating piecewise linear transfer functions. Neural Computation, 13, 1811–1825. Wersing, H., Steil, J. J., & Ritter, H. (2001). A competitive layer model for feature binding and sensory segmentation. Neural Computation, 13, 357–387. Wu, C. W., & Chua, L. O. (1997). A more rigorous proof of complete stability of cellular neural networks. IEEE Trans. Circuits and Systems, I, 44, 370–371. Yi, Z., Heng, P. A., & Fu, A. W. C. (1999).Estimate of exponential convergence rate and exponential stability for neural networks. IEEE Trans. Neural Networks, 10, 1487–1493. Yi, Z., Heng, P. A., & Leung, K. S. (2001). Convergence analysis of cellular neural networks with unbounded delay. IEEE Trans. Circuits and Systems, I, 48, 680–687. Yi, Z., Heng, P. A., & Vadakkepat, P. (2002). Absolute periodicity and absolute stability of delayed neural networks. IEEE Trans. Circuits and Systems, I, 49, 256–261. Zhang, K. (1996). Representation of spatial orientation by the intrinsic dynamics of the head-direction cell ensemble: a theory. J. Neurosci., 16, 2112–2126. Received February 14, 2002; accepted September 4, 2002.

LETTER

Communicated by Bruno Olshausen

Simple-Cell-Like Receptive Fields Maximize Temporal Coherence in Natural Video Jarmo Hurri jarmo.hurri@hut.Ž

Aapo Hyv¨arinen aapo.hyvarinen@hut.Ž Neural Networks Research Centre, Helsinki University of Technology, 02015 HUT, Finland

Recently, statistical models of natural images have shown the emergence of several properties of the visual cortex. Most models have considered the nongaussian properties of static image patches, leading to sparse coding or independent component analysis. Here we consider the basic time dependencies of image sequences instead of their nongaussianity. We show that simple-cell-type receptive Želds emerge when temporal response strength correlation is maximized for natural image sequences. Thus, temporal response strength correlation, which is a nonlinear measure of temporal coherence, provides an alternative to sparseness in modeling simple-cell receptive Želd properties. Our results also suggest an interpretation of simple cells in terms of invariant coding principles, which have previously been used to explain complex-cell receptive Želds. 1 Introduction The functional role of simple cells has puzzled scientists since the structure of their receptive Želds was Žrst mapped by Hubel and Wiesel in the 1950s (Palmer, 1999). The Žrst hypothesis concerning their role was based on their visual appearance, that is, their similarity with edges and bars. The second major theory was the local spatial frequency analysis theory, which is based on generally applicable signal processing principles. The current view of the functionality of sensory neural networks emphasizes learning and the relationship between the structure of the cells and the statistical properties of the information they process (see, e.g., Field, 1994; Simoncelli & Olshausen, 2001). A major advance was achieved when Olshausen and Field (1996) showed that simple-cell-like receptive Želds emerge when sparse coding is applied to natural image data. Similar results were obtained with independent component analysis (ICA) shortly after (Bell & Sejnowski, 1997; van Hateren & van der Schaaf, 1998). In the case of image data, ICA is closely related to sparse coding (Hyv¨arinen, Karhunen, & Oja, 2001; Olshausen & Field, 1997). Neural Computation 15, 663–691 (2003)

c 2003 Massachusetts Institute of Technology °

664

J. Hurri and A. Hyva¨ rinen

In this article, we show that an alternative principle, temporal coherence (Becker, 1993; Foldi´ ¨ ak, 1991; Kayser, Einh¨auser, Dummer, ¨ Konig, ¨ & K¨ording, 2001; Mitchison, 1991; Stone, 1996; Wiskott & Sejnowski, 2002), leads to the emergence of simple-cell receptive Želds from natural image sequences. This Žnding is signiŽcant because it means that temporal coherence provides a complementary theory to sparse coding as a computational principle behind the formation of simple-cell receptive Želds. The results also link the theory of achieving invariance by temporal coherence (Foldi´ ¨ ak, 1991) to real-world visual data and measured properties of the visual system. Whereas previous research has focused on establishing this link for complex cells, we show that such a connection exists even on the simple-cell level. Temporal coherence is based on the idea that when processing temporal input, the representation changes as little as possible over time. Foldi´ ¨ ak (1991) was one of the Žrst to suggest the usefulness of temporal coherence in computational neuroscience. He developed a two-layer network that was able to learn to identify a Žxed feature, such as a line with a Žxed orientation, even if the way the feature was expressed in the data changed, for example, if the line was translated. Foldi´ ¨ ak used temporal coherence as a tool to learn translation invariances: artiŽcially generated input data were temporally coherent (consecutive input frames contained translated versions of a line with the same orientation), and by using competition and short-term memory, the output was also taught to be temporally coherent. This associated translated versions of a feature with each other. Several other researchers have studied temporal coherence and other forms of coherence, such as coherence with respect to different views of the same scene. For example, in Becker and Hinton (1992) and Stone (1996), surface depth was discovered from stereograms by a multiple layer nonlinear network. In Becker and Hinton (1992), this was done by using stereograms representing different views of the same randomly generated scene and maximizing mutual information between outputs. In Stone (1996), learning was achieved by using a temporal sequence of slightly different stereograms and maximizing temporal smoothness of output while preserving variability in output. In these studies, the input data sets were generated so that there was an underlying coherent parameter in the data, and the objective was to Žnd that parameter by using coherence. Therefore, the main result was the demonstration of the usefulness of coherence using simulated data. The contribution of this article is to show that when the input consists of natural image sequences, the linear Žlters that maximize temporal response strength correlation are similar to simple-cell receptive Želds. We Žrst describe temporal response strength correlation, which is a measure of temporal coherence, and an algorithm capable of optimizing the measure. In section 3, we apply this algorithm to natural image sequences. In addition to the main results, we describe several control experiments that were made to ensure the validity and novelty of our results. Finally, in section 4, we give an intuitive explanation of why optimization of the objective function pro-

Simple-Cell Receptive Fields and Temporal Coherence

665

duces such results, discuss the spatiotemporal and nonlinear (nonnegative) extensions of the model, and conclude by discussing the implications of this work. 2 Temporal Response Strength Correlation In the basic model, we restrict ourselves to consider linear spatial models of simple cells. Linear simple-cell models are commonly used in studies concerning the connections between visual input statistics and simple-cell receptive Želds (Bell & Sejnowski, 1997; Olshausen & Field, 1996; van Hateren & van der Schaaf, 1998), because linearity seems to approximately characterize most simple cells (DeAngelis, Ohzawa, & Freeman, 1993b). Extensions of this basic framework are discussed in sections 4.4 and 4.5. The model uses a set of spatial Žlters (vectors) w1 ; : : : ; wK to relate input to output. Let signal vector x.t/ denote the input of the system at time t: A vectorization of image patches can be done by scanning images column-wise into vectors. For windows of size N £ N; this yields vectors with dimension N 2 : The output of the kth Žlter at time t, denoted by signal yk .t/; is given by the dot-product yk .t/ D wTk x.t/:

(2.1)

Let matrix W D [w1 ; : : : ; wK ]T denote a matrix with all the Žlters as rows. Then the input-output relationship can be expressed in vector form by y.t/ D Wx.t/;

(2.2)

where signal vector y.t/ D [y1 .t/; : : : ; yK .t/]T : Temporal response strength correlation, the objective function, is defined by f .W/ D

K X kD1

Et fg.yk .t//g.yk .t ¡ 1t//g;

(2.3)

where the nonlinearity g is strictly convex, even (rectifying), and differentiable. The symbol 1t denotes a delay in time. The nonlinearity g measures the strength (amplitude) of the response of the Žlter and emphasizes large responses over small ones (see section 4). Examples of choices for this nonlinearity are g1 .x/ D x2 ; which measures the energy of the response, and g2 .x/ D ln cosh x; which is a robust version of g1 : A set of Žlters with a large temporal response strength correlation is such that the same Žlters often respond strongly at consecutive time points, outputting large (either positive or negative) values. This means that the same Žlters will respond strongly over short periods of time, thereby expressing temporal coherence of a population code.

666

J. Hurri and A. Hyva¨ rinen

To keep the outputs of the Žlters bounded, we enforce the unit variance constraint on each of the output signals yk .t/; that is, we enforce the constraint Et fy2k .t/g D wTk Cx wk D 1 for all k; where matrix Cx D Et fx.t/xT .t/g: Additional constraints are needed to keep the Žlters from converging to the same solution. Standard methods (Hyv¨arinen et al., 2001) are either to force the set of Žlters to be orthogonal or to force their outputs to be uncorrelated, from which we choose the latter. This introduces additional constraints wTi Cx wj D 0; i D 1; : : : ; K; j D 1; : : : ; K; j 6D i: These uncorrelatedness constraints limit the number of Žlters K we can Žnd so that K · N2 : The unit variance constraints and the uncorrelatedness constraints can be expressed by the single matrix equation, WCx WT D I:

(2.4)

Note that if we use a nonlinearity g.x/ D x2 and 1t D 0; the objecPK 4 tive function becomes f .W/ D kD1 Et fyk .t/g: In this case, the optimization of the objective function under the unit variance constraint is equivalent to optimizing the sum of kurtoses of the outputs. Kurtosis is a commonly used measure in sparse coding. Similarly, in the case of nonlinearity g.x/ D ln cosh x and 1t D 0; the objective function can be interpreted as a nonquadratic measure of the nongaussianity of Žlter outputs. We return to this issue in section 3. Thus, the receptive Želds are learned in our model by maximizing the objective function 2.3 under the constraint 2.4. The optimization algorithm used for this constrained optimization problem is a variant of the gradient projection method of Rosen (for the original algorithm, see Luenberger, 1969). The optimization approach employs whitening, that is, a temporary change of coordinates, to transform the constraint 2.4 into an orthonormality constraint. Then a gradient projection algorithm employing optimal symmetric orthogonalization can be used. See the appendix for details. 3 Experiments on Natural Image Sequences 3.1 Data Collection. The natural image sequences used as data were a subset of those used in van Hateren and Ruderman (1998). The original data set consisted of 216 monochrome, noncalibrated video clips of 192 seconds each, taken from television broadcasts. More than half of the videos feature wildlife; the rest show various topics, such as sports and movies. Sampling frequency was 25 frames per second, and each frame was blockaveraged to a resolution of 128 £ 128 pixels. For our experiments, this data set was pruned to remove the effect of human-made objects and artifacts. First, many of the videos feature human-made objects, such as houses and furniture. Such videos were removed from the data set, leaving us with 129 videos. Some of these 129 videos had been grabbed from television broadcasts, and there was a wide black bar with height 15 pixels at the

Simple-Cell Receptive Fields and Temporal Coherence

667

top of each image, probably because the original broadcast had been in wide screen format. Our sampling procedure never took samples from this topmost part of the videos. If these artifacts were not removed from the data set, the static ICA results computed for comparison showed longer horizontal and vertical receptive Želds than results obtained from scenes without the artifacts. The Žnal, preprocessed (see below) data set consisted of 200,000 pairs of consecutive 11 £ 11 image windows (patches) at the same spatial position, but 1t milliseconds apart from each other. Depending on the experiment, 1t varied between 40 ms and 960 ms: However, because of the temporal Žltering used in preprocessing, initially 200,000 longer image sequences with a duration of 1t C 400 ms; and the same spatial size 11 £ 11; were sampled with the same sampling rate. A second data set was needed for computing the corresponding (static) ICA solution for comparison. This data set consisted of 200,000 11 £ 11 images sampled from the same video data. 3.2 Preprocessing. The preprocessing in the main experiment consisted of three steps: temporal decorrelation, subtraction of local mean, and normalization. (The same preprocessing steps were applied in the control experiments. Whenever preprocessing was varied in control experiments, it is explained separately below.) Temporal decorrelation can be motivated in two ways. First, it can be motivated biologically as a model of temporal processing at the lateral geniculate nucleus (Dong & Atick, 1995). Second, for 1t D 0; the objective function can be interpreted as a measure of sparseness. Therefore, it is important to rule out the possibility that there is barely any change in short intervals in video data, since this would imply that our results could be explained in terms of sparse coding or ICA. To make the distinction between temporal response strength correlation and measures of sparseness clear, temporal decorrelation was applied because it enhances temporal changes. Note, however, that this still does not remove all of the static part in the video, an issue addressed in the control experiments that follow. To investigate the effect of temporal decorrelation, we Žrst examined the histogram of the distances between subsequent image windows separated by 1t D 40 ms; shown in Figure 1A. (This histogram also shows that there are indeed large changes between subsequent time points even without temporal decorrelation.) The local mean has been removed from these windows, and they have been normalized (see below); note that 2 is maximal distance because of normalization. Temporal decorrelation was performed with a temporal Žlter, shown in Figure 1B. The Fourier magnitude of the Žlter was computed by inverting the amplitude spectrum of input data. A Wiener Žlter approach was used in order not to amplify high-frequency noise. (We determined the Žlter directly from data—see Dong & Atick, 1995—for an analytic solution.) Noise power was estimated by assuming that it is equal to signal power at 5:5 Hz; a value also used in Dong and Atick (1995). Phases

668

J. Hurri and A. Hyva¨ rinen

B 2£104 1£104 0 0

0.5

1

1.5

Euclidean distance

2

0.1 0.05 0 0

0.1

0.2

0.3

0.4

0.5

1

1.5

2

time (seconds)

D occurrences

occurrences

C

¯ lter coe± cient

occurrences

A

2£104 1£104 0 0

0.5

1

1.5

Euclidean distance

2

2£104 1£104 0 0

Euclidean distance

Figure 1: Temporal decorrelation enhances temporal changes. (A) Distribution of Euclidean distances between consecutive samples at the same spatial position, but 40 ms apart from each other, without temporal decorrelation. Note that two is the maximum value because of normalization. (B) The temporally decorrelating Žlter. (C) Distribution of Euclidean distances between consecutive samples at the same spatial position, but 40 ms apart from each other, after temporal decorrelation. (D) Distribution of Euclidean distances between consecutive samples at the same spatial position, but 120 ms apart from each other, after temporal decorrelation. Note that removal of DC component and normalization were also performed in all of these cases.

were determined by adding a minimum energy delay constraint on the Žlter (see Oppenheim & Schafer, 1975). Filter length was originally 2500 ms but was truncated to 400 ms because this part contains over 99% of Žlter energy. When short image sequences of length 1t C 400 ms are Žltered with this temporal Žlter, the resulting sequences are of length 1t: From each of these temporally Žltered short sequences, two windows separated by 1t were taken. Figure 1C shows the distribution of the distances between such temporally decorrelated windows, after removal of local mean and normalization (see below), when 1t D 40 ms: This histogram shows that temporal decorrelation enhances temporal changes in the data. Figure 1D shows the distribution of the distances between temporally decorrelated windows, after removal of local mean and normalization, when 1t D 120 ms: In this case, consecutive windows are, on average, quite far from each other when measured with the Euclidean norm. In fact, the peak of the histogram is

Simple-Cell Receptive Fields and Temporal Coherence

669

approximately at 1.4, which for normalized vectors means that a large part of the consecutive windows are approximately orthogonal to each other. After temporal decorrelation, the local mean (DC component) was subtracted from each window. This reduces the number of dimensions of the data by one, so with image window size N D 11; the number of Žlters K · 120: Finally, the sample vectors (vectorized windows) were normalized to have unit Euclidean norm, which can be considered a form of contrast gain control (Carandini, Heeger, & Movshon, 1997; Heeger, 1992). Note that no spatial low-pass Žltering or dimensionality reduction was performed during preprocessing. In the case of the second data set, which was used to compute the corresponding static ICA solution, preprocessing consisted of removal of the local mean, followed by normalization. Temporal decorrelation was not performed here, since it has no meaning in the case of static image data. 3.3 Results: Temporally Coherent Filters of Natural Image Sequences. The main experiment consisted of running the symmetric gradient projection algorithm 50 times using different random initial values, followed by a quantitative analysis of these results, as well as results obtained with a corresponding ICA algorithm. In this experiment, 1t was 40 ms: The number of extracted Žlters was set at the maximum value K D 120: Nonlinearity g in objective function 2.3 was chosen to be g.x/ D ln cosh x because of its robustness against outliers (Hyv¨arinen et al., 2001). Figure 2 shows the resulting Žlters (i.e., rows of matrix W) of the Žrst run. The Žlters have been ordered according to Et fg.yk .t//g.yk .t ¡ 1t//g; that is, according to their “contribution” into the Žnal objective value (Žlters with largest values top left). The Žlters resemble Gabor Žlters. They are localized and oriented and have different scales. These are the main features of simplecell receptive Želds (Palmer, 1999). When compared qualitatively with earlier reported results, obtained with sparse coding and independent component analysis (Bell & Sejnowski, 1997; Olshausen & Field, 1996; van Hateren & van der Schaaf, 1998), our results show a larger variety of different spatial scales. To compare the results quantitatively, we extracted a corresponding set of 50 ICA separation matrices using the symmetric Žxed-point ICA (FastICA) algorithm, with robust nonlinearity tanh (Hyv¨arinen et al., 2001). This algorithm is a symmetric version of that used in van Hateren and van der Schaaf (1998). The symmetric nature of the algorithm facilitates extracting a balanced set of Žlters (Hyv¨arinen et al., 2001). Filters maximizing temporal response strength correlation were compared against the rows of the separating matrix (the ICA Žlters), since these Žlters are the natural counterparts in ICA (van Hateren & van der Schaaf, 1998). Figure 3 shows the ICA Žlters obtained from the Žrst run. In the quantitative comparison of the results, we measured the most important properties of the receptive Želds. The results are shown in Figure 4. The measured properties were peak spatial frequency (see Figures 4A and

670

J. Hurri and A. Hyva¨ rinen

Figure 2: Temporally coherent Žlters of natural image sequences, given by the Žrst run of the main experiment. The Žlters were estimated from natural image sequences by optimizing temporal response strength correlation with the symmetric gradient projection algorithm (here nonlinearity g.x/ D ln cosh x). The Žlters have been ordered according to Et fg.yk .t//g.yk .t¡1t//g; that is, according to their contribution into the Žnal objective value (Žlters with largest values at top left).

4B; note the logarithmic scale and units cycles per pixel), peak orientation (see Figures 4C and 4D), spatial frequency bandwidth (see Figures 4E and 4F), and orientation bandwidth (see Figures 4G and 4H). See van Hateren and van der Schaaf (1998) for deŽnitions of these measures. Although there are some differences, the most important observation here is the similarity of the histograms. This supports the idea that ICA=sparse coding and temporal coherence are complementary theories, in that both result in the emergence of simple-cell-like receptive Želds. As for the differences, the results obtained using temporal response strength correlation have a slightly

Simple-Cell Receptive Fields and Temporal Coherence

671

Figure 3: For comparison, ICA Žlters estimated from natural image sequences by using the symmetric Žxed-point algorithm. Note that these Žlters have a much less smooth appearance than in most published ICA results; this is because for comparison, we show here the Žlters and not the basis vectors, and further, no low-pass Žltering or dimension reduction was applied in the preprocessing.

smaller number of high-frequency receptive Želds. Also, temporal response strength correlation seems to produce receptive Želds that are somewhat more localized with respect to both spatial frequency and orientation. When the results are compared against the results in van Hateren and van der Schaaf (1998), the most important difference is the peak at zero bandwidth in Figures 4E and 4F. This difference is probably a result of the fact that no dimensionality reduction, antialiasing, or noise reduction was performed here, which results in the appearance of very small, checkerboard-like receptive Želds. This effect is more pronounced in ICA, which also explains the stronger peak at the 45 degree angle in Figure 4D.

672

J. Hurri and A. Hyva¨ rinen

temporal coherence

ICA

2000 1000 0 0.1

0.2

0.4 0.7

peak spatial frequency (c/pixel)

occurrences

C

occurrences

B 2000 1000 0 0.1

0.4 0.7

D 1000 500 0 ¡ 90 ¡ 45

0

45

90

1000 500 0 ¡ 90 ¡ 45

peak orientation (degrees)

0

45

90

peak orientation (degrees)

F 1500 1000 500 0 0

1

2

3

1500 1000 500 0

occurrences

E occurrences

0.2

peak spatial frequency (c/pixel)

occurrences

occurrences

A

0

1

2

0

50

100

3

G

H 2000 1000 0 0

50

100

150

orientation bandwidth (degrees)

occurrences

spatial freq. bandwidth (octaves)

occurrences

spatial freq. bandwidth (octaves)

2000 1000 0 150

orientation bandwidth (degrees)

Figure 4: Comparison of properties of receptive Želds obtained by optimizing temporal response strength correlation (left column, histograms A, C, E, and G) and estimating ICA Žlters (right column, histograms B, D, F, and H). See the text for details.

Simple-Cell Receptive Fields and Temporal Coherence

673

3.4 Control Experiments. To ensure the novelty and validity of our results, we made eight control experiments. All other aspects, except those speciŽcally mentioned here, were similar in these experiments as in the main experiment. 3.4.1 Control Experiment I: No Temporal Decorrelation. In the main experiment, we used temporally decorrelated data. However, as was seen in Figure 1, there is considerable temporal change in natural video data even without temporal decorrelation. Therefore, it is reasonable to ask whether temporal decorrelation is necessary for achieving results like those shown in Figure 2. To answer this question, the algorithm was run for data that were not temporally decorrelated. The results are shown in Figure 5A. Although the Žlters with highest-frequency components seem to be somewhat less localized, the results remain qualitatively very similar to those in Figure 2 in that they also resemble Gabor Žlters. This suggests that the simple-cell-like properties of the results of the main experiment are not a consequence of temporal decorrelation. 3.4.2 Control Experiments II–IV: Longer 1t. In the main experiment, consecutive samples were separated by 40 ms: Does the phenomenon found in the main experiment hold only for very small 1t? To answer this question, we examined the case 1t D 120 ms in control experiment II. As can be seen in Figure 1D, with this time separation, preprocessed consecutive sample windows are typically much farther from each other than when 1t D 40 ms: Figure 5B shows the results of applying the symmetric gradient projection algorithm to this data. Although the receptive Želds are now larger than in the main experiment, the results are still qualitatively similar. This implies that the discovered relationship applies to short-time natural image sequences in general, not only for some speciŽc value of 1t: It seems natural that there would be an upper limit for which the results are no longer qualitatively similar to those in Figure 2. This is indeed the case. The degree of spatial localization decreases when 1t increases. This can already be seen to some degree in Figure 5B and is more pronounced in the results of control experiment III, in which 1t D 480 ms (see Figure 5C). In the 480 ms case, most of the Žlters are poorly localized, resembling Fourier basis vectors corresponding to different frequencies. As 1t becomes large enough, spatial localization disappears, and Žlters also start to lose their orientation selectivity. This is illustrated in Figure 5D for 1t D 960 ms (control experiment IV). 3.4.3 Control Experiment V: Randomly Selected Consecutive Windows. Control experiment V was made to ensure that the results reect the dynamics of natural image sequences, and not just the relationship between any arbitrary image patches. Instead of using samples separated by 1t; random image samples were chosen as consecutive window pairs. No temporal

674

J. Hurri and A. Hyva¨ rinen

A

B

C

D

Figure 5: Results of control experiments I–IV. (A) Results of control experiment I in which no temporal decorrelation was performed. (B) Results of control experiment II in which 1t D 120 ms: (C) Results of control experiment III in which 1t D 480 ms: (D) Results of control experiment IV in which 1t D 960 ms:

decorrelation was done here, since random window pairs do not have a temporal relationship. Figure 6A shows the resulting spatial Žlters, which correspond to noise patterns, indicating that the original results in Figure 2 do reect natural image sequence dynamics. 3.4.4 Control Experiment VI: Complete Removal of the Static Part of Video. In most experiments, the data were temporally decorrelated, leading to enhanced temporal change. For example, note from Figure 1D that in the case 1t D 120 ms; after preprocessing (including the normalization step), the peak of the histogram of distances between preprocessed consecutive windows is at approximately 1.4. Remembering that the samples have been normalized, this indicates that a large part of the preprocessed sample con-

Simple-Cell Receptive Fields and Temporal Coherence

A

B

C

D

675

Figure 6: Results of control experiments V–VIII. (A) Results of control experiment V in which consecutive window pairs were chosen randomly. (B) Results of control experiment VI in which the static part of natural image sequences was removed altogether by employing Gram-Schmidt orthogonalization to consecutive windows. (C) Results of control experiment VII in which observer (camera) movement was compensated by using a tracking mechanism. (D) Results of control experiment VIII in which ordinary linear correlation between output values was maximized.

sists of consecutive windows that are approximately orthogonal to each other. This supports the idea that our results are not a consequence of the static part of natural image sequences. However, since preprocessing does not remove the static part of the video completely, we made another control experiment (control experiment VI), with 1t D 120 ms; in which the static part was removed altogether. This was done by modifying the preprocessing step so that no temporal decorrelation was done; instead, after removal of the local mean, consecutive windows

676

J. Hurri and A. Hyva¨ rinen

were orthonormalized using the Gram-Schmidt procedure. This removes the static part, the part present already at time t ¡ 1t; from the window at time t completely. No other form of temporal Žltering was performed in this experiment. Figure 6B shows the results of this control experiment. The resulting Žlters are still localized and oriented and have different scales. This shows that the qualitative nature of our results is not a consequence of the static part of natural image sequences. The Žlters are more localized than in the corresponding experiment with temporal decorrelation, probably because removal of the static part is more likely to remove large features from the data (see section 4). If the same orthogonalization procedure is applied in the case 1t D 40 ms; the results (not shown) are even more localized. This suggests that here too, as in the case of temporally decorrelated data, the degree of spatial localization is a function of 1t: 3.4.5 Control Experiment VII: Compensation of Observer Movement. Control experiment VII was made to study the role of observer (camera) movement. To compensate for this movement, a simple correlation-based tracking mechanism was implemented into the sampling procedure. Tracking was applied before temporal Žltering (temporal decorrelation), so each 440 ms (D 1t C 400 ms) sequence was tracked. Let x1 be the Žrst vectorized sample window in a 440 ms sequence, and X n be the set of candidate windows in the nth video frame of the sequence, differing at most 10 pixels from the spatial position of the Žrst window x1 : The nth window xn was chosen by xT x

n¡1 xn D arg maxx2X n kxn¡1 kkxk : The results, shown in Figure 6C, are qualitatively similar to the original results, showing that the main results are not a consequence of observer movement. The number of low-frequency receptive Želds seems to be smaller in the control results. This change is probably caused by decreased large-scale movement (see section 4).

3.4.6 Control Experiment VIII: Ordinary Linear Correlation. Finally, the purpose of control experiment VIII was to show that higher-order correlation is indeed needed for the emergence of simple-cell-like Žlters. To study this, we computed the optimal Žlter solutions for maximizing linear correlation f` .wk / D E t fyk .t/yk .t ¡ 1t/g (see also Mitchison, 1991). 1 The unit variance constraint is used here again, so the problem is equivalent to minimizing E t f.yk .t/ ¡ yk .t ¡ 1t//2 g with the same constraint. A 1 Note that this objective function is deŽned for single Žlters. A similar single-unit rule for optimizing Et fg.yk .t//g.yk .t ¡ 1t//g can be deŽned and optimized for temporal response strength correlation. The optima for this objective function are similar to those in Figure 2, but the receptive Želds are more elongated. In addition, there are problems with obtaining a complete basis because a deationary algorithm (Hyv¨arinen et al., 2001), in which Žrst-extracted solutions dominate, has to be used. In the case of linear correlation, we do not have this problem since a closed-form solution can be found.

Simple-Cell Receptive Fields and Temporal Coherence

677

closed-form solution can be derived from necessary Karush-Kuhn-Tucker (KKT) conditions and an additional zero DC constraint. Let Cx D EDET ; from which the DC eigenvector corresponding to zero eigenvalue has been dropped out. The necessary conditions are fulŽlled by those eigenvectors of ED¡1 ET Et f.x.t/ ¡ x.t ¡ 1t//.x.t/ ¡ x.t ¡ 1t//T g that correspond to nonzero eigenvalues. Analysis of sufŽcient KKT conditions reveals that the Žlters fulŽlling the Žrst-order conditions behave as in the case of principal components; that is, after the minimizing Žlter is found and removed from the set of eigenvectors, another eigenvector from the set will be the next minimum, assuming that the output of the next selected Žlter has to be uncorrelated with the outputs of the previously selected ones. The eigenvector corresponding to the smallest eigenvalue is the global minimum, and the next minimum is the one with the next smallest eigenvalue. Figure 6D shows the resulting Žlters, sorted according to the corresponding eigenvalue (smallest eigenvalue top left). These Žlters resemble Fourier basis vectors, and not simple-cell receptive Želds. Thus, we see that emergence of localized receptive Želds requires the use of a nonlinear temporal correlation measure. 4 Discussion 4.1 Temporal Coherence of Large Responses. Temporal response strength correlation, as deŽned by equation 2.3, does not explicitly measure the rate of change of the output signal. Therefore, it is important to examine what type of temporal coherence the objective function measures. In order to do this, consider three different temporally uncorrelated signals, y1 .t/; y2 .t/; and y3 .t/; depicted in Figure 7. (Note that in the main experiment, the output signals yk .t/ are temporally uncorrelated because they are spatially linearly Žltered from temporally decorrelated input data.) All of these signals have unit energy and a gaussian marginal distribution. Signals y2 .t/ and y3 .t/ have been created by reordering the samples of y1 .t/ so that they contain two intervals of high amplitude (see the Žgure caption for details). Signal y3 .t/ has the largest temporal response strength correlation of these signals, as measured by equation 2.3. This is because the objective function emphasizes the temporal coherence of large amplitudes. This is not truepfor an arbitrary measure of amplitude correlation. For example, for g.x/ D jxj (concave on interval ]0; 1]), signal y2 .t/ has a larger measure than y3 .t/ ( f .y2 .t// ¼ 0:73; f .y3 .t// ¼ 0:71). 4.2 Temporal Coherence vs. Sparseness. We saw in the previous section that our objective function gives large values when both yk .t/ and yk .t ¡ 1t/ have large amplitudes, thus emphasizing the correlation of large activations. This property must not, however, be confused with sparseness. Sparseness of yk .t/ means that very large amplitudes, as well as very small ones, are relatively common. It is thus a property of the marginal distribution of

678

J. Hurri and A. Hyva¨ rinen

y (t)

ln cosh y(t)

A

B ln cosh y1 (t)

y 1 (t )

3 0 ¡ 3 0

200

time index

1 0 0

time index

200

400

0

time index

200

400

0

time index

200

400

D ln cosh y2 (t)

3

y 2 (t )

2

400

C 0 ¡ 3 0

200

time index

3 2 1 0

400

E

F ln cosh y3 (t)

3

y 3 (t )

3

0 ¡ 3 0

200

time index

400

3 2 1 0

Figure 7: Temporal response strength correlation emphasizes temporal coherence of large amplitudes. Three temporally uncorrelated signals y.t/ with unit energy and gaussian marginal distributions (A,C,E) and their rectiŽed magnitudes ln cosh y.t/ (B,D,F). The signals in C and E were obtained by rearranging the time indices of the signal in A. This was done so that the two intervals of high amplitude in these signals contain the samples with the largest amplitudes, in random order. In C, the total length of the intervals is half of the total signal length; in E, this ratio is 15 : The signal in E has the largest temporal response strength correlation, as measured by equation 2.3 ( f .y1 .t// ¼ 0:13; f .y2 .t// ¼ 0:23; f .y3 .t// ¼ 0:29). This illustrates the fact that the objective function emphasizes the temporal correlation of large amplitudes. See the text for discussion.

Simple-Cell Receptive Fields and Temporal Coherence

A

679

B

Figure 8: Examples of transformations inducing approximate local translations. The local area is marked with a dashed square. (A) Planar rotation. (B) Bending.

yk .t/: The temporal correlation of large amplitudes says nothing about their frequency or any other aspect of the marginal distribution of yk .t/: All the signals in Figure 7 (on the left) have a gaussian marginal distribution, yet the signals vary considerably in their temporal coherence, as measured by our objective function. Our measure of temporal coherence could indirectly measure the marginal distribution (sparseness) only where there is little change in the data, that is, if the static part of the image sequence dominates. However, this possibility was ruled out by the use of temporal decorrelation, which ensures that there is quite a large amount of change in the data, as shown in Figure 1. Ultimately, control experiment VI showed that the use of temporal coherence produced similar results even if the static part was removed completely, thus proving that the principle of temporal coherence is distinct from sparse coding. 4.3 An Intuitive Explanation of Results. 4.3.1 The Importance of Translation in Natural Image Sequences. The most universal local visual properties of objects are edges and lines, so we limit the discussion here to their dynamic properties. Objects can undergo a number of transformations in image sequences: translation, rotation, occlusion, and, for nonrigid objects, deformation. A transformation in the 3D space can induce a different transformation in an image sequence. For example, a translation toward the camera induces a change of object size in the image sequence. Our hypothesis is that for local edges and lines and during short time intervals, most 3D object transformations result in local translations of these elements in image sequences. This is, of course, true for 3D translations of objects. Figure 8 illustrates this phenomenon for two other transformations: a planar rotation and bending of an object. Note that the effect illustrated in Figure 8A is even more pronounced if object rotation is not purely planar.

680

J. Hurri and A. Hyva¨ rinen

4.3.2 Why Gabor-Like Filters Maximize Correlation of Square-RectiŽed Responses. In order to demonstrate the correlation of square-rectiŽed responses at consecutive time points, we will consider the interaction of features and Žlters in one dimension (orthogonal to the orientation of the Žlter). This allows us to consider the effect of local translations in a simpliŽed setting. Figure 9 illustrates, in a simpliŽed case, why the temporal response strengths of lines and edges correlate positively as a result of Gabor-like Žlter structure. Prototypes of two different types of image elements—the proŽles of a line and an edge—which both have a zero DC component, are shown in the topmost row of the Žgure. The leftmost column shows the proŽles of three different Žlters with unit norm and zero DC component: a Gabor-like Žlter, a sinusoidal (Fourier basis–like) Žlter, and an impulse Žlter. The rest of the Žgure shows the square rectiŽed responses of the Žlters to the inputs as functions of spatial displacement of the input. Consider the rectiŽed response of the Gabor-like Žlter to the line and the edge. The squared response at time t ¡ 1t (spatial displacement zero) is strongly positively correlated with response at time t; even if the line or edge is displaced slightly. This shows how small local translations of basic image elements still yield large values of temporal response strength correlation for Gabor-like Žlters. If you compare the responses of the Gabor-like Žlter to the responses of the sinusoidal Žlter, you can see that the responses to the sinusoidal Žlter are typically much smaller. This leads to a lower value of our measure of temporal response strength correlation that emphasizes large values. Also, while the response of an impulse Žlter to an edge correlates quite strongly over small spatial displacements, when the input consists of a line, even a very small displacement will take the correlation to almost zero. Thus, we can see that when considering three important classes of Žlters— Žlters that are maximally localized in space, maximally localized in frequency, or localized in both—the optimal Žlter is a Gabor-like Žlter localized in both space and frequency. If the Žlter is maximally localized in space, it fails to respond over small spatial displacements of very localized image elements. If the the Žlter is maximally localized in frequency, its responses to the localized image features are not strong enough. Figure 10 shows why we need nonlinear correlations instead of linear ones: raw output values might correlate either positively or negatively, depending on the displacement. Thus, we see why ordinary linear correlation is not maximized for Gabor-like Žlters, whereas the rectiŽed (nonlinear) correlation is.

4.3.3 Emergence of Simple-Cell-Like Filters. Future research is needed to provide a detailed analysis of which properties of natural image sequence data are needed for the emergence of simple-cell-like Žlters. However, at

Simple-Cell Receptive Fields and Temporal Coherence

681

input

¡ 2¡ 1 0 1 2 3

¯ lter

¡ 2¡ 1 0 1 2 3

square-recti¯ ed output

¡ 2¡¡ 1 0 1 2 3

¡ 1 0 1

¡ 1 0 1

¡ 2¡ 1 0 1 2 3

¡ 1 0 1

¡ 1 0 1

¡ 2¡ 1 0 1 2 3

¡ 1 0 1

¡ 1 0 1

Figure 9: A simpliŽed illustration of why a Gabor-like Žlter, localized in both space and frequency, yields larger values of temporal response strength correlation than a Žlter localized only in space or only in frequency. Top row: cross sections of a line (left) and an edge (right) as functions of spatial position. Leftmost column: cross sections of three Žlters with unit norm and zero DC component: a Gabor-like Žlter (top), a sinusoidal Žlter (middle), and an impulse Žlter (bottom). The other plots in the Žgure show the responses of the Žlters to the inputs as a function of spatial displacement of the input. The Gabor-like Žlter yields fairly large positively correlated values for both types of input. The sinusoidal Žlter yields small response values. The impulse Žlter yields fairly large positively correlated values when the input consists of an edge, but when the input consists of a line, even a small displacement yields a correlation of almost zero.

682

J. Hurri and A. Hyva¨ rinen

input

¡ 2¡ 1 0 1 2 3

¯ lter

¡ 2¡¡ 1 0 1 2 3

¡ 2¡ 1 0 1 2 3

raw output

¡ 1 0 1

¡ 1 0 1

Figure 10: A simpliŽed illustration of why nonlinear correlation is needed for the emergence of the phenomenon. Raw response values of the Gabor-like Žlter to the line and edge may correlate positively or negatively, depending on the displacement. (See Figure 9 for an explanation of the layout of the Žgure.)

this point, we can provide a set of hypotheses as to why oriented, localized Žlters with multiple scales emerge from the data: Orientation. The Žlters are oriented because the contours (edges and lines) in image sequences are oriented. Because our objective function emphasizes strong responses, the Žlters need to be matched to the dominant features in the data (see Figure 9). Localization. As illustrated in Figure 9, the features are limited in width because lines and edges are mostly limited in width as well, and because short-time translations are very local. This second point is supported by the results of control experiments II through IV (see Figures 5B–5D), which showed that when 1t is increased, localization decreases. The features are limited in length for the same reason that they are limited in ICA or sparse coding: contours in real images have some curvature. Even a weak curvature makes the match (dot-product) with an elongated Gabor quite weak, and gives poor temporal coherence (Hoyer & Hyv¨arinen, 2002). Multiple scales. The Žlters respond to multiple scales for two reasons. The Žrst reason is the scale invariance of natural image sequences. This explanation is supported by the results of control experiment VI (Figure 6B), in which the static part of image sequences was removed completely. The results exhibit a narrower range of different scales because removal of the

Simple-Cell Receptive Fields and Temporal Coherence

683

static part of the sequences is more likely to remove large features. The second reason is that the features move with many different velocities in the data. This is supported by the results of control experiment VII (see Figure 6C), in which tracking was used to compensate for observer movement. The results exhibit a narrower range of different scales, because tracking is likely to reduce the velocity of moving features in the data. 4.4 The Case of Spatiotemporal Receptive Fields. Due to limited computational resources, we are currently unable to estimate the most temporally coherent spatiotemporal receptive Želds. However, argumentation similar to the intuitive explanation provided above can be given to illustrate a similar phenomenon in the spatiotemporal case. Figure 11 illustrates a case in which a vertical line is moving in the image sequence. The response of a simple-cell-like motion-selective spatiotemporal Žlter (DeAngelis, Ohzawa, & Freeman, 1993a), whose spatiotemporal position and orientation match the initial position of the line and its direction of movement, is large in magnitude at consecutive time points. This illustrates how large temporal response strength correlation could arise in the case of spatiotemporal receptive Želds. 4.5 The Case of a Nonlinear (Nonnegative) Cell Model. Linear Žlters with negative coefŽcients or negative-valued data have signed outputs. Their widespread use as simple-cell models is often motivated by an interpretation in which the term simple cell in fact refers to a unit of two cells. These two positive-output cells, with reversed polarities, are modeled using a single linear Žlter with signed output.2 This two-cell approach will be explained in detail below. At this point, notice that this coupling is very different from complex-cell pooling of simple cells. In complex-cell pooling, the coupled cells respond to similar features at different spatial positions, whereas two cells with opposite polarities respond to similar features at the same spatial position. For single simple cells, a more realistic basic model of the mapping from input to output via a receptive Želd is a combination of linear Žltering and a nonlinearity called half-wave rectiŽcation (Heeger, 1992). Using the same notation as above, the output of the cell, yk .t/; is computed by yk .t/ D maxf0; wTk x.t/g;

(4.1)

instead of the purely linear input-output relationship, equation 2.1. In this model, the output of a cell is never negative. 2 Another possibility would be to consider the negative and positive values as changes from maintained Žring rate. However, simple cells have a low maintained Žring rate, which makes this approach undesirable (Heeger, 1992).

684

J. Hurri and A. Hyva¨ rinen

A

B

C

t

t

t

x

x

x

Figure 11: An illustration of how temporal response strength correlation could be exhibited by the outputs of simple-cell-like spatiotemporal receptive Želds. A phenomenon analog to translation in the spatial case can be observed in the spatiotemporal case. Let x and y denote the horizontal and vertical spatial coordinates, respectively, and let t denote the temporal coordinate. (A) The spatiotemporal trace (solid line) of a moving vertical line is shown here in the x–t coordinate system. The plot is similar for all y-coordinates because the moving line is vertical. Two different overlapping spatiotemporal input windows, separated by a small time difference, are also marked: one with dashed line and the other with dotted line. (B) A simple-cell-like spatiotemporal receptive Želd, with position and orientation that match the initial position of the line and its direction of movement, responds strongly to the moving line. Here the spatiotemporal Žlter has been superimposed over the dashed temporal window. White indicates large, positive values in the Žlter, dark indicates large negative values, and middle gray indicates zero values. (C) When the same spatiotemporal receptive Želd, at the same spatial position, is applied to the same input a moment later (dotted spatiotemporal input window), the response is still strong, but the sign changes. Therefore, the temporal response strength correlation of the outputs of the simple-cell-like spatiotemporal receptive Želd would be large for this kind of input.

The purely linear model 2.1 combines the outputs of two such positiveoutput simple cells with reversed polarities. This is implemented so that the positive output values correspond to the output of one cell, and the negative values correspond to the output of another cell, with otherwise similar receptive Želd except for a change of the sign of all the connection weights. The exact way an input pattern is mapped into a response in such a model is as follows. Let yk;1 .t/ and yk;2 .t/ denote the outputs of two cells with reversed polarities. Their outputs are given according to equation 4.1 by yk;1 .t/ D maxf0; wTk x.t/g and yk;2 .t/ D maxf0; ¡wTk x.t/g: The overall output

Simple-Cell Receptive Fields and Temporal Coherence

685

is deŽned as yk .t/ D yk;1 .t/ ¡ yk;2 .t/:

(4.2)

It is straightforward to show that this model is equivalent to a purely linear model, that is, that yk;1 .t/ ¡ yk;2 .t/ D wk x.t/: Therefore, one might interpret our model as measuring the temporal coherence of this two-cell unit, where the cells have similar receptive Želds with reversed polarities. A large value of the objective function indicates that either of the two cells responds strongly to the input. As mentioned above, using the linear model is a common approach. The same model is used, for example, in Bell and Sejnowski (1997), Olshausen and Field (1996), and van Hateren and van der Schaaf (1998), which makes the comparison of results obtained with these different models feasible. However, a natural question is whether the same principle applies to the outputs of individual, half-wave rectiŽed simple cells. To address this issue, we computed the optimal solution for half-wave rectiŽed cell outputs, that is, replaced the original linear input-output relationship (see equation 2.1) with the half-wave rectiŽed relationship (see equation 4.1), and computed the optimal solution for this model. The same constraint (see equation 2.4) was used in this case for simplicity. Instead of using the temporally decorrelated data set, we used the orthonormalized data set with 1t D 120 ms; also used in control experiment VI (see section 3.4.4). This is because by using the orthonormalized data set, we excluded the possibility of obtaining simple-cell-like receptive Želds because of the static part of the video. This is important here since the static part might yield positive responses at consecutive time instances. The results of this experiment are shown in Figure 12A. Although the features are not as well deŽned as before, the resulting Žlters still show orientation, localization, and different spatial scales. What is the reason for the emergence of such features even in this case when the negative part of the linear Žlter response has been discarded and the static part of the video has been removed completely? First, as in the purely linear case, the objective function emphasizes the temporal correlation of large responses, and oriented and localized Žlters respond strongly to lines and edges. But how does temporal correlation arise? An illustration of a possible explanation is shown in Figure 12B. When a Gabor-like Žlter is applied to a line, the half-wave rectiŽed output still correlates positively over time, but more weakly and over longer spatial displacements. Finally, let us note that some form of temporal coherence of simple-cell outputs is implicitly assumed in many studies. The Žring rate of a simple cell is typically assumed to code for the output of a linear Žlter, possibly after some simple nonlinear transformations, such as half-wave rectiŽcation. This requires that the output of the linear Žlter has some temporal coherence. If the output of the linear Žlter changes quite randomly, the Žring rate cannot

686

J. Hurri and A. Hyva¨ rinen

A

B

¡ 2¡ 1 0 1 2 3

¡ 2¡¡ 1 0 1 2 3

¡ 1 0 1

Figure 12: Temporal response strength correlation for a half-wave rectiŽed (nonlinear, nonnegative) cell model. (A) The results of running the algorithm for a nonlinear cell model with half-wave rectiŽcation of cell output. (B) When a simple-cell-like Žlter is applied to an input containing a moving line, the halfwave rectiŽed output is still correlated positively over time, but the correlation is weaker, and the spatial displacement needed for the correlation is larger. Cross sections of a line (left) and a Žlter (middle), and the half-wave rectiŽed output (right). See also Figure 9.

provide much information on the output, because then the Žring rate is very noisy when computed over short time intervals. Thus, maximization of temporal coherence could have a relation to maximization of the efŽciency of a code based on Žring rates. 4.6 Implications of Our Results. In this article, we have shown that simple-cell-type receptive Želds maximize temporal response strength correlation at cell output when the input consists of natural image sequences. Temporal response strength correlation, or temporal correlation of recti-

Simple-Cell Receptive Fields and Temporal Coherence

687

Žed cell outputs, can be considered as a measure of temporal coherence. Our Žndings have several important implications. First, temporal coherence provides an alternative or complementary theory to sparse coding as a computational principlebehind the formation of simple-cell receptive Želds. The application of either one of these principles results in the emergence of simple-cell properties from natural data. Second, Foldi´ ¨ ak and others (Foldi´ ¨ ak, 1991; Kayser et al., 2001; Kohonen, Kaski, & Lappalainen, 1997; Wiskott & Sejnowski, 2002) have proposed that invariant visual representations, such as those found in complex cells, can be found by maximizing temporal coherence. (For an alternative complexcell model using sparse coding, see Hyv¨arinen & Hoyer, 2001.) Our results show that this principle is applicable to the visual system even on the level of simple cells, which usually are not considered as invariant detectors. Although in some complex-cell models (Kayser et al., 2001; Kohonen et al., 1997) simple-cell receptive Želds are obtained as by-products, the learning is strongly modulated by the complex cells and therefore is very different from our model, which considers only the statistics of simple-cell outputs. Moreover, the simple-cell receptive Želds learned in Kayser et al. (2001) and Kohonen et al. (1997) do not seem to have the important properties of spatial localization and multiresolution (different scales). Furthermore, whereas most earlier research results linking temporal coherence and properties of visual system have been based on theoretical considerations and simulated data, the results published in this article have been computed from natural image sequence data. To our knowledge, this is the Žrst time that localized and oriented receptive Želds, with different scales, have been shown to emerge from natural data using the principle of temporal coherence. A step like this is important for any theory that tries to explain the structure and functionality of sensory neural networks using the statistical properties of natural input data.

Appendix: Details of the Symmetric Gradient Projection Algorithm This appendix gives a detailed description of the optimization algorithm. First, the maximization of objective function 2.3 under constraint 2.4 can be made easier by employing whitening, a temporary change of coordinates, so that constraint 2.4 is transformed into an orthonormality constraint. Let equation Cx D EDET denote the eigenvalue decomposition of matrix Cx : If an eigenvalue of Cx is zero, then that eigenvalue can be dropped out of eigenvalue matrix D; and the corresponding eigenvector can be removed from eigenvector matrix E: This was the case in our experiments because preprocessing reduced the dimensionality of input data x.t/ by one when the DC component was removed from data. This dimensionality reduction also means that the number of Žlters that can be extracted is K · N 2 ¡ 1:

688

J. Hurri and A. Hyva¨ rinen

DeŽning matrix U D WED1=2 ;

(A.1)

constraint 2.4 can be expressed as an orthonormality constraint, UUT D I:

(A.2)

This simpler constraint will be easier to handle in the optimization algorithm below. To express objective function 2.3 using the same transformed Žlter matrix U; we have to solve equation A.1 for matrix W: This is equivalent to solving matrix equation ET D1=2 WT D UT : In the case where the DC component has been removed, this is an underdetermined set of linear equations. By imposing the additional zero DC constraint on the Žlters in rows of W; the solution is given by W D UD¡1=2 ET

(A.3)

(see Hurri, 1997, for details). Substituting this into equation 2.2 gives y.t/ D UD¡1=2 ET x.t/ D Uz.t/;

(A.4)

where signal vector z.t/ D D¡1=2 ET x.t/: This transformation from input data x.t/ to z.t/ is called PCA whitening (Hyv¨arinen et al., 2001). The above shows that after input data x.t/ are whitened, we can optimize f .U/ D

K X kD1

Et fg.yk .t//g.yk .t ¡ 1t//g;

(A.5)

where output y.t/ is given by equation A.4, with respect to orthonormality constraint A.2. When the solution to this problem is found, the solution to the original problem is given by equation A.3. The actual optimization algorithm used for this constrained optimization problem is a variant of the gradient projection method of Rosen (for the original algorithm, see Luenberger, 1969). Let ®.m/ be a nonnegative decreasing sequence of real numbers, which converges to zero (initial step length ®.0/ is changed adaptively to speed up convergence). Let U.0/ be a random orthonormal matrix, and let U.n/ be the value of matrix U at iteration n: The algorithm Žnds a new candidate point by projecting matrix U.n/ C ®.m/ df .U.n// onto the constraint surface deŽned by equation A.2. If dU the candidate point is not an improvement, m is increased by one to Žnd a new candidate point. The critical step in the algorithm is the projection onto the constraint surface. This is achieved by optimal symmetric orthogonalization. Let A

Simple-Cell Receptive Fields and Temporal Coherence

689

be a square matrix with full rank. Then B D A.AT A/¡1=2 is the nearest orthonormal matrix to A with respect to Frobenius matrix norm (see Fan & Hoffman, 1955, theorem 1, for a generalized proof of this). This is exactly the property required to achieve the projection step. The resulting algorithm for maximizing temporal response strength correlation (TRSC) is shown below. The input arguments of the program are whitened data vectors z.t/; convergence tolerance ²; and initial step length ®.0/ (in this version ®.m/ D ®.0/ 2m ). Adaptation of initial step length ®.0/ is not included in this description. The algorithm assumes convergence when the objective function changes very little between successive steps. In our experiments, convergence tolerance ² varied between 10¡3 and 10¡6 : Note df .U/ that denoting U D [u 1 ¢ ¢ ¢ uK ]T ; the kth row of dU is given by the transpose of @ f .U/ @ D Et fg.yk .t//g.yk .t ¡ 1t//g @uk @uk D Et fg0 .yk .t//g.yk .t¡1t//z.t/ C g.yk .t//g0 .yk .t¡1t//z.t¡1t/g: funct U D max TRSC.z.t/; ²; ®.0// U.0/ :D rand./ comment: A random initial starting point. U.0/ :D symmetricOrth.U.0// f Old :D 0 n :D 0 while f .U.n// ¡ f Old > ² f Old :D f .U.n// m :D 0 ± ± ²² df .U.n//

while f symmetricOrth U.n/ C ®.0/ · f Old 2m dU m :D m C 1 end ± ² df .U.n// U.n C 1/ :D symmetricOrth U.n/ C ®.0/ m 2 dU

n :D n C 1 end U :D U.n/

: funct B D symmetricOrth.A/ B :D A.AT A/¡1=2 : Acknowledgments We thank Hans van Hateren for permission to use the natural image sequences and for providing us with the program used to measure receptive Želd properties. We also thank Patrik Hoyer for helpful comments.

690

J. Hurri and A. Hyva¨ rinen

References Becker, S. (1993). Learning to categorize objects using temporal coherence. In S. J. Hanson, J. D. Cowan, & C. L. Giles (Eds.), Advances in neural information processing systems, 5 (pp. 361–368). San Mateo, CA: Morgan Kaufmann. Becker, S., & Hinton, G. (1992). Self-organizing neural network that discovers surfaces in random-dot stereograms. Nature, 355(6356), 161–163. Bell, A., & Sejnowski, T. J. (1997). The independent components of natural scenes are edge Žlters. Vision Research, 37(23), 3327–3338. Carandini, M., Heeger, D. J., & Movshon, J. A. (1997). Linearity and normalization in simple cells of the macaque primary visual cortex. Journal of Neuroscience, 17(21), 8621–8644. DeAngelis, G. C., Ohzawa, I., & Freeman, R. D. (1993a). Spatiotemporal organization of simple-cell receptive Želds in the cat’s striate cortex. I. General characteristics and postnatal development. Journal of Neurophysiology, 69(4), 1091–1117. DeAngelis, G. C., Ohzawa, I., & Freeman, R. D. (1993b). Spatiotemporal organization of simple-cell receptive Želds in the cat’s striate cortex. II. Linearity of temporal and spatial summation. Journal of Neurophysiology,69(4), 1118–1135. Dong, D. W. & Atick, J. (1995). Temporal decorrelation: A theory of lagged and nonlagged responses in the lateral geniculate nucleus. Network: Computation in Neural Systems, 6(2), 159–178. Fan, K., & Hoffman, A. J. (1955). Some metric inequalities in the space of matrices. Proceedings of the American Mathematical Society, 6, 111–116. Field, D. (1994). What is the goal of sensory coding? Neural Computation, 6(4), 559–601. Foldi´ ¨ ak, P. (1991). Learning invariance from transformation sequences. Neural Computation, 3(2), 194–200. Heeger, D. J. (1992). Normalization of cell responses in cat striate cortex. Visual Neuroscience, 9, 181–198. Hoyer, P. O., & Hyva¨ rinen, A. (2002). A multi-layer sparse coding network learns contour coding from natural images. Vision Research, 42(12), 1593–1605. Hurri, J. (1997). Independent component analysis of image data. Master’s thesis, Helsinki University of Technology. Available on-line: http://www. cis.hut.Ž/jarmo/publications/. Hyva¨ rinen, A., & Hoyer, P. O. (2001). A two-layer sparse coding model learns simple and complex cell receptive Želds and topography from natural images. Vision Research, 41(18), 2413–2423. Hyva¨ rinen, A., Karhunen, J., & Oja, E. (2001). Independent component analysis. New York: Wiley. Kayser, C., Einh a¨ user, W., Dummer, ¨ O., Konig, ¨ P., & Kording, ¨ K. (2001). Extracting slow subspaces from natural videos leads to complex cells. In G. Dorffner, H. Bischof, & K. Hornik (Eds.), ArtiŽcial neural networks—ICANN 2001 (pp. 1075–1080). New York: Springer-Verlag. Kohonen, T., Kaski, S., & Lappalainen, H. (1997). Self-organized formation of various invariant-feature Žlters in the adaptive-subspace SOM. Neural Computation, 9(6), 1321–1344.

Simple-Cell Receptive Fields and Temporal Coherence

691

Luenberger, D. G. (1969). Optimization by vector space methods. New York: Wiley. Mitchison, G. (1991). Removing time variation with the anti-Hebbian differential synapse. Neural Computation, 3(3), 312–320. Olshausen, B. A., & Field, D. (1996). Emergence of simple-cell receptive Želd properties by learning a sparse code for natural images. Nature, 381(6583), 607–609. Olshausen, B. A., & Field, D. (1997). Sparse coding with an overcomplete basis set: A strategy employed by V1? Vision Research, 37, 3311–3325. Oppenheim, A., & Schafer, R. (1975). Digital signal processing. Upper Saddle River, NJ: Prentice Hall. Palmer, S. E. (1999). Vision science—Photons to phenomenology. Cambridge, MA: MIT Press. Simoncelli, E. P., & Olshausen, B. A. (2001). Natural image statistics and neural representation. Annual Review of Neuroscience, 24, 1193–1216. Stone, J. (1996). Learning visual parameters using spatiotemporal smoothness constraints. Neural Computation, 8(7), 1463–1492. van Hateren, J. H., & Ruderman, D. L. (1998). Independent component analysis of natural image sequences yields spatio-temporal Žlters similar to simple cells in primary visual cortex. Proceedings of the Royal Society of London B, 265(1412), 2315–2320. van Hateren, J. H., & van der Schaaf, A. (1998). Independent component Žlters of natural images compared with simple cells in primary visual cortex. Proceedings of the Royal Society of London B, 265(1394), 359–366. Wiskott, L., & Sejnowski, T. J. (2002). Slow feature analysis: Unsupervised learning of invariances. Neural Computation, 14(4), 715–770. Received November 14, 2001; accepted August 23, 2002.

LETTER

Communicated by Michael Schmitt

Continuous-Time Symmetric HopŽeld Nets Are Computationally Universal Ï õ ma JiÏrõ´ S´ [email protected] Institute of Computer Science, Academy of Sciences of the Czech Republic, P.O. Box 5 182 07 Prague 8, Czech Republic

Pekka Orponen [email protected].Ž Laboratory for Theoretical Computer Science, Helsinki University of Technology, P.O. Box 5400, FIN-02015 HUT, Finland

We establish a fundamental result in the theory of computation by continuous-time dynamical systems by showing that systems corresponding to so-called continuous-time symmetric HopŽeld nets are capable of general computation. As is well known, such networks have very constrained Lyapunov-function controlled dynamics. Nevertheless, we show that they are universal and efŽcient computational devices, in the sense that any convergent synchronous fully parallel computation by a recurrent network of n discrete-time binary neurons, with in general asymmetric coupling weights, can be simulated by a symmetric continuous-time HopŽeld net containing only 18n C 7 units employing the saturated-linear activation function. Moreover, if the asymmetric network has maximum integer weight size wmax and converges in discrete time t¤ , then the corresponding HopŽeld net can be designed to operate in continuous time 2.t¤ ="/ for any " > 0 such that wmax 212n · "21=" . In terms of standard discrete computation models, our result implies that any polynomially space-bounded Turing machine can be simulated by a family of polynomial-size continuoustime symmetric HopŽeld nets. 1 Introduction In recent years, a number of studies have sought to understand the computational characteristics of “natural” dynamical systems. The results achieved include universal computation results for several types of ordinary differential equations (Asarin & Maler, 1994; Branicky, 1994), partial differential equations (Omohundro, 1984), and discrete iterations (Koiran, Cosnard, & Garzon, 1994; Moore, 1990; Siegelmann & Sontag, 1995). One much studied class of systems are those that are deŽned by various neural network models (Golden, 1996; Haykin, 1999; Siegelmann, 1999). This interest is motivated Neural Computation 15, 693–733 (2003)

c 2003 Massachusetts Institute of Technology °

694

Ï õ ma and P. Orponen J. S´

partly by the quest to understand the fundamental limits and possibilities of practical neurocomputing and partly by the realization that despite their formal simplicity, neural networks are computationally quite powerful, and thus may serve as a useful reference model for investigating more complicated systems. In general, the computational capabilities of discrete-time systems are fairly well understood, but in the area of continuous-time systems, much work remains to be done. (For an overview of the issues, see Moore, 1998, and Orponen, 1997a.) Continuous-time recurrent neural networks are an attractive class of computational models with applications in control, optimization, and signal processing (Cichocki & Unbehauen, 1993; Medsker & Jain, 2000). Probably the best-known and most widely used continuous-time recurrent network model is that popularized by HopŽeld (1984) and often called the continuous-time HopŽeld model. The dynamics of this model had actually been analyzed earlier by Cohen and Grossberg (1983) in a more general setting, but because of the afŽnity to the very inuential discrete-time binary-state version of the model (HopŽeld, 1982), this additive special case of the Cohen-Grossberg equations has become associated with HopŽeld’s name. As practical neural networks, proposed uses of HopŽeld nets include associative memory (HopŽeld, 1984) and fast approximate solution of combinatorial optimization problems (HopŽeld & Tank, 1985), and designs exist for implementing them in analog electrical (HopŽeld, 1984) and optical (Stoll & Lee, 1988) hardware. In this article, we prove a fundamental result concerning the computational power of continuous-time symmetric HopŽeld nets. It is well known (Cohen & Grossberg, 1983; HopŽeld, 1984) that the dynamics of any HopŽeld net with a symmetric coupling weight matrix is governed by a Lyapunov, or energy function. This is a bounded function deŽned on the state-space of a network, whose values are properly decreasing along any nonconstant trajectory of the network’s dynamics. In particular, such a symmetric network always converges from any initial state toward some stable equilibrium state. This is a very useful property for obtaining guaranteed behavior in practical applications, but would at Žrst sight seem to limit the networks’ general dynamical capabilities severely. For instance, nondamping oscillations of the network state, which seem to be an essential prerequisite of general computation, obviously cannot be created under this constraint (i.e., strictly speaking, continuous-time HopŽeld nets cannot simulate even a single alternating bit), whereas such oscillations are easily obtained in networks with asymmetric coupling weights. Nevertheless, we shall show that inŽnite oscillations are the only feature of general-purpose digital computation that cannot be reproduced in continuous-time symmetric HopŽeld nets. More precisely, we prove that any converging discrete-time computation described by a recurrent network of n ¸ 1 binary neurons working under synchronous fully parallel mode and with in general asymmetric coupling weights can be embedded in a

Continuous-Time Symmetric HopŽeld Nets

695

continuous-time HopŽeld net of 18n C 7 units with a symmetric coupling weight matrix and using the saturated-linear activation function. In particular, the simulation of t¤ discrete update steps can be achieved in continuous time 2.t¤ ="/, for any " > 0 such that wmax 212n · "21=" where wmax ¸ 1 is the maximum integer weight size of the discrete network. An experimental Ï õ ma & Orponen, 2000). validation of this result appeared previously (S´ A key observation is that any terminating computation by a discretetime deterministic network with state-space f0; 1gn must converge within 2n steps. A basic technique used in our proof is the construction of a symmetric continuous-time clock network—a simulated .n C2/-bit binary counter— that, using 8nC7 units, produces a sequence of 2n well-controlled oscillations (generated by the simulated second least signiŽcant counter bit) before it converges. This sequence of clock pulses is used to drive the rest of the network where each discrete neuron is simulated by a symmetrically coupled subnetwork of 10 continuous-time units. The continuous-time clock is already of some interest from a dynamical systems perspective because it provides, to our knowledge, the Žrst known example of a continuoustime Lyapunov system whose convergence time grows exponentially in the Ï õ ma & Orponen, 2001a). system dimension (S´ A similar, but considerably simpler, construction was used in the discretetime setting (Orponen, 1996) to prove the computational equivalence of symmetric and convergent asymmetric binary neural networks. Our construction also provides the technique for improving this discrete-time simulation by reducing its overhead from a quadratic number of units to the asymptotÏ õ ma, Orponen, & Antti-Poika, 2000). The original ically optimal linear size (S´ idea for the discrete-time clock network used by Orponen (1996), and on which our current construction is based, stems from the result by Goles and Mart´õ nez (1989). Another related work (Orponen, 1997b) concerns the simulation of discrete-time binary networks by continuous-time asymmetric networks that do not adhere to the Lyapunov property. It is quite easy to see (Lepley & Miller, 1983; Orponen, 1996) that families of polynomial-size discrete neural networks are computationally equivalent to (nonuniform) polynomially space-bounded Turing machines; more precisely, they compute the complexity class PSPACE=poly (Balca´ zar, D´õ az, & Gabarr o, ´ 1995). By the result in this article, we now know that continuoustime symmetric HopŽeld nets are also at least as powerful; given any polynomially space-bounded Turing machine, we can construct a family of polynomial-size continuous-time symmetric HopŽeld nets (one network for each input length) for simulating it. This is, to our knowledge, the Žrst result concerning the computational power of continuous-time symmetric networks, and it is somewhat surprising that they turn out to be computationally universal in this complexity-limited sense. It remains an open question whether this is also an upper bound on the power of such networks. A related line of study concerns the computational capabilities of Žnite discrete-time analog-state neural networks (Siegelmann, 1999). Here it

696

Ï õ ma and P. Orponen J. S´

is known that the computational power of asymmetric networks using the saturated-linear activation function increases with the Kolmogorov complexity of the real weight parameters (Balcazar, ´ Gavalda, ` & Siegelmann, 1997). With integer weights, such networks coincide with binary networks and so are equivalent to Žnite automata (Horne & Hush, 1996; Indyk, 1995; Ï õ ma & Wiedermann, 1998), while with rational weights, arbitrary Turing S´ machines can be simulated (Indyk, 1995; Siegelmann & Sontag, 1995). With arbitrary real weights, the networks can even have “super-Turing” computational capabilities (Siegelmann & Sontag, 1994). To some extent, similar results also apply to discrete-time symmetric analog networks that are computationally equivalent to their asymmetric counterparts when an external Ï õ ma et al., 2000). On the other hand, clock pulse sequence is provided (S´ it is known that any amount of analog noise reduces the computational power of discrete-time analog recurrent networks to that of Žnite automata (Casey, 1996; Maass & Orponen, 1998) or even less (Maass & Sontag, 1999; Siegelmann, Roitershtein, & Ben-Hur, 2000). A preliminary version of this article appeared as an extended abstract Ï õ ma & Orponen, 2001b). This article is organized as follows. After a brief (S´ review of the basic deŽnitions in section 2, our main construction of the symmetric continuous-time HopŽeld net simulating a given discrete-time binary neural network is outlined in section 3, where its dynamics is informally explained. The formal veriŽcation of this construction, which has the form of a rather tedious case analysis, is given in section 4. In section 5, a numerical simulation example witnessing the validity of the construction is presented. Section 6 concludes with some open problems. 2 Preliminaries We Žrst specify the model of a Žnite discrete-time binary-state recurrent neural network (DRNN). Such a network consists of n simple computational binary units or neurons, indexed as 1; : : : ; n, that are connected into a generally cyclic directed graph or architecture. Each edge .i; j/ in this graph, leading from neuron i to j, is labeled with an integer coupling weight w.i; j/ D wji 2 Z. Note that for binary neurons, integer weights can be assumed without loss of generality (see Parberry, 1994). The absence of an edge in the architecture indicates a zero weight between the respective neurons, and vice versa. An instantaneous state of a DRNN at time t ¸ 0 is described by a vector .t/ n y.t/ D .y.t/ 1 ; : : : ; yn / 2 f0; 1g composed of current binary outputs (states) .t/ yj from particular neurons j D 1; : : : ; n. We consider here only the synchronous fully parallel dynamics in which the evolution of the network state y.t/ is determined for discrete time instants t D 0; 1; : : : ; as follows. At the beginning of a computation, the network is placed in some initial state y.0/ , which may include an external input. At discrete time t ¸ 0, each neuron j D 1; : : : ; n collects its local binary inputs from the states y.t/ i 2 f0; 1g of

Continuous-Time Symmetric HopŽeld Nets

697

incident neurons i and determines its integer excitation, .t/

»j

D

n X

wji y.t/ i ;

iD0

j D 1; : : : ; n;

(2.1)

as the respective weighted sum of these inputs. Moreover, sum 2.1 includes an integer bias wj0 2 Z local to each neuron j, which is formally modeled as a coupling weight from an additional neuron with index 0 and constant .tC1/ unit output y.t/ 0 ´ 1. At the next instant t C 1, the new network state y is computed by applying an activation function to all excitations componentwise, .tC1/

yj

D H.»j.t/ /;

j D 1; : : : ; n;

where the Heaviside or threshold function, ( 1 for » ¸ 0 H.» / D ; 0 for » < 0

(2.2)

(2.3)

is employed. We also denote by wmax D

max

jD1;:::;nIiD0;:::;n

jwji j

(2.4)

the maximum weight size in the network. Similarly, a Žnite continuous-time analog neural network is composed of m analog units that operate (in our case) with the saturated-linear activation function 8 > <1 for » ¸ 1 (2.5) ¾ .» / D » for 0 < » < 1 > : 0 for » · 0: Hence, the states yp .t/ of analog units p D 1; : : : ; m at time t ¸ 0 are real numbers within the interval [0; 1], and the coupling weights (including biases), denoted by v.p; q/ 2 R for edges from p to q, are reals as well. The computational dynamics of a continuous-time network is deŽned for every real t > 0, for example, by the following system of ordinary differential equations in real variables y1 ; : : : ; ym 2 [0; 1]: dyp dt

.t/ D ¡yp .t/ C ¾ .»p .t//;

p D 1; : : : ; m;

(2.6)

where »p .t/ D

m X qD0

v.q; p/yq .t/

(2.7)

Ï õ ma and P. Orponen J. S´

698

is the real-valued excitation for unit p at time t ¸ 0 (cf. equation 2.1) including bias v.0; p/ 2 R, which is associated with an additional formal variable y0 whose value is constant y0 .t/ ´ 1 in time. The initial network state y.0/ 2 [0; 1]m then determines the boundary conditions for system 2.6. In particular, we shall consider the continuous-time (symmetric) HopŽeld networks, whose architecture is an undirected graph with symmetric weights, v.p; q/ D v.q; p/

for every 1 · p; q · m:

(2.8)

The dynamics of a continuous-time symmetric HopŽeld net described by system 2.6 is controlled by the following Lyapunov or energy function, introduced by Cohen and Grossberg (1983): E.y/ D ¡

m X m m m Z yp X X 1X v.q; p/yq yp ¡ v.0; p/yp C ¾ ¡1 .y/ dy: 2 pD1 qD1 0 pD1 pD1

(2.9)

The characteristic properties of function E.y/ are that it is bounded on the system’s state-space [0; 1]m and that it is properly decreasing (i.e., dE=dt < 0) along any nonconstant trajectory of the network’s dynamics. It then follows that the continuous-time HopŽeld net always converges, from any initial state, toward some stable equilibrium state with dyp =dt D 0 for all p D 1; : : : ; m. 3 Constructing the Continuous-Time HopŽeld Net We now show how to simulate the convergent computations of a given DRNN N on a somewhat bigger continuous-time HopŽeld network H. In our simulation, the binary output values 0 and 1 of discrete neurons in N will be represented by excitations 2.7 of corresponding analog units in H that are below the lower saturation threshold of 0 or above the upper saturation threshold of 1, respectively, for activation function 2.5. For brevity, we simply say that a unit p is saturated at 0 or 1 at time t if its excitation satisŽes »p .t/ · 0 or »p .t/ ¸ 1, respectively. We say that p is unsaturated when 0 < »p .t/ < 1. Note that we use the excitations »p .t/ of continuous-time units p 2 H rather than their actual states yp .t/ to represent the binary values since starting at any point within the open interval .0; 1/, the outputs yp .t/ of the units saturated at 0 or 1 converge only to limit values 0 or 1, respectively, and never reach these boundaries (see point 1 in lemma 2 below). In fact, this is the main difference between the discrete dynamics 2.2 and its simulation within the continuous dynamics 2.6. The following theorem summarizes the result: Theorem 1. Any fully parallel computation by a recurrent neural network N of n ¸ 1 binary neurons with asymmetric integer weights of maximum size wmax ¸ 1,

Continuous-Time Symmetric HopŽeld Nets

699

converging within t¤ discrete update steps, can be simulated by a continuous-time symmetric HopŽeld network H with m D 18n C 7 analog units employing the saturated-linear activation function, within continuous time 2.t¤ ="/ for any real " > 0 such that wmax 212n · "21=" :

(3.1)

Proof. Since the DRNN N deŽnes a deterministic dynamical system over the state-space f0; 1gn , any converging computation by N must terminate within t¤ · 2n discrete steps. To achieve the simulation of N by HopŽeld net H , we Žrst construct in section 3.1 a continuous-time symmetric clock subnetwork C D C nC1 with 8n C 7 units that simulates a discrete .n C 2/bit binary counter. When the network C is initialized in the zero state, its interface unit x1 will exhibit a sequence of 2n well-controlled oscillations before C converges. Each “clock pulse” from x1 2 C is then exploited to “drive” a simulation of one parallel discrete update iteration by a simulating subnetwork S of size 10n units that is constructed in section 3.2. In particular, n G S D [jD1 j consists of n continuous-time symmetric gate subnetworks G j (j D 1; : : : ; n) each having 10 units for simulating one discrete neuron j 2 N . We Žrst discuss the operation of the continuous-time HopŽeld net H D C [ S intuitively; its correctness will be veriŽed formally in section 4. 3.1 The Clock Network. The construction of the continuous-time symmetric clock network C D C nC1 simulating an .n C 2/-bit binary counter of “order (n+1)” will be described by induction on n. The induction starts with the 2-bit counter network C 1 presented in Figure 1. The undirected edges connecting units in this graph are labeled with the corresponding symmetric weights, whereas the oriented edges drawn without an originating unit correspond to the biases. Assume that initially all of the states of the units indicated in this network are zero. The least signiŽcant counter unit c0 of “order 0” has positive bias v.0; c0 / D " > 0, corresponding to its initial positive excitation. Because of its feedback coupling v.c0 ; c0 / D 1 C " > 1, the state of c0 gradually grows from initial 0 toward 1. Eventually c0 saturates at 1, at which point we say that the unit c0 becomes active or Žres. Recall that we associate the simulated discrete counter behavior to the excitations of the units rather than their outputs. The external state of c0 evolves smoothly converging to 1 and exhibits no abrupt “Žring” transitions. Thus, c0 simply implements counting from 0 to 1 as required. This trick of gradual transition from 0 to 1, formally described in lemma 3, is used repeatedly throughout our construction of C . The remaining six units c1 ; a1 ; x1 ; b1 ; d1 ; z1 in the 2-bit counter network C 1 (see Figure 1) are of order 1. They function similarly to the corresponding units of higher orders, so we will describe only the inductive construction for general order k > 1. The eventual interconnection of the clock network C to the simulating subnetwork S , that is, to the gate subnetworks G j

Ï õ ma and P. Orponen J. S´

700

1+e 1+e

1+e 1

c1

1

3 1+e 1+e

1+e

e

c0

1+e 1

a1 W+4 x1

2 1+e 1

b1

1+ e 3

1+e (12u+39) n

3

d1 1+e

z1

1+e 3+e

W

A

B

Figure 1: A 2-bit continuous-time counter C 1 .

(j D 1; : : : ; n), is taken into account in the weight v.a1 ; x1 / D W C 4, which includes a large, positive integer parameter, 0 0 11 n X 3 jwj0 jAA > 0; (3.2) W D @nu C 28 @n C 2 jD1 where 0 u D 28 max @¡ jD1;:::;n

1 X 1·i·nIwji ·0

wji ;

X 1·i·nIwji ¸0

wji A ¸ 0;

(3.3)

and wji are the weights of the original DRNN N to be simulated. This large weight is needed because the interface unit x1 transfers the pulses generated by the clock C to all of the gate subnetworks G j , and its operation within C must not be affected by feedback effects from S (see section 3.2). For the induction step depicted in Figure 2, assume that an “order .k¡1/” counter network C k¡1 (1 < k · n C 1) has been constructed, containing the Žrst k counter units c0 ; : : : ; ck¡1 2 C k¡1 , together with auxiliary units a` ; x` ; b` ; d` ; z` 2 C k¡1 (` D 1; : : : ; k ¡ 1), and for k > 2 also x0` ; b0` (` D 2; : : : ; k ¡ 1), for a total of mk D jC

k¡1 j

D k C 5.k ¡ 1/ C 2.k ¡ 2/ D 8k ¡ 9

(3.4)

Continuous-Time Symmetric HopŽeld Nets

701

units. Then the next counter unit ck is connected to all the mk units p 2 C k¡1 via unit weights v.p; ck / D 1, which, together with its bias v.0; ck / D ¡mk C " and feedback weight v.ck ; ck / D 1 C ", cause ck to Žre shortly after all these units are active (see lemma 3). This includes the Žrst k active counter bits c0 ; : : : ; ck¡1 , which means that the simulated counting from 0 to 2k ¡ 1 has been accomplished, and, hence, the next counter bit ck must Žre. In addition, unit ck is connected to a sequence of seven auxiliary units ak ; x0k ; b0k ; xk ; bk ; dk ; zk with feedback weights v.ak ; ak / D v.x0k ; x0k / D v.b0k ; b0k / D v.xk ; xk / D v.bk ; bk / D v.dk ; dk / D v.zk ; zk / D 1 C ", which are being, one by one, activated after ck Žres (see lemma 3). This is implemented by the weights v.ck ; ak / D mk , v.ak ; x0k / D ¡v.x0k ; x1 / C 1, v.x0k ; b0k / D v.xk ; bk / D v.bk ; dk / D 1, v.b0k ; xk / D Vk C v.x0k ; x1 /, v.dk ; zk / D Vk ¡ mk , and biases v.0; ak / D ¡mk C ", v.0; x0k / D v.0; xk / D v.0; dk / D ¡1 C ", v.0; b0k / D v.0; bk / D ¡1 C "=3, v.0; zk / D mk ¡ Vk C ", where v.x0k ; x1 / < 0 is in its absolute value a sufŽciently large, negative weight and Vk > 0 is a sufŽciently large, positive parameter, whose values will be determined by formulas 3.5 and 3.8, respectively, so that units x0k ; xk ; zk are not directly inuenced by the computations occurring in units from C k¡1 except via ck . The purpose of the auxiliary units ak ; b0k ; bk ; dk is only to slow down the continuous-time state ow in order to synchronize the computation. The units x0k ; xk are used to reset all the lower-order units in C k¡1 back to values near 0 by saturating them at 0 (see lemma 2, point 2b) after ck Žres, which is consistent with the correct counter computation. The units that saturate at 0 are then called passive. To achieve this effect, x0k is connected to x1 via a large, negative weight, v.x0k ; x1 / D ¡Skx1 ¡ .12u C 39/n; and similarly, xk is linked with each p 2 C

(3.5) k¡1 nfx1 g

via negative weight

v.xk ; p/ D ¡Skp ;

(3.6)

where parameter 2 Skp D 6 6v.ck ; p/ C 6 q2C

3 X k¡1 Iv.q;p/>0

v.q; p/7 7; 7

p2C

k¡1

(3.7)

exceeds the positive inuence of units in C k¡1 [ fck g on p, and term ¡.12u C 39/n in equation 3.5 balances the total positive inuence on the interface unit x1 originating from units A in the gate subnetworks G j (see section 3.2). Now, the value of parameter Vk D 1 ¡ v.x0k ; x1 / ¡

X p2C

k¡1 nfx1 g

v.xk ; p/

(3.8)

Ï õ ma and P. Orponen J. S´

702

1+e m k+e

mk

ck

Ck

1

mk m k+e

ak

x1 1+e

1 v(x’k ,x1) 1

v(x’k ,x1)+1 1+e 1+ e 3

x’k 1

bk’

p 1+e

v(xk ,p) 1

1+e 1+e

xk 1+e

Vk m k 1

Vk v( x’k ,x1)+1

Vk+ v(x’k ,x1) 1+e

1

1

bk 1+ e 3

1+e

1

dk 1+e

Vk m k

zk

1+e

m k Vk +e

Figure 2: Inductive construction of C k .

is determined so that unit xk Žres after b0k is activated in spite of the negative contributions through weights 3.6 from p 2 C k¡1 nfx1 g to the excitation of xk . Note that the interface unit x1 2 C k¡1 is Žrst suppressed separately by x0k (see lemma 4), while the remaining units in C k¡1 nfx1 g are active. In particular, units a1 and b1 incident on x1 that are of the same order 1 remain activated after x1 becomes passive because of active c1 and d1 , respectively, and unit c0 is connected with x1 via a negative weight (see Figure 1). Furthermore, units in C k¡1 of order greater than 1 are not directly inuenced by passive x1 . Thus, only after x1 becomes passive and b0k ; xk are activated, the other units from C k¡1 nfx1 g are reset by xk . Finally, unit zk balances the negative inuence of x0k ; xk on C k¡1 so that the Žrst k counter bits can again count from 0 to 2k ¡ 1, but now with ck being

Continuous-Time Symmetric HopŽeld Nets

703

active. This is achieved by exact positive weights ( v.zk ; p/ D

¡v.x0k ; x1 / ¡ 1 ¡v.xk ; p/ ¡ 1

for p D x1 for p 2 C k nfx1 g;

(3.9)

in which ¡v.x0k ; x1 / and ¡v.xk ; p/ eliminate the inuence of x0k and xk , respectively, on p 2 C k¡1 , whereas ¡1 compensates for v.ck ; p/ D 1 for each p 2 C k¡1 . Clearly, units p 2 C k¡1 cannot reversely activate zk since their maximal contribution to the excitation of zk , X X v.p; zk / D ¡mk ¡v.x0k ; x1 /¡ v.xk ; p/ D Vk ¡mk ¡1; (3.10) p2C

k¡1

p2C

k¡1 nfx1 g

according to equations 3.9 and 3.8, cannot overcome its bias v.0; zk / D mk ¡ Vk C ". This completes the induction step of the counter network construction. 3.2 The Simulating Network. The high-level scheme of the simulating subnetwork S is depicted in Figure 3. Network S consists of a sequence of Žve layers, each linked to the subsequent one. A signal is propagated through these layers (see point 2b of lemma 2) only in one direction (from top to bottom in Figure 3), although the respective connections are symmetric. For this purpose, a device used previously for symmetric implementation of feedforward networks (Parberry, 1994) is applied here. In particular, the absolute values of weights and biases in the symmetric network are arranged in a decreasing sequence, which prevents the signal from backpropagating. However, to make recurrent computation possible, the “bottom” layer is further connected to the “top” layer—but only by weights of small absolute values that need additional strong support from the clock C in order to transmit a signal. Therefore, Figure 3 also shows the clock subnetwork C that has been constructed in section 3.1. Particularly, interface unit x1 of C is connected to the Žrst layer denoted by A D f®j ; ¯j I j D 1; : : : ; ng by large, positive weights, and at the same time x1 is linked to the fourth layer B D fÁj ; Âj I j D 1; : : : ; ng by negative weights of large absolute values (see Figure 1). Within the simulated .n C 2/–bit binary counter C nC1 , unit x1 is of the second least signiŽcant order 1 and thus Žres 2n times. To each discrete computational step of network N that converges after t¤ · 2n updates corresponds a twophase continuous-time state dynamics of S , controlled by one oscillation of x1 2 C . In particular, the next simulated state of N is computed in the Žrst phase when unit x1 is passive. In this phase, the Žrst layer A is locked; no signal can pass through A, since there is no support from C . At the same time, the negative inuence of x1 on the fourth layer B is suppressed. The third layer of S serves as a memory that encodes the current binary state y.t/ of the

Ï õ ma and P. Orponen J. S´

704

1.layer: feedback transfer

A= {a j , bj ; j=1,...,n} copying

+

CLOCK

2.layer: memory driver

{z j ,hj ; j=1,...,n}

x1

storing 3.layer: memory

y (t)

{p j , rj ; j=1,...,n}

computing

­

4.layer: memory driver

B= {fj , c j ; j=1,...,n} storing

5.layer: memory

y (t+1)

{yj ,wj ; j=1,...,n}

Figure 3: A high-level scheme of the simulation.

simulated network N at discrete-time instant t. This memory is properly initialized by the initial state y.0/ of N . The new state y.tC1/ of N at each time instant t C 1 is then computed from y.t/ in the subsequent fourth layer by using the original weights of N , and it is further stored in the following Žfth memory layer. This completes the Žrst phase, and S is stabilized until x1 Žres. In the second phase, when unit x1 is active, the new simulated state of N replaces the old one. The fourth layer B is now locked due to the large negative weights from x1 , while the Žrst layer A is unlocked by the positive support from C . The new state y.tC1/ is transmitted from the Žfth

Continuous-Time Symmetric HopŽeld Nets

705

memory layer back to the Žrst layer in a cycle, and further transferred to the subsequent second layer that updates the contents of the third memory layer by new state y.tC1/ . Thus, one simulation step is completed, and the simulating subnetwork S is stabilized until x1 becomes passive again. Now, the detailed implementation of the simulating subnetwork S will be presented. As indicated in Figure 3, each of the Žve layers in S is composed of n pairs of continuous-time units—one pair for each discrete neuron j in the network N to be simulated. Each system of Žve unit pairs (one from each layer) that are associated with the same neuron j creates the symmetric gate subnetwork G j with 10 units for simulating discrete neuron j 2 N as depicted in Figure 4. Besides the deŽnition of weights in G j , this Žgure again shows the interface clock unit x1 2 C that is linked to each G j . In what follows, we informally describe the dynamics of G j (j D 1; : : : ; n) that simulates the discrete-time updates of neuron j 2 N . For simplicity of presentation, we assume (see point 2b of lemma 2) that particular units in the continuoustime network G j approximately follow the discrete update rule 2.1–2.3 when their inputs are saturated for the duration of a certain period. At the beginning of the simulation, all units in G j are placed in zero initial states, except possibly for units ¼j ; %j 2 G j that doubly encode the initial state of discrete neuron j 2 N , y¼j .0/ D y%j .0/ D yj.0/ ;

(3.11)

which means that both units ¼j ; %j are initially saturated at value yj.0/ 2 f0; 1g. The Žrst phase of a general simulation step corresponding to the update of state of neuron j at discrete-time instant t C 1 starts when the interface unit x1 2 C becomes passive. Consequently, units ®j ; ¯j 2 A saturate at 0 and remain passive due to their negative biases (see Figure 4). Only then (see lemma 4) are the states of units Áj ; Âj 2 B allowed to evolve since the inuence of negative weights from x1 disappears with passive x1 . Assume now that the simulated state yj.t/ 2 f0; 1g of neuron j 2 N at discrete-time instant t is doubly represented by continuous-time units ¼j ; %j 2 G j that are both saturated at value yj.t/ . In particular, units ¼j ; %j are either both passive due to their negative biases and passive ³j , if yj.t/ D 0, or they are saturating each other at 1 by their positive coupling weight v.¼j ; %j / while ´j is passive, if yj.t/ D 1.

The value corresponding to the new state yj.tC1/ 2 f0; 1g of neuron j is computed by unit Áj 2 G j , based on its connections to the units ¼i 2 G i (i D 1; : : : ; n) that represent the simulated states y.t/ at the i of neurons i 2 N previous discrete-time instant t. Therefore, these connections are labeled with symmetric weights v.¼i ; Áj / D 28wji , and unit Áj has bias v.0; Áj / D 28wj0 C 14, where wji are the original weights of N . The large coefŽcients are used to guarantee that Áj is not inuenced by Ãj ; !j while the original

Ï õ ma and P. Orponen J. S´

706

­ 12u ­ 40 3

­ 12u ­ 37

aj 4 u +9 ­ 4 u­ 7

­ u­ 1

12u +39

hj

­ 2 u­ 3

pj

u u 28wj 0 +14

4 u +9

12u +39

zj

2 u +3

2 u +3

rj

­ 28wji

fj

­ 3 (u +28|wj 0|+28) 2

cj

yj

6

6

u ­ 28wj0 ­ 14

­6

wj

­4 Figure 4: A continuous-time gate network G

­ u­ 1 u

x1

­6

­1

­ 2 u­ 3 2 u +3

28wji

6

­3

bj

­1 j

simulating the discrete neuron j.

function of j is preserved (Parberry, 1994). Similarly, unit Âj computes the negation of yj.tC1/ from units %i 2 G i by using the opposite weights v.%i ; Âj / D ¡28w.i; j/ and bias v.0; Âj / D ¡28wj0 ¡ 14, which also prevent Âj from being inuenced by Ãj ; !j . Note that because units ¼i ; %i have biases v.0; ¼i / D v.0; %i / D ¡u ¡ 1, where u is the large nonnegative parameter deŽned by formula 3.3, the state evolution of Áj ; Âj cannot reversely affect the dynamics of units ¼i ; %i , respectively. In addition, units Áj ; Âj 2 G j are never simultaneously active; either they are both passive, or each of them represents the negation of a binary state encoded by the other. Therefore, the positive parameter W introduced in equation 3.2 and implanted in weight v.a1 ; x1 / is sufŽcient for

Continuous-Time Symmetric HopŽeld Nets

707

activating the clock unit x1 2 C regardless of the total negative inuence on x1 from units in B (see Figure 1). Furthermore, units Áj ; Âj store the new state yj.tC1/ into both Ãj and !j in such a way that either Áj ensures activating Ãj ; !j through positive weights, if yj.tC1/ D 1, or Âj arranges for their resetting by means of negative weights, if yj.tC1/ D 0. Consequently, continuous-time units Ãj ; !j store the simulated

state yj.tC1/ even if Áj ; Âj later become passive, since they are either both passive due to their negative biases, if yj.tC1/ D 0, or they are saturating each

other at 1 via their positive coupling weight v.Ãj ; !j /, if yj.tC1/ D 1. Since units ®j ; ¯j 2 A are temporarily locked, the gate network G j becomes stable until unit x1 is activated. In the second phase, interface unit x1 2 C becomes active and locks units Áj ; Âj 2 B by saturating them at 0 before releasing units ®j ; ¯j 2 A (see lemma 4). Unit ®j then receives yj.tC1/ from unit Ãj , as the positive support from x1 balances the negative bias of ®j so that the small, positive weight v.Ãj ; ®j / can have an inuence on ®j . Similarly, unit ¯j computes the negation of the binary state yj.tC1/ received from !j by means of the support from .tC1/ and its negation are further propagated in G j from ®j ; ¯j to x1 . State yj ³j ; ´j , respectively, and these units then update the simulated state of neuron j stored in ¼j ; %j by its new value yj.tC1/ . Since the negative weights v.x1 ; Áj / D v.x1 ; Âj / D ¡3.u C 28jwj0 j C 28/=2 from x1 overcome the total positive inuence on Áj ; Âj that originates in their incident units, network G j is temporarily stabilized until x1 becomes passive again and computing the new discrete state yj.tC2/ is initiated. Recall also that units ®j ; ¯j 2 G j are never simultaneously active, which implies that the term ¡.12u C 39/n in deŽnition 3.5 of v.x0k ; x1 / (k > 1) sufŽces to suppress the total positive inuence from A on x1 (see Figure 1). This completes the construction of the n G . simulating subnetwork S D [jD1 j

4 Formal VeriŽcation Now the correct state evolution of the continuous-time HopŽeld network H described in section 3 needs to be veriŽed. This is achieved by a sequence of lemmas analyzing the behavior of the corresponding system of differential equations 2.6. Lemma 1 Žrst upper bounds the maximum sum of absolute values of weights incident on any unit in H. Lemma 2 then describes explicitly the continuous-time state evolution for saturated units. An analysis of how the decreasing defects (i.e., distances from limit values in the states of saturated units) affect the excitation of any other unit reveals that the units in H actually approximate the discrete update rule 2.1–2.3 of corre-

Ï õ ma and P. Orponen J. S´

708

sponding binary neurons after a certain transient time, provided that the incident saturated units stay saturated. Furthermore, the transfer of the activity in the clock C from a unit to a subsequent one, when all the incident units are saturated, will be analyzed explicitly in lemma 3. (But note that the dynamics of unit c0 at time t D 0 slightly differs from this analysis.) A crucial fact for the proof of theorem 1 is that the duration time of this transfer turns out to be constant and unaffected by any initial defect. This introduces a “discrete time” into the clock operation and enables the clock to synchronize the simulation. In point 2 of lemma 3, the result is also partially generalized to the case when some of the incident units may become unsaturated. Finally, lemma 4 will verify that the clock interface unit x1 2 C correctly synchronizes the simulation by network S ; locking the units in A and B always precedes unlocking the units in B and A, respectively. Lemma 1. For any unit p 2 H in the HopŽeld network constructed in section 3, the sum of absolute values of its incident weights (excluding its local bias) is upper bounded by 4p D

m X qD1

jv.q; p/j < "21=" :

(4.1)

Proof. Consider Žrst the input weights to the clock interface unit x1 2 C as indicated in Figures 1, 2, and 4: 4x1 D v.x1 ; x1 / C jv.c0 ; x1 /j C v.a1 ; x1 / C v.b1 ; x1 / C C

nC1 X kD2 n X jD1

.v.ck ; x1 / C jv.x0k ; x1 /j C v.zk ; x1 // .v.®j ; x1 / C v.¯j ; x1 / C jv.Áj ; x1 /j C jv.Âj ; x1 /j/;

(4.2)

which reduces to 4x1 D 3W C 2.12u C 39/n C 9 C " C 2

nC1 X kD2

jv.x0k ; x1 /j

(4.3)

by using deŽnitions 3.9 and 3.2. Weights v.x0k ; x1 / deŽned by formulas 3.5 and 3.7 can be expressed recursively, v.x0k ; x1 / D v.x0k¡1 ; x1 / ¡

X q2C

v.q; x1 /

k¡1 nC k¡2 Iv.q;x1 />0

D v.x0k¡1 ; x1 / ¡ v.ck¡1 ; x1 / ¡ v.zk¡1 ; x1 / D 2v.x0k¡1 ; x1 /;

(4.4)

Continuous-Time Symmetric HopŽeld Nets

709

for k D 3; : : : ; n C 1 according to Figure 2 and equation 3.9, whereas v.x02 ; x1 / D ¡dv.c2 ; x1 /Cv.x1 ; x1 /Cv.a1 ; x1 /Cv.b1 ; x1 /e¡.12uC39/n D ¡.W C .12u C 39/n C 8/

(4.5)

follows from Figure 1. Hence, v.x0k ; x1 / D ¡2k¡2 .W C .12u C 39/n C 8/

(4.6)

for k D 2; : : : ; n C 1, which can be substituted into formula 4.3: 4x1 D 2nC1 .W C .12u C 39/n C 8/ C W ¡ 7 C ":

(4.7)

Furthermore, as indicated in Figure 2, the input weights to the clock unit znC1 2 C of the highest order n C 1 give 4znC1 D v.znC1 ; znC1 / C v.dnC1 ; znC1 / C

X

p2C

v.p; zk / n

D 2VnC1 ¡ 2mnC1 C " D 2VnC1 ¡ 16n C 2 C "

(4.8)

according to equations 3.10 and 3.4. Parameter VnC1 will be computed recursively starting with V2 D 1 ¡ v.x02 ; x1 / ¡

X p2C

2 D 1C

X p2C

1

v.x2 ; p/ 1 nfx1 g

6v.c2 ; p/ C 6 6 q2C

3 X 1 Iv.q;p/>0

v.q; p/7 7 C .12u C 39/n 7

D 2W C .12u C 39/n C 48;

(4.9)

which follows from equations 3.5 through 3.8 and Figure 1. For 2 < k · nC1, deŽnition 3.8 of Vk can be rewritten as Vk D 1 ¡ v.x0k ; x1 / ¡

X p2C

k¡2 nfx1 g

v.xk ; p/ ¡

X p2C

v.xk ; p/:

(4.10)

k¡1 nC k¡2

Similarly as in equation 4.4, a recursive formula for the weights v.xk ; p/ for p 2 C k¡2 nfx1 g (k > 2) deŽned by formulas 3.6 and 3.7 can be derived, v.xk ; p/ D v.xk¡1 ; p/ ¡

X

q2C

v.q; p/

k¡1 nC k¡2 Iv.q;p/>0

D v.xk¡1 ; p/ ¡ v.ck¡1 ; p/ ¡ v.zk¡1 ; p/ D 2v.xk¡1 ; p/;

(4.11)

Ï õ ma and P. Orponen J. S´

710

according to Figure 2 and equation 3.9. By introducing formulas 4.4 and 4.11 into equation 4.10 and using deŽnition 3.8, the recursive formula for Vk is obtained: X (4.12) Vk D 2Vk¡1 ¡ 1 ¡ v.xk ; p/: p2C

k¡1 nC k¡2

The weights v.xk ; p/ for p 2 C k¡1 nC k¡2 D fck¡1 ; ak¡1 ; x0k¡1 ; b0k¡1 ; xk¡1 ; bk¡1 ; dk¡1 ; zk¡1 g in equation 4.12 can be calculated from Figure 2 and by deŽnitions 3.6 and 3.7 in which dv.ck ; p/ C v.p; p/e D 3, as follows: ¡v.xk ; ck¡1 / D 3 C v.ak¡1 ; ck¡1 / C

X

q2C

v.q; ck¡1 /

k¡2 Iv.q;ck¡1 />0

D 2mk¡1 C 3

(4.13)

D ¡v.x0k¡1 ; x1 / C mk¡1 C 4

(4.14)

¡v.xk ; ak¡1 / D 3 C v.ck¡1 ; ak¡1 / C v.x0k¡1 ; ak¡1 / ¡v.xk ; x0k¡1 / ¡v.xk ; b0k¡1 /

D D D D

¡v.xk ; xk¡1 / D D

3 C v.ak¡1 ; x0k¡1 / C v.b0k¡1 ; x0k¡1 / ¡v.x0k¡1 ; x1 / C 5 3 C v.x0k¡1 ; b0k¡1 / C v.xk¡1 ; b0k¡1 / Vk¡1 C v.x0k¡1 ; x1 / C 4

(4.15)

(4.16)

3 C v.b0k¡1 ; xk¡1 / C v.bk¡1 ; xk¡1 / Vk¡1 C v.x0k¡1 ; x1 / C 4

(4.17)

¡v.xk ; bk¡1 / D 3 C v.xk¡1 ; bk¡1 / C v.dk¡1 ; bk¡1 / D 5

(4.18)

¡v.xk ; dk¡1 / D 3 C v.bk¡1 ; dk¡1 / C v.zk¡1 ; dk¡1 / D Vk¡1 ¡ mk¡1 C 4 ¡v.xk ; zk¡1 / D 3 C v.dk¡1 ; zk¡1 / C

X q2C

D 2Vk¡1 ¡ 2mk¡1 C 2;

(4.19) v.q; zk¡1 /

k¡2 Iv.q;zk¡1 />0

(4.20)

where formula 3.10 has been employed in equation 4.20. Weights 4.13 through 4.20 are summed up as ¡

X p2C

k¡1 nC k¡2

v.xk ; p/ D 5Vk¡1 C 31;

(4.21)

which is plugged in formula 4.12: Vk D 7Vk¡1 C 30:

(4.22)

Continuous-Time Symmetric HopŽeld Nets

711

It follows from equations 4.9 and 4.22 that VnC1 D 7n¡1 .2W C .12u C 39/n C 53/ ¡ 5;

(4.23)

which gives 4znC1 D 2 ¢ 7n¡1 .2W C .12u C 39/n C 53/ ¡ 16n ¡ 8 C ";

(4.24)

according to equation 4.8. Observe from Figure 4 that the maximum value of 4p among units p 2 S within the simulating subnetwork is reached by ®j ; ¯j 2 S , that is, 4®j D 4¯j D 16u C 51 < 4x1 ;

j D 1; : : : ; n;

(4.25)

which is still less than 4x1 associated with the clock interface unit x1 2 C according to equation 4.7. On the other hand, the maximum value of 4p among units p 2 C within the clock subnetwork is reached by unit znC1 2 C of the highest order n C 1 for n ¸ 2—for example, compare formula 4.8 to 4xnC1 D 2VnC1 ¡ 2v.x0nC1 ; x1 / C 1 C "

D 2VnC1 ¡ 2n .W C .12u C 39/n C 8/ C 1 C " < 4znC1

(4.26)

computed by formula 4.6—while 4x1 > 4z2 dominates for n D 1 according to equations 4.7 and 4.24. Hence, 4p · max.4znC1 ; 4x1 /

(4.27)

for every unit p 2 H . Clearly, " · 0:0625

(4.28)

follows from assumption 3.1 in which n ¸ 1, wmax ¸ 1, and "21=" is decreasing for " > 0. In addition, the following two upper bounds, W · 42n2 wmax C 42nwmax C 42n u · 28nwmax ;

(4.29) (4.30)

are obtained from deŽnitions 3.2 and 3.3, respectively, which together with equations 4.7 and 4.24 and inequality 4.28 ensure that condition 4.27 implies 4p < 12 ¢ 7n .10n2 wmax C 11nwmax C 3n C 2/;

(4.31)

which, for n ¸ 1, gives 4p < wmax 212n · "21=" according to assumption 3.1.

(4.32)

Ï õ ma and P. Orponen J. S´

712

Lemma 2. 1. Let p 2 H be a unit saturated at b 2 f0; 1g with a defect ±p .t/ D jb ¡ yp .t/j;

(4.33)

for the duration of a continuous-time interval ¿ D [t0 ; t f ] for some t0 ¸ 0. Then the state dynamics of p converging toward value b can be explicitly solved as yp .t/ D jb ¡ ±p e¡.t¡t0 / j

(4.34)

for t 2 ¿ , where ±p D ±p .t0 / is p’s initial defect. 2a. Let Q µ H be a subset of units saturated for the duration of time interval ¿ D [t0 ; t f ]. Then the dynamics of the excitation »p .t/ for any unit p 2 H can be described as »p .t/ D v.0; p/C for t 2 ¿ , where 1pQ D

X v.q; p/C q2QI»q .t/¸1

X

q2QI»q .t0 /·0

v.q; p/±q ¡

X

v.q; p/yq .t/C1pQ e¡.t¡t0 / (4.35)

q2H nQ

X

v.q; p/±q

(4.36)

q2QI»q .t0 /¸1

is the initial total weighted defect of Q affecting »p .t0 /. 2b. In addition, let tf > t0 C t1 where t1 D

ln 2 ; "

(4.37)

and assume that the respective weights in H satisfy either X X v.0; p/ C v.q; p/ C v.q; p/ < ¡"

(4.38)

q2H nQIv.q;p/>0

q2QI»q .t0 /¸1

or v.0; p/ C

X q2QI»q .t0 /¸1

v.q; p/ C

X q2H nQIv.q;p/<0

v.q; p/ > 1 C ":

(4.39)

Then p is saturated at either 0 or 1, respectively, for the duration of time interval [t0 C t1 ; t f ]. Proof. 1. According to dynamics 2.5 and 2.6, the state yp .t/ of unit p saturated at b 2 f0; 1g for t 2 ¿ is independent of outputs from the remaining units, and

Continuous-Time Symmetric HopŽeld Nets

713

its continuous-time dynamics is described by a differential equation, dyp dt

.t/ D ¡yp .t/ C b;

(4.40)

with a boundary condition, yp .t0 / D jb ¡ ±p j;

(4.41)

obtained from formula 4.33 for initial defect ±p . Hence, its explicit solution 4.34 follows. 2a. The excitation »p .t/ of unit p deŽned by formula 2.7 is split among the contributions from units outside Q and from those in Q saturated at 0 and 1 whose dynamics for t 2 ¿ is given by equation 4.34: »p .t/ D v.0; p/ C C

X

X q2H nQ

q2QI»q .t/¸1

v.q; p/yq .t/ C

X

v.q; p/±q e¡.t¡t0 /

q2QI»q .t/·0

v.q; p/.1 ¡ ±q e¡.t¡t0 / /:

(4.42)

By introducing the initial total weighted defect 4.36 into formula 4.42, the dynamics 4.35 follows. 2b. The defect term 1pQ e¡.t¡t0 / in equation 4.35 vanishes quickly as time proceeds, and its absolute value can be bounded for t 2 [t0 Ct1 ; t f ] as follows, j1pQ e¡.t¡t0 / j · 4p e¡t1 < ";

(4.43)

by using lemma 1 and equation 4.37. Hence, unit p is saturated at either 0 or 1 for the duration of time interval [t0 C t1 ; tf ] when condition 4.38 or 4.39, respectively, is assumed in equation 4.35. Lemma 3. 1. Consider a situation as depicted in Figure 5, where a clock unit p 2 C with fractional part of bias " 0 2 f"; "=3g and feedback weight v.p; p/ D 1 C " is supposed to receive a signal from preceding clock unit o 2 C , activate itself, and further transfer the signal to a subsequent clock unit r 2 C with bias fraction " and v.r; r/ D 1 C " via weight v.p; r/ ¸ 1. Let all the units incident on p; r excluding p; r be saturated for the duration of some sufŽciently large time interval ¿ D [t0 ; t f ] (e.g., t f > t0 C t2 where t2 is deŽned by formula 4.49 below), starting at a time t0 > 0 when »p .t0 / D 0. Assume that the initial defects meet ±p C 1rQ < "

(4.44)

Ï õ ma and P. Orponen J. S´

714

1+e

1+e

v ( p , r) > 1

p ..+e’

r ..+e

Figure 5: Activity transfer from unit p to unit r in the clock subnetwork C .

for Q D Hnfpg. Further assume that the respective weights satisfy X v.0; p/ C v.q; p/ D " 0

(4.45)

q2QI»q .t0 /¸1

v.0; r/ C

X q2QI»q .t0 /¸1

v.q; r/ D " ¡ v.p; r/:

(4.46)

Then p is unsaturated with the state dynamics yp .t/ D

" 0 .e".t¡t0 / ¡ 1/ " 0 C 1pQ e¡.t¡t0 / ¡ 1C" ".1 C "/

(4.47)

exactly for the duration of time interval .t0 ; t0 C t01 /, where t01 D

ln.1 C

" "0 /

(4.48)

"

(note t01 D t1 for " 0 D " and t01 D 2t1 for " 0 D "=3), and r is saturated at 0. In addition, p is saturated at 1 for the duration of time interval [t0 C t01 ; t0 C t2 ] and remains further saturated independently of the output from r, while r unsaturates from 0 at time t0 C t2 where t2 D ln

v.p; r/.." C " 0 /.1 C

" 1=" "0 /

C 1pQ / ¡ .1 C "/1 rQ

".1 C "/

¸ t01 :

(4.49)

2. Consider a situation as depicted in Figure 6, where a clock unit p 2 C with bias v.0; p/ D ¡1 C " and feedback weight v.p; p/ D 1 C " is supposed to receive a signal from preceding clock unit o 2 C , activate itself, and further transfer the signal to a subsequent clock unit r 2 C with v.0; r/ D ¡1C"=3 and v.r; r/ D 1 C", via weight v.p; r/ D 1, while the units in a given subset R µ H , incident on p (with no connections to r), may unsaturate after p unsaturates from 0. Let all the units

Continuous-Time Symmetric HopŽeld Nets

715

R

q

1+e

o

1+e 1

p

r 1+ e 3

1+e

Figure 6: Activity transfer from unit p to r when units q 2 R unsaturate.

incident on p; r, excluding p; r, and R, be saturated for the duration of a sufŽciently large time interval ¿ D [t0 ; tf ] (e.g., at least until r unsaturates from 0) starting at a time t0 > 0 when »p .t0 / D 0. In Figure 6, Bt0 D fq 2 BI »q .t0 / ¸ 1g µ S

(4.50)

denotes the subset of units from simulating subnetwork S that are saturated at 1 at time t0 . Assume that the initial defects meet ±r < "2¡1="

(4.51)

1rQ0 < "2¡1="

(4.52)

for Q0 D Hn.R [ fpg/, and .1 C "/±p C

X

q2RI»q .t0 /·0

v.q; p/±q ¡

X q2RI»q .t0 /¸1

v.q; p/±q · "2¡1="

(4.53)

outside Q0. Further, assume that the respective weights satisfy v.0; p/ C

X q2Q0 I»q .t0 /¸1

v.q; p/ C

X q2RIv.q;p/<0

X q2Q0 I»q .t0 /¸1

v.q; p/ D "

(4.54)

v.q; r/ D 0:

(4.55)

Ï õ ma and P. Orponen J. S´

716

Then p saturates at 1 in time at most t0 C 2t1 , remaining saturated until time at least t f , and r unsaturates from 0 only after p is saturated at 1. Proof. 1. A summary of the dynamics of units p; r under discussion here is presented in Table 1, which is veriŽed step by step below. Excitation »p .t/ D " 0 C .1 C "/yp .t/ C 1pQe¡.t¡t0 /

(4.56)

of unit p at time t 2 [t0 ; t0 C t2 ] is derived from equation 4.35 by assumption 4.45 and determines p’s state dynamics 2.6 by differential equation dyp dt

.t/ D ¡yp .t/ C " 0 C .1 C "/yp .t/ C 1pQ e¡.t¡t0 /

(4.57)

when p is unsaturated. The corresponding boundary condition,

yp .t0 / D

¡" 0 ¡ 1pQ D ±p ; 1C"

(4.58)

comes from »p .t0 / D 0, which is applied to formula 4.56, and also determines the initial defect ¡ 1pQ D " 0 C .1 C "/±p ;

(4.59)

Table 1: Chronology of the Dynamics of Units p; r.

t = t0 2 (t0 ; t0 + tg ) = t0 + tg 2 (t0 + tg ; t0 + t01 ) = t0 + t01 2 (t0 + t01 ; t0 + t2 ) = t0 + t2 2 (t0 + t2 ; tf ]

¹ p (t) yp (t) =0 =¯ p >0

dyp (t) dt

¹ r (t)

yr (t) =¯ r

<0

= ¯ r e¡(t¡t0 )

=0 >0

|

<0

eq. 4.47 = 0

=1

eq. 4.64

>1

eq. 4.70

>0

Continuous-Time Symmetric HopŽeld Nets

717

which can be bounded as ¡ 1 ¡ " ¡ " 0 · 1pQ · ¡" 0 < 0

(4.60)

due to 1 ¸ ±p ¸ 0. The explicit solution 4.47 for differential equation 4.57 follows when initial condition 4.58 is provided. By plugging solution 4.47 in equation 4.57, an explicit formula for p’s state derivative can be calculated: dyp dt

.t/ D

e¡.t¡t0 / ." 0 e.1C"/.t¡t0 / C 1pQ / : 1C"

(4.61)

It follows that before the state yp .t/ of unit p starts to grow, it is initially nonincreasing within time t 2 [t0 ; t0 C tg ] where ¡1pQ

ln "0 tg D ; 1C"

(4.62)

since p’s state derivative 4.61 is nonpositive for t 2 [t0 ; t0 C tg ], and tg ¸ 0 according to inequality 4.60. Further, by introducing solution 4.47 into equation 4.56, the dynamics of p’s excitation can also be expressed explicitly: »p .t/ D

" 0 .e".t¡t0 / ¡ 1/ > 0: "

(4.63)

This ensures that unit p is unsaturated for the duration of the whole time interval .t0 ; t0 C t01 / even though its state yp .t/ is initially decreasing for t 2 .t0 ; t0 Ctg /. Eventually unit p saturates at 1 exactly at time instant t0 Ct01 where t01 is given by formula 4.48, derived from equation 4.63 for »p .t0 C t01 / D 1. The state of unit p at t0 C t01 can be computed by substituting formula 4.48 into equation 4.47: yp .t0 C t01 / D

¡ 1 ¡ " 0 ¡ 1pQ 1 C 1C"

¢ " ¡1=" "0

:

(4.64)

Also, t01 > tg according to inequality 4.28, which conŽrms the actual growth of p’s state. Notice that the length t01 of the period when p is unsaturated is constant and independent of the initial defects. This introduces a notion of “discrete time” into the clock operation based on t1 . Recall that the detailed chronology of the dynamics of units p; r during the activity transfer is shown in Table 1. Similarly, excitation »r .t/ D " ¡ v.p; r/ C v.p; r/yp .t/ C 1rQ e¡.t¡t0 /

(4.65)

Ï õ ma and P. Orponen J. S´

718

of unit r in time t 2 [t0 ; t0 C t2 ] is derived from equation 4.35 by assumption 4.46. In order to verify that the state dynamics of unit p is indeed controlled by equation 4.57 for the duration of time interval .t0 ; t0 C t01 /, it must also be checked that unit r remains saturated at 0 in this period, that is, »r .t/ · 0. Since v.p; r/ ¸ 1, according to equation 4.65, it sufŽces to show »r .t/ · " C yp .t/ ¡ 1 C 1rQ e¡.t¡t0 / · 0

(4.66)

for all t 2 [t0 ; t0 C t01 ], which can be rewritten as "." 0 C.1C"/.±p C1rQ //e¡.t¡t0 / C" 0 .e".t¡t0 / ¡1/¡" · "." 0 ¡" 2 /

(4.67)

by substituting solution 4.47 for yp .t/ in which ¡1pQ is replaced by formula 4.59. Inequality 4.67 further reduces to "." 0 C ".1 C "//e¡.t¡t0 / C " 0 .e".t¡t0 / ¡ 1/ ¡ " · "." 0 ¡ " 2 /

(4.68)

due to assumption 4.44. For t 2 [t0 ; t0 C t" ], where t" D ln

" 0 C ".1 C "/ ; "0 ¡ "2

(4.69)

term e".t¡t0 / reaches its maximum at time instant t0 C t" , whereas e¡.t¡t0 / · 1, which implies inequality 4.68 by using condition 4.28. For t 2 [t0 Ct" ; t0 Ct01 ], on the other hand, term "." 0 C ".1 C "//e¡.t¡t0 / in inequality 4.68 achieves its maximum "." 0 ¡ " 2 / at time instant t0 C t" , while " 0 .e".t¡t0 / ¡ 1/ ¡ " · 0 reaches 0 at time instant t0 C t01 . Hence, unit r is saturated at 0 within the period .t0 ; t0 C t01 / when unit p is unsaturated. The state yp .t/ of unit p saturated at 1 follows further the dynamics equation 4.34, that is, 0

yp .t/ D 1 ¡ ±p .t0 C t01 /e¡.t¡t0 ¡t1 / ± ¡ ¢¡1=" ² ¡.t¡t ¡t0 / 0 1 " C " 0 C 1pQ 1 C ""0 e D 1¡ ; 1C"

(4.70)

for t 2 [t0 C t01 ; tf ], where the corresponding defect ±p .t0 C t01 / D 1 ¡ yp .t0 Ct01 / is calculated from formula 4.64. By substituting formula 4.70 into equation 4.56, the dynamics of p’s excitation is obtained, 0

»p .t/ D 1 C ." C " 0 /.1 ¡ e¡.t¡t0 ¡t1 / / ¸ 1;

(4.71)

for t 2 [t0 C t01 ; t0 C t2 ], which conŽrms that unit p remains saturated at 1 at least until unit r becomes unsaturated from 0. Also, excitation »r .t/ of

Continuous-Time Symmetric HopŽeld Nets

719

unit r saturated at 0 after p saturates at 1 can be expressed by introducing formula 4.70 into equation 4.65 as follows: 0

»r .t/ D " ¡ v.p; r/±p .t0 C t01 /e¡.t¡t0 ¡t1 / C 1rQ e¡.t¡t0 / ;

(4.72)

which reaches 0 at time instant t0 C t2 , that is, »r .t0 C t2 / D 0, which gives formula 4.49 for t2 by using equation 4.48. Hence, unit r is unsaturated from 0 after time t0 C t2 , and the dynamics of r’s state is further described by a differential equation of the form 4.57. In this way, the activity of unit p is transferred to r. Finally, it must be checked that unit p remains saturated at 1 when r is unsaturated from 0. For this purpose, excitation 4.56 of unit p can be rewritten as »p .t/ D " 0 C 1 C " C v.p; r/yr .t/

¡ .1 C "/±p .t0 C t2 /e¡.t¡t0 ¡t2 / C .1pQ ¡ v.p; r/±r /e¡.t¡t0 / (4.73)

for t 2 [t0 Ct2 ; t f ] according to equation 4.35 in which the subset of saturated units Q is now replaced with Q1 D H nfrg, while the initial defects ±r ; 1pQ in equation 4.73 are still related to time instant t0 and Q D Hnfpg. The defect ±p .t0 C t2 / at time instant t0 C t2 of unit p saturated at 1 can be calculated from equation 4.70, ±p .t0 C t2 / D 1 ¡ yp .t0 C t2 / D

" v.p; r/ ¡

.1C"/1rQ

¡

¢1=" "

."C" 0 / 1C " 0

;

(4.74)

C1pQ

where formula 4.49 is used. In order to prove that »p .t/ ¸ 1 for all t 2 [t0 C t2 ; tf ], the underlying negative defect terms in equation 4.73 having the least value for t D t0 C t2 will be lower bounded by ¡" 0 ¡ ", whereas v.p; r/yr .t/ ¸ 0 is neglected, ensuring that unit p remains saturated at 1 independently of the output from r. Thus, it is sufŽcient to prove ¡.1 C "/±p .t0 C t2 / C .1pQ ¡ v.p; r/±r /e¡t2 ± ² ¡ ¢1=" ¡".1C"/ ." C " 0 / 1C ""0 Cv.p; r/±r ± ² D > ¡" 0 ¡" ¡ ¢ " 1=" 0 C1pQ ¡ .1C"/1rQ v.p; r/ ." C " / 1C "0

(4.75)

where formulas 4.74 and 4.49 have been employed, which further reduces to ³ ± ´ " ²1=" C 1rQ .1 C "/." C " 0 / " 1 C 0 " ³ ´ ± " ²1=" 0 2 0 C." C " /1pQ ¡".1C"/±r : (4.76) < v.p; r/ ."C" / 1C 0 "

Ï õ ma and P. Orponen J. S´

720

According to assumption 4.44 and equation 4.59, it sufŽces to show inequality 4.76 with 1rQ and 1pQ replaced by " and ¡" 0 ¡ .1 C "/", respectively. In addition, v.p; r/ ¸ 1 and ±r · 1, which leads to ³ ± ´ " ²1=" C" .1 C "/." C " 0 / " 1 C 0 " ± ² 1=" " ¡ " 0 ." C " 0 / ¡ ".1 C "/.1 C " C " 0 /; < ." C " 0 /2 1 C 0 "

(4.77)

which holds due to inequality 4.28. This completes the argument for unit p to be saturated at 1 after r becomes unsaturated from 0. 2. Note that unit o saturates at 1 according to case 1 of this lemma before p is unsaturated from 0 at time instant t0 . Excitation of unit p derived from equation 4.35 can be lower bounded as »p .t/ ¸ " C .1 C "/yp .t/ C 1pQ0 e¡.t¡t0 /

(4.78)

for t 2 ¿ by assumption 4.54. According to dynamics equation 2.6, this also provides the following lower bound on the state derivative of unsaturated p: dyp dt

.t/ ¸ "yp .t/ C " C 1pQ0 e¡.t¡t0 / :

(4.79)

Moreover, in the beginning of time interval ¿ , the state evolution of unit p is determined by formula 4.47 before the Žrst unit q 2 R becomes unsaturated, since assumption 4.45 coincides with assumption 4.54 due to " 0 D " and »q .t0 / ¸ 1 for all q 2 R with v.q; p/ < 0 (see the table in Figure 6). Also, the initial total weighted defect 1pQ0 for Q0 D H n.R [ fpg/ can be expressed in terms of 1pQ D ¡" ¡ .1 C "/±p for Q D H nfpg from equation 4.59 as follows, 1pQ0 D ¡" ¡ .1 C "/±p ¡

X q2RI»q .t0 /·0

v.q; p/±q C

X

v.q; p/±q ; (4.80)

q2RI»q .t0 /¸1

according to deŽnition 4.36. Hence, 1pQ0 ¸ ¡".1 C 2¡1=" /

(4.81)

by assumption 4.53. By introducing inequality 4.81 into formula 4.79, the state derivative of unsaturated p is further lower bounded as dyp dt

.t/ ¸ "yp .t/ C " ¡ ".1 C 2¡1=" /e¡.t¡t0 / :

(4.82)

Continuous-Time Symmetric HopŽeld Nets

721

Since "yp .t/ ¸ 0, it follows that dyp dt

.t/ ¸ " ¡ " 2 > 0

(4.83)

for t ¸ t0 C td , where td D ln

1 C 2¡1=" ; "

(4.84)

provided that p is still unsaturated. This implies that yp .t/ for t ¸ t0 C td grows at least as fast as the straight line with equation ." ¡ " 2 /.t ¡ t0 ¡ td / ¡ y D 0

(4.85)

until unit p saturates at 1. Hence, p saturates at 1 certainly before t0 Ctd Cts < t0 C 2t1 where ts D

1 " ¡ "2

(4.86)

because »p .t/ > yp .t/ from equation 2.6 due to p’s state derivative 4.83 is positive for t ¸ t0 C td . In addition, it will be proved that the subsequent unit r is saturated at 0 at least until p saturates at 1. Excitation »r .t/ D ¡1 C

" C yp .t/ C 1rQ0 e¡.t¡t0 / 3

(4.87)

of unit r saturated at 0 can be derived from equation 4.35 by assumption 4.55. Let ty > 0 be the least local time instant at which yp .t0 C ty / D 1 ¡

" ¡ 1rQ0 e¡ty 3

(4.88)

when r is still saturated at 0 since »r .t0 C ty / D 0 follows from equation 4.87. Thus, it sufŽces to prove that the excitation of unit p can be lower bounded at time instant t0 C ty as ± ² " »p .t0 C ty / ¸ " C .1 C "/ 1 ¡ ¡ 1rQ0 e¡ty C 1pQ0 e¡ty ¸ 1 3

(4.89)

according to inequality 4.78. By substituting the error bounds 4.52 and 4.81, this reduces to 5 ¡ " ¡ 3.1 C .2 C "/2¡1=" /e¡ty ¸ 0;

(4.90)

Ï õ ma and P. Orponen J. S´

722

which holds for condition 4.28 due to e¡ty < 1, ensuring that unit p is already saturated at 1 at time instant t0 C ty . Finally, it must be checked that unit p remains saturated at 1 after t0 C ty when unit r may become unsaturated from 0. Inequality 4.78 now reads as ± ±" ² ² C 1rQ0 e¡ty e¡.t¡t0 ¡ty / C yr .t/ »p .t/ ¸ " C .1 C "/ 1 ¡ 3 C .1pQ0 ¡ ±r /e¡.t¡t0 / ;

(4.91)

since the state dynamics of unit p saturated at 1 is controlled by equation 4.34. In order to prove that »p .t/ ¸ 1 for all t 2 .t0 C ty ; tf ], it sufŽces to show 6 ¡ .1 C "/e¡.t¡t0 ¡ty / ¡ 3.1 C .3 C "/2¡1=" /e¡.t¡t0 / ¸ 0

(4.92)

according to inequality 4.91 in which yr .t/ ¸ 0 and the error bounds 4.51, 4.52, and 4.81 have been applied. Inequality 4.92 follows from e¡.t¡t0 / · e¡.t¡t0 ¡ty / · 1 and condition 4.28. This completes the argument for unit p to be saturated at 1 after r becomes unsaturated from 0. Lemma 4. All the units in A [ B µ S are saturated at 0 when the state of the clock interface unit x1 2 C equals 2=3. 1. The state of unit x1 unsaturated from 0 at time t0 ¸ 0 (i.e., »x1 .t0 / D 0) after reaching value yx1 .t0 C t0` / D 2=3 from below at time t0 C t0` , remains further increasing at least until x1 is again unsaturated from 1 by the next x0k unit (2 · k · n C 1). 2. Let unit x0k (2 · k · n C 1) cease its saturation at 0 at time t0 ¸ 0 when »x0k .t0 / D 0. Following this, let unit x1 unsaturate from 1 immediately after time instant t0 C tu > t0 , where »x1 .t0 C tu / D 1, and reach its state value yx1 .t0 C t` / D 2=3 from above at time t0 C t` (tu < t` ). Let 1x1 Q1 > ¡" for Q1 D

H nfx0k g

(4.93)

and

1x1 Q01 > ¡"2¡1="

(4.94)

for Q01 D H n.At0 [ B [ fx1 ; x0k g/ where At0 D fq 2 AI »q .t0 / ¸ 1g:

(4.95)

Further assume that X

±x1 < "2¡1="

(4.96)

¡1="

v.q; x1 /±q < "2

(4.97)

v.q; x1 /±q > ¡"2¡1=" :

(4.98)

q2At0

X q2B

Continuous-Time Symmetric HopŽeld Nets

723

Then the state of unit x1 keeps decreasing after time t0 C t` at least until x1 is again unsaturated from 0 by a1 . Proof. It can Žrst be easily checked from Figure 4 that the excitation of all ®j ; ¯j ; Áj ; Âj 2 A [ B for j D 1; : : : ; n is negative when yx1 D 2=3: »®j · v.0; ®j / C v.Ãj ; ®j / C v.³j ; ®j / C v.x1 ; ®j / ¢ »¯j · v.0; ¯j / C v.´j ; ¯j / C v.x1 ; ¯j / ¢

2 D ¡2 3

2 D ¡2 3

»Áj · v.0; Áj / C u C v.Ãj ; Áj / C v.!j ; Áj / C v.x1 ; Áj / ¢ »Âj · v.0; Âj / C u C v.x1 ; Âj / ¢

(4.99) (4.100)

2 · ¡2 3

(4.101)

2 · ¡42: 3

(4.102)

1. Obviously, yx1 .t/ is further increasing for t ¸ t0 Ct` since .dyx1 =dt/.t/ > 0 for yx1 .t0 C t` / D 2=3 according to inequality 4.82, and hence, unit x1 then saturates at 1. 2. Excitation »x1 .t0 C tu / D W C .12u C 39/n C 3 C 2" C v.x0k ; x1 /yx0k .t0 C tu / C 1x1 Q1 e¡tu D 1

(4.103)

of unit x1 at t0 C tu is derived from equation 4.35 for Q1 where v.0; x1 /C

X

q2Q1 I»q .t/¸1

v.q; x1 / D v.0; x1 /Cv.x1 ; x1 /Cv.c0 ; x1 /Cv.a1 ; x1 / C v.b1 ; x1 / C v.ck ; x1 / C

X v.q; x1 / q2At0

D W C .12u C 39/n C 3 C 2"

(4.104)

follows from Figures 1 and 2, deŽnition 3.9, and the fact that ®j 2 At0 iff ¯j 2 = At0 , for all j D 1; : : : ; n (see section 3.2). Equation 4.103 gives yx0k .t0 C tu / ¸

W C .12u C 39/n C 2 C " ¡v.x0k ; x1 /

(4.105)

by assumption 4.93 and v.x0k ; x1 / < 0. We already know from inequality 4.82 where e¡.t¡t0 / < 1 that dyx0

k

dt

.t/ ¸ "yx0k .t/ ¡ "2¡1=" :

(4.106)

Ï õ ma and P. Orponen J. S´

724

We prove that the state yx0k .t/ of unit x0k keeps increasing after x1 is unsaturated from 1 for t > t0 C tu . Since yx0k is continuous, it sufŽces to show that its derivative satisŽes dyx0

k

dt

.t0 C tu / > 0

(4.107)

at time instant t0 C tu according to inequality 4.106, which further reduces to dyx0

k

dt

.t0 C tu / ¸

".W C .12u C 39/n C 2 C "/ ¡ "2¡1=" > 0 ¡v.x0k ; x1 /

(4.108)

by using condition 4.105. Inequality 4.108 follows from ¡v.x0k ; x1 / < "21=" according to lemma 1. Similarly, excitation X »x1 .t/ D v.0; x1 / C v.q; x1 / C .1 C "/yx1 .t/ C v.x0k ; x1 /yx0k .t/ C

X q2At0

q2Q01 I»q .t/¸1

v.q; x1 /yq .t/ C

X q2B

v.q; x1 /yq .t/ C 1x1 Q01 e¡.t¡t0 / (4.109)

of unit x1 for time t ¸ t0 is obtained from equation 4.35 for Q01 . Hence, the derivative of unsaturated x1 for t ¸ t0 C tu can be expressed as dyx1 dt

.t/ D v.0; x1 / C C

X q2At0

X q2Q01 I»q .t/¸1

v.q; x1 / C "yx1 .t/ C v.x0k ; x1 /yx0k .t/

v.q; x1 /yq .t/C

X q2B

v.q; x1 /yq .t/C1x1 Q01 e¡.t¡t0 / (4.110)

according to equation 2.6. Denote by 0 < t0d < t` the least local time instant such that dyx1 dt

.t0 C t0d / D 0:

(4.111)

Note that t0d > tu since unit x1 is still saturated at 1 at time t0 C tu when its state yx1 is increasing according to equation 4.34. The values deŽning the boundary time instants (printed in bold) for the variables associated with units x0k ; x1 and their signs or dynamics are summarized in Table 2. Now, the change in values of particular terms in the state derivative 4.110 of unit x1 will be estimated between time instants t0 C t` and t0 C t0d . Thus, yx1 .t0 C t0d / > yx1 .t0 / D 1 ¡ ±x1 > 1 ¡ "2¡1=" >

2 D yx1 .t0 C t` / 3

(4.112)

Continuous-Time Symmetric HopŽeld Nets

725

Table 2: The Review of the Boundary Time Instants for Units x0k ; x1 .

t= t0 t0 + tu t0 + t0d t0 + t`

¹

dyx 0

x0k (t)

k

dt

=0

<0

|

>0

(t)

¹

x1 (t)

>1 =1 |

yx1 (t) = 1 ¡ ¯ x1 = 1 ¡ ¯ x1 e¡tu | 2 = 3

dyx 1 dt

(t)

>0 =0 <0

by assumption 4.96, which implies " "yx1 .t0 C t` / ¡ "yx1 .t0 C t0d / < ¡ C " 2 2¡1=" : 3

(4.113)

Since v.x0k ; x1 / < 0 and yx0k .t/ is increasing for t ¸ t0 C tu , it follows that v.x0k ; x1 /yx0k .t/ ¡ v.x0k ; x1 /yx0k .t0 C t0d / < 0

(4.114)

for t ¸ t0 C t0d . In addition, v.q; x1 / > 0 for q 2 At0 µ A and all the units in At0 are still saturated at 1 at t0 C t0d because yx1 .t/ is nondecreasing for t0 · t · t0 C t0d , which gives X

q2At0

v.q; x1 /yq .t/ ¡

X

q2At0

v.q; x1 /yq .t0 C t0d / < "2¡1="

(4.115)

for t ¸ t0 C t0d according to assumption 4.97. Similarly, v.q; x1 / < 0 for q 2 B, and all the units in B are still saturated at 0 at t0 C t0d , which provides X q2B

v.q; x1 /yq .t/ ¡

X q2B

v.q; x1 /yq .t0 C t0d / < "2¡1="

(4.116)

for t ¸ t0 C t0d by assumption 4.98. Also, the defect term in equation 4.110 satisŽes 0

1x1 Q01 e¡.t¡t0 / ¡ 1x1 Q01 e¡td < "2¡1="

(4.117)

for t ¸ t0 C t0d according to assumption 4.94. By summing inequalities 4.113 through 4.117 for differences in particular terms of equation 4.110, we obtain dyx1 dt

.t0 C t` / D

dyx1 dt

.t0 C t` / ¡

dyx1 dt

.t0 C t0d /

< ¡"=3 C ".3 C "/2¡1=" < 0

(4.118)

Ï õ ma and P. Orponen J. S´

726

from inequality 4.28. The state derivative of x1 is further negative as inequality 4.118 is valid even for t ¸ t0 C t` until x1 saturates at 0, since bounds 4.114 through 4.117 still apply, while condition 4.113 holds for continuous yx1 .t/, which is further decreasing according to inequality 4.118. The correct timing of the simulation still needs to be veriŽed to ensure a sufŽciently fast decrease in the defects of the continuous-time correlates of binary states, because the analysis in lemmas 3 and 4 is valid only if the defect bounds 4.44, 4.51 through 4.53, 4.93, 4.94, and 4.96 through 4.98 are satisŽed. According to inequality 4.43, the absolute value of the total weighted defect of saturated units affecting any unit in H is bounded by " after transient time t1 , decreasing further to "2¡1=" by time 2t1 . On the other hand, t1 lower bounds the time necessary for activating a typical clock unit p 2 C (see the table in Figure 5) by point 1 in lemma 3. We will Žrst concentrate on the clock subnetwork C excluding its connection to simulating subnetwork S via interface unit x1 , which will be resolved later, together with the synchronization analysis for S . In order to validate assumption 4.44, consider a unit o0 2 C that has also been activated according to point 1 of lemma 3 last before unit p 2 C from formula 4.44 starts its activation. Clearly, unit o0 coincides with unit o 2 C in the table in Figure 5 except for o D x0k (k D 2; : : : ; n C 1) and o D xk (k D 1; : : : ; n C 1), whose activation is analyzed in point 2 of lemma 3 instead, and therefore 8 ak > > > b > k > : o

for p D b0k ; 2 · k · n C 1 for p D b1 for p D bk ; 2 · k · n C 1 otherwise:

(4.119)

It follows from Figures 1 and 2 that v.o0 ; r/ ¸ 0. In fact, only v.z1 ; ck / D 1 and v.zk ; c1 / > 0 for k D 2; : : : ; n C 1 are positive, while the remaining pairs o0 ; r are unconnected corresponding to v.o0 ; r/ D 0. In addition, v.p; r/ ¸ 1, and hence the defect in formula 4.44, can be upper bounded as follows: ±p C 1rQ · v.p; r/±p C 1rQ C v.o0 ; r/±o0 D 1rQ2 ;

(4.120)

where Q2 D H nfo0 g, according to deŽnition 4.36. Assume Žrst that unit r 6D x1 is not connected to any unit in S , which gives 1rQ2 D 1rQ02 ;

(4.121)

where Q02 D Q2 \ C . For o0 6D b0k (2 · k · n C 1) and o0 6D bk (1 · k · n C 1), all the units in Q02 are saturated when o0 is being activated, which takes time t1 . On the other hand, the activation of o0 D b0k ; bk takes time 2t1 due to its bias

Continuous-Time Symmetric HopŽeld Nets

727

v.0; o0 / D ¡1 C "=3, while simultaneously units in R \ C µ Q02 (for particular cases of R, see the table in Figure 6 in which o0 now corresponds to r) saturate within a time period of length t1 according to point 2b of lemma 2. Note that the corresponding unit, x0k or xk , preceding o0 D b0k or o0 D bk , respectively, is already saturated at 1 by point 2 of lemma 3 before o0 unsaturates from 0. Furthermore, all the units in Q02 are saturated for the duration of the next t1 period certainly before unit p from formula 4.44 unsaturates. It follows from inequality 4.43 that 1rQ02 < ", which implies condition 4.44 according to inequality 4.120 and equation 4.121. Similarly, for r D x1 (i.e., p D a1 ), it will be shown below that all the units in Q2 that are connected to r, especially units in A [ B t0 µ R from simulating subnetwork S , are saturated before o0 D c1 unsaturates, which clearly sufŽces for proving condition 4.44. Analogously, the stronger defect bounds 4.51 through 4.53 in point 2 of lemma 3 are met for p D x0k (2 · k · n C 1) and p D xk (1 · k · n C 1) since according to point 1 of lemma 3, the transient time 2t1 that is sufŽcient to decrease the underlying defects due to inequality 4.43 is guaranteed by the successive separate (all the units in C are saturated but one) activations of the preceding two units ck ; ak with v.ck ; r/ D v.ak ; r/ D 0 before unit p unsaturates from 0. It remains only to check inequality 4.53 for p D x1 when A [ Bt0 µ R, which again follows from the fact that units in A [ Bt0 µ S are saturated before o0 D c1 unsaturates, whose proof is given below. Now, the synchronization of the simulating subnetwork S will be analyzed. The Žrst phase of any simulated discrete step is delimited by the period during which the clock interface unit x1 2 C is passive. Recall that unit x1 is reset by x0k for some 2 · k · n C 1, which saturates at 1 by point 2 of lemma 3 before the subsequent unit b0k is unsaturated from 0. The activation of b0k then takes time 2t1 by point 1 of lemma 3, from which the Žrst period of length t1 sufŽces for x1 to saturate at 0 according to point 2b of lemma 2. Thus, the second t1 period of activating the b0k already belongs to the Žrst simulation phase. This phase then also includes the successive separate activations of units bk ; dk ; zk ; c0 ; c1 ; a1 , each of length t1 , except for the activation of bk , which requires time 2t1 according to point 1 of lemma 3. Altogether the length of the Žrst phase is certainly at least 8t1 . Note that in this consideration, the length of activating the xk is omitted for which only the upper bound is given in point 2 of lemma 3. Actually, the Žrst two t1 periods of the Žrst simulation phase are sufŽcient to stabilize the simulating subnetwork S , that is, to saturate all its units, even before dk unsaturates from 0. In particular, according to lemma 4, units in A are and remain saturated at 0 after yx1 D 2=3, while units in Bt0 only then saturate at 1 by point 2b of lemma 2 within the Žrst t1 period, long before o0 D c1 unsaturates, as required above. The following second t1 period ensures that the units in the second and Žfth layers of S saturate (see Figure 3). Similarly, the second phase of any simulated discrete step is deŽned as the period during which the clock interface unit x1 2 C is active. Recall that unit x1 is activated by a1 and saturates at 1 by point 2 of lemma 3 before the

728

Ï õ ma and P. Orponen J. S´

subsequent unit b1 becomes unsaturated from 0. Thus, the second simulation phase includes the successive separate activations of units b1 ; d1 ; z1 ; c0 ; ck ; ak for some 2 · k · n C 1, each of length t1 except for the activation of b1 , which requires time 2t1 according to point 1 of lemma 3. Altogether the length of the second phase is certainly at least 7t1 . In fact, already the Žrst three t1 periods of the second simulation phase are sufŽcient to stabilize the simulating subnetwork S , that is, to saturate all its units even before z1 unsaturates from 0. In particular, according to lemma 4, units in B are and remain saturated at 0 after yx1 D 2=3, while units in At0 only then saturate at 1 by point 2b of lemma 2 within the Žrst t1 period. The following two t1 periods ensure that the units in the second layer of S , followed by third-layer units, saturate according to point 2b of lemma 2 (see Figure 3). Obviously, the length of the second simulation phase that is lower bounded by 7t1 also guarantees the defect bounds 4.93, 4.94, and 4.96 through 4.98 in point 2 of lemma 4. Finally, the lower bound Ä.t¤ ="/ on the total simulation time follows immediately from the previous time analysis and equation 4.37. In addition, every unsaturated unit in H saturates within time at most O.t1 / according to point 2b of lemma 2 and lemma 3. Also during the simulation, all the units in H can simultaneously be saturated for a period of at most t1 before the respective defects decrease below " according to inequality 4.43, and the next unit unsaturates. This implies the corresponding upper bound O.t¤ ="/ and completes the proof of the theorem. 5 A Simulation Example A computer program, HNGEN, has been created to automate the construction from theorem 1. The input for HNGEN is a text Žle containing the asymmetric weights and biases of a given DRNN N , as well as its initial state. The program generates the system of differential equations 2.6 describing the dynamics of continuous-time symmetric HopŽeld net H together with its boundary conditions in the form of a FORTRAN subroutine corresponding to N to be simulated. This FORTRAN procedure is then presented to a solver from the NAG library (Numerical Algorithms Group, 2002) that provides a numerical solution for this system. By using HNGEN, the underlying construction has been successfully tested on several examples. For instance, consider the simple three-neuron cycle DRNN N depicted in Figure 7, initiated in a state where neuron 1 has output 1 and neurons 2 and 3 output 0. Then the computation of the network consists simply of propagating the unit signal around the cycle. Implementing this system on the HNGEN generator results in a system of 61 differential equations describing the dynamics of a continuous-time symmetric HopŽeld net H with 61 units. Figure 8 shows the numerical evolution of the states corresponding to the clock interface unit x1 and the three units ¼1 , ¼2 , ¼3 that represent binary states of the original discrete neurons from N , for

Continuous-Time Symmetric HopŽeld Nets

729

1

3 1 1

1

1 1

2

1

Figure 7: A three-neuron cycle DRNN.

a period of eight (23 ) simulated discrete steps, conŽrming the correctness of the simulation. A parameter value of " D 0:3 was used in this numerical simulation, showing that the theoretical bound on " in theorem 1, which would require about " < 0:024 in the present case, is actually quite conservative. 6 Conclusion and Open Problems We have proved that an arbitrary convergent discrete-time recurrent neural network can be simulated by a symmetric continuous-time HopŽeld net with only a linear increase in the network size. The existence of Lyapunov functions for HopŽeld nets precludes the use of unbounded oscillations in such a simulation; nevertheless, we are able to base the construction on the bounded but exponentially long sequence of pulses generated by the continuous-time clock subnetwork. From the point of view of understanding analog computation in general, this technique is somewhat unsatisfying, since we are still basically discretizing the continuous-time computation. It would be most interesting to develop some theoretical tools (e.g., complexity measures, reductions, universal computation) for “natural” continuous-time computations that exclude the use of discretizing oscillations. First steps along this direction have recently been established (Ben-Hur, Siegelmann, & Fishman, 2002; Gori & Meer, 2001). Another challenge for further research is to prove upper bounds on the power of continuous-time systems. Note that in the case of discrete-time analog-state neural networks, a single Žxed-size network with rationalnumber parameters can be computationally universal, that is, able to simulate a universal Turing machine on arbitrary inputs (Siegelmann & Sontag, 1995). Can this strong universality result be generalized for continuoustime systems? Also, we have established an exponential lower bound on

730

Ï õ ma and P. Orponen J. S´

Figure 8: Continuous-time symmetric simulation of three-neuron cycle DRNN for " D 0:3.

Continuous-Time Symmetric HopŽeld Nets

731

Ï õ ma the convergence time of symmetric continuous-time HopŽeld nets (S´ & Orponen, 2001a). Can a matching upper bound be proved, or the lower bound be increased?

Acknowledgments Ï was partially supported by grants GA AS CR B2030007, GA CR No. 201= J. S. 02=1456. P. O. was partially supported by grant 81120 from the Academy of Finland.

References Asarin, E., & Maler, O. (1994). On some relations between dynamical systems and transition systems. In Proceedings of the 21st ICALP’94 International Colloquium on Automata, Languages, and Programming, LNCS 820 (pp. 59–72). Berlin: Springer-Verlag. Balc´azar, J. L., D´õ az, J., & Gabarr´o, J. (1995). Structural complexity I (2nd ed.). Berlin: Springer-Verlag. Balc´azar, J. L., Gavald`a, R., & Siegelmann, H. T. (1997). Computational power of neural networks: A characterization in terms of Kolmogorov complexity. IEEE Transactions on Information Theory, 43(4), 1175–1183. Ben-Hur, A., Siegelmann, H. T., & Fishman, S. (2002). A theory of complexity for continuous time systems. Journal of Complexity, 18(1), 51–86. Branicky, M. (1994). Analog computation with continuous ODEs. In Proceedings of the PhysComp’94 Workshop on Physics and Computation (pp. 265–274). Los Alamitos, CA: IEEE Computer Society Press. Casey, M. (1996). The dynamics of discrete-time computation, with application to recurrent neural networks and Žnite state machine extraction. Neural Computation, 8(6), 1135–1178. Cichocki, A., & Unbehauen, R. (1993). Neural networks for optimization and signal processing. Chichester: Wiley. Cohen, M. A., & Grossberg, S. (1983). Absolute stability of global pattern formation and parallel memory storage by competitive neural networks. IEEE Transactions on Systems, Man, and Cybernetics, 13(5), 815–826. Golden, R. M. (1996). Mathematical methods for neural network analysis and design. Cambridge, MA: MIT Press. Goles, E., & Mart´õ nez, S. (1989). Exponential transient classes of symmetric neural networks for synchronous and sequential updating. Complex Systems, 3(6), 589–597. Gori, M., & Meer, K. (2001). A step towards a complexity theory for dynamical systems (Tech. Rep. ALCOMFT-TR-01-191). Department of Computer Science, University of Aarhus, Denmark. Haykin, S. (1999). Neural networks: A comprehensive foundation (2nd ed.). Upper Saddle River, NJ: Prentice Hall.

732

Ï õ ma and P. Orponen J. S´

HopŽeld, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences, 79, 2554–2558. HopŽeld, J. J. (1984). Neurons with graded response have collective computational properties like those of two-state neurons. Proceedings of the National Academy of Sciences, 81, 3088–3092. HopŽeld, J. J., & Tank, D. W. (1985). “Neural” computation of decisions in optimization problems. Biological Cybernetics, 52(3), 141–152. Horne, B. G., & Hush, D. R. (1996). Bounds on the complexity of recurrent neural network implementations of Žnite state machines. Neural Networks, 9(2), 243–252. Indyk, P. (1995). Optimal simulation of automata by neural nets. In Proceedings of the 12th STACS’95 Annual Symposium on Theoretical Aspects of Computer Science, LNCS 900 (pp. 337–348). Berlin: Springer-Verlag. Koiran, P., Cosnard, M., & Garzon, M. (1994). Computability with lowdimensional dynamical systems. Theoretical Computer Science, 132, 113–128. Lepley, M., & Miller, G. (1983).Computational power for networks of thresholddevices in an asynchronous environment (Unpublished manuscript). Cambridge, MA: Department of Mathematics, MIT. Maass, W., & Orponen, P. (1998). On the effect of analog noise in discrete-time analog computations. Neural Computation, 10(5), 1071–1095. Maass, W., & Sontag, E. D. (1999). Analog neural nets with gaussian or other common noise distribution cannot recognize arbitrary regular languages. Neural Computation, 11(3), 771–782. Medsker, L. R., & Jain, L. C. (Eds.). (2000). Recurrent neural networks: Design and applications. Boca Raton, FL: CRC Press. Moore, C. (1990). Unpredictability and undecidability in physical systems. Physical Review Letters, 64(20), 2354–2357. Moore, C. (1998). Finite-dimensional analog computers: ows, maps, and recurrent neural networks. In Proceedings of the 1st International Conference on Unconventional Models of Computation (pp. 59–71). Berlin: Springer-Verlag. Numerical Algorithms Group. (2002). Fortran library routine D02PCF. Available on-line: http://www.nag.co.uk/numeric//manual/pdf/D02/d02pcf.pdf. Omohundro, S. (1984). Modelling cellular automata with partial differential equations. Physica, 10D, 128–134. Orponen, P. (1996). The computational power of discrete HopŽeld nets with hidden units. Neural Computation, 8(2), 403–415. Orponen, P. (1997a). A survey of continuous-time computation theory. In D.Z. Du & K.-I. Ko (Eds.), Advances in algorithms, languages, and complexity (pp. 209–224). Dordrecht: Kluwer Academic Publishers. Orponen, P. (1997b). The computational power of continuous time neural networks. In Proceedings of the 24th SOFSEM’97 Seminar on Current Trends in Theory and Practice of Informatics, LNCS 1338 (pp. 86–103). Berlin: SpringerVerlag. Parberry, I. (1994). Circuit complexity and neural networks. Cambridge, MA: MIT Press.

Continuous-Time Symmetric HopŽeld Nets

733

Siegelmann, H. T. (1999). Neural networks and analog computation: Beyond the Tur¨ ing limit. Boston: Birkhauser. Siegelmann, H. T., Roitershtein, A., & Ben-Hur, A. (2000). Noisy neural networks and generalizations. In S. A. Solla, T. K. Leen, & K.-R. Muller ¨ (Eds.), Advances in neural information processing systems (NIPS ’99), 12 (pp. 335–341). Cambridge, MA: MIT Press. Siegelmann, H. T., & Sontag, E. D. (1994). Analog computation via neural networks. Theoretical Computer Science, 131(2), 331–360. Siegelmann, H. T., & Sontag, E. D. (1995). Computational power of neural networks. Journal of Computer System Science, 50(1), 132–150. Ï õ ma, J., & Orponen, P. (2000). A continuous-time HopŽeld net simulation of S´ discrete neural networks. In Proceedings of the 2nd NC’2000 International ICSC Symposium on Neural Computing (pp. 36–42). Wetaskiwin, Canada: ICSC Academic Press. Ï õ ma, J., & Orponen, P. (2001a). Exponential transients in continuous-time symS´ metric HopŽeld nets. In Proceedings of the 11th ICANN’2001 Conference on ArtiŽcial Neural Networks, LNCS 2130 (pp. 806–813). Berlin: Springer-Verlag. Ï õ ma, J., & Orponen, P. (2001b). Computing with continuous-time Liapunov S´ systems. In Proceedings of the 33rd STOC’2001 Annual ACM Symposium on Theory of Computing (pp. 722–731). New York: ACM Press. Ï õ ma, J., Orponen, P., & Antti-Poika, T. (2000). On the computational complexity S´ of binary and analog symmetric HopŽeld nets. Neural Computation, 12(12), 2965–2989. Ï õ ma, J., & Wiedermann, J. (1998). Theory of neuromata. Journal of the ACM, S´ 45(1), 155–178. Stoll, H. M., & Lee, L.-S. (1988). A continuous-time optical neural network. In Proceedings of the IEEE International Conference on Neural Networks (Vol. II, pp. 373–384). New York: IEEE Press. Received May 29, 2002; accepted August 22, 2002.

ARTICLE

Communicated by Rajesh P. N. Rao

Modeling Reverse-Phi Motion-Selective Neurons in Cortex: Double Synaptic-Veto Mechanism Chun-Hui Mo [email protected] Division of Biology, and Division of Chemistry, California Institute of Technology, Pasadena, CA 91125, U.S.A.

Christof Koch [email protected] Division of Biology, Division of Engineering and Applied Science, California Institute of Technology, Pasadena, CA 91125, U.S.A.

Reverse-phi motion is the illusory reversal of perceived direction of movement when the stimulus contrast is reversed in successive frames. Livingstone, Tsao, and Conway (2000) showed that direction-selective cells in striate cortex of the alert macaque monkey showed reversed excitatory and inhibitory regions when two different contrast bars were ashed sequentially during a two-bar interaction analysis. While correlation or motion energy models predict the reverse-phi response, it is unclear how neurons can accomplish this. We carried out detailed biophysical simulations of a direction-selective cell model implementing a synaptic shunting scheme. Our results suggest that a simple synaptic-veto mechanism with strong direction selectivity for normal motion cannot account for the observed reverse-phi motion effect. Given the nature of reverse-phi motion, a direct interaction between the ON and OFF pathway, missing in the original shunting-inhibition model, it is essential to account for the reversal of response. We here propose a double synaptic-veto mechanism in which ON excitatory synapses are gated by both delayed ON inhibition at their null side and delayed OFF inhibition at their preferred side. The converse applies to OFF excitatory synapses. Mapping this scheme onto the dendrites of a direction-selective neuron permits the model to respond best to normal motion in its preferred direction and to reverse-phi motion in its null direction. Two-bar interaction maps showed reversed excitation and inhibition regions when two different contrast bars are presented. 1 Introduction Reverse-phi motion was Žrst demonstrated by Anstis (1970; Anstis & Rogers, 1975). Subjects perceived the reverse direction of motion when the contrast of a moving object reverses in the second frame of a two-frame Neural Computation 15, 735–759 (2003)

c 2003 Massachusetts Institute of Technology °

736

C. Mo and C. Koch

shift experiment. A repetitive four-stroke cycle of reverse-phi motion gives a strong illusion of unidirectional apparent motion (Anstis & Rogers, 1986). The reverse-phi illusion involves the short-range motion process pathway (Chubb & Sperling, 1989). Random dot cinematograms (RDC) studies suggest that Dmax , the maximum distance dots can move from one frame to the other while still preserving the sense of motion, for reverse-phi motion is comparable to Dmax for normal motion, compatible with the notion that the same short-range direction-selective mechanism is most likely responsible for both normal and reverse-phi motion (Sato, 1989). Reverse-phi-like effects have also been reported during electrophysiological experiments from direction-selective complex cells in cat striate cortex (Emerson, Citron, Vaughn, & Klein, 1987), the H1 cell in the y’s lobula plate (Egelhaff & Borst, 1992), and the optical tract of the wallaby (Ibbotson & Clifford, 2001). Recent recordings from direction-selective cells in the alert monkey show that cells in cortical areas V1 and MT reverse facilitation and suppression regions in the two-bar interaction map when two different contrast bars are presented (Livingstone, Tsao, & Conway, 2000; Conway & Livingstone, 2001). This implies that these cells respond to reverse-phi motion in the reversed direction. Space-time plots of reverse-phi motion show energy in the reverse direction (see Figure 1A). Although the bar movement direction is to the right, the left motion energy unit aligns better with the stimuli and extracts more motion energy than the right motion energy unit. Therefore, both the motion energy model (Adelson & Bergen, 1985) and the equivalent Reichardt model (Reichardt, 1961; Santen & Sperling, 1985) can account for the reverse-phi motion. At the core of the Reichardt detector is a correlation step that involves multiplication between inputs. Mathematically, if the sign of one of the inputs being multiplied is reversed, as in the case of reverse-phi motion, the sign of the Žnal product is also reversed. However, there is no experimental evidence that either a single neuron or a neural network can perform a clean, four-quadrant multiplication operation. Even for a single identiŽable cell that performs multiplication, such as the LGMD neuron in the locust’s visual system (Hatsopoulos, Gabbiani, & Laurent, 1995; Gabbiani, Mo, & Laurent, 2001), it is unlikely that the sign of the cell’s output could be reversed for a reversed signed input, given multiple half-wave rectiŽcation mechanisms and narrow operating ranges for most biophysical processes involved. It is therefore of interest to study a direction-selective mechanism that can be implemented by neurons and can account for both normal and reverse-phi motion. The Žrst computational step in visual processing is half-wave rectiŽcation and separation into ON and OFF channels. It is unknown whether direction selectivity is generated between nonlinear interactions of these half-wave rectiŽed signals or between simple cells that carry the reconstructed full wave signal. DL-2-amino-4-phosphonobutyric acid (APB) reversibly blocks the ON response in the mammalian retina (Schiller, 1982), but the detection of motion direction is largely unaffected in rabbit, cat, and monkey in electro-

Modeling Reverse-Phi Motion

737

Figure 1: Space-time plot of normal and reverse-phi motion and connectivity diagram of the model that accounts for direction and reverse-phi selectivity. (A) Space-time plot of a one-dimension white bar moving from left to right in normal motion (left panel) and reverse-phi motion (right panel). A rightmotion energy unit aligns well with the normal motion plot, but triggers a much reduced response for the reverse-phi motion. Instead, this strongly stimulates a left motion energy unit. (Adapted from Fig. 16 in Adelson & Bergen, 1985.) (B) Connectivity diagram of normal and reverse-phi motion direction-selective model. Input to LGN neurons comes from a one-dimensional array of 179 pixels. The intensities from those pixels were summed through difference of gaussian spatial Žlters on to LGN cells. There are one ON and one OFF center geniculate cell at one of six spatially offset locations. Each of the middle four ON center LGN cells provided excitatory input to one branch of the model cell dendrites, while the ON center LGN cell immediately to the right and the OFF center LGN cell immediately to the left provide delayed on-the-path inhibition. The converse connection scheme for the OFF center LGN cells’ excitatory inputs is not shown.

738

C. Mo and C. Koch

physiological experiments (Knapp & Mistler, 1983; Horton & Sherk, 1983; Schiller, Sandel, & Maunsell, 1986). Similarly, existing direction-selective models treat ON and OFF inputs separately (Koch & Poggio, 1985; Suarez, Koch, & Douglas, 1995; Rao & Sejnowski, 2000). However, the nature of reverse-phi motion stimuli suggests an interaction between ON and OFF channels. Different rectiŽcation schemes affect the ON-OFF interaction differently, and therefore carefully constructed reverse-phi stimuli were used in experiments to separate the Žrst-order, second-order, and third-order motion (Lu & Sperling, 1999; Mather & Murdoch, 1999). The requirement for ON-OFF interactions constrains cellular models of direction selectivity. V1 direction-selective cells show slanted excitatory regions in their receptive Želd map (Livingstone, 1998). The asymmetry in the summation of excitatory inputs at dendrites alone was not sufŽcient to account for the directional response based on a modeling study by Anderson, Binzegger, Kahana, Martin, and Segev (1999). Therefore, asymmetrical delayed inhibition is likely to be the mechanism that underlies direction selectivity. Such a mechanism based on shunting inhibition (i.e., an increase in a chlorine-based GABAA conductance that reverses close to the cell’s resting position) was proposed for the cortex by Koch and Poggio (1985). Recent experiments in the retina provide evidence in favor of at least some nonlinear interactions between excitatory and shunting inhibitory inputs that take place within the dendrites of direction-selective ganglion cells (Taylor, He, Levick, & Vaney, 2000; for a dissenting view, see Borg-Graham, 2001). Large conductance changes that reverse around the cell’s resting potential have been observed in V1 during visual stimuli (Anderson, Carandini, & Ferster, 2000; Borg-Graham, Monier, & Fr´egnac, 1998). Here we show how a double synaptic-veto mechanism, derived from the traditional asymmetrical delayed shunting inhibition model, can account for both normal and reverse-phi motion direction selectivity. 2 Methods We followed a two-step compartmental simulation strategy using the program NEURON (Hines & Carnevale, 1997). We Žrst investigated the performance of an idealized neuron (see Figure 4B) before we implemented our synaptic assembly onto a reconstructed V1 cell (see Figure 7A). The idealized cell morphology is shown in Figure 4B. Eight dendrites (width, 0.5 ¹m; length, 100 ¹m) were directly connected to the soma (width, 16 ¹m; length, 16 ¹m). Each dendrite had 20 compartments, for a total of 180 compartments. The dendrites were passive, while the cell body contained a number of voltage-dependent currents that give rise to fast Hodgkin-Huxley-like action potentials. The biophysical parameters were as follows: Ra D 250 IJcm2 , Cm D 0:5¹F/cm2 , Eleak D ¡60 mV, Rm D 10 kIJcm; gNa D 0:024 S/cm2 , gK D 0:020 S/cm2 , ENMDA D 0 mV, gNMDA D 2:5 nS, ¿NMDA on D 0:1 ms, ¿NMDA off D 80 ms, E AMPA D 0 mV, gAMPA D 2:5 nS, ¿AMPA on D 0:1

Modeling Reverse-Phi Motion

739

ms, ¿AMPA off D 2 ms, E GABA D ¡60 mV, gGABA D 6:0 nS, ¿GABA on D 1 ms, ¿GABA off D 80 ms. Synaptic input was modeled using the point process in NEURON (adopted from Archie & Mel, 2000). We adapted a layer 4 stellate cell model from Mainen and Sejnowski (1996; see Figure 7A). The model cell contained sodium channels at the soma and dendrites, as well as fast potassium channels at the soma and the axon. Both calcium- and voltage-dependent slow potassium channels and high-threshold calcium channels were present at the soma and dendrites. All passive and active parameters were the same as those used in their original paper. All synapse parameters were the same as described above except EGABA D ¡70 mV, gGABA D 6:9 nS. The input connection scheme to both models is shown in Figure 1B. The spatial resolution of the stimulus was 1 minute of arc and the temporal resolution 0.1 ms. The lateral geniculate nucleus (LGN) layer was modeled as a transfer function. Stimuli projected onto the LGN layer consisted of two 6 £ 1 arrays of spatial Žlters modeling ON center and OFF center cells that covered 3 degrees of visual space. The spatial kernel was a difference of gaussian (DOG) function (adapted from Worg¨ ¨ otter & Koch, 1991). The gaussian kernel was G(x) = (K/2 ¼ ¾ 2 ) exp.¡x2 =2¾ 2 ), ¾center D 10:6 minute, ¾surround D 31:8 minute, and Kcenter =Ksurround D 17=16. Stimuli were Žrst passed through these spatial Žlters and then through low-pass temporal Žlters, with a delay between the center and the surround component (¿center D 10 ms, ¿surround D 20 ms delaysurround D 3 ms). The resulting values were scaled to give a time-dependent, stimulus-driven LGN instantaneous Žring rate with a maximum value of 200 Hz. A background Žring rate of 5 Hz was added, and all negative values were set to 0 (half-wave rectiŽcation). OFF center cells were modeled as the reverse of their ON center counterpart. Each of the middle four ON center LGN cells provided the excitatory input to one branch of the model cell dendrites, while the ON center LGN cell immediately to the right (preferred side) and the OFF center LGN cell immediately to the left (null side) provided delayed inhibition. Each of the middle four OFF center LGN cells provided excitatory input to one branch of the model cell dendrites, while the OFF center LGN cell immediately to the right and the ON center LGN cell immediately to the left provided delayed inhibition. The delay was 12 ms. There were 8 excitatory synapses and 16 inhibitory synapses in the model. Excitatory synapses were mapped to the dendrite compartment 60 ¹m away from the soma; the same type of inhibitory input was located 50 ¹m and a different type of inhibition 40 ¹m away from the cell body. Given that shunting inhibition was on the direct path between excitation and the soma, it could effectively and speciŽcally veto the excitatory input to that branch while only minimally affecting the excitatory input from neighboring branches (Koch, Poggio, & Torre, 1982). For the layer 4 stellate cell, eight triplets of excitatory-inhibitoryinhibitory synapses were mapped to eight separate terminal branches. This

740

C. Mo and C. Koch

arrangement was replicated four times and mapped onto 32 terminal branches. The total synaptic count was 32 excitatory and 64 inhibitory synapses. 3 Results We started by testing how well the reverse-phi effect was explained by the original asymmetric-delayed inhibition model of Barlow and Levick (1965), as implemented with shunting inhibition (Torre & Poggio, 1978; Poggio & Torre, 1978; Koch et al., 1982). We then carried out compartmental simulation in NEURON using an idealized dendritic geometry to prove our concept and compare the model against experimental data. Finally, we mapped our synaptic connection scheme to a more realistic cortical cell morphology and demonstrated how it could account for both normal and reverse-phi direction selectivity. 3.1 Asymmetric-Delayed Shunting Inhibition Model. The traditional Barlow and Levick (1965) inhibitory-based scheme is shown in Figure 2A. Inhibition was assumed to be of the shunting type (that is, with an inhibitory reversal potential around the local resting potential) so that synaptic interactions are restricted to local branches. We plotted the time course of excitatory and inhibitory inputs to the model from two adjacent cells in the LGN input layer (see Figure 2B). When a white bar moved in a normal fashion in the preferred direction, there was a temporal shift between excitatory and inhibitory inputs (see Figure 2B.a), while in the null direction, excitation and inhibition overlapped and therefore cancelled each other (see Figure 2B.c). If the same bar moved in the reverse-phi fashion, there was little difference in the temporal alignment of excitatory and inhibitory inputs between preferred and null direction movement (see Figures 2B.e–h). For normal motion Figure 2: Facing page. Normal shunting inhibition cannot account for reversephi motion. (A) Asymmetrical delayed inhibition scheme resulted in direction selectivity. ON excitation was gated by a delayed ON inhibition at its preferred side and the converse for OFF excitation (symbols as in Figure 1). (B) Excitatory and inhibitory inputs to model cell when a bright thin bar moves across its receptive Želd at 10 degrees per second in preferred normal (PN) motion direction (a,b); in null normal (NN) motion direction (c,d), preferred reverse-phi (PR) motion direction (e,f), and null reverse-phi (NR) motion direction (g,h). Solid lines: Excitatory inputs from ON center cells (a,c,e,g) and OFF center cells (b,d,f,h). Dashed lines: Delayed inhibitory inputs (20 ms delay) were plotted in negative. Inputs were calculated as the stimuli passed through the spatial-temporal Žlters mentioned in section 2. (C) The direction index DI for the eight-armed dendritic model cell for different inhibitory delay times and contrast reversal rates. Over a wide range of inhibitory input delay times, the model cell was direction selective to normal motion but only weakly to reverse-phi motion.

Modeling Reverse-Phi Motion

A

741

P referred Direction

Input O rder in V is ua l S pace

DS O N-O N

Input S trength

B

a

PN

b

PN

c

NN

d

NN

e

PR

f

PR

g

NR

h

NR

0

C

O F F -O F F

100 200 T ime [ms ]

0

1.0

Norma l Motion 50Hz R evers eP hi

a

Direction Index

100 200 T ime [ms ]

0.0 1.0

Norma l Motion 75Hz R evers eP hi

b

c

0.0 1.0

Norma l Motion 100Hz R evers eP hi

0.0 0

10

20

30

40

50

Delay [ms ]

60

70

80

742

C. Mo and C. Koch

only, the ON branch received signiŽcant input, while for reverse-phi motion, the input was spread between ON and OFF channels. The combined areas under the excitation or inhibition curves for both branches were much less for reverse-phi motion than normal motion. Because of the low-pass Žltering and the rectiŽcation inherent in the LGN layer, both the excitatory and inhibitory inputs to the model cell during reverse-phi motion were reduced. We compared the direction index (DI) of the model cell using different stimulus reversal rates and the delay factor for the inhibition (see Figure 2C). DI was calculated as the response in the preferred direction minus the null direction response, divided by the sum of the preferred and the null direction. The value of preferred direction was calculated as the sum of excitation minus inhibition, with all negative values set to zero. Thus, DI ranges from 0 (for a nondiscriminatory system) to 1 (for a system that does not respond at all to null direction motion). The model was direction selective to normal motion for a wide range of delay values but only weakly direction selective to reverse-phi motion. DI for reverse-phi motion increased when the stimulus reversal rate increased, but the direction preference was the same as that of normal motion, contrary to the experimental data. Therefore, it appears that a traditional inhibition scheme cannot easily account for reverse-phi motion. Given the nature of reverse-phi motion, a direct nonlinear interaction between ON and OFF branches, missing in traditional schemes, is needed to account for reverse-phi motion-direction selectivity. We here propose a “reversed” shunting inhibition scheme (see Figure 3A), in which an ON excitation is gated by a delayed OFF inhibition at its null side and an OFF excitation is gated by a delayed ON inhibition at its null side (see Poggio, 1982; Koch & Poggio, 1987, for other synaptic logic models involving ONOFF interaction). Not surprisingly, this model responds equally strongly to both directions for normal motion, as inhibition is activated only by a bar of the opposite contrast from that of excitation. However, there is a difference in the temporal alignment of excitatory and inhibitory inputs between the Figure 3: Facing page. Reversed shunting inhibition scheme was direction selective to reverse-phi motion. (A) Reversed asymmetrical delayed-inhibition scheme. ON excitation was gated by a delayed OFF inhibition at its preferred side and the converse for OFF excitation. (B) Excitatory and inhibitory inputs to the model cell when a bright thin bar moves across its receptive Želd at 10 degrees per second in the preferred normal (PN) motion direction (a,b); the null normal (NN) motion direction (c,d), the preferred reverse-phi (PR) motion direction (e,f), and the null reverse-phi (NR) motion direction (g,h). Solid lines: Excitatory inputs from ON center cells (a,c,e,g) and OFF center cells (b,d,f,h). Dashed lines (delayed inhibitory inputs—20 ms delay) are plotted in a reverse negative. (C) The DI calculated for reverse-phi motion. The model is not direction selective to normal motion, but when inhibitory input delay is optimal, the model was direction selective to reverse-phi motion.

Modeling Reverse-Phi Motion

A

743

Input O rder in V is ual S pace

P referred Direction

RS ON-O F F

Input S trength

B

PN

b

PN

c

NN

d

NN

e

PR

f

PR

g

NR

h

NR

0

Direction Index

C

OF F -O N

a

100 200 T ime [ms ]

0

100 200 T ime [ms ]

a

Normal Motion 50Hz R evers eP hi

b

Normal Motion 75Hz R evers eP hi

1.0

0.0 1.0

0.0 1.0

Normal Motion 100Hz R evers eP hi

c

0.0 0

10

20

30

40

50

Dela y [ms ]

60

70

80

744

C. Mo and C. Koch

preferred and null direction reverse-phi movement. When the inhibitory delay matches the stimuli reversal rate, DI is close to 1. DI decreases when the stimulus reversal rate increases. We conclude that the ON-OFF and OFFON vetoing scheme can discriminate the direction of reverse-phi motion. Given that the experimental data demonstrated that reverse-phi motion in the cell’s null direction elicited more vigorous responses than reverse-phi motion in the preferred direction (DI < 0), the delayed inhibitory input needs to reside at the excitatory input’s preferred side instead of its null side. 3.2 Double Synaptic-Veto Mechanism. A traditional shunting inhibition scheme can account for normal motion-direction selectivity, while a “reversed” shunting inhibition scheme can account for reverse-phi motiondirection selectivity. In order to account for both, we need to combine both synaptic schemes. One way to achieve this is to construct a model cell with four dendritic branches. Two of them implement a traditional shunting inhibition scheme and the other two a “reversed” shunting inhibition scheme. Such a connection scheme will be selective to both types of motion; however, the “traditional” branches offer a nongated path for reverse-phi excitatory inputs, while the “reversed” branches offer a nongated path for normal motion excitation. Such nongated excitatory inputs result in a high background level of depolarization at the soma and therefore low DI values. We propose a double synaptic-veto mechanism that combines the two synaptic schemes in a more sophisticated way at the microcircuitry level (see Figure 4A). In the new connection scheme, an ON excitatory synapse is gated by both a delayed ON inhibition at its null side (see Figure 1B, right side of the cell) and a delayed OFF inhibition at its preferred side (see Figure 1B, left side of the cell), and an OFF excitatory synapse is gated by both a delayed OFF inhibition at its null side and a delayed ON inhibition at its preferred side. Same-type inhibition at the null side vetoes nulldirection normal motion, while different-type inhibition at the preferred side vetoes preferred-direction reverse-phi motion. We mapped this triplet of synapses—one excitatory and two shunting inhibitory ones—onto the Figure 4: Facing page. A double synaptic-veto mechanism can account for both normal and reverse-phi motion direction selectivity. (A) Connection scheme of the double synaptic veto mechanism. ON excitation is gated by a delayed ON inhibition at its preferred side and a delayed OFF inhibition at its null side; the converse is true for OFF excitation. (B) Mapping the double synaptic-veto scheme onto the eight-armed cable model with spiking at the cell body. Each excitation-inhibition-inhibition triplet is mapped onto its own dendritic branch with the excitation at the far side of cell body. (C) Model cell’s response to a bright bar moving at 10 degrees per second across its receptive Želd. The model cell responds best when a normal motion stimulus moves in its preferred direction and a reverse-phi motion stimulus moves in its null direction. The stimuli reverse rate is 75 Hz.

Modeling Reverse-Phi Motion

A

745

Input O rder in V is ual S pace

B

DS

P referred Direction

C

P referred Normal Motion

P referred R everse P hi

40

40

20

20

0

0

-20

-20

-40

-40

-60

-60 0

100

200

300

Null Normal Motion

40

0

0

0

-20

-20

-40

-40

V m [mv]

20

-60

200

300

Null R evers e P hi

40

20

100

-60 0

100 200 T ime [ms ]

300

0

100 200 T ime [ms ]

300

746

C. Mo and C. Koch

abstract cell model with a soma and eight dendritic branches (see Figure 4B), creating a simulacrum with four direction-selective subunits, each of which implements the synaptic connection of Figure 4A (the detailed connection scheme is explained in section 2). The model responded best to normal motion in its preferred direction (DI = 1) and reverse-phi motion (DI = ¡1) in its null direction (see Figure 4C). Note that the amplitude of the response to normal motion (6 spikes) was twice as large as the amplitude to reverse-phi motion (3 spikes), reecting the fact that both the excitatory and inhibitory inputs for normal motion stimuli were stronger. In a broad range of model parameters, Cm = 0.5 to 1 ¹F/cm2 , Eleak D ¡70 to ¡60 mV, Delayinhibitory input D 8 to 15 ms, gAMPA + gNMDA D 1 to 10 nS, gAMPA =gNMDA D 0:1 to 10, and gGABA =.gNMDA C gNMDA / D 1 to 10, the simulation showed direction selectivity for both types of motion, but only speciŽc parameter sets resulted in high DI. When DI was small, null-direction normal motion or preferred-direction reverse-phi motion also elicited spikes, but the timing of the Žrst spike was late compared to the case of the preferred-direction movement (data not shown). Even if gNMDA was set to zero, the cell responded in a differential way to null- and preferreddirection motion. Since sodium and potassium channels are placed only at the soma, the voltage-dependent dendritic current was not required for the model’s direction selectivity. However, NMDA currents increased somatic voltage during preferred-direction movement and thus increased DI. The inhibitory synapses were always located between the excitatory input and the spike-triggering zone at the cell body, thereby fulŽlling the “on-thepath” condition (Koch et al., 1982). The inhibitory conductance change in most cases needs to be only a little bit larger than the excitatory conductance change to achieve a “veto” effect. To test whether shunting inhibition was required for the direction selectivity we observed, we decreased the GABA channel reversal potential from ¡60 mV to ¡90 mV in 5 mV steps. At the same time, we decreased the amplitude of gGABA accordingly so that the model always responded to a preferred-direction normal-motion stimuli with six spikes and nulldirection reverse-phi stimuli with three spikes. DI for both types of motions decreased when the GABA channel reversal potential was decreased. At ¡90 mV, the model responded with Žve spikes to null-direction normalmotion stimuli and with three spikes to preferred-direction reverse-phi motion stimuli. Thus, the direction selectivity was lost. 3.3 Receptive Field and Two-Bar Interaction Maps. The direction-selective cell shows a slant in the space-time plot (see Figure 5A). A recent study of direction-selective cells in awake monkey V1 shows an excitatory region on the cell’s preferred side and an inhibitory region on the cell’s null side, consistent with our connection scheme (Livingstone, 1998). To compare the experimental data with our model, we incorporated a synaptic noise source (AMPA conductance only) at the soma to achieve a reasonably

Modeling Reverse-Phi Motion

747

Figure 5: Comparison of space-time response mapping of monkey V1 cell and model cell. (A) Poststimulus time histogram (PSTHs) obtained from a layer 5/6 complex cell in primary visual cortex of the alert macaque monkey in response to ashed bars, presented in random order at a series of positions across the cell’s receptive Želd. (From Livingston, 1998, Fig. 3A.) This cell’s preferred direction was from the bottom toward the top. Flash bar duration = 56 ms; interstimulus delay = 100 ms; 75 stimulus presentations. (B) PSTHs obtained from the model neuron to ashed bars at 20 spatial locations across its receptive Želd. The model’s preferred direction is from the bottom toward the top. It shows a decrease in the response onset time and an increase in the response transiency, as does the V1 complex cell.

high background Žring rate and then presented ashing bar stimuli at 20 different spatial locations across the receptive Želd (see Figure 5B). There was good agreement between the experimental data and the response of our eight-arm model. Both space-time plots showed a progressive shortening of the response onset time and a more transient response going from the cell’s preferred side to its null side. In our model, the shortening of response onset time and the increase in response transiency were due to asymmetrical delayed inhibition and the basic property of integrate-and-Žring neurons of LGN layer. The response onset time was determined mainly by the excitatory input since the inhibitory input was relatively small and delayed. Going from the preferred to the null side, the bar moves from the edge to the center of the receptive Želd of the Žrst LGN cell that provided excitatory input, increasing the excitatory input amplitude and decreasing the time needed to charge the membrane to Žre the Žrst spike. The response transiency is determined primarily by how quickly inhibition can overcome excitation and shut off the response. Moving toward the null side, excitatory input strength decreases, while inhibitory input strength increases, as does the response transiency. The slant of the excitatory and inhibitory regions in the space-time plot is related to the cell’s velocity tuning (Livingstone, 1998;

748

C. Mo and C. Koch

McLean & Palmer, 1989; Reid, Soodak, & Shapley, 1991). Since we did not include any synaptic delay between the retina and V1, the model responds much earlier to the visual stimulus than actual striate cortex neurons do. The periodic Žring is caused by the deterministic synaptic input. Random noise channels at the soma are responsible for the background Žring and jitter of spikes around the peak Žring time. Rao and Sejnowski (2000) showed a similar space-time plot for their direction-selective model. In their network model, the decrease of response onset time was due to the increase of the excitatory synaptic input strength from the cell at the preferred side toward the model cell itself. This effect is expected for any direction-selective model based on asymmetrical inhibition. The experimental data also show reversed excitatory and inhibitory regions in the two-bar interaction map of direction-selective cells in V1 and MT when opposite-contrast bars, instead of same-contrast bars were presented (MT: see Figure 6A, from Livingstone, 2001, Fig. 1; V1: Livingstone,

Modeling Reverse-Phi Motion

749

personal communication, April 2002). To compare with the experimental data, we tested our model’s response to two sequentially presented bars (see Figure 6B). The x-axis corresponds to the position of the Žrst ashed bar and the y-axis to the position of the second ashed bar. The diagonal line corresponds to both bars being presented at the same spatial location. A nondirection-selective cell should have excitation regions both above and below the diagonal line, while a direction-selective cell should have asymmetric excitation regions with respect to the diagonal line. When the two bars are of the same contrast, the excitatory region of our model cell’s response map is mostly above the diagonal, where the second presenting Figure 6: Facing page. Two-bar interaction maps. (A) Interaction map for two MT cells in the alert monkey (Fig. 1 in Livingstone et al., 2001). (B) Same interaction map for the eight-armed model neuron. Pairs of bars (8 minutes of width) were ashed sequentially for 13 ms each at different spatial locations. Spikes were counted for 100 ms from the start of the Žrst bar ash. The x-axis is the Žrst ash bar’s position, and the y-axis is the second bar’s position. The diagonal line is where two bars presented at the same spatial location. From left to right, Panel 1: Both bars were white. Areas above the diagonal line, where the second bar position was more positive than the Žrst bar position, were more active than the area below the diagonal line. Panel 2: Both bars were black. Panel 3: The Žrst bar was black and the second bar was white. Areas below the diagonal line were more active than the area above the diagonal line. Panel 4: The Žrst bar was white, and the second was black. Panel 5: The same contrast conditions minus the inverted contrast conditions (1 C 2 ¡ 3 ¡ 4). The model neuron preferred two same-contrast bars ashed sequentially in its preferred direction and two inverted-contrast bars ashed sequentially in its null direction. Given the symmetrical input, the white-to-white and black-to-blackmodel interactions are identical, as are the black-to-white and the white-to-black one. (C) Space-time two-bar interaction map of the model cell following the technique pioneered by Emerson et al. (1987). The reference bar was presented at time 0 at four different locations across the receptive Želd. For each reference bar position, the probing bar was presented at different locations and times relative to the reference bar. Spikes were counted for 100 ms from the start of the Žrst bar ash. Four such maps were added together to give a position-invariant ds-dt map of two bar interactions. From left to right, Panel 1: Both bars were white. Areas along the diagonal line, where the two-bar presenting sequence matched the preferred direction and speed, were more active than areas orthogonal to the diagonal line, where the two-bar presenting sequence matched the null direction. Panel 2: Both bars were black. Panel 3: The reference bar was black, and the probing bar was white. Panel 4: The reference bar was white, and the probing bar was black. Panel 5: Same-contrast conditions minus the inverted-contrast conditions. The model neuron preferred two same-contrast bars ashed sequentially in its preferred direction and two inverted-contrast bars ashed sequentially in its null direction.

750

C. Mo and C. Koch

bar ’s position is located more toward the null side of the Žrst presenting bar. However, when the two bars have opposite signs of contrast, the excitatory region is mostly below the diagonal, whereas the second presenting bar’s position is located more toward the preferred side of the Žrst bar, signaling a reverse in the direction preference of the model. There is no difference between the white-white and black-black plots, given the symmetric ON and OFF inputs the model receives (the same is true for the black-white and black-white plots). The difference of the excitatory and inhibitory regions between the same- and inverted-contrast bar presentations is more clearly evident in Figure 6B: excitation is mostly above and inhibition mostly below the diagonal. Note that excitation was stronger at the model cell’s preferred side (minus side of spatial scale), while inhibition was stronger at the model cell’s null side (plus side of spatial scale). This was again due to the spatial asymmetry of excitation and inhibition. Although the model is direction selective to both presentations, direction selectivity was higher for samecontrast bar presentations than for different-contrast bar presentations. This translates into a weaker response to reverse-phi than to normal motion. The experimental data from MT cells in Figure 6A show the same overall trend but with a much larger receptive Želd and better overall position invariance across the receptive Želd. There were a few major differences between the empirically determined MT cell response maps (see Figure 6A) and our V1 model cell response (see Figure 6B). Figure 6A shows a diagonal organization, while Figure 6B shows a more circular organization. Part of this difference can be explained by the receptive-Želd size difference between MT and V1 cells. This difference can also arise due to differences in the number and density of direction-selective subunits along the preferred direction and the extent of the nonlinear boost of the Žnal output stage. Some V1 direction-selective complex cells do show a circular interaction region, while other V1 complex cells reveal a more diagonal organization (M. Livingstone, personal communication, April 2002). In addition, the diagonal region in the model (see Figure 6B) is above baseline, whereas it appears to be below baseline in the data (see Figure 6A). This elevation is likely due to the reverse-correlation technique used in the experiment and further nonlinear excitation mechanisms that are missing from our model. Despite these differences, the “same minus inverted” maps (the rightmost panel in Figures 6A and 6B), which demonstrate the nonlinear interactions, do show a substantial amount of similarity. Finally, the cross-shaped backdrop in Figure 6B does not appear in the experimental data. Again, this is due to the difference in receptive-Želd size between the recorded MT cell and our V1 model cell. If Figure 6A were evaluated within a larger spatial range (for example, within C= ¡ 10 degrees), the same cross-shaped background would appear (M. Livingstone, personal communication, April 2002). We also generated space-time two-bar interaction maps (see Figure 6C) to compare with the experimental data published by Emerson et al. (1987) on

Modeling Reverse-Phi Motion

751

complex cells in cat striate cortex. The x-axis corresponds to the time of the ashed probe bar relative to the time of presentation of the ashed reference bar. The y-axis corresponds to the position of the ashed probe bar relative to the position of the ashed reference bar (for more details, see Emerson et al., 1987). A direction-selective cell should have obliquely oriented excitatory regions (Figure 2 in Emerson et al., 1987). When the two bars are of the same contrast, the excitatory region of our model cell’s response map is mostly found along the diagonal, whereas the second presenting bar’s position is located more toward the null side of the Žrst presenting bar. However, when the two bars presented have opposite contrast sign, the excitatory region mostly anks the diagonal area, whereas the second presenting bar’s position is located more toward the preferred side of the Žrst bar, signaling a reverse in the direction preference of the model. The difference of excitatory and inhibitory regions between the same and inverted contrast bar presentations is more clearly evident in the “same minus inverted” panel of Figure 6C: excitation is mostly along the diagonal line, while inhibition mostly anks the diagonal area. Note that excitation is stronger at the model cell’s preferred side (minus side of spatial scale), while inhibition is stronger at the model cell’s null side (plus side of spatial scale). This is due to the spatial asymmetry of excitation and inhibition. The nonlinear facilitation observed in the experiment by Emerson and colleagues may derive from a speciŽc excitatory directional interaction that is not addressed in our model or from generic nondirectional facilitations (such as network feedback or active channels on the dendrites that are masked by directional suppressions). 3.4 Layer 4 Stellate Cell Model. Although all of our parameters are physiologically plausible and the model displayed direction selectivity for a broad range of parameters, we wanted to ensure that the effect we observed was not due to the model cell’s cable structure. We therefore mapped our double synaptic-veto arrangement onto an anatomically correct layer 4 stellate cell model (Mainen & Sejnowski, 1996; see Figure 7A). All active and passive parameters of the original model cell remained unchanged. The new model’s response to different types of motions is illustrated in Figure 7B. The stellate cell showed direction selectivity for normal motion and the opposite selectivity for reverse-phi motion, with DI = 1 in both cases. We also tested our connection scheme on the layer 5 pyramidal cell model from Mainen and Sejnowski (1996). The pyramidal cell model showed the same directional preference when we mapped our synapse triples onto basal dendrites alone or basal and apical dendrites together (data not shown). 4 Discussion We demonstrate here that our double synaptic-veto mechanism can account for the reversal of direction selectivity in reverse-phi motion. Our biophys-

752

C. Mo and C. Koch

A

B 40 20 0 -20 -40 -60 -80

0

100

200

300

40 20 0 -20 -40 -60 -80 0

300

40 20 0 -20 -40 -60 -80 0

Null Normal Motion

V m [mv]

40 20 0 -20 -40 -60 -80

P referred Normal Motion

0

100 200 T ime [ms ]

P referre d R evers e P hi

100

200

300

Null R evers e P hi

100 200 T ime [ms]

300

Figure 7: Mapping the double synaptic-veto mechanism onto a layer 4 stellate cell model caused it to respond differentially for normal as well as reverse-phi motion. (A) Input synapse location on a layer 4 stellate cell. (Cell model from Mainen & Sejnowski, 1996.) (B) Stellate cell model’s response to a bright bar moving at 10 degrees per second across its receptive Želd. The model responds best when a normal motion stimulus moves in its preferred direction and a reverse-phi motion stimulus moves in its null direction, as do many V1 cells in the macaque monkey (Livingstone et al., 2000).

Modeling Reverse-Phi Motion

753

ical simulations are obviously a mere proof of concept that such a scheme might be implemented in a plausible manner by cortical cells. Given the large number of degrees of freedom of any detailed biophysical simulations and the few constraints, except for order-of-magnitude estimations on the relevant parameters, little else is possible at this time. However, the fact that different cell models with distinct dendritic morphologies and voltagedependent currents can be driven by the same synaptic arrangement to replicate the experimental data in terms of direction selectivity shows that our double synaptic-veto mechanism is not implausible from a physiological point of view. This scheme makes several predictions that can be evaluated using extra cellular recordings. First, the nondirection-selective zone for reverse-phi motion should reside at the cell’s null side instead of the preferred side. Due to our inhibitory synaptic connection scheme, the nondiscriminating zone of direction-selective cells demonstrated experimentally in retinal and cortical neurons (Livingstone, 1998; He, Jin, & Masland, 1999) should also reverse its location for reverse-phi stimuli. Second, the response amplitude to normal motion in the preferred direction should be larger than to reversephi in the null direction (see Figures 4C and 7B). The inputs from the LGN layer to the model cell are weaker and spread into both ON and OFF channels for reverse-phi stimuli. Third, DI for reverse-phi motion should be more sensitive to parameter tuning than normal motion, especially for inhibitory input delay. Our input time course analysis (see Figures 2 and 3) suggests DI of reverse-phi motion is quite sensitive to the reversal rate: the higher the reversal rate is, the weaker are the input signals feeding into the direction-selective cell. This last property might show up in appropriate psychophysical studies as an increase in the motion detection threshold (increase of percentage of motion coherence or a decrease of Dmax ). In this study, we assume that direction selectivity is generated at the single cell level in a feedforward manner, without the aid of local feedback circuits. There are extensive feedback interactions among V1 cells, and these feedback currents are likely to be important for sharpening directional tuning (Douglas, Koch, Mahowald, Martin, & Suarez, 1995; Maex & Orban, 1996). As we stated before, our models respond in an appropriate directionselective manner to both types of motions over a broad parameter range, although DI was not always high. When DI was low, the response onset time for the preferred-direction motion was less than for the null-direction motion. A network could use this difference in response onset time to increase DI if we assume that neurons with the same direction preference have more excitatory feedback connection among themselves. In fact, this difference alone is enough to generate direction selectivity through network interaction (Maex & Orban, 1996; Suarez et al., 1995). Excitatory feedback might also help to produce balanced response amplitude for normal-motion and reverse-phi motion. In psychophysics experiments, human subjects have the same detection threshold for normal-motion and reverse-phi motion

754

C. Mo and C. Koch

(Sato, 1989). Monkey V1 cell’s response amplitudes to normal motion and reverse-phi motion are also comparable (M. Livingstone, personal communication, April 2002). In our simulation, although DI = 1 for normal motion and DI = ¡1 for reverse-phi motion, our model responds with many more spikes for the normal motion due to the low-pass nature and half-wave rectiŽcation of our vision system. This difference can be decreased by network interactions if the feedforward input triggers the cortical response and sets the directional bias, while the network itself determines the amplitude of the response. Of course, the use of shunting inhibition to compute the normal and reverse-phi direction selectivity does not rule out additional biophysical mechanisms to sharpen up this selectivity (Mel, 1993; Archie & Mel, 2000; Mel, Ruderman, & Archie, 1998) such as facilitation in the preferred direction (Emerson et al., 1987). We use the terms excitation and inhibition rather than facilitation or suppression in this article. Facilitation and suppression usually refer to the nonlinear part of a cell’s response. We did not isolate linear responses from nonlinear responses in our analysis, although the “same minus inverted” map (see Figures 6B and 6C) is a plot of the nonlinear interaction and thus shows facilitation and suppression (Emerson et al., 1987; Livingstone, Pack, & Born, 2001). The nonlinear suppression in the null direction comes from shunting inhibition. There is no signiŽcant nonlinear facilitation in our model other than NMDA synaptic inputs. This might provide positive nonlinear interaction in the real neuron. Because the excitatory input for reverse-phi motion spreads between ON and OFF channels, such facilitation could further increase the difference of response amplitudes. Excitatory feedback, missing in our feedforward model, might underlie the facilitation in the preferred direction and might Žll in the gap of response amplitude difference between normal and reverse-phi motion. In the traditional inhibition-based direction-selective scheme, shunting inhibition is not necessarily required if the neuron does not possess subunit structures. However, shunting inhibition is required for our double synaptic-veto mechanism to restrict the interaction to local branches. As we stated in section 3, direction selectivity for both normal and reverse-phi motion decreases to almost zero when the inhibitory channel reversal potential decreases from ¡60 mV to ¡90 mV. Our double synaptic veto scheme requires branch-speciŽc computations that cannot be achieved by nonshunting inhibition. Archie and Mel (2000) demonstrated disparity tuning and reverse-phi-like effects without shunting inhibition in a similar modeling study. The nonlinear interaction underlying disparity tuning in their model is clustering (nonlinear excitatory interaction). The basic nonlinear direction interaction in our model is between the excitation and shunting inhibition; shunting inhibition is required for the on-path veto and direction selectivity. We only used one type of GABA synapse that has a long off-ramp (80 ms) in our simulation. This long off-ramp is necessary only to shut off the long-lasting NMDA currents. If we set NMDA conductance to zero and

Modeling Reverse-Phi Motion

755

GABA synapse off-ramp to 2 ms, the model is still direction selective. In real neurons, fast and slow inhibitions coexist. The key to our double synaptic gating mechanism is that excitatory inputs are half-wave rectiŽed and carry separate ON or OFF signals, while the inhibitory input carries both ON and OFF signals (in a spatially segregated manner). In our model, inhibition originates (via interneurons) from LGN ON and OFF center cells. The inhibition could, in principle, also be supplied by a cortical simple cell, with spatially offset ON and OFF regions within its receptive Želd. The separation of ON and OFF channels occurs at the very Žrst mammalian visual processing stage in the retina (Rodieck, 1998). Within the mammalian visual system, it is not until V1 that these two channels combine their information. Since the detection of reverse-phi requires interaction between ON and OFF channels and the simple cell might be the Žrst place that this interaction occurs, Livingstone et al. (2001) proposed that direction selectivity in the monkey might arise between the interactions of two simple cells that carry the full-wave signal. In the visual system of the y, the reverse-phi effect observed in higher-order visual cells (Egelhaaf & Borst, 1992) was used to support the argument that there is no ON-OFF channel separation. Although ON-OFF interaction is necessary to account for reverse-phi motion, our results suggest that only inhibition needs to carry the full-wave signal. The excitation to the direction-selective cell can originate from direct geniculate projection. The synaptic triplet arrangement proposed here for our double synapticveto scheme requires rather sophisticated synapse placement during the development process. Two inhibitory ON and OFF inputs need to synapse close by their associated excitatory input or between this input and the cell body in order to be able to veto excitation effectively (Koch et al., 1982). This is in contrast to traditional shunting inhibition schemes that require only the pairing of one inhibitory process for each excitatory one. It is possible that activity-dependent, temporally asymmetric Hebbian learning rules (Markram, Lubke, Frotscher, & Sakmann, 1997) might be used to establish such speciŽc connection schemes as shown for normal direction selectivity by Rao and Sejnowski (2000). It does remain a major challenge to understand how direction-selective neurons with subunit structures could be established in an unsupervised manner on the basis of such learning rules. Reverse-phi motion stimuli do not appear to be a common feature of natural spatiotemporal scenes. It therefore remains unclear why cortical cells should invert their direction selectivity for reverse-phi motion. We believe that the synaptic circuitry responsible for detecting reverse-phi motion has to be established as a by-product of developing normal direction selectivity rather than as a stand-alone training process. If the inputs to directionselective cortical neurons have a signiŽcant background Žring rate, then OFF inhibition at the preferred side of ON excitation is released when a white bar moves in the preferred direction and thus helps to increase direction preference. If the inhibition comes from simple cells, it naturally

756

C. Mo and C. Koch

contains spatially offset ON and OFF signals. The challenge is to align the ON and OFF regions properly with excitatory inputs. We are investigating learning rules that could account for our double synaptic-veto mechanism. The “prediction and sequence learning” mechanism proposed in Montague and Sejnowski (1994) and Montague, Dayan, Person, and Sejnowski (1995) might play an important role in the establishment of the reverse-phi motiondetection circuit. Acknowledgments This project originated in a joint discussion with Tomaso Poggio and Margaret Livingstone. We thank both for their help and criticism. We also thank Kevin Archie, Bartlett Mel, and Terry Sejnowski for providing us with cell models, help, and criticism. This research was supported by grants from the NSF-sponsored Engineering Research Center at Cal Tech, the National Institutes of Health, and the National Institute of Mental Health. References Adelson, E. H., & Bergen, J. R. (1985). Spatiotemporal energy models for the perception of motion. J. Opt. Soc. Am. A, 2, 284–299. Anderson, J. C., Binzegger, T., Kahana, O., Martin, K. A. C., & Segev, I. (1999). Dendritic asymmetry cannot account for directional responses of neurons in visual cortex. Nature Neuroscience, 2, 820–824. Anderson, J. S., Carandini, M., & Ferster, D. (2000). Orientation tuning of input conductance, excitation, and inhibition in cat primary visual cortex. J. Neurophysiology, 84, 909–926. Anstis, S. M. (1970). Phi movement as a subtractive process. Vision Research, 10, 1411–1430. Anstis, S. M., & Rogers, B. J. (1975). Illusory reversal of visual depth and movement during changes of contrast. Vision Research, 15, 957–961. Anstis, S. M., & Rogers, B. J. (1986). Illusory continuous motion from oscillating postive-negative patterns: Implications for motion perception. Perception, 15, 627–640. Archie, K. A., & Mel, B. W. (2000). A model for intradendritic computation of binocular disparity. Nature Neuroscience, 3, 54–63. Barlow, H. B, & Levick, R. W. (1965). The mechanism of directional selectivity in the rabbit’s retina. J. Physiology, 173, 477–504. Borg-Graham, L. J. (2001).The computation of directional selectivity in the retina occurs presynaptic to the ganglion cell. Nat Neurosci., 4, 176–183. Borg-Graham, L. J., Monier, C., & Fr´egnac, Y. (1998). Visual input evokes transient and strong shunting inhibition in visual cortical neurons. Nature, 393, 369–373. Chubb, C., & Sperling, G. (1989). Two motion perception mechanisms revealed through distance-driven reversal of apparent motion. Proc. Natl. Acad. Sci. USA, 86, 2985–2989.

Modeling Reverse-Phi Motion

757

Conway, B. R., & Livingstone, M. S. (2001). Space-time maps and two-bar interactions of direction-selective cells in macaque V-1. Manuscript submitted for publication. Douglas, R. J., Koch, C., Mahowald, M., Martin, K. A. C., & Suarez, H. H. (1995). Recurrent excitation in neocortical circuits. Science, 269, 981–985. Egelhaaf, M., & Borst, A. (1992). Are there separate ON and OFF channels in y motion vision? Visual Neuroscience, 8, 151–164. Emerson, R. C., Citron, M. C., Vaughn, W. J., & Klein, S. A. (1987). Nonlinear directionally selective subunits in complex cells of cat striate cortex. J. Neurophysiology, 58, 33–65. Gabbiani, F., Mo, C., & Laurent, G. (2001). Invariance of angular threshold computation in a wide-Želd looming-sensitive neuron. J. Neuroscience, 21, 314–329. Hatsopoulos, N., Gabbiani, F., & Laurent, G. (1995). Elementary computation of object approach by a wide-Želd visual neuron. Science, 270, 1000– 1003. He, S., Jin, Z. F., & Masland, R. H. (1999). The nondiscriminating zone of directionally selective retinal ganglion cells: Comparison with dendritic structure and implications for mechanism. J. Neuroscience, 19, 8049–8055. Hines, M. L., & Carnevale, N. T. (1997). The NEURON simulation environment. Neural Computation, 9, 1179–1209. Horton, J. C., & Sherk, H. (1983). Receptive Želd properties in the cat’s lateral geniculate nucleus in the absence of ON-center retinal input. J. Neuroscience, 4, 374–380. Ibbotson, M. R., & Clifford, C. W. G. (2001). Interactions between ON and OFF signals in directional motion detectors feeding the NOT of the wallaby. J. Neurophysiol., 86, 997–1005. Knapp, A. G., & Mistler, L. A. (1983). Response properties of cells in rabbit’s lateral geniculate nucleus during reversible blockade of retinal ON-center channel. J. Neurophysiology, 50, 1236–1245. Koch, C., & Poggio, T. (1985). The synaptic veto mechanism: Does it underlie direction and orientation selectivity in the visual cortex? In D. Rose & V. Dobson (Eds.), Models of the visual cortex (pp. 408–419). New York: Wiley. Koch, C., & Poggio, T. (1987). Biophysics of computation: Neurons, synapses, and membranes. In G. M. Edelman, W. E. Gall, & W. M. Cowan (Eds.), Synaptic function (pp. 637–697). New York: Wiley. Koch, C., Poggio, T., & Torre, V. (1982). Retinal ganglion cells: A functional interpretation of dendritic morphology. Philosophical Transactions of the Royal Society B, 298, 227–264. Livingstone, M. S. (1998). Mechanisms of direction selectivity in macaque V1. Neuron, 20, 509–526. Livingstone, M. S., Pack, C. C., & Born, R. T. (2001). 2-D substructure of MT receptive-Želds. Neuron, 30, 781–793. Livingstone, M. S., Tsao, D. Y., & Conway, B. R. (2000).What happens if it changes contrast when it moves? Society for Neuroscience Abstracts, 26, 162.6. Lu, Z. L., & Sperling, G. (1999). Second-order reversed phi. Perception and Psychophysics, 61, 1075–1088.

758

C. Mo and C. Koch

Maex, R., & Orban, G. A. (1996). Model circuit of spiking neurons generating directional selectivity in simple cells. J. Neurophysiol., 75, 1515–1545. Mainen, Z. F., & Sejnowski, T. J. (1996). Inuence of dendritic structure on Žring pattern in model neocortical neurons. Nature, 382, 363–366. Markram, H., Lubke, J., Frotscher, M., & Sakmann, B. (1997). Regulation of synaptic efŽcacy by coincidence of postsynaptic APs and EPSPs. Science, 275, 213–215. Mather, G., & Murdoch, L. (1999). Second-order processing of four-stroke apparent motion. Vision Research, 39, 1795–1802. McLean, J., & Palmer, L. A. (1989). Contribution of linear spatiotemporal receptive Želd structure to velocity selectivity of simple cells in area 17 of cat. Vision Res., 29, 675–679. Mel, B. W. (1993). Synaptic integration in an excitable dendritic tree. J. Neurophysiol., 70, 1086–1101. Mel, B. W., Ruderman, D. L., & Archie, K. A. (1998). Translation-invariant orientation tuning in visual “complex” cells could derive from intradendritic computations. J. Neuroscience, 18, 4325–4334. Montague, P. R, Dayan, P., Person, C., & Sejnowski, T. J. (1995). Bee foraging in uncertain environments using predictive Hebbian learning. Nature, 377, 725–728. Montague, P. R., & Sejnowski, T. J. (1994). The predictive brain: Temporal coincidence and temporal order in synaptic learning mechanisms. Learn Mem, 1, 1–33. Poggio, T. (1982). Visual algorithms (AI Memo No. 683). Cambridge, MA: MIT ArtiŽcial Intelligence Laboratory. Poggio, T., & Torre, V. (1978). A new approach to synaptic interactions. In R. Heim & G. Palm (Eds.), Theoretical approaches to complex systems (pp. 89–115). Berlin: Springer-Verlag. Rao, R. P. N., & Sejnowski, T. J. (2000). Predictive sequence learning in recurrent neocortical circuits. In S. A. Solla, T. K. Lee, & K.-R. Muller (Eds.), Advances in neural information processing systems, 12 (pp. 164–170). Cambridge, MA: MIT Press. Reichardt, W. (1961). Autocorrelation: A principle for the evaluation of sensory information by the nervous system. In W. A. Rosenblith (Ed.), Sensory communication. New York: Wiley. Reid, R. C., Soodak, R. E., & Shapley, R. M. (1991). Directional selectivity and spatio-temporal structure of receptive Želds of simple cells in cat striate cortex. J. Neurophysiol., 66, 505–529. Rodieck, R. W. (1998). The Žrst steps in seeing. Sunderland, MA: Sinauer Associates. Santen, J. P. H., & Sperling, G. (1985). Elaborated Reichardt detectors. J. Opt. Soc. Am. A, 2, 300–320. Sato, T. (1989). Reversed apparent motion with random dot patterns. Vision Res., 29, 1749–1758. Schiller, P. H. (1982). Central connections of the retinal ON and OFF pathways. Nature, 297, 580–583.

Modeling Reverse-Phi Motion

759

Schiller, P. H., Sandel, J. H., & Maunsell, J. H. R. (1986). Functions of the ON and OFF channels of the visual system. Nature, 322, 824–825. Suarez, H., Koch, C., & Douglas, R. (1995). Modeling direction selectivity of simple cells in striate visual cortex within the framework of the canonical microcircuit. J. Neuroscience, 15, 6700–6719. Taylor, W. R., He, S., Levick, W. R., & Vaney, D. I. (2000). Dendritic computation of direction selectivity by retinal ganglion cells. Science, 289, 2347–2350. Torre, V., & Poggio, T. (1978). A synaptic mechanism possibly underlying directional selectivity to motion. Proc. Roy. Soc. Lond. B, 202, 409–416. Worg ¨ otter, ¨ F., & Koch, C. (1991). A detailed model of the primary visual pathway in the cat: Comparison of afferent excitatory and intracortical inhibitory connection schemes for orientation selectivity. J. Neuroscience, 11, 1959–1979. Received April 3, 2002; accepted October 7, 2002.

LETTER

Communicated by Steven Nowlan

Visual Development and the Acquisition of Motion Velocity Sensitivities Robert A. Jacobs [email protected] Department of Brain and Cognitive Sciences, University of Rochester, Rochester, NY 14627, U.S.A.

Melissa Dominguez [email protected] Department of Computer Science, Universityof Rochester,Rochester,NY 14627,U.S.A.

We consider the hypothesis that systems learning aspects of visual perception may beneŽt from the use of suitably designed developmental progressions during training. Four models were trained to estimate motion velocities in sequences of visual images. Three of the models were developmental models in the sense that the nature of their visual input changed during the course of training. These models received a relatively impoverished visual input early in training, and the quality of this input improved as training progressed. One model used a coarseto-multiscale developmental progression (it received coarse-scale motion features early in training and Žner-scale features were added to its input as training progressed), another model used a Žne-to-multiscale progression, and the third model used a random progression. The Žnal model was nondevelopmental in the sense that the nature of its input remained the same throughout the training period. The simulation results show that the coarse-to-multiscale model performed best. Hypotheses are offered to account for this model’s superior performance, and simulation results evaluating these hypotheses are reported. We conclude that suitably designed developmental sequences can be useful to systems learning to estimate motion velocities. The idea that visual development can aid visual learning is a viable hypothesis in need of further study. 1 Introduction With relatively few exceptions, relationships between development and learning have largely been ignored by the neural computation community. This is surprising because development may be nature’s way of biasing biological learning systems so that they achieve better performance, and may represent an effective means for engineers to bias machine learning systems. It is well known in the machine learning literature that learning Neural Computation 15, 761–781 (2003)

c 2003 Massachusetts Institute of Technology °

762

R. Jacobs and M. Dominguez

systems are inherently faced with the bias-variance dilemma (Geman, Bienenstock, & Doursat, 1995). Systems with little or no bias tend to interpolate in unpredictable ways and thus have highly variable generalization performance. Systems with larger bias, in contrast, tend to show better generalization performance when exposed to those training sets that they can adequately learn. We speculate that development may be an effective means of adding suitable bias to a system, thereby enhancing the generalization performance of that system. In previous work, we studied the effects of different types of developmental sequences on the performances of systems trained to estimate the binocular disparities present in pairs of visual images (Dominguez & Jacobs, in press-a, in press-b). The systems consisted of three components. The Žrst component was a pair of right-eye and left-eye images. For example, the images may have depicted a light or dark object against a gray background. The second component was a set of binocular energy Žlters, which are widely used to model the binocular sensitivities of simple and complex cells in primary visual cortex of primates (Ohzawa, DeAngelis, & Freeman, 1990). Based on local patches of the right-eye and left-eye images, each Žlter acted as a disparity feature detector at a coarse, medium, or Žne scale depending on whether the Žlter was tuned to a low, medium, or high spatial frequency, respectively. The third component was an artiŽcial neural network. The outputs of the binocular energy Žlters were the inputs to this network. The network was trained to estimate the disparity of the object, which was deŽned as the amount that the object was shifted between the right-eye and left-eye images. A nondevelopmental system was compared to three developmental systems. The network of the nondevelopmental system received the outputs of all binocular energy Žlters throughout the entire training period. The networks of the developmental systems, in contrast, were trained in three stages. The network of the coarse-to-multiscale system received the outputs of binocular energy Žlters tuned to a low spatial frequency during the Žrst training stage. It received the outputs of Žlters tuned to low and medium spatial frequencies during the second training stage, and it received the outputs of all Žlters during the third training stage. The network of the Žneto-multiscale system was trained in an analogous way, though its Žlters were added in the opposite order: it received the outputs of Žlters tuned to a high frequency during the Žrst training stage, and the outputs of lowerfrequency Žlters were added during subsequent stages. The network of the random developmental model was also trained in stages, though its inputs were chosen at random at each stage and thus were not organized by spatial frequency content. The results show that the coarse-to-multiscale and Žne-to-multiscale systems consistently outperformed the nondevelopmental and random developmental systems. The fact that they outperformed the nondevelopmental system is important because this demonstrates that models that undergo

Visual Development and Motion Velocity Sensitivities

763

a developmental maturation can acquire a more advanced perceptual ability than one that does not. The fact that they outperformed the random developmental system is important because this demonstrates that not all developmental sequences can be expected to provide performance beneŽts. To the contrary, only sequences whose characteristics are matched to the task should lead to superior performance. In conjunction with other results (not described here), these Žndings suggest that the most successful systems at learning to detect binocular disparities are those that are exposed to visual inputs at a single scale early in training and for which the resolution of their inputs progresses in an orderly fashion from one scale to a neighboring scale during the course of training. At a more general level, these results suggest that the idea that visual development aids visual learning is a viable hypothesis in need of further study. This article studies this hypothesis in the context of visual motion velocity estimation. There are enough similarities between the tasks of binocular disparity estimation and motion velocity estimation that it is reasonable to believe that a developmental approach that is useful for the former task may also be advantageous for the latter task. However, the differences between the two tasks mean that this belief needs to be checked. Indeed, our simulations show that the two tasks do not yield the same pattern of results. Although a developmental approach to the velocity estimation task is shown to be beneŽcial, it is not the case that all developmental progressions that lead to performance advantages on the disparity estimation task also lead to advantages on the velocity estimation task. In particular, a coarseto-multiscale developmental system outperformed nondevelopmental and random developmental systems on the velocity estimation task, but a Žneto-multiscale system did not. We hypothesize that the performance advantage of the coarse-to-multiscale system relative to the Žne-to-multiscale system is due to the fact that the coarse-to-multiscale system learned to make greater use of motion energy Žlters tuned to a low spatial frequency. Analyses suggest that coarse-scale motion features are more informative for the velocity estimation task than Žne-scale features. 2 Developmental and Nondevelopmental Systems The structure of the developmental and nondevelopmental systems is illustrated in Figure 1. The input to each system was a sequence of 88 retinal images where each image was a one-dimensional array 40 pixels in length. This sequence depicted an object moving at a constant velocity in front of a stationary background. The retinal array was treated as if it were shaped like a circle in the sense that the left-most and right-most pixels were regarded as neighbors. This wraparound of the left and right edges was done to avoid edge artifacts. Although a one-dimensional retina is a simpliŽcation, its use is justiŽed by the need to keep the simulation times within reason. The sequence of retinal images was Žltered using motion energy Žlters.

764

R. Jacobs and M. Dominguez

time = 1 time = 2

time = 88 Low SF Med SF High SF

Sequence of

Motion energy filters

retinal images

Artificial neural network

Figure 1: The developmental and nondevelopmental systems shared a common structure. The input to the systems was a sequence of retinal images, Žltered by motion energy Žlters. The set of Žlters can be partitioned into subsets tuned to low, medium, and high spatial frequencies. The outputs of the Žlters were the inputs to an artiŽcial neural network that was trained to estimate the object velocity depicted in the image sequence.

Based on neurophysiological results, Adelson and Bergen (1985) proposed motion energy Žlters as a way of modeling the motion sensitivities of simple and complex cells in primary visual cortex. A sequence of one-dimensional images can be represented using a two-dimensional array where one dimension encodes space and the other dimension encodes time. In this case, motion energy Žlters are two-dimensional Žlters that extract motion information in local patches of the spatiotemporal space. The receptive Želd proŽle of a simple cell can be described mathematically as a Gabor function, which is a sinusoid multiplied by a gaussian envelope. A quadrature pair of such functions with even and odd phases tuned to leftward (¡) and rightward (C) directions of motion is given by ( ) 1 x2 t2 D exp ¡ 2 ¡ cos.!x x § wt t/ 2¼¾x ¾t 2¾x 2¾t2 ( ) 1 x2 t2 § exp ¡ 2 ¡ sin.!x x § wt t/; go D 2¼¾x ¾t 2¾x 2¾t2 g§ e

(2.1)

(2.2)

where x and t are the spatial and temporal distances to the center of the gaussian, ¾x2 and ¾t2 are the spatial and temporal variances of the gaussian,

Visual Development and Motion Velocity Sensitivities

765

and !x and !t are the spatial and temporal frequencies of the sinusoids. The ratio !t =!x determines the orientation of a Gabor function in the spatiotemporal space, which in turn determines the velocity sensitivity of the function. The activity of a simple cell is given by the square of the convolution of the cell’s receptive Želd proŽle with the spatiotemporal pattern, denoted f . 2 For example, . f ¤ gC e / is the activity of a simple cell with even phase that is sensitive to rightward motion. The activities of simple cells with even and odd phases are summed in order to form the activity of a complex cell. This sum is known as a motion energy. Thus, leftward and rightward motion energies are given by 2 ¡ 2 E¡ D . f ¤ g ¡ e / C . f ¤ go /

(2.3)

E D .f ¤

(2.4)

C

2 gC e /

C .f ¤

2 gC o / :

Because a complex cell’s activity is based on the combined properties of simple cells with even and odd phases, this activity is phase insensitive, meaning that this value is relatively insensitive to the exact position of a motion within the complex cell’s receptive Želd. In our simulations, we used a subset of the possible receptive-Želd locations in the two-dimensional (40 pixels £ 88 time frames) spatiotemporal space. This subset formed a 20 £ 4 uniform grid located in the middle of the space such that receptive Želds were centered on odd-numbered pixels and odd-numbered time frames. An advantage of this choice of locations was that edge artifacts were avoided because all receptive Želds fell entirely within the spatiotemporal space. Fifteen complex cells corresponding to three spatial frequencies and Žve temporal frequencies were centered at each receptive-Želd location. All cells were tuned to rightward motion because we restricted our data sets to include only objects that were moving to the right. The parameter values of these cells are shown in Table 1. The spatial and temporal frequencies were each separated by an octave. Importantly, temporal frequencies were chosen so that the set of cells at each spatial frequency had the same pattern of velocity tunings. SpeciŽcally, the sets tuned to low, medium, and high spatial frequencies had velocity tunings of 0.25, 0.5, 1.0, 2.0, and 4.0 pixels per time frame. This was achieved by correlating the spatial and temporal frequency tunings of the cells: low-spatial-frequency cells were tuned to a comparatively low range of temporal frequencies, and high-spatial-frequency cells were tuned to a high range of temporal frequencies. 1 A cell’s spatial and 1 Recent neuroscientiŽc studies indicate that the preferred velocity of velocity-tuned neurons in area MT of primates is largely independent of spatial frequency (Perrone & Thiele, 2001). Simoncelli and Heeger (2001) speculated that a velocity-tuned MT neuron pools the outputs of a set of V1 cells whose spatial and temporal frequency tunings are positively correlated.

766

R. Jacobs and M. Dominguez

Table 1: Parameter Values for the Complex Cells at Each Receptive-Field Location. Spatial Frequency (cycles per pixel)

Temporal Frequency (cycles per time frame)

Velocity (pixels per time frame)

0.0625

0.015625 0.03125 0.0625 0.125 0.25 0.03125 0.0625 0.125 0.25 0.5 0.0625 0.125 0.25 0.5 1.0

0.25 0.5 1.0 2.0 4.0 0.25 0.5 1.0 2.0 4.0 0.25 0.5 1.0 2.0 4.0

0.125

0.25

Spatial Frequency (cycles per pixel)

Spatial Standard Deviation (¾x )

0.0625 0.125 0.25

4.0 2.0 1.0

Temporal Frequency (cycles per time frame)

Temporal Standard Deviation (¾t )

0.015625 0.03125 0.0625 0.125 0.25 0.5 1.0

16.0 8.0 4.0 2.0 1.0 0.5 0.25

Note: The temporal frequencies were chosen so that the same velocity tunings existed at each spatial frequency.

temporal standard deviations were set to be inversely proportional to its spatial and temporal frequencies, respectively. The outputs of the complex cells within each spatial frequency band were normalized using a softmax nonlinearity, E.!xi ;!tj /=¿

O xi ; !tj / D Pe E.!

k

eE.!xi ;!tk /=¿

(2.5)

Visual Development and Motion Velocity Sensitivities

767

where !xi and !tj are the spatial and temporal frequencies to which a complex cell was tuned, E.!xi ; !tj / was the initial output of the complex cell, O x ; !t / was the normalized output, ¿ is a scaling parameter (its value E.! i j was set to 0.001), and !tk ; k D 1; : : : ; 5; indexed the Žve temporal frequencies corresponding to spatial frequency !xi . As a result of this normalization, complex cells tended to respond to relative contrast in the image sequence rather than absolute contrast (Heeger, 1992; Nowlan & Sejnowski, 1994). The normalized outputs of the complex cells were the inputs to an artiŽcial neural network. The network had 1200 input units (the complex cells had 80 receptive-Želd locations, and there were 15 cells at each location). The network’s hidden layer contained 18 hidden units, which were organized into three groups of 6 units each. The connectivity of the hidden units was set so that each group had a limited receptive Želd and neighboring groups had overlapping receptive Želds. A group of hidden units received inputs from 32 receptive-Želd locations at the complex cell level, and the receptive Želds of neighboring groups overlapped by 8 receptive-Želd locations. The hidden units used a logistic activation function. The output layer consisted of a single linear unit; this unit’s output was an estimate of the object velocity depicted in the sequence of retinal images. The weights of an artiŽcial neural network were initialized to small, random values and were adjusted during the course of training to minimize a sum of squared error cost function using a conjugate gradient optimization procedure (Press, Teukolsky, Vetterling, & Flannery, 1992). Advantages of this procedure are that it tends to converge quickly and has no free parameters (e.g., no learning rate or momentum parameters). Weight sharing was implemented at the hidden unit level so that corresponding units within each group of hidden units had the same incoming and outgoing weight values and so that a hidden unit had the same set of weight values from each receptive-Želd location at the complex unit level. This provided the network with a degree of translation invariance and dramatically decreased the number of modiŽable weight values in the network. It therefore decreased the number of data items and the amount of time needed to train the network. Models were trained and tested using separate sets of training and test data items. Each set contained 250 randomly generated items. Training was terminated after 100 iterations through the training set. The results reported are based on the data items from the test set. Three developmental systems and one nondevelopmental system were simulated. The coarse-to-multiscale system (model C2M) was trained using a coarse-to-multiscale developmental sequence implemented as follows. The training period was divided into three stages. During the Žrst stage, the neural network portion of the model received only the outputs of complex cells tuned to the low spatial frequency (the outputs of other complex cells were set to zero). During the second stage, the network received the outputs of complex cells tuned to low and medium spatial frequencies; it received

768

R. Jacobs and M. Dominguez

the outputs of all complex cells during the third stage. The training of the Žne-to-multiscale system (model F2M) was identical to that of model C2M except that its training used a Žne-to-multiscale developmental sequence. During the Žrst stage of training, its network received the outputs of complex cells tuned to the high spatial frequency. This network received the outputs of complex cells tuned to high and medium spatial frequencies during the second stage and received the outputs of all complex cells during the third stage. The training of the random developmental system (model RD) also used a developmental sequence, though this sequence was generated randomly and thus was not based on the spatial frequency tunings of the complex cells. The collection of complex cells was randomly partitioned into three equal-sized subsets, with the constraint that each subset included one-third of the cells at each receptive-Želd location. During the Žrst stage of training, the neural network portion of the model received the outputs of the complex cells only in the Žrst subset. It received the outputs of the cells in the Žrst and second subsets during the second stage of training and the outputs of all complex cells during the third stage. In contrast, the training period of the nondevelopmental system (model ND) was not divided into separate stages; its neural network received the outputs of all complex cells throughout the entire training period. 3 Data Sets and Simulation Results The performances of the four models were evaluated on two data sets. In all cases, the images were gray scale with luminance values between 0 and 1, and motion velocities were rightward with magnitudes between 0 and 4 pixels per time frame. Fifteen simulations of each model on each data set were conducted. In the solid object data set, images consisted of a moving light or dark object in front of a stationary gray background. The object’s gray-scale values were randomly chosen to be in the range from 0.0 to 0.1 or from 0.9 and 1.0, whereas the gray-scale value of the background was always 0.5. The size of the object was randomly chosen to be an integer between 6 and 12 pixels, its initial location was a randomly chosen pixel on the retina, and its velocity was randomly chosen to be a real value between 0 and 4 pixels per time frame. Because the velocity was real-valued, the boundaries between an object’s internal gray-scale values, or the boundaries between the ends of an object and the background, could fall at a real-valued location within a retinal pixel. In these (common) cases, the luminance value of a pixel was appropriately linearly interpolated. Given a sequence of images, the task of a model was to estimate the object’s velocity. The top portion of Figure 2 gives an example of 10 frames of an image sequence from the solid object data set. The bar graph in Figure 3 illustrates the results. The horizontal axis of each graph gives the model, and the vertical axis gives the root mean squared

Visual Development and Motion Velocity Sensitivities

769

Solid object data item

Noisy object data item

Figure 2: (Top) Ten frames of an image sequence from the solid object data set. (Bottom) Ten frames of an image sequence from the noisy object data set.

error (RMSE) on the data items from the test set at the end of training (the error bars give the standard error of the mean). The labels for the developmental models C2M, F2M, and RD include a number. Recall that the training of these models was divided into three training stages (or developmental stages). The number in the label gives the length of developmental stages 1 and 2 (the length of developmental stage 3 can be calculated using the fact that the entire training period lasted 100 iterations). For example, the label C2M-5 corresponds to a version of model C2M in which the Žrst stage was 5 iterations, the second stage was 5 iterations, and the third stage was 90 iterations. In regard to model RD, we simulated four versions of this model (RD-5, RD-10, RD-20, and RD-30). Only the version that performed best is included in the graph. Model C2M signiŽcantly outperformed all other models. Its bestperforming version was C2M-20, which had an 11.5% smaller generaliza-

770

R. Jacobs and M. Dominguez

solid object data set 0.55

RMSE

0.50

0.45

F2M-30

F2M-20

F2M-10

F2M-5

C2M-30

C2M-20

C2M-10

C2M-5

RD-20

ND

0.40

Figure 3: The RMSE on the test set data items for model ND, the best-performing version of model RD, and different versions of models C2M and F2M after training on the solid object data set. The error bars give the standard error of the mean.

tion error than model ND (the difference between the mean performances of these models is statistically signiŽcant; t D 2:50; p < 0:02 using a two-tailed t-test with 28 degrees of freedom). In addition, C2M-20 had a 9.6% smaller error than the best version of model F2M (t D 3:57; p < 0:01) and a 7.2% smaller error than the best version of model RD (t D 2:30; p < 0:05). The images in the second data set, referred to as the noisy object data set, were meant to resemble random-dot kinematograms frequently used in behavioral experiments. Images contained a noisy object that was moving to the right and a noisy background that was stationary. The gray-scale values of the object pixels and the background pixels were set to random numbers between 0 and 1. The size of the object was randomly chosen to be an integer between 6 and 12 pixels, its initial location was a randomly chosen pixel on the retina, and its velocity was randomly chosen to be an integer between 0 and 4 pixels per time frame. As before, the task was to map an image sequence to an estimate of an object velocity. The bottom portion of Figure 2 gives an example of 10 frames of an image sequence from the noisy object data set. The results are shown in Figure 4. Model C2M once again outperformed the other models. Relative to model ND, all versions of model C2M showed

Visual Development and Motion Velocity Sensitivities

771

noisy object data set 0.80

RMSE

0.75

0.70

F2M-30

F2M-20

F2M-10

F2M-5

C2M-30

C2M-20

C2M-10

C2M-5

RD-20

ND

0.65

Figure 4: The RMSE on the test set data items for model ND, the best-performing version of model RD, and different versions of models C2M and F2M after training on the noisy object data set. The error bars give the standard error of the mean.

superior performance (ND vs. C2M-5: t D 2:69; p < 0:02; ND vs. C2M-10: t D 2:78; p < 0:01; ND vs. C2M-20: t D 3:03; p < 0:01; ND vs. C2M-30: t D 4:14; p < 0:001). The best-performing version of model C2M was C2M30. On average, it had an 8.9% smaller generalization error than model ND (t D 4:14; p < 0:001), a 6.1% smaller error than the best version of model F2M (t D 2:95; p < 0:01), and a 4.3% smaller error than the best version of model RD (t D 2:10; p < 0:05). Taken as a whole, the simulation results are interesting for a number of reasons. Most important for our purposes, the fact that model C2M outperformed model ND demonstrates that a model that undergoes a developmental maturation can acquire a more advanced perceptual ability than one that does not. The fact that model F2M performed similarly to or worse than model ND, and worse than model C2M, is important because this demonstrates that not all developmental sequences provide performance beneŽts. It is tempting to hypothesize that only sequences whose characteristics are matched to the task should lead to superior performance. However, the Žnding that the best version of model RD outperformed model ND is inconsistent with this hypothesis because it is difŽcult to understand why a random developmental progression would be well matched to a ve-

772

R. Jacobs and M. Dominguez

0.0

0.0 C,M,F

0.5

Fine

0.5

Coarse

1.0

C,M,F

1.0

Fine

1.5

Coarse

1.5

Random

noisy object data set (10 epochs) 2.0

Random

RMSE

solid object data set (10 epochs) 2.0

Figure 5: Performances of several networks after 10 epochs of training. Their RMSE on the test set data items are shown for the (left) solid object data set and (right) noisy object data set.

locity estimation task. Future work will need to examine this unexpected Žnding. To understand these results better, we conducted additional simulations. We compared the performances of several different types of neural networks on the solid object and noisy object data sets. These networks were not developmental systems; each received the same set of inputs throughout the entire training period. Some networks received only the outputs of the lowspatial-frequency motion energy Žlters (labeled “coarse” networks), only the outputs of medium-frequency Žlters (labeled “medium” networks), or only the outputs of high-frequency Žlters (labeled “Žne” networks). Other networks received the outputs of multiple subsets of Žlters; for example, C,M networks received the outputs of low- and medium-frequency Žlters, and C,M,F networks received the outputs of all Žlters (identical to model ND). The inputs to networks labeled “random” were the outputs of a randomly selected one-third of the motion Žlters. Furthermore, we examined the performances of these networks at a relatively early point in their training periods (after 10 epochs) and also at the end of training (after 100 epochs). The results at a relatively early point in training are shown in Figure 5. The results at the end of training on the solid object and noisy object data sets are shown in Figures 6 and 7, respectively. For our purposes, the most important result to emerge from the data in these Žgures is that motion features at different scales are not equally informative for the task of velocity estimation. To the contrary, coarse motion features (i.e., the outputs of low-spatial-frequency motion energy Žl-

Visual Development and Motion Velocity Sensitivities

773

solid object data set (100 epochs) 2.0

RMSE

1.5

1.0

0.5

C,M,F

M,F

C,M

Fine

Medium

Coarse

Random

0.0

Figure 6: The RMSE of several networks after 100 epochs of training on the solid object data set.

ters) are more informative than medium-scale motion features, which in turn are more informative than Žne motion features. Evidence for this includes the facts that the coarse network outperformed the medium network, and the medium network outperformed the Žne network. In addition, the C,M network performed better than the M,F network. These performance rankings were found using both solid object and noisy object data sets and were found at an early point in training and at the end of training. Based on these results, we can speculate as to why model C2M showed the best performance and why model F2M showed comparatively poor performance. Because the only inputs to model C2M at the start of training were the outputs of the coarse motion energy Žlters and because these outputs were the only set that it received at all stages of training, it is likely that this model made extensive use of these signals. The analyses suggest that the coarse motion signals are the most informative for the velocity estimation task. Model C2M’s extensive use of these highly informative signals presumably allowed it to achieve a high level of performance. In contrast, model F2M is likely to have made greater use of the outputs of Žne motion energy Žlters because these outputs were its only inputs at the start of

774

R. Jacobs and M. Dominguez

noisy object data set (100 epochs) 2.0

RMSE

1.5

1.0

0.5

C,M,F

M,F

C,M

Fine

Medium

Coarse

Random

0.0

Figure 7: The RMSE of several networks after 100 epochs of training on the noisy object data set.

training and the only set that it received at all stages of training. The analyses suggest that the Žne motion signals are the least informative for the velocity estimation task. Model F2M’s extensive use of these least informative signals presumably is responsible for this model’s comparatively poor performance. In earlier work, we found that the most successful systems at learning a binocular disparity estimation task were those that (1) received inputs at a single-frequency scale early in training and (2) for which the resolution of their inputs progressed in an orderly fashion from one scale to a neighboring scale during the course of training (Dominguez & Jacobs, in press-a, in pressb). Condition 1 allowed a system to combine and compare input features at an early training stage without the need to compensate for the fact that these features could be at different spatial scales. If condition 2 was satisŽed, when a system received inputs at a new spatial scale, it was close to a scale with which the system was already familiar. To test the importance on the velocity estimation task for the resolution of a system’s inputs to progress in an orderly fashion from one scale to a neighboring scale, we compared the performances of Žve systems. Three of the systems—ND, C2M-20, and F2M-20—were described above; the other

Visual Development and Motion Velocity Sensitivities solid object data set

775

noisy object data set 0.80

0.50

0.75

0.45

0.70

0.40

0.65 F-CF-CMF-20

C-CF-CMF-20

F2M-20

C2M-20

ND

F-CF-CMF-20

C-CF-CMF-20

F2M-20

C2M-20

ND

RMSE

0.55

Figure 8: The RMSE of several systems on the test set data items for the (left) solid object data set and (right) noisy object data set.

two systems are new. The neural network of model C-CF-CMF-20 received coarse motion features during the Žrst developmental stage; coarse and Žne motion features during the second developmental stage; coarse, medium, and Žne features during the third stage. The neural network of model FCF-CMF-20 received Žne motion features in the Žrst stage; coarse and Žne features in the second stage; and coarse, medium, and Žne features during the Žnal developmental stage. Because the inputs to systems C-CF-CMF-20 and F-CF-CMF-20 did not proceed from one scale to a neighboring scale, we predicted that these systems would perform poorly. The results on the solid object and noisy object data sets are shown in Figure 8. On the solid object data set, C2M-20 outperformed C-CF-CMF-20, and F2M-20 outperformed F-CF-CMF-20. On the basis of these data, we conclude that it is important for a system’s inputs to proceed from one scale to a neighboring scale. On the noisy object data set, the results are different. The performances of C2M-20 and C-CF-CMF-20 are similar, and F2M-20 performed worse than F-CF-CMF-20. We do not believe that these results are necessarily inconsistent with the hypothesis that we are considering. Instead, the results may indicate that on this task, it is more important for a system to receive the outputs of the low-spatial-frequency motion Žlters as early in training as possible than it is for a system to receive inputs whose resolution changes in an orderly manner. Overall, we believe that our results imply that it is moderately, but not highly, important for a developmental system learning to estimate motion velocities to receive inputs whose reso-

776

R. Jacobs and M. Dominguez

lution progresses in an orderly fashion from one scale to a neighboring scale during training.2 We have presented analyses suggesting that coarser-scale motion features are more informative than Žner-scale features on the velocity estimation task. Several factors may help explain this Žnding. As Weiss and Adelson (1998) discussed, motion signals tend to be less ambiguous when the stimulus is viewed for a long duration and more ambiguous when the stimulus is viewed for a short duration. Their reasoning applies to the activities of motion energy Žlters with receptive Želds in the spatiotemporal domain. Coarse-scale Žlters tend to have larger receptive Želds than those of Žne-scale Žlters. Consequently, there is less ambiguity in the activities of coarse-scale Žlters relative to the activities of Žne-scale Žlters. In addition, coarse-scale Žlters have large, overlapping receptive Želds and thus form a high-resolution coarse code of the spatiotemporal space (Milner, 1974; Hinton, 1981; Ballard, 1986). This code could provide a network with accurate information as to the location of the moving object at each moment in time. For example, the activities of these Žlters may have coded with high accuracy the fact that the moving object was at location A at time tA and location B at time tB . If so, a network could have easily learned to estimate the object velocity accurately by calculating .B ¡ A/=.tB ¡ tA /. In contrast, Žne-scale Žlters have smaller, less overlapping receptive Želds, which form a lower-resolution coarse code.3 An interesting prediction follows from this line of reasoning about the advantages of Žlters with large, overlapping receptive Želds. In general, Žlters with larger receptive Želds tend to be tuned to slow velocities, whereas

2 Dominguez and Jacobs (in press-a, in press-b) found that this was a highly important factor for a developmental system learning to estimate binocular disparities, whereas we Žnd on a motion velocity estimation task that it is only a moderately important factor. Unfortunately, it is difŽcult to compare our current and prior Žndings in a detailed way, for a number of reasons. First, whereas the disparity estimation task involved two images (lefteye and right-eye images), the motion estimation task involved a sequence of 88 images. Second, differences between binocular energy Žlters and motion energy Žlters make the two sets of simulations difŽcult to compare. For instance, binocular energy Žlters make extensive use of the relative phases of Žlters, whereas motion energy Žlters do not. As a second example, the binocular energy Žlters that we used earlier were one-dimensional (we were interested only in horizontal disparities, not vertical disparities), whereas the motion energy Žlters used here were two-dimensional. 3 Analyses by Hinton (1981) and Ballard (1986) provide an understanding of the relationship between receptive-Želd size and resolution in the case of binary units that each become active when a stimulus falls within its receptive Želd. If D is the diameter of a unit’s receptive Želd, k is the dimensionality of the space to be represented, and N is the desired number of just noticeable differences in each dimension (i.e., the desired resolution), then the required number of units is N k =Dk¡1 . For a Žxed number of units, a high-resolution code (that is, one with a large N) requires units with large receptive Želds (Želds with a large D), whereas a low-resolution code can be achieved with units with small receptive Želds.

Visual Development and Motion Velocity Sensitivities

777

solid object data set 0.65

RMSE

0.55

F2M-20 s f

F2M-10 s f

C2M-20 s f

C2M-10 s f

RD-20 s f

RD-10 s f

0.35

ND s f

0.45

Figure 9: The RMSE of several systems on the test set data items from the solid object data set with slow (s) or fast (f) object velocities.

Žlters with smaller receptive Želds tend to be tuned to fast velocities. Consequently, we predict that all models should be better at estimating slow velocities than fast velocities. However, we expect that this tendency will be comparatively strong for model C2M, because the receptive Želds of lowspatial-frequency Žlters tuned to slow velocities are much bigger than those of low-frequency Žlters tuned to fast velocities. In contrast, we expect that this tendency will be weak for model F2M, because the receptive Želds of high-frequency Žlters tuned to slow velocities are only mildly bigger than those of high-frequency Žlters tuned to fast velocities. To test this prediction, we evaluated the accuracy of seven systems at estimating slow (0.0–1.333 pixels per frame) and fast (2.667–4.0 pixels per frame) object velocities after training on the solid object data set. The results are shown in Figure 9. The horizontal axis gives both the system and the object velocity (s = slow, f = fast); the vertical axis gives the RMSE. All systems are generally good at estimating both slow and fast object velocities in the sense that they all show subpixel accuracy. Most important for our current purposes, models ND, RD, and C2M were more accurate at estimating

778

R. Jacobs and M. Dominguez

slow velocities than at estimating fast velocities. This trend was strongest for model C2M, in agreement with the predicted outcome. Also in agreement with our prediction is the fact that model F2M showed roughly equal accuracy at estimating slow and fast velocities. 4 Conclusion We have compared four models on a visual motion velocity estimation task. Three of the models were developmental in the sense that the nature of their visual input changed during the course of training. Model C2M used a coarse-to-multiscale developmental progression, meaning that it received coarse-scale motion features early in training and Žner-scale features were added to its input as training progressed, model F2M used a Žne-tomultiscale progression, and model RD used a random progression. The Žnal model, model ND, was nondevelopmental in the sense that the nature of its input remained the same throughout the training period. The simulation results show that model C2M performed best, and model F2M often performed worst. The fact that model C2M outperformed model ND is important because this demonstrates that a model that undergoes a developmental maturation can acquire a more advanced perceptual ability than one that does not. The fact that model F2M performed similar to or worse than model ND, and worse than model C2M, is important because this demonstrates that not all developmental sequences provide performance beneŽts. It is tempting to hypothesize that only sequences whose characteristics are matched to the task should lead to superior performance. However, the Žnding that the best version of model RD outperformed model ND is inconsistent with this hypothesis because it is difŽcult to understand why a random developmental progression would be well matched to a velocity estimation task. In general, we conclude that the idea that visual development can aid visual learning is a viable hypothesis in need of further study. Model C2M’s superior performance is interesting because its developmental progression resembles that of human infants. The spatial acuity of newborns is roughly one-Žfteenth to one-thirtieth that of adults with normal eyesight. In other words, newborns are sensitive only to low spatial frequencies; they cannot see Žne details. Acuity improves approximately linearly from these low levels at birth to near adult levels by about eight months of age (Norcia & Tyler, 1985). Our simulation results suggest that this developmental sequence may provide important functional beneŽts for the acquisition of motion velocity estimation. Model C2M’s superior performance is also interesting because this Žnding is broadly consistent with a theory of child development known as the “less is more” hypothesis (Newport, 1990). According to this view, cognitive, perceptual, and motor limitations early in infancy (such as being able to perceive visually only low spatial frequencies) are helpful, perhaps nec-

Visual Development and Motion Velocity Sensitivities

779

essary, stages in development. Limited mental abilities reect simple neural representations, which are useful stepping-stones or building blocks for the subsequent development of more complex representations (Turkewitz & Kenney, 1982). Consistent with the “less is more” hypothesis, our view of perceptual development is that early developmental periods set the stage for later periods in the sense that early periods bias the performance of a biological system during these later periods. Although machine learning researchers rarely use the term development when referring to their learning algorithms, many researchers have pursued a “stage-setting” strategy—for example: ² Systems that perform clustering frequently are trained in two stages. During the Žrst stage, the K-means algorithm is used to locate the cluster centers approximately. These centers are then used to initialize the mean vectors of a mixture of normal distributions, which is trained using an expectation-maximization (EM) algorithm during a second training period (Bishop, 1995). ² Bayesian systems are frequently trained in two stages. In the Žrst stage, point estimates of a system’s parameters are obtained using a maximum likelihood estimation algorithm such as the EM algorithm. These point estimates are then used to initialize a Markov chain Monte Carlo (MCMC) sampler such as a Gibbs sampler. The initialization of MCMC samplers in this manner can lead to dramatic speed-ups in their convergence (Peng, Jacobs, & Tanner, 1996). ² Jordan (1994) proposed that probabilistic decision trees (also known as hierarchical mixtures of experts; Jordan & Jacobs, 1994) be trained in two stages. In the Žrst stage, a deterministic decision tree is trained using, for example, the CART or C4.5 algorithms. The resulting tree is then used to initialize both the shape and the parameters of a probabilistic decision tree, which is trained using the EM algorithm in a second training stage. ² Neural networks are sometimes trained in two stages. During the Žrst stage, a genetic algorithm is used to identify a good network structure and a good set of initial weights. The backpropagation algorithm is then used to modify these initial weights in a second stage of training (Belew, McInerney, & Schraudolph, 1991). These examples highlight the fact that machine learning researchers are exploring multistaged strategies for biasing their learning systems so as to enhance their performances. We believe that a developmental approach based on biological principles as presented here represents a promising, but understudied, method for suitably biasing learning systems.

780

R. Jacobs and M. Dominguez

Acknowledgments We thank the anonymous reviewers for their comments and suggestions, which resulted in important improvements to this article. This work was supported by NIH research grant R01-EY13149. References Adelson, E. H., & Bergen, J. R. (1985). Spatiotemporal energy models for the perception of motion. Journal of the Optical Society of America A, 2, 284–299. Ballard, D. H. (1986). Cortical connections and parallel processing: Structure and function. Behavioral and Brain Sciences, 9, 67–120. Belew, R. K., McInerney, J., & Schraudolph, N. N. (1991). Evolving networks: Using the genetic algorithm with connectionist learning. In Proceedings of the Second ArtiŽcial Life Conference. Reading, MA: Addison-Wesley. Bishop, C. M. (1995). Neural networks for pattern recognition. New York: Oxford University Press. Dominguez, M., & Jacobs, R. A. (in press-a). Does visual development aid visual learning? In P. Quinlan (Ed.), Connectionist models of development. East Sussex, UK: Psychology Press. Dominguez, M., & Jacobs, R. A. (in press-b). Developmental constraints aid the acquisition of binocular disparity sensitivities. Neural Computation, in press. Geman, S., Bienenstock, E., & Doursat, R. (1995). Neural networks and the bias=variance dilemma. Neural Computation, 4, 1–58. Heeger, D. J. (1992). Normalization of cell responses in cat striate cortex. Visual Neuroscience, 9, 181–197. Hinton, G. E. (1981). Shape representation in parallel systems. In A. Drina (Ed.), Proceedings of the Seventh International Joint Conference on ArtiŽcial Intelligence. Menlo Park, CA: Morgan Kaufmann. Jordan, M. I. (1994). A statistical approach to decision tree modeling. In M. Warmuth (Ed.), Proceedings of the Seventh Annual ACM Conference on Learning Theory. New York: ACM Press. Jordan, M. I., & Jacobs, R. A. (1994). Hierarchical mixtures of experts and the EM algorithm. Neural Computation, 6, 181–214. Milner, P. M. (1974). A model for visual shape recognition. Psychological Review, 81, 521–535. Newport, E. L. (1990). Maturational constraints on language learning. Cognitive Science, 14, 11–28. Norcia, A., & Tyler, C. (1985). Spatial frequency sweep VEP: Visual acuity during the Žrst year of life. Vision Research, 25, 1399–1408. Nowlan, S. J., & Sejnowski, T. J. (1994). Filter selection model for motion segmentation and velocity integration. Journal of the Optical Society of America A, 11, 3177–3200. Ohzawa, I., DeAngelis, G. C., & Freeman, R. D. (1990). Stereoscopic depth discrimination in the visual cortex: Neurons ideally suited as disparity detectors. Science, 249, 1037–1041.

Visual Development and Motion Velocity Sensitivities

781

Peng, F., Jacobs, R. A., & Tanner, M. A. (1996). Bayesian inference in mixtures-ofexperts and hierarchical mixtures-of-experts models with an application to speech recognition. Journal of the American Statistical Association, 91, 953–960. Perrone, J. A., & Thiele, A. (2001). Speed skills: Measuring the visual speed analyzing properties of primate MT neurons. Nature Neuroscience, 4, 526–532. Press, W. H., Teukolsky, S. A., Vetterling, W. T., & Flannery, B. P. (1992). Numerical recipes in C: The art of scientiŽc computing. Cambridge: Cambridge University Press. Simoncelli, E. P., & Heeger, D. J. (2001). Representing retinal image speed in visual cortex. Nature Neuroscience, 4, 461–462. Turkewitz, G., & Kenney, P. A. (1982). Limitations on input as a basis for neural organization and perceptual development: A preliminary statement. Developmental Psychobiology, 15, 357–368. Weiss, Y., & Adelson, E. H. (1998). Slow and smooth: A Bayesian theory for the combination of local motion signals in human vision (Center for Biological and Computational Learning Paper No. 158). Cambridge, MA: Massachusetts Institute of Technology. Received May 13, 2002; accepted October 16, 2002.

LETTER

Communicated by Zoubin Ghahramani

Modeling Cross-Modal Enhancement and Modality-SpeciŽc Suppression in Multisensory Neurons Paul E. Patton [email protected] Beckman Institute, University of Illinois at Urbana=Champaign, Urbana, IL 61801, U.S.A.

Thomas J. Anastasio [email protected] Beckman Institute and Department of Molecular and Integrative Physiology, University of Illinois at Urbana=Champaign, Urbana, IL 61801, U.S.A.

Cross-modal enhancement (CME) occurs when the neural response to a stimulus of one modality is augmented by another stimulus of a different modality. Paired stimuli of the same modality never produce supraadditive enhancement but may produce modality-speciŽc suppression (MSS), in which the response to a stimulus of one modality is diminished by another stimulus of the same modality. Both CME and MSS have been described for neurons in the deep layers of the superior colliculus (DSC), but their neural mechanisms remain unknown. Previous investigators have suggested that CME involves a multiplicative ampliŽer, perhaps mediated by N-methyl D-aspartate (NMDA) receptors, which is engaged by cross-modal but not modality-speciŽc input. We previously postulated that DSC neurons use multisensory input to compute the posterior probability of a target using Bayes’ rule. The Bayes’ rule model reproduces the major features of CME. Here we use simple neural implementations of our model to simulate both CME and MSS and to argue that multiplicative processes are not needed for CME, but may be needed to represent input variance and covariance. Producing CME requires only weighted summation of inputs and the threshold and saturation properties of simple models of biological neurons. Multiplicative nodes allow accurate computation of posterior target probabilities when the spontaneous and driven inputs have unequal variances and covariances. Neural implementations of the Bayes’ rule model account better than the multiplicative ampliŽer hypothesis for the effects of pharmacological blockade of NMDA receptors on the multisensory responses of DSC neurons. The neural implementations also account for MSS, given only the added hypothesis that input channels of the same modality have more spontaneous covariance than those of different modalities.

Neural Computation 15, 783–810 (2003)

c 2003 Massachusetts Institute of Technology °

784

P. Patton and T. Anastasio

1 Introduction Multisensory integration occurs at numerous sites in the mammalian nervous system (Stein & Meredith, 1993) and has been studied extensively in the deep layers of the superior colliculus (DSC) of cats and monkeys (Meredith & Stein, 1986b; Wallace, Wilkinson, & Stein, 1996; Wallace, Meredith, & Stein, 1998). Enhancement is a form of multisensory integration in which the response to a stimulus of one modality is augmented by a stimulus of another modality (Meredith & Stein, 1986b; Stein & Meredith, 1993). Cross-modal enhancement (CME) was thought to be mediated by multiplicative interactions among inputs of different modalities, perhaps through N-methyl D-aspartate (NMDA) receptors present in DSC neurons (Meredith & Stein, 1986b; Stein, Huneycutt, & Meredith, 1988; Meredith, Wallace, & Stein, 1992; Stein & Meredith, 1993; Binns & Salt, 1996; Binns, 1999). CME contrasts sharply with the responses of multisensory neurons to paired stimuli of the same modality presented within their excitatory receptive Želds. Such presentations never produce robust enhancement of the sort seen in CME and sometimes result in modality-speciŽc suppression (MSS), in which the response to a stimulus of one modality is reduced by a second stimulus of the same modality (Stein & Meredith, 1993). The neural mechanisms of CME and MSS remain unknown. Open questions concern whether CME requires multiplicative interactions, the possible role of NMDA receptors, and the paradox of MSS. In previous work we modeled CME on an abstract level using Bayes’ rule (Anastasio, Patton, & Belkacem-Boussaid, 2000). Here we use neural implementations of the Bayes’ rule model to explore mechanisms of multisensory integration in DSC and other neurons. The DSC receives visual, auditory, and somatosensory inputs organized as a multimodal topographic map (Middlebrooks & Knudsen, 1984; Meredith & Stein, 1990; Clemo & Stein, 1991). Activation of cells at a particular location in this map can result in an orienting movement toward the corresponding location in space containing the stimulus source (Wurtz & Goldberg, 1972; Peck, 1990). Individual DSC neurons may receive input from two, or occasionally three, different sensory modalities (Meredith et al., 1992; Wallace et al., 1996). Receptive Želds for multiple modalities are large and spatially coincident. CME in multisensory DSC neurons occurs when stimuli of different modalities are presented together within their excitatory receptive Želds (Meredith & Stein, 1996; Kadunce, Vaughan, Wallace, & Stein, 2001). Enhancement can be expressed as a percentage of the largest single stimulus response SRmax , %E D

CR ¡ SRmax £ 100%; SRmax

(1.1)

where CR is the neural response to the combined stimuli. CME exhibits the property of inverse effectiveness, wherein weak unimodal stimuli, when

A Neural Model of Cross-Modal Enhancement

785

combined, produce a larger %E than do stronger stimuli (Meredith & Stein, 1986b; Wallace & Stein, 1994; Wallace et al., 1996). Multimodal neurons exhibiting CME have also been described in cat extraprimary sensory cortex (Wallace, Meredith, & Stein, 1992; Jiang, Lepore, Ptito, & Guillemot, 1994; Stein & Wallace, 1996), and cross-modal enhancement phenomena have been noted in the behavioral responses of cats (Stein et al., 1988; Stein, Meredith, Huneycutt, & McDade, 1989). MSS observed for DSC neurons can be expressed as negative percentage enhancement. Modality-speciŽc suppressive interactions have also been reported for neurons in a number of primate cortical visual structures (Sato, 1989, 1995; Reynolds, Chelazzi, & Desimone, 1999). We have proposed previously that DSC neurons use stochastic sensory input to compute the posterior probability of a target using Bayes’ rule (Anastasio et al., 2000). We showed that CME and inverse effectiveness arise as straightforward consequences of this hypothesis. The initial study did not address the issue of how neurons might implement the needed computation. That posterior probabilities are a natural outcome of neural computation is well known to specialists in neural networks and pattern classiŽcation (Richard & Lippmann, 1991; Duda, Hart, & Stork, 2001). We apply these results in a neurobiological context and extend them to account for experimental observations. To illustrate how the Bayes’ rule model of CME (Anastasio et al., 2000) might be implemented by neurons, we consider three simple perceptron models. The simplest involves a single perceptron with sensory input channels that are Poisson distributed and conditionally independent. This model demonstrates that enhancement can be produced using only properties generically present in biological neurons and that no special multiplicative interactions are needed. The second, an augmented perceptron model, computes posterior probabilities under the more general circumstances in which inputs are multivariate gaussian. We show that it accounts better than the multiplicative interaction hypothesis for experimental Žndings on the effects of NMDA receptor blockade on the responses of DSC neurons. The third implementation extends the Bayes’ rule model to provide an explanation for the otherwise puzzling phenomenon of MSS. The augmented perceptron produces suppression given neurobiologically plausible assumptions that include covariance among inputs of like modality in the absence of sensory stimulation. 2 A Perceptron Model with Poisson Inputs Can Simulate CME We Žrst consider a model in which the sensory input likelihoods are Poisson distributed and conditionally independent given the target. Under these conditions, a single perceptron is capable of computing the posterior probability of a target and simulating CME. The target is represented as binary random variable T (target absent, T D 0; target present, T D 1). The prior

786

P. Patton and T. Anastasio

probability distribution of the target is assigned arbitrarily as P.T D 1/ D 0:1 and P.T D 0/ D 0:9. According to the Bayes’ rule model (Anastasio et al., 2000), the posterior probability of a target is computed by using sensory input to modify this prior probability. Input from each modality i is modeled as random variable Mi .i D 1; 2; : : : ; k/, which can assume any discrete nonnegative value mi . The mi represent numbers of neural impulses per unit time (Žring rate). The input likelihoods, which are conditional distributions of the inputs given the target, are modeled as Poisson densities: P.Mi D mi j T D t/ D

i ¡¸ it ¸m it e : mi !

(2.1)

The Poisson density was used in the original formulation of the Bayes’ rule model (Anastasio et al., 2000) because it requires the fewest assumptions and reasonably approximates neuronal Žring-rate distributions (Gabbiani & Koch, 1998). The mean and variance of a Poisson distribution are equal and are speciŽed by parameter ¸. The input likelihoods under spontaneous (target absent, P.Mi D mi j T D 0/) or driven (target present, P.Mi D mi j T D 1/) conditions are described using equation 2.1 with means ¸i0 and ¸i1 , respectively (¸i1 > ¸i0 ) (see Figure 1B). The posterior target probability is Figure 1: Facing page. (A) A perceptron model of multisensory integration. The model receives two conditionally independent Poisson-distributed sensory inputs, M1 D V and M2 D A, having synaptic weights wv and wa . The bias input b is Žxed and might represent tonic inhibitory input or biophysical properties intrinsic to a neuron. The sigma node 6 computes the weighted sum u of the inputs (see equation 2.5). The output is passed through the logistic squashing function f .u/ (see equation 2.6). (B) The likelihoods of observing discrete numbers of impulses per unit time (0.25 s) are shown for sensory input Mi under spontaneous (P.Mi D mi j T D 0/, spont, dot-dashed line) and stimulus driven (P.Mi D mi j T D 1/, driven, solid line) conditions. The likelihoods are given by Poisson distributions (see equation 2.1) with ¸i0 D 2 and ¸i1 D 6. (C) The bimodal perceptron model simulates CME. The response is equal to the posterior probability of a target as computed by the perceptron ( f .u/ D P.T D 1 j m/) using equations 2.5, 2.6, and 2.8. For both modalities V and A, the input likelihoods are assigned spontaneous mean ¸v0 D ¸a0 D 2 and driven mean ¸v1 D ¸a1 D 6. In the cross-modal both-driven case (stars), both modalities are driven by a target stimulus. Inputs V D v and A D a are set equal and varied over a range of impulses per unit time. In the modality-speciŽc spontaneous-driven case (circles), one modality is driven, while the other is Žxed at the spontaneous mean. The difference between the two plots represents CME. The dashed line indicates prior target probability P.T D 1/. The both-driven and spontaneous-driven plots represent slices through the full 2D posterior. (D) Cross-modal enhancement for the perceptron model is computed using equation 1.1, where CR and SRmax values are taken from the both-driven and spontaneous-driven curves of C.

A Neural Model of Cross-Modal Enhancement

787

788

P. Patton and T. Anastasio

given using Bayes’ rule: P.m j T D 1/P.T D 1/ : P.m/

P.T D 1 j m/ D

(2.2)

In this equation, m is a vector representing the activity mi of each sensory modality Mi . We assume initially that the likelihoods for the various modalities are conditionally independent given the target. According to this simplifying assumption, the visibility of a target indicates nothing about its audibility, and vice versa. The joint likelihoods P.m j T D 1/ and P.m j T D 0/ can then be computed as P.m j T D t/ D

k Y iD1

P.Mi D mi j T D t/:

(2.3)

The unconditional joint probability P.m/ is given by the principle of total probability: P.m/ D P.T D 1/P.m j T D 1/ C P.T D 0/P.m j T D 0/:

(2.4)

We will use the perceptron as the basis for our model of a DSC neuron. The perceptron computes the weighted sum u of its inputs where each input mi has an associated synaptic weight wi : uDbC

k X

wi mi :

(2.5)

iD1

The constant bias input b might represent the biophysical properties of a neuron or a tonic input. The weighted input sum is passed through the logistic squashing function, f .u/ D

1 ; 1 C e¡u

(2.6)

which approximates the threshold and saturation properties of real neurons. We seek a formula for u such that the value of the logistic function f .u/ is equal to the posterior probability of the target given inputs m. f .u/ D P.T D 1 j m//. We substitute equation 2.6 for f .u/ and substitute Poisson distributions 2.1 for the input likelihoods in equations 2.2, 2.3, and 2.4. In solving for u, we obtain: µ u D ln

¶ X ´´ ¶ k µ³ ³ P.T D 1/ ¸i1 C ln mi C .¸i0 ¡ ¸i1 / : P.T D 0/ ¸i0 iD1

(2.7)

A Neural Model of Cross-Modal Enhancement

789

Equation 2.7 is linear with respect to mi . Thus, given the basic neuronal nonlinearities of threshold and saturation as represented by the squashing function, the posterior probability is computed by the weighted sum of sensory inputs. Multiplying inputs together is not necessary. This Žnding contrasts with previous ideas about the neural basis of CME (see section 5). Equation 2.7 can be rearranged to yield the input weights wi and bias b for a simple perceptron (see Figure 1A): ³ wi D ln

¸i1 ¸i0

´

µ and b D ln

¶ X k P.T D 1/ C .¸i0 ¡ ¸i1 /: P.T D 0/ iD1

(2.8)

The bias contains a term related to the target prior. Because ¸i1 > ¸i0 , the bias term will be negative whenever P.T D 1/ < P.T D 0/. This Žnding has interesting implications for how the target prior might be represented neurobiologically (see section 5). We propose that the response of a DSC neuron is proportional to posterior target probability. A perceptron with weights set according to equation 2.8 was used to simulate the results of a bimodal CME experiment. We consider visual V and auditory A inputs, but the simulation is valid for any combination of two sensory modalities. To simulate the response of a bimodal DSC neuron to cross-modal stimulation, modalities M 1 D V and M2 D A were set equal and varied from 0 to 25 impulses per unit time (both-driven). To simulate a bimodal DSC neuron’s response to modalityspeciŽc stimulation, one input was varied from 0 to 25 impulses per unit time, while the other was held Žxed at its spontaneous mean (spontaneousdriven). The results are illustrated in Figures 1C and 1D. A single perceptron is thus sufŽcient to implement the Bayes’ rule model of CME for inputs that are Poisson distributed and conditionally independent given the target. 3 An Augmented Perceptron Model and the Role of NMDA Receptors As a more general form of our model, we consider a case in which sensory inputs are represented as gaussian distributions, and we drop the assumption that inputs from different modalities are conditionally independent given the target. Input to a DSC neuron from a single sensory modality is derived by combining signals from many different sensory neurons. According to the classical central limit theorem, the distribution of the combined input will be gaussian, provided that the individual signals are independent (Fristedt & Gray, 1997). We assume that input from a single modality will be approximately gaussian, despite possible dependencies between signals carried by individual neurons of the same modality (see below). We model the multimodal input and dependencies between inputs of different

790

P. Patton and T. Anastasio

modalities using a multivariate gaussian distribution (Duda et al., 2001): µ ¶ 1 1 T ¡1 exp ¡ (3.1) P.m j T D t/ D .m¡ ¹ / § .m¡ ¹ / : t t t 2 .2¼/k=2 j§t j1=2 In this equation, k is the number of modalities, and ¹t is a vector containing the means of the distributions associated with each modality when T D t. The symbol T denotes the transpose operation. §t is a k by k covariance matrix whose diagonal terms are the variances associated with individual modalities and whose off-diagonal terms are the covariances among modalities. The model classiŽes input patterns into two categories: T D 0 and T D 1. For this simple case, u can be expressed as a discriminant function (Duda et al., 2001): ³ ´ ³ ´ ³ ´ P.T D 1 j m/ P.m j T D 1/ P.T D 1/ D ln Cln u D g.m/ D ln P.T D 0 j m/ P.m j T D 0/ P.T D 0/ D ln P.m j T D 1/Cln P.T D 1/¡[ln P.m j T D 0/Cln P.T D 0/]:

(3.2)

Substituting equation 3.1 and simplifying, we obtain: uD

1 T ¡1 ¡1 ¡1 T [m .§1 ¡ §¡1 0 /m C 2m .§1 ¹ 1 ¡ 60 ¹0 / 2 T ¡1 ¡ ¹T1 §¡1 1 ¹1 C ¹0 § 0 ¹0 ] ³ ´ j§0 j1=2 P.T D 1/ C ln : j§1 j1=2 P.T D 0/

(3.3)

As before, the subscript 1 refers to an input driven by the target stimulus (T D 1), and 0 to one active spontaneously in the absence of a stimulus (T D 0). Because of the quadratic and multiplicative terms involved, this formula cannot, in general, be implemented by a single perceptron. A multilayered perceptron is capable of approximating any continuous function (Cybenko, 1989). Such a network, given m as input and appropriate synaptic weights, should be capable of supplying an approximation of u in equation 3.3 to a logistic output node representing a DSC neuron. Another plausible implementation involves the introduction of pi nodes, or product units (Rumelhart, Hinton, & McClelland, 1986; Durbin & Rumelhart, 1989), to create an augmented-perceptron model (see Figure 2A). Pi nodes multiply their inputs together just as sigma nodes add them. Neurobiologically, they represent possible voltage-sensitive processes in the dendritic arbor of a neuron (Durbin & Rumelhart, 1989; Koch & Poggio, 1992). The augmentedperceptron model can be expressed as uDbC

k X iD1

wi mi C

k X k X iD1 jDi

½ij .mi mj /:

(3.4)

A Neural Model of Cross-Modal Enhancement

791

The Žrst two terms on the right-hand side are identical with those of the simple perceptron, equation 2.5. The last term speciŽes the sigma-pi function with weights ½ij . For the case of two sensory inputs V D v and A D a, equation 3.4 can be rewritten as u D b C wv v C wa a C ½v v2 C ½a a2 C ½va va:

(3.5)

Using equation 3.3, the constants ½v , ½a , ½va , wv , wa , and b can be speciŽed as Á ! Á ! Á ! 2 2 2 2 2 2 ¾a1 ¾v1 ¾va0 1 ¾a0 1 ¾v0 ¾va1 ¡ ¡ ¡ ½v D ; ½a D ; ½va D ; (3.6) 2 ¯0 2 ¯0 ¯1 ¯1 ¯1 ¯0 Á ! Á ! 2 2 2 ¡ 2 ¡ ¹a1 ¾va1 ¹a0 ¾va0 ¹v0 ¾ a1 ¹v1 ¾a1 C (3.7) wv D ; ¯1 ¯0 Á ! Á ! 2 2 2 2 ¡ ¹v1 ¾va1 ¡ ¹a0 ¾v1 ¹a1 ¾v1 ¹v0 ¾va0 C (3.8) wa D ; ¯1 ¯0 bD

2 2 C 2¾ 2 2 2 [¡¾a1 ¹v1 va1 ¹v1 ¹a1 ¡ ¾v1 ¹a1 ] 2¯1 2 2 ¡ 2¾ 2 2 2 [¾a0 ¹v0 va0 ¹v0 ¹a0 C ¾v0 ¹a0 ] 2¯0 "Ás ! # ¯0 P.T D 1/ C ln : ¯1 P.T D 0/

C

(3.9)

2 2 ¡ 4 and 2 2 ¡ 4 . The terms In these equations, ¯0 D ¾v0 ¾a0 ¾vao ¯1 D ¾ v1 ¾a1 ¾va1 2 2 2 , and 2 are variances for the spontaneous and driven cases for ¾v0 ; ¾a0 ; ¾v1 ¾a1 2 2 each modality, and ¾va0 and ¾va1 are the covariances for the spontaneous and driven conditions, respectively. An augmented perceptron as speciŽed by equations 3.5 through 3.9 can compute posterior target probability and thereby simulate CME (see Figures 2B and 2C). Note that as for the Poisson-input model, CME arises by summation in a nonlinear neuron. When the spontaneous and driven 2 D variances and the spontaneous and driven covariances are equal (¾ v0 2 2 D 2 , and 2 D 2 ¾v1 ; ¾a0 ¾a1 ¾va0 ¾va1 /, the pi-node terms disappear (½v D ½a D ½va D 0) and the computation can again be performed by a simple perceptron. The multiplicative terms associated with the pi nodes compensate for differences between spontaneous and driven variance and covariance rather than produce CME. Targets vary greatly in the efŽcacy with which they drive different sensory input neurons. It seems likely that the variance of a sensory input of any modality should be larger in the driven than in the spontaneous case.

792

P. Patton and T. Anastasio

A Neural Model of Cross-Modal Enhancement

793

2 2 In the two-channel model, we set the spontaneous variances ¾v0 D ¾a0 D5 2 2 and the driven variances ¾v1 D ¾a1 D 6. It also seems likely that two sensory inputs of different modalities should covary more under driven than under spontaneous conditions. Inputs of different modalities are segregated in modality-speciŽc pathways prior to their arrival in the DSC (Sparks & Hartwich-Young, 1989; Wallace, Meredith, & Stein, 1993). Such input channels are unlikely to interact under spontaneous conditions, and sponta-

Figure 2: Facing page. (A) An augmented-perceptron model of multisensory integration. Sensory inputs are gaussian distributed. The model receives input from two sensory modalities: M1 D V and M2 D A. Each provides weighted input to summing node 6 with weights wv and wa . The summing node also receives a Žxed bias input b. The set of three pi nodes 5 multiplies both modalities by themselves and by each other. The pi nodes provide inputs to the summing node with weights ½v ; ½a ; and ½va . The output is passed through the squashing function f .u/. (B) The augmented-perceptron model simulates CME. Input likelihoods are gaussian distributed. For both modalities V and A, the spontaneous likelihood mean is ¹v0 D ¹a0 D 2, and the driven likelihood mean is ¹v1 D ¹a1 D 6. Spontaneous and driven variances for both modalities are 2 2 D ¾a02 D 5 and ¾v1 D ¾a12 D 6. The spontaneous and driven covariances are ¾v0 2 2 ¾va0 D 0:1 and ¾va1 D 2:8. The responses are equal to posterior target probability computed using the augmented perceptron. The cross-modal both-driven curve (stars) simulates a case in which both modalities are driven by a stimulus. Inputs V D v and A D a are set equal and varied from 0 to 25 impulses per unit time. The modality-speciŽc spontaneous-driven curve (circles) simulates a case where only one modality is driven. The driven input varies from 0 to 25 impulses per unit time, while the input representing the nondriven modality was held Žxed at the mean of the spontaneous likelihood distribution. The both-driven and spontaneous-driven plots represent slices through the full 2D posterior. The no pi both-driven (asterisks) and no pi spontaneous-driven (triangles) curves illustrate the inaccurate probability computation performed for the both-driven and spontaneous-driven cases, respectively, by a network in which the pi nodes have been removed by setting ½v D ½a D ½va D 0. (C) Cross-modal enhancement (see equation 1.1) is plotted for the intact augmented perceptron (stars) and for a version from which the pi nodes have been removed (asterisks). (D) The amount by which removal of the pi nodes reduces the augmented perceptron response to input is illustrated for the both-driven (asterisks) and spontaneous-driven (circles) cases. In each case, reduction R is computed using RD

P.T D 1 j m/ ¡ L.m/ £ 100%; P.T D 1 j m/

where P.T D 1 j m/ is computed using the intact network, and L.m/ is the corresponding value computed by the lesioned network following removal of the pi nodes.

794

P. Patton and T. Anastasio

neous covariance between channels of different modalities should be low. In contrast, the sensory attributes of natural targets are likely to covary, and channels of different modalities will covary as they are driven by these targets. Accordingly, in the two-channel model, we set spontaneous covariance 2 2 D 0:1 and driven covariance ¾ va1 D 2:8. ¾va0 The speciŽc variance and covariance values chosen are consistent with these plausible assumptions and produce an accurate simulation of neurophysiological Žndings on DSC neurons when used to parameterize the two-channel augmented perceptron. An augmented-perceptron model DSC neuron receiving inputs of two modalities (see Figure 2A) is studied in the both-driven case, in which both modalities are driven equally by a stimulus, and in the spontaneous- driven case, in which one modality is driven by a stimulus while the other is spontaneously active. As the number of impulses increases, target probability increases from nearly zero to nearly one, and it increases faster in the both-driven than in the spontaneous-driven case (see Figure 2B). CME reaches its highest values in the region where the spontaneous-driven responses are still small (see Figures 2B and 2C). Presumably, experimental effects on CME would be studied in this region, which will be referred to as the enhancement zone. Binns and Salt (1996) investigated the effects of pharmacologically blocking NMDA receptors on the responses of DSC neurons to modality-speciŽc and cross-modal stimuli. We postulate that NMDA receptors may serve the functional role assigned to pi nodes in the augmented-perceptron model. To investigate this hypothesis, the effects on the model of removing the pi nodes (i.e., setting ½v D ½a D ½va D 0) were compared with the effects reported by Binns and Salt (1996). Following pi node removal, the model no longer accurately computes the posterior probability of a target (see Figure 2B). The effect of pi node removal is to reduce the both-driven and spontaneousdriven responses to a given input below the values computed by the intact model (see Figures 2B and 2D). Removal of the pi nodes does not eliminate enhancement but reduces it within the enhancement zone (see Figure 2C). These effects are produced over a wide range of model variance and covariance parameters. Binns and Salt (1996) also report that blockade of NMDA receptors in DSC neurons produces reductions in both spontaneous-driven and both-driven responses. For the chosen parameters, the percentage by which the model response is reduced is greater for both-driven than for spontaneous-driven input within the enhancement zone (see Figure 2D), as also observed experimentally. The experimental data are thus more consistent with our hypothesis than with the view that NMDA receptors function as a multiplicative ampliŽer for CME (see section 5).

4 Multiple Input Channels and Modality-SpeciŽc Suppression The augmented-perceptron model can be extended to account for the fact that cross-modal stimuli produce robust enhancement of DSC neuron re-

A Neural Model of Cross-Modal Enhancement

795

sponses, whereas paired modality-speciŽc stimuli do not (Stein & Meredith, 1993). As an implementation of the Bayes’ rule model (Anastasio et al., 2000), the augmented perceptron accounts for CME as an increase in the posterior target probability computed by a DSC neuron due to integration of a second input of a different modality. The extension to MSS is based on the intuition that a second input should increase target probability less if it covaries more with the Žrst input and that spontaneous covariance, in particular, is likely to be stronger between inputs of the same modality. Analysis shows this intuition to be correct. Surprisingly, given speciŽc but neurobiologically plausible assumptions concerning the inputs, it can even account for the counterintuitive phenomenon of MSS. Bayes’ rule can thus account for both CME and MSS, and both can be implemented using the same augmented perceptron. For convenience, we refer to visual and auditory modalities in what follows, although the result also applies to other modality combinations. A DSC neuron’s visual receptive Želd is produced by combining input from multiple sources, many of which are known to have much smaller receptive Želds than those of DSC neurons (Olson & Graybiel, 1987; Sparks & Hartwich-Young, 1989; Meredith & Stein, 1990). These visual neurons would have separate but overlapping receptive Želds within the receptive Želd of the DSC neuron. We assume that the paired modality-speciŽc stimuli used to produce MSS will activate separate but overlapping sets of visual neurons. We will refer to the set of visual neurons that can be activated by the Žrst stimulus of the pair as channel V and those activated by the second stimulus as channel X. Receptive Želd overlap will cause the V and X channels to covary under driven conditions. Many of the neurons that provide visual input to the DSC are spontaneously active (see the discussion in Anastasio et al., 2000). Overlap in the sets of visual neurons that comprise channels V and X will cause them to covary under spontaneous conditions. Overlap of spontaneously silent neurons will contribute additional covariance under driven conditions. In addition to the two visual channels, we assume a third channel A representing auditory input. This three-channel case is treated using an augmented perceptron (see Figure 3A). The means, variances, and covariances for the multivariate gaussians, equation 3.1, representing the spontaneous and driven likelihoods of V, X, and A, are set according to the assumptions made above, which are that means, variances, and covariances are higher under driven than under spontaneous conditions and that covariances, especially spontaneous covariances, are higher between modality-speciŽc than between cross-modal channels. The parameters in the augmented-perceptron model can be computed using equation 3.3 as for the two-channel case above. The model can simulate the Žnding that crossmodal input pairs (V and A) produce CME, whereas modality-speciŽc input pairs (V and X) do not (see Figures 3B and 3C). To illustrate the explanatory power of the model, the modality-speciŽc case simulated in Figures 3B and 3C is actually one of suppression (MSS).

796

P. Patton and T. Anastasio

A Neural Model of Cross-Modal Enhancement

797

In the model, MSS arises over a speciŽc but neurobiologically plausible range of parameters. Figure 4 shows the effects of varying spontaneous and 2 and 2 ) on peak driven covariance between the V and X channels (¾vx0 ¾ vx1 CME and MSS. For the simple case considered, all three modalities have equal spontaneous means (¹v0 D ¹x0 D ¹a0 D 2), driven means (¹v1 D 2 2 2 D ¾x0 D ¾a0 D 2), and driven ¹x1 D ¹a1 D 6), spontaneous variances (¾v0 2 2 2 variances (¾v1 D ¾x1 D ¾a1 D 6). Driven covariances between cross-modal 2 2 2 channels are larger than spontaneous covariances (¾va0 D ¾xa0 D 0:1; ¾va1 D 2 ¾xa1 D 2:8). These parameters are the same as those of Figure 3. A real-valued multivariate gaussian distribution, equation 3.1, exists only 2 2 2 when j6t j > 0. This condition is met whenever ¾v2 ¾ a2 ¾x2 C 2¾va ¾vx ¾ax > Figure 3: Facing page. (A) An augmented-perceptron model of multisensory integration for a three-channel case: M1 D V, M2 D X, and M3 D A. Each channel provides weighted input to summing node 6 with weights wv , wx , and wa . The summing node also receives a Žxed bias input b. Each modality multiplies itself and each of the other modalities on the set of six pi nodes 5. The pi nodes provide inputs to the summing node with weights ½v ; ½x ; ½a ; ½vx; ½xa ; and ½va . The output of the summing node is passed through the squashing function f .u/. (B) A three-channel augmented-perceptron model is used to simulate both CME and MSS. The input likelihoods are multivariate gaussians. The spontaneous input likelihoods for all three channels have means ¹v0 D ¹x0 D ¹a0 D 2 and variances 2 2 D ¾x0 D ¾a02 D 2. Driven input likelihoods have means ¹v1 D ¹x1 D ¹a1 D 6 ¾v0 2 2 2 and variances ¾v1 D ¾x1 D ¾a12 D 6. Within-modality covariances ¾vx 0 D 1:6 and 2 ¾vx1 D 3:6 are higher than the corresponding between-modality covariances 2 2 2 2 ¾va 0 D ¾xa0 D 0:1 and ¾va1 D ¾xa1 D 2:8. The responses are equal to the posterior probability computed by the augmented perceptron. Plots for four different input conditions are shown. For each condition, a driven channel is one where the input varies over the range from 0 to 15 impulses per unit time (abscissa), and a spontaneous channel is one where the input is held Žxed at the spontaneous mean. The V or X alone condition (circles) simulates an experiment in which a single visual stimulus is presented. One of the visual channels, V or X, is driven, while the other visual channel and the auditory channel are spontaneous. The A alone case (triangles) simulates an experiment in which an auditory stimulus is presented alone. Channel A is driven, and V and X are spontaneous. The V and X condition (squares) simulates an experiment in which two visual stimuli are presented simultaneously in a DSC neuron’s visual receptive Želd. Channels V and X are driven, and A is spontaneous. The V and A case (stars) simulates an experiment in which a visual and an auditory stimulus are presented simultaneously. Channels V and A are driven, and X is spontaneous. Each plot constitutes a slice through the full 3D posterior. (C) Percentage enhancement is computed as described for Figure 1D. Presentation of two stimuli of different modalities (V and A) produces positive enhancement or CME, while presentation of two stimuli of the same modality (V and X) produces negative enhancement or MSS.

798

P. Patton and T. Anastasio

4 4 2 C ¾a2 ¾vx C ¾x2 ¾va . The chart in Figure 4 is bounded at the right and ¾v2 ¾ax on the bottom by a region for which no real-valued multivariate gaussian distribution exists. Peak CME (white bars) is always greater than 200% (asterisks) for spontaneous covariance values less than 0.6 (not shown). Suppression (black bars) becomes evident when the spontaneous covariance 2 approaches the spontaneous variance 2 D 2 D 2 D 2, but is rel¾vx0 ¾v0 ¾x0 ¾a0 2 atively uninuenced by the driven covariance ¾ vx1 . Similar relationships are observed when the variances for different modalities are unequal. This

A Neural Model of Cross-Modal Enhancement

799

parametric analysis further implicates spontaneous covariance as a factor causing suppression of one input channel by another. Graphical analysis illustrates how spontaneous covariance between input channels can lead to suppression. It also shows that spontaneous covariance is necessary for suppression but not sufŽcient, because spontaneous covariance leads to suppression only when driven variance is greater than spontaneous variance. Multivariate gaussian input likelihood distributions (P.m j T D t/; see equation 3.1) for the three-channel model are plotted in Figure 5, for a series of different variance and covariance conditions. Figure 6 plots posterior target probabilities (P.T D 1 j m/; see equation 2.2) for the same series of conditions. The heavy dashed curve in Figure 5 indicates the intersection of the spontaneous and driven likelihood distributions, P.m j T D 0/ D P.m j T D 1/. For inputs falling on this curve, the posterior probability of a target is equal to its prior probability, P.T D 1 j m/ D P.T D 1/. The same intersection curve is again shown in Figure 6, which plots posterior target probabilities. In the white region of Figure 6, P.T D 1 j m/ is nearly 0, and in the gray region nearly 1. In each panel of Figures 5 and 6, the dotted diagonal line indicates the case where channels V and X are held equal and driven over the range from 0 to 15, as in Figure 3. The dot-dashed horizontal line indicates the case where V alone is driven from 0 to 15, and X is held Žxed at its spontaneous mean. The dot-dashed vertical line indicates the case where X alone is driven from 0 to 15, with V remaining Žxed at its spontaneous mean. In all cases, A channel input is held Žxed at its spontaneous mean ¹a0 D 2. Figure 4: Facing page. Spontaneous covariance is associated with MSS in the three-channel augmented-perceptron model. Spontaneous and driven covari2 2 ance (¾vx 0 and ¾vx1 ) between the V and X channels are varied, while all other parameters are as speciŽed for Figure 3. Each of the rectangles corresponds to 2 2 one combination of ¾vx 0 and ¾vx1 , as indicated. VX enhancement was computed by comparing the combined response CR and the maximum single modality response SRmax using equation 1.1. For CR, V and X were both driven by a stimulus over the range from 0 to 30, while A was held Žxed at its spontaneous mean. For SRmax , V alone was driven over the range from 0 to 30, and X and A were held Žxed at their spontaneous means. The maximum percentage enhancement over the range was computed and is indicated for each covariance combination by the height of a white bar above the horizontal line representing zero enhancement. Maximum enhancements exceeding 200% are indicated by a white bar with an asterisk. Spontaneous covariances less than 0.6 (not shown) produced enhancements greater than 200%. The minimum enhancement is computed for inputs that are larger than those at which the maximum occurred. If the minimum observed was less than zero, it is indicated as a black bar extending downward from the horizontal zero enhancement line. The depth of the bar indicates suppression on a scale of 0 to ¡200%.

800

P. Patton and T. Anastasio

Figure 5: Multivariate gaussian input likelihood distributions for the threechannel model under spontaneous and stimulus-driven conditions for different settings of variance and covariance. In each panel, spontaneous and driven likelihood is plotted as a function of V and X channel input. A channel input is held Žxed at its spontaneous mean (a D ¹a0 D 2). Each plot represents a slice through a 3D likelihood distribution. Likelihood value is indicated by contour lines and shading, where darker shading indicates a higher likelihood. In each panel of Figures 5 and 6, the diagonal dotted line indicates v D x, which corresponds to the V- and X-driven case of Figure 3. The horizontal dotdashed line corresponds to the V-alone case, where v varies from 0 to 15 and x D a D ¹x0 D ¹a0 D 2. The vertical dot-dashed line corresponds to the X-alone case. The heavy dashed curve in each panel indicates the points of intersection of the spontaneous and driven likelihood distributions (P.m j T D 0/ D P.m j T D 1/). (A) The spontaneous and driven covariances between all channels

A Neural Model of Cross-Modal Enhancement

801

2 2 2 2 2 Figure 5 (cont.): Facing page. are zero (¾va 0 D ¾xa0 D ¾vx0 D 0; ¾va1 D ¾xa1 D 2 D 0). All other parameters are as speciŽed in Figure 3. The concentric ¾vx 1 contour lines about the point v D 2, x D 2 indicate the spontaneous likelihood distribution. From outermost to innermost, the contour lines indicate P.m j T D 0/ D 0:005; 0:01; 0:015, and 0.02, respectively. The concentric contour lines about the point v D 6, x D 6 indicate the driven likelihood distribution. From outermost to innermost P.m j T D 1/ D 2£10¡4 ; 4£10¡4 ; 6£10¡4 ; 8£10¡4 , and 1 £ 10¡3 respectively. The corresponding posterior target probability distribution is shown in Figure 6A. (B) The spontaneous and driven covariances are 2 2 2 nonzero and assigned the same values as for Figure 3 (¾va 0 D ¾xa0 D 0:1; ¾vx0 D 2 2 2 1:6; ¾va1 D ¾xa1 D 2:8, and ¾vx1 D 3:6). All other parameters are also as speciŽed in Figure 3. The concentric contour lines centered about v D 2, x D 2 indicate the spontaneous likelihoods P.m j T D 0/ D 0:005; 0:01; 0:015; 0:02; 0:025; 0:03; and 0.035. The second set of concentric contour lines indicates the driven likelihoods P.m j T D 1/ D 2 £ 10¡4 ; 4 £ 10¡4 ; 6 £ 10¡4 ; 8 £ 10¡4 ; 0:001; and 0.0012. The corresponding target posterior probability distribution is shown in Figure 6B. (C) The spontaneous and driven covariances between the V, X, and A channels are zero, but the variances are larger for the V channel than for the X and A 2 2 2 2 channels (¾v0 D 8; ¾x0 D ¾a02 D 2; ¾v1 D 16; ¾x1 D ¾a12 D 6). Distribution means are as in Figure 3. The concentric contour lines about v D 2, x D 2 indicate the spontaneous likelihoods P.m j T D 0/ D 0:002; 0:004; 0:006; 0:008; and 0.01. The concentric contour lines centered about v D 6, x D 6 indicate the driven likelihoods P.m j T D 1/ D 1 £ 10¡4 ; 2 £ 10¡4 ; 3 £ 10¡4 ; 4 £ 10¡4 ; 5 £ 10¡4 ; and 6 £ 10¡4 . The corresponding target posterior probability distribution is shown in Figure 6C. (D) The spontaneous and driven covariances are nonzero 2 2 2 2 2 as in Figures 3 and 5B (¾va 0 D ¾xa0 D 0:1; ¾vx0 D 1:6; ¾va1 D ¾xa1 D 2:8, and 2 ¾vx1 D 3:6), but the spontaneous variance is larger than the driven variance 2 2 2 2 (¾v0 D ¾x0 D ¾a02 D 8; ¾v1 D ¾x1 D ¾a12 D 6). Distribution means are again as for Figure 3. The concentric contour lines about v D 2, x D 2 indicate the spontaneous likelihoods P.m j T D 0/ D 0:0005; 0:001; 0:0015; 0:002; and 0.0025. The second set of concentric contour lines indicates the driven likelihoods P.m j T D 1/ D 2 £ 10¡4 ; 4 £ 10¡4 ; 6 £ 10¡4 ; 8 £ 10¡4 ; 0:001; 0:0012; 0:0014; and 0.0016. The corresponding target posterior probability distribution is shown in Figure 6D.

In Figure 5A the driven variances are equal, the spontaneous variances are equal in all three channels, and driven variance is greater than spontaneous variance, as in Figure 3. Unlike Figure 3, there is no covariance between any of the channels in Figure 5A. Under these circumstances, the spontaneous (T D 0) and driven (T D 1) likelihood contours (see Figure 5A), and their associated posterior probability contours (see Figure 6A), are circular. The heavy vertical line in Figure 6A connects a point along the V and X both-driven line with the corresponding point along the V-only,

802

P. Patton and T. Anastasio

spontaneous-driven line. The both-driven point has posterior probability near one, while the spontaneous-driven point has posterior probability near zero, indicating very large enhancement. For the circular case, Figure 6A illustrates that both-driven posteriors will always be higher than the corresponding spontaneous-driven posteriors and, consequently, suppression cannot occur. In Figures 5B and 6B, the driven variances are equal, the spontaneous variances are equal again in all three channels, and driven variance is greater than spontaneous variance. Cross-modal channels are assigned a very small spontaneous covariance and a larger driven covariance. The V and X channels are assigned a much larger spontaneous covariance than the crossmodal channels and a larger driven covariance. All parameters have the

A Neural Model of Cross-Modal Enhancement

803

same values as for Figure 3. Due to nonzero covariance, the spontaneous and driven likelihood contours (see Figure 5B) are elliptical and elongated in the diagonal direction. The place of intersection of the spontaneous and driven likelihoods, and the posterior probability contours (see Figure 6B), are similarly elliptical and diagonally elongated. This effect is due primarily to the large VX spontaneous covariance. The heavy vertical line in Figure 6B connects a both-driven point having posterior probability near zero with a corresponding spontaneous-driven point having posterior probability near one, indicating suppression. Figure 6B illustrates how suppression can arise from the Bayes’ rule model and be simulated by a neural implementation of it when modality-speciŽc channels exhibit spontaneous covariance and driven variance exceeds spontaneous variance. Figure 6: Facing page. Posterior target probability (P.T D 1 j m/) is plotted as a function of V and X channel input for the three-channel model described in Figure 3. Posterior target probability is indicated in each panel by shading and contour lines. White indicates P.T D 1 j m/ values near zero, and gray indicates P.T D 1 j m/ values near one. Contour lines are at 0.1 unit intervals. The heavy dotted curve in each panel indicates the input values for which the posterior and prior target probabilities are equal .P.T D 1 j m/ D P.T D 1/ D 0:1/, which is the same as the P.m j T D 0/ D P.m j T D 1/ curve in the corresponding panels of Figure 5. The dotted and dot-dashed lines are as described in Figure 5. (A) The likelihood parameters are as given for Figure 5A. An input parameter combination that produces enhancement of 1075% is shown by the heavy vertical line terminated with circular dots. The V and X both-driven posterior probability P.T D 1 j V D 6; X D 6; A D 2/ D 0:94 (upper dot) and the V alone posterior P.T D 1 j V D 6; X D 2; A D 2/ D 0:08 (lower dot). (B) The likelihood parameters are as given for Figure 5B. An input parameter combination that produces enhancement of ¡83:3% (suppression) is shown by the heavy vertical line terminated with circular dots. The V and X both-driven posterior probability P.T D 1 j V D 5:8; X D 5:8; A D 2/ D 0:16 (upper dot) and the V alone posterior P.T D 1 j V D 5:8; X D 2; A D 2/ D 0:96 (lower dot). (C) The likelihood parameters are as given for Figure 5C with different variances for the V and X channels. An input parameter combination that produces enhancement of only 40.3% is shown by the heavy vertical line terminated with circular dots. The V and X both-driven posterior probability P.T D 1 j V D 7; X D 7; A D 2/ D 0:94 (upper dot) and the V alone posterior P.T D 1 j V D 7; X D 2; A D 2/ D 0:67 (lower dot). (D) The likelihood parameters are as described in Figure 5D in which the spontaneous variance is larger than the driven variance. Here, the V and X both-driven posterior probability reaches a maximum value of only 0.27, and thus only two contour lines are visible. An input parameter combination that produces enhancement of 8337% is shown by the heavy vertical line terminated with circular dots. The V and X both-driven posterior probability P.T D 1 j V D 10; X D 10; A D 2/ D 0:27 (upper dot) and the V alone posterior P.T D 1 j V D 10; X D 2; A D 2/ D 0:0032 (lower dot).

804

P. Patton and T. Anastasio

Covariance is necessary for suppression. Elongation of the spontaneous or driven likelihoods by itself, without covariance, can decrease enhancement but cannot produce suppression. The spontaneous and driven covariances are set to zero in Figure 5C, but the likelihoods are elongated along the X channel axis (ordinate) by setting the V variance greater than the X variance under both spontaneous and driven conditions. The equal likelihood (see Figure 5C) and posterior probability (see Figure 6C) contours are also elongated along the X channel axis. The heavy vertical line in Figure 6C connects a both-driven point with a corresponding spontaneous-driven point that have almost equal posterior probabilities, indicating very little enhancement. Increasing V variance further would bring the posterior probability contours closer to parallel with the X channel axis. This would further reduce enhancement but never lead to suppression. Suppression requires the elongation combined with diagonalization of likelihood distributions (see Figures 5B and 6B) that can be produced only by covariance. Covariance is necessary but not sufŽcient for suppression. Suppression requires spontaneous covariance (with or without driven covariance), and that driven variance is greater than spontaneous variance. In Figures 5D and 6D, channel covariances are as in Figures 5B and 6B, and the spontaneous variances are equal and the driven variances are equal for all three channels. Unlike the case of Figures 5B and 6B, the driven variance in Figures 5D and 6D is smaller than the spontaneous variance. Now the heavy dashed curve indicating equal spontaneous and driven likelihood (see Figure 5D) and equal prior and posterior probability (see Figure 6D) forms a closed ellipse containing the point speciŽed by the driven V and X means. The spontaneous-driven lines never intersect this ellipse. Within the ellipse, the posterior probability of a target never exceeds 0.27. The heavy vertical line in Figure 6D connects a both-driven point having posterior probability 0.27 with a spontaneous-driven point having posterior probability near zero, producing sizable enhancement only because of the small magnitude of the latter value. In summary, suppression can occur between two channels when driven variance is larger than spontaneous variance and spontaneous covariance approaches the value of spontaneous variance (with or without driven covariance). These conditions are very likely to be met by channels of the same modality that provide input to neurons in the DSC and elsewhere in the brain (see above). It is very likely that driven variance will be higher than spontaneous variance, because driven variance reects the huge variability in stimulus effectiveness, in addition to the stochastic properties of the nervous system that determine spontaneous variance. It is also very likely that input channels of the same modality will covary under both spontaneous and driven conditions, because they may share some of the same input axons. The seemingly paradoxical phenomenon of MSS arises naturally from the Bayes’ rule model of CME, given these neurobiologically plausible conditions of input channel variance and covariance. The three-channel neural

A Neural Model of Cross-Modal Enhancement

805

implementation presented in Figure 3 illustrates an example in which CME and MSS can arise in the same model neuron. The implementation can be extended to any number of cross-modal and modality-speciŽc channels. A two-channel, single-modality model was also investigated, and suppression was found to occur under similar parametric circumstances. 5 Discussion CME is often referred to as supra-additive or multiplicative, since a neuron’s response to multimodal stimuli is often greater than the sum of the responses to the single modality stimuli presented alone (Meredith & Stein, 1986b; Stein et al., 1988; Meredith et al., 1992; Stein & Meredith, 1993). It has been suggested that CME requires a special nonlinear ampliŽer, such as an ion channel, that responds to multimodal but not unimodal input (Stein & Meredith, 1993; Binns & Salt, 1996; Binns, 1999). The implementations we have examined here suggest a different view of cross-modal enhancement. We have previously argued that CME arises as a consequence of DSC neurons computing the posterior probability of a target using Bayes’ rule (Anastasio et al., 2000). Here we use neural implementations of the Bayes’ rule model to address issues of neurobiological mechanism. A single perceptron is capable of computing the target posterior when the inputs are conditionally independent and Poisson distributed, or gaussian distributed without the assumption of conditional independence, provided variances and covariances for the spontaneous likelihood distribution are equal to those for the driven likelihood distribution. When the above equality is not satisŽed, an augmented perceptron, which allows inputs to be multiplied together, is needed to compute the posterior probability of a target. In either case, CME itself is produced by weighted summation in a nonlinear model neuron. Multiplication of inputs is not needed to produce CME. Instead, it serves a function not envisioned by the multiplicative ampliŽer hypothesis. It serves to compensate for differences between spontaneous and driven variance and covariance, to allow computation of posterior target probability when these parameters are unequal. It has been suggested that NMDA-sensitive glutamate conductances present in DSC neurons might provide a multiplicative ampliŽer for CME (Stein & Meredith, 1993; Binns & Salt, 1996; Binns, 1999). Binns and Salt (1996) tested this hypothesis by applying the NMDA receptor antagonist AP5 to multisensory DSC neurons in the cat during cross-modal and modality-speciŽc stimulation. The results were equivocal with respect to the ampliŽer hypothesis, since AP5 did not selectively eliminate CME. Instead, application of AP5 resulted in a reduction of both modality-speciŽc and crossmodal responses, with the percentage reduction being greater for the latter. It is possible to reconcile these Žndings with the multiplicative hypothesis by proposing, for example, that NMDA receptors involved in cross-modal responses have different channel kinetics from those involved in modality-

806

P. Patton and T. Anastasio

speciŽc responses (Binns, 1999). Our model, however, can account for the Žndings without any such special assumptions. The function of the pi nodes in the augmented perceptron model might be implemented by dendritic voltage-sensitive conductances in DSC neurons (Koch & Poggio, 1992; Mel, 1993). We propose that rather than serving as multiplicative ampliŽers for CME, NMDA receptors in DSC neurons might perform the function of these pi nodes. Simulations in which pi nodes were removed from the model, over a suitable range of parameters, yielded results similar to those observed in the NMDA channel blockade studies. Removing the pi nodes from the augmented-perceptron model similarly produced a reduction in response to both modality-speciŽc and cross-modal stimuli, with the percentage reduction being greater for the latter (see Figure 2C). Our model provides a straightforward explanation for the NMDA channel blockade Žndings, whereas the ampliŽer model of CME does not. In addition to multimodal interactions, Stein and Meredith (1993) have examined interactions between two stimuli of the same modality. Given the assumptions that driven variances are greater than the spontaneous variances and that signiŽcant spontaneous covariance should occur between modality-speciŽc but not between cross-modal input channels, the augmented-perceptron implementation simulated the absence of enhancement for modality-speciŽc stimuli and the presence of enhancement for cross-modal stimuli. Furthermore, our model proved capable of simulating the counterintuitive Žnding of modality-speciŽc suppression. For suppression to occur in the model, the spontaneous covariance between the V and X channels must be less than, but close to, the spontaneous variances. When these conditions are met, the VX pi node invariably has a negative weight .½vx < 0/. Neurobiologically, a pi node with a negative weight might correspond, for example, to an inhibitory interneuron with a nonlinear conductance perhaps produced by NMDA receptors. Depressive interactions, in which the neural response to a cross-modal stimulus is less than that elicited by a single stimulus, can occur when the two stimuli are sufŽciently spatially disparate (Meredith & Stein, 1986a, 1986b, 1996; Kadunce, Vaughan, Wallace, Benedek, & Stein, 1997; Kadunce et al., 2001). Pairs of modality-speciŽc stimuli can also produce such effects when one of the two is outside the excitatory receptive Želd (Kadunce et al., 1997). This is distinct from the modality-speciŽc suppression discussed above, in which both stimuli are presented within the excitatory receptive Želd. Depression due to spatial disparity appears to result from competitive interactions involved in target selection (Munoz & Istvan, 1998; Findlay & Walker, 1999) rather than target detection, which is the focus of this study. Our results do not preclude competitive interaction as a possible explanation for modality-speciŽc suppression. They do demonstrate that modality-speciŽc suppression can be simulated using the Bayes’ rule model of CME and neural implementations thereof and is therefore a plausible consequence of target detection processes.

A Neural Model of Cross-Modal Enhancement

807

CME in the DSC apparently depends on cortical-descending input. The DSC of the cat receives descending cortical input from certain areas of extraprimary sensory cortex, including the anterior ectosylvian cortex (AES) and the lateral suprasylvian sulcus (Berson & McIlwain, 1983; Harting, Updyke, & Van Lieshout, 1992; Wallace et al., 1993). Inactivation of these areas, and of AES in particular, has been shown to selectively abolish CME, while preserving multisensory responsiveness in DSC neurons (Wallace & Stein, 1994; Jiang, Wallace, Jiang, Vaughan, & Stein, 2001). The model suggests that the cortical input could produce CME simply by adding a component to the DSC response to subcortical input, but it offers no insight into how the cortical and subcortical contributions may be partitioned. We are exploring a new model that subsumes the implementations proposed here within a broader model of the development of CME, using intrinsic Hebbian mechanisms and both ascending and cortical-descending inputs to the DSC. For the case of conditionally independent Poisson distributed inputs, the bias b is negative whenever P.T D 1/ < P.T D 0/ and ¸i0 < ¸i1 . For the more general case, the value of b is inuenced by all of the other parameters in the model (see equation 3.9). A number of restricted cases were examined, and it was found that b was negative for all but a small portion of parameter space (near asymptotes) whenever a set of plausible parameter requirements was maintained. The inhibitory bias might correspond to some biophysical mechanism intrinsic to DSC neurons or to an extrinsic source of tonic inhibition. The DSC contains inhibitory interneurons and receives extrinsic inhibitory input from the substantia nigra, zona incerta, contralateral colliculus, and a variety of brainstem structures (Karabelas & Moschovakis, 1985; Ficalora & Mize, 1989; Appell & Behan, 1990; Mize, 1992; May, Sun, & Hall, 1997). Both substantia nigra and zona incerta cells are tonically active (Hikosaka & Wurtz, 1983; Ma, 1996). Some substantia nigra cells respond to visual or auditory stimulation with a decrease or cessation of Žring. Because the bias contains a term related to the target prior probability, it is possible that the sensory responses of nigral cells could modulate CME in the DSC according to changes in the prior expectation of targets. Acknowledgments We thank Jeffrey Bilmes, Ehtibar Dzhafarov, Jesse Reichler, and Liudmila Yafremava for helpful discussions and comments on the manuscript prior to submission. We also thank anonymous reviewers of previous versions of this article for useful comments. This work was supported by NSF grants IBN 92-21823 and IBN 00-80789, ONR grant N00014-01-0249, and a grant from the Critical Research Initiatives of the State of Illinois, all to T. J. A. The code used to generate the results presented here will be made available at our web site: http://csn.beckman.uiuc.edu/ .

808

P. Patton and T. Anastasio

References Anastasio, T. J., Patton, P. E., & Belkacem-Boussaid, K. (2000). Using Bayes’ rule to model multisensory enhancement in the superior colliculus. Neural Computation, 12, 997–1019. Appell, P. P., & Behan, M. (1990). Sources of subcortical GABAergic projections to the superior colliculus in the cat. J. Comp. Neurol., 302(1), 143–158. Berson, D. M., & McIlwain, J. T. (1983). Visual cortical inputs to the deep layers of cat’s superior colliculus. J. Neurophysiol., 50, 1143–1155. Binns, K. E. (1999). The synaptic pharmacology underlying sensory processing in the superior colliculus. Prog. Neurobiol., 59(2), 129–159. Binns, K. E., & Salt, T. E. (1996). Importance of NMDA receptors for multimodal integration in the deep layers of the cat superior colliculus. J. Neurophysiol., 75(2), 920–930. Clemo, H. R., & Stein, B. E. (1991). Receptive Želd properties of somatosensory neurons in the cat superior colliculus. J. Comp. Neurol., 314(3), 534–544. Cybenko, G. (1989). Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals, and Systems, 2, 303–314. Duda, R. O., Hart, P. E., & Stork, D. G. (2001). Pattern classiŽcation (2nd ed.). New York: Wiley. Durbin, R., & Rumelhart, D. E. (1989). Product units: A computationally powerful and biologically plausible extension to backpropagation networks. Neural Computation, 1, 133–142. Ficalora, A. S., & Mize, R. R. (1989). The neurons of the substantia nigra and zona incerta which project to the cat superior colliculus are GABA immunoreactive: A double label study using GABA immunocytochemistry and lectin retrograde transport. Neurosci., 29, 567–581. Findlay, J. M., & Walker, R. (1999). A model of saccade generation based on parallel processing and competitive inhibition. Behav. Brain Sci., 22, 661–721. Fristedt, B., & Gray, L. (1997). A modern approach to probability theory. Boston: Birkh a¨ user. Gabbiani, F., & Koch, C. (1998). Principles of spike train analysis. In C. Koch & I. Segev (Eds.), Methods in neuronal modeling (2nd ed., pp. 312–360). Cambridge, MA: MIT Press. Harting, J. K., Updyke, B. V., & Van Lieshout, D. P. (1992). Corticotectal projections in the cat: Anterograde transport studies of twenty-Žve cortical areas. J. Comp. Neurol., 324(3), 379–414. Hikosaka, O., & Wurtz, R. H. (1983). Visual and oculomotor functions of the monkey substantia nigra pars reticulata. I. Relation of visual and auditory responses to saccades. J. Neurophysiol., 49, 1254–1267. Jiang, H., Lepore, F., Ptito, M., & Guillemot, J.-P. (1994). Sensory interactions in the anterior ectosylvian cortex of cats. Exp. Brain Res., 101, 385–396. Jiang, W., Wallace, M. T., Jiang, H., Vaughan, J. W., & Stein, B. E. (2001). Two cortical areas mediate multisensory integration in superior colliculus neurons. J. Neurophysiol., 85, 506–522. Kadunce, D. C., Vaughan, J. W., Wallace, M. T., Benedek, G., & Stein, B. E. (1997). Mechanisms of within- and cross-modality suppression in the superior colliculus. J. Neurophysiol., 78(6), 2834–2847.

A Neural Model of Cross-Modal Enhancement

809

Kadunce, D. C., Vaughan, J. W., Wallace, M. T., & Stein, B. E. (2001). The inuence of visual and auditory receptive Želd organization on multisensory integration in the superior colliculus. Exp. Brain Res., 139, 303–310. Karabelas, A. B., & Moschovakis, A. K. (1985). Nigral inhibitory termination on efferent neurons of the superior colliculus: An intracellular horseradish peroxidase study in the cat. J. Comp. Neurol., 239(3), 309–329. Koch, C., & Poggio, T. (1992). Multiplying with synapses and neurons. In T. McKenna, J. L. Davis, & S. F. Zornetzer (Eds.), Single neuron computation (pp. 315–345). San Diego, CA: Academic Press. Ma, T. P. (1996). Saccade related omnivectorial pause neurons in the primate zona incerta. NeuroReport, 7, 2713–2716. May, P. J., Sun, W., & Hall, W. C. (1997). Reciprocal connections between the zona incerta and the pretectum and superior colliculus of the cat. Neurosci., 77(4), 1091–1114. Mel, B. W. (1993). Synaptic integration in an excitable dendritic tree. J. Neurophysiol., 70(3), 1086–1101. Meredith, M. A., & Stein, B. E. (1986a). Spatial factors determine the activity of multisensory neurons in cat superior colliculus. Brain Res., 365(2), 350– 354. Meredith, M. A., & Stein, B. E. (1986b).Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. J. Neurophysiol., 56(3), 640–662. Meredith, M. A., & Stein, B. E. (1990). The visuotopic component of the multisensory map in the deep laminae of the cat superior colliculus. J. Neurosci., 10(11), 3727–3742. Meredith, M. A., & Stein, B. E. (1996). Spatial determinants of multisensory integration in cat superior colliculus neurons. J. Neurophysiol., 75(5), 1843– 1857. Meredith, M. A., Wallace, M. T., & Stein, B. E. (1992). Visual, auditory and somatosensory convergence in output neurons of the cat superior colliculus: Multisensory properties of the tecto-reticulo-spinal projection. Exp. Brain Res., 88(1), 181–186. Middlebrooks, J. C., & Knudsen, E. I. (1984). A neural code for auditory space in the cat’s superior colliculus. J. Neurosci., 4(10), 2621–2634. Mize, R. R. (1992). The organization of GABAergic neurons in the mammalian superior colliculus. Prog. Brain Res., 90, 219–248. Munoz, D. P., & Istvan, P. J. (1998). Lateral inhibitory interactions in the intermediate layers of the monkey superior colliculus. J. Neurophysiol., 79(3), 1193–1209. Olson, C. R., & Graybiel, A. M. (1987). Ectosylvian visual area of the cat: Location retinotopic organization, and connections. J. Comp. Neurol., 261, 277–294. Peck, C. K. (1990). Neuronal activity related to head and eye movements in cat superior colliculus. J. Physiol. (Lond.), 421, 79–104. Reynolds, J. H., Chelazzi, L., & Desimone, R. (1999). Competitive mechanisms subserve attention in macaque areas V2 and V4. Journal of Neuroscience, 19(5), 1736–1753. Richard, M. D., & Lippmann, R. P. (1991). Neural network classiŽers estimate Bayesian a posteriori probabilities. Neural Computation, 3, 461–483.

810

P. Patton and T. Anastasio

Rumelhart, D. E., Hinton, G. E., & McClelland, J. L. (1986). A general framework for parallel distributed processing. In D. E. Rumelhart & J. L. McClelland (Eds.), Parallel distributed processing: Explorations in the microstructure of cognition (Vol. 1, pp. 45–76). Cambridge, MA: MIT Press. Sato, T. (1989). Interactions of visual stimuli in the receptive Želds of inferior temporal neurons in awake macaques. Exp. Brain Res., 77, 23–30. Sato, T. (1995). Interactions between two different visual stimuli in the receptive Želds of inferior temporal neurons in macaques during matching behaviors. Exp. Brain Res., 105, 209–219. Sparks, D. L., & Hartwich-Young, R. (1989). The deep layers of the superior colliculus. In R. H. Wurtz & M. Goldberg (Eds.), The neurobiology of saccadic eye movements (Vol. 3, pp. 213–255). Amsterdam: Elsevier. Stein, B. E., Huneycutt, W. S., & Meredith, M. A. (1988). Neurons and behavior: The same rules of multisensory integration apply. Brain Res., 448(2), 355–358. Stein, B. E., & Meredith, M. A. (1993). The merging of the senses. Cambridge, MA: MIT Press. Stein, B. E., Meredith, M. A., Huneycutt, W. S., & McDade, L. (1989). Behavioral indices of multisensory integration: Orientation to visual cues is affected by auditory stimuli. J. Cog. Neurosci., 1(1), 12–24. Stein, B. E., & Wallace, M. T. (1996). Comparisons of cross-modality integration in midbrain and cortex. Prog. Brain Res., 112, 289–299. Wallace, M. T., Meredith, M. A., & Stein, B. E. (1992). Integration of multiple sensory modalities in cat cortex. Exp. Brain Res., 91(3), 484–488. Wallace, M. T., Meredith, M. A., & Stein, B. E. (1993). Converging inuences from visual, auditory, and somatosensory cortices onto output neurons of the superior colliculus. J. Neurophysiol., 69(6), 1797–1809. Wallace, M. T., Meredith, M. A., & Stein, B. E. (1998). Multisensory integration in the superior colliculus of the alert cat. J. Neurophysiol., 20(2), 1006–1010. Wallace, M. T., & Stein, B. E. (1994). Cross-modal synthesis in the midbrain depends on input from cortex. J. Neurophysiol., 71(1), 429–432. Wallace, M. T., Wilkinson, L. K., & Stein, B. E. (1996). Representation and integration of multiple sensory inputs in primate superior colliculus. J. Neurophysiol., 76(2), 1246–1266. Wurtz, R. H., & Goldberg, M. E. (1972).Activity of superior colliculus in behaving monkey. III. Cells discharging before eye movements. J. Neurophysiol., 35(4), 575–586. Received May 24, 2000; accepted September 11, 2002.

LETTER

Communicated by Wulfram Gerstner

Event-Driven Simulation of Spiking Neurons with Stochastic Dynamics Jan Reutimann [email protected]

Michele Giugliano [email protected]

Stefano Fusi [email protected] Computational Neuroscience, Institute of Physiology, University of Bern, 3012 Bern, Switzerland

We present a new technique, based on a proposed event-based strategy (Mattia & Del Giudice, 2000), for efŽciently simulating large networks of simple model neurons. The strategy was based on the fact that interactions among neurons occur by means of events that are well localized in time (the action potentials) and relatively rare. In the interval between two of these events, the state variables associated with a model neuron or a synapse evolved deterministically and in a predictable way. Here, we extend the event-driven simulation strategy to the case in which the dynamics of the state variables in the inter-event intervals are stochastic. This extension captures both the situation in which the simulated neurons are inherently noisy and the case in which they are embedded in a very large network and receive a huge number of random synaptic inputs. We show how to effectively include the impact of large background populations into neuronal dynamics by means of the numerical evaluation of the statistical properties of single-model neurons under random current injection. The new simulation strategy allows the study of networks of interacting neurons with an arbitrary number of external afferents and inherent stochastic dynamics. 1 Introduction Most of the observed in vivo cortical phenomena are believed to be the expression of the collective dynamics of large populations of interacting neurons. The richness and the complexity of these phenomena call for powerful new techniques for investigating in a systematic way the emergent behavior of large networks of neurons. In this respect, computer simulations are becoming an increasingly important tool to model realistic cortical modules (104 –105 neurons and 107 –109 synapses), to relate single-neuron and synapse properties to the collective dynamics of large networks and to Neural Computation 15, 811–830 (2003)

c 2003 Massachusetts Institute of Technology °

812

J. Reutimann, M. Giugliano, and S. Fusi

check the validity of the numerous assumptions underlying the theories of neural networks as dynamical systems. Traditional Žxed time-step (synchronous) computer simulations are based on numerical methods for solving a set of coupled differential equations that describe the dynamics of the basic elements of the networks (the neurons and the synapses). The time axis is uniformly discretized, and each state variable is updated at every time step. Under such conditions, the algorithmic complexity usually scales with the number of elements contained in the network, multiplied by the number of time steps into which the simulated interval is divided. Such an approach also sets a cutoff on the temporal resolution of the simulation (the minimal time step), which is imposed on all the state variables, no matter whether they are characterized by completely different timescales. This represents an obvious drawback when slow and fast dynamic variables coexist, as all of them are updated at the very same pace, determined by the fastest variable. When the number of neurons and synapses approaches realistic numbers for a cortical column (» 105 ), the traditional techniques become impracticable, and new alternative simulation strategies should be devised. One possible solution takes natural inspiration from the way the neurons exchange information and interact: most of the dynamical variables are coupled for relatively short time intervals only, during the emission and the transmission of action potentials. The emission of a spike is a relatively rare event (the typical cortical frequencies being of the order of several Hz) and well localized in time (i.e., the width of a spike is of the order of 1 ms) with respect to the overall simulation times. Interestingly, it can be observed that within the interval between two consecutive relevant events (e.g., spikes as in our case, although our approach can be generalized to other classes of events), the neurons and the synapses are essentially isolated, and each state variable is usually described by a deterministic, uncoupled differential equation. Therefore, given the initial state at the beginning of the interval and the length of the interval, the Žnal state can be determined without iteratively updating the state variable at every time step. This approach is usually known as event driven; it constitutes the fundamental principle of modern operating systems and has already been applied successfully to large-scale simulations of networks of spiking neurons (Watts, 1994; Delorme, Gautrais, van Rullen, & Thorpe, 1999; Giugliano, 2000; Mattia & Del Giudice, 2000). The CPU time of these simulations depends mostly on the frequency of occurrence of relevant events, not on the time simulated. If the relevant events are rare, as would be the spikes in the cortex, then the event-driven approach reduces the computational loads dramatically. One drawback of this strategy manifests itself when the neuronal dynamics does not evolve deterministically between the emission of two successive spikes—for example, when the network to be studied is embedded in a very large population of neurons that provide an external, noisy background activity. This spike activity is usually meant to emulate the sensory

Event-Driven Simulation of Spiking Neurons

813

inputs or, in general, the inputs coming from other areas of the brain (Amit & Brunel, 1997a; Wang, 1999). This interaction, at least on a Žrst approximation, is modeled as a one-way connection (feedforward input) in the sense that the local simulated activity of the network of neurons does not affect the background activity that represents the external input. Each simulated (internal) neuron receives synaptic inputs from a group of Mext external neurons. The external population is assumed to be large, and the overlap among the external groups corresponding to different internal neurons is usually small. Hence, the total synaptic currents generated by the external neurons and injected into different internal neurons can be considered statistically independent. These inputs provide the external stimulation and allow the network to sustain the spontaneous activity observed in cortical recordings (Amit & Brunel, 1997a). In the case of the event-based simulation approach of Mattia and Del Giudice (2000), such an external activity was implemented by generating, for each of the N neurons in the simulated networks, a large number of additional random presynaptic spike events, Žred on average at a frequency ºext by each of the Mext neurons of the external background populations. Such hypotheses give rise to an additional algorithmic complexity that can be quantiŽed as O .N ¢ºext ¢ Mext /. As a consequence, the increased realism introduced in the simulated neuronal dynamics, by appropriately increasing the size of the individual background populations for each unit of the network, has to be paid in terms of a considerable portion of CPU loads, substantially slowing the event-driven approach. Moreover, as the size of the external population increases, the external spikes can very quickly saturate the list of events that have to be handled by the simulator, and the inter-event interval can be as short as a few nanoseconds (it scales as O .1=.N ¢ ºext ¢ Mext //). In particular, such short time intervals can easily produce roundoff errors (see Press, Flannery, Teukolsky, & Vetterling, 1986) and require double-precision oating-point variables to encode the time of occurrence of the events. In this article, we propose a possible solution to these problems. If the synaptic background activity is irregular, as in the in vivo phenomena to be modeled, the impact of the external neurons on each simulated neuron of the network can be approximated by a continuous random current injection with appropriate statistical properties, directly related to ºext and M ext (Gerstein & Mandelbrot, 1964; Tuckwell, 1988). Furthermore, by numerically solving the master equation that governs the density distribution of the state variables, it is possible to predict the behavior of each neuron under such a noisy stimulating current, during those time intervals when no synaptic input (relevant event) from the internal simulated neurons occurs. The same approach can be applied to the case in which the state variables are inherently stochastic because of some source of noise related to the single-neuron dynamics. Here we focus on an example in which the dynamics are deterministic and the simulated neurons are embedded in a very large network. If the

814

J. Reutimann, M. Giugliano, and S. Fusi

(excitatory or inhibitory) postsynaptic potentials evoked by a single presynaptic spike are small compared to the threshold for triggering postsynaptic action potentials (i.e., diffusion approximation), the density equation can be reduced to a simpler Fokker-Planck equation, which in many cases can be solved analytically (Ricciardi, 1977). These theoretical approaches have already proved to constitute a powerful tool for studying network properties in stationary conditions, such as in studies on spontaneous activity, on the persistent stimulus-selective delay activity observed in the cortex (Amit & Brunel, 1997b; Yakovlev, Fusi, Berman, & Zohary, 1998; Wang, 1999; Fusi & Mattia, 1999; Brunel, 2000), or to study the network response properties to particular transient or oscillatory inputs (for a review, see Knight, 1972; Nykamp & Tranchina, 2000; Gerstner & Kistler, 2002). By combining these strategies with an event-driven simulation approach, we show in this article how to implement the global background activity, effectively reducing its computational complexity to O .N ¢ ºext /. 2 The Simulation Algorithm 2.1 The General Idea. In the event-driven approach described in Mattia and Del Giudice (2000), the dynamics of the neurons and the synapses were updated only on the arrival of a spike. Therefore, the action potential was the relevant event to trigger an update of the state variables. The new state was then computed on the basis of two elements: (1) the initial state, following the previous update, and (2) the time passed since the previous update. Because the dynamics of both synapses and neurons was deterministic in this interval, the Žnal state could be determined for an arbitrary time interval by the solution of the differential equations governing the dynamics of the neurons. In our case, the development of the neuronal state variables in the interspike interval is not deterministic. In particular, the external spikes are not explicitly treated as synaptic events. Instead, we assume that the external activity is highly irregular and the total synaptic current coming from outside can be described as a stochastic process with a known distribution. In a traditional Žxed time-step simulation, at each step the external current should be randomly generated, independently for each neuron, and the state variables affected by it should be updated. Therefore, the dynamics of the state variables would be nondeterministic, and the Žnal state could not be determined by the initial condition and the time since the previous update. However, given the distribution of the external current, it is possible to determine the distribution of the state variables as a function of time. It is as if each state variable starts from some determined initial value at the preceding update time (when the distribution is a delta function) and then evolves along all the possible trajectories determined by the different realizations of the input currents. This packet of trajectories would evolve until another relevant event occurs. At this

Event-Driven Simulation of Spiking Neurons

815

time, the packet should be reduced to a single trajectory, which is the one that has actually been followed by the neuron. The advantage is that it is not necessary to integrate iteratively, and presumably many times, the state variables during the interval between two relevant events. Deciding at the end what was the actual trajectory is completely equivalent to what would be obtained in a Žxed time-step approach, provided that the trajectory is chosen according to the correct distribution of the state variables. 2.2 An Example: Networks of Integrate-and-Fire Neurons. The approach for the proposed new event-driven algorithm is quite general and can be applied to a large class of neuronal models and synaptic interactions (see also Mattia & Del Giudice, 2000; Giugliano, 2000). In order to illustrate the algorithm, we will focus on a network of randomly connected integrate-and-Žre (IF) model neurons, whose state is fully determined by a single internal variable, the membrane potential V. In section 4, we show how to generalize this approach to more complex models. Formally, the differential equation that governs the dynamics of the membrane voltage below threshold for a generic neuron can be written as follows, dV D ¡L.V; t/ C Iint .t/ C Inoise .t/ dt

Vmin · V < V# ;

(2.1)

where L.V; t/ is a generic leakage term. We will show two examples: (1) the case in which L.V; t/ D .Vrest ¡ V/=¿m , where Vrest is the resting potential and ¿m is the passive membrane time constant, and (2) the case in which the leak is constant: L.V; t/ D ¸. The two models respond in the same way to noisy currents provided that the dynamics of the neuron with the constant leakage are complemented by the condition that the membrane voltage is limited from below by a rigid barrier at Vrest . Interestingly, the stationary response function of these two model neurons to noisy currents can be Žtted to the response of cortical pyramidal cells measured in in vitro experiments (Rauch, La Camera, Luscher, ¨ Senn, & Fusi, 2002). Iint .t/ represents the net charging current generated by synaptic inputs from other neurons within the simulated network, and Inoise .t/ is the noisy current that in our case represents the external background activity, coming from the feedforward neurons. We assume that each internal neuron receives the external input from a different group of M ext external neurons. If V reaches a threshold V# , the neuron emits a spike, and V is reset to a value Vreset < V# , where it remains during an absolute refractory period ¿arp . A rigid (reecting) barrier Vmin was included to restrict the membrane potential to the interval Vmin · V.t/ < V# . Although unusual compared to the IF dynamics described in the literature, such a choice is reasonable, as the physiological range for V in a real neuron is bounded by the ionic concentration ratios across the membrane.

816

J. Reutimann, M. Giugliano, and S. Fusi

If the spikes are point processes (i.e., their time width is zero), Iint .t/ can be written as a sum of Dirac deltas, X Iint .t/ D Jk ±.t ¡ tk /; k

where the sum extends over all the afferent spikes. The efŽcacy of each spike is given by Jk . In the interval between two successive synaptic inputs, Iint .t/ is 0, and V follows a random walk induced by the noisy input Inoise .t/. If the external spikes are statistically independent, the total noisy current impinging on a generic neuron of the network can be well approximated by a Gauss-distributed, delta-correlated stochastic process characterized by a mean ¹ and a variance ¾ 2 per unit time (Amit & Tsodyks, 1991; Gerstein & Mandelbrot, 1964; Tuckwell, 1988). ¹ and ¾ 2 are explicitly related to the mean Žring rate ºext of the external population, its probability of feedforward connectivity cext, its size Mext , and the average synaptic strength Jext , according to the following equations: ¹ D ºext ¢ cext ¢ Mext ¢ Jext

¾ 2 D ºext ¢ cext ¢ Mext ¢ Jext 2 :

(2.2)

As a consequence, the temporal evolution of the membrane potential V in the intervals between two consecutive internal spikes can be regarded as a stochastic process whose statistical properties are completely described by PV .V; t/, representing the probability density of having V.t/ D V (Ricciardi, 1977; Tuckwell, 1988). Given that the neuron starts from V.t0 / D V0 at time t0 (the time of the last relevant event), PV .V; t/ describes the packet of depolarization trajectories generated by the external input when the neuron had a particular initial value V0 . Before the arrival of the next spike emitted by one of the network neurons, it might happen that the external synaptic current drives the voltage across the threshold V# . These crossings occur at any time t at a rate P fpt .t0 ; V0 I t/, which expresses the probability density of having a Žrst passage time at t, given the initial state V D V0 at time t0 . These probabilities can be computed numerically by solving the diffusion equations that govern PV .V; t/ (see appendix A). A solution is shown in Figure 1, where the temporal evolution of the probability density for the V and the corresponding interspike interval distribution are plotted. Note that there is an overall decrease of the distribution in time, because the fraction of neurons that cross the Žring threshold are eliminated (we are interested in the Žrst passage time only). As apparent from the inspection of the interspike interval distribution, there is at any time a nonzero probability of crossing the Žring threshold at V# D ¡50 mV. 2.3 The Extended Event-Driven Algorithm. Unlike in the fully deterministic case, where the state of a neuron is inuenced only by events having taken place before the current time, we now have to consider the possibility of having uncertain events—predictions about the neuron’s future that may or may not happen. By drawing a random number according to the

Event-Driven Simulation of Spiking Neurons

817

Figure 1: Numerical solution of the Fokker-Planck equation for the linear IF neuron (see appendix A): the temporal evolution of the probability density for the V (surface plot) and the interspike interval distribution (thick line) are shown. Neuron parameters: ¿m D 20 ms, Vrest D ¡70 mV, V# D ¡50 mV, Vmin D ¡80 mV, ¿arp D 2 ms, V0 D ¡70 mV, ¹¿m D 14:5 mV, and ¾ 2 ¿m D 10:2 mV2 .

probability distribution given by P fpt , we obtain the predicted time (T1 ) of a Žrst threshold crossing. However, if the state of the neuron is inuenced in any way before T1 , a new prediction has to be made based on the current state, and the old prediction becomes irrelevant. To describe in some detail our algorithm, we consider a generic neuron, whose initial condition is V.t0 / D V0 (see Figure 2), embedded in a network. First, we draw a pseudorandom number according to the distribution given by P fpt .t0 ; V0 I T1 /, which yields time T1 as the predicted time of the next threshold crossing according to our neuron model (see Figure 2, A1). Two possible situations might now occur, and they can be outlined as follows: 1. No presynaptic (internal) spike arrives in the time interval [t0 ; T1 ] from other neurons within the network. At time t D T1 , the neuron emits a spike (as determined in the Žrst step) and enters the absolute

818

J. Reutimann, M. Giugliano, and S. Fusi Initial Event

A1

P (t ,V ; T ) fpt 0

VJ

0

Next Event

A2

1

VJ

P (T +t fpt

1

arp

,V

;T)

reset

2

V

0

V t

T

0

reset

T +t

1

1

B1

T

arp

2

B2 Pfpt(t0,V0; T1)

VJ

V

V

0

1

t

0

sp 0 1

DV

P (t ,V ,t ; V ) v 0

t

VJ

sp 1

V +DV 1

sp fpt 1

P (t ,V +DV ; T )

1

1

1

2

1

1

sp 1

T

t

1

C1

T

2

C2 Pfpt(t0,V0; T1)

VJ

VJ

DV V

V

0

1

0

arp

,V

;T)

reset

2

1

V t

sp fpt 1

P (t +t

sp t 1

T

1

reset sp 1

t +t

arp

T

2

Figure 2: An illustration of the extended algorithm (see section 2.3). Column 1: Initial event in three different situations. Column 2: Corresponding next event. Initial voltage is V.t0 / D V0 for all cases (A1, B1, C1). (A1) First-passage-time T1 is determined randomly according to P fpt .t0 ; V0 /. As no further event occurs, a spike is emitted at time T1 . (A2) After the absolute refractory period ¿arp , the voltage is set to Vreset , and the next Žrst-passage-time T2 is determined according to P fpt .T1 C¿arp ; Vreset I T2 /. (B1) After T1 has been determined (as in A1), a synaptic sp sp sp event occurs at time t1 .t0 < t1 < T1 /. The current membrane voltage V1 D V.t1 / sp is chosen randomly according to PV .t0 ; V0 ; t1 / and subsequently updated by the synaptic contribution 1V1 . T1 is now obsolete. (B2) Since the membrane voltage remains below threshold, the next Žrst-passage-time T2 is drawn according to sp P fpt .t1 ; V1 C 1V1 I T2 /. (C1) Same as B1, but 1V1 is large enough to cause a threshold crossing and emission of a spike. (C2) After the refractory period, the sp next Žrst-passage-time is determined randomly according to P fpt .t1 C ¿arp ; Vreset /.

Event-Driven Simulation of Spiking Neurons

819

refractory period, after which the neuron continues at the Žrst step using V0new D Vreset and tnew D T1 C ¿arp (see Figure 2, A2). The spike 0 emission leads to the creation of further events, which represent the synaptic interactions that will take place in response to this spike after some axonal delay period. sp

sp

2. A single internal spike arrives at time t1 .t0 < t1 < T1 /. At this point, sp the membrane voltage V1 is drawn from PV .t0 ; V0 I V; t1 / and subsequently updated according to the synaptic strength 1V1 D J associated with the incoming spike. Again, two scenarios are possible, depending on whether the new membrane potential .V1 C 1V1 / is (a) below threshold, in which case the neuron continues at step 1 ussp ing V0new D V1 C 1V1 and tnew D t1 (see Figure 2B), or (b) equal to 0 or above threshold V# , in which case the neuron emits a spike and the membrane voltage is reset to Vreset during the absolute refractory period, after which the neuron continues at step 1 using V0new D Vreset sp and tnew D t1 C ¿arp (see Figure 2C). As before, the spike emission 0 results in the generation of new events to be processed later in the simulation. This is repeated during all the simulation time, or until no more events are to be processed. A critical situation occurs when the statistics of the background activity change, because this also affects the probability distributions. In order to propagate the change to the whole network, all neurons must immediately be visited, and the current state variables must be updated according to PV , using the old statistics. Then, using P fptwith the new statistics, spontaneous spike times are determined as usual, and the simulation continues as described above. It is clear from this procedure that the statistics of the background activity can change in time only by discrete steps rather than in a continuous way. This is an inherent limitation of the event-driven approach. However, most physiological situations can be adequately described by piecewise constant (discontinuous) changes in the statistical properties of the external background populations. Examples of such changes are given in Figures 3B and 4. For an extension to more complex transients or periodical noisy inputs, see section 4. 2.4 Implementation of the Event List. As a consequence of the eventbased simulation approach, a large number of events have to be processed in chronological order. The strategies involved in handling the events have to be chosen with care, given that a lack of efŽciency could lower the performance of the algorithm considerably. The problem reduces to having a single ordered list with random insertions (when creating new events), repetitive deletion of the lowest element

820

J. Reutimann, M. Giugliano, and S. Fusi

(when the next event is processed), and occasional random deletions (when an uncertain event becomes irrelevant). An appropriate data structure for the storage of the events is given by the class of balanced trees, which ensure good performance: searching for one element out of E costs O .log.E// operations. However, maintaining the balance of the tree causes considerable overhead, especially in our situation where repetitive deletions occur at one place (Wirth, 1986). In our implementation, we used the “skip list” data structure, which is a probabilistic alternative to balanced trees (Pugh, 1990; see also appendix B), based on linked lists. In this structure, maintenance of the balance comes as a consequence of the probabilistic nature of the structure, and the insertion and deletion algorithms are as simple as those for linked lists. Therefore, skip lists strongly reduce the overhead of the event handling, while offering the performance beneŽts of balanced trees. 2.5 Look-up Tables. In the general case, the processing of an event requires a partial differential equation (e.g., equation A.1) to be solved numerically, which is time-consuming. Although this computational load does not directly depend on the assumed size of the external populations (Mext ), and therefore does not affect the run-time performance scaling of our algorithm with respect to Mext , it nevertheless reduces the overall performance of the simulation in a dramatic way. However, we can make use of the fact that the parameters governing the evolution of the probability distributions can change only in a discontinuous manner, and prepare tables of all needed distributions off-line. These tables can be stored and used during the simulation, therefore reducing this computationally delicate task to simple table look-ups. For a speciŽc example, refer again to Figure 1, where such an off-line evaluated solution has been graphically represented: the time-varying probability density distribution PV and the corresponding interspike interval distribution density P fpt are plotted for a particular V0 D ¡70 mV. The density distribution represented by the surface starts as a Dirac-delta located at V D V0 , and as time goes by with no internal event occurring, it drifts toward the asymptotic value (¹¿m D 14:5 mV) and diffuses at a rate ¾ (with ¾ 2 ¿m D 10:2 mV2 ). Accordingly, the Žrst-passage time probability density distribution P fpt Žrst increases and then slowly decreases to zero for large time intervals. Of course, different initial conditions V0 yield different proŽles of PV and P fpt . 3 Performance Evaluation The proposed algorithm provides fast simulations of large networks. In order to check the simulation results for correctness, the emergent behaviors were compared to the mean-Želd theory predictions.

Event-Driven Simulation of Spiking Neurons

821

3.1 Numerical Accuracy. The introduction of tabular discretizations will introduce numerical approximations that inuence the accuracy of the simulations. The fact that a higher table resolution produces more accurate results leads to a trade-off between accuracy and table resolution (which directly affects memory consumption). This is illustrated in Figure 3: the different lines in Figure 3B correspond to different table resolutions. The mean frequency response to noisy external inputs with stationary statistics is compared to the one predicted analytically by the mean Želd theory (see Amit & Brunel, 1997a). The deviations from the predicted frequency in Figure 3B (23.6 Hz in the high noise regime) are plotted in Figure 3C as a function of table resolution. Of the resolutions tested in Figure 3C, the best results were obtained with 0:1 mV and 0:5 ms in the voltage and time domains, respectively, and the number of elements contained in the corresponding table was of the order of 106 . Figure 3D demonstrates that the accuracy is very sensitive to the discretization of the voltage and less sensitive to the time resolution. Minor discrepancies in the global Žring rate were expected, since an additional reecting lower barrier for the depolarization of the single neuron was set at V D Vmin , slightly altering the IF dynamics compared to those employed in the theory (see equation A.1). The achieved accuracy was nevertheless satisfying, and the typical transient overshoots in the collective mean Žring rate, expected after a sudden increase in the statistics of the background activity, were captured. We also compared directly the simulations and the theoretical prediction of the transient response of a population of coupled neurons to a stepwise change in the statistics of the external current (see Figure 4). In this particular case, we were using the IF neuron with a constant leakage because the analytical solution of the network dynamics during transients is simpler (Mattia & Del Giudice, 2002). Note that changing the neuron model affects only the generation of the look-up tables, as all the information about the neural dynamics is contained within these tables. Figure 4 shows simulations of a network of 5000 interacting, inhibitory neurons. At time t D 0 (2 seconds after the start of the simulation), the statistics of the external current are changed from (¹ D 125 mV/s, ¾ D 5:15 mV/s) to (¹ D 700 mV/s, ¾ D 12:20 mV/s). The transient response of the simulation follows very closely the theoretical prediction reported in Mattia and Del Giudice (2002). In order to maximize accuracy and minimize memory usage, nonuniform discretizations and efŽcient interpolation methods can be used. In particular, voltages near the threshold and short time intervals should be represented with a reŽned discretization. In this last example, we obtain the exact steady-state values of the population frequency, which reects the fact that we were using nonuniform, high-resolution discretizations of the underlying tables (minimal resolution of 0:1 mV at the reecting barrier Vmin , and maximal resolution of 0:01 mV near the threshold V# ).

822

J. Reutimann, M. Giugliano, and S. Fusi

Figure 3: Simulation of a population of IF neurons for two different regimes of the external input (¹1 ¿m D 14:5 mV, ¾12 ¿m D 10:2 mV2 , ¹2 ¿m D 16:5 mV, ¾22 ¿m D 27 mV2 ). (A) Raster plot of 30 of 500 neurons and the corresponding instantaneous network Žring rate (B, average over 100 runs) compared to the analytical prediction (dotted line) by the mean-Želd theory (see text; Amit & Tsodyks, 1991; Brunel, 2000; Fusi & Mattia, 1999). The three lines correspond to three different look-up table resolutions that were used. (C) Estimate of the inaccuracies induced by the discretizations of the look-up tables, measured in the last 200 ms of B. (D) Sensitivity of the inaccuracies to changes in table resolution (v: change in voltage axis [0:4 mV £ 0.5 ms, 0:2 mV £ 0.5 ms, 0:1 mV £ 0.5 ms; from right to left], t: change in time axis [0:1 mV £ 2 ms, 0:1 mV £ 1 ms, 0:1 mV £ 0.5 ms]).

Event-Driven Simulation of Spiking Neurons

823

25

n(t) [Hz]

20

15

10

5

0

0

100

200

300 time [ms]

400

500

600

Figure 4: Transient response to a step change in the statistics of the external current: simulation versus theory. For t < 0, the network is in an asynchronous stationary state with mean emission rate º D 0:2 Hz. At t D 0, an instantaneous increase of the external current from (¹ D 125 mV=s, ¾ D 5:15 mV=s) to (¹ D 700 mV=s, ¾ D 12:20 mV=s) drives the activity toward a new stable state with º D 20 Hz. The solid black line is the mean of the activity from 10 simulations of a coupled network (5000 inhibitory LIF neurons; see the text). The thick gray line is the theoretical prediction. (Used with permission from Mattia & Del Giudice, 2002).

3.2 Run-Time Performance. All the simulations were performed on a Pentium III 850 MHz system running Linux, and they conŽrmed that excellent run-time performance can be achieved if the computation of the probabilistic dynamics (i.e., the on-line probability tables search and look-up, and the consequent random number extractions) is fast enough. Figure 5A reports a benchmark comparing our algorithm with the event-driven approach of Mattia and Del Giudice (2000). Both simulations were performed over a simulated time of 1 second with a network of 1000 IF neurons, characterized by a weak average internal synaptic coupling (J D 0:1 mV) and by a low connectivity probability (c D 0:1). With the aim of accounting for differences in the output frequency, the execution times of both simulation approaches were normalized and plotted as a function of the mean number of spikes received by each neuron from the external neurons (background activity, Fext ) divided by the number of spikes received by the other simu-

824

J. Reutimann, M. Giugliano, and S. Fusi

execution time vs. ratio of external/internal spikes

A

extended algorithm Mattia & Del Giudice approach

execution time / n [s/Hz]

100

10

1

0.1

0.1

1

F

10 /F

100

1000

ext

100

10 execution time [s]

execution time [s]

B vs. number of neurons

1

0.01 100

1000 N

10000

C

vs. output frequency

8 6 4 2 0

0

20

40 n [Hz]

60

80

Figure 5: Execution performance of the proposed algorithm. (A) Comparison of the execution times of the state-of-the-art event-driven approach (as described in Mattia & Del Giudice, 2000, circles) and of the extended event-driven approach (crosses), as a function of the ratio between the number of the external and internal spikes (see section 3). The execution time is given per simulated second and averaged over Žve runs of 4 s simulation time each. (B) The execution time scales with the square of the number of simulated neurons (c D 0:05, mean output frequency 4 Hz). (C) Execution time scales almost linearly with output frequency (c D 0:05, J D 0:1 mV).

Event-Driven Simulation of Spiking Neurons

825

lated neurons (internal activity, F): Fext Mext ¢ cext ¢ ºext D ; F N ¢c¢º where c and cext are set to 0:1 and 1, respectively, ºext is calculated from the statistics of the background populations, and º is the fraction of simulated neurons that emit a spike per unit time. The ratio Mext =N was varied systematically (0.1, 1, 10, 100, 1000). The Žgure demonstrates that the execution time of the extended event-driven approach scales very well (does not increase) with respect to the increase in the size of the background populations. Only in the case of very few external spikes per unit time does the approach of Mattia and Del Giudice perform better than ours, which can be explained by the lower time required to process a single event: only one random number has to be generated compared to at least two in the extended algorithm (one for PV and P fpt each and for each newly inserted event in the event list). The execution time is also affected by the size N of the network to be simulated, scaling with the square of N (see Figure 5B), and for a given network size, it increases almost linearly with the output frequency, as expected from the analysis of the traditional event-driven simulation paradigm. As demonstrated by the simulation results, the main strength of the proposed approach is therefore related to the fact that the external populations were not modeled explicitly, resulting in an execution time independent of the number of external neurons (Mext) and of the ratio of the external to internal spikes: O .N ¢ ºext /. As a consequence, Mext enters the simulation only as a parameter in the expressions of the mean ¹ and the variance ¾ 2 of the external current (see equation 2.2). 4 Discussion The considerable speed-up of simulation times makes our approach suitable for the investigation of the emergent collective behaviors in large-scale networks of neurons and extends the perspectives of the event-driven simulation approach. In our tests, a simulation of 1000 neurons with a connectivity probability of 0.1 was running in real time (i.e., 1 s of the simulated network was executed in 1 s, while the mean frequency was 4 Hz), no matter how large the external population was. 1 It is important to stress that our approach does not imply any approximation provided that the number of independent external afferents to be mimicked is large enough and that the solution of the density equation is accurate. Our simulations produce the same results as any traditional Žxed-time-step approach, in which the external afferents are statistically independent. In this, our strategy differs from 1 The software implementing the described algorithm can be obtained from the authors upon request.

826

J. Reutimann, M. Giugliano, and S. Fusi

that of Knight (1972), who suggested studying the whole network dynamics by solving the density equations that govern the assembly of neurons. His strategy strongly relies on the assumption that the large number of neurons in the simulated network lose their identity and, hence, that the network state can be fully described by the probability density function of the state variables. In many cases, this represents a good approximation, although Žnite-size effects should be added artiŽcially to the dynamic equations of the probability density function, to account for the Žnite size of the network under consideration (see Brunel & Hakim, 1999; Brunel, 2000; Mattia & Del Giudice, 2002). Moreover, the weak statistical correlations introduced by the synaptic couplings, which cannot be modeled in a density approach, must be neglected. In our case, all of these elements emerge naturally from the simulation, which still constitutes a useful tool to check the hypotheses of the density approach. Our simulation strategy can be easily extended to events that are not well localized in time. For instance, when the synaptic currents generated by the arrival of presynaptic spikes are not delta correlated, the depolarization V cannot be updated instantaneously. A persistent current would affect the state variable of the neuron throughout the duration of the synaptic input. If the synaptic current goes instantaneously up upon the arrival of a spike and then decays exponentially with a time constant ¿ , we have that (see also Srinivasan & Chiel, 1993; Destexhe, Mainen, & Sejnowski, 1994; Lytton, 1996; Giugliano, 2000; Nykamp & Tranchina, 2001) X X Iint .t/ D Jk e¡.t¡tk /=¿ D e¡t=¿ Jk etk =¿ D e¡t=¿ Isp ; k

k

where Iint , the total synaptic current, is updated when a spike arrives (a term is added to the sum Isp ) and decays exponentially during the interspike intervals. Such a dynamic behavior can be introduced in our approach by deŽning an effective external mean current ¹ef f .t/: ¹ef f .t/ D ¹.t/ C e¡t=¿ Isp : The density equations should then be solved for every Isp . This makes the look-up tables for the probability density functions more complicated (it adds a dimension corresponding to Isp ), but it provides the possibility of having more realistic currents. Another interesting extension to our model can deal with situations in which we have a time-varying periodic change in the background statistics (i.e., an oscillatory, noisy input current) or a transient with a complex temporal proŽle that is limited in time. Such a situation can be simulated without changing continuously the statistics of the input current, which would require a frequent update of all the state variables of the neurons even in the absence of incoming spikes (see section 2.3). The solution is to add a time-dependent state variable describing the neuron’s position within

Event-Driven Simulation of Spiking Neurons

827

the phase #0 of the period. At the update time, the neuronal state variable would be updated according to the appropriate look-up table, that is, the one corresponding to the neuronal dynamics under the time-varying stimulus starting at the update time. As an example, P fpt .t0 ; V0 I t/ would then become P fpt .t0 ; V0 ; µ0 I t/, and P V .t0 ; V0 I V; t/ would have to be extended to PV .t0 ; V0 ; µ0 I V; t/. The extra cost of this approach is an added dimension in the look-up tables whose extension depends on the time correlation length of the input signal, the length of the temporal proŽle (the period in the case of a cyclic input), and the time resolution. However, the simulation performance in terms of execution time is not impaired. In general, simulations with more complex model neurons or synapses also require the addition of other dimensions in the probability density function. Moreover, solving high-dimensional partial differential equations is difŽcult, and the look-up tables would require a huge amount of memory. This is a major limitation of our approach because the problem becomes intractable even with few state variables. Nevertheless, our strategy (as most of the event-driven approaches) is efŽcient when the problem and the model are already well deŽned, the relevant internal variables are known, and extensive simulations are needed to explore the parameter space. Appendix A: Fokker-Planck Equation The probability density function PV .V; t/ that describes the statistics of the dynamics during the intervals between two successive spikes evolves in the voltage-time domain according to the Fokker-Planck equation (Cox & Miller, 1965), @ @ PV .V; t/ D ¡ Á.V; t/; @t @V

(A.1)

2

@ where Á .V; t/ D ¡ ¾2 ¢ @V p.V; t/ C [.¹ ¡ L.V; t// ¢ p.V; t/]. In the most general case, equation A.1 must be complemented by the initial condition P V .V; t0 / D ±.V ¡ V0 / and by appropriate boundary conditions restricting, in the case of the leaky IF neuron, the potential

² at V D Vmin , in terms of a reecting barrier, since no neuron can have a depolarization that passes below Vmin 8 t; Á.Vmin ; t/ D 0

(A.2)

² at V D V# , in terms of an absorbing barrier, since neurons that are crossing such a threshold are absorbed and leave the interval (Vmin I V# ): 8 t; PV .V# ; t/ D 0:

(A.3)

828

J. Reutimann, M. Giugliano, and S. Fusi

Finally, at any time t, the probability density PV .V; t/ must satisfy a normalization condition, Z

V# Vmin

Z p.V; t/ dV C

t t0

º.t0 / dt0 D 1

(A.4) 2

@ where º.t/ D Á.V# ; t/ D P fpt .t/ D ¡ ¾2 ¢ @V PV .V; t/jV# , represents the rate at which neurons are crossing the threshold V# . We note that equation A.4 accounts for the deating time course of the surface PV .V; t/, sketched in Figure 1: with time, the realizations of the process V.t/ leave the integration interval (Vmin ; V# ) after absorption at the threshold, and they irreversibly R accumulate in the term tt0 º.t0 / dt0 . By preliminarily choosing a Žxed set of values for (¹; ¾ ) and discretizing the interval (Vmin I V# ) into a Žnite number of bins, the numerical integration of equation A.1 determines the desired time-dependent probability density distributions PV .¹; ¾ 2 ; t0 ; V0 ; t1 I V1 / and P fpt .¹; ¾ 2 ; t0 ; V0 I t/, as a function of (¹; ¾ ) and given that V.t0 / D V0 .

Appendix B: Skip List Skip lists are a data structure introduced by W. Pugh. Skip lists can be used in place of balanced trees (Wirth, 1986), and they use probabilistic balancing rather than strictly enforced balancing. The underlying data structure is an ordered linked list of length n. When searching a linked list, we may have to visit all the nodes of the list. If every second node also has a pointer to the node two ahead it in the list, we have to examine no more than dn=2e C 1 nodes, since we can skip over half the nodes and go back only one node in case we had skipped over the element we were looking for. If every .2i /th node has a pointer 2i nodes ahead, the number of nodes that must be examined can be reduced to dlog2 ne while only doubling the number of pointers. A node that has k forward pointers is called a level k node. In a skip list, the levels of nodes are chosen randomly; the probabilities of choosing a certain level follow a geometric series (50% are level 1, 25% level 2, 12.5% level 3, and so on). A node’s ith forward pointer, instead of pointing 2i¡1 nodes ahead, points to the next node of level i or higher. Thus, insertions and deletions require only local modiŽcations; the level of a node, chosen randomly when the node is inserted, never has to be changed. In summary, the data structure is a linked list with extra pointers that skip over intermediate nodes—hence the name. Skip lists are balanced by consulting a random number generator and do not require rearranging of the structure, as is the case for balanced trees. Furthermore, the simplicity of skip list algorithms (only local modiŽcations) makes them easier to implement and provides signiŽcant constant factor speed improvements over

Event-Driven Simulation of Spiking Neurons

829

balanced and self-adjusting tree algorithms. A more detailed analysis of the algorithmic performance can be found in Pugh (1990). Acknowledgments We thank Maurizio Mattia for providing his software and assistance and for very useful discussions and remarks on the manuscript. This work was supported by SNF grants 21-57076.99 and 31-61335.00 and by the Human Frontier Science Foundation (LT00561=2001-B). References Amit, D. J., & Brunel, N. (1997a). Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex. Cereb. Cortex, 7(3), 237–252. Amit, D. J., & Brunel, N. (1997b). Dynamics of a recurrent network of spiking neurons before and following learning. Network: Computation in Neural Systems, 8, 373–404. Amit, D. J., & Tsodyks, M. V. (1991). Quantitative study of attractor neural networks retrieving at low spike rates I: Substrate-spikes, rates and neuronal gain. Network, 2(3), 259–274. Brunel, N. (2000). Dynamics of sparsely connected networks of excitatory and inhibitory spiking neurons. J. Comp. Neurosci., 8, 183–208. Brunel, N., & Hakim, V. (1999). Fast global oscillations in networks of integrateand-Žre neurons with low Žring rates. Neural Comp., 11(7), 1621–1671. Cox, D. R., & Miller, H. D. (1965). The theory of stochastic processes. London: Methuen. Delorme, A., Gautrais, J., van Rullen, R., & Thorpe, S. (1999). SpikeNET: A simulator for modeling large networks of integrate and Žre neurons. Neurocomputing, 26=27, 989–996. Destexhe, A., Mainen, Z. F., & Sejnowski, T. J. (1994). An efŽcient method for computing synaptic conductances based on a kinetic model of receptor binding. Neural Comp., 6, 14–18. Fusi, S., & Mattia, M. (1999). Collective behavior of networks with linear (VLSI) integrate and Žre neurons. Neural Computation, 11, 633–652. Gerstein, G. L., & Mandelbrot, B. (1964). Random walk models for the spike activity of a single neuron. Biophys. J., 4, 41–68. Gerstner, W., & Kistler, W. (2002). Spiking neuron models: Single neurons, populations, plasticity. Cambridge: Cambridge University Press. Giugliano, M. (2000). Synthesis of generalized algorithms for the fast computation of synaptic conductances with Markov kinetic models in large network simulations. Neural Comput., 12(4), 771–799. Knight, B. W. (1972). Dynamics of encoding in a population of neurons. Journal of General Physiology, 59, 734–766. Lytton, W. W. (1996). Optimizing synaptic conductance calculation for network simulations. Neural Comp., 8, 501–509.

830

J. Reutimann, M. Giugliano, and S. Fusi

Mattia, M., & Del Giudice, P. (2000). EfŽcient event-driven simulation of large networks of spiking neurons and dynamical synapses. Neural Comput., 12(10), 2305–2329. Mattia, M., & Del Giudice, P. (2002). Population dynamics of interacting spiking neurons. Phys. Rev. E 66(5), 051917. Nykamp, D. Q., & Tranchina, D. (2000).A population density approach that facilitates large-scale modeling of neural networks: Analysis and an application to orientation tuning. J. Comp. Neurosci., 8, 19–50. Nykamp, D. Q., & Tranchina, D. (2001).A population density approach that facilitates large-scale modeling of neural networks: Extension to slow inhibitory synapses. Neural Comp., 13, 511–546. Press, W. H., Flannery, B. P., Teukolsky, S. A., & Vetterling, W. T. (1986).Numerical recipes:The art of scientiŽc computing. Cambridge: Cambridge University Press. Pugh, W. (1990). Skip lists: A probabilistic alternative to balanced trees. Communications of the ACM, 33(6), 668–676. Rauch, A., La Camera, G., Luscher, ¨ H.-R., Senn, W., & Fusi, S. (2002). Neocortical pyramidal cells respond as integrate-and-Žre neurons to in vivo-like input currents. Manuscript submitted for publication. Ricciardi, L. M. (1977). Diffusion processes and related topics in biology. Berlin: Springer-Verlag. Srinivasan, R., & Chiel, H. J. (1993). Fast calculation of synaptic conductances. Neural Comp., 5, 200–204. Tuckwell, H. C. (1988). Introduction to theoretical neurobiology, Vol. 2: Non linear and stochastic theories. Cambridge: Cambridge University Press. Wang, X.-J. (1999). Synaptic basis of cortical persistent activity: The importance of NMDA receptors to working memory. J. Neurosci., 19(21), 9587–9603. Watts, L. (1994). Event-driven simulation of networks of spiking neurons. In J. D. Cowan, G. Tesauro, & J. Alspector (Eds.), Advances in neural information processing systems, 6 (pp. 927–934). San Mateo, CA: Morgan Kaufmann. Wirth, N. (1986). Algorithms and data structures. Upper Saddle River, NJ: Prentice Hall. Yakovlev, V., Fusi, S., Berman, E., & Zohary, E. (1998). Inter-trial neuronal activity in infero-temporal cortex: A putative vehicle to generate long term associations. Nature Neuroscience, 1(4), 310–317. Received May 3, 2002; accepted October 1, 2002.

Communicated by Rajesh P. N. Rao

LETTER

Isotropic Sequence Order Learning Bernd Porr [email protected]

Florentin Worg ¨ otter ¨ [email protected] Department of Psychology, University of Stirling, Stirling FK9 4LA, Scotland

In this article, we present an isotropic unsupervised algorithm for temporal sequence learning. No special reward signal is used such that all inputs are completely isotropic. All input signals are bandpass Žltered before converging onto a linear output neuron. All synaptic weights change according to the correlation of bandpass-Žltered inputs with the derivative of the output. We investigate the algorithm in an open- and a closed-loop condition, the latter being deŽned by embedding the learning system into a behavioral feedback loop. In the open-loop condition, we Žnd that the linear structure of the algorithm allows analytically calculating the shape of the weight change, which is strictly heterosynaptic and follows the shape of the weight change curves found in spike-time-dependent plasticity. Furthermore, we show that synaptic weights stabilize automatically when no more temporal differences exist between the inputs without additional normalizing measures. In the second part of this study, the algorithm is is placed in an environment that leads to closed sensormotor loop. To this end, a robot is programmed with a prewired retraction reex reaction in response to collisions. Through isotropic sequence order (ISO) learning, the robot achieves collision avoidance by learning the correlation between his early range-Žnder signals and the later occurring collision signal. Synaptic weights stabilize at the end of learning as theoretically predicted. Finally, we discuss the relation of ISO learning with other drive reinforcement models and with the commonly used temporal difference learning algorithm. This study is followed up by a mathematical analysis of the closed-loop situation in the companion article in this issue, “ISO Learning Approximates a Solution to the Inverse-Controller Problem in an Unsupervised Behavioral Paradigm” (pp. 865–884). 1 Introduction A central goal of every autonomous agent is to maintain homoeostasis (Ashby, 1956), without which it will eventually disintegrate (“die”). A generic way to achieve this is by reacting to a disturbance of the homoeostasis with a closed-loop negative feedback mechanism (a reex), which will Neural Computation 15, 831–864 (2003)

c 2003 Massachusetts Institute of Technology °

832

B. Porr and F. Worg ¨ otter ¨

compensate for the disturbance by means of a (motor) reaction. Thus, the simplest form of sensible autonomous behavior can be obtained by designing an agent whose (re-)actions are reex based (Brooks, 1989). This type of behavior is found even in rather primitive animals like amoebas, which retract their philopodia when encountering a potentially damaging chemical gradient. Such sensor-motor reex loops represent typical feedback reaction systems, because a reex will always be elicited only after a sensor event has already been encountered, as the word feedback implies. The reaction delay, which is unavoidably associated with every reex loop, can even lead to fatal situations in the worst case. Thus, in any kind of improved behavior, the acting agent will try to avoid reexes, for example, by predicting one sensor event from another earlier occurring event (at a different sensor). This takes place when predicting pain from the heat that radiates from a hot surface in order to prevent a retraction reex by means of an anticipatory avoidance reaction. In this example, heat radiation and pain are causally related. Many other similar causal relations exist during the life of an animal, for example, between smell and taste when foraging or between vision and touch when exploring. In all of these cases, a temporal sequence of sensor events occurs, which needs to be learned in order to avoid reex reactions to the later event. Thus, temporal sequence learning is a dominant aspect of animal behavior. It requires a late event, which serves as a reference to which the earlier event temporally relates. The goal is to learn this speciŽc temporal relation and turn reactive into proactive behavior. In artiŽcial systems, temporal sequence learning can be achieved, for example, by classical Hebbian learning (Hebb, 1949) in combination with delays (Levy & Minai, 1993), by differential Hebbian learning (Kosco, 1986; Klopf, 1986), or by the very inuential temporal difference (TD) reinforcement learning algorithm (Sutton, 1988; Montague, Dayan, & Sejnowski, 1993; Dayan & Sejnowski, 1994; Abbott & Blum, 1996; Dayan, Kakade, & Montague, 2000; Rao & Sejnowski, 2001; Haruno, Wolpert, & Kawato, 2001; Schultz & Suri, 2001). In TD learning, the “later event” is represented by a designated reference signal (mostly a reward or punishment signal) to which the prediction of the learner is explicitly compared. The reference signal thus represents an explicitly deŽned so-called evaluative feedback for the learning, which stops when prediction and reward match. This may pose a problem, as pointed out by Klopf (1988), who had emphasized that evaluative feedback cannot exist in autonomously acting agents, which normally cannot rely on any external, evaluative, (teacher-like) signal. Klopf’s differential Hebbian algorithm is indeed nonevaluative and belongs to the so-called class of drive reinforcement models. This issue, however, is still rather controversial. Klopf’s arguments are convincing, yet evidence exists that dopamine could indeed serve as such a possible reward-like reference signal in the brains of higher mammals (Schultz, Dayan, & Montague, 1997; Schultz & Suri, 2001), which can respond to complex learning situations

Isotropic Sequence Order Learning

833

such as instrumental (operand) conditioning. Less complex forms of learning such as basic classical conditioning, however, can be observed even in very simple creatures (for example, Aplysia), which do not have a reward system (Kandel et al., 1983). One aspect of the current study, therefore, is to design an algorithm in which sequence order learning takes place in a reward-free, unsupervised way by means of a temporal Hebb learning rule that is isotropic with respect to the inputs (hence the name ISO learning, which stands for isotropic sequence order learning).1 Thus, the algorithm is strictly based on the causal relation between its inputs, which is in reality often given by the “properties of the world,” as described by the examples above. The reference signal is just the latest occurring signal (which often has the highest initial synaptic weight), a situation that can change during learning. The article is organized in the following way. First, we introduce the algorithm in an open-loop paradigm. Its linear structure allows an analytical treatment of some of its main characterizing features. More complex aspects are addressed with simulations. In this part of the study, it will become clear that all input lines are mathematically equivalent in our algorithm. Furthermore, we will show that the algorithm performs strict heterosynaptic learning. A detailed comparison of ISO learning with other algorithms is given in appendix B. In the second part of this study we embed our algorithm in a behavioral loop by means of a robot experiment. This creates a self-referential system and leads to stability. As a consequence of the fact that learning is heterosynaptic, we Žnd that synaptic weights will self-stabilize as soon as the reex input becomes silent. One central aspect of this and the companion article, “ISO Learning Approximates a Solution to the Inverse-Controller Problem in an Unsupervised Behavioral Paradigm,” is to show that unsupervised open-loop ISO learning inherently turns into a reference-based system as soon as it is embedded into a nonevaluative environment that leads to a closed sensor-motor loop. This could be expected from the results of Klopf (1988), but we will show analytically in the companion article that such a closed-loop system creates, by means of the learning process, a forward model of its environment. Temporal sequence learning using the ISO learning algorithm can therefore be understood as Žnding a solution to the speciŽc inverse controller problem that replaces a reex by its forward model.

1 The self-referential structure of this abbreviation is meant to hint at the self-referential behavioral loop introduced in the second part of this article.

834

B. Porr and F. Worg ¨ otter ¨

Figure 1: The basic circuit in the time domain.

2 Open Loop: ISO Learning First, we describe the algorithm itself and its characteristics without behavioral feedback. We consider a system of N C 1 linear Žlters h receiving inputs x and producing outputs u. The Žlters connect with corresponding weights ½ to one output unit v (see Figure 1). In section 1, we emphasized that all input lines of our algorithm are mathematically equivalent. It should be remembered, however, that functionally, many times there are distinctive differences among them. As a consequence, we will use x0 to denote the one unit that will later represent the reex pathway. This has no mathematical consequences and is done only for convenience. The output v is then given as v D ½0 u0 C

N X

½ k uk :

(2.1)

kD1

Learning (weight change) takes place according to a differential Hebb rule, d ½j D ¹uj v0 dt

¹ ¿ 1;

(2.2)

where the weight change depends on the correlation between uj and the derivative of v. An extensive discussion how this rule relates to TD learning and to other differential Hebbian learning rules, as introduced by Klopf (1986, 1988) and Kosco (1986), is given in section 4 and appendix B. Here, we note only that other differential Hebbian learning rules use Žltering and derivates in different pathways as compared to ISO learning (see Figure 12 for circuit diagrams). All weights can change (also ½0 ). The constant ¹ is adjusted such that all weight changes occur on a much longer timescale (i.e., very slowly) as

Isotropic Sequence Order Learning

835

compared to the decay of the responses u. Thereby the system operates in the steady-state condition. In general, the system that we consider shall operate in continuous time (e.g., with neuronal rate codes), and it shall be able to handle continuous input functions x.t/ of arbitrary shape. The transfer function h shall be that of a bandpass that transforms a ±pulse input into a damped oscillation (see Figure 2A) and is speciŽed in the Laplace domain, h.t/ $ H.s/ D

1 ; .s C p/.s C p¤ /

(2.3)

where p¤ represents the complex conjugate of the pole p D a C ib. It is important to note that such a bandpass is stable only if its pole pair is located on the left complex half-plane; otherwise, an ampliŽed oscillation is obtained. Real and imaginary parts of the poles are given by a :D Re.p/ D ¡¼ f=Q q b :D Im.p/ D .2¼ f /2 ¡ a2 ;

(2.4) (2.5)

where f is the frequency of the oscillation. The damping characteristic of the resonator is reected by Q > 0:5. Small values of Q lead to a strong damping. The use of resonators (bandpass Žlters) is motivated by biology because oscillatory neuronal responses (Traub, 1999) and bandpass-Žltered response characteristics (at virtually all sensory front ends, cell membranes [Shepherd, 1990], and ion channels like NMDA) are very prevalent in neuronal systems. Several examples for the utilization of such bandpass-Žltered responses provide Grossberg and Schmajuk (1989) with their spectral timing model, which they have used in different applications (Grossberg, 1995; Grossberg & Merrill, 1996). Thus, the main idea is to use a neuron that gets bandpass-Žltered sensor signals at its inputs and generates a motor output. Later, one of these bandpasses (h0 ) has the special task of providing the input for a reex-like reaction. The other bandpass-Žltered sensor signals are candidates for generating an earlier motor reaction through learning. 2.1 Analytical Findings: Open-Loop Condition. 2.1.1 Timing Dependence of Weight Change. Here we address the question how the timing between the input signals inuences the weight change. In order to perform analytical calculations, we introduce two restrictions,

836

B. Porr and F. Worg ¨ otter ¨

Figure 2: Input functions and the initial weight change for t D 0 according to equations 2.13 and 2.14. (a) The inputs x, the impulse responses u for a choice of two different resonators h, and the derivative of the output v0 . (b) The initial weight change ½1 .T/tD0 for H1 D H0 ; Q D 1; f D 0:01 (arbitrary units) and (c) for resonators with different frequencies f0 D 0:01; f1 D 0:02 but with the same Q D 1. The solid lines in b and c represent the analytical solutions derived from equations 2.13 and 2.14, and the dots the simulation results from the numerical integration of equation 2.9 with the same parameters for f and Q. For that purpose, the two Žlters H0 and H1 get two different inputs x1 .t/ D ±.t/ and x0 .t/ D ±.t ¡ T/. This pulse sequence was repeated every 2000 time steps. After 400,000 time steps, the weight ½ was measured and plotted against the temporal difference T. The learning rate was set to ¹ D 0:001. (d) Schematic explanation of the mutual weight change at a strong (A) and a weak synapse (B) with two subsequent delta pulses at the inputs x1 and x0 . For further explanations, see the text.

which we use often throughout the theoretical parts of this article: 1. We will consider only two resonators, thus, N D 1.

2. Accordingly we have to deal with only two input functions x0 ; x1 , and we deŽne them as (delayed) ±-pulses: x0 .t/ D ±.t ¡ T/; x1 .t/ D ±.t/:

T¸0

(2.6) (2.7)

The Žrst restriction is necessary because the analytical treatment of the case N > 1 is very intricate and largely impossible. Concerning the second

Isotropic Sequence Order Learning

837

restriction, we note that the theory of signal decomposition allows composing any causal input function from ±-pulses. Thus, the second constraint is not really a restriction. The delay T ensures a well-deŽned causal relation between both inputs, where x0 (the latter of the two) is the timing reference (the reex input). Especially the section on the robot implementation will show that the algorithm (with N > 1) is very robust with respect to variations in T. In general, we use as an initial condition ½0 D 1 and ½1 D 0. For the analytical treatment, we consider only the weight change at ½1 . (In fact, we later show that the algorithm normally operates always in a domain where ½0 changes very little.) Because we assume steady state, we can rewrite the product in the learning rule (see equation 2.2) as a correlation integral between input and output: ½1 ! ½1 C 1½1 Z 1 1½1 .T/ D ¹ u1 .T C ¿ /v0 .¿ / d¿:

(2.8) (2.9)

0

Similar to other approaches (Oja, 1982), we compute the weight change for the initial development of the weights as soon as learning starts, because this is indicative of the continuation of the learning. Therefore, we assume ½1 .t/ D 0 for t D 0, and equation 2.9 turns into Z

½1 .T/tD0 D ¹

1 0

u1 .T C ¿ /u00 .¿ / d¿:

(2.10)

In simple cases (e.g., for h0 D h1 ), this integral can be solved directly. A general solution, which can be extended to cover more than two inputs, requires applying the Laplace transform using the notational convention x.t/ $ X.s/ for a transformation pair of functions in the time and the Laplace domain. The linearity of our system allows solving the integral in equation 2.10 analytically, which is possible with the help of Plancherel’s theorem (see appendix A for this rather unknown theorem). Applying it together with the shift theorem x.t ¡ t0 / ! X.s/e¡t0 s to equation 2.10, we get: 1 1½1 D ¹ 2¼ D¹

1 2¼

Z Z

C1 ¡1

C1

¡1

H1 .¡i!/[i!e¡Ti! H0 .i!/] d!

(2.11)

H1 .i!/[¡i!eTi! H0 .¡i!/] d!:

(2.12)

Note that the symmetry of Plancherel’s theorem is broken due to the exponential term. Equation 2.11 represents a Fourier transform, and equation 2.12 is its inverse. Both integrals can be evaluated with the method of

838

B. Porr and F. Worg ¨ otter ¨

residuals. Equation 2.12, however, offers the advantage that we can neglect the right complex half-plane, because it leads to contributions for negative time (i.e., t < 0) only (McGillem & Cooper, 1984; Stewart, 1960). Thus, of the four residuals (poles) for H1 and H0 , only those of H1 need to be considered because those of H0 have ipped their sign in equation 2.12 and appear now on the right complex half-plane. We get as the Žnal result, ½1 .T/tD0 D ¹ ½1 .T/tD0 D ¹

b1 M cos.b1 T/C.a1 PC2a0 jp1 j2 / sin.b1 T/ ¡Ta1 e b1 .PC2a1 a0 C2b1 b0 /.PC2a1 a0 ¡2b1 b0 / b0 M cos.b0 T/C.a0 PC2a1 jp0 j2 / sin.b0 T/ ¡Ta1 e b0 .PC2a0 a1 C2b0 b1 /.PC2a0 a1 ¡2b0 b1 /

T ¸ 0 (2.13) T < 0; (2.14)

where M D jp1 j2 ¡ jp0 j2 and P D jp1 j2 C jp0 j2 . If we assume identical resonators H0 D H1 D H, we get 1½1 .T/tD0 D ¹

1 sin.bT/e¡aT ; 4ab

(2.15)

which is identical to the impulse response of the resonator itself apart from a different scaling factor. The corresponding weight change curves are plotted in Figures 2b and 2c. The curves show that synaptic weights are strengthened if the presynaptic signal arrives before the postsynaptic signal, and vice versa. The biological relevance of the learning curves becomes especially clear in the case H0 D H1 . This learning curve with identical resonators is similar to the curves obtained in neurophysiological experiments exploring spike-timingdependent synaptic plasticity (STDP or temporal Hebb; Markram, Lubke, ¨ Frotscher, & Sakman, 1997; Bi & Poo, 1998; Zhang, Tao, Holt, Harris, & Poo, 1998; Abbott & Nelson, 2000; Fu et al., 2002). 2 Furthermore we Žnd for this case (see Figure 2b) that the location of the maximum of the learning curve Topt falls in the interval ¸ ¸ < Topt < ; 2¼ 4

1 < Q < 1; 2

(2.16)

where ¸ D 1=f is the wavelength of the resonator. The isotropic setup of the algorithm in principle also leads to weight changes at ½0 . It is, however, evident that the change in ½0 is (very) small when the contribution from the other inputs ½k ; k ¸ 1 is small. This is most easily seen when considering Figure 2d, which shows a situation that arises 2 In order to reproduce STDP in a biophysical model, the signals x and u require a different interpretation involving NMDA conductances and backpropagating action potentials. This is the topic of a follow-up study currently in preparation (Saudargiene, Porr, & W¨orgo¨ tter, 2003).

Isotropic Sequence Order Learning

839

after some learning by using the standard initial conditions. The size of the synapses depicts the momentarily existing weight values. The input sequence is such that a weight increase arises at synapse B from the inuence of input line A onto line B (CT in learning curve), whereas weight decrease occurs at synapse A due to the inverse causal (¡T) inuence of input line B onto line A. The degree of change is depicted by the plus and minus signs, showing that the decrease of A is smaller than the increase of B. For two similar inputs, a simple rule of thumb is that the weight change 1½ roughly follows the weight value of the other input scaled by the learning rate ¹, while the sign of the change depends on the temporal sequence of events: 1½late input ¼ ¹½early input

1½early input ¼ ¡¹½late input:

(2.17) (2.18)

As a result, the strong input roughly maintains its strength while the contributions from the other inputs are small. This is the typical case when learning is guided by a strong reex and the organism has the task of building up predictive pathways that should be weaker but more precise in order to prevent the disturbance. We note that the above analytical results can be extended to cover the most general system structure as represented in Figure 1 with N > 1. Equation 2.1 turns into V.s/ D

N X

(2.19)

½k Uk .s/;

kD0

keeping it in the Laplace domain, because then we can directly obtain 1½j .T/ D ¹

1 2¼

Z

C1

¡1

¡i!V.¡i!/Uj .i!/ d!;

(2.20)

which is the general form of equation 2.9 in the Laplace domain. It should be noted that for all 1½j , this integral can still be evaluated analytically in the same way as in the special case with two resonators. In the following equations, we will always use the index j for the input weights and k for the summation of the output signal v. 2.1.2 Weight Change When x0 Becomes Zero. In this section, we address the question of weight development when the reference input (reex) becomes silent (x0 D 0) at some point during learning. This is motivated by the cases discussed in section 1, where the goal of learning is to avoid (late, painful, damaging) reex reactions. Thus, setting x0 D 0 corresponds to the condition when the reex has successfully been avoided. Note that we are now left with just one input (x1 ) asking if its synaptic weight will continue

840

B. Porr and F. Worg ¨ otter ¨

to change. This would correspond to a situation of homosynaptic learning (e.g., homosynaptic long-term potentiation; Guo-Quing & Poo, 1998). This section will show that our algorithm does not perform homosynaptic learning. Instead, the synaptic weight of x1 stabilizes as soon as x0 D 0. Thus, ISO learning is purely heterosynaptic learning. We use the same two restrictions as above and start with equation 2.20, inserting equation 2.19 into it. We set x0 D 0 $ X0 D 0, and the weight change becomes 1½j D ¹

Z C1 N 1 X ¡i!Hk .¡i!/Hj .i!/ d!: ½k 2¼ kD1 ¡1

For N D 1, we get Z C1 1 ¡i!H1 .¡i!/H1 .i!/ d! 1½1 D ¹ ½1 2¼ ¡1 Z C1 i D ¡¹ ½1 !jH1 .i!/j2 d!: 2¼ ¡1

(2.21)

(2.22) (2.23)

H1 .i!/H1 .¡i!/ D jH.i!/j2 is valid since the transfer functions can always be expressed as products of complex conjugate pole pairs. Multiplying H1 .i!/ with H1 .¡i!/ leads to products of a complex number with its conjugate counterpart, which renders its absolute value. Since all transfer functions are symmetrical in relation to the real axis, the frequency response jH.i!/j2 is also symmetrical, which leads to symmetrical responses in equation 2.23 at jH1 .i!/j2 . Due to ! in equation 2.23, the entire integral becomes antisymmetrical and thus zero.3 Thus, the weights stabilize if only x1 is active. This result can be summarized in a rather intuitive way: With N D 1 and x0 D 0, there is an input signal only at x1 . The weight change in that case is a correlation of a damped sine wave with its derivative, which is a damped cosine wave. The correlation of a sine with a cosine is always zero. We have not attempted to calculate the behavior of the weights for N > 1, which is very tedious, if not impossible. Instead, we will show simulation results for this later. However, the above argument can be extended by the Fourier theorem of wave decomposition to more inputs because each sine wave from a resonator is multiplied by its cosine counterpart. Thus, we also expect for N > 1 a zero correlation and a stop of the weight development as soon as x0 D 0. 3 In a practical application (e.g., a digital inŽnite impulse response Žlter), this is true only if the frequency responses of the input X1 and the transfer function H1 vanish for high frequencies to avoid the integral’s becoming ill deŽned (1 ¡ 1). In other words, the transfer functions must contain a low-pass term. This reects the aspect that the time course of the input functions must be predictable (Kalman Žlter property).

Isotropic Sequence Order Learning

841

Figure 3: Simulation results with a circuit with two inputs, hence N D 1 (see Figure 1). Input pulse sequences were repeated every 100 time steps, the Žrst starting at zero. Both resonators had values of Q 0;1 D 1 and f0;1 D 0:1. The other parameters were ¹ D 0:01 and T D 2. Results for (a) t D 0 and (b) t D 900.

2.2 Simulations: Open-Loop Condition. In this section, we perform simulations with the neuronal circuit from Figure 1. The simulations have the purpose of validating the theoretical results from the previous section and exploring more complex situations (especially N > 1) that are not analytically treatable. The simulations were performed under Linux on an Athlon processor using C++. The resonators were implemented as time-discrete inŽnite impulse response Žlters in the z-domain. We used the impulse invariant transformation from the s-plane to the z-plane and calculated the coefŽcients for the Žlters according to McGillem and Cooper (1984). We used normalized time steps resulting in normalized Žlter frequencies in the range f D [0; : : : ; 0:5]. In all applications, we used frequencies less than or equal to fmax D 0:1 in order to avoid sampling artifacts. 2.2.1 One Filter in the Predictive Pathway: N = 1. As before, we begin with the simplest case, N D 1: one resonator in the reex pathway x0 and one resonator in the predictive pathway x1 , and use both restrictions (1,2) noted previously. Signal shape. Figure 3a shows for t D 0 the ±-pulses at x0;1 and the responses u0 and u1 from the resonators H0 and H1 , respectively. Before learning, the output v is identical to the signal u0 because the weights were set to ½0 D 1 and ½1 D 0. The actual weight change of ½1 is caused by repeated pairing of the ±-pulses at x0 and x1 . The result after nine pairings is depicted in Figure 3b. The comparison between Figures 3a and 3b shows that the onset of the output v has shifted toward the earlier event x1 . Before learning, it was identical to the resonator response u0 in the reex pathway. After learning, the output is a superposition of both signals u0;1 , which leads to an onset that occurs together with the early onset of u1 .

842

B. Porr and F. Worg ¨ otter ¨

Thus, the circuit is able to “detect” the ±-pulse at x1 as a predictor of the ±-pulse x0 . Learning curve. Using the same setup, we can vary the interval T and plot the change of ½1 in dependence of T for the initial learning step (i.e., for t D 0 after one correlation). This was simulated using identical resonators H0 D H1 but also with different resonators H0 6D H1 . The results are shown together with the analytical Žndings in Figures 2b and 2c, having used the same parameters in both the simulation and the analytical calculation. Thus, the analytically calculated weight change curves are reproduced by the simulation results. Development of ½0 . In all cases discussed so far, both weights were allowed to change, and substantial changes in ½1 were found for about 10 to 50 pairings, while we have claimed that ½0 remains stable. An easy intuition why this basically holds can be gained by using the rule of thumb deŽned in equations 2.17 and 2.18. From this, it is clear that the change of ½0 remains tiny for a prolonged time in our setup because ½1 equals zero at the beginning and ¹ is very small. In simulations, we found that ½0 starts to change by more than 1% only after about 50,000 learning steps when using a standard learning rate of ¹ D 0:001 and ½1 D 0; ½0 D 1 as the usual initial conditions. Note that in the robot experiments shown later, the learning goal is reached after not more than 20 pairings. During this time, the change in ½0 is minuscule. Several other relevant cases could occur. ² Another interesting initial condition would be setting the weights to the same initial values (e.g., ½0 D ½1 D 0:5). This will still lead to a weight growth at ½1 (until about learning step 100,000), but now ½0 will drop from the beginning. Functionally, this could be interpreted as a situation where the reex input becomes weaker, while the anticipatory pathway continues to take over. This could reect a situation where the reex has not been used for a long time, because then it is reasonable to allow the reex to disappear, leading to ½0 D 0. The only measure that has to be taken is to stop the weight from changing its sign by keeping it at zero. ² In conditions where the reex saves the organism from life-threatening situations, the weight ½0 can always be set to a Žxed value. ² In conditions where we have multiple synaptic weights of similar strength (i.e., N > 1), we can expect that the system’s development will be dominated by stimulus-sequence-induced symmetry-breaking effects. This can lead to rather complex patterns, which would require a more detailed analysis (and is beyond the scope of this article).

Isotropic Sequence Order Learning

843

Figure 4: Simulated development of the weight ½1 for the case of two inputs (N D 1). Parameters were f0;1 D 0:01 and Q0;1 D 1. The inputs are triggered at a temporal difference of T D 15: x0 D ±.t ¡ T/ and x1 D ±.t/. The pairing of the delta pulses is repeated every 2000 time steps. The learning rate is set to (a) ¹ D 0:001 and (b) ¹ D 0:01.

Weight stabilization for x0 D 0. The analytical results (see equation 2.22) predict that ½1 should stabilize as soon as x0 D 0. This, however, also requires that the learning rate ¹ is zero, which in reality cannot be ultimately achieved. The following simulation results show the effect of the learning rate on the development of the weights and compare the analytically obtained result with those obtained for more realistic situations. The simulation to test this was performed in the following way. First, we triggered the two resonators with paired ±-pulses. Then the input x0 was switched off (i.e., x0 D 0) at t D 400,000 and only the input x1 was still active. Figure 4 shows the weight development of ½1 over time for two different learning rates ¹. With a low learning rate, the weight ½1 approximately stabilizes when the input x0 is switched off (see Figure 4a), whereas with a higher learning rate, the weight continues to grow. Weight stabilization can be very desirable during learning, but so is a high learning rate. These conicting demands therefore lead to a trade-off, which needs to be taken care of in practical applications (like the robot application). 2.2.2 More Than One Filter in the Predictive Pathway. The setup with only one resonator (N D 1) in the predictive pathway has the disadvantage that there is only one speciŽc temporal interval Topt where learning (weight change) is at the maximal rate. The use of an array of resonators with different frequencies in the predictive pathway removes this disadvantage (see the inset in Figure 5). The system should now be able to learn more than only one time interval properly. We have set up such a system with an array of 10 resonators in the predictive pathway. We triggered this array with the same ±-pulse (x1 D ±.t/). The reex pathway was triggered by a delayed

844

B. Porr and F. Worg ¨ otter ¨

±-pulse (x0 D ±.t ¡ T/I T D 10). The initial condition for learning was set to ½0 D 1I ½k D 0I k ¸ 1 as before.

Signal shape. Figure 5 shows the resonator responses uk scaled with their momentarily existing weights ½k (top) at time t D 390,000 during learning. The scaled response of u0 (a, dashed line) is still the biggest at this time. The diagram also shows the output signal v and its derivative during the learning process (also t D 390,000, bottom). Additionally, the output signal generated when silencing the input x0 is shown (c, dotted line, bottom, t D 400,000). The output v is a superposition of all resonator outputs. It can be seen that it has a Žrst and a second maximum (marked with 1 and 2 in Figure 5). The second maximum is due to the resonator response from the reex pathway u0 and vanishes when the input x0 is switched off (see the dotted curve in c). The Žrst maximum is generated by superposition of the responses ½k uk ; k > 0 (i.e., all except u0 ). In general, we have observed that this superposition process will always try to generate the Žrst maximum as close as possible to x0 . This can be understood by the ongoing ampliŽcation of an initially existing asymmetry in the system in the following way. At the Žrst learning step, the derivative of v is zero before x0 and then follows the shape of the v0 curve, as shown in the diagram. Thus, there is one resonator response whose shape matches the v0 curve best (best positive correlation). Obviously, it is that particular resonator that has its maximum at (or closest to) the maximum of the v0 curve (second cusp; the Žrst is still zero). For this resonator, we get the highest correlation result (see equation 2.9) and, thus, the strongest weight growth at the beginning of learning. The other weights grow less strong, and their growth rate is approximately (inversely) related to the distance of their resonator maximum from x0 . As a consequence, we get a distribution of weight values that follows the shape outlined by the y-position of the resonator maxima, as shown in the top panel by the dots on the curves. Superposition of these weighted responses thus leads to a maximum of v at x0 . This line of argumentation continues to hold for the following learning steps, because the theoretical results suggest that the contribution of the correlation of the Žrst part of the v0 curve (Žrst cusp) with the uk ; k > 0, which would correspond to homosynaptic learning, is zero in all cases (see equations 2.21–2.23), thereby not affecting the weight change. Thus, weight change continues to follow the distribution of the maxima in Figure 5a. The resonator with the lowest frequency ( fl ) determines the longest delay Tmax D 1fl , which can be learned. Equivalently, the shortest delay is Tmin D f1h , where fh is the resonator with the highest frequency. Within the range [Tmin ; Tmax ], any T causes an output with a maximum that always coincides with the location of x0 , provided there are enough resonators to allow for a sufŽciently accurate superposition process.

Isotropic Sequence Order Learning

845

Figure 5: Multiple Žlters (N D 10) in the predictive pathway: (a) Filter responses, (b) the neuronal circuit, and (c) its output during learning and after learning. The neuronal circuit (b) consists of a Žlter bank where the Žlter frequencies are 5 set to fk D kf0 I k ¸ 1 and f0 D 0:01. The learning rate was set to ¹ D 0:0005 and Q D 1. The Žlter bank gets two different inputs: x0 .t/ D ±.t/ (reex pathway) and x1 .t/ D ±.t ¡ T/ (predictive pathway), T D 10. The delta pulses are repeated every 2000 time steps. After 400,000 time steps, x0 is set to zero. The contribution of the signals uk ½k to the output v triggered by x1 .t/ is called HV and is marked by the shaded box in b. The weighted resonator responses ½k uk after learning are shown in a. The output signals during learning (time step 390,000) and after learning (after time step 400,000) are shown in c.

Learning curve. As in the case of only two resonators, the dependence of the weight change on the temporal distance T can be explored. Now, however, we have to monitor N changeable weights. For this experiment, we have chosen the same standard setup using paired ±-pulses with a temporal delay of T, but now we use 15 resonators (N D 15) in the predictive pathway. Their frequencies are chosen such that 10 resonators have a frequency that is higher and 5 resonators one that is lower than f0 (see Figure 6). Every second weight change curve is shown in Figure 6 for t D 0, where we varied

846

B. Porr and F. Worg ¨ otter ¨

Figure 6: Weight changes ½j dependent on the temporal distance T with a Žlter bank of resonators (N D 15) set up as in Figure 5b. The Žlter frequencies are set to 5f fk D k0 I k ¸ 1 with f0 D 0:01 and Q D 1. The learning rate was set to ¹ D 0:0001 and Q D 1. The case f0 D fk is marked with a thick line and reproduces the curve in Figure 2b. The Žlter bank gets two different inputs: x1 .t/ D ±.t/ (predictive pathway) and x0 .t/ D ±.t ¡ T/ (reex pathway). The delta pulses are repeated every 2000 time steps. After 400,000 time steps, the weight ½j was measured and plotted against the temporal difference T. Only every second curve is plotted.

T from ¡150 to 150. Every curve in this diagram represents one weight ½k of a speciŽc resonator hk in dependence of T. The curve plotted with the thick line belongs to the resonator hk , which has the same frequency as the resonator h0 , hence, fk D f0 . The other weight change curves belong to resonators in the predictive pathway, which have different frequencies compared to f0 . It can be seen that every weight change curve has a speciŽc T where weight change is maximal, or (in support of the argument used to explain the Žrst maximum in Figure 5) the other way around: for speciŽc values of T and large N, there always exists one particular resonator that shows maximum weight change. Another interesting result is that the weight change curve with fk D f0 is identical to the weight change curve with only one resonator (see Figure 6). The fact that both weight change curves are the same is due to the linearity of our model.

Isotropic Sequence Order Learning

847

Figure 7: Weight change of multiple resonators N D 10 in dependence of the learning rate. The neuronal circuit (see Figure 5b) consists of a Žlter bank where the Žlter frequencies are set to fk D 0:1 I k ¸ 1, where the index k is also used as a k label for the different curves in this Žgure (Q D 1 in both cases). The Žlter bank gets two different inputs: x1 .t/ D ±.t/ (predictive pathway) and x0 .t/ D ±.t ¡ T/ (reex pathway) with T D 10. The delta pulses are repeated every 2000 time steps. After 400,000 time steps, x0 was set to zero. The learning rate was set to (a) ¹ D 0:0001 and (b) ¹ D 0:001.

In summary, in an array of different resonators, every resonator is responsible only for a speciŽc and limited range of temporal intervals so that such an array is able to cover a wide range of different temporal intervals. The weight change curves for the different weights give precise information as to which resonator yields the maximum contribution to the output signal. Weight stabilization for x0 D 0. Next, we ask whether the weights also stabilize in a multiresonator setup if the reex pathway x0 becomes zero (compare to Figure 7). We use the same setup as before (N D 10 and paired ±-pulses with T D 10). The test was performed in the same way as above by setting x0 to zero at time t D 400,000. Figure 7 shows that the weights stabilize in the limit of ¹ ! 0. Thus, we Žnd again that the crucial parameter for an approximate weight stabilization is the learning rate ¹, which is too high in Figure 7b. Because of the complexity of the mathematics, we were not able to give robust analytical arguments for weight stabilization in the multiresonator case. We could argue only that the individual resonator responses (sine waves) should be orthogonal to the derivative of the output (cosine wave) as soon as x0 D 0 (see the dashed curve in Figure 6), leading to zero value of the correlation integral. The experimental Žndings in Figure 7 support this notion. Thus, in the multiresonator case, we obtain the desired property of weight stabilization in the limit of ¹ ! 0. Homosynaptic learning does not take place even with more than two resonators.

848

B. Porr and F. Worg ¨ otter ¨

Let us in the context of N > 1 also briey consider more than one predictive pathway with several sensors that operate independently. In this case, the isotropy of the algorithm leads to the situation that learning will continue between those sensor inputs even after the reference (reex) input x0 has become silent. Weight changes of the other (nonreex) weights, however, will normally remain small because after learning, the absolute values of them are small, which leads only to minor cross inuences, as will be shown in the robot example (see Figure 11). Thus, even in such a situation, weights will be (approximately) stable. Another stabilizing factor arises if we place the learning algorithm in a closed behavioral loop (see also the next section). In the closed-loop paradigm analyzed in our companion article, we found that a perturbation of the weight ½1 (which disturbs the Žnal condition x0 D 0) leads to a counterforce that reestablishes the original weight. Taken together, these arguments show that weights might indeed undergo small drifts after removal of the reex, but these drifts do not lead to a divergence. This is supported by the robot experiments, where we never observed weight divergence even after prolonged runs. 3 Closed Loop: The Robot Experiment The task in this robot experiment is collision avoidance. The built-in reex behavior is a retraction reaction after the robot has hit an obstacle (see Figure 8, solid pathway). This represents a typical feedback mechanism with the desired state that the signal at the collision sensor should remain zero. In order to prevent the robot from leaving the desired state, it can use other sensor modalities that can predict a looming collision. In our case, this is achieved with range Žnders (see Figure 8, dashed pathway). The learning algorithm has the task of learning the existing temporal correlation between the range Žnder and the collision sensor signals. After learning, the robot can generate a motor reaction in response to the range Žnder signals and thereby avoid the retraction reex. Functionally, the reex will be eliminated, and the “predictive pathway” takes over after learning. Up to this point, the algorithm had been treated in a pure open-loop condition, where learning was entirely unsupervised. The robot experiments create a situation where the behavioral reaction inuences the sensor inputs, thereby creating a closed-loop situation (see Figure 8). Unsupervised learning thereby turns into something that we would call self-referenced learning in order to distinguish it from reinforcement learning, which requires an explicitly deŽned reference signal (punishment or reward), which is not present in closed-loop ISO learning. The theoretical treatment of this situation in our companion article will clarify that these two situations are fundamentally different. The robot’s circuit diagram is shown in Figure 9; a detailed description, which includes a list of the robot’s control parameters, is given in

Isotropic Sequence Order Learning

849

Figure 8: Simple sensor motor feedback with prediction that is made explicit with the example of collision avoidance. The solid lines depict a prewired reex loop that exists before learning. This reex loop performs a reex reaction—in this case, a retraction reaction (motor response) when the collision sensor (reex eliciting signal) has been triggered. The task is to learn that the earlier range Žnder signal (predictive signal, dashed pathway) can be used to generate an earlier motor reaction to prevent the collision (reex).

appendix C. The robot has three collision sensors and two range Žnders. All signals are Žltered by bandpass Žlters and converge onto two neurons, which generate two different motor outputs: one controls the robot’s speed and the other the robot’s steering angle. The speed of the robot is set to a Žxed value and its steering to zero so that the undisturbed robot drives straight forward. The built-in retraction behavior is generated by the dotted pathways where the collision sensors trigger highly damped sine waves in the corresponding resonators. This signal is sign inverted and directly transmitted to the motors. Essentially, it consists of just a single half-wave, which leads to the retraction reaction. The weights are initially set to minus one and effectively do not change during learning, so that the retraction behavior always remains the same. The dotted collision sensor pathways with their strong weights that determine the motor output are, together with the arising behavioral feedback, equivalent to the reex loop discussed in Figure 8. The range Žnder signals (solid lines) react at a distance of about 15 cm from an obstacle and are therefore able to predict a collision. However, the temporal delay between the range Žnder signal and the collision signal is variable and depends on the actual motion trajectory of the robot. In order to cope with a rather wide range of temporal delays, we used the same approach as in section 2.2.2 and implemented two resonator Žlter banks, which get their signals from the two range Žnders. Filter banks consist of 10

850

B. Porr and F. Worg ¨ otter ¨

Figure 9: Robot circuit: The robot consists of three collision sensors (CS), two range Žnders (RF), and two output neurons for speed (ds) and steering angle (dÁ). P These output neurons represent simple summation circuits (indicated by ). The robot has a reex behavior established by the signals from the collision sensors (dotted lines), which are fed into four bandpass Žlters H0 with f0 D 1 Hz and Q0 D 0:6. The output of the bandpass Žlters is summed at the neurons for speed (ds) and steering angle (dÁ). The corresponding weights are adjusted in such a way that the robot performs an appropriate retraction reaction if either of the collision sensors is triggered. The task of learning is to use the signals from the range Žnders (RF) to predict the trigger of the collision sensor (CS). The two signals from the left and the right range Žnder are fed into two Žlter banks, with I k ¸ 1 and Q D 1 throughout. N D 10 resonators with frequencies of fk D 1Hz k The 20 signals from the two Žlter bank converge on both the speed neuron and the neuron responsible for the steering angle. Learning rate was ¹ D 0:00002. L depicts the implementation of the learning rule (see equation 2.2).

resonators covering approximally a temporal interval between 50 ms and 500 ms. These resonator signals converge onto both the speed neuron and the steering neuron. Their weights are initially set to zero. Depending on the initial conditions, the robot found different solutions to avoid obstacles. One solution, for example, is that after learning, the robot simply stops in front of an obstacle or slightly oscillates back and forth. This

Isotropic Sequence Order Learning

851

type of behavior may look trivial but is entirely compatible with the learning goal of avoiding obstacles. More commonly, we observed a different type of solution where the robot continuously drives around and uses mainly its steering to generate avoidance movements. Other solutions do not seem to be possible and have not been observed. Furthermore, we observed that the robot always found one of these solutions after sufŽciently long learning. Figure 10 shows episodes of the robot behavior and its signals for one selected example trajectory. The signals shown in Figures 10c and 10d correspond to a situation where the robot still collides with the walls. Corresponding collision points are marked in Figure 10a by c and d. As expected, learning leads to a change of the temporal relation between the range Žnder signal and the collision signal. This can be seen by the different lengths of T depicted in Figures 10c and 10d and is due to the learned motor output, which is increasingly dominated by the range Žnder signal. This supports the Žlter bank approach, which we have used in the robot experiment. Finally, Figure 10e depicts a situation where the robot has learned to avoid the obstacles (CS D 0). Note that the low-pass component of the bandpass Žlters smoothes the rater noisy range Žnder signals, which substantially adds to the robustness of the algorithm. Furthermore, pure noise signals are not correlated to other sensor signals and do not contribute to learning. The change of the weights in the robot example shall now be compared with the results from the simulations. We Žnd that the weights approximately stabilize in our robot experiments (see Figure 11). Their actual values depend on the solution found. The situation in the robot experiment, however, is more complicated than in the simulations shown earlier, because the ds- and dÁ-neurons get signals from more than two sensors at the same time. Thus, often triplets of temporal correlations exist; for example, during a slanted wall approach, we Žrst obtain a signal from the right, then one from the left range Žnder, and Žnally one from the right collision sensor. After successful learning, the collision sensor remains silent, but we are left with sequences of range Žnder events. Thus, learning continues, though at a smaller rate, even after the last collision. As a central observation, this shows that our system continues to operate without a designated reference signal (because x0 is zero now). Learning continues between the remaining inputs. This can be seen in Figure 11 when looking at the development of the weight from the left range Žnder to dÁ, which continues to change after the last collision has occurred (at t D 85s). Ultimately, the earlier of the two range Žnder signals would dominate, but this will lead to a stable situation only for very simple (e.g., circular) trajectories, where an unchanging relation between both range Žnder signals is forced on the robot. An equivalent reward-retrieval situation has also been simulated. These results shall not be presented here in order to limit the length of this article but can be viewed on-line at http://www.cn.stir.ac.uk/predictor/animat/.

852

B. Porr and F. Worg ¨ otter ¨

Figure 10: (a) Manually reconstructed robot movement trace in an arena (240cm £ 200cm) with three obstacles (shaded) at the onset of learning. Motors were not entirely balanced, leading to a curved start of the trajectory. Many collisions (solid circles show forward collision and dashed circles backward collision) occur, and trapping at obstacles happens. After a collision, a fast reexlike retraction and turning reaction is elicited. (b) Robot movement trace after successful learning of the temporal correlation between signals at RF and CS. No more collisions occur; the trajectory is smooth. A complete movie of this trial can be viewed at http://www.cn.stir.ac.uk/predictor/real—movie 1. (c–e) Signals at RF (top), CS (middle), and motor control signal ds (bottom) for different learning stages. (c) Signals occurring at the early collision marked c in part a. A stereotyped motor reaction is elicited in response to the CS signal. (d) Signals occurring at the late collision d. Motor reactions occur in response to RF but are not sufŽcient to avoid the collision. When it occurs, a strong motor reaction is again elicited. (e) Signals occurring at the curve marked e in part b. Smooth motor reactions occur in response to RF; CS remains silent because no collision occurs.

Isotropic Sequence Order Learning

853

Figure 11: (a) Complete motor signal traces for ds and dÁ and (b) development of the synaptic weights for the same trial as in Figure 10.

4 Discussion In this study, we have developed an isotropic algorithm for sequence order learning (ISO learning) in which learning relies only on the temporal order of its inputs. This has the advantage that all input signals are treated equally and that learning takes place between all of them. Thus, it represents a form of unsupervised sequence order learning. 4.1 Basic Properties. ISO learning is driven only by the temporal relation between pre- and postsynaptic signals. As a consequence, our learning rule is related to learning based on spike-timing-dependent plasticity (STDP; Gerstner, Kempster, van Hemmen, & Wagner, 1996; Gerstner, Kreiter, Markram, & Herz, 1997; Markram et al., 1997; Zhang et al., 1998; Bi & Poo, 1998; Roberts, 1999; Xie & Seung, 2000; Kistler & van Hemmen, 2000; Song, Miller, & Abbott, 2000; Song & Abbott, 2001; Fu et al., 2002), but our algorithm uses time-continuous functions and not spike trains as input signals. The measured curves for STDP are based on the relation between individual (pre- and postsynaptic) spikes. Curves with different characteristic shapes have been observed (Abbott & Nelson, 2000), such as that similar to our Figure 2, but also inverted versions of it have been measured. Thus, the question arises how these curves relate to the average Žring rate of the neurons (Gerstner et al., 1997). This question was speciŽcally addressed in the studies of Roberts (1999) and Xie and Seung (2000), who found that the temporal derivative of the postsynaptic impulse rate directly relates to the temporal Hebbian STDP curve, or with sign inversion to the anti-Hebbian curve. This shows that a computational link exists between rate-based ISO learning and the spike-based STDP results because we found that the Hebbian STDP curve will be obtained analytically by integrating the ISO learning rule over time (see equations 2.10 through 2.14).

854

B. Porr and F. Worg ¨ otter ¨

In the second part of this study, we introduced a closed-loop situation by means of behavioral feedback. We implemented a primary reex loop, which is distinguished from all other inputs only by the fact that it initially carries the largest synaptic weight. In general, such closed-loop reex loop situations have the disadvantage that any reaction will occur only after an incoming sensor event. This inherent disadvantage of feedback loops leads to a general objective for improving animal behavior, which is to Žnd a mechanism that prevents the reex (Palm, 2000; Wolpert & Ghahramani, 2000). Sequence order learning can achieve this by creating earlier, anticipatory actions. In addition, we have shown that weights stabilize as soon as the reex has been successfully avoided. Due to the isotropy of the inputs, any other input line can take on the role of the reference signal during learning, and the initial reex can even be unlearned or reduced in strength, a situation observed in many physiological reexes. 4.2 Practical Aspects. A convenient aspect of ISO learning that leads to a very limited computational effort is the use of inŽnite impulse response Žlters in our approach. With such Žlters, it is possible to generate a smooth and long-lasting response when using only two resonators. Such a response can bridge a very long temporal difference T and is therefore able to generate a basic predictive reaction. Additional Žlters contribute to increasingly precise timing. Thus, the basic temporal correlation between X1 and X0 can be established by one Žlter (N D 1) and then improved by adding more and more Žlters. In other approaches (Sutton & Barto, 1981, 1988; Klopf, 1988), delay lines are often used for the predictive input X1 . The discrete structure of these algorithms requires many more delay elements as compared to the analog operating ISO learning because they need a delay element for every unit time-step. Thus, the computational effort is much higher if a broad temporal range has to be covered. 4.3 Evaluative versus Nonevaluative Models. A fundamental difference exists between reference-based (reward-based) algorithms (e.g., TD learning) and so-called drive reinforcement algorithms, such as differential Hebbian learning and ISO learning. Probably the most inuential method for reference-based temporal sequence learning is the TD learning algorithm (Sutton, 1988). TD learning has the goal of generating an output v, which predicts a reference (reward r) by the help of its (sensorial) input signals x. This goal is achieved by minimizing a prediction error ± between reference and output. Thus, learning relies on the predeŽned reference, which acts like a teacher signal in supervised learning. The direct comparison between the two algorithms shows that the reference (reward) pathway and the error calculation of TD learning are replaced by the reex pathway in our algorithm. Mathematically, the reex pathway is not functionally distinct from the other pathways in ISO learning;

Isotropic Sequence Order Learning

855

however, it drives the output with an initially strong weight. As described in section 2.2.1, the strongest input dominates the learning behavior of the other inputs and weights. Klopf (1988) called this drive reinforcement learning. In appendix B, we compare the different drive reinforcement models in the literature. Here, we just note that our algorithm belongs to the class of pure unsupervised learning algorithms opposed to TD learning, which is supervised using the reference (reward) as teaching signal. The initially strong weight in the reex pathway of ISO learning can be interpreted as a boundary condition, preventing the output from becoming arbitrary. Introducing boundary conditions is typical practice of unsupervised, especially Hebbian, learning (Miller, 1996). The structural differences of our learning algorithm and TD learning suggest different neuronal substrates. The TD learning circuit consists of two components: the predictive circuit and the error signal circuit. Usually, these two circuits are identiŽed with different neuronal subsystems: the error circuit with the dopamine system and the predictive circuit with cortical or other dopamine-modulated brain areas. Strong supporting evidence is found in Schultz et al. (1997), and it is also known that reward-based learning plays a substantial role in animal behavior, such as during instrumental (operand) conditioning paradigms and action planning (Dayan & Abbott, 2001). Our algorithm, on the other hand, suggests only one neuronal circuit because all pathways are equivalent, as supported by Hauber, Bohn, and Grietler (2001). It is conceivable that such a system coexists with the rewardbased learning systems, because in an autonomous agent, any reward-based system needs to be bootstrapped by Žrst correlative experiences, such as those used by our system to drive learning. In the one-circuit scenario, our learning rule, based on temporal relation between pre- and postsynaptic signals, would have to be represented by internal neuronal variables like NMDA dynamics or Ca2C concentration. The direct relation of our learning rule with the shape of the STDP curves (see Figure 2b) indicates that it should be relatively straightforward to redesign our model into a biophysically more realistic one, which directly relies on such internal neuronal variables and uses spike trains as inputs. This has recently been attempted by Rao and Sejnowski (2001) using the TD learning algorithm but the relation between TD learning and STDP is less direct, and, accordingly, the transition between those two models is bit more intricate (Dayan, 2002). When talking to specialists in the Želd of temporal sequence learning, we were asked to explain to what degree our learning rule is different from the one used in TD learning. This aspect is quite technical, and we refer readers to appendix B for an in-depth discussion. 4.4 Closed-Loop Condition. Hebbian learning rules like the one used here belong to the class of unsupervised learning rules. Unsupervised learn-

856

B. Porr and F. Worg ¨ otter ¨

ing seems to be the obvious choice for creating the Žrst and earliest stages of autonomous behavior, because it does not require external (teacher-like) knowledge. Instead, it relies purely on self-organization based on the correlation structure of the inputs. Such unguided self-organization processes, however, can also lead to a situation where nonsensical correlations are learned, leading in the end to an undesired network behavior. The standard solution to avoid this problem is the introduction of boundary conditions, which keep the self-organization process within sensible margins. In practice, this is done either heuristically by the network designer or, as a better choice, boundary conditions are introduced such that they intrinsically (and in a natural way) represent the structure of the problem to which the selforganization process is applied. In the case of our unsupervised temporal sequence learning algorithm, the same is achieved by embedding the learning circuit in an environment that leads to a closed-loop situation. The causal relation that naturally exists between many different pairs of sensor events (e.g., pain follows heat, taste follows smell) as described in section 1 creates an implicit boundary condition for our algorithm by using the latest incoming event (the one that drives the reex) as the temporal reference for learning. The environment has two properties in our model: it provides feedback, and it contains disturbances, but very clearly it does not provide any reward or any other teaching signal. Klopf (1988) called this feedback loop nonevaluative since there is nothing in the environment that evaluates the organism’s performance. Instead, here ISO learning becomes self-referenced (von Foerster, 1960; Maturana and Varela, 1980): the actions of the learner inuence its own learning without any evaluation process. Robotics is the discipline that can clarify the concepts of autonomous behavior and interaction with a complex environment quite naturally (Brooks, 1997). In the Želd of temporal sequence learning, Vershure has been working for over 10 years in using robot applications (Verschure & Pfeifer, 1992; Verschure & Voegtlin, 1998). In his words, every organism undergoes three steps of development: prewired reex (Žxed connections), adaptive control (classical Hebbian learning of sequences of sensor inputs), and reective, contextual control (goal-oriented learning). In Vershure’s terminology, adaptive control has no goals but builds up temporal associations with “proximal” and “distal” sensors. At the stage of the reective control, a goal is introduced in the form of a reward or punishment when, for example, an object has successfully been found. Our study shows that this distinction may be too rigid. The behavioral pattern observed in our robot seems to be punishment guided, which would place it at the advanced level of reective control. The unsupervised, nonevaluative, but self-referenced structure of the robot’s interaction with the environment, however, places it at the simpler level of adaptive control. This shows that autonomous agents can develop rather complex behavioral patterns by means of simple nested feedback loop systems, without hav-

Isotropic Sequence Order Learning

857

ing to evaluate their own behavior. Of course, we would not argue against the importance of higher learning schemes, and it is also quite sensible to distinguish between increasingly higher levels starting with nonevaluative schemes, which are surpassed by evaluative (reward and punishment) schemes, which are followed up by contextual learning and Žnally by the different stages of cognitive learning. But it seems advisable to treat these different stages in a less separatist way, allowing for a broader transition range between them. In our companion article, we derive a theoretical treatment of the closedloop ISO learning situation. We will show analytically that the predictive pathway learns to approximate the inverse controller of the reex pathway, thereby creating a forward model of the control situation. Appendix A: Plancherel’s Theorem This theorem is not widely known, so we state it here as Z

1 0

f1 .t/ f2 .t/ dt D

1 2¼

D

1 2¼

Z Z

C1 ¡1

C1

¡1

F1 .i!/F2 .¡i!/ d!

(A.1)

F1 .¡i!/F2 .i!/ d!

(A.2)

where F is the Laplace transform of f (Stewart, 1960). If we set f1 D f2 D f , it becomes the more commonly used theorem of Parseval. Appendix B: Comparison Between TD, ISO Learning, and Other Differential Hebbian Learning Algorithms One has to distinguish drive reinforcement models such as those by Sutton and Barto (1981), Klopf (1986, 1988), and Kosco (1986) (to which ISO learning also belongs) from reference-based reinforcement models such as TD learning (Sutton, 1988; Dayan & Sejnowski, 1994). We Žrst discuss the differences between the different drive reinforcement models, which are shown in Figures 12a through 12c. The central difference between ISO learning and the other techniques is that in ISO learning, all inputs are Žltered before they are summed at the output neuron. In the other algorithms, the inputs are summed in an unŽltered way. A low-pass Žlter is applied only to the conditioned stimulus when it enters the learning pathway (i.e., before the correlator £). This leads to a fundamentally different behavior of ISO learning because as a result, ISO learning produces an orthogonal behavior between input and output (after Žltering, the inputs resemble sine waves; thus, the derivative of the output resembles a sum of cosines; see Figure 2). This orthogonal behavior, which crucially relies on the Žltering of all pathways, leads to the inherent and desired

858

B. Porr and F. Worg ¨ otter ¨

Figure 12: Comparison of (a–c) three drive reinforcement algorithms and (d) TD learning in Laplace notation. Transfer functions are denoted as H, K, and the derivative operator as s. X0 represents the unconditioned and X1 the conditioned input. The ampliŽer symbol denotes the changing synaptic weight. Note that c is drawn with a Žxed weight at X0 to make it more easily comparable to the other diagrams. All models use a derivative of the postsynaptic signal in order to control the weight change. Both Sutton and Barto models (a, d) use low-pass Žlters K only in the conditioned pathway. Klopf’s model b is identical to model a with the exception of an additional temporal derivative at x1 . Only in ISO learning are all inputs Žltered, which together with the output-derivative generates orthogonal behavior, leading to weight stabilization. For further explanation, see the text.

weight stabilization property of ISO learning, which does not arise in the other drive-reinforcement algorithms without additional measures taken. Furthermore, we note that Klopf has introduced an additional derivative at the conditioned input, because he focuses on signal changes. TD learning belongs to yet another category of sequence learning algorithms. The difference arises from the fact that TD is evaluative (reference based; see Figure 12d), whereas the drive reinforcement models operate in a nonevaluative way. We have already discussed this elementary difference

Isotropic Sequence Order Learning

859

at great length. Here, we focus on the aspect that TD learning also uses some kind of derivative, which suggests a strong structural similarity between TD and ISO learning methods. In spite of this apparent similarity, however, our approach is more strongly related to Kalman Žltering than to TD learning. This is due to the combination of linear Žltering and applying a derivative. The predictive property of our algorithm thereby arises from the fact that every low-pass Žltered function is smooth, which leads to the situation that its der