No title

ARTICLE

Communicated by Bard Ermentrout

Asynchronous States and the Emergence of Synchrony in Large Networks of Interacting Excitatory and Inhibitory Neurons D. Hansel [email protected] Laboratoire de Neurophysique et de Physiologie du Syst`eme Moteur, UniversitÂe RenÂe Descartes, 75270 Paris Cedex 06, France, and InterdisciplinaryCenter for Neural Computation, Hebrew University of Jerusalem, 91904 Jerusalem, Israel

G. Mato [email protected] ComisiÂon Nacional de EnergÂõ a AtÂomica and CONICET, Centro AtÂomico Bariloche and Instituto Balseiro (CNEA and UNC), 8400 San Carlos de Bariloche, R.N., Argentina

We investigate theoretically the conditions for the emergence of synchronous activity in large networks, consisting of two populations of extensively connected neurons, one excitatory and one inhibitory. The neurons are modeled with quadratic integrate-and-Žre dynamics, which provide a very good approximation for the subthreshold behavior of a large class of neurons. In addition to their synaptic recurrent inputs, the neurons receive a tonic external input that varies from neuron to neuron. Because of its relative simplicity, this model can be studied analytically. We investigate the stability of the asynchronous state (AS) of the network with given average Žring rates of the two populations. First, we show that the AS can remain stable even if the synaptic couplings are strong. Then we investigate the conditions under which this state can be destabilized. We show that this can happen in four generic ways. The Žrst is a saddle-node bifurcation, which leads to another state with different average Žring rates. This bifurcation, which occurs for strong enough recurrent excitation, does not correspond to the emergence of synchrony. In contrast, in the three other instability mechanisms, Hopf bifurcations, which correspond to the emergence of oscillatory synchronous activity, occur. We show that these mechanisms can be differentiated by the Žring patterns they generate and their dependence on the mutual interactions of the inhibitory neurons and cross talk between the two populations. We also show that besides these codimension 1 bifurcations, the system can display several codimension 2 bifurcations: Takens-Bogdanov, Gavrielov-Guckenheimer, and double Hopf bifurcations.

Neural Computation 15, 1–56 (2003)

° c 2002 Massachusetts Institute of Technology

2

D. Hansel and G. Mato

1 Introduction Synchrony can be deŽned in several ways. In its broader deŽnition, the activities of two neurons are said to be synchronized if some kind of temporal correlations exist between them. In this sense, in any system of interacting neurons, some level of synchrony should exist. In this respect, observing synchrony between directly interacting neurons should not be surprising. In fact, several studies in the 1960s and 1970s were devoted to determining what the shape of the cross-correlations of a pair of neurons can tell us about their interactions (Abeles, 1990). The implicit assumption in this approach was that correlations are dominated by direct interactions between the neurons. Other types of synchronous states can emerge through cooperative effects. In these states, the correlations in the activity of a pair of neurons in the network depend weakly on their direct interaction (Hansel & Sompolinsky, 1996). Instead, they are mostly mediated by the inputs to the neurons coming from all other neurons belonging to the part of the network with which they interact. To deŽne this type of synchronous state, it is appropriate to consider macroscopic observables, for example, the instantaneous activity of the neurons averaged across assemblies of neurons (multi-unit activities) or local Želd potentials that presumably reect electrical activity induced by many neurons. In an asynchronous state, the temporal uctuations of these quantities (suitably normalized) are small—on the order of 1 / M where M is the size of the neuronal assembly. When synchrony emerges, these uctuations become Žnite even if the population is large. Understanding the conditions for the emergence of these states in the brain is one of the goals of recent research in the Želd of collective neuronal dynamics. Central issues concern the roles of the cellular properties of the neurons; the nature (excitatory, inhibitory), the strength, and the kinetics of the synaptic interactions; and the local and global architectures of the networks. The ways in which noise, heterogeneities, and spatial uctuations of connectivity patterns affect synchrony are also among the fundamental questions investigated. One possible approach to these questions is to study cooperative dynamical states in simpliŽed network models of local neuronal populations in the brain. Within limits, these models can be studied in depth analytically. Subsequently, these analytical investigations can be extended using numerical simulations beyond the limits that can be treated analytically. This approach has led to substantial progress in our understanding of the general mechanisms of neuronal synchrony. However, most of the theoretical studies have addressed the case of one population of neurons, much less is known about networks comprising several populations. The goal of this article is to study the emergence of synchronous activity in large networks consisting of two interacting populations of neurons, one excitatory and the other inhibitory. We investigate the conditions

Asynchronous States and the Emergence of Synchrony

3

under which asynchronous states (AS) become unstable due to interactions. More speciŽcally, we inquire: (1) Under which conditions can the network display a stable asynchronous state? Is this possible when the couplings are strong? (2) What are the different mechanisms through which the AS can become unstable in the network? (3) In what ways do they depend on the recurrent interactions in the excitatory neurons, the mutual interactions of the inhibitory neurons, and the excitatory-inhibitory feedback connections between the two populations? (4) What patterns of synchrony emerge when the AS is unstable? In particular, can one differentiate unambiguously between the regimes of parameters in which neurons spike in synchrony from regimes in which they burst in synchrony? (5) More quantitatively, what determines the period of the synchronous oscillations and the phase shift between the oscillations in the two populations? The article is organized as follows. The model is introduced in section 2. In section 3, the method of AS stability analysis is presented, and a spectral equation for the eigenmodes of the linearized dynamics is established. Assuming that the synaptic time constants of the excitatory and inhibitory synapses are identical simpliŽes the analysis of this equation. This allows us in section 4 to study the instabilities of AS to a large extent completely analytically. The various bifurcations of the AS are characterized. The results of section 4 are subsequently generalized in section 5. Results from numerical simulations are presented in section 6. Section 7 is devoted to a discussion of our results. 2 The model 2.1 The Quadratic Integrate-and-Fire Model. Near the onset of Žring, it can be shown that the detailed dynamics of any type I conductance-based model can be replaced by the reduced model (Ermentrout & Kopell, 1986; Ermentrout, 1996), ¡ ¢2 dV CN D A V ¡ V ? C I ¡ Ic , dt

(2.1)

where Ic is the rheobase of the neuron. Analytical formulas can be derived N the constant A, and V ?. Using these to compute the effective capacitance C, formulas, one can compute these parameters numerically as functions of the biophysical parameters of the full conductance-based model (see appendixes A and B). The solution of equation 2.1 diverges in Žnite time. This divergence corresponds to the Žring of an action potential. If one supplements equation 2.1 with the condition that the variable V has to be reset to ¡1 following a spike, this yields a reduced model that accurately describes the dynamics of the full conductance-based model in the limit I ! Ic . In

4

D. Hansel and G. Mato

this model, the relationship between the Žring rate º and the current I is given by p º D B I ¡ Ic ,

(2.2)

p N where B D A / p C. When I ¡ Ic is not small, this reduction is no longer exact. However, one can generalize it heuristically as follows. We assume that action potentials are Žred whenever V crosses some threshold, Vt (from below), and that it is immediately reset to a value Vr . The relationship between the discharge rate and the external current of the model is now given by p

p

Vt A Vr A 1 1 arctan( pI¡Ic ) ¡ arctan( pI¡Ic ) p . D pB º I ¡ Ic

(2.3)

The two parameters, Vt and Vr , are phenomenological. They are not determined by the exact reduction method near the Žring onset, and additional constraints are required to determine them. In the following, we impose that the º ¡ I curve of the quadratic integrate-and-Žre (QIF) model matches the º ¡ I curve of the full model as much as possible. It is convenient to rewrite the reduced model in terms of the dimensionless variables VN and IN deŽned by A VN D t0 (V ¡ V ?) CN A 2 IN D It0 , CN 2

(2.4) (2.5)

where t0 has a dimension of a time. The dynamics are governed by t0

¯ dV N 2 C IN ¡ INc . DV dt

(2.6)

The dynamical variable VN is reset to VNr D A ( Vr ¡ V ?)t 0 / CN whenever N A similar model has it reaches the threshold value: VN t D A ( Vt ¡ V ?)t0 / C. been used by Latham, Richmond, Nelson, and Nirenberg (2000) to study with numerical simulations a two-population network model. In this article, we use a QIF model with parameters derived from the Wang-Buszaki (WB) conductance-based model described in appendix A. Using the approach described above, one Žnds that Ic D 0.1601 mA / cm2 , V ? D ¡59.5462 mV, CN D 0.9467 m F / cm2 , and A D 0.012875 mS / cm2 / mV. Requiring that the º-I curve of the reduced and the full model agrees well up to Žring rates as large as ºmax D 100 spikes per second yields for the other parameters Vr ¡ V ? D ¡4.6 mV and Vt ¡ V ? D 33.2 mV (VN r D ¡0.626,

Asynchronous States and the Emergence of Synchrony

5

100

n (Hz)

75 50 25 0 0.0

0.5

1.0 1.5 I (mA/cm 2)

2.0

Figure 1: Firing rate of a neuron versus the external current. Circles: WangBusza ki model. Solid line: QIF model with A D 0.012875 mS / cm2 / mV, Vr ¡V ¤ D N D 0.9467 m F / cm2 . ¡4.6 mV, Vt ¡ V ¤ D 33.2 mV, Ic D 0.1601 muA / cm2 and C

VN t D 4.52). The º-I curves of the full model and the QIF model with these parameters are plotted in Figure 1. Clearly the Žt is excellent over a broad range of Žring rates. The traces of the membrane potential for neurons Žring at 50 spikes per second and 10 spikes per second are shown in Figures 2A and 2B, respectively, for this QIF model. For comparison, we have also plotted the traces of a WB neuron that Žres at the same frequencies. For a frequency of 10 Hz, the traces are very similar. When the frequency increases, the traces of the QIF neuron deviate from those of the WB neuron. 2.2 The Two-Population Network Model. We deal with a network of two heterogeneous populations: one excitatory and the other inhibitory. The dynamics of the neurons is modeled by equation 2.6, dV ia syn 2 th C Iia C Iia ( t ) ¡ Ia D Via dt

(2.7)

where i D 1, . . . , Na are the indices of the neurons in population a (a D E, I). For simplicity, the Žring threshold of the neurons Vt , the resetting value Vr , and the rheobase currents are the same for all the neurons. Time is measured in units of t0 D 10 ms. The current Iia represents the effect of an external stimulus that is assumed to be constant in time but different from one neuron to another. We consider the case where Na is very large and the external input currents to the neurons in population a are distributed according to a density syn distribution Qa ( I ) . The synaptic current, Iia ( t) , on the ith cell, is the sum of

6

D. Hansel and G. Mato A

V (mv)

20 20 60 100

0

20 t (msec)

40 B

V (mv)

20 20 60 100

0

100

200 t (msec)

300

400

Figure 2: Traces of a WB neuron (solid line) and QIF neuron (dashed line). Parameters of the model are the same as in Figure 1. (A) The neuron Žres at 50 Hz. (B) The neuron Žres at 10 Hz. syn

all contributions Iijab from all the presynaptic cells on it: syn Iia (t) D

X

syn

Iijab (t ).

(2.8)

jb syn

In this work, the current Iijab ( t) is modeled by syn

syn

Iijab ( t) D Gijab sjb (t) ,

(2.9)

syn

where Gijab measures the strength of the synapses that neuron j in population b makes on neuron i in population a and sjb evolves with time according to sjb ( t) D

X spikes

fb ( t ¡ tspike,j ) ,

(2.10)

Asynchronous States and the Emergence of Synchrony

7

where the summation is performed over all the spikes emitted by the presynaptic neuron with an index j at times tspike,j . The function fb ( t) is fb ( t ) D

t2b

µ ³ ´ ³ ´¶ 1 t t exp ¡ ¡ exp ¡ H (t) . ¡ t1b t1b t2b

(2.11)

Here, H ( t) is the Heaviside function, and the normalization is such that the integral of fb ( t) is one. The characteristic times t1b and t2b are the rise and decay times of the synapse, respectively. In case t1b D t2b D tb , one obtains the so-called alpha function (Rall, 1967): fb ( t ) D

³ ´ t t exp ¡ . tb tb

(2.12)

syn

Note that Gijab has the dimension of a current density and that excitatory syn

syn

(resp. inhibitory) interactions correspond to Gijab > 0 (resp. Gijab < 0). The synaptic current, which a cell i of the ath population receives from a cell j from the bth population, is syn

Iiab ( t) D

X

syn

Iijab ( t) ,

(2.13)

j

where a D E, I, i D 1, . . . , Na , j D 1, . . . , Nb , and Na is the number of neurons in the ath population. As a limiting case of a network with extensive connectivity, we assume full connectivity between the neurons, which simpliŽes considerably our analytical approach. For simplicity, we also assume that the strength of a synapse between two neurons depends on only the populations to which they belong. Therefore, syn

syn

Gijab D Gab .

(2.14)

In the case of an extensively connected network, the average number of synaptic inputs per neuron varies proportionally to the size of the system, and the synaptic strengths are varied in inverse proportion to the system size so that (see Hansel & Sompolinsky, 1996) syn

Gab D

gab , Nb

(2.15)

where the normalized synaptic strengths, gab are independent of the Na ’s. Our approach can be applied to the case where the kinetics of excitatory (resp. inhibitory) synapses are different when the postsynaptic neuron belongs to the excitatory or the inhibitory population. However, this entails

8

D. Hansel and G. Mato

an increase in the number of parameters of the model. Therefore, in this article, we consider that the synaptic rise time and decay time depend on only the nature, excitatory or inhibitory, of the synapses, and not on the synapse target. The rise and decay times of the synapses are denoted by t1b , t2b , b D E, I. We neglect axonal propagation delays. 3 The Asynchronous State and Its Stability 3.1 The Distribution of Neuron Firing Rates in the Asynchronous State. In the asynchronous state, the synaptic current is the same for all neurons in a population and does not depend on time. It is given by X syn syn (3.1) Iia (t) D gabºb ´ Ia , b D E,I

where ºb is the average Žring rate of the neurons in population b and gab is a 2 £ 2 matrix that characterizes the strengths of the synaptic interactions within and between the populations. Integrating the dynamical equations, equation 2.7, one calculates the Žring rate of each of the neurons in the network. This yields "Z ºia D

Vt Vr

Á ´ F Iia

# ¡1

dV syn

V2 C Iia C Ia ¡ Iath ! X C gabºb ¡ Iath ,

(3.2)

b D E,I

where F (I) D

p

I p arctan(Vt / I ) ¡ arctan(Vr / I ) p

(3.3)

if I > 0 and F ( I ) D 0 otherwise. These equations, together with ºa D

1 X ºia, Na i

(3.4)

determine the distribution of Žring rates in each population. Note that neusyn rons with an external current smaller than Iamin D Ith a ¡ Ia are below threshold and cannot Žre. The distribution Pa (º) of the Žring rates of the neurons in population a in the asynchronous state can be related to the distribution of external currents via the input-output relation, Z 0

P a (º) D Qa ( F ¡1 (º) ) |F ¡1 (º) | C d (º)

Imin a ¡1

dIQ a (I ) ,

(3.5)

Asynchronous States and the Emergence of Synchrony

9 A

Q(I)

1

0.5

0 -1

0

1 I (mA/cm 2 )

2 B

0.03

P(n)

0.02

0.01

0

0

25

50 n (Hz)

75

100

Figure 3: (A) Gaussian distribution of external current with a standard deviation s D 0.4. (B) Distribution of Žring rates for the QIF model (parameters as in Figure 1) and a distribution of currents as in A. The average Žring rate is ºN D 30 Hz. The delta function at º D 0 is not shown.

where the last contribution comes from the neurons that receive a total input too small to make them Žre. The distribution of Žring rates is plotted in Figure 3 for a gaussian distribution of currents:

Qa ( I ) D p

³ ´ ( I ¡ Ia ) 2 exp ¡ . 2sa2 2p sa2 1

(3.6)

We emphasize that the distribution of Žring rates can be kept constant even if the coupling strengths change. This can be done simply by adjusting the distribution of external currents with an offset that depends on the coupling and the average Žring rate of the population. In this way, one can separate the effect of coupling strength and the effect of the average rate on synchronization properties.

10

D. Hansel and G. Mato

3.2 Stability Analysis of the Asynchronous State. The stability of the AS in a homogeneous network of leaky integrate-and-Žre neurons has been studied by Abbott and van Vreeswijk (1993). We have generalized their approach to the case of heterogeneous network of QIF neurons (Hansel & Mato, 2001). For each of the active neurons in the population a, the variable yia can be deŽned as Z Via dx P (3.7) yia D ºia , 2 C I C x ia Vr b gab ºb where ºi and ºa are the Žring rates of neuron i and the average population Žring rate in the AS, respectively, and gab are the strengths of the synapses from population b to population a. This dynamical variable evolves in time according to X dyia D ºia C gab Kb ( yia ) 2 dt b

b ( t) ,

(3.8)

where 2 b ( t) D

Nb 1 X (ºjb (t) ¡ ºb ) Nb j D1

(3.9)

is the deviation of the Žring rate of neuron j at time t from its value in the AS, and Ka ( yia ) D

ºia P . 2 CI C Via i b gabºb

(3.10)

The probability density, r a ( y, º, t ) satisŽes the continuity equation, @r a ( y, º, t) @t

D ¡

@Ja ( y, º, t ) @y

,

(3.11)

where the ux Ja ( y, º, t) is Á X Ja ( y, º, t) D ºa C gab Kb ( y )2

! b ( t)

r a ( y, º, t) .

(3.12)

b

In the asynchronous state, Ja ( y, º, t) D ºa . The deviations from this value, ja ( y, º, t) D Ja ( y, º, t) ¡ ºa , due to perturbations from the AS state satisfy @ja ( y, º, t) @t

D

X b

gab Kb ( y )

d2

b ( t)

dt

¡ ºa

@ja ( y, º, t) @y

.

(3.13)

Asynchronous States and the Emergence of Synchrony

11

For synaptic interactions modeled with a difference of exponentials (see equations 2.11), it is straightforward to show that equation 3.13 is equivalent to a ( t) D ¡c 1a2 a ( t) C ha ( t) dt X dha ( t) D ¡c 2a ha ( t) C c 1ac 2a ja (1, º, t) , dt º

d2

(3.14) (3.15)

where c ka D 1 / tka for k D 1, 2. The perturbation ja has to be evaluated at y D 1 because this corresponds to the crossing of the threshold, V D Vt (see equation 3.7). The AS is stable if small perturbations from the AS eventually decay. Assuming that ja ( y, º, t) , 2 a ( t) , ha (t) are proportional to exp(lt) and integrating equation 3.13, one Žnds after a straightforward but tedious calculation that l satisŽes Y µ (l C c 1a ) (l C c 2a ) ¡ c 1ac 2a gaa Ua ¶ (3.16) D gEI gIE UI UE , c 1ac 2a aD E,I where Z Ua (l) D

1 0

dº

Ha (º) 1 ¡ exp(¡l /º)

(3.17)

and Pa (º)º Ha (º) D 2F ¡1 (º)

" 1 ¡ exp(¡l /º)

(cos( A C w ) C sin(A C w ) Aº ) ) (cos(w ) C sin(w ) Aº ) l ¡ exp(¡l /º l C 2 ( 1 C Aº/ l)

# (3.18)

with p A D 2 F ¡1 (º) /º

p w D 2 arctan(Vr / F ¡1 (º) ) .

(3.19) (3.20)

The spectral equation, equation 3.16, determines the eigenvalues of the dynamical equations linearized around the AS. Note that this spectral equation also holds if a fraction of the neurons are subthreshold and do not Žre in the AS. Indeed, an inŽnitesimal perturbation cannot make these neurons Žre. Therefore, they do not contribute to the destabilization of the AS.

12

D. Hansel and G. Mato

3.3 The Spectral Equation at the Onset of an Instability. The AS is stable if all the eigenvalues, the solutions to equation 3.16, have a negative real part. Therefore, continuous onset of instabilities occurs when at least one of the eigenvalues crosses the imaginary axis when some parameter is changed. At this onset, l D im and singularities appear in the integral in equation 3.17. These singularities can be isolated, and one Žnds: ³

´ p m A1a C A2a ¡ i A3a , 2 m 2

(3.21)

dºHa (º)

(3.22)

Ua ( im ) D where Z A1a D A2a D A3a D

1 0

1 X

± m ² ± m ²2 2p n 2p n

Ha

nD 1

1 Z X nD 1

p

dy ctg

± y ² Ha ( Cm ) y 2np 2

¡p

Z

C

p

dy ctg 0

(3.23)

( y C 2np ) 2

± y ² Ha ( m ) y 2

y2

.

(3.24)

These integrals are nonsingular. In general, they have to be evaluated numerically. The onset of instabilities of the AS is determined by taking the real and imaginary part of equation 3.16. The simultaneous solution of these equations determines m and the coupling at the onset of instabilities as a function of the other parameters of the model. At instability onset, the synaptic currents in the unstable mode oscillate with a frequency given by the imaginary part of the eigenvalue, m . The dephasing d between the oscillations of the two populations is exp(id) / 2

/

I 2 E

D

gEE UE ¡ (l /c 1E C 1) (l /c 2E C 1) . | gEI |UI

(3.25)

A positive (resp. negative) value of the phase lag d means that the oscillation of the inhibitory population is in advance (resp. delayed) over the excitatory population. The solutions of equation 3.16 can be classiŽed according to the temporal 6 0, the properties of the corresponding unstable modes of instability. If m D instability occurs through a Hopf bifurcation (Strogatz, 2000). This can happen in two ways. In the Žrst, called a supercritical Hopf bifurcation, the AS loses stability, and simultaneously stable oscillations of the network activity appear with vanishingly small amplitude and Žnite frequency. The latter

Asynchronous States and the Emergence of Synchrony

13

is equal to m / 2p . The phase lag between the inhibitory and the excitatory population activity is d (see equation 3.25). In the second, called a subcritical Hopf bifurcation, the AS loses its stability by coalescing with an unstable state in which the activity of the network oscillates. This is a necessary (but not sufŽcient) condition for having a region in the parameter space where the stable AS coexists with two states of synchronous oscillations—one stable and one unstable. The stable oscillatory state persists in some regions where the AS is unstable and the unstable oscillatory state no longer exists. The values of m and d determined by the stability analysis now correspond to the frequency and the phase lag between the two populations in the unstable oscillatory state at bifurcation. However, if the amplitude of the stable oscillations at the bifurcation is small, these values should also provide a good estimate of the frequency and the phase lag in the stable oscillatory state near the bifurcation. This point needs to be veriŽed with numerical simulations (see section 6). Finally, if m D 0, there is a saddle node (SN) bifurcation (Strogatz, 2000). To study how the stability of the asynchronous states depends on the synaptic properties, we construct phase diagrams for a Žxed distribution of the Žring rates of the two populations, PE (º) and PI (º) , taking as parameters the strength of the four interactions. This implies that when the interaction strengths are changed, the external average inputs (or the Žring thresholds) have to be modiŽed accordingly to satisfy equation 3.2 for the two populations. Because of the normalization of the synaptic interactions, changing the synaptic time constants does not affect the Žring-rate distribution in the AS. To study their role on the emergence of synchrony, the synaptic time constants can be varied while maintaining constant all the other parameters. 4 Synchrony in a Two-Population Model: The Symmetric Case A detailed analysis of the conditions of the AS stability as a function of synaptic coupling strength, synaptic kinetics, and distributions of Žring rates is a formidable task. In this section, we focus on a situation where, t1E D t1I D t1 , t2E D t2I D t2 , and P E (º) D PI (º). We term this situation the symmetric case. It turns out that to a large extent, this case can be investigated analytically and that the results thus obtained are highly instructive for understanding the general properties of the two-population network. 4.1 The Instabilities of the Asynchronous State. In the symmetric case, for all a, Ua (l) D U (l) where U (l) is given by equation 3.17. Therefore equation 3.16 becomes (im C c 1 ) 2 ( im C c 2 ) 2 ¡c 1c 2 ( im C c 1 ) ( im C c 2 ) UT C c 12c 22 U 2 D D 0,

(4.1)

where U is a function of m , T D gEE C gII and D D gEE gII ¡ gEI gIE . The latter parameters are the trace and the determinant of the matrix gab , respectively.

14

D. Hansel and G. Mato

Figure 4: Phase diagram of a two-population QIF network in the symmetric case. The determinant and the trace of the coupling matrix are D D gEE gII ¡gEI gIE and T D gEE C gII , respectively. The standard deviation of the current distribution is sE D sI D 0.1. The average Žring rates in the AS are ºE D ºI D 50 Hz. The synaptic time constants are t1E D t1I D 1 msec, t2E D t2I D 4 msec. In the gray region, the AS is stable. See the text for an explanation of the different lines. The dotted line is the limit of L4 in the limit of inŽnite synaptic time constants (see section 5.3).

Therefore, one can completely describe the role of the synaptic strength in a two-dimensional plane spanned by these two effective parameters. Let us consider point X 0 D (D D 0, T D 0). A particular realization of point X 0 is obtained in the absence of coupling. Therefore, at X 0, the AS is marginally stable if the system is homogeneous, that is, for sE D sI D s D 0. If one introduces any level of heterogeneities, the AS becomes stable in some region around this point. The size of this region depends on s (or, equivalently, on the dispersion of the Žring rates in the AS). This result holds independent of the parameters and the Žring rates distribution provided the symmetry is preserved. It shows that there is always some domain in the D , T plane where the AS is stable. What are the boundaries of this region, and what are the instabilities on these boundaries? An example of the network phase diagram is shown in Figure 4. It was obtained by solving equation 4.1 for synaptic rise and decay times of 1 msec and 4 msec, respectively, the average Žring rate of the two populations, º D 50 Hz, and the standard deviation of the external input distribution, s D 0.1. The distribution of Žring rates for these parameters is shown in Figure 3. The region of stability of the AS is bounded by four lines.

Asynchronous States and the Emergence of Synchrony

15

Line L1 is deduced from equation 4.1 by taking m D 0. It is straightforward to see that the equation of the line is given by

T D U (0) D C 1 / U (0)

(4.2)

with Z U (0) D

1 0

dº

µ ¶ sin(A C w ) ¡ sin(w ) P (º)º C 1 . 2F¡1 (º) A

(4.3)

For the parameters of Figure 4, U (0) D 0.36. On this line, an SN bifurcation occurs. Above the line, the AS with ºE D ºI D 50 Hz is unstable through an SN bifurcation. At the instability onset, the unstable mode grows exponentially but does not oscillate. This corresponds to a change in the average Žring rates of the two populations, ºE and ºI , but not to the emergence of synchrony in the network. Before discussing the general meaning of transition line L2, we consider two cases: 1. P ´ gEI gIE D 0, gII D 0, gEE > 0. The two populations are noninteracting, and the inhibitory neurons are decoupled. For such parameters, the states of the network are represented in the (D , T ) plane by the line segment: D D 0, T > 0. There is one instability with real eigenvalue for the unstable mode on this segment at point XE D (0, 1 / U (0) ) , which belongs to L1. All other instabilities in this segment occur in the region above L1 (not shown in Figure 4). Therefore, for the parameters of the Žgure, the excitatory population cannot develop synchrony by itself. 2. P D gEI gIE D 0, gEE D 0, gII < 0. As in the previous case, the two populations are noninteracting, but now the excitatory neurons, rather than the inhibitory ones, are decoupled. This situation is mapped in the phase diagram on segment D D 0, T < 0. An instability with a pure imaginary eigenvalue l D im c occurs on this segment at point XI D (0, gc ) where gc and m c are solutions to the complex equation: ( im c C c 1 ) ( im c C c 2 ) D c 1c 2 gc U ( im c ).

(4.4)

Solving this equation yields gc D ¡0.98, m c D 3.02, that is, m c / (2p ºI ) ¼ 0.96. Therefore, at XI , the mutual inhibitory interactions I-I become strong enough to destabilize the AS with a Hopf bifurcation where synchrony emerges on the timescale of the spiking average period. Other solutions of equation 4.4 exist, but they are not relevant since they occur in the region below XI , where the AS is already unstable. This mechanism, where synchrony of neural activity emerged in one population of neurons, has been studied by several authors

16

D. Hansel and G. Mato

(van Vreeswijk,1996; Wang & Busz Âaki, 1996; White, Chow, Ritt, SotoTrevino, ˜ & Kopell, 1998; Neltner, Hansel, Mato, & Meunier, 2000; Golomb & Hansel, 2000; Golomb, Hansel, & Mato, 2001). 6 We now consider the general case, P D gEI gIE D 0, in which the two populations interact. It is straightforward to see that if T and D satisfy the relationship

T D

D C gc , gc

(4.5)

then equation 4.1 is satisŽed with m D m c . Equation 4.5 deŽnes a straight line (L2) in the (T , D ) plane on which a Hopf bifurcation occurs. Obviously XI belongs to this line. The intersection of L1 and L2 deŽnes a point, A ¼ (¡2.72, 1.8) , where the real parts of the two eigenvalues vanish. One corresponds to an SN bifurcation and the other to a Hopf bifurcation. At that point, the AS becomes unstable through a codimension 2 bifurcation called a Gavrielov-Guckenheimer bifurcation (Kuznetsov, 1998). The points on L1 and L2 to the left of A are not relevant as instability onset since they are inside a region where the AS is already unstable. The pattern of synchrony into which population a (a D E, I) settles depends in general on the ratio Ra D

m . 2p ºa

(4.6)

Here Ra is close to one, and neurons Žre on the average about one spike per period in the unstable mode. However, the phase relationship between the spikes and the collective oscillations can change from cycle to cycle. Consequently, the phase distribution is broad but unimodal. It can be easily shown that on L2, 2 I

2 E

D

gEE ¡ gc . | gEI |

(4.7)

Therefore, on this line, the phase lag is constant, d D 0, that is, at the instability onset, the activities in the two populations oscillate in phase. Point B on L2 deŽned by D D g2c ´ DB , T D 2gc ´ TB is a multicritical point where two eigenvalues have a vanishing real part. This can be shown by expanding equation 3.16 in the vicinity of B. One Žnds two other critical lines, L3 and L4, tangent to L2 in B. SufŽciently close to B, these lines are given by the equation

T ¡ TB D b1 ( D ¡ DB ) C b2 (D ¡ DB ) 2 ,

(4.8)

Asynchronous States and the Emergence of Synchrony

17

where b1 D

1 gc

b2 D ¡

|W 0 (m c ) | 2 . 4g3c <e(W 0 (m c ) ) 2

(4.9) (4.10)

Here we have deŽned W (m ) D c 1c 2 U ( im ) / ( im C c 1 ) ( im C c 2 ) , and W 0 (m ) is the derivative of W with respect to m . For the parameters of Figure 4, b1 D ¡1.02 and b2 D 47.9. Hence, the expansion, equation 4.8, truncated at second order, is a good approximation to L3 and L4, only very close to B. This is why it seems to be a cusp in point B. Beyond B, L2 is no longer relevant for the instability of the AS. We conclude that on AB, the AS undergoes an instability to spike-to-spike synchrony. This instability, driven by the I-I interactions, is the extension in the framework of two populations of the instability that occurs in one population of inhibitory neurons. The interaction between the two populations makes this mechanism less efŽcient. Indeed, from equation 4.5, one sees that when the feedback between the two populations increases, a stronger mutual inhibitory coupling, gII , is required for the AS to be destabilized on L2. Point D is deŽned as the intersection of L4 and L1. At that point, the AS is unstable through a codimension 2 bifurcation called a Takens-Bogdanov bifurcation (Kuznetsov, 1998). The coordinates of D can also be obtained by expanding equation 3.16 in the limit m ! 0. One Žnds that in D, D D U (0) 2 , T D 2U (0) . The frequency of the unstable mode, m , varies continuously on L3 and L4, as shown in Figure 5. On L4, m decreases continuously when D increases, from m D m cII D 3.02 at B, to m D 0 at D. On L3, m increases slowly when D increases. In the limit of large D , m converges asymptotically to a Žnite value, m 1 , and L3 is asymptotic to the straight line deŽned by T D D / g1 C gc , where g1 and m 1 satisfy the equation (im 1 C c 1 ) (im 1 C c 2 ) D c 1c 2 g1 U ( im 1 ).

(4.11)

Note that this equation is identical to the spectral equation describing the stability of the AS in one population of neurons. For the parameters of Figure 4, m 1 ¼ 3.82 and g1 ¼ 6.5. The ratio between the frequency of the unstable mode and the average Žring rate of the neurons, R ´ RE D RI (see equation 4.6), is plotted in Figure 5. This ratio is larger than 1 on L3. This means that the unstable mode oscillates faster than the average frequency of the neurons. However R increases by less than 25% on L3. Therefore, the synchrony that emerges is on the timescale of the spikes. In contrast, on L4, R varies from R ¼ 1 near B to R D 0 in the vicinity of D. This means that along L4, the pattern

18

D. Hansel and G. Mato

Figure 5: The ratio R D m / 2p º versus the determinant of the coupling matrix, D, on lines L3 (dashed line) and L4 (dashed-dotted line) shown in Figure 4. Points B and D correspond to B and D shown in Figure 4.

of synchrony changes continuously from spike-to-spike synchrony to burst synchrony in which the neurons have the time to Žre several spikes on average during one period of oscillation of the unstable mode. The phase lag, d, on L3 and L4 can be expressed analytically as a function of D , T , and gEE . From equation 4.1 one Žnds that ( im C c 1 ) (im C c 2 ) D

p c 1c 2 U (im ) ( T § T 2 ¡ 4D ). 2

(4.12)

Analyzing the numerical solutions of the spectral equation, one Žnds that it is negative on L4 and positive on L3. Substitution in equation 3.25 gives for the phase lag the equation p =m ( T 2 ¡ 4 D ) d D § arctan , gEE ¡ gII

(4.13)

where the positive (resp. negative) sign is for the phase lag on L4 (resp. on L3). The condition T 2 ¡ 4D D 0 deŽnes a line in the phase diagram to which B, D and the point T D D D 0 belong. Therefore, T 2 ¡ 4D < 0 on L3, except in B and D where T 2 ¡ 4D D 0 (see deŽnition of points B and D). Since gEE ¡ gII > 0, cosd > 0.

(4.14)

The phase lag, d, is a function of the three variables gEE , gII , and gEI gIE and not only the reduced variables T and D . In contrast to what happens on L2,

Asynchronous States and the Emergence of Synchrony

19 A

1.6

d

1.2 0.8 0.4 0 -8

0

-4 g

4 g

II

8

EE

B

0

d

-0.4 -0.8 -1.2 -1.6 -8

-4 g

II

0

4 g EE

8

Figure 6: Phase lag of the population oscillations. Solid lines were obtained from equation 3.25. The squares are results from numerical simulations of a twopopulation network with NE D NI D 1600. The positive (resp. negative) part of the x-axis corresponds to variation of gEE with gII D 0 (resp. gII with gEE D 0). (A) In the vicinity of L3. The simulations were done keeping a Žxed distance equal to 1 from L3 and with gIE D ¡gEI . The positive (resp. negative) part of the x-axis corresponds to variation of gEE (resp. gII ). The standard deviation of the current distribution is s D 0.1. Other parameters are as in Figure 4. (B) In the vicinity of L4. The simulations were done keeping a Žxed distance equal to 2 from L4 and with gIE D ¡gEI . The standard deviation of the current distribution is s D 0.4. Other parameters are as in Figure 4.

on L3 and L4, it changes continuously and nonmonotonically. It vanishes in B and D. This is shown, for example, in Figure 6A, where d is plotted against gEE (with gII D 0) along the two-line segment L3 and L4 and against gII (with gEE D 0) for the same two lines. Equation 4.14 implies that in general, |d| < p / 2. In particular, the activity of the two populations can never be in antiphase. From equation 4.13, one

20

D. Hansel and G. Mato

Figure 7: Phase diagram of a two-population QIF network in the symmetric case. The determinant and the trace of the coupling matrix are D D gEE gII ¡gEI gIE and T D gEE C gII , respectively. The standard deviation of the current distribution is sE D sI D 0.4. Other parameters are as in Figure 4. In the gray region, the AS is stable. See the text for an explanation of the different lines. The dotted line is the limit of L4 for very large synaptic time constants (see section 5.3).

sees that the maximum phase lag at the onset of instability, |d| D p / 2, is achieved only for gEE D gII D 0 (d D ¡p / 2 on L3 and d D p / 2 on L4). Finally, on line L3, in the limit of very large D and T , the phase lag returns to 0, as can be seen in equation 3.25 using equation 4.11. These results are also conŽrmed in the example of Figure 6B. The two line segments, L3 and L4, intersect at point C where D ( C) ¼ 7.9 and T ( C) ¼ 4.3. Note that m has different values on the two lines at C, as can be seen from Figure 5. Therefore, at C, the AS is unstable through a double-Hopf bifurcation (Kuznetsov, 1998). Whether L3 or L4 is the border of stability of the AS depends on T . This is shown in Figure 4 for T > T ( C) . The AS loses stability on L4, whereas for T < T ( C) , this occurs on L3. 4.2 The Effect of Heterogeneities. We now consider how the phase diagram for the stability of the AS is modiŽed when the level of heterogeneities varies. Figure 7 shows the phase diagram in the symmetric case for sE D sI D 0.4, with all other parameters the same as in Figure 4. Comparing these two Žgures, one sees that in spite of a signiŽcant increase in the level of heterogeneities, the transition line (L1) has moved only slightly. In contrast, L2 has substantially moved toward more negative values of T and D . This can be easily understood because to compensate for the increased desynchronizing effect of the heterogeneities, stronger synapses are required to destabilize the AS. Line segment L4 is barely modiŽed in the neighborhood of D, its crossing point with L1. A stronger effect occurs in the vicinity of

Asynchronous States and the Emergence of Synchrony

21

Figure 8: The ratio R D m / 2p º versus the trace of the coupling matrix, T , on line L4 shown in Figure 7. Points B and D correspond to B and D shown in Figure 7.

B, which moves this point toward more negative values of T and D . Most of the changes in line segment L4 when s increases can be explained by this stronger effect. A more dramatic change occurs for the line segment L3. Indeed, when the heterogeneities are strong enough, L3 moves completely to the right of L4. Moreover, on L4, T increases very slowly with D . Therefore, all L3 becomes located well into the region where the AS is already unstable, and this instability scenario is now irrelevant. As in Figure 4, along line L4, the frequency of the unstable mode depends continuously on the location. However, in contrast to that case, L4 is relevant as an instability line all the way between points B and C. In Figure 8, we plotted the ratio of the frequency of the emerging oscillations to the average Žring rate showing the continuous and monotonic variation of this quantity along L4 from a value slightly larger than 1 at B corresponding to spike-tospike synchrony to 0 at D in the vicinity of which the network Žres in synchronous bursts. To characterize further how the instabilities of the AS depend on the level of heterogeneities, we computed the value of D on the four lines L1, L2, L3, L4 as functions of the heterogeneity level, s, for T with all other parameters constant. The results are shown in Figure 9. As already noted from the comparison of Figures 4 and 7, the location of L1 depends only very weakly on the level of the heterogeneities. In contrast, the destabilization of line L2 is much more sensitive to this parameter. In Figure 9B, the line segment L3 is clearly much more sensitive to heterogeneities than line segment L4. Indeed, the coupling strength required on the former line increases much more rapidly with s than on the latter line. A consequence of this difference in sensitivity to heterogeneities is that, as already noted, the two line segments

22

D. Hansel and G. Mato

Figure 9: Determinant of the coupling matrix, D, versus the standard deviation, s on the bifurcation lines L1, L2, L3, L4 for T D 1. The synaptic time constants are as in Figure 4. (A) L1 (solid line) and L2 (long dashed line). (B) L3 (dashed line) and L4 (dashed-dotted line).

L3 and L4 interchange their positions in the (D ,T ) phase diagram when s is large enough. This happens at s ¼ 0.15. For s larger than this value, line L3 is located beyond the instability line L4 in a region where the AS is already unstable. Therefore, in that case only, one instability scenario driven by the E-I-E interaction loop is relevant. 4.3 The Effect of the Firing Rate. For a Žxed architecture and coupling strength, the average Žring rate of the network depends on the external input. To investigate how the stability of the AS depends on this input, we studied the solutions of the spectral equation as a function of the average Žring rate, keeping the variance of the threshold distribution constant, s D 0.1. Figure 10A shows how the minimal coupling strength gc required for the destabilization of the AS in one inhibitory population determined by equa-

Asynchronous States and the Emergence of Synchrony

23

Figure 10: (A) The synaptic coupling, gII , at which the AS becomes unstable in one inhibitory population versus the average Žring rate of the neurons. The standard deviation of the input distribution is s D 0.1. The synaptic time constants are as in Figure 4. (B) The determinant, D, on the bifurcation lines L3 (dashed line) and L4 (dashed-dotted line) versus the average Žring rate of the neurons in a two-population network (symmetric case) for T D 1. Other parameters are as in Figure 4.

tion 4.4 varies with the average Žring rate. One Žnds that gc is a nonmonotonic function of the average Žring rate. The critical coupling is minimum (gc D ¡0.98) for a Žring rate of º D 50 spikes per second. Note, however, that the extremum of the curve is very shallow. Already for gc D ¡1.5, the AS is unstable in a broad range of average Žring rates. Figure 10B plots D c , the value of D at the instability onset on L3, and L4, for T D 1. This Žgure shows that on L3, Dc decreases monotonically with increasing Žring rate. Moreover, this instability disappears for º < ºmin D 20 spikes per second at which value Dc diverges. In contrast, on L4, Dc is a nonmonotonic function of the average Žring rate and exists at all values of the Žring rate. The two lines intersect for º ¼ 40 spikes per second. Below

24

D. Hansel and G. Mato

this value of the Žring rate, the relevant instability of the AS is on L3, whereas above this value, it is on L4. 4.4 Summary of the Results in the Symmetric Case. Our analysis of the symmetric case reveals that in a two-population heterogeneous network, the AS can lose stability in four ways. Two of these are not speciŽc to two-population networks and are an extension in this framework of instabilities that already exist for one population: the rate instability that occurs on L1 and the spike-to-spike synchrony instability that occurs on L2. At the onset of the former type of instability, which depends only weakly on heterogeneities, no synchrony occurs. In the latter case, which is sensitive to heterogeneities, at the onset of the instability, activities in the two populations are synchronized on the timescale of the spikes and oscillate in phase. The two other types of instability that occur on L3 and L4 are speciŽc to two-population networks. On L3, the activities in the two populations oscillate at a frequency on the order of the average Žring rate, and the inhibitory population is oscillating ahead of the excitatory population. On L4, the frequency of the population activity oscillations can become very slow. This happens if both the recurrent excitation and the recurrent inhibition (the feedback between the two populations) are strong. On L4, the oscillation of the excitatory population is ahead of the oscillation of the inhibitory population. Finally, L3 and L4 differ in the way they depend on the level of heterogeneities. L3 becomes irrelevant if the heterogeneities are large or the average Žring rates are small. 5 Synchrony in Two-Population Networks: The General Case If the distributions of the Žring rates of the two populations are different in the AS or the synaptic time constants of the excitation and inhibition are not the same, the stability of the AS (at given Žring-rate distributions) depends on three synaptic coupling parameters gEE , gII , gEI gIE and also on the synaptic time constants. 5.1 The Effect of the Synaptic Coupling Strength. Figure 11A displays the phase diagram of the AS in the plane ¡gEI gIE , gEE when gII D 0. The synaptic time constants are t1E D 1 msec, t2E D 3 msec, t1I D 1 msec, and t2I D 6 msec. The average Žring rate of the excitatory (resp. inhibitory) population is 20 Hz (resp. 40 Hz), and sE D sI D 0.2 for both populations. As in the symmetric case, if the recurrent excitation is strong enough, SN instability occurs. In the plane ¡gEI gIE , gEE , this instability occurs on a straight line, the equation of which is

gEE D

1 ¡ gEI gIE UI (0) , UE (0)

(5.1)

Asynchronous States and the Emergence of Synchrony

25

Figure 11: Phase diagram in the asymmetric case. The average Žring rate of the excitatory (resp. inhibitory) population is ºE D 20 Hz (resp. ºI D 40 Hz). The synaptic time constants are t1E D t1I D 1 msec, t2E D 3 msec, t2I D 6 msec. The standard deviations of the input distributions are sE D sI D 0.2. The AS is stable in the gray region. In this Žgure, only the physical branches of the instability lines are drawn. In particular, L3, which branches at B, is not drawn. (A) gII D 0. A saddle-node bifurcation occurs on L1 (solid line) and a Hopf bifurcation on L4 (dashed-dotted line). (B) gEE D 0. Hopf bifurcations occur on L2 (long-dashed line) and L4 (dashed-dotted line). The difference between these two bifurcation lines is explained in the text. At E, the instability of the AS is due to the interactions between the two populations since gEE D gII D 0 at that point.

26

D. Hansel and G. Mato

where Ua (0) , a D E, I is given by equation 4.3, replacing the distribution P by Pa , the Žring-rate distribution in population a. For the parameters of the Žgure, one Žnds UE (0) D 0.53 and UI (0) D 0.4. When gEE increases, this instability can be prevented by increasing the negative feedback between the two populations measured by gEI gIE < 0. When this feedback is strong enough, the AS becomes unstable through a Hopf bifurcation, which occurs on the line to the right of the phase diagram. On this line, the period of the population rhythm varies continuously from m ¼ 1.25 at its intersection with the x-axis, E, (gEE ( E ) D 0) to m D 0 at D. The value of m at E corresponds to a pattern in which the inhibitory (resp. excitatory) neurons Žre approximately two spikes (resp. one spike) per period of the population oscillation. Near D, it corresponds to a pattern in which neurons tend to Žre synchronous bursts with many spikes. The properties of the instability on this line are similar to those found on line L4 in the phase diagrams of the symmetric case. Because of these similarities, we have denoted this line by L4 here as well. Figure 11B displays the phase diagram of the model in the plane ¡gEI gIE , gII when gEE D 0. The other parameters are the same as in Figure 11A. The region of stability of the AS is bounded by two lines. One, L2, corresponds to the destabilization of the AS when gII is strong enough. Like line L2 in Figures 4 and 7, this line corresponds to the emergence of synchrony through the I-I interaction. In contrast to the symmetric case, here m / 2p and d are not constant on L2. However, m / 2p always remains just slightly smaller, by less than 4%, than the average Žring rate of the inhibitory population, and d is always smaller than 15% of the total period. Another difference with the symmetric case is that L2 is not straight anymore. However, it deviates only slightly from a straight line. In similarity with the symmetric case, the slope of L2 is negative. This means that the feedback between the two populations plays against synchrony through the I-I coupling. A stronger I-I coupling is required to compensate for this feedback. For strong enough feedback, gEI gIE , the AS can lose stability even if gII is small. This transition occurs on the line to the right in Figure 11B. On this line, the population frequency varies signiŽcantly. Indeed, at B, m / 2p ¼ 40 Hz, and it is two times smaller at point E. Embedded into the three-dimensional space, (gEE , gII , gEI gIE ), this line connects to L4 from Figure 11A. We also denote this line by L4 since the instabilities occurring on these two lines can be thought of as corresponding to a similar synchrony mechanism. Note that the slope of L4 in Figure 11B is negative. In other words, increasing gII plays against the destabilization of the AS through this mechanism. 5.2 The Effect of Synaptic Time Constants. In this section, we analyze the effect of synaptic kinetics on the stability of the AS for Žxed distributions of Žring rates. First, we focus on the effect of the decay times.

Asynchronous States and the Emergence of Synchrony 0.02

g2E(1/msec)

A

0.01

0.00 0.0

AS

0.1

0.2

g2I(1/msec)

g2E(1/msec)

0.8

27

B

0.6

AS

0.4 0.2 0.0 0.0

0.1

0.2

g2I(1/msec)

Figure 12: Phase diagrams in the plane c 2I ´ 1 / t2I , c 2E ´ 1 / t2I for two different strengths of the I-I interactions. The average Žring rate of the excitatory and inhibitory populations are ºE D 20 Hz and ºI D 30 Hz, respectively. (A) gII D 0. (B) gII D ¡7.4, Other parameters are t1E D t1I D 1 msec, gEE D 6, gEI gIE D ¡36 and sE D sI D 0.2. In the gray region, the AS is stable. On L2 (long-dashed line) and L4 (dashed-dotted line), Hopf bifurcations occur. Along L4, the frequency of the unstable mode m decreases with c 2I . For instance, in A, m / 2p varies from 11 Hz for c 2I D 0.3 msec ¡1 to 0 for c 2I D 0. Along L2, m remains Žnite and, its variation is smaller.

Figure 12 displays the phase diagram in the (c 2I ,c 2E ) plane for gEE D 6, gEI gIE D ¡36 and two different values of gII . We consider Žrst the case where gII D 0. In the limit t2E ! 1, the AS is always stable. This is because in that limit, the excitatory input to the inhibitory population is not modulated in time. Therefore, it is equivalent to a positive constant current that excites the inhibitory population. The only way synchrony can occur in this case is through the I-I interactions, which we have assumed to be zero. For Žnite t2E , the AS is unstable provided t2I is sufŽciently large. This deŽnes a transition line c 2E D f (c 2I ) where function f (c ) is an increasing function that vanishes linearly when c ! 0. On this line, which is plotted in Figure 12A, m is a decreasing continuous

28

D. Hansel and G. Mato

function of c 2I . It goes to 0 in the limit c 2I ! 0. Since the decay of the inhibition cannot be faster than its rise time (see equation 2.11), m cannot be larger than m D 0.79, its value for t2I D 1 msec, on the transition line in Figure 12A. Therefore, all along the line, the transition leads to a synchronous bursting state. The bursts are short for fast inhibition and long when the inhibition is slow. In Figure 12A, the AS stabilizes for fast inhibition and slow enough excitation. In particular, the fastest excitation compatible with a stable AS (attained for very fast inhibition with t1I D t2I D 1 msec) is t2E ¼ 50 msec, in the range of NMDA synaptic time constants. The phase diagram in the (c 2I ,c 2E ) plane remains qualitatively the same as in Figure 12A, provided that gII is sufŽciently small. If gII is large enough, a second instability line appears in the region of the phase diagram corresponding to fast inhibition and slow excitation. This is because for a large enough gII , the I-I interaction can induce spike-to-spike synchrony in the inhibitory population. This situation is shown in Figure 12B. Note that the transition line found for gII D 0 (see Figure 12A) is still present but has been shifted to much higher values of c 2E . In contrast to what happens for small values of gII , when the inhibition is fast (t2I < 9 msec), the AS is destabilized for slow enough excitation. For instance, for t2I D 6 msec, this happens for t2E larger than 20 msec. Note that for the parameters of Figure 12B, the AS is stable if the excitatory and inhibitory synapses are in a range corresponding to AMPA and GABAA synapses, respectively. 5.3 The Limit of Slow Synapses. The spectral equation, equation 3.16, can be simpliŽed if one assumes that the synapses have inŽnitely long decay time constants. We assume that c 2E D gcN2E and c 2I D gcN2I where g ¿ 1 and cN2E , cN2I are Žnite and we make the ansatz l D glN .

(5.2)

Substituting in equation 3.16, expanding at the leading order in g, and usR1 ing the fact that 0 dyK ( y ) D F0 (F ¡1 (º) ) , one Žnds that at the onset of an instability lN D imN with mN solution of the equation, ( ( imN C cN2E ) ¡cN2E gEE UE ) ( ( imN C cN2I ) ¡cN2I gII UI ) D gEI gIEcN2EcN2I UE UI ,

(5.3)

where the quantities Z UN a D

1 0

dºP a (º) F0 (F ¡1 (º) )

do not depend on mN .

(5.4)

Asynchronous States and the Emergence of Synchrony

29

Equation 5.3 is quadratic in mN . For mN D 0, one recovers the condition for the SN bifurcation obtained above (line L1). Another transition line is 6 found for mN D 0 on which synchronous oscillations emerge and have a low frequency, m / g ¿ 1. During one period of these oscillations, the neurons burst synchronously since they have the time to Žre several spikes in one oscillation of the populations. If one assumes c 2E D c 2I D c , it is convenient to deŽne X D gEE UE C gII UI and Y D D UI UE . One Žnds that an instability to oscillations occurs on the line X D 2 in the ( X, Y ) plane and that m / g ¿ 1 varies continuously on this line. Therefore, sufŽciently strong gEE and not too large gII are required for burst synchrony. More 6 c 2I , this onset of slow bursting occurs on a surface generally, when c 2E D in the three-dimensional space ( gEE , gII , ¡gEI gIE ) . One also Žnds here that a sufŽciently strong gEE is required for this instability. In general, one also Žnds that increasing gII prevents the emergence of synchronous bursting. Equation 5.3 is also the spectral equation for the stability of the Žxedpoint solutions of the system of two coupled equations, 1 dºE D ¡ºE C GE ( gEEºE C gEIºI ) cN2E dt 1 dºI D ¡ºI C GI ( gIEºE C gIIºI ) , cN2I dt

(5.5) (5.6)

where the functions Ga ( hE , hI ) are Z Ga ( x) D

1 ¡1

dIQ a (I ) F ( I C x ).

(5.7)

Let us consider a two-population rate model in which the neurons have an input-output transfer function F (x ) and receive a random external input with probability density function Qa (I ) (a D E, I). In this model, the Žring rate of neuron i in population a, ºia, follows the equation 1 dºia syn D ¡ºia C F ( Iia C Iia ) , cN2a dt syn

(5.8)

where the total synaptic current, Iia , is given by equation 3.1 and Iia is its external input. It is easy to check that the dynamics of this model can be reduced to two effective degrees of freedom: the average Žring rate in the two populations, ºa ( t) , a D E, I, which satisfy equations 5.5 and 5.6. These two equations are nothing other than a two-population mean-Želd model with transfer function Ga (x ) given by equation 5.7. An example of such a function is plotted in Figure 13. Therefore, in the limit of very slow synaptic decay, the instabilities of our spiking model are the same as those of a rate model (see also Ermentrout, 1994; Bressloff & Coombes, 2000).

30

D. Hansel and G. Mato

n (Hz)

150

100

50

0

1

0

1 2 I (mA/cm2 )

3

Figure 13: Input-output transfer function of the rate model that corresponds to the QIF model for s D 0.4. Other parameters as in Figure 1.

Finally, one can show that if the rise time of the synapses is also very slow and on the same order of magnitude as their decay time, the quadratic equation for m , equation 5.5, is replaced by a quartic equation. This situation is similar to the previous one except that now the instability to burst synchrony does not require mutual interactions between the excitatory neurons. Here too the spectral equation can be derived from the mean-Želd equations of a rate model. However, now it is deŽned by four coupled equations—two equations for each of the two populations. 5.4 The Effect of the Average Firing Rates. The effect of the Žring rates of the two populations on the stability of the AS has been studied above in the symmetric case in which the average Žring rates of the two populations were necessarily equal. Here, we consider the case in which the distributions of Žring rates in the two populations are different to understand how this affects the stability of the AS. In particular, we want to investigate whether some speciŽc relationship (“resonance”) between the average Žring rates in the two populations is favorable to the emergence of synchrony in the network. The SN instability (m D 0) depends on the Žring rates through UE (0) and UI (0). The maximal value of the recurrent excitatory feedback, gEE , is given by gEE D

1 UI (0) C |gEI | gIE . 1 C | gII |UI (0) UE (0)

(5.9)

The quantities Ua (0), a D E, I are decreasing functions of the Žring rate. For large Žring rates, they converge to a Žnite value. Therefore, increasing

Asynchronous States and the Emergence of Synchrony

31

the Žring rate of the excitatory population or decreasing the Žring rate of the inhibitory population increases gEE (i.e., stabilizes the AS with respect to the SN instability). In particular, the maximal value of gEE at this onset of instability, when ºI changes, is 1 / UE (0) C | gEI | gIE / gII . It is reached in the limit of very small Žring rates of the inhibitory neurons. This is because the sensitivity of the neurons to changes in their input increases when their Žring rate decreases. Therefore, the inhibitory neurons are more efŽcient at low Žring rates than at high Žring rates at preventing the runaway of the activity of the excitatory neurons due to the excitatory feedback. In the symmetric case, we have found that the interaction with an excitatory population stabilizes the AS with respect to the spike-to-spike instability generated by the I-I interactions. Figure 11B provides an example that this remains true for the general case. In order to gain more insight into this property, we computed the value of gII on L2 for different values of ºE and ºI with all the other parameters of the model held constant. The result for ºI D 50 Hz is plotted in Figure 14A; in the limit, ºE ! 0, gII ! gc , where gc is the critical value of the I-I coupling at which the AS would lose stability in the inhibitory population if it were isolated from the excitatory population. This is expected since in this limit, the excitatory population has no effect because it is inactive. In the limit of large ºE , also, gII ! gc . This is because in this limit, the modulation of the excitatory drive on the inhibitory population becomes small, and therefore the excitatory population has no dynamical effect on the inhibitory population. For intermediate values of the excitatory Žring rate, gII is a nonmonotonic function of ºE . The minimum value is reached for ºE ¼ ºI . It is about 25% more negative than gc . Note that the position and the maximum of gII depend on ºI . As a rule, we have found that when ºI is large, this maximum is more pronounced, and it is located around a value of ºE < ºI as depicted in Figure 14A. These results show that in general, the feedback between the two populations disrupts the emergence of synchrony through the mutual interactions between the inhibitory neurons unless the average frequencies of the two populations differ sufŽciently. The frequencies of the two populations affect the instability on L4 in a different way, as shown in Figure 14B. Here, we have plotted the critical value of the feedback, ¡gEI gIE , between the two populations versus the frequencies ºE and ºI for gEE D gII D 0. The feedback required to destabilize the AS is minimum, around ºE D ºI ¼ 27.5 Hz. Note that this minimum is deep but that its curvature is small. Therefore, in contrast to what happens on L2, synchrony is favored on L4 if the average frequencies in the two populations are similar. The behavior for line L3 is similar to the one found for L4. The critical value of the feedback is smaller when the two populations have the same Žring rate. However, in this case, the system is much more sensitive to the difference in the Žring rates. For instance, a factor of 2 between ºE and ºI generates an increase of a factor of 10 in the critical

32

D. Hansel and G. Mato

A

-2.4 -2.6 -2.8 -3 g

II

-3.2 -3.4 -3.6 -3.8

0

20

40 60 n (Hz)

80

100

E

B -g g

IE EI

24 16 8 0 0

20 n E (Hz)

40

60 0

20

40

60

80

n (Hz) I

Figure 14: (A) The strength of the mutual inhibition, gII , for which the AS becomes unstable on line L2 versus the average frequency of the excitatory population, ºE . The average frequency of the inhibitory population is ºI D 50 Hz. Other parameters are gEE D 2 and gEI gIE D ¡4. (B) ¡gEI gIE versus the average Žring rates of the two populations for which a Hopf bifurcation (L4) occurs. Here, gEE D gII D 0. In A and B, the synaptic time constants are t1E D t1I D 1 msec, t2E D 3 msec, t2I D 6 msec and the standard deviations of the input distributions are sE D sI D 0.2.

¡gEI gIE (data not shown here), while the change on L4 is less than 100%. This behavior is similar to the one found in the previous section: line L3 is a much less robust transition than line L4.

Asynchronous States and the Emergence of Synchrony

33

6 Numerical Simulations The study of the solutions of equation 3.16 has allowed us to determine in the parameter space the AS stability region and the different ways this state can become unstable. The goal of this section is to complete the characterization of these instabilities and the patterns of synchrony that emerge beyond their onset. We focus on the following questions: • Are the transitions at the onset of instabilities sub- or supercritical when they occur through a Hopf bifurcation? • To what extent is the imaginary part of the unstable eigenvalue on the instability lines a good estimator of the frequency of the population in the stable oscillatory state in which the network settles beyond but close to these lines? Is the phase lag, d, computed from the linearized dynamics, a good estimator for the actual phase lag between the oscillations of the activities in the two populations in the stable state near the instability onset? The answer to these two questions depends critically on the nature of the Hopf bifurcation. • Which one of the excitatory or inhibitory populations is more synchronized near onset of synchrony? How can their relative level of synchrony be controlled? In principle, the answer to the Žrst question can be determined analytically by expanding the dynamics beyond the linear order. However, this calculation is extremely involved and is beyond the scope of this article. Therefore, to clarify all these issues, we relied on numerical simulations. These were performed by integrating the dynamics of the network using a second-order Runge-Kutta scheme with a Žxed time step, dt D 0.1 msec. This algorithm was supplemented by an interpolation for the determination of the Žring times of the neurons (Hansel, Mato, Neltner, & Meunier, 1998; Shelley & Tao, 2001). 6.1 The Nature of the Hopf Bifurcations. In a subcritical Hopf bifurcation, the stable Žxed point, a stable limit cycle, and an unstable limit cycle can coexist in some range of the control parameter near the onset of bifurcation. This does not happen if the bifurcation is supercritical. One can use this criterion to characterize the nature of a Hopf bifurcation in numerical simulations by checking whether the system displays a hysteresis when the control parameter is changed continuously from a value where only the Žxed point exists up to values where only the limit cycle is stable and backward. We performed numerical simulations in the vicinity of the three lines L2, L3, and L4. The state of the network was characterized by computing the synchrony level  in the two populations, as explained in appendix C. Simulations with different network sizes were compared to check that syn-

34

D. Hansel and G. Mato

chrony was not due to a Žnite-size effect. An example of the results of this study is shown in Figures 6.1A through 15C for s D 0.1 and network size NE D NI D 12, 560 near instability points chosen on L2, L3, and L4, respectively. No hysteresis is apparent in the vicinity of the instability points on L2 (see Figure 15A). Although a small hysteresis cannot be completely excluded, our simulations were unable to detect it. In contrast, near L3 and L4,

Asynchronous States and the Emergence of Synchrony

35

the network displays some hysteresis (see Figures 15B and 15C). However, this is a small effect since the range of coupling strength in which it occurs, as well as the value of the synchrony parameters, Â, on the left branches of the hysteresis, are small. When s increases, a hysteresis also appears near the instability points on L2. This effect can be seen by comparing Figures 15A and 15D, which were obtained for s D 0.1 and s D 0.4, respectively. However, this is a small effect, which can hardly be distinguished from sample-to-sample uctuations if the network size is too small (result not shown). Near L4, for this value of s, the hysteresis has shrunk and cannot be seen. 6.2 The Frequency of the Synchronous Oscillations and the Phase Lag Between the Two Populations. Our simulations indicate that a supercritical Hopf bifurcation occurs on L4 for large levels of heterogeneities. Therefore, one expects that the pattern of synchrony should be correctly predicted by our linear stability analysis near the bifurcation. This is conŽrmed in Figure 16A, where simulation results for the frequency of the population oscillations near L4 are plotted for the parameters of the phase diagram in Figure 7. The simulations were performed along a line parallel to line L4, and the results are compared with the value predicted from a linear stability analysis. In Figure 17A, the potentials of one excitatory neuron and one inhibitory neuron are plotted for T D 5.25, D D 8. On this point, the global frequency is much smaller than the Žring rate of the neurons. This means that the system must be bursting, as observed in the simulation.

Figure 15: Facing page. The level of synchrony in the excitatory population close to the Hopf bifurcations lines in the symmetric case. For each value of the bifurcation control parameter (corresponding to the x-axis), the network was simulated during 60,000 time steps. At the end of the run, the Žnal state of the network was kept and used as initial conditions to run a new simulation after the control parameter was incremented. In each Žgure, two lines are shown. One corresponds to a sweep positive increment and the other to a negative increment. The standard deviation of the input is s D 0.1 (A) The strength of the recurrent excitation is Žxed gEE D 2 and gII was changed. For each value of gII , gEI gEI is varied to keep D D 0. Predicted transition on L2 at T D ¡1.11 (vertical arrow). (B) The E-E and I-I couplings are Žxed: gEE D gII D 0. The E-I and I-E coupling are varied with gEI D ¡gIE . Predicted transition on line L3 at D D 1.98 (vertical arrow). (C) The E-E and the I-I interactions are Žxed, gEE D 5, gII D 0 and gEI and gIE are changed simultaneously with gEI D ¡gIE . Predicted transition on L4 at D D 7.9 (vertical arrow). (D) gII and gEI , gIE are changed such that gEI D ¡gIE and D D ¡1. Predicted transition on L2 at T D ¡4.2 (vertical arrow). (E) The E-E and the I-I interactions are Žxed: gEE D gII D 0 and gEI and gIE are changed simultaneously with gEI D ¡gIE . Predicted transition on L4 at D D 8.11.

36

D. Hansel and G. Mato

Figure 16: The frequency of the population activity oscillations near the onset of instability of the AS. The lines represent the predicted frequency from the solution of the spectral equation, equation 3.16. The squares are the results from simulations. A network of NE D NI D 1600 was simulated during 60,000 time steps and the frequency oscillation was obtained by evaluating the autocorrelation of the average membrane potential. (A) Parameters as in Figure 7. The simulations were done near L4 at distance 2 away from L4 with gIE D ¡gEI and keeping gII D ¡2. (B) Parameters as in Figure 4. The simulations were done at a distance 1 away from line L3 with gIE D ¡gEI and keeping gII D ¡2. (C) Parameters of Figure 7. The simulations were done at a distance 2 away from line L2 varying gIE D ¡gEI and keeping gEE D 0.

The Hopf bifurcations that occur on L3 (for small levels of heterogeneities) as well as on L2 if the level of heterogeneities is sufŽcient were found to be subcritical. Therefore, one expects that the actual oscillation frequencies and phase lags near these lines should deviate from the values obtained for the

Asynchronous States and the Emergence of Synchrony

37

A

20 mV

0

100 200 300 400 500 t (msec) B

10 mV

0

100 200 300 400 500 t(msec)

Figure 17: Synchronous oscillations in which neurons Žre bursts of spikes. Parameters as in Figure 7 with T D 5.25 and D D 8. (A) Excitatory neuron (lower trace) and inhibitory neuron (upper trace). (B) The membrane potential of the excitatory (resp. inhibitory) neurons averaged over all the population (lower line, resp. upper line).

unstable mode in the linear stability analysis. In spite of this, a reasonable agreement between the predictions and the simulation values was found as shown in Figures 16B and 16C. This is because the hysteresis near L2 has a small range and a small amplitude (see, e.g., Figure 15B). In Figure 6, the phase lag between the oscillations of the activity of the two populations is plotted as a function of gEE for gII D 0 (A) and as a function of gII for gEE D 0 (B). The solid and dashed lines correspond to the predictions from our analytical results. Results from numerical simulations

38

D. Hansel and G. Mato

performed in the vicinity of the instability lines are represented by squares. They were estimated from the position of the cross-correlation peak of the activities of the two populations. In agreement with the linear theory, the phase lag on L3 and L4 are nonmonotonic functions of the coupling. For instance, on line L4, it reaches a maximum of p / 2 when gEE D gII D 0 and then decreases as one approaches points B or D in the phase diagram. The negative sign of the phase lag means that the oscillation of the activity of the excitatory population is ahead of the inhibitory one. Note that on L3, the lag has the opposite sign, that is, the inhibitory population oscillates ahead of the excitatory one. On line L4 near D, the linear theory predicts that the phase lag should become small and vanish at D. This is conŽrmed by our numerical simulations, as shown in Figure 17B. Finally, the linear theory predicts that at the instability onset on L2, the phase lag should be zero. Therefore, d is expected to be small near L2 when synchrony occurs. Our simulations are in line with this prediction. 6.3 The Level of Synchrony in the Two Populations. For strong enough gII , the activity of the inhibitory neurons can synchronize as a result of their mutual coupling. If the feedback between the two populations, ¡gEI gIE is weak, the dynamical state of the inhibitory population should not be perturbed much. However, the excitatory population is now entrained by the inhibitory one. Therefore, it develops synchrony. The level of synchrony in the excitatory population depends on the coupling gEI . For small gEI , one expects the excitatory population to be weakly synchronized, and the level of synchrony should increase with gEI . These expectations were conŽrmed by our numerical simulations. Moreover, under these conditions, the inhibitory population was always substantially more synchronized than the excitatory one (results not shown). More generally, we found this property to be true in the vicinity of L2 even for large values of ¡gEI gIE . This can be seen in Figure 18, where the synchrony measures, ÂE and ÂI , are plotted against ¡gIE gEI , for gEI D ¡gIE / 4 (A) and for gEI D ¡4gIE (B). In both cases, the transition point on L2 occurs for the same value of ¡gIE gEI , namely, ¡gIE gEI ¼ 3.25. This agrees with the fact that the spectral equation depends on only gEI and gIE through their product. However, the level of synchrony in the two populations also depends on the ratio gEI / gIE . On both curves, the inhibitory population is always more synchronized than the excitatory one. The synchrony in the excitatory population varies nonmonotonically with gEI gIE , with an intermediate region where the AS is stable. This is because the feedback between the two populations has two opposing effects. On the one hand, it drives the excitatory population, and on the other, it disrupts the synchrony induced by the mutual I-I interactions in the inhibitory population. Finally, increasing gEI (for Žxed gEI gIE ) increases ÂE . This is because the larger the | gEI |, the stronger is the inhibitory drive on the excitatory population.

Asynchronous States and the Emergence of Synchrony

39 A

0.5 0.4 0.3 c

L2

0.2

L4

0.1 0

1

3

5 -g

7

g

EI IE

B

0.5 0.4

c

0.3 L2

0.2

L4

0.1 0

1

3

5 -g

EI

g

7

IE

Figure 18: The synchrony level of the excitatory (solid line) and the inhibitory populations (dashed line) versus ¡gEI gIE . The strength of the E-E and I-I interactions are Žxed: gEE D 2, gII D ¡2. Other parameters ºE D ºI D 50 Hz, sE D sI D 0.2. A network with NE D NI D 1600 neurons was simulated for different values of gEI and a Žxed ratio gEI / gIE . (A) gEI / gIE D ¡1 / 4. (B) gEI / gIE D ¡4. The level of synchrony was calculated as explained in appendix C (average over 6 seconds). The theory predicts that the AS becomes unstable at ¡gEI gIE D 3.25 (on L2) and ¡gEI gIE D 5.7 (on L4) as indicated by the arrows.

In all our numerical simulations, we found that in the vicinity of L2, the level of synchrony is larger in the inhibitory population than in the excitatory one. This is in contrast with what happens near L3. This is shown in Figure 19. Here, for a given value of ¡gEI gIE , the relative level of synchrony depends strongly on the balance between gEI and gIE . For large gEI and small gIE , the synchrony in the excitatory population is higher than in the inhibitory one. The opposite happens if gIE is large and | gEI | is small. The same Žgure also shows that the synchrony levels near line L4 are affected in the same way by the balance between gEI and gIE . Making the excitatory to

40

D. Hansel and G. Mato A

0.5 0.4 0.3 c

L2

0.2

L3

0.1 0

1

3

5 -g

EI

g

7

IE

B

0.5 0.4 0.3 L3

c

L2

0.2 0.1 0

1

3

5

7

-g g

EI EI

Figure 19: The synchrony level of the excitatory (solid line) and the inhibitory populations (dashed line) vs. ¡gEI gIE . The strength of the E-E and I-I interactions is Žxed: gEE D 2, gII D ¡2. Other parameters ºE D ºI D 50 Hz, sE D sI D 0.1. A network with NE D NI D 1600 neurons was simulated for different values of gEI and a Žxed ratio gEI / gIE . (A) gEI / gIE D ¡1 / 4 (B) gEI / gIE D ¡4. The level of synchrony of the two populations was calculated as explained in appendix C (average over 6 seconds). The theory predicts that the AS becomes unstable at ¡gEI gIE D 2.96 (on L2) and ¡gEI gIE D 5.42 (on L3) as indicated by the arrows.

inhibitory interaction stronger makes the inhibitory population more synchronized, and vice versa. 7 Discussion In this work, we have analyzed the synchronization properties of a fully connected two-population system. We have not intended to present a complete model of the cortex, but a simple network model that is amenable to analytical treatment. A natural but nontrivial extension of our work would

Asynchronous States and the Emergence of Synchrony

41

be to study the stability of asynchronous states in networks with more structured patterns of connectivity such as the ones found in the visual cortex, which include local or intracolumnar connections between neurons with similar feature preference, vertical connections between different layers, and feedback connections from higher cortical areas. For instance, one expects that in a network with space-dependent interactions, instabilities of the AS may lead to traveling waves or other spatiotemporal patterns in which hot spots of activity move across the system (Ben Yshai, Hansel, & Sompolinsky, 1997; Hansel & Sompolinsky, 1998; Golomb, 1998; Golomb & Ermentrout, 1999). 7.1 Three Mechanisms for Synchrony in Two-Population Networks. 7.1.1 The Mutual Inhibition Mechanism. The activity of inhibitory neurons can be synchronized on the timescale of the spikes through their mutual interactions. It has been proposed that hippocampal c rhythms can be driven by populations of inhibitory interneurons whose activity is synchronized through this mechanism (Buzs a ki & Chrobak, 1995). Synchrony in one population of inhibitory neurons has been extensively studied in recent years theoretically (van Vreeswijk, Abbott, & Ermentrout, 1994; Hansel, Mato, & Meunier, 1995; van Vreeswijk, 1996; Wang & Busz a ki, 1996; White et al., 1998; Chow, 1998; Chow, White, Ritt, & Kopell, 1998; Golomb & Hansel, 2000; Neltner et al., 2000; Whittington, Traub, Kopell, Ermentrout, & Buhl, 2000) and experimentally (Whittington, Traub, & Jefferys, 1995; Traub, Whittington, Colling, Buzs Âaki, & Jefferys, 1996; Traub, Jefferys, & Whittington, 1999, for a review). Here, we have generalized this mechanism to take into account the effect of a second excitatory population. When the cross-talk interaction between the two populations is not too large, we have found that the AS becomes unstable if the mutual inhibition is sufŽciently strong. Excitation plays against this destabilization in two ways. For a given value of the feedback between the two populations (Žxed value of | gEI gIE |), the strength of the mutual inhibition gII required to destabilize the AS increases with gEE . Also, for a given gEE , a larger mutual inhibition is required if the feedback between the two populations is increased. If |gEI gIE | is too strong, another mechanism for the destabilization of the AS can occur, as discussed below. These desynchronizing effects of the excitatory population are stronger when the average Žring rates of the two populations are more similar. In the example shown in Figure 14, | gII | is maximal when the Žring rates of the two populations are similar. Therefore, as far as the value of gII is concerned, the excitatory population has a strong inuence. In contrast, we have found that the frequency of the unstable mode at the transition is mainly determined by the average Žring rate of the inhibitory population and depends only weakly on the properties of the excitatory population and on the way the two populations interact. Related to this fact, we have shown that the phase shift between the two populations is very small even

42

D. Hansel and G. Mato

if their average Žring rates are very different. In view of these features, this mechanism can arguably be based on the synchronization of the inhibitory population. Our analysis focuses on the mechanisms for destabilization of the AS. We address this problem by studying local bifurcations of the AS. Therefore, we did not address the issue of rhythm properties under strong synchrony. Strongly synchronized states could coexist with the weakly synchronous states or even the AS. The only way to address this question (and the question of the size of the attraction basins of the stable states) is with extensive numerical simulations. The fact that the frequency of the unstable mode is determined to a large extent by the average Žring rate of the neurons in the AS does not contradict previous results indicating that when the synchrony is substantial, the rhythm frequency can depend on the synaptic time constant of mutual inhibition (Whittington et al., 1995; Chow et al., 1998; White et al., 1998). 7.1.2 The Cross-Talk Mechanisms. We have demonstrated that the AS can be destabilized with two qualitatively different mechanisms by the crosstalk interactions between the two populations. In the Žrst mechanism (on L4), the inhibitory population lags behind the excitatory one. Therefore, in the unstable mode, an increase in activity of the excitatory population makes the inhibitory neurons increase their activity. This generates in return an increase in inhibition of the excitatory neurons, which tends to prevent simultaneous Žring of many of these neurons. SufŽciently strong feedback between the two populations will prevent the perturbation from decaying, and synchrony will emerge. Mutual excitation of excitatory neurons favors the development of this instability since it helps to amplify perturbations in the E population. Mutual inhibition of inhibitory neurons disrupts the development of this instability since it reduces the excitability of this population. In this mechanism, synchrony is induced by recurrent inhibition of the excitatory population. It has been investigated mostly by numerical simulations in previous studies and has been proposed as a mechanism for c oscillations (Jefferys, Traub, & Whittington, 1996). However, as shown here, this mechanism can lead to slow oscillations. The oscillations can be so slow that the neurons develop bursts of action potentials. In the second type of cross-talk instability (on L3), the inhibitory population tends to Žre in advance with respect to the excitatory one, preventing the excitatory neurons from Žring. In return, this deexcitation reduces the activity of the inhibitory neurons. This instability does not decay provided that the feedback between the two populations is sufŽciently strong. The global oscillations near this instability are on the order of or faster than the average Žring rate of the neurons. These two mechanisms are qualitatively different from the mutual inhibition mechanism. Indeed, we have found that the frequency of population

Asynchronous States and the Emergence of Synchrony

43

activity as well as the phase shift between the oscillations in the two populations vary substantially and continuously on the corresponding instability surfaces. Also, the emerging oscillations in the two populations can display a large phase shift, d. However, in all the examples we have studied, the numerical solution of the spectral equation reveals that |d| < p2 . We did not prove this statement in general. However, we have proved that in the symmetric case, |d| D p / 2 on onset instability lines L3 and L4 at gEE D gII D 0. We have also shown that d has a local maximum at that point. Without the assumption of symmetry between the two populations, we have also shown that the oscillations of the two populations become in phase when the emerging rhythm becomes very slow. 7.1.3 Robustness of Synchrony to Heterogeneities. A systematic and comparative study of the robustness of the various modes of destabilization would be a very tedious task. Nevertheless, some general trends emerge from the study of the spectral equation in the symmetric case. The coupling strengths required for destabilization on L2 and L3 depend crucially on the heterogeneity level (see Figure 9). This is in contrast with what happens for L1 and L4 where the dependence is much weaker. Thus, the recurrent inhibition is the most robust mechanism for synchronous oscillations. Another difference between line L3 and the other lines is that for sufŽciently large s, L3 lies completely inside a region, which is beyond instability line L4. Therefore, when the level of heterogeneities increases, this line becomes physically irrelevant. In this sense, the mechanism corresponding to this instability line is much less robust than the other mechanisms. Similar trends were also found in all the examples we investigated after relaxing the symmetry assumption. Recent studies have suggested that because of heterogeneities, synchrony in inhibitory networks can be difŽcult to achieve (White et al. 1998; Chow et al., 1998). Our study provides an example of a model in which the desynchronizing effect of large levels of heterogeneities can be overcome provided that the external excitatory drive and the synaptic interactions are sufŽciently strong. Simulation results in the framework of a conductance-based model were found to be consistent with this conclusion (Neltner et al., 2000). This is in contrast with the behavior of inhibitory networks of leaky integrate-andŽre models where for sufŽciently large level of heterogeneities, the AS is always stable (Golomb et al., 2001). For sufŽciently strong coupling, the AS is stable even if the network is homogeneous. This is yet another difference with the present model. Below, we elaborate further on these differences. 7.2 Getting Rid of Synchrony. The frequent occurrence of synchronous neuronal activity in physiopathological states in the brain (see Bergman et al., 1998; McCormick & Contreras, 2001) suggests that in many situations, synchronous states of activity need to be prevented. The issue is to understand how synchrony can be prevented in particular in systems

44

D. Hansel and G. Mato

where massive feedback interactions are observed. Our analysis shows that even if the interactions are strong, a suitable balance between excitation and inhibition stabilizes the AS. We have proved this statement for our model in full generality and analytically under the assumption of symmetry between the two populations. Indeed, for D D T D 0, the AS is marginally stable without heterogeneities, and it becomes stable when heterogeneities are added. Besides the presence of heterogeneities, this implies that if the excitatory feedback connections, gEE , are strong, the two populations must interact strongly, and the mutual interactions of the inhibitory neurons must be strong to ensure that the AS is stable. In particular, if gII is not strong enough, the system tends to Žre synchronous bursts. Increasing gII reduces the excitability of the inhibitory population, preventing the synchronous bursts from developing. This desynchronizing effect of mutual inhibition also holds in the nonsymmetric case, as shown in Figure 11.

7.3 Application to the Stability of Persistent Activity. Hebb (1949) proposed that the representation of an object in short-term memory consists of the set of cortical cells that this object activates when it is presented. According to this hypothesis, the reverberation of excitation across this set is able to maintain activity even when the stimulus has been removed, thus preserving a memory of the object. In other words, the network displays multistability. The fact that persistent activity requires strong excitatory feedback poses the problem of control of neural activity in persistent states at Žring rates in the range 10 to 30 spikes per second, as observed in experiments. Wang (1999) has argued, mostly on the basis of numerical simulations, that to achieve a stable persistent state with activities in a physiological range (10–50 Hz), recurrent excitatory synapses must be dominated by a slow component. It was subsequently proposed that NMDA synapses were required to achieve such states. Another possibility is that inhibition controls the Žring rate (Rubin and Sompolinsky, 1989). However, recurrent inhibition can also induce synchrony, and this can be incompatible with low rate persistent activity. Indeed, if the neurons Žre synchronously in some short time window, the fast excitatory feedback will decay before the neurons Žre their next spike. Note, however, that weak synchrony may be compatible with persistent activity. The results shown in Figure 12 indicate that strong mutual inhibitory interactions with synaptic time constants in a range corresponding to GABAA synapses stabilize the AS even if the recurrent excitation is mediated by nonsaturating AMPA synapses. Therefore, strong GABAA mutual inhibition should help achieve a stable persistent state. This is in line with the conclusions of our previous study (Hansel & Mato, 2001), in which the threshold distribution had a Žnite support. Here, we show that this is true for a gaussian distribution of threshold without truncation (see Figure 3).

Asynchronous States and the Emergence of Synchrony

45

At Žrst glance, this contradicts the results of Wang (1999). However, I-I interactions were not included in Wang’s study. This may explain why it concluded that stable persistent activity was impossible to achieve with AMPA excitatory synapses and that NMDA synapses were necessary. One should note that recent anatomical and physiological studies have revealed that I-I connections are numerous and strong in the cortex (Sik, Penttonen, Ylinen, & Buzs a ki, 1995; Tarczy-Hornoch, Martin, Jack, & Stratford, 1998; Gupta, Wang, & Markram, 2000). The role we suggest for these connections in the stability of persistent activity Žts nicely with these experimental results, without excluding that NMDA synapses could also contribute to delay period activity in cortex. 7.4 Comparison with Networks of Leaky (Linear) Integrate-and-Fire Networks. In the absence of noise, a very precise Žne tuning of the external input is required to get a LIF neuron to Žre below 8 spikes per second. This is due to the logarithmic behavior of the º-I curve near current threshold. Furthermore, there is an exponential divergence of the function H (º) (the equivalent of equation 3.18 for the leaky integrate-and-Žre neuron; Golomb et al. 2001) when º ! 0 unless P ( x) vanishes at least exponentially fast when x ! 0. In this situation, U also diverges, and the spectral equation is not well deŽned. This divergence is due to the fact that at very low rates, the LIF model spends an exponentially long time near the threshold. This is because leaky integrate-and-Žre neurons integrate their inputs in a purely passive way. As a consequence, if the external currents the neurons receive are distributed according to a gaussian, the resulting distribution of Žring rate does not vanish sufŽciently fast at zero Žring rate for the integrals in H (º) to converge. To avoid this problem, a gaussian distribution of period was chosen in Golomb et al. (2001). In contrast, for the QIF model in the limit of very low rates, H (º) goes as (exp(¡l /º) ¡1) P (º) /º. Therefore, in this limit, U is proportional to the average period of the population, which is deŽned even if the distribution of Žring rates does not vanish at zero frequency. Another difference between the two models is that in LIF inhibitory networks, instability of the AS can lead to cluster states (van Vreeswijk, 1996; Neltner et al., 2000; Golomb & Hansel, 2000). The number of clusters thus generated can be rather large, especially at low Žring rates. This does not happen in our model, since the numerical solution of the spectral equation reveals that the onsets to clustering instabilities are always within the regions where the AS is already unstable. These instabilities are therefore irrelevant for the AS. Of course, this does not exclude the possibility that clustering occurs far from the onset of the instabilities in our model as well. The origin of this different behavior can be clariŽed in the weak coupling limit. In this limit, the instabilities of the AS can be related to the Fourier expansion of the response function, Z (w ), of the neurons, where w is the phase variable describing the position of the neuron on its limit cycle (Kuramoto, 1984; Neltner et al., 2000). Destabilization of the AS through an n-cluster

46

D. Hansel and G. Mato

instability mode in a heterogeneous network requires a large Fourier coefŽcient of order n. In the LIF model, the function Z is an exponential and is discontinuous at w D p / 2. Therefore, all the Fourier coefŽcients are large and even diverge in the limit of small Žring rates. One can further show that in this limit, the order of the unstable leading mode increases. In contrast, for the QIF, the amplitude of the response function diverges but proportionally to cos(w ) (up to a constant). Therefore, the dominant mode becomes the Žrst one. This difference has its origin in the way the membrane potential approaches the threshold potential at low rate. In the LIF the derivative of the membrane potential that is just before the spike threshold goes to zero with the Žring rate. This is not the case for the QIF where this derivative is 6 0. proportional to Vt2 D 7.5 Relation to Previous Studies. Mechanisms of neuronal synchrony have been addressed in recent years, following two different and complementary theoretical approaches. One has been to study the conditions under which fully synchronized states or more general phase-locked states are stable in fully connected networks of identical neurons (Wang & Rinzel, 1992; Hansel, Mato, & Meunier, 1993b; van Vreeswijk et al., 1994; Hansel, et al., 1995; White et al., 1998; Chow, 1998; Crook, Ermentrout, & Bower, 1998). The other approach has been to investigate the conditions of the stability of the asynchronous state of a large neuronal system (Abbott & van Vreeswijk, 1993; Gerstner & van Hemmen, 1993; Treves, 1993; van Vreeswijk, 1996, 2000; Brunel & Hakim, 1999; Brunel, 2000; Golomb & Hansel, 2000; Neltner et al., 2000; Gerstner, 2000). The latter approach was followed here. This article aims to contribute to a comprehensive understanding of the patterns of synchrony in two-population heterogeneous neuronal networks with full connectivity. In order to make the model more amenable to analytical calculations, noise was not introduced. An expected consequence of adding input noise is to increase the parameter domain where the AS is stable. Still, if the noise is not too strong, simulations indicate that three instabilities may occur for the AS. However, another type of instability can also be found if the noise is strong enough. It leads to a state in which the population oscillations are faster than the average Žring rate of the neurons. In this state, the Žring pattern of the neurons can be very irregular, and averaging over a large number of neurons is necessary to reveal the existence of the underlying rhythmic component in the network activity. Such patterns of synchrony, called fast oscillations, were studied Žrst by Brunel and Hakim (1999) in sparse networks of inhibitory leaky integrate-and-Žre neurons. The results of this study were subsequently generalized to the case of two populations with sparse connectivity (Brunel, 2000; N. Brunel, C. Geisler, & X.-J. Wang, private communication Dec. 2001). More recently, Brunel and Hansel (2002) have shown that sparse connectivity was not required for fast oscillations to take place but that they can occur in a fully connected network of inhibitory neurons in the presence

Asynchronous States and the Emergence of Synchrony

47

of strong input noise. Increasing the coupling destabilizes the AS, leading to a state where the neurons Žre synchronously at a rate that can be signiŽcantly smaller than the population oscillations. Therefore, when noise is also present, the AS can become unstable through four generic mechanisms. The all-to-all connectivity considered here can be thought of as the extreme case of a system in which the connectivity is high. Indeed, we have checked that our results still hold when the connectivity is random and sufŽciently dense (results not shown). However, an important difference exists between the collective behavior of densely and sparsely connected networks. In densely connected networks in the absence of input noise, the neurons discharge in a very regular manner. This is because the uctuations in the synaptic input are small—of the order of O (1 / K ) , where K is the connectivity. In contrast, in sparse networks with strong excitatory and inhibitory synapses, the Žring of the neurons can be highly irregular even if no input noise is present (van Vreeswijk & Sompolinsky, 1996, 1998; Brunel, 2000). In this state, the stochasticity of the Žring is of deterministic origin, and the uctuations in the synaptic input remain large due to a balance between excitation and inhibition. We have shown that in the presence of an excitatory population, a stronger coupling is required to synchronize activity through mutual inhibition. A similar conclusion was reported in Traub, Jefferys, and Whittington (1997) relying on a simulation of a detailed conductance-based model. These authors found that excitation can destroy a synchronous state generated by mutual inhibition. We should also remark that the desynchronizing effect of excitation is a characteristic feature of type I neurons. For type II neurons (such as in the Hodgkin-Huxley model), excitation can have a strongly synchronizing effect, especially at low Žring rates (Hansel, Mato, & Meunier, 1993b; Hansel et al., 1993a, 1995). The fact that mutual inhibition can prevent recurrent inhibition to induce synchrony was noted by Tsodyks, Skaggs, Sejnowski, and McNaughton (1997) in the framework of a two-population rate model. In this work, strengthening the I-I interactions was shown to prevent the two populations from developing synchronous oscillatory rate modulations. Since we have seen that in the limit of inŽnitely slow synapses, the spectral equation becomes equivalent to the spectral equation of a rate model, it is not surprising that in that limit, our conclusions agree with the results of Tsodyks et al. (1997). However, mutual inhibition is a powerful way to generate synchrony on a timescale comparable to the timescale of AMPA and GABAA interactions. Therefore, it was not clear a priori that this mechanism would still hold for these synapses. Our detailed stability analysis of the AS shows that this is indeed the case. A desynchronizing effect of mutual inhibition was also reported by Bush and Sejnowski (1996) in simulations of a two-population model of a column in primary visual cortex. It was found that when the inhibition between the

48

D. Hansel and G. Mato

basket cells is increased, the temporal modulation of the local Želd potential is decreased and the neurons are less bursty. However, in this work, the Žring rate of the neurons changed when the mutual inhibition was increased. Another difference with our work is that in the simulations presented by Bush and Sejnowski, the neurons were intrinsic bursters. Therefore, in their case, the desynchronizing effect of mutual inhibition could also have been the consequence of a disruption of the intrinsic bursting when the inhibitory synaptic input became stronger. Excitatory cells in cortex display strong spike adaptation, whereas interneurons do not. To reduce the number of parameters of our two-population model, we assumed the dynamics of the two populations were identical and spike adaptation was not introduced in the excitatory dynamics. Recent theoretical studies have investigated the dynamics of excitatory networks of neurons with spike adaptation (van Vreeswijk & Hansel, 2001; Fuhrman, Markram, & Tsodyks, 2002). In these works, it was demonstrated, mostly analytically, that excitation and spike adaptation can cooperate to create synchronous oscillatory activity and that the frequency of these oscillations depends on the kinetics of the adaptation process. In particular, if the excitatory recurrent feedback and the adaptation is sufŽciently strong and slow, the network settles into a state in which the neurons Žre synchronous bursts (van Vreeswijk & Hansel, 2001). How an additional population of inhibitory neuron modiŽes this type of synchronous state was subsequently studied, relying on numerical simulations (van Vreeswijk & Hansel, 2001). The main conclusion was that recurrent inhibition of the excitatory population (E-I-E feedback loop) plays against the emergence of the synchronous bursts but that mutual inhibition (I-I interactions) favors it. In contrast, we have found in our model that recurrent inhibition favors the synchronous bursting that emerges on instability lines L3 and L4 but that mutual inhibition disrupts it. Keeping in mind that spike-to-spike synchrony can be induced by mutual inhibition, inhibitory interactions can thus have a spectrum of qualitatively different effects in two-population networks. Appendix A: The Wang-Busz a ki Model The dynamical equations for the conductance-based neuron model used in this work are (Wang & Busz a ki, 1996):

C

dV D Iext ¡gNa m31 h ( V ¡VNa ) ¡gK n4 ( V ¡ VK ) ¡gl ( V ¡Vl ) C Isyn dt dh h1 ( V ) ¡ h D th ( V ) dt dn n1 ( V ) ¡ n . D tn ( V ) dt

(A.1) (A.2) (A.3)

Asynchronous States and the Emergence of Synchrony

49

The parameters gNa , gK , and gl are the maximum conductances per surface unit for the sodium, potassium, and leak currents, respectively, and VNa , VK , and Vl are the corresponding reversal potentials. The capacitance per surface unit is denoted by C. The external stimulus on the neuron is represented by an external current Iext , and the synaptic current due to the interactions with other neurons in the network is Isyn. The functions m1 ( V ), h1 (V ) , and n1 ( V ) and the characteristic times (in milliseconds) tm , tn , th , are given by x1 ( V ) D ax / (ax C bx ) , tx D 1 / (ax C bx ) with x D m, n, h and: am D ¡0.1(V C 35) / (exp(¡0.1( V C 35) ) ¡ 1) bm D 4 exp(¡(V C 60) / 18)

ah D 0.35 exp(¡(V C 58) / 20)

bh D 5 / (exp(¡0.1( V C 28) ) C 1).

(A.4) (A.5) (A.6) (A.7)

The other parameters of the sodium current are gNa D 35 mS / cm2 and VNa D 55 mV. The delayed rectiŽer current is described in a similar way as in the HH model with an D ¡0.05 (V C 34) / (exp(¡0.1( V C 34) ) ¡ 1) bn D 0.625 exp(¡(V C 44) / 80)

(A.8) (A.9)

and gK D 9 mS / cm2 and VK D ¡90 mV. Other parameters of the model are C D 1 m F/cm2 , gl D 0.1 mS/cm2 , and Vl D ¡65 mV. This neuron model displays an SN bifurcation at the current threshold to repetitive Žring, Ic , in the vicinity of which the relationship between p the current, I, injected in the neuron and its Žring rate behaves like f / I ¡ Ic for I > Ic . Therefore, they are able to Žre at an arbitrarily small rate. This is also seen in Figure 1, which represents the frequency-current relation of this model. Appendix B: Reduction of a Conductance-Based Model to the QIF Model We consider a type I conductance-based model. The equations of the singleneuron dynamics can be written in compact form: dX D F(X) C G(X) . dt

(B.1)

50

D. Hansel and G. Mato

In the case of the WB model, X, F, and G are three-dimensional vectors with: X1 D V

(B.2)

X2 D h

(B.3)

X3 D n

(B.4)

F1 (X) D ( Ic ¡gNa m31 h ( V ¡VNa ) ¡gK n4 (V ¡VK ) ¡gl ( V ¡Vl ) ) / C F2 (X) D ( h1 (V ) ¡ h ) / th (V )

F3 (X) D ( n1 (V ) ¡ n ) / tn (V ) G1 D I ¡ Ic

(B.5) (B.6) (B.7) (B.8)

and G2 D G3 D 0. Following Ermentrout (1996), one can study this dynamics in the vicinity of Žring onset. One writes: G(X) D 2 N(X) ,

(B.9)

where 2 ¿ 1. For 2 < 0, the neuron is not Žring. It displays a stable Žxed ¯ . For 2 > 0, the neuron Žres action potentials. At 2 D 0, a point: X D X(I) bifurcation occurs. The Taylor expansion of F (X) around X? D X¯ (Ic ) is F (X) D F(X?) C A(X ¡ X?) C Q(X ¡ X?, X ¡ X?) C ¢ ¢ ¢

(B.10)

where A and Q are, respectively, the Jacobian and the Hessian matrices of F evaluated at X?. Since an SN bifurcation occurs at X?, the matrix A has one zero eigenvalue. The corresponding normalized eigenvector will be denoted by e. Similarly, AT , the transpose of A, has a zero eigenvalue. We will denote by f the corresponding eigenvector satisfying f.e D 1. For small 2 > 0, the solution X(t) of equation B.1 can be expanded, X (t ) D X? C

p

2 z (t) e

(B.11)

where the scalar quantity z satisŽes (Ermentrout & Kopell, 1986) dz p 2 D 2 (g C qz ) , dt

(B.12)

where: g D f.N(X?) and q D f.Q(e, e). This equation can easily be solved. Assuming that g and q have the same sign, one Žnds r z ( t) D

g p tan gq2 ( t ¡ t0 ) , q

(B.13)

where t0 is a constant. Function z diverges when the argument of the tangent is a multiple of p / 2. These divergences correspond to the Žring of an action

Asynchronous States and the Emergence of Synchrony

51

potential in the original system (Ermentrout & Kopell, 1986). Therefore, in the limit 2 ! 0, the Žring frequency of the neuron vanishes as p º D B I ¡ Ic,

(B.14)

p gq . BD p

(B.15)

with

CoefŽcient B can be calculated by diagonalizing matrix A. It turns out that due to the particular structure of this matrix, expressions for e, f, Q(e, e) can be derived analytically in terms of the gating functions (this fact has apparently escaped attention in previous papers). In particular, for the WB model, one Žnds that 0

0

e D K (1, h1 , n1 ) ³ ´ 1 @F1 @F1 fD 1, th , tn LK @h @n

(B.16) (B.17)

where KD p

1

(B.18)

0 1 C ( h1 ) 2 C (n1 ) 2

L D 1 C th

0

@F1 @h

0

h1 C tn

@F1 @n

0

n1 .

(B.19) 0

The derivative of function f (V ) with respect to V has been denoted f . In these expressions, all the functions have to be evaluated for V, n1 , h1 at the Žxed point of the dynamics. Finally we Žnd: v Á ! u 2F 2F X 0 @2 F1 X 00 @F1 1 u 1 @ @ 1 1 0 t C2 C ( n1 ) 2 C . BD Xi Xi p L 2 @V 2 @V @Xi @n2 iD 2,3 @Xi i D2,3

(B.20)

The Taylor expansion of the dynamics, equation B.10, is valid in the limit of low Žring rate except during an action potential where the variable z diverges. If I is sufŽciently close to threshold, an approximate trajectory for the vector X(t) can be computed from z ( t) and the vector e. This approximation is a good description of the behavior of the neuron except during the spikes, which are of short duration. It becomes exact in the limit I ! Ic . Therefore, near the Žring onset, one can replace the full model, equation B.1, by the reduced model, equations B.11 and B.12, where all the parameters can be

52

D. Hansel and G. Mato

computed from the full model. In particular, the shape of the f-I curve near threshold is given by equation B.14. When I ¡ Ic is not small, this reduction is no longer exact. However, one can generalize it heuristically as follows. We consider that action potentials are Žred whenever z crosses (from below) some threshold, zt . Then z is immediately reset to a value zr and it evolves again according to equation B.12. The two phenomenological parameters, zt and zr , cannot be determined by the exact reduction method detailed above. Instead, we estimate them by requiring that the f ¡ I curve Žts the f ¡ I curve of the full model as well as possible. Appendix C: A Measure for the Synchrony Level To characterize the degree of synchrony in a population of N neurons in the simulations, we measure the temporal uctuations of the membrane potential averaged over the population (Hansel & Sompolinsky, 1992; Golomb & Rinzel, 1993, 1994; Ginzburg & Sompolinsky, 1994). The quantity

V ( t) D

N 1 X Vi (t) N iD 1

(C.1)

is evaluated over time, and the variance sV2 D h[V (t )]2 it ¡ [hV (t) it ]2 of its temporal uctuations is computed where h. . .it denotes time averaging. After normalization of sV to the average over the population of the single cell membrane potentials sV2i D h [Vi ( t) ]2 it ¡ [hVi ( t) it ]2 , one deŽnes  ( N ) ,  (N) D

1 N

sV2 PN

2 i D 1 sVi

,

(C.2)

which is between 0 and 1. The central limit theorem implies that in the limit, N ! 1  ( N ) behaves as  ( N ) D  (1) C

³ ´ 1 a CO , N N

(C.3)

where a > 0 is a constant. In particular,  (N ) D 1, if the system is fully synchronized (i.e., Vi ( t) D V ( t) for all i), and  (N) D O (1 / N ) if the state of the system is asynchronous. In the asynchronous state,  (1) D 0. More generally, the larger  (1) > 0, the more the population is synchronized. In networks with more than one population of neurons, we deŽne a synchrony measure  for each population separately.

Asynchronous States and the Emergence of Synchrony

53

Acknowledgments This work was partially supported by ECOS. The work of G.M. was partially supported by grant PICT97 03-00000-00131 from ANPCyT and by Fundaci Âon Antorchas. Fruitful discussions with N. Brunel, B. Ermentrout, D. Golomb, H. Sompolinsky, and C. van Vreeswijk are acknowledged. We thank N. Brunel, D. Golomb, and C. Meunier for careful and critical reading of the manuscript. References Abbott, L. F., & van Vreeswijk, C. (1993). Asynchronous state in networks of pulse-coupled oscillators. Phys. Rev., E48, 1483–1490. Abeles, M. (1990). Corticonics: Neural circuits of the cerebral cortex. Cambridge: Cambridge University Press. Ben Yshai, R., Hansel, D., & Sompolinsky, H. (1997). Traveling waves and processing of weakly tuned inputs in cortical module. J. Comput. Neurosci., 4, 57–77. Bergman, H., Feingold, A., Nini, A., Raz, A., Slovin, H., Abeles, M., & Vaadia E. (1998). Physiological aspects of information processing in the basal ganglia of normal and parkinsonian primates. Trends in Neuroscience, 21, 32–38. Bressloff, P. C., & Coombes, S. (2000). Dynamics of strongly-coupled spiking neurons. Neural Comput., 12, 91–129. Brunel, N. (2000). Dynamics of sparsely connected networks of excitatory and inhibitory spiking neurons. J. Comput. Neurosci., 8, 183–208. Brunel, N., & Hakim, V. (1999). Fast global oscillations in networks of integrateand-Žre neurons with low Žring rates. Neural Comput., 11, 1621–1671. Brunel, N., & Hansel, D. (2002). How noise affects synchronization in recurrent networks of inhibitory neurons. Unpublished manuscript. Bush, P., & Sejnowski, T. (1996). Inhibition synchronizes sparsely connected cortical neurons within and between columns of realistic network models. J. Comput. Neurosci., 3, 91–110. BuzsÂa ki, G., & Chrobak, J. J. (1995). Temporal structure in spatially organized neuronal ensembles: A role for interneuron networks. Curr. Opin. Neurobiol., 5, 504–510. Chow, C. C. (1998). Phase-locking in weakly heterogeneous neuronal networks. Physica, D118, 343–370. Chow, C. C., White, J. A., Ritt, J., & Kopell, N. (1998). Frequency control in synchronized networks of inhibitory neurons. J. Comput. Neurosci., 5, 407– 420. Crook, S. M., Ermentrout, G. B., & Bower, J. M. (1998). Spike frequency adaptation affects the synchronization properties of networks of cortical oscillators. Neural Comput., 10, 837–854. Ermentrout, G. B. (1994). Reduction of conductance based models with slow synapses to neural nets. Neural Comput., 6, 679–695.

54

D. Hansel and G. Mato

Ermentrout, G. B. (1996). Type I membranes, phase resetting curves, and synchrony. Neural Comput., 8, 979–1001. Ermentrout, G. B., & Kopell, N. (1986). Parabolic bursting in an excitable system coupled with a slow oscillation. SIAM J. Appl. Math., 46, 233–253. Fuhrman, G., Markram, H., & Tsodyks, M. (2002). Spike frequency adaptation and neocortical rhythms. J. Neurophysiol., 88, 761–770. Gerstner, W. (2000). Population dynamics of spiking neurons: Fast transients, asynchronous state, and locking. Neural Comput., 12, 43–89. Gerstner, W., & van Hemmen, J. L. (1993). Coherence and incoherence in a globally coupled ensemble of pulse-emitting units. Phys. Rev. Lett., 71, 312–315. Ginzburg, I., & Sompolinsky, H. (1994). Theory of correlations in stochastic neuronal networks. Phys. Rev., E50, 3171–3191. Golomb, D. (1998). Models of neuronal transient synchrony during propagation of activity through neocortical circuitry. J. Neurophysiol., 79, 1–12. Golomb, D., & Ermentrout, G. B. (1999). Continuous and lurching traveling pulses in neuronal networks with delay and spatially decaying connectivity. Proc. Natl. Acad. Sci. USA. 96, 13480–13485. Golomb, D., & Hansel, D. (2000). The number of synaptic inputs and the synchrony of large sparse neuronal networks. Neural Comput., 12, 1095–1139. Golomb, D., Hansel, D., & Mato, G. (2001). Theory of synchrony of neuronal activity. In S. Gielen & F. Moss (Eds.), Handbook of biological physics. Amsterdam: Elsevier. Golomb, D., & Rinzel, J. (1993).Dynamics of globally coupled inhibitory neurons with heterogeneity. Phys. Rev., E48, 4810–4814. Golomb, D., & Rinzel, J. (1994). Clustering in globally coupled inhibitory neurons. Physica, D72, 259–282. Gupta, A., Wang, Y., & Markram, H. (2000). Organizing principles for a diversity of GABAergic interneurons and synapses in the neocortex. Science, 287, 273– 278. Hansel, D., & Mato, G. (2001). Existence and stability of persistent states in large neuronal networks. Phys. Rev. Lett., 86, 4175–4178. Hansel, D., Mato, G., & Meunier, C. (1993a).Phase dynamics for weakly coupled Hodgkin-Huxley neurons. Europhys. Lett., 23, 367–372. Hansel, D., Mato, G., & Meunier, C. (1993b). Phase reduction and neural modeling. in functional analysis of the brain based on multiple-site recordings. Concepts in Neuroscience, 4, 192–210. Hansel, D., Mato, G., & Meunier, C. (1995). Synchrony in excitatory neural networks. Neural Comput., 7, 307–337. Hansel, D., Mato, G., Neltner, L., & Meunier, C. (1998).On numerical simulations of integrate-and-Žre neural networks. Neural Comput., 10, 467–483. Hansel, D., & Sompolinsky, H. (1992). Synchrony and computation in a chaotic neural network. Phys. Rev. Lett., 68, 718–721. Hansel, D., & Sompolinsky, H. (1996). Chaos and synchrony in a model of hypercolumn in visual cortex. J. Comput. Neurosci., 3, 7–34. Hansel, D., & Sompolinsky, H. (1998) Modeling feature selectivity in local cortical circuits. In C. Koch & I. Segev. (Eds.), Methods in neuronal modeling: from ions to networks. Cambridge, MA: MIT Press.

Asynchronous States and the Emergence of Synchrony

55

Hebb, D. (1949). The organization of behavior. New York: Wiley. Jefferys, J. G. R., Traub, R. D., & Whittington, M. A. (1996). Neuronal networks for induced “40 Hz” rhythms. Trends Neurosci., 19, 202–208. Kuramoto, Y. (1984). Chemical oscillations, waves and turbulence. New York: Springer-Verlag. Kuznetsov, Y. (1998). Elements of applied bifurcation theory (2nd ed.). New York: Springer-Verlag. Latham, P. E., Richmond, B. J., Nelson, P. G., & Nirenberg, S. (2000). Intrinsic dynamics in neuronal networks. I. Theory. J. Neurophysiol., 83, 808–827. McCormick, D. A., & Contreras, D. (2001). On the cellular and network bases of epileptic seizures. Annu. Rev. Physiol., 64, 815–846. Neltner, L., Hansel, D., Mato, G., & Meunier, C. (2000). Synchrony in heterogeneous networks of spiking neurons. Neural Comput., 12, 1607–1641. Rall, W. (1967). Distinguishing theoretical synaptic potentials computed for different some-dendritic distribution of synaptic input. J. Neurophysiol.,30, 1138– 1168. Rubin, N., & Sompolinsky, H. (1989). Neural networks with low local Žring rates. Europhys. Lett., 10, 465–470. Shelley, M. J., & Tao, L. (2001). EfŽcient and accurate time-stepping schemes for integrate-and-Žre neuronal networks. J. Comput. Neurosci., 11, 111–119. Sik, A., Penttonen, M., Ylinen, A., & BuzsÂa ki G. (1995). Hippocampal CA1 interneurons: An in vivo intracellular labeling study. J. Neurosci., 15, 6651–6665. Strogatz, H. S. (2000). Nonlinear dynamics and chaos. Cambridge: Perseus. Tarczy-Hornoch, K., Martin, K. A., Jack J. J., & Stratford K. J. (1998). Synaptic interactions between smooth and spiny neurones in layer 4 of cat visual cortex in vitro. J. Physiol., 508, 351–363. Traub, R. D., Jefferys, J. G., & Whittington, M. A. (1997). Simulation of gamma rhythms in networks of interneurons and pyramidal cells. J. Comput. Neurosci., 4, 141–150. Traub, R. D., Jefferys, J. G. R., & Whittington, M. A. (1999). Fast oscillations in cortical circuits. Cambridge, MA: MIT Press. Traub, R. D., Whittington, M. A., Colling, S. B, BuzsÂaki, G., & Jefferys, J. G. R. (1996). Analysis of gamma rhythms in the rat hippocampus in vitro and in vivo. J. Physiol. (London), 493, 471–484. Treves, A. (1993). Mean Želd analysis of neuronal spike dynamics. Network, 4, 259–284. Tsodyks, M. V., Skaggs, W. E., Sejnowski, T. J., & McNaughton B. L. (1997). Paradoxical effects of external modulation of inhibitory interneurons. J. Neurosci., 17, 4382–4388. van Vreeswijk, C. (1996). Partial synchronization in populations of pulsecoupled oscillators. Phys. Rev., E54, 5522–5537. van Vreeswijk, C. (2000). Analysis of the asynchronous state in networks of strongly coupled oscillators. Phys. Rev. Lett., 84, 5110–5113. van Vreeswijk, C., Abbott, L. F., & Ermentrout, G. B. (1994). When inhibition not excitation synchronizes neural Žring. J. Comput. Neurosci., 1, 313–321. van Vreeswijk, C., & Hansel, D. (2001). Patterns of synchrony in neural networks with adaptation. Neural Comput., 13, 959–992.

56

D. Hansel and G. Mato

van Vreeswijk, C., & Sompolinsky, H. (1996). Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science, 274, 1724–1726. van Vreeswijk, C., & Sompolinsky, H. (1998). Chaotic balanced state in a model of cortical circuits. Neural Comput., 10, 1321–1372. Wang, X.-J. (1999). Synaptic basis of cortical persistent activity: The importance of NMDA receptors to working memory. J. Neurosci., 19, 9587–9603. Wang, X.-J., & BuzsÂaki, G. (1996). Gamma oscillation by synaptic inhibition in a hippocampal interneuronal network model. J. Neurosci., 16, 6402–6413. Wang, X.-J., & Rinzel, J. (1992). Alternating and synchronous rhythms in reciprocally inhibitory model neurons. Neural Comput., 4, 84–97. White, J. A., Chow, C. C., Ritt, J., Soto-Trevino, ˜ C., & Kopell, N. (1998). Synchronization and oscillatory dynamics in heterogeneous, mutually inhibited neurons. J. Comput. Neurosci., 5, 5–16. Whittington, M. A., Traub, R. D., & Jefferys, J. G. R. (1995). Synchronized oscillations in interneuron networks driven by metabotropic glutamate receptor activation. Nature, 373, 612–615. Whittington, M. A., Traub, R. D., Kopell, N., Ermentrout, B., & Buhl, E. H. (2000). Inhibition-based rhythms: Experimental and mathematical observations on network dynamics. Int. J. of Psychophysiology,38, 315–336. Received April 25, 2002; accepted June 27, 2002.

NOTE

Communicated by Erkki Oja

A Constrained EM Algorithm for Principal Component Analysis Jong-Hoon Ahn [email protected]

Jong-Hoon Oh [email protected] Department of Physics, Pohang University of Science and Technology, Pohang, KyÆ ongbuk, Korea

We propose a constrained EM algorithm for principal component analysis (PCA) using a coupled probability model derived from single-standard factor analysis models with isotropic noise structure. The single probabilistic PCA, especially for the case where there is no noise, can Žnd only a vector set that is a linear superposition of principal components and requires postprocessing, such as diagonalization of symmetric matrices. By contrast, the proposed algorithm Žnds the actual principal components, which are sorted in descending order of eigenvalue size and require no additional calculation or postprocessing. The method is easily applied to kernel PCA. It is also shown that the new EM algorithm is derived from a generalized least-squares formulation. 1 Introduction Principal component analysis (PCA) is a statistical technique that analyzes the covariance structure of multivariate data. It takes a data set in a given orthonormal basis in Rd and Žnds a new orthonormal basis Rq with its axes ordered (d > q). This new basis is rotated in such a way that the axes are oriented along the direction in which the data have their highest variance. The principal components are the new coordinates, ordered by their corresponding variances. The reduced data can best preserve the variance of data; thus, they are widely used in data compression, image analysis, visualization, pattern recognition, regression, and time-series prediction. Situations dealing with high-dimensional data with low-dimensional structure require an efŽcient PCA technique as a dimensionality-reduction method, and a variety of methods for PCA have been introduced in the literature (Jolliffe, 1986). Nevertheless, the existing PCA methods had one common shortcoming: the absence of an associated probability density model. The derivation of PCA from the perspective of density estimation offers a number of important advantages, and the PCA model in the noiseless case provides a very Neural Computation 15, 57–65 (2003)

° c 2002 Massachusetts Institute of Technology

58

J. Ahn and J. Oh

efŽcient method for extracting a few principal components from a highdimensional data set (Tipping & Bishop, 1999; Roweis, 1997). Classic eigenvalue analysis is a quick and exact method for use with low-dimensional data, but it is unsuitable for very high-dimensional data sets. Computing a complete covariance matrix is laborious and inefŽcient when only a few principal components are required. However, the EM algorithm for PCA has rotational ambiguity, which means that it Žnds only vector sets that are linear superpositions of the principal components. In this article, we propose a coupled latent variable model for extracting the actual principal components. 2 Probabilistic Models 2.1 The Single Latent Variable Model. The single latent variable model for PCA is referred to as the probabilistic model (Tipping & Bishop, 1999). The latent variable model is linear: t D Wx C m C 2 ,

(2.1)

where the q-dimensional column vector x has a unit isotropic distribution N (0, I ). The noise is given as a normal distribution such that 2 » N (0, Y) , with Y diagonal, the d £ q matrix W contains the factor loadings, and m is a constant vector with a maximum likelihood estimator that is the mean of the given data set ftkg. Given this formulation, t is also modeled as the normal density N (m , Y C WW T ) . It is assumed that the diagonal elements of the covariance matrix Y should be identical. When the diagonal elements of Y are equal, the factor loadings eventually become the principal axes. Tipping and Bishop showed that by considering a model with an isotropic noise structure, the maximum likelihood estimator W ML is a matrix whose columns are the scaled and rotated principal eigenvectors of the sample’s covariance matrix S, W D Uq (L q ¡ s 2 I ) 1 / 2 R,

(2.2)

where the q-column vectors in Uq are eigenvectors of S, with corresponding eigenvalues in the diagonal matrix L q , and R is an arbitrary q £q orthogonal matrix. The true principal axes can be recovered when the columns of RT are equal to the eigenvectors of the W T W matrix. Although postprocessing needs few costs, it may be unwanted from a theoretical viewpoint. In order to remove the rotational ambiguity in a probabilistic PCA model, we shall consider a coupled latent variable model constructed from single models under some constraints. 2.2 A Coupled Latent Variable Model. It was shown that the q-dimensional single latent variable model Žnds q-dimensional principal subspace.

A Constrained Algorithm for PCA

59

Now let us consider a ( q ¡ 1)-dimensional single model prior to the qdimensional model. This certainly recovers (q ¡ 1)-dimensional principal subspace. Assuming that the columns of Uq are sorted in descending order of eigenvalue size, we can constrain the q £ q rotational matrix R to Rqi D 0, where i D 1, . . . , q ¡ 1. This is because a ( q ¡ 1)-dimensional principal subspace should be spanned by the set of Žrst ( q ¡ 1) principal axes. Because matrix R is orthogonal, the condition leads to Riq D 0 and Rqq D 1. Thus, we can obtain an exact qth principal vector through the coupled model constructed from two single models. We can then deduce that a coupled model constructed from single latent variable models with full dimensionality should recover the exact principal components on their convergence. The ith model in the coupled model is given by yi D WIi x C m C 2

(2.3)

i

where the q£q diagonal matrix Ii contains the factor selection with Ii ( j, j) D 1 for j D 1, . . . , i and Ii ( j, j) D 0, for j D i C 1, . . . , q. For the ith isotropic noise model 2 i » N (0, si2 I ) , the coupled model implies a probability distribution over multiple y spaces for x, given by p ( y1 D t, . . . , yq D t | x ) D D

q Y

p (t | xI i)

iD 1 q Y

(2p si2 ) ¡d / 2

iD 1

(2.4) µ

¶ 1 2 exp ¡ 2 kt ¡WIi x¡m k , 2si

(2.5)

where p ( t | xI i) is the conditional density for the ith single model. Note that R 6 1. With the latent variables x deŽned by p ( t, . . . , t | x) dt D µ ¶ 1 p (x ) D (2p ) ¡q/ 2 exp ¡ xT x , 2

(2.6)

we obtain the marginal distribution of coupled y spaces in the form of Z p (t, . . . , t) D D

p ( t, . . . , t | x) p ( x ) dx µ ¶ 1 (2p si2 ) ¡d / 2 |M| ¡1 / 2 exp ¡ ( t ¡ m ) T C ¡1 ( t ¡ m ) , 2 iD 1

q Y

(2.7)

where the model covariance is " q #¡1 X I ¡1 T T ¡ WQM Q W CD , s2 iD 1 i

(2.8)

60

J. Ahn and J. Oh

Pq Pq where M D I C iD 1 IiT W T WIi / si2 and Q D iD1 Ii / si2 . Using Bayes’ rule, the posterior distribution of the latent variables x may be calculated: µ 1 ¡q / 2 1/ 2 ( ) (2p ) | |M| . . . exp ¡ fx ¡ M ¡1 QT W T ( t ¡ m )gT p x t, ,t D 2 ¶ (2.9) ¡1 T T ( £ Mfx ¡ M Q W t ¡ m ) g , where the posterior covariance matrix is given by MDIC

q X Ii W T WIi . si2 i D1

(2.10)

M is q £ q matrix while C is d £ d matrix. 3 Derivation of Constrained EM Algorithm An expectation-maximization (EM) algorithm for PCA Žnds the principal subspace as the noise level sq2 becomes inŽnitely small (Roweis, 1997). The coupled model for the new EM algorithm also comes to be valid for inŽnitely small noises si2 . Further, we need to consider priorities among the single models that are associated with the coupled model. By the constrained algorithm, we mean that the ith single model should be treated prior to the (i C 1)th single model so that the columns of Uq are sorted in descending order of eigenvalue size. The priorities for such models are decided by the limits si2 si2C 1

¡! 0.

(3.1)

These conditions certainly give the constrained EM algorithm for PCA. 3.1 EM Algorithm. As the latent variable x is considered missing data in the EM approach, it can be expected by the posterior density, equation 2.9, and the noise condition, equation 2.11, Z | E[x t] D xp (x | t, . . . , t) dx D M ¡1 QT W T ( t ¡ m )

(3.2)

! fL(W T W ) g¡1 W T ( t ¡ m ) , where elementwise operator L is deŽned as L ( aij ) D aij for i ¸ j and L ( aij ) D 0 for i < j. The E-step may be also represented in matrix form, H D fL(W T W ) g¡1 W T V,

(3.3)

A Constrained Algorithm for PCA

61

where H is q £ n matrix, which has column vectors fx kg, and V is d £ n matrix, which has column vectors ftk ¡m g. In the M-step, the new parameter value W new that maximizes the expected value of the log likelihood for the complete data given at each iteration is determined. The part that includes P W in the complete data log likelihood LC D nkD 1 ln[p ( tk, . . . , tk | x k ) p ( xk ) ] is given by LW

" # ´ q ³ X 1 HHT T T T T D tr 2VH Q W ¡ W Ii 2 Ii W . 2 si iD 1

(3.4)

If we consider condition 3.1, this is maximized by W new as follows, ( W new

T

D VH Q

! VH

T

´) ¡1 q ³ X HHT I i 2 Ii si i D1

T fU(HHT )g ¡1

(3.5)

,

where elementwise operator U is deŽned as U ( aij ) D aij for i · j and U ( aij ) D 0 for i > j. 4 Discussion We treat the coupled latent variable model constructed from single probabilistic models in order to remove the rotational ambiguity involved in the single probabilistic model. The constrained EM algorithm derived from them gradually recovers the actual principal components, and it is naturally applied to the kernel PCA, H D fL(C T KC ) g ¡1 C T K

C

new

T

D H fU(HH

T )g ¡1

,

(4.1) (4.2)

which are derived in appendix A. This article proposes using the coupled latent variable model to perform the PCA. The source vector x is considered a hidden variable that is linearly related to the observed vectors yi . Different isotropic noises 2 i are added to the observed data vectors under different model conditions Ii , but all the observed vectors are modeled by the same vector t. By the coupled model, we mean this model description. The EM algorithm is employed to Žt linear combinations of these source distributions to the observed data. The parameter to be estimated is the factor loading W (Dempster, Laird, & Rubin, 1977). The forms of iteration rules are similar to those used in the previous EM algorithm, where simple pseudo-inverse solutions are used for the leastsquare formulation (Roweis, 1997). But it is clearly distinguished by the

62

J. Ahn and J. Oh

lower and upper operators that leave the matrix elements unchanged or changed to zero.1 Thus, there is virtually no additional or postprocessing cost in using the method. The cost per iteration is the same for both algorithms, via elapsed time or total oating-point operations. Ignoring constants, they are both O ( n £ d £ q) , where n = data number, d = original dimensionality, and q = reduced dimensionality. The postprocessing cost for the EM algorithm to Žnd the orthogonal vectors is O (n £ q2 C d £ q2 ) , which is smaller than the cost of one iteration. The previous EM algorithm for PCA minimizes the squared error function monotonically. However, the constrained EM algorithm shows a slightly different convergent behavior. It is also derived from the solution for minimizing the generalized squared error function, which is given by a weighted squared error function summed over the required dimensionality with positive constants ai :

JD

q X iD 1

ai kV ¡ WIi Hk2 .

(4.3)

Using the generalized least-squares formulation and condition 3.1 as a ( i C 1) / ai ! 0, we can obtain the same iteration rules 3.3 and 3.5 that are derived in appendix B. Thus, we can learn that the iterative rules B.5 and B.6 monotonically minimize the generalized squared error function 4.3. Although the proposed method for squared cost function is generally slower than the EM algorithm while converging, the two methods Žnally recover the principal subspace in the same number of iterations. The iterative number required to recover the orthogonal axis is based on the ratios of eigenvalues li / li C 1 . Appendix A: Constrained EM Algorithm for Kernel PCA The constrained EM algorithm for PCA can be naturally applied to kernel PCA (Rosipal & Girolami, 2001). The basic idea is that the factor loadings W can be expressed as W D VW C,

(A.1)

where VW is a nonlinearly mapped data matrix for observed data V 2 Rd£n and C 2 Rn£q contains coefŽcients. Using this expression and the kernel 1 The explicit forms of the operators are related to our assumption that the columns of Uq are sorted in descending order of the eigenvalue size. Thus, we can obtain well-ordered principal components.

A Constrained Algorithm for PCA

63

matrix K, we can write the matrix W T W and W T VW as T W T W D C T VW VW C

D C T KC

(A.2)

D C T K.

(A.3)

T W T VW D C T VW VW

Thus, from the E-step, equation 3.3, the matrix H 2 Rq£n can be updated as H D fL(C T KC )g ¡1 C T K.

(A.4)

Considering expression A.1, we can write the M-step, equation 3.5, in the matrix form VW C new D VW H T fU(HHT )g ¡1 . Then the M-step follows: C new D HT fU(HHT )g ¡1 .

(A.5)

Appendix B: The Generalized Squared Error Formulation The treatment of a coupled model is supplanted by introducing a generalized squared error function. Our problem is to seek the weight matrix W and the hidden matrix H that minimize the error 4.3. The problem of minimizing the single squared error is a classic one that can be solved by a gradient search procedure. Similarly, the generalized squared error problem is solved by forming the gradient,

rH J D rW J D

q X iD 1 q X iD 1

2ai ( WIi ) T (V ¡ WIi H )

(B.1)

2ai ( V ¡ WIi H ) ( Ii H ) T ,

(B.2)

and setting it equal to zero. This yields the necessary condition (

q X

) ai ( WIi

)T

VD

iD 1

(

V

q X i D1

(

) ai ( Ii H

)T

q X

) ai ( WIi

)T(

WI i ) H

i D1

(

D W

q X i D1

(B.3)

) ai ( Ii H ) ( Ii H

)T

.

(B.4)

64

J. Ahn and J. Oh

Pq Pq Because iD 1 ai (WIi ) T (WIi H ) and iD1 ai ( WIi H ) ( Ii H ) T are nonsingular, we can solve for H and W uniquely as ( q )¡1 Á q ! X X T H D ai ( WIi ) (WIi ) ai Ii W T V iD 1

D

Á

f(W T W ) ¯ Ag¡1 2

D

4(WTW ) ¯

D

f(W T

D

T

q X

ai Ii W T V

iD 1

8Á q < X :

i D1

!

i D1

W ) ¯ Dg

¡1

Á q X

!(

!¡1 93 ¡1 = ai Ii A 5 WT V ;

(B.5)

WTV

and W

D

VH VH

T

i D1 Á q X

D

a i ( Ii H ) ( Ii H

)T

)¡1

iD 1

!

ai Ii f(HHT ¯ Ag ¡1

8 Á !¡1 93¡1 q < = X 5 VH T 4 ( HH T ) ¯ A ai Ii : ; D1 2

D

i D1

ai Ii

q X

i

VH T f(HHT )

¯

DT g¡1 ,

where ¯ denotes the Hadamard product and A is given by Pq Pq 0 Pq 1 ¢ ¢ ¢ aq i D 1 ai i D 2 ai i D3 ai B Pq C Pq Pq B ¢ ¢ ¢ aq C i D 2 ai i D3 ai B iD 2 ai C B Pq C Pq Pq B C ¢ ¢ ¢ a a a a qC A D B iD 3 i iD 3 i i D3 i B C B . C .. .. .. B .. C . . . @ A aq

aq

and D is represented as Pq 0 ai 1 P iqD 2 a B iD 1 i B B B1 1 B DDB B B. .. B. . @. 1

(B.6)

1

aq

Pq a PiqD 3 i iD 1

ai

Pq a PiqD 3 i iD 2

ai

¢¢¢

¢¢¢

Paqq

¢¢¢

Paqq

.. .

.. .

¢¢¢

¢¢¢

aq 1 ai C

C C C C a iD 2 i C . C .. C C . A iD 1

1

(B.7)

(B.8)

A Constrained Algorithm for PCA

65

These iterative rules can minimize the generalized squared error most efŽciently. If we take condition 3.1 as ai C 1 / ai ! 0 , all upper elements in the D matrix approach zero, and the iteration rules become identical to equations 3.3 and 3.5. Acknowledgments This research was funded by the Brain Science and Engineering Research Program, Ministry of Science and Technology, in Korea. References Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm Journal of Royal Statistical Society, 39, 1–38. Jolliffe, I. T. (1986). Principal component analysis. New York: Springer-Verlag. Rosipal, R., & Girolami, M. (2001). An expectation-maximization approach to nonlinear component analysis. Neural Computation, 13, 505–510. Roweis, S. R. (1997). EM algorithms for PCA and SPCA. In M. Kearns, M. Jordan, & S. Solla (Eds.), Advances in neural information processing systems, 10 (pp. 626– 632). Cambridge, MA: MIT Press. Tipping, M. E., & Bishop, C. M. (1999). Mixtures of probabilistic principal component analyzers. Neural Computation, 11, 443–482. Received July 26, 2001; accepted June 20, 2002.

LETTER

Communicated by Emilio Salinas

Higher-Order Statistics of Input Ensembles and the Response of Simple Model Neurons Alexandre Kuhn [email protected] Ad Aertsen [email protected] Stefan Rotter [email protected] Neurobiology and Biophysics, Biology III, Albert-Ludwigs-University, D-79104 Freiburg, Germany

Pairwise correlations among spike trains recorded in vivo have been frequently reported. It has been argued that correlated activity could play an important role in the brain, because it efŽciently modulates the response of a postsynaptic neuron. We show here that a neuron’s output Žring rate critically depends on the higher-order statistics of the input ensemble. We constructed two statistical models of populations of spiking neurons that Žred with the same rates and had identical pairwise correlations, but differed with regard to the higher-order interactions within the population. The Žrst ensemble was characterized by clusters of spikes synchronized over the whole population. In the second ensemble, the size of spike clusters was, on average, proportional to the pairwise correlation. For both input models, we assessed the role of the size of the population, the Žring rate, and the pairwise correlation on the output rate of two simple model neurons: a continuous Žring-rate model and a conductance-based leaky integrate-and-Žre neuron. An approximation to the mean output rate of the Žring-rate neuron could be derived analytically with the help of shot noise theory. Interestingly, the essential features of the mean response of the two neuron models were similar. For both neuron models, the three input parameters played radically different roles with respect to the postsynaptic Žring rate, depending on the interaction structure of the input. For instance, in the case of an ensemble with small and distributed spike clusters, the output Žring rate was efŽciently controlled by the size of the input population. In addition to the interaction structure, the ratio of inhibition to excitation was found to strongly modulate the effect of correlation on the postsynaptic Žring rate.

Neural Computation 15, 67–101 (2003)

° c 2002 Massachusetts Institute of Technology

68

A. Kuhn, A. Aertsen, and S. Rotter

1 Introduction

The brain consists of a huge number of neurons that are heavily interconnected. These cells interact by producing rapid depolarizations of their membrane, the action potentials (spikes) that are propagated along the axon and excite or inhibit postsynaptic neurons by inducing postsynaptic potentials (PSPs). The integration of PSPs can bring the postsynaptic neurons, in turn, to Žre a spike. Such interactions within the neuronal network lead to temporal correlations between spike trains of different neurons. Pairwise correlations of spike trains are routinely observed in virtually all regions of the brain. They have been related to the representation of sensory information (DeCharms & Merzenich, 1996) and motor behavior (Baker, Spinks, Jackson, & Lemon, 2001), but also to more abstract processing levels (Steinmetz et al., 2000; Salinas & Sejnowski, 2001). The effect of correlated synaptic input on the neuronal response has been investigated in several recent theoretical and numerical studies (Feng & Brown, 2000; Salinas & Sejnowski, 2000; Svirskis & Rinzel, 2000; Feng & Zhang, 2001; Stroeve & Gielen, 2001). They showed that temporal correlation can have a tremendous effect on the rate and variability of the output of model neurons. Pairwise correlations are generally used to characterize the statistical dependence of parallel spike trains. However, as pointed out, for example, by Bohte, Spekreijse, and Roelfsema (2000), an ensemble of spike trains is not completely described by pairwise interactions. Thus, the effect of pairwise correlation cannot be studied without also taking into account correlations among three or more spike trains (i.e., higher-order interactions). This point is illustrated by the following example. Under the general assumption that only the timing of presynaptic spikes and not their form is important, we can model series of incoming spikes as point processes (Tuckwell, 1988; Cox & Isham, 1980). A presynaptic ensemble of N spike trains can then be thought of as N parallel point processes. We deŽne an interaction as the occurrence of simultaneous spikes in different spike trains. The interactions can be classiŽed as second, third, . . . , Nth-order, corresponding to clusters of 2, 3, . . . , N simultaneous spikes. For simplicity, we assume that the ensemble is homogeneous, that is, the Žring rates of all spike trains are identical and, similarly, the rates of second, third, . . . , Nth-order interactions are identical for any subset of 2, 3, . . . , N spike trains picked from the ensemble. We now consider a homogeneous ensemble of N parallel spike trains where all spikes are associated with pairwise (second-order) interactions: there are no interactions of order 3 or higher, nor are there isolated spikes, that is, spikes in one spike train without a counterpart in another one. If k is the rate of each second-order interaction, the discharge rate r of an individual spike train is r D ( N ¡ 1)k . The correlation coefŽcient between two point processes in a homogeneous system is the rate of interactions with simultaneous spikes on both processes, divided by the

Higher-Order Statistics of Input Ensembles

69

discharge rate of an individual process (see appendix A). Thus, in the case of an ensemble with only second-order interactions, the correlation coefŽcient is c D 1 / ( N ¡1) . It is approximately inversely proportional to the number of presynaptic inputs. For N D 100 correlated spike trains, the correlation coefŽcient will be about 10¡2 . Stronger pairwise correlations can be obtained only by introducing higher-order interactions. A single third-order interaction (or triplet), for instance, contributes to the pairwise correlations in three pairs of spike trains, while introducing only one spike per process. The same second-order interaction is obtained by having one pair of spikes in each of the three pairs of spike trains. This results in two spikes in each single process, that is, twice as many as in the triplet case, which results in a halved correlation coefŽcient. In a homogeneous high-dimensional system, certain combinations of rates and pairwise correlations thus imply the existence of higher-order interactions. Biological neural networks are typically not homogeneous (in the sense deŽned above), but very little is known about their actual interaction structure. The assessment of the statistical signiŽcance of simultaneous events across more than two neurons is a nontrivial problem. Different statistical methods have been proposed (Martignon, von Hasseln, Grun, ¨ Aertsen, & Palm, 1995; Martignon et al., 2000; Gutig, ¨ Aertsen, & Rotter, 2002). Here, we report on a study of the effects of higher-order interactions on the response of model neurons. SpeciŽcally, we constructed two different spike train ensembles and compared their effects on the response of a Žringrate (FR) neuron model and a conductance-based leaky integrate-and-Žre (IF) model. For both neuron models, we assessed the effects of the pairwise correlation in the input on the output Žring rate of the neuron. Analytical results were derived for the FR neuron. Preliminary results have been presented elsewhere (Kuhn, Rotter, & Aertsen, 2002). 2 Two Models for Correlated Neuronal Populations

The realization x (t) of a stochastic point process consists of a sequence of points in time . . . , tk, tkC 1 , . . . that can be represented by a pulse train, X d ( t ¡ tk ) , x ( t) D k

where d (t ) is the Dirac delta function. To reduce the number of parameters deŽning the input ensemble, we model spike trains as parallel Poisson point processes with homogeneous interaction structures. Moreover, we restrict our treatment to instantaneous statistical dependence, that is, the interaction among spike trains originates from simultaneous spikes only. 2.1 Single Interaction Process Model. The single interaction process (SIP) model was motivated by results of Deppisch et al. (1993) and Bohte et al. (2000). They simulated a network of integrate-and-Žre neurons and

70

A. Kuhn, A. Aertsen, and S. Rotter

observed that it could show periods of random looking, unsynchronized neuronal activities, but also population bursts, with almost all neurons Žring synchronously. The SIP model is deŽned as follows. First, a single realization wu ( t) of a Poisson process with stationary rate a is generated. Each process xi (t ) in a set of N spike trains is then deŽned by xi ( t) D wu ( t) C wi (t )

(i D 1, 2, . . . , N ) ,

where wi (t) is an independent realization of a Poisson process with rate b. The superposition of independent Poisson processes is again a Poisson process with a rate equal to the sum of the individual rates (see Cox & Isham, 1980). Thus, the rate of xi (t ) is r D a C b. Figure 1A shows the realization wu ( t) (top) that is copied into each individual spike train xi ( t) (bottom), introducing a statistical dependence between them. We characterized the statistical dependence between two spike trains by the count correlation coefŽcient, c D a/ (a C b ) . Note that the ensemble model is completely constrained by the rate r and the pairwise count correlation coefŽcient c. The relations a D rc and b D r (1 ¡c) are used for the numerical simulation of a set of spike trains with given rate and given pairwise correlation. In the literature, the pairwise dependence is frequently expressed by the cross-correlation. The cross-correlation function R ( D t) of two spike trains xi ( t) and xj ( t) is R (D t) D ad ( D t) C (a C b ) 2 . A derivation of the cross-correlation function and the count correlation coefŽcient is presented in appendix A. When using the SIP model as input to a point neuron, all individual spike trains xi ( t) are summed up. The compound process can, in turn, be decomposed into two independent processes: a Poisson process with rate a, in which each point accounts for N simultaneous spikes, and a Poisson process of rate Nb, resulting from the sum of all independent realizations wi (t) . The dependence between spike trains in the SIP model is thus described by a single interaction process (of order N). We can write X X X [wu (t) C wi ( t) ] D Nwu (t ) C xi ( t ) D wi (t) . i

i

i

The decomposition into two independent processes greatly facilitates the analysis of the neuronal response. This model of correlated spike trains also appears in Feng and Brown (2000) and Stroeve and Gielen (2001).

Higher-Order Statistics of Input Ensembles

A

Single Interaction Process

B

71

Multiple Interaction Process

w

u

wg

x1 x2

x1 x2

x

x

N

N

time

time

C

SIP

­1

sum (s )

spike train index

1

D

MIP 1

10

10

20

20

30

30

40

40

104

104

0

0

0.1 0.2 0.3 0.4 0.5 time (s)

0 0

0.1 0.2 0.3 0.4 0.5 time (s)

Figure 1: Two models of spike train ensembles with identical rates and pairwise correlations but different higher-order statistics. (A) The single interaction process (SIP) model. Each spike train xi ( t) of the ensemble is composed of a Žxed realization wu ( t) of a Poisson process with rate a and an independent realization of a Poisson process with rate b. (B) The multiple interaction process (MIP) model. Each spike train xi ( t) is realized by randomly deleting spikes from a Žxed realization w g ( t) . The generating process is a realization of a Poisson process with rate c . (C) Fifty spike trains with a rate of 20 spikes per second and pairwise correlation coefŽcient 0.1, generated according to the SIP model. Their summed activity is shown below (bin size 5 ms). (D) Fifty spike trains with the same rate and pairwise correlation as in A, but generated according to the MIP model.

72

A. Kuhn, A. Aertsen, and S. Rotter

2.2 Multiple Interaction Process Model. In the second model of a spiking population, the interaction structure is composed of spike clusters involving only subsets of the spike trains. The multiple interaction process (MIP) model is constructed as follows. Each individual spike train is a thinned version (Cox & Isham, 1980) of one and the same realization w g ( t) of a Poisson process. Individual spike trains xi (t ) are thus produced by random deletions of pulses in w g ( t) . The deletions are independent. The thinning process is repeated for all spike trains in the ensemble. If the rate of w g ( t) is c and the probability of deletion is (1 ¡ 2 ), then the rate of each individual spike train is

r Dc2 . Figure 1B shows the generating process w g ( t) (top) and the resulting correlated spike trains x1 (t) , . . . , xN (t ) (bottom). Note that the pulses in each individual spike train are Poisson (Snyder & Miller, 1991). The count correlation coefŽcient is cD2 . Again, the rate r and correlation coefŽcient c together completely constrain the model, and the relations 2 D c and c D r/ c are used for the numerical simulation of the ensemble of spike trains. The cross-correlation function of any two spike trains xi ( t) and xj (t ) is R (D t) D c 2 2d ( D t) C (c 2 ) 2 . The derivation of the cross-correlation function and the count correlation coefŽcient is again given in appendix A. The sum of the N individual spike trains results in a pulse train similar to the generating realization wg ( t) , but with each pulse scaled differently. The scalar amplitude n represents the number of (synchronous) spikes at this point in time. As the deletion of a certain pulse in wg ( t) occurs with a Žxed probability and is independent for each of the N individual spike trains, the number n follows a binomial distribution: Á ! N n n » B(nI 2 , N ) D 2 (1 ¡ 2 ) N¡n . n The scalar n is independent for each point in the compound process. The sum process can thus be decomposed into N C 1 independent Poisson processes (Snyder & Miller, 1991), each process containing only points with identical multiplicity. Such a process is called an interaction process of order n, with n 2 f0, 1, . . . , Ng. Evidently, the rate of the interaction process of order n is c B(nI 2 , N ) . Figures 1C and 1D show realizations of an ensemble of 50 spike trains, generated according to the SIP and the MIP model, respectively. In both

Higher-Order Statistics of Input Ensembles

73

cases, the individual input rate r and the count correlation coefŽcient c were identical (r D 20 spikes per second, c D 0.1). The compound activity of all spike trains is shown below the spike rasters. Although any single spike train, as well as any pair of spike trains picked from the two ensembles, would be statistically indistinguishable, the population activity is clearly different for both models. 3 Results

The effects of both ensemble models were tested on two neuron models. To reduce the complexity of the input, we Žrst focused on a simpliŽed setting. We assume the input population to be composed of two subpopulations, one excitatory and one inhibitory, with the same size and rate of the excitatory and the inhibitory populations. The excitatory presynaptic spike trains were correlated, generated according to the SIP or the MIP model, whereas the inhibitory presynaptic spike trains were modeled as independent Poisson processes. We assessed the inuence of the input parameters, and in particular of the excitatory pairwise correlation, on the mean output rate of two types of model neurons: the FR neuron and the conductance-based leaky IF neuron. The output of the FR neuron is represented by a continuous function of time, the instantaneous Žring rate of the neuron, and could be treated analytically. The response of the IF neuron was studied by means of numerical simulations. In addition, the latter model was studied for inputs with different proportions of excitation and inhibition. 3.1 Response of the Firing Rate Neuron. The continuous FR neuron is widely used in neural network studies. Its membrane potential U ( t) is described by

C

1 d U ( t) D ¡ U ( t) C I ( t) , dt R

(3.1)

with C the membrane capacitance, R the membrane resistance, and I ( t) the input current. The membrane potential U (t) is a low-pass version of the input current I ( t) . The output Žring rate ro ( t) of the neuron is given by the activation function, which is assumed to be an instantaneous function of the potential. Here, we used the Heaviside step function H ( x ) , ( H ( x) D

0 1

for x < 0 for x ¸ 0.

(3.2)

In this case, the output rate of the neuron is expressed by ro ( t) D KH ( U ( t) ¡ h ) ,

(3.3)

74

A. Kuhn, A. Aertsen, and S. Rotter

where h represents the threshold and K is a constant with dimension spike per second. Alternatively, a sigmoidal function is often used as activation function. In our case, however, we used the step function because it greatly facilitates the analysis of the neuronal response. Since we are interested only in the qualitative behavior of the output rate, the constant K will not be considered any further, and ro ( t) will be dimensionless. An example of the response of the membrane potential U ( t) to the input described above is shown in Figure 2. Synapses were modeled by current pulses. The PSPs were thus represented by an instantaneous jump of potential (positive or negative, according to the excitatory or inhibitory nature of the synapse), followed by an exponential decay toward rest, with time constant t D RC D 10 ms. Excitatory and inhibitory PSPs had the same absolute amplitude, Žxed at a value of 1. The excitatory and inhibitory input population size was N D 100 in this example, and the individual excitatory and inhibitory input Žring rate was r D 20 spikes per second. The excitatory input was modeled by the SIP model (see Figure 2A), or by the MIP model (see Figure 2B), and the pairwise correlation coefŽcient was c D 0.4, in both cases. Observe that for both input models, the membrane potential shows large depolarizations caused by the occurrence of synchronous excitatory presynaptic spikes. These depolarizing events had, however, different characteristics for both input models, as we will see in more detail in the following. 3.1.1 Amplitude Distribution of the Membrane Potential. The probability density function (pdf) of the amplitude distribution of the membrane potential is represented in Figure 2 next to sample time courses of the membrane potential. The histogram is an estimate generated from a 200 s simulation, with a bin size of 4 (in units of the PSP amplitude). The thick black lines represent an analytical approximation of part of the pdf. As the output rate of the FR neuron is an instantaneous function of the membrane potential, knowledge of the pdf of the potential is sufŽcient to derive the mean output rate of the neuron. We now describe the main assumptions made to obtain the approximation of the pdf; a full derivation for the potential and the mean output rate is given in appendix B. The approximation of the pdf is based on the decomposition of the input into independent components (cf. section 2) and the use of a classical result of shot noise theory. The membrane potential of a FR neuron can be considered as a Žltered point P process. Assuming identical synapses, it can be represented by U (t ) D k h ( t ¡ tk ) , with t k denoting the times of occurrences of synaptic events and h ( t) the impulse response of the Žlter realized by the synapse and the membrane. For synapses modeled as current pulses, h (t) is simply the impulse response of the membrane, given by ( h ( t) D

0 Ae¡t / t

for t < 0 for t ¸ 0,

(3.4)

Higher-Order Statistics of Input Ensembles

A

120

B

SIP

120

40 0

C

120

0.2 0.4 time (s)

­ 40 0

10­ 3 10­ 1 p.d.f.

SIP, approximated

D

120

0.2 0.4 time (s)

10­ 3 10­ 1 p.d.f.

MIP, approximated

80 potential

potential

40 0

80 40 0 ­ 40 0

MIP

80 potential

potential

80

­ 40 0

75

40 0

0.2 0.4 time (s)

10­ 3 10­ 1 p.d.f.

­ 40 0

0.2 0.4 time (s)

10­ 3 10­ 1 p.d.f.

Figure 2: Time course (left) and probability density function (pdf, right) of the membrane potential of the Žring rate (FR) neuron. The membrane potential is expressed in units of the PSP amplitude. (A) The excitatory input to the FR neuron consisted of 100 excitatory spike trains, generated according to the SIP model (individual Žring rate is 20 spikes per second, pairwise correlation 0.4). The inhibitory input was composed of 100 independent Poisson processes with a rate of 20 spikes per second. The pdf estimated from a 200 s simulation of the membrane potential is shown on the right (histogram with a bin width of 4). A segment of the analytical approximation of the pdf (see text) is overlaid (thick black line). The dotted line indicates the Žring threshold. (B) Same input rate and pairwise correlation as in A, but using the MIP model for excitatory inputs. (C) Approximation (see text for details) of the membrane potential with the SIP model as excitatory input. The approximation retained the large uctuations caused by clusters of synchronous input spikes (compare with A). The estimated pdf of the approximated membrane potential (from a 200 s simulation of the approximated membrane potential, bar histogram) matched the analytically calculated pdf (thick black line, same as in A). (D) Approximation of the membrane potential using the MIP model for excitatory inputs. Again, the estimated pdf (histogram) matched the analytical expression (thick black line, same as in B).

76

A. Kuhn, A. Aertsen, and S. Rotter

with A being the postsynaptic potential amplitude and t the membrane time constant. If the synaptic events follow a Poisson statistics, the Žltered process is called a shot noise (Papoulis, 1991). If the rate of the incoming events is large compared to the time constant of the Žlter, the distribution of membrane potentials is approximately gaussian (see Papoulis, 1991). In the case of our two input models, however, the interaction structure does not admit a normal approximation. The right panels in Figures 2A and 2B show that the membrane potential densities are clearly not gaussian. Gilbert and Pollak (1960) derived an exact expression for the pdf of a shot noise with exponential impulse response. They showed that for A D 1 and t D 1, the shot noise U (t) with underlying rate of pulses l D 1, has density ( for 0 · U < 1 e ¡g P (U ) D ¡g e (1 ¡ log U ) for 1 · U < 2, R1 where g D ¡ 0 e ¡t log t dt, Euler ’s constant. In fact, the analytic expression of P ( U ) changes at U D 1, 2, 3, . . . The expression for a given segment is derived recurrently from the expression of the preceding segment. The case with parameters A, t , and l arbitrary can be extended from the results of Gilbert and Pollak (1960) and is shown in appendix B.

3.1.2 Analytic Approximation of Membrane Potential Distribution. What is the use of this result for the derivation of the membrane potential density of the FR neuron to the SIP and MIP input models? If the proportions of excitation and inhibition are equal (i.e., the input is balanced), the neuron’s output Žring rate is driven by the uctuations of the membrane potential above the threshold. Synchronous clusters of input spikes induce large excursions of the membrane potential that are likely to constitute the major contribution to the neuronal output rate. If, in addition, the nonsynchronous input spikes (excitatory or inhibitory) do not signiŽcantly contribute to the membrane potential excursions, but only to the mean membrane potential level, we can approximate the membrane potential by the Žltered interaction process (i.e., the process with simultaneous spikes), together with a constant membrane potential offset due to the nonsynchronous input spikes. For the input with the SIP model, the interaction process has Žxed amplitude N and rate rc, and the nonsynchronous input consists of an excitatory spike train with rate Nr (1 ¡ c) and an inhibitory spike train with rate Nr. The resulting mean membrane potential offset is given by Campbell’s theorem and amounts to ¡Nrct (see appendix B). We can write P (U ) ¼ PeN ( U C Nrct ) , where the pdf of the potential response to the SIP model is approximated by a shifted version of the pdf of the Žltered Nth-order interaction process PeN ( U ) . The latter can be derived on the basis of Gilbert and Pollak’s method.

Higher-Order Statistics of Input Ensembles

77

The rigorous justiŽcation of this approximation is given in appendix B. Figure 2C shows an example of the time course of the approximated membrane potential, with the same parameters as in Figure 2A (left panel). Observe that the small uctuations caused by the nonsynchronous input spikes disappeared, but their mean level was retained. A similar approximation can be made for input deŽned by the MIP model. To be able to use the shot noise density directly, we make the additional assumption that the stochastic spike clusters generated by the MIP model can be approximated by the average cluster; we replace the N C 1 interaction processes of order 0 to N by a single interaction process of order (amplitude) Nc and rate r/ c. The density of the potential response to the MIP model thereby becomes P (U ) ¼ PenN ( U C Nrt ) , where PenN (U ) is the pdf of the Žltered average interaction process and ¡Nrt the mean membrane potential offset, caused by the independent inhibitory input. Again, we refer to appendix B for a complete derivation of this result. An example of the time course of the approximated membrane potential response to the input with the MIP model is shown in Žgure 2D (left panel). Because of the piecewise expression of the shot noise pdf, the convenience of our approximation depends on the number of segments that must be taken into account. For a membrane’s impulse response of amplitude A (see equation 3.4), the expression changes its form at integer multiples of A. For the SIP model (see Figure 2C, right), the neuron’s response to a synchronized spike cluster has amplitude N. We used the Žrst segment of the pdf, extending over [¡Nrct, ¡N ( rct ¡ 1) [. (We use the conventional mathematical notation x 2 [a, b[ to indicate a · x < b.) For the parameter set considered here, the approximation of the potential holds for U 2 [¡8, 92[ in units of A. Comparison with the histogram estimated from 200 second simulation of the approximated membrane potential shows an excellent correspondence. For the MIP model (see Figure 2D, right), we used the Žrst two segments. As we reduced the model’s interaction structure to the average spike cluster, the neuron’s response to a synchronized cluster has amplitude Nc, and the Žrst two segments cover [¡Nrt, ¡N(rt ¡ 2c) [, which corresponds to [¡20, 60[. Note that the change of the analytic form of the pdf can be recognized from the sharp bend at U D ¡N ( rt ¡ c) D 20. Again, a numerical estimate from the simulation of the approximated membrane potential (see the histogram in Figure 2D, right) shows excellent correspondence. If we now compare the analytical approximation of the pdf with the pdf estimated from a simulation of the full input model (Figures 2A and 2B, right), the agreement is very good. A small discrepancy can be observed only near the lower bound of the domain considered. There, our approximation neglected the uctuations due to the nonsynchronous input spikes that spread the membrane potential around the mean value.

78

A. Kuhn, A. Aertsen, and S. Rotter

3.1.3 Mean Response Rate. We can now consider the mean output rate ro for the FR neuron, Z 1 (3.5) ro ´ E[ (ro (t) ] D P ( U ) dU D 1 ¡ Q (h ) , h

Rx

where Q ( x ) D ¡1 P ( U ) dU, the cumulative probability distribution function of the membrane potential. We thus obtain an approximation of the mean output rate by integrating over the approximated density, which has to be known between ¡1 and h . In Figure 2, the threshold h is represented by a dotted line and was set to 30. The number of segments of the pdf that must enter the calculation of ro is thus determined by the value of the input parameters, the threshold, and the membrane time constant. For the SIP model, an approximation of the mean output rate based on the Žrst segment of the probability distribution is µ ³ ´rct ¶ 1 e h C rct ro ¼ 1 ¡ D , N rct

(3.6)

R e D e ¡rct g / C ( rct ), where C ( z) D 1 tz¡1 e¡t dt is the gamma function. with D 0 It is valid for ¡rct · h / N < ¡(rct ¡ 1) . For larger h , additional segments of the distribution have to be included. For the MIP model, the approximation based on the Žrst two segments is given in appendix B in equation B.2. We used these approximations to study the output rate of the FR neuron in response to both types of input. The input was described by three parameters: the size N of the input population, the rate r of individual inputs, and the correlation coefŽcient c between any two individual excitatory inputs. N and r were identical for the excitatory and the inhibitory populations. We examined how the input correlation coefŽcient c inuenced the mean output rate ro of the FR neuron. In addition, we systematically varied N and r. The results are shown in Figure 3. Analytical approximations are represented by thick gray lines, and results obtained by numerical simulations are given by black dots. Figure 3A shows the output rate ro as a function of the input correlation coefŽcient c for several input population sizes N D 40, 100, 500, 1000 and a Žxed input rate of r D 20 spikes per second. The SIP model was used as the excitatory input ensemble. For all N, ro increased monotonically with the correlation coefŽcient. Indeed, as c grows larger, the rate of occurrence of simultaneous excitatory input spikes, given by rc, increases, and membrane potential excursions above threshold become more frequent, leading to an increase of the mean output rate. Nevertheless, observe that the overall increase of ro does not scale with N and tends to saturate rapidly for larger input population sizes. In the limit N ! 1, we have ro ¼ 1 ¡

( rct ) rct ¡1 ¡rct g . e C ( rct )

Higher-Order Statistics of Input Ensembles

79

This limit is plotted as a dashed line in Figure 3A. Figure 3B features the output rate of the FR neuron for identical input conditions, with an interaction structure of the excitatory population generated according to the MIP model. For small values of c, the response of the FR neuron showed a considerably stronger dependence on the ensemble size than in the SIP model. For N D 40, for instance, the initial increase of ro was smaller than with the SIP model. By contrast, for N D 1000, a small input correlation induced a much larger output rate with the MIP model. For larger N, a further increase of c led to a decrease of ro . Note that for c D 1, both correlated input models are statistically identical, and the responses of the FR neuron were thus the same. For both the SIP and MIP input models, the analytical approximations of ro for the FR neuron model were in very good agreement with the simulation results, showing that the assumptions made were appropriate. As hypothesized, the response of the neuron is driven by the occurrence of excitatory spike clusters for the largest part of the correlation range. If c approaches 0, the rate of these synchronous events tends to 0, and the approximate treatment of the SIP model degrades. The reason is that it neglects the contribution of nonsynchronous excitatory input spikes, the rate of which is Nr (1¡c), increasing for smaller c. This is shown in the inset of Figure 3A, depicting the output rate of the FR neuron for c D 0.005, 0.01, . . . , 0.1. Whereas the approximation tends to 0, the neuron response to the full SIP input model has nonzero values at c D 0, for N D 1000. The success of the approximation near c D 0 thus critically depends on the values of r and N. For the response to the MIP model, we limited the approximation to the Žrst two segments of the pdf and thus could not approximate the response of the FR neuron over the whole range of c (see Figure 3B). For small correlation coefŽcients, the size of the mean synchronous spike cluster Nc is small compared to the threshold, and many segments of the shot noise density must be taken into consideration. The success of the analytical approximation here indicates that the stochastic nature of the size of synchronous clusters does not play a signiŽcant role for the mean output rate. This independence of the stochastic nature of the cluster size allows us to explain the observed effects in terms of the average cluster. For small values of c, the rate r/ c of the mean cluster is large. The frequent occurrence of small clusters (with size proportional to c) results in their accumulation in the membrane potential, which drives the potential above threshold h (see Figure 2B) and increases the output rate. Nevertheless, the rate of occurrence of the synchronous input events decreases with increasing c. The clusters become larger but less frequent with increasing c. If their mean inter-event interval (inverse rate) is smaller than the membrane time constant, they do not accumulate, and the increase of the cluster size cannot compensate for their less frequent occurrence. For a large input population size, these two opposing effects result in a non-monotonic dependence of the output rate on the input correlation (see Figure 3B).

80

0.05

0.3

0 0

0.05 0.1

0.2

100

0.1 0 0

C

0.5

D

SIP 90 s­ 1 ­1 50 s

output

20 s­ 1 10 s

0.1

0.3

1000 500

0.2 0 0

0.2 0.4 0.6 0.8 1 correlation coefficient

0.3

0 0

MIP

100

0.1

40

0.4 0.2

0.5 0.4

¥ 1000 500

output

output

0.4

B

SIP

0.5

­1

0.5 0.4

output

A

A. Kuhn, A. Aertsen, and S. Rotter

0.3 0.2

40

0.2 0.4 0.6 0.8 1 correlation coefficient MIP 90 s­­ 11 50 s 20 s­ 1 ­1

10 s

0.1 0.2 0.4 0.6 0.8 1 correlation coefficient

0 0

0.2 0.4 0.6 0.8 1 correlation coefficient

Figure 3: Mean output of the Žring rate (FR) neuron as a function of the pairwise correlation coefŽcient in the population of excitatory inputs, for the two statistical input models. Relative units are used (see text). Thick gray lines are obtained from analytical approximations (see text); the black dots represent the results of simulations (duration 500 s each). (A) The excitation was generated according to the SIP model, and the inhibition was represented by a set of independent Poisson processes. Excitatory and inhibitory population size and rate were identical. The output rate of the FR neuron is shown for different input population sizes (N D 40, 100, 500, 1000). The individual input rate was held constant (r D 20 spikes per second). The analytically computed output rate of the FR neuron for inŽnite population size is plotted as a dashed line. Inset: Zoom-in of the graph for low pairwise correlation coefŽcients (0 · c · 0.1). (B) The excitation was generated according to the MIP model, everything else as in A. (C) Same input setting as in A, but for a Žxed input population size (N D 500) and different individual input rates (r D 10, 20, 50, 90 spikes per second). (D) The excitation was generated according to the MIP model. Everything else as in C. Note that for c D 0 and c D 1, the SIP and the MIP models coincide.

Figure 3C shows the output rate ro for a Žxed population size of N D 500 and for different input rates r D 10, 20, 50, 90 spikes per second, with the SIP model being used as excitatory input ensemble. The output rate ro increased with increasing c, and more so for larger r. The form of the dependence of ro on c was more saturating for larger r. The output rate of the FR neuron with identical input parameters, but using the MIP model as excitatory input, is

Higher-Order Statistics of Input Ensembles

81

shown in Figure 3D. We observe a distinct difference with the response to the SIP input, particularly for small input correlations. Here, for small c, an increase of c leads to a large increase of ro due to the large N. A consecutive decrease of ro for larger c occurred for small r only. Indeed, for r D 50, 90 spikes per second, the inter-cluster interval did not become small compared to the membrane time constant. After having explored the effects of each of the three input parameters, we now consider how modiŽcations of the FR neuron and, more precisely, of the membrane time constant t , the thresholdh of the activation function, and the form of the activation function could affect these results. With regard to t , note that the output rate of the FR neuron with the SIP model (see equation 3.6) depends only on the product rt . Thus, an increase (decrease) of the time constant has the same effect as the proportional increase (decrease) of the input rate. This is the case for the density function of any shot noise with exponential kernel (Gilbert & Pollak, 1960). Taking into consideration that the membrane potential of the FR neuron response (for both the SIP and the MIP models) can be decomposed into a sum of several elementary shot noise processes with different amplitudes and rates (see appendix B), it can be concluded that the equivalence of t and r is valid for all input parameters of both input models. An analogous but somewhat weaker statement can be made concerning the threshold h of the activation function. The output rate of the neuron depends only on the quotient h / N (see equation 3.6). This is the case for the approximated response to both the SIP and the MIP models (see appendix B). Thus, as long as the approximation is good, the effect of an increase of h is equivalent to the proportional decrease of N. Finally, we turn to the effects of the form of the activation function. For an arbitrary function, it would be necessary to know the entire density function of the potential to be able to derive the mean output rate of the FR neuron. The piecewise deŽnition of the density function, however, makes it difŽcult to derive an approximation of the output rate. Nevertheless, our method would still be applicable, and an approximation of the neuronal rate could be derived, if the input clusters had a low rate. Indeed, in this case, the entire density function is well approximated by the Žrst few segments. Note that if the activation function was linear, the mean output rate of the FR neuron would not be sensitive to any aspect of the input statistics other than the mean input rate, because the mean output of a linear system is determined by its mean input only. A nonlinear activation function is thus a prerequisite for the sensitivity of the neuron to input correlations and, a fortiori, to higherorder statistics. In turn, any sufŽciently nonlinear activation function will result in different mean responses of the FR neuron to both input models. Indeed, the response potential densities to both the SIP and MIP model are different, and the mean output rate will typically be different. 3.2 Response of the Integrate-and-Fire Neuron. In order to compare the effects of different input ensemble statistics under more realistic condi-

82

A. Kuhn, A. Aertsen, and S. Rotter

tions, we simulated a conductance-based leaky IF neuron (Tuckwell, 1988; Koch, 1999). Its membrane potential was (on average) depolarized with respect to the resting state, and it uctuated. Its input conductance was also increased, similarly to what is typically observed in vivo (ParÂe, Shink, Gaudreau, Destexhe, & Lang, 1998). The essential difference to the membrane potential dynamics of the FR neuron is that synapses here were modeled not as current pulses but as transient membrane conductance changes. The subthreshold membrane potential of the IF neuron was described by 1 d U ( t) D [Ur ¡ U ( t) ] C Ge ( t) [Ue ¡ U (t )] C Gi ( t) [Ui ¡ U (t) ]. dt R Ur is the resting membrane potential. Ge ( t) and Gi ( t) are the excitatory and inhibitory synaptic conductances, and Ue and Ui are the excitatory and inhibitory synaptic reversal potentials, respectively. We use ge (t) and gi (t ) to denote the conductance changes elicited by a single excitatory or inhibitory presynaptic spike. Both excitatory and inhibitory conductance changes were modeled by an a-function (Koch, 1999; Rotter & Diesmann, 1999), X ge ( t) D ge ate1¡at H ( t) , Ge (t) D ge (t ¡ tek ) , C

gi ( t) D gi ate1¡at H ( t) ,

Gi (t) D

k X k

gi ( t ¡ tik ) ,

where ge (gi ) is the peak excitatory (inhibitory) conductance change, and 1 / a is the time constant of the conductance change. Both types of synapses had the same time constant. The times tek and tik represent the occurrences of excitatory and inhibitory input spikes, respectively. The voltage-dependent conductances responsible for the action potential generation were not included in the model. Instead, when the membrane potential reached a threshold Uh , an action potential was generated and the membrane potential was clamped to a reset value Ureset for a period trefr . In addition, all synaptic currents were shunted. This latter mechanism modeled the refractory period. The various model parameter values were chosen to be in line with the experimental literature (McCormick, Connors, Lighthall, & Prince, 1985): C D 500 pF, R D 30 MV, Ur D ¡70 mV, Uh D ¡50 mV, Ureset D ¡60 mV (Troyer & Miller, 1997), and trefr D 2 ms. The excitatory synapses modeled fast glutamatergic (AMPA) synapses, and the inhibitory synapses modeled fast GABAergic (GABAA ) synapses. We set Ue D 0 mV and Ui D ¡70 mV (Koch, 1999). The time constant of the synaptic conductance changes was 1 / a D 1 ms, and the peak conductances were ge D 1 nS and gi D 3.4 nS, respectively. 3.2.1 Membrane Potential Response. A depolarized membrane potential was obtained by bombarding the neuron with Poisson spike trains. We used independent excitatory and inhibitory spike trains whose rates were

Higher-Order Statistics of Input Ensembles

83

adjusted such that the membrane potential uctuated around a mean value of ¡54.3 mV (LÂeger, Stern, Aertsen, & Heck, 2002). We used an excitatory rate of 9000 spikes per second and an inhibitory rate of 5500 spikes per second, which is equivalent to 9000 independent individual excitatory inputs and 5500 independent individual inhibitory inputs, both with a rate of 1 spike per second. We refer to this input as background input. The membrane potential uctuated with a standard deviation of 1.5 mV, and the model neuron Žred with a rate of 1 spike per second under these conditions. The excitatory postsynaptic potential (EPSP) or inhibitory postsynaptic potential (IPSP) produced by an additional excitatory (inhibitory) input spike had, on average, an amplitude of 0.16 mV (¡0.15 mV). This is in accordance with Matsumura, Chen, Sawaguchi, Kubota, & Fetz (1996), who found that EPSPs and IPSPs recorded intracellularly in vivo had similar amplitudes and time courses. We let two additional populations of excitatory and inhibitory inputs impinge on the neuron. Both populations had the same size, and the individual excitatory and inhibitory discharge rates were identical, as with the FR neuron. The excitatory spike trains were correlated, with parameters represented by either the SIP or the MIP model; the inhibitory inputs were always independent. Figure 4A shows a 500 ms realization of the excitatory and inhibitory population activity (bottom), with N D 100 and r D 20 spikes per second. Both the input and the IF neuron were simulated with a time step of 0.1 ms. In the Žgure, the number of excitatory input spikes per time step is plotted as a positive integer and the number of inhibitory input spikes per time step as a negative integer. The SIP model was used for the excitatory population, and the correlation coefŽcient was c D 0.1. Note that the background synaptic input is not shown. The membrane potential response of the IF neuron is plotted in the upper part of the Žgure. It uctuated randomly, characterized by occasional fast depolarizations leading to an output spike. These were mostly caused by spike clusters involving the whole correlated excitatory population. Note that the second spike in Figure 4A was due to the background of independent synaptic input. The same input parameters were used for Figure 4B, but here the MIP model replaced the SIP model. The excitatory input activity consisted of a large number of clusters of smaller size; the output spikes resulted from the summation of several input clusters. This summation mechanism was rendered stochastic by the background uctuations of the membrane potential. Due to the reduced size of the input clusters, the depolarization preceding a response spike was slower than with the SIP model. In Figure 4C, we used the SIP model but increased the pairwise correlation coefŽcient to 0.4. As a consequence, the rate of occurrence of the input clusters increased, accompanied by a corresponding increase of the output rate. Figure 4D shows the response of the neuron to the MIP model, with c D 0.4. The size of the clusters increased compared to c D 0.1 (see Figure 4B), but their rate of occurrence decreased, as already explained in section 3.1.

84

A. Kuhn, A. Aertsen, and S. Rotter

­ 70 100 50 0 0

input

­ 60

0.1 0.2 0.3 time (s)

0.4 0.5

­ 50 ­ 60 ­ 70 100 50 0 0

0.1 0.2 0.3 time (s)

MIP, c = 0.1 ­ 50 ­ 60 ­ 70 100 50 0 0

0.1

D

SIP, c = 0.4

U (mV)

U (mV)

U (mV)

­ 50

C

input

B

SIP, c = 0.1

input

input

U (mV)

A

0.4 0.5

0.2 0.3 0.4 time (s)

0.5

MIP, c = 0.4 ­ 50 ­ 60 ­ 70 100 50 0 0

0.1

0.2 0.3 0.4 time (s)

0.5

Figure 4: Time course of the membrane potential of the leaky integrate-andŽre (IF) neuron (top) and the corresponding input (bottom), with the excitation generated by either the SIP or the MIP model. (A) The excitation was generated according to the SIP model, with a pairwise correlation coefŽcient of 0.1; the inhibition was a population of independent Poisson processes. The input population size (N D 100) and individual Žring rate (r D 20 spikes per second) were the same for the excitation and the inhibition. The number of excitatory (inhibitory) input spikes per bin (size 0.1 ms) is plotted as a positive (negative) number in the lower panel. Note that the additional independent background inputs that kept the membrane potential in a depolarized state (see text) are not shown. The dotted line in the upper panel indicates the Žring threshold. (B) The excitation was generated according to the MIP model. The pairwise correlation and other input parameters were identical to A. (C) The excitation was generated according to the SIP model, with an excitatory input pairwise correlation coefŽcient of 0.4. Other parameters as in A. (D) The excitation was generated according to the MIP model. The pairwise correlation and other input parameters were identical to C.

The output rate was larger than with c D 0.1. However, the clusters were still not large enough to reliably elicit an output spike individually. 3.2.2 Mean Response Rate. Similarly to the FR neuron, we compared the response of the IF neuron for the two different input models SIP and

Higher-Order Statistics of Input Ensembles

85

MIP by studying the dependence of the output Žring rate on the pairwise correlation of the excitatory input. In addition, we systematically varied N and r. Figure 5A shows the output rate ro of the IF neuron to SIP input as a function of c, for a Žxed input rate r D 20 spikes per second, and for different input population sizes N D 20, 40, 100, 1000. The output rate slowly increased with the input correlation coefŽcient. For population sizes beyond 100, the output rate ro did not increase any further. This behavior was caused by the membrane potential resetting of the IF neuron. Evidently, if the input population size is large enough that the depolarizations induced by an input cluster reach the threshold, a further increase of N has no impact on the output rate because the rate of occurrence of the input clusters does not depend on N. In that case, the mean output rate ro of the IF neuron can thus be approximated by (see Murthy & Fetz, 1994) ro D r (1 ¡ trefr ro ) ,

(3.7)

where r is the rate of input clusters. For the SIP model, we have r D rc. This approximation is plotted as a thick gray line in Figure 5A and expresses that the output rate ro is equal to the rate of spike clusters r, subtracting the clusters arriving while the neuron is refractory. Of course, the approximation is not valid if output spikes are (also) caused by the background bombardment of non-synchronized excitatory spikes (as is the case for c D 0, for instance). Figure 5B shows the response of the IF neuron as a function of c but with the MIP model for the excitation and for r D 20 spikes per second and N D 20, 40, 100, 300, 500, 1000. For small values of c, the dependence of the output rate on c was far more strongly modulated than with the SIP model. For N D 1000, for instance, a small c generated a very large output Žring rate, much larger than with the SIP model. Only for very small populations (N D 20) was the output rate smaller than with the SIP model, over the whole range of c (see Figure 5A, inset). For large N, as for the FR neuron, the dependence of ro on c was non-monotonic. The initial increase of ro with c was caused by the integration of increasingly large input clusters. However, when the clusters were large enough that the occurrence of a single input cluster reliably triggered an output spike, a further increase of the input correlation was detrimental to the output Žring rate. Indeed, a further increase of c, resulting in even larger input clusters, causes ro to decrease, because the occurrence rate of clusters decreases with increasing c. For these larger values of c, the mean output rate ro of the IF neuron can again be approximated by equation 3.7. We assume that the stochastic nature of the spike cluster size does not play a role and replace each cluster by one of average amplitude. The rate of the mean amplitude clusters is r/ c. The approximation given by equation 3.7, with r D r/ c, is represented as a thick gray line in Figure 5B. The amplitude of the mean input spike cluster is proportional to N, and the range of validity of the approximation is larger

86

A. Kuhn, A. Aertsen, and S. Rotter

for large values of N. Note that for larger values of ro , the approximation slightly overestimated the output rate. This is due to the Žnite rise time of the conductances used to model the synapses, which allowed synchronized clusters to impinge on the cell during the depolarization caused by a Žrst cluster. Such an event happened more frequently at higher rates. Figure 5C shows the mean output rate of the IF neuron with the SIP model, for N D 500, and r D 10, 20, 50, 90 spikes per second. N was large enough so that each input cluster elicited an output spike. The relation between ro and c depended on r, because the rate of occurrence of the input clusters is proportional to r. The mean output rate was approximated by equation 3.7, with r D rc (thick gray lines). Again, note that the approximation is more precise for small values of ro . We replaced the a-function conductance with a conductance pulse and repeated these simulations. In that case, the (small) discrepancy between the simulated output rate and the approximation, observed for larger values of ro , disappeared completely (data not shown). The response to exactly the same input parameters, but with the MIP model instead of the SIP model, is shown in Figure 5D. As N was large, a non-monotonic dependence of ro on c could be observed. This was the case for all r. The approximation of the mean output rate ro (see equation 3.7, r D r/ c) is plotted as thick gray lines. For the different input rates r, the left bound of the range of the approximated output rate was chosen as the value of c producing the maximal output rate in the numerical simulations. The approximation gave better results for small values of output rates, as explained above. For large r (50, 90 spikes per second), two local maxima appeared in the simulation results for the dependence between ro and c. For the smaller value of c, the interaction structure was thus (locally) optimal with regard to the generation of output spikes, so that both an increase and a decrease of c caused a decrease of ro . To Žrst approximation, we can make the same assumption here as for the response of the FR neuron and explain this effect in terms of the mean cluster. The mean cluster generated by the MIP model, for a value of c producing the locally maximal output rate, was not big enough to depolarize the membrane potential up to the spiking threshold alone and thus needed support from other inputs. An efŽcient arrangement consists of pairs of clusters whose sizes were such that within the integration time of the membrane, they jointly cause a depolarization that can just bring the neuron to Žre a spike. A pair of larger clusters would result in the same effect, but it would occur less frequently, as the mean cluster rate is inversely related to its mean size. Note that the positions of both peaks are shifted to higher values of c for increasing input rate r. This is because not only excitation but also inhibition increases with r, necessitating larger clusters to reach the threshold. Interestingly, this can lead to a nonmonotonic dependence of ro on r. For c around 0.35, for instance, an increase of the input rate from 50 to 90 spikes per second resulted in a decreased output rate.

Higher-Order Statistics of Input Ensembles

140 120 100 80 60 40 20 0 0

B

SIP 2 0 0

SIP®

­1

output rate (s )

4

®MIP N=20 0.5 1 100, 1000 40 20

0.2 0.4 0.6 0.8 1 correlation coefficient

D

SIP

­1

­1

output rate (s )

C

120 100 80 60 40 20 0 0

output rate (s )

­1

output rate (s )

A

90 s­ 1 ­1

50 s

s­­ 11

20 10 s

0.2 0.4 0.6 0.8 1 correlation coefficient

87

MIP 120 100 1000 80 60 40 20 40 20 0 0 0.2 0.4 0.6 0.8 1 correlation coefficient 140 120 100 80 60 40 20 0 0

MIP

90 s­ 1 ­1

50 s

20 s­­ 11 10 s

0.2 0.4 0.6 0.8 1 correlation coefficient

Figure 5: Mean output rate of the leaky integrate-and-Žre (IF) neuron as a function of the pairwise correlation coefŽcient of the excitatory input, for the two different input ensemble statistics. Thick gray lines represent an analytical approximation of the output Žring rate under the assumption of large spike clusters in the input (see text for details). (A) The excitation was generated according to the SIP model; the inhibition was a set of independent Poisson processes. Excitatory and inhibitory population size and rate were identical. The output rate is shown for different input population sizes (N D 20, 40, 100, 1000) and constant individual input rate (r D 20 spikes per second). The output rate for N D 100 and N D 1000 produced undistinguishable curves. The inset compares the response of the IF neuron for the SIP and the MIP input model as excitation, for N D 20 and r D 20 spikes per second. It shows that for small population sizes, the output rate with the SIP model was larger over the whole range of c. (B) The excitation was generated according to the MIP model. The output rate is shown for different input population sizes (N D 20, 40, 100, 300, 500, 1000) and a Žxed input rate (r D 20 spikes per second). (C) Same input setting as in A, for different individual input rates (r D 10, 20, 50, 90 spikes per second) and a Žxed population size (N D 500). (D) The excitation was generated according to the MIP model. All input parameters as in C. Again, note that for c D 0 and c D 1 the SIP and the MIP model coincide.

88

A. Kuhn, A. Aertsen, and S. Rotter

The input scenarios discussed thus far were composed of equal proportions of excitation and inhibition. The ratio of excitation and inhibition received by a cortical neuron in vivo, however, can only be roughly estimated. Anatomical data suggest that excitation amounts to more than four times the inhibition (Braitenberg & Schuz, ¨ 1998). On the other hand, it has been argued on the basis of the irregularity of the discharge of cortical neurons that they operate with equal proportions of excitation and inhibition (van Vreeswijk & Sompolinsky, 1996; Shadlen & Newsome, 1998). We deŽne the balance of the input, bD

Ni ri , Ne re

where Ne (Ni ) is the excitatory (inhibitory) input population size and re (ri ) the Žring rate of individual excitatory (inhibitory) input lines. We varied the input balance of the IF neuron by changing the amount of inhibition (the product Ni ri ). Note that the same variation of b could be obtained by changing either Ne or re , not necessarily leading to exactly the same effects. Figure 6A shows the mean output rate of the IF neuron as a function of the excitatory pairwise correlation, for different levels of inhibition, using the SIP model for the excitatory input. The excitation was composed of Ne D 300 and re D 20 spikes per second. The inhibition was Ni ri D 0, 1200, 3000, 6000, or 9000 spikes per second, corresponding to b D 0, 0.2, 0.5, 1, 1.5. For low levels of inhibition (b D 0, 0.2, 0.5), the output Žring rate for weak pairwise correlations in the input was much larger than for the balanced input. For these values of b, an increase of c caused a decrease of the output rate. The occurrence of excitatory clusters of size Ne induced large depolarizations—larger than actually necessary to reach the spiking threshold. The excess excitatory spikes hence did not contribute to generating output spikes, which had a detrimental effect on the output rate. Observe that for b D 0.5, ro decreased down to a value of 10 spikes per second at c D 0.45 and increased up to about 20 spikes per second at c D 1. Thus, the initial decrease of ro , caused by spikes participating in the (too) large synchronization clusters, could be partially compensated for by the increased cluster rate. Using more inhibition than excitation (b D 1.5), no output spikes were produced for independent inputs. Due to the hyperpolarized membrane potential, the uctuations of the potential were not large enough to induce spontaneous spiking. Nevertheless, an increase of the input correlation and the associated occurrence of synchronized clusters brought the neuron to Žre, with an output rate again tending to the maximal rate of 20 spikes per second for c D 1. This is shown by the overlap with the thick gray line, representing the approximation expressed by equation 3.7 (with r D rc). The response of the IF neuron to the same input parameters, but with the MIP model instead of the SIP model for the excitatory input, is shown in Figure 6B. We again observe considerable differences with the response

Higher-Order Statistics of Input Ensembles

80

SIP b=0

60 40 20 0 0

B ­1

100

output rate (s )

­1

output rate (s )

A

¬b=1,1.5

0.2 0.4 0.6 0.8 1 correlation coefficient

89

100 80

MIP b=0

60 40 20 0 0

b=1.5

0.2 0.4 0.6 0.8 1 correlation coefficient

Figure 6: Mean output rate of the leaky integrate-and-Žre (IF) neuron as a function of the pairwise correlation coefŽcient of the excitatory input, for the two different input ensemble statistics, with different ratios of inhibition to excitation. Thick gray lines represent an analytical approximation of the output Žring rate, assuming that output spikes are caused only by large spike clusters in the input. (A) The excitation was generated according to the SIP model. The excitatory population size was Ne D 300, and the excitatory individual Žring rate was re D 20 spikes per second. The product of the inhibitory population size and rate was Ni ri D 0, 1200, 3000, 6000, 9000, corresponding to an input balance (see text) of b D 0, 0.2, 0.5, 1, 1.5. The response of the IF neuron for b D 1.5 and b D 1 differed only for very low excitatory input correlations. (B) The excitation was generated according to the MIP model. All input parameters were identical to A.

to the SIP input, particularly for small input correlations. At small values of c, a small increment induced a large increase of ro for b D 1 and 1.5. For b D 0.2, a small increase of c had little inuence on ro , and for b D 0, ro decreased with c. Small values of c generated small synchronized clusters that could drive the output rate up or down, depending on the net excitation level. This is because for high inhibition levels, the membrane potential uctuates strongly, and an increase of excitatory input synchrony results in an even higher variance of the membrane potential, leading to more frequent threshold crossings. By contrast, if the excitation level is high, the synaptic input, on average, tends to depolarize the neuron above threshold (see Shadlen & Newsome, 1998). In that case, the summation of rapidly occurring excitatory spikes is efŽcient, and synchrony does not greatly accelerate the depolarization of the membrane potential to threshold. For b D 0.2, ro was initially insensitive to changes in c, because the loss of spikes in the synchronized clusters was exactly compensated by the greater efŽcacy of synchronous input. The degree of input balance at which this effect occurs depends on the input parameters N and r. For larger c, as N was large, the effect of synchrony was detrimental for all b values. As the output spikes were then exclusively caused by clusters of synchronous

90

A. Kuhn, A. Aertsen, and S. Rotter

input spikes, the mean response rate was well described by the approximation expressed by equation 3.7 (with r D r/ c), plotted as a thick gray line in Figure 6B. The interaction structure of the input can thus have a dramatic inuence on the output Žring rate. A striking example is given by the response of the IF neuron for b D 0.5. With the interaction structure of the SIP type, an increase of c from 0 provoked a decrease of the output Žring rate, but with the MIP model, for the same input balance level, the same increase of correlation induced an increase of the output rate. 4 Discussion

Individual Žring rates and pairwise correlations of a population of neurons do not determine the ensemble uniquely. We showed that neurons can be very sensitive to the details of the interaction structure of their input ensembles. Thus, it is difŽcult to draw any deŽnitive conclusions about the effect of particular input parameters (like pairwise correlation) on a neuron’s output rate, as long as physiological input ensembles are not characterized in a more complete manner, in particular, by specifying their higher-order correlation structure. 4.1 Input Ensemble Statistics. In model studies, it is often assumed that the total input or the membrane potential can be represented by a process with a gaussian probability density function (Tuckwell, 1988; Feng & Brown, 2000; Svirskis & Rinzel, 2000). Doing so has the advantage that the statistical properties of the model can be constrained easily. Nevertheless, as the third and higher cumulants are forced to be zero, this model is obviously not suited to study the impact of the full ensemble statistics. As we have shown in this work, it is crucial to take all moments of the input ensemble into account. Several studies have tackled the impact of correlated spike trains on the output of neuron models. Shadlen and Newsome (1998) and Salinas and Sejnowski (2000) generated correlated spike trains by simulating neurons with partial common input. Two sets of inputs were randomly drawn from a Žnite pool, resulting in a certain fraction of the inputs being used for both neurons. This introduces a correlation in the output spike trains of the two neurons. Such spike trains have broader cross-correlograms, which is more realistic than the exact synchrony used in this study. However, the higher-order interactions in such ensembles of spike trains were not considered. Another drawback is that the ensembles were not deŽned analytically. Destexhe and ParÂe (1999) used correlated synaptic input to reproduce the neuronal properties observed in vivo. They used Poisson processes as inputs and introduced stochastic dependence by drawing a smaller number of input processes than there were synapses (similarly to Salinas & Sejnowski, 2000), a single Poisson process being used as input to several synapses.

Higher-Order Statistics of Input Ensembles

91

Again, the relation between the pairwise correlation and the higher-order correlation structure was not stated explicitly. Statistically, the interaction structure of an input ensemble has a large number of degrees of freedom. We picked two extreme situations. In the SIP model, all of the doublets of spikes on any pair of spike trains occur simultaneously, resulting in spike clusters synchronized over the whole population. In the MIP model, the cluster size is a stochastic variable with a mean increasing linearly with the pairwise correlation. To realize a pairwise correlation coefŽcient identical to the SIP model, the average cluster rate must be higher than in the SIP model. Some studies used an input ensemble related to the MIP model (Murthy & Fetz, 1994; Bernander, Koch, & Usher, 1994) in order to investigate the effects of input synchrony. Their input ensemble differed from ours in that the rate of occurrence of spike clusters was held constant. As a consequence, they parameterized their ensemble in terms of the absolute synchrony (the size of spike clusters), and not in terms of the correlation coefŽcient. Interestingly, Bernander et al. (1994) also introduced a temporal jitter of the synchronization and showed that the precision of the spike correlation can strongly modulate the output Žring rate, depending on the values of the input parameters. The issue of the ensemble statistics was speciŽcally tackled by Bohte et al. (2000), who studied an input distribution with maximal entropy. Assuming homogeneous rates and homogeneous pairwise correlations, they derived a maximally at distribution of input clusters. The size of an input cluster was deŽned as the number of neurons spiking within a single time bin. They showed that under the assumption of maximal entropy, spike clusters involving a large number of neurons already exist for a small pairwise correlation. The SIP and MIP models were chosen because they represent two extreme cases of input ensembles. The question arises of how similar they are to the inputs that a real cortical neuron receives. To date, this question remains largely unanswered. Let us mention two lines of research that could help solve this problem. The Žrst is the massively parallel recording of multiple single cell activities in vivo (Nicolelis, 1998). Statistical methods suited to the analysis of the corresponding data are beginning to be available (Gerstein & Aertsen, 1985; Martignon et al., 1995, 2000; Gutig ¨ et al., 2002). Nevertheless, the pool of neurons forming the actual presynaptic input ensemble to a single cortical neuron cannot be inferred from such studies. Therefore, a second source of answers could come from the detailed investigation of membrane potential uctuations gained from intracellular recordings in vivo (Borg-Graham, Monier, & Fre gnac, 1998; Azouz & Gray, 1999; LÂeger, Stern, Aertsen, & Heck, 2002). Analysis of membrane potential uctuations or their time courses, possibly combined with simultaneous recordings from neighboring cells, could help to shed light on the interaction structure of the input ensembles seen by cortical neurons.

92

A. Kuhn, A. Aertsen, and S. Rotter

4.2 Dependence of Response on Input Ensemble Statistics. We used the SIP and MIP models of spike train ensembles to describe the excitatory input into neurons. The inhibitory inputs were assumed independent throughout this study. With the SIP model and for a Žxed pairwise input correlation, an increase of the input population size induced only a limited increase of the output rate for two very different neuron models (see Figures 3A and 5A). This was true at all input rates and over the entire range of input correlations. Interestingly, this was the case not only for the IF neuron but also for the FR neuron, whose membrane potential did not undergo any reset following a threshold crossing. Probing the model neurons with an interaction structure generated by the MIP model, we observed that for any Žxed pairwise correlation in the input, the range of output rates obtained by varying the size of the input ensemble was much larger than with the SIP model (see Figures 3B and 5B). For small input correlations and in contrast to what happened with an interaction structure of the SIP type, the output rate increased steadily with the size of the input ensemble and could thus attain much larger values. The most striking difference between the response to both interaction structures was observed for large input populations. By large, we mean that an input spike cluster involving the whole population elicits a membrane depolarization that reaches the Žring threshold. With the SIP model, the rate of spike clusters was proportional to both the individual input rate and the pairwise correlation. For a large input population, the output rate of the IF neuron was thus approximately linearly related to these two input parameters (see Figure 5C). By contrast, with input statistics of the MIP type, the dependence of the IF response on the input Žring rate of a large input ensemble could be non-monotonic (see Figure 5D). Note that for the FR neuron, the variance of the membrane potential was the same with the SIP or the MIP input model. Indeed, the autocovariance function (and thereby the variance) of a linear system’s output is entirely determined by the autocovariance function of its input (Papoulis, 1991). As the autocovariance of the summed input ensemble is given by the sum of the autocovariances and cross-covariances of the individual spike trains (both of which were identical for the SIP and the MIP input models), the autocovariance of the membrane potential is identical for both input ensembles. Thus, the size of the uctuations, as measured by the variance of the membrane potential, does not alone explain the output rate response, as shown by the very different response of the FR neuron to both input ensembles (see Figure 3). This also implies that the assumption of a gaussian distribution of the membrane potential, which requires knowing only the Žrst and second moments of the potential, may fail to predict the mean response rate. Moreover, it can be easily shown that the variance of the membrane potential increases (monotonically) with the input pairwise correlation c (see appendix B for the complete expression of the variance of the membrane potential). Thus, despite an increasing vari-

Higher-Order Statistics of Input Ensembles

93

ance of the membrane potential, the response rate can decrease (see Figures 3B and 3D), depending on the higher-order interactions of the input ensemble. Many of the neuronal response properties investigated here were similar for both the FR and the IF neuron. The differences could originate from either the postspike reset mechanism built into the IF neuron or from the use of conductances to model the synapses. Tiesinga, JosÂe, and Sejnowski (2000) studied the differences in the neuronal response to an uncorrelated input ensemble when using currents or conductances to model synapses. In the case of the MIP model, we have previously shown that the mean output of a continuous FR neuron with reversal potentials can be in very good correspondence to the output rate observed in an otherwise identical neuron that included a membrane potential reset mechanism (Kuhn, Rotter, & Aertsen, 2002). Hence, depending on the higher-order statistics of the input ensemble, input parameters can have very different effects on the neuron’s output rate. We studied the effects of pairwise correlation in the excitatory population across its entire range from 0 to 1. However, in experimental studies, the correlation coefŽcient is rarely found to be larger than 0.1 to 0.2 (Aertsen & Gerstein, 1985; Das & Gilbert, 1999; Eggermont & Smith, 1995; Nelson, Salin, Munk, Arzi, & Bullier, 1992). Considering this range of input correlations, it appears that the two different realizations of input correlations considered here impose different constraints on the way the neuronal response can be modulated. If the activity of a presynaptic network is characterized by the synchronization of large pools of neurons, the neuronal output is approximately linearly modulated by the input rate. By contrast, if the population activity of the presynaptic network is composed of small, distributed clusters, the size of the input population constitutes an ideal parameter to modulate the output Žring rate of the postsynaptic neuron over a large range. The latter case would thereby suggest an operation mode based on the dynamic recruitment of input lines. 4.3 Balance of Excitation and Inhibition. We changed the mean ratio of inhibition to excitation of the input (the input balance) and observed that the effects of pairwise correlation in the excitatory population on the output rate could be reversed. With the SIP model as excitatory input structure, the monotonic increase of the output rate as a function of the correlation, observed for equal proportions of excitation and inhibition, could change into a monotonically decreasing relation (see Figure 6A). Such modulation of the relative effect of synchrony by the net excitatory level was also observed with the MIP model as interaction structure for low values of the input correlation coefŽcient (see Figure 6B). This detrimental effect of input synchrony on the postsynaptic Žring, at high levels of (net) excitation, has already been reported (Murthy & Fetz, 1994), albeit with a different input ensemble (see above).

94

A. Kuhn, A. Aertsen, and S. Rotter

Stroeve and Gielen (2001) used an input population identical to the SIP model and reported that the pairwise correlation of excitation affects the postsynaptic Žring rate only if the inhibitory Žring rate is sufŽciently high. It must be pointed out that this statement follows from their particular choice of input parameters. They used a small input size (N D 120), which was effectively smaller compared to the input used here because the membrane potential of their model neuron was not in a depolarized state. As shown in this study, the effects of input correlation depend strongly on the size of the population. More important, their input parameters were such that a compensation effect, similar to what was described for Figure 6B (b D 0.2, c < 0.2), was in effect. This is clearly demonstrated by the irregularity of inter-spike intervals (Stroeve & Gielen, 2001) that increased with the input correlation. This shows that the neuron’s Žring was increasingly caused by large, randomly occurring spike clusters. As such input clusters induced membrane depolarizations beyond the Žring threshold, the corresponding “loss of spikes” had to be compensated for by the increased efŽcacy of synchrony. Hence, as demonstrated in Figure 6B for low input correlation, it is appropriate to consider the net excitation level as a powerful means of modulating the effects of input correlation. This possibility was mentioned earlier by Salinas and Sejnowski (2000). Finally, we would like to stress that both input and output investigated here were stationary and that an important and obvious step toward more realistic conditions would be to study the impact of nonstationary input ensemble statistics. Appendix A

In this appendix, we derive the cross-correlation function and count correlation coefŽcient for pairs of spike trains generated by either the SIP or the MIP model. Consider a homogeneous Poisson point process x ( t) with rate E[x(t)] D l. The autocorrelation Rxx ( D t) of this process is (Papoulis, 1991) Rxx ( D t) D E[x(t)x(t C D t) ] D ld ( D t) C l2 . For two independent Poisson point processes x (t) and y ( t) with rates l and m , respectively, the cross-correlation function Rxy ( D t) is Rxy ( D t) D E[x(t) y ( t C D t) ] D lm . We now consider dependent processes. Let v ( t) and w (t) be deŽned as v ( t) D x ( t) C z ( t) w ( t) D y ( t) C z (t) , where x ( t) , y ( t) , and z ( t) are independent homogeneous Poisson processes with rate l, m , and º, respectively. The cross-correlation function of v ( t) and

Higher-Order Statistics of Input Ensembles

95

w (t) is Rvw (D t) D Rxy ( D t) C Rxz ( D t) C Ryz (D t) C Rzz ( D t) 2 D ºd ( D t) C º C lº C mº C lm

D ºd ( D t) C (º C l) (º C m ) .

The count correlation coefŽcient for the two point processes v (t) and w (t) with respect to a Žxed observation interval of duration T is deŽned in terms of the stochastic variables V and W giving the respective numbers of points encountered during the observation interval: cVW D p

cov(V, W ) . var ( V ) var (W )

For arbitrary point processes, the count correlation coefŽcient depends in general on the observation interval. For the two dependent Poisson processes v ( t) and w (t) considered above, we have in particular var(V) D var (X ) C var (Z ) var(W) D var (Y ) C var(Z) cov(V, W ) D var (Z ) , with X, Y, and Z the stochastic variables giving the respective numbers of points in the processes x (t) , y ( t) , and z ( t) , in the interval considered. For an observation interval of duration T, we have var(X) D lT,

var(Y) D m T,

var (Z ) D ºT,

and thus obtain cVW D p

º . (º C l) (º C m )

Note that this is independent of the observation length T. Thus, we may also simply use the term “pairwise correlation coefŽcient” for “count correlation coefŽcient.” Normalizing the cross-correlation function of the two processes by the geometric mean of their respective rates, we obtain an expression of the cross-correlation as a function of the count correlation coefŽcient, p

p Rvw ( D t) D cVWd ( D t) C (º C l) (º C m ) . (º C l) (º C m )

The count correlation coefŽcient thus corresponds to the net integral over the central peak in the (normalized) cross-correlation function. The general case represented by v ( t) and w ( t) is now readily applied to our two models of spike train ensembles. For the SIP model, we have º D a

96

A. Kuhn, A. Aertsen, and S. Rotter

and l D m D b. Hence, the cross-correlation function and count correlation coefŽcient between any two spike trains are R (D t) D ad ( D t) C (a C b ) 2 , a . cD aC b For the MIP model, it holds that the rate of coincident spikes in any pair of spike trains is º D c 2 2 and l D m D c 2 ¡c 2 2 . The cross-correlation function and count correlation coefŽcient thus are R (D t) D c 2 2d ( D t) C (c 2 ) 2 , cD2 . Appendix B

In this appendix, we develop an analytical approximation of the response of the FR neuron to the two models of spike train ensembles (SIP and MIP). The FR neuron was described above (see equations 3.1 through 3.3). The corresponding synaptic input current is described by a stationary stochastic process, and we are interested in the stationary output rate of the neuron. The knowledge of the stationary membrane potential distribution is sufŽcient to derive the mean output rate for this particular neuron model (see equation 3.5). If we use linear synapses and Poisson-driven synaptic events, the membrane potential is a shot noise. For pulse-like synaptic currents, the membrane realizes a Žrst-order low-pass Žlter with time constant t D RC (see equation 3.4). Gilbert and Pollak (1960) derived an integral equation for the shot noise distribution and solved it for several particular choices of the Žlter. For the Žlter h ( t) deŽned in equation 3.4, with A D 1 and t D 1, the shot noise S (t ) with underlying rate l has an amplitude distribution with density P (S ) given by P (S ) D dSl¡1

for 0 · S < 1.

For larger values of S, P (S ) is found by recursion from the integral equation, " # Z P (S ) D Sl¡1 d ¡ l The constant d is given by dD

e ¡lg , C (l)

S

1

P (x ¡ 1) x¡l dx .

Higher-Order Statistics of Input Ensembles

97

R1 R1 where g D ¡ 0 e¡t log t dt is Euler ’s constant and C ( z ) D 0 tz¡1 e ¡t dt is the gamma function. Thus, the analytic expression of P ( S ) changes at integer values of S. For t arbitrary, the density is easily shown to be ( P? ( S ) D

DSlt ¡1 DSlt ¡1 w (lt , S )

where

"³

w (lt , S) D 1 ¡

1 S ¡1

for 0 · S < 1

(B.1)

for 1 · S < 2

´¡lt

# F (lt , lt, 1 C ltI 1 ¡ S )

and F is the hypergeometric function F (a, b, cI z ) D

1 X ( a ) k ( b) k z k ( c) k k! kD 0

with ( a) k :D a ( a C 1) ¢ ¢ ¢ ( a C k ¡ 1). The constant D is DD

e ¡lt g . C (lt )

If lt D 1, w (lt , S ) D 1 ¡ log S, and P? (S ) simpliŽes to ( for 0 · S < 1 e¡g P? (S ) D ¡g for 1 · S < 2, e (1 ¡ log S ) which is the expression presented in Gilbert and Pollak (1960). We now formulate the approximation of the membrane potential density, P ( U ) , for correlated excitatory and independent inhibitory input. The neuron’s impulse response h ( t) is given explicitly in equation 3.4. We Žx the PSP amplitude at A D 1. Let us Žrst consider the membrane response to the SIP model. Let Ne (Ni ) be the number of excitatory (inhibitory) input spike trains and re (ri ) the rate of each individual excitatory (inhibitory) spike train. The excitation can be decomposed into two independent Poisson processes (see section 2.1). Let UeN ( t) be the Žltered Ne th-order interaction process and Ue1 (t) the Žltered Žrst-order process. Ui ( t) is the Žltered inhibitory input spike train with total rate Ni ri . Thus, if tekN , tek1 , and tik denote the times of events in the corresponding pulse trains, we have UeN (t) D Ue1 (t) D Ui (t) D

X k

X k

X k

Ne h ( t ¡ tekN ) h ( t ¡ tek1 ) ¡h(t ¡ tik ) .

98

A. Kuhn, A. Aertsen, and S. Rotter

Note that we chose the same absolute amplitude for the excitatory and inhibitory synaptic currents. As we are dealing with a linear system, we have U ( t) D UeN ( t) C Ue1 (t ) C Ui ( t) , and thus P ( U ) D ( PeN ¤ Pe1 ¤ Pi ) ( U ) , the convolution product of the densities of the three independent components. We now useR Campbell’s theorem to obtain the mean and variance of each R density. As h ( t) dt D t and h ( t) 2 dt D t / 2, we have E (UeN ) D Ne at D Ne re ct, E (Ue1 ) D Ne bt D Ne re (1¡ c)t , E (Ui ) D ¡Ni ri t,

var (UeN ) D Ne2 at / 2 D Ne2 re ct / 2, var (Ue1 ) D Ne bt / 2 D Ne re (1¡ c) t / 2, var (Ui ) D Ni ri t / 2.

For a large range of parameters, in particular for large excitatory populations, we have var (UeN ) À var ( Ue1 ) C var (Ui ) , and we can approximate the convolution (Pe1 ¤ Pi ) ( U ) by a Dirac delta function located at E(Ue1 C Ui ) . We obtain P (U ) ¼ PeN ( U ¡ Ne re (1 ¡ c) t C Ni ri t ) . The density of the membrane potential response of the FR neuron to the MIP model is approximated in a similar manner. Let Ue ( t) be the Žltered excitatory input. The interacting point processes in the MIP model can be decomposed into Ne C 1 independent interaction processes (see section 2.2). Let Uen ( t) be the Žltered nth-order interaction process, with rate B(nI c, Ne ) re / c. The membrane potential of the FR neuron is then given by U ( t ) D Ue ( t ) C Ui ( t ) D

N X nD 0

Uen ( t) C Ui (t ).

The variance of Ue (t ) is var (Ue ) D Ne re (1 ¡ c C cNe )t / 2. If the relation var(Ue ) À var (Ui ) holds true, we can approximate P i (U ) by a Dirac delta function located at E(Ui ) and write P (U ) ¼ Pe (U C Ni ri t ) .

P In addition, we approximate Ue ( t) by UenN ( t) D Ne c k h (t ¡ teknN ) , where teknN are the times of pulses in the generating process (wg ( t) ; see section 2.2). Effectively, we let the order of interaction be deterministic and approximate the N interaction processes by a single process with amplitude E(n) D Ne c and with the same rate re / c as the generating process. We then get P (U ) ¼ PenN ( U C Ni ri t ). The membrane potential density for both input models can thus be approximated by the expression of the shot noise density (see equation B.1), using a linear variable transformation U D kS ¡ l. We Žnally obtain ³ ´ 1 UCl P ( U ) ¼ P? , k k

Higher-Order Statistics of Input Ensembles

99

with an appropriate choice of k, l, and of the parameters of the density function P? (see equation B.1). The mean output rate ro of the FR neuron is then obtained by integrating over the tail of the potential distribution (see equation 3.5) and is expressed in a piecewise manner too. The Žrst two terms are 8 > > >1 ¡ D > > > > > > > > > > > 1¡D > > > < ro ¼

> > > > > > > > > > > > > > > > > :

´ h C l lt for ¡l · h < k ¡ l k ³ ´ ´ ³ ´¡lt Zh ³ 1 h C l lt U C l lt ¡1 k ¡ > k k U C l¡k : lt k¡l ³ ´ UCl ¢F lt, lt, 1 C lt I 1 ¡ k 9 > dU = ¢ for k ¡ l · h < 2k ¡ l. k > ; 1 lt 8 > <

³

(B.2)

It remains to Žx values for l, k, and l, in the expressions above, with the parameters from the input models. For the SIP model, l D re c, k D Ne , and l D ¡Ne re (1 ¡ c) t C Ni ri t . For the MIP model, l D re / c, k D Ne c, and l D Ni ri t . Note that if N D Ne D Ni and r D re D ri , the expression (h C l) / k simpliŽes to h / N C rct for the SIP model and to h / (Nc) C rt / c for the MIP model. Hence, the threshold h and the size of the input population N appear only as the ratio h / N. Acknowledgments. This work was supported in part by the Human Frontier Science Program, the Deutsche Forschungsgemeinschaft, and the German-Israeli Foundation for ScientiŽc Research and Development. References Aertsen, A., & Gerstein, G. L. (1985). Evaluation of neuronal connectivity: Sensitivity of cross-correlation. Brain Research, 340, 341–354. Azouz, R., & Gray, C. M. (1999). Cellular mechanisms contributing to response variability of cortical neurons in vivo. J. Neurosci., 19(6), 2209–2223. Baker, S. N., Spinks, R., Jackson, A., & Lemon, R. N. (2001). Synchronization in monkey motor cortex during a precision grip task. I. Task-dependent modulation in single-unit synchrony. J. Neurophysiol., 85(2), 869–885. ¨ Koch, C., & Usher, M. (1994). The effect of synchronized inputs Bernander, O., at the single neuron level. Neural. Comput., 6, 622–641. Bohte, S. M., Spekreijse, H., & Roelfsema, P. R. (2000). The effects of pair-wise and higher-order correlations on the Žring rate of a postsynaptic neuron. Neural. Comput., 12, 153–179.

100

A. Kuhn, A. Aertsen, and S. Rotter

Borg-Graham, L. J., Monier, C., & FrÂegnac, Y. (1998). Visual input evokes transient and strong shunting inhibition in visual cortical neurons. Nature, 393, 369–373. Braitenberg, V., & Schuz, ¨ A. (1998). Cortex: Statistics and geometry of neuronal connectivity (2nd ed.). Berlin: Springer-Verlag. Cox, D. R., & Isham, V. (1980). Point processes. London: Chapman & Hall. Das, A., & Gilbert, C. D. (1999).Topography of contextual modulations mediated by short-range interactions in primary visual cortex. Nature, 399(6737), 655– 661. DeCharms, R. C., & Merzenich, M. M. (1996). Primary cortical representation of sounds by the coordination of action-potential timing. Nature, 381, 610–613. Deppisch, J., Bauer, H.-U., Schillen, T., Konig, ¨ P., Pawelzik, K., & Geisel, T. (1993). Alternating oscillatory and stochastic states in a network of spiking neurons. Network, 4, 243–257. Destexhe, A., & ParÂe, D. (1999). Impact of network activity on the integrative properties of neocortical pyramidal neurons in vivo. J. Neurophysiol., 81(4), 1531–1547. Eggermont, J. J., & Smith, G. M. (1995). Rate covariance dominates spontaneous cortical unit-pair correlograms. Neuroreport, 6(16), 2125–2128. Feng, J., & Brown, D. (2000). Impact of correlated inputs on the output of the integrate-and-Žre model. Neural Comput., 12(3), 671–692. Feng, J., & Zhang, P. (2001). Behavior of integrate-and-Žre and Hodgkin-Huxley models with correlated inputs. Phys. Rev. E, 63, 051902:1–11. Gerstein, G. L., & Aertsen, A. M. H. J. (1985). Representation of cooperative Žring activity among simultaneously recorded neurons. J. Neurophysiol., 54(6), 1513–1528. Gilbert, E. N., & Pollak, H. O. (1960). Amplitude distribution of shot noise. Bell Syst. Tech. J., 39, 333–350. Gutig, ¨ R., Aertsen, A., & Rotter, S. (2002). Analysis of higher-order neuronal interactions based on conditional inference. Manuscript submitted for publication. Koch, C. (1999). Biophysics of computation: Information processing in single neurons. New York: Oxford University Press. Kuhn, A., Rotter, S., & Aertsen, A. (2002). Correlated input spike trains and their effects on the leaky integrate-and-Žre neuron. Neurocomputing, 44–46, 121–126. LÂeger, J.-F., Stern, E. A., Aertsen, A., & Heck, D. (2002). Synaptic integration in the cortex shaped by network activity. Manuscript submitted for publication. Martignon, L., Deco, G., Laskey, K., Diamond, M., Freiwald, W., & Vaadia, E. (2000). Neural coding: Higher-order temporal patterns in the neurostatistics of cell assemblies. Neural. Comput., 12, 2621–2653. Martignon, L., von Hasseln, H., Grun, ¨ S., Aertsen, A., & Palm, G. (1995). Detecting higher-order interactions among the spiking events in a group of neurons. Biol. Cybern., 73, 69–81. Matsumura, M., Chen, D., Sawaguchi, T., Kubota, K., & Fetz, E. E. (1996). Synaptic interactions between primate precentral cortex neurons revealed by spiketriggered averaging of intracellular membrane potentials in vivo. J. Neurosci., 16(23), 7757–7767.

Higher-Order Statistics of Input Ensembles

101

McCormick, D. A., Connors, B. W., Lighthall, J. W., & Prince, D. A. (1985). Comparative electrophysiology of pyramidal and sparsely spiny neurons of the neocortex. J. Neurophysiol., 54, 782–806. Murthy, V. N., & Fetz, E. E. (1994). Effects of input synchrony on the Žring rate of a three-conductance cortical neuron model. Neural Comput., 6, 1111–1126. Nelson, J. I., Salin, P. A., Munk, M. H. J., Arzi, M., & Bullier, J. (1992). Spatial and temporal coherence in cortico-cortical connections: A cross-correlation study in areas 17 and 18 in the cat. Visual Neurosci., 9, 21–37. Nicolelis, M. A. L. (Ed.). (1998).Methodsfor neural ensemble recordings. Boca Raton, FL: CRC Press. Papoulis, A. (1991). Probability, random variables, and stochastic processes (3rd ed.). New York: McGraw-Hill. ParÂe, D., Shink, E., Gaudreau, H., Destexhe, A., & Lang, E. (1998). Impact of spontaneous synaptic activity on the resting properties of cat neocortical pyramidal neurons in vivo. J. Neurophysiol., 79, 1450–1460. Rotter, S., & Diesmann, M. (1999). Exact digital simulation of time-invariant linear systems with applications to neuronal modeling. Biol. Cybern., 81, 381– 402. Salinas, E., & Sejnowski, T. J. (2000). Impact of correlated synaptic input on output Žring rate and variability in simple neuronal models. J. Neurosci., 20(16), 6193–6209. Salinas, E., & Sejnowski, T. J. (2001). Correlated neuronal activity and the ow of neural information. Nat. Rev. Neurosci., 2(8), 539–550. Shadlen, M. N., & Newsome, W. T. (1998). The variable discharge of cortical neurons: Implications for connectivity, computation, and information coding. J. Neurosci., 18(10), 3870–3896. Snyder, D. L., & Miller, M. I. (1991). Random point processes in time and space (2nd ed.). New York: Springer-Verlag. Steinmetz, P., Roy, A., Fitzgerald, P., Hsiao, S., Johnson, K., & Niebur, E. (2000). Attention modulates synchronized neuronal Žring in primate somatosensory cortex. Nature, 404(6774), 187–190. Stroeve, S., & Gielen, S. (2001). Correlation between uncoupled conductancebased integrate-and-Žre neurons due to common and synchronous presynaptic Žring. Neural Comput., 13(9), 2005–2029. Svirskis, G., & Rinzel, J. (2000). Inuence of temporal correlations of synaptic input on the rate and variability of Žring in neurons. Biophys. J., 79, 629–637. Tiesinga, P. H. E., JosÂe, J. V., & Sejnowski, T. J. (2000). Comparison of currentdriven and conductance-driven neocortical model neurons with HodgkinHuxley voltage-gated channels. Phys. Rev. E, 62(6), 8413–8419. Troyer, T. W., & Miller, K. D. (1997).Physiological gain leads to high ISI variability in a simple model of a cortical regular spiking cell. Neural Comput., 9, 971–983. Tuckwell, H. C. (1988). Introduction to theoreticalneurobiology (Vol. 2). Cambridge: Cambridge University Press. van Vreeswijk, C., & Sompolinsky, H. (1996). Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science, 274, 1724–1726. Received May 10, 2002; accepted June 4, 2002.

LETTER

Communicated by Ad Aertsen

Duality of Rate Coding and Temporal Coding in Multilayered Feedforward Networks Naoki Masuda [email protected] Department of Mathematical Engineering and Information Physics, Graduate School of Engineering, University of Tokyo, Tokyo, Japan Kazuyuki Aihara [email protected] Department of Complexity Science and Engineering, Graduate School of Frontier Sciences, University of Tokyo, Tokyo, Japan A functional role for precise spike timing has been proposed as an alternative hypothesis to rate coding. We show in this article that both the synchronous Žring code and the population rate code can be used dually in a common framework of a single neural network model. Furthermore, these two coding mechanisms are bridged continuously by several modulatable model parameters, including shared connectivity, feedback strength, membrane leak rate, and neuron heterogeneity. The rates of change of these parameters are closely related to the response time and the timescale of learning. 1 Introduction

Experimental data on information coding by Žring rates have been widely reported, but it is still controversial whether the brain really uses such coding for information processing. Calculating Žring rates on the basis of spike trains, if it requires time averaging, may consume more time than is biologically acceptable to account for brain functions (Thorpe, Fize, & Marlot, 1996). Accordingly, functional roles of precisely timed spiking and synchronous Žring have been explored. Our research is motivated by experimental results showing that neurons with overlapping receptive Želds Žre synchronously when activated by common preferred stimuli in the cat primary visual cortex (Eckhorn et al., 1988; Gray, Konig, ¨ Engel, & Singer, 1989). Synchronous Žring is also experimentally observed in the cat superior colliculus (Brecht, Singer, & Engel, 1999), the monkey primary motor cortex (Hatsopoulos, Ojakangas, Paninski, & Donoghue, 1998), and the monkey somatosensory cortex (Steinmetz et al., 2000). Furthermore, the Žring of a single neuron is also precise to the millisecond order in, for example, rat neocortical slices (Mainen & Sejnowski, 1995) and b-lobe neurons (MacLeod, B¨acker, & LauNeural Computation 15, 103–125 (2003)

° c 2002 Massachusetts Institute of Technology

104

N. Masuda and K. Aihara

rent, 1998). Synchronous Žring has been examined by various mathematical models with pulse couplings to clarify physiological mechanisms (Domany, van Hemmen, & Schulten, 1994; Bottani, 1995; Bressloff & Coombes, 2000; Guardiola, DÂõ az-Guilera, Llas, & PÂerez, 2000; Feng, Brown, & Li, 2000; Masuda & Aihara, 2001). It has been shown that network behavior is sensitive to model details and that stable asynchronous and clustered states may also appear. Temporal coding models, alternatives to the rate coding, have been presented on the basis of precisely timed spiking, coincidence, and synchronization. Among them are the multilayered synŽre chain model (Abeles, 1991; Domany et al., 1994; Diesmann, Gewaltig, & Aertsen, 1999), the detectors of periodic Žring patterns (Fujii, Ito, Aihara, Ichinose, & Tsukada, 1996), a learning algorithm model for coincidence detection by updating synaptic strengths (Gerstner, Kempter, van Hemmen, & Wagner, 1996), and the chaotic attractor reconstructor model with intersynchronization intervals (Tokuda & Aihara, 2000; Masuda & Aihara, 2002b; Aihara & Tokuda, 2002). These spatiotemporal computation models can process information at higher speeds and require less energy than is required by the rate coding (Abeles, 1991). Nevertheless, the functional roles of precisely timed neural activities are still unclear. On the other hand, short-term Žring rates can be estimated reliably by a population average of 100 or fewer neurons without resorting to temporal averaging or measurement of precisely timed Žring (Shadlen & Newsome, 1998; Mar, Chow, Gerstner, Adams, & Collins, 1999; van Rossum, Turrigiano, & Nelson, 2002). The population rate coding with such spatial averaging produces a biologically plausible coding scheme with Žring rates. Balanced excitation and inhibition models (Shadlen & Newsome, 1998) can also reproduce the experimentally observed high variability of Žring time, which was once thought to be generated more easily with temporal coding schemes with coincidence detectors (Abeles, 1991; Softky & Koch, 1993). Although these two coding mechanisms are usually considered alternatives, they may work in a complementary rather than exclusive way (Abeles, 1982, 1991; Tsodyks & Markram, 1997). It is shown that both the temporal coding and the population rate coding can work in a single neural network and that they are bridged by changing the intensity of additive gaussian noise (Masuda & Aihara, 2002b; van Rossum et al., 2002). In the feedforward network model depicted in Figure 1, each sensory neuron integrates the external stimulus S ( t) to encode its different aspect. Each cortical neuron gathers small contributions from many sensory neurons to encode signal information (Shadlen & Newsome, 1998; Masuda & Aihara, 2002a). When sufŽciently weak noise is applied to the cortical neurons, they tend to synchronize and encode a signal crudely but robustly in intersynchronization intervals (synchronous coding), but moderately large noise desynchronizes them (Tokuda & Aihara, 2000; Masuda & Aihara, 2002b; Aihara & Tokuda, 2002). In the latter case, each cortical neuron represents a different feature of

Rate Coding and Temporal Coding

105

IF x1

LIF v1

x2

v2

synchronous coder population rate coder

S(t) xn1 sensory layer

vn2 cortical layer

Figure 1: Architecture of the feedforward neural network model. The sensory layer and the cortical layer consist of n1 integrate-and-Žre (IF) neurons and n2 leaky integrate-and-Žre (LIF) neurons with leak rate c , respectively. Each sensory neuron integrates the external stimulus S ( t ) and Žres when the membrane potential reaches the threshold. A cortical neuron receives spatiotemporal incident spikes with amplitude 2 from n01 sensory neurons where n01 · n1 . After Žring, it sends a feedback spike with amplitude 2 and synaptic delay t to all the other cortical neurons. The spatiotemporal spike trains from all the cortical neurons are used to evaluate Žring synchrony and the instantaneous Žring rate.

S ( t) , and Žne approximation to the temporal waveform of S ( t) is obtained by the population rate coding deŽned by the instantaneous ensemble average of spike counts (Shadlen, Britten, Newsome, & Movshon, 1996; Shadlen & Newsome, 1998; Mar et al., 1999; van Rossum et al., 2002). Our results suggest that the two coding schemes can work dually, depending on physiological conditions and anatomical structure. Noise intensity is not the only physical cause of switch between the population rate coding and the synchronous coding. For example, a theoretical study suggests that transmembrane leak can also change rapidly with background activities (Bernander, Douglas, Martin, & Koch, 1991). Figure 2 suggests that the rate code and synchronous code modes can coexist when heterogeneity in Žring thresholds is modulated. We continuously bridge these two coding schemes in this article to illuminate their complementary roles. We identify dominant coding schemes determined by the correlation of inputs to cortical neurons, the feedback coupling strength, the membrane leak rate, and inhomogeneity in constituent neurons. We also consider the

106

N. Masuda and K. Aihara B 1

1

0.75

0.75 s _ 2 (i)

s _ 2 (i)

A

0.5 0.25 0

0.5 0.25

0

200

400

600

0

800

0

200

time (ms) C

600

800

D 30

30

20

20

n eu ron

n eu ron

400 time (ms)

10 0

0

200

400 time (ms)

600

800

10 0

0

200

400

600

800

time (ms)

Figure 2: The histograms (A, B) and rastergrams (C, D) in the regime of the synchronous coding (A, C) when D H D 0 and the population rate coding (B, D) when D H D 0.25, with the deŽnition of D H given in section 3.4. The other parameters are equal to those used for Figure 6C.

combination of these effects in multilayered neural networks where both feedforward and feedback connections exist. We describe the model neural network and the methods that we use to evaluate the functional coding schemes in section 2. In section 3, we examine how the functional coding schemes switch when various model parameters are modulated. Section 4 is devoted to the uniŽed effects of simultaneous parameter changes. In section 5, we discuss the relations between our results and dual coding schemes, multilayered feedforward networks, synŽre chains, and the timescale of learning. 2 Model 2.1 Neural Network. The neural network model used in this work is depicted in Figure 1. The network consists of two layers: the sensory layer with n1 neurons and the cortical layer with n2 neurons. Each sensory neuron is an integrate-and-Žre (IF) neuron (Tuckwell, 1988; Koch, 1999) that receives a common continuous external input S ( t) . The membrane potential of the ith sensory neuron is obtained by the perfect integration of suprathreshold S (t) . The ith sensory neuron Žres when the membrane potential sum reaches 1, af-

Rate Coding and Temporal Coding

107

ter which the potential is instantaneously reset to a resting potential of 0 and the neuron restarts integration. The initial phases of the neurons are chosen randomly from a uniform distribution on [0, 1]. This is a key assumption required for the network to operate as an efŽcient population rate coder. The correlation between the signal and the population rate would be reduced if the phases were not random (Masuda & Aihara, 2002a; van Rossum et al., 2002). Dynamical noise could disperse the phases in real situations to make them uniformly distributed even in the absence of randomness of initial phases (Masuda & Aihara, 2002b). We adopt this neuron model because of its simplicity and the theoretical evidence that interspike intervals of the IF neuron can encode the deterministic information on S (t) (Sauer, 1994). The results below can be extended to more realistic sensory neurons with a leak (Masuda & Aihara, 2002a). When a sensory neuron Žres, it sends a spike to some cortical neurons. Every cortical neuron receives instantaneous incident spike trains from n01 · n1 sensory neurons. A pair of cortical neurons generally shares in average n012 / n1 incident spatiotemporal spike trains from the sensory neurons connecting to both neurons. Therefore, we deŽne the shared connectivity r ´ ( n012 / n1 ) / n01 D n01 / n1 . Generally, the network dynamics of coupled leaky integrate-and-Žre (LIF) neurons are more sensitive to network conŽguration (Knight, 1972; Bressloff & Coombes, 2000) in contrast to the dynamics of coupled IF neurons. We treat the cortical neurons as LIF neurons without an external bias (Tuckwell, 1988; Masuda & Aihara, 2002a, 2002b) since dynamical transitions of the network states are our primary concern. The ith cortical neuron (1 · i · n2 ) has a membrane potential vi ( t) , a threshold for Žring (vi (t) D H i , E[H i ] D H 0 D 1, where E means the expectation), and a resting potential (vi ( t) D 0). After Žring, this cortical neuron emits an instantaneous feedback spike with amplitude 2 and synaptic delay t to the other cortical neurons. The dynamics of the ith cortical neuron are described by n2 XX X X dvi 0 D 2 d ( t ¡ Ti0 , j ) C 2 d ( t ¡ Ti0 , j ¡ t ) ¡ c vi ( t ) , dt 6 i,i0 D 1 j i0 D i0 2S j i

(1 · i · n2 ) ,

(2.1)

where c > 0 is the membrane leak rate and d is the delta function. Ti0 , j is the time when the jth spike generated by the i0 th sensory neuron arrives at the ith cortical neuron. Here, the delay between the sensory and cortical layers is uniformly zero or equivalently uniformly constant. Ti00 , j is the jth Žring time of the i0 th cortical neuron, and 2 is the amplitude of a feedforward spike. The ith cortical neuron receives incident spikes only from the i0 th sensory neuron with i0 2 Si where |Si | (D n01 ) is the number of randomly selected sensory neurons. Although we do not consider noise sources such as ongoing activity (Arieli, Sterkin, Grinvald, & Aertsen, 1996; Azouz & Gray, 1999) composed of spikes from neurons outside the network, they

108

N. Masuda and K. Aihara

would also modulate the network behavior and preferred coding schemes in a similar way to dynamical gaussian white noise (Masuda & Aihara, 2002b; van Rossum et al., 2002). The delay t D 2.6 ms (Abeles, 1991; Diesmann et al., 1999), and the Žring rate is kept at 20 Hz (de Oliveira, Thiele, & Hoffmann, 1997; Zohary, Shadlen, & Newsome, 1994). The membrane leak rate should be around 0.05 to 0.1 ms¡1 (Abeles, 1982; Koch, 1999), but we use larger values to emphasize the duality of the two complementary coding schemes. We analyze the network dynamics in the high-input regime by setting n01 D 240 and n2 D 30, where 100 to 200 incident spikes are necessary for a cortical neuron to Žre (Gerstner et al., 1996; Shadlen & Newsome, 1998; Diesmann et al., 1999). The spike trains emitted by the cortical neurons are observed by the virtual synchronous coder and the virtual population rate coder, as shown in Figure 1. We measure the degree of synchrony by calculating syn, which is deŽned as the ratio of the number of synchronous Žring events to the average number of single neuron spikes. A synchronous Žring event is deŽned as spike arrivals from more than n2 / 2 cortical neurons within tw D 1.5 ms (Softky & Koch, 1993; Hatsopoulos et al., 1998; Aihara & Tokuda, 2002). Full synchrony gives syn D 1, whereas fully asynchronous Žring results in syn D 0. The population rate of synchronous Žring usually represents the input signal more robustly but less accurately than that of asynchronous Žring. The accuracy of the synchronous coding is equal to that of coding by a single neuron (Masuda & Aihara, 2002b; van Rossum et al., 2002), although the noise torelance is better (Aihara & Tokuda, 2002). The performance of population rate coding is evaluated by the correlation coefŽcient corr between the input signal S ( t) and the short-time population Žring rate of the cortical neurons, that is, the number of spikes from all the cortical neurons in each bin of width w D 3.8 ms (Shadlen et al., 1996). Given Z 1 iw ( ) s1 i D S (t ) dt w ( i¡1) w and s2 ( i) D the number of spikes observed for t 2 [(i ¡ 1)w, iw], the correlation coefŽcient between the two time series fs1 ( i)g and fs2 ( i)g deŽnes corr. 2.2 External Inputs. The chaotic input S ( t) was generated from the Lorenz system, the dynamics of which is represented by the following equations: 8 dx > < dt D a (10y ¡ 10x) , dy (2.2) D a (28x ¡ y ¡ xz ) , dt > : dz 8 D a (¡ 3 z C xy ). dt

Rate Coding and Temporal Coding

109

The continuous input signal is deŽned as S ( t) D 2.25£10¡2 C 7.00£10¡4 x and models nonrandom dynamical rules with complexity contained in complex external stimuli such as visual scenes and sounds. The change rate a is Žxed at 0.03 so that the characteristic timescale of S ( t) is smaller than the interspike interval. As a result, the synchronous coding cannot capture rapid signal change, and the Žring rate can encode only roughly the temporal waveform of S ( t). This fact is revealed by low values of corr for the synchronous Žring. Population Žring rates with asynchronous Žring are more strongly correlated with S ( t). Our results qualitatively remain the same when we apply stochastic inputs, which are often used to study information processing by neural networks (see section 3.4). 3 Selection of Effective Coding Modes

We investigate which coding mechanism is effective when values of model parameters are changed. The parameters examined here have different functional relevance and affect the choice of the optimal coding in different ways. 3.1 Shared Connectivity. Two neurons often share incident connections, especially when the neurons are geometrically or functionally close to each other (Aertsen, Gerstein, Habib, & Palm, 1989; Zohary et al., 1994; Shadlen et al., 1996). They usually share only a part of inputs, and the feedforward and also feedback coupling structure is far from all-to-all. Incomplete coupling introduces quenched noise that disperses the membrane potentials of two neurons starting from the same initial level. The shared connectivity r D n01 / n1 , that is, the ratio of the average number of the shared connections to that of the total connections (n01 ), is biologically between 0.1 and 0.3 (Zohary et al., 1994; Shadlen et al., 1996; Shadlen & Newsome, 1998; Hatsopoulos et al., 1998; Diesmann et al., 1999). Shared connections, or common incident spikes, are likely to drive the neural population to synchrony, as do common feedback spikes delivered to many neurons (Bottani, 1995). Synchronous Žring can occur theoretically even without feedback couplings (Knight, 1972; Feng, Brown, & Li, 2000), though biologically it may be produced by cooperation of feedback interactions and correlated inputs (Eckhorn et al., 1988; Hatsopoulos et al., 1998; Brecht et al., 1999). We examine how r inuences network behavior and the efŽciency of the two codings. As shown in Figure 3, we change the number of shared connections between the sensory layer and the cortical layer by changing n1 and keeping all the other parameters constant. Since the simulation results depend on initial conditions, syn and corr shown in this Žgure, as well as in the following Žgures, are the averages of 50 trials. Figure 3 demonstrates that the population rate coding performs better for smaller r, whereas the neurons are more synchronous for larger r. This result holds regardless of the feedback strength within the cortical layer (see Figure 4).

110

N. Masuda and K. Aihara

Figure 3: Change in the degree of synchrony measured by syn and performance of the population rate coder measured by corr when the shared connectivity r varies. We set n1 D n01 / r, n2 D 30, n01 D 240, 2 D 0.007, c D 40 ms ¡1 , and 2 D 0.002.

A

B 1 0.4 0.3

0.6 0.4

r=0.50 r=0.25 r=0.03

0.2 0

0

0.0025 0.005 0.0075 0.01 coupling strength

c o rr

syn

0.8

0.2 r=0.50 r=0.25 r=0.03

0.1 0

0

0.0025 0.005 0.0075

0.01

coupling strength

Figure 4: Change in syn (A) and corr (B) when the feedback strength 2 varies. n2 D 30, n01 D 240, 2 D 0.007,c D 40 ms ¡1 , and 2 D 0.002. The lines in each panel correspond to the three examined cases, with n1 D 480, n1 D 960, and n1 D 8000, corresponding to r D 0.50, r D 0.25, and r D 0.03, respectively.

Rate Coding and Temporal Coding

111

Correlated inputs with large r produce relatively invariant responses of the cortical neurons and interfere with efŽcient signal averaging, which results in biases and a loss of reliability in signal estimation. The information transmission capacity from the sensory layer to the cortical layer is decreased in this situation (Stein, 1967; Shadlen & Newsome, 1998). However, robust temporal coding with synchronous Žring is possible based on coincidence detection. When r is small, cortical neurons are likely to desynchronize to participate in reliable rate coding. Effective pooling of independent information from asynchronous cortical neurons leads to better estimation of S (t ) (Shadlen et al., 1996; Shadlen & Newsome, 1998). Changing r continuously inuences the efŽciency of the synchronous coding and the population rate coding in opposite directions. 3.2 Feedback Strength. We examine the inuence of feedback coupling 2 among cortical neurons. Feedback interactions generally compel networks

toward characteristic states such as full synchrony, clustered, and phaselocked states (Domany et al., 1994; Bressloff & Coombes, 2000). Figure 4 shows that the cortical layer is a reliable rate coder when 2 is small over a wide range of shared connectivity, whereas the cortical neurons synchronize when 2 is large. 3.3 Membrane Leak Rate. The membrane leak rate c of the cortical neurons is another modulatable parameter. Simulation studies suggest that it can vary rapidly in response to the background activity (Bernander et al., 1991). Networks of LIF neurons with large c synchronize more readily than LIF neurons with small c or nonleaky (c D 0) neurons (Knight, 1972; Bressloff & Coombes, 2000; Feng et al., 2000; Masuda & Aihara, 2001). Therefore, we expect that the preferred coding mode would change gradually from the population rate coding to the synchronous coding as c increases. Changes in Žring rates cause signiŽcant biases in the performance measures syn and corr when a in equation 2.2 is Žxed. Consequently, to make the Žring rates Žxed for the sake of fair comparisons, we adjust 2 for each c in the numerical analyses. The simulation result shown in Figure 5, however, supports our conjecture only for c 2 [0.00, 0.02]. EfŽcient population rate coding as well as a certain level of synchrony is achieved for larger c . The simultaneous large values of syn and corr for large c are contrasted with the opposed effects on these quantities of varying r or 2 . The relevant parameter range includes the biologically plausible leak rate of c D 0.05 ¡ 0.1 ms ¡1 (Abeles, 1982; Diesmann et al., 1999; Koch, 1999). Although large c could cause signal estimation to deteriorate because of the weighted integration of the signal performed by the LIF neuron (Knight, 1972; Masuda & Aihara, 2002a), its contribution to corr is actually marginal. However, c also modulates the input preference. An LIF neuron with large c operates as a coincidence detector that Žres only when multiple spikes arrive in a short

112

N. Masuda and K. Aihara

Figure 5: Change in syn and corr when the membrane leak c varies. n1 D 480, n01 D 240, n2 D 30, and 2 D 0.003. We adjust 2 to keep the Žring rates constant.

time window (Abeles, 1991; Softky & Koch, 1993; Fujii et al., 1996; Koch, 1999). Such a cortical neuron detects the peaks of S ( t) at which the sensory neurons send relatively many coincident spikes to the cortical neurons (Masuda & Aihara, 2002a). As a result, the cortical neurons Žre synchronously for increased c , and the rate of synchronous Žring events encodes the amplitude of S (t ). Since the temporal waveform of S (t ) is effectively characterized by the location and the amplitude of its peaks, corr becomes large. Introduction of a refractory period will prevent synchronous bursting with high frequency and therefore eliminate the emergence of simultaneously large syn and corr. 3.4 Heterogeneity. Unlike the homogeneous model neurons considered above, real neurons are naturally heterogeneous. Heterogeneity drives neural populations out of synchrony (White, Chow, Ritt, Soto-Trevino, ˜ & Kopell, 1998; Tiesinga & JosÂe, 2000). We consider heterogeneity in feedforward coupling strength, membrane leak rate, and Žring threshold. For each neuron or synapse, one of these parameters is assigned a value randomly according to the uniform distributions [2 0 (1 ¡ D2 ) , 2 0 (1 C D 2 ) ], [c 0 (1 ¡ Dc ) , c 0 (1 C D c ) ], or [H 0 (1 ¡ D H ) , H 0 (1 C D H ) ], where 2 0 D 0.007, c 0 D 0.025 ms¡1 , and H 0 D 1. The degrees of heterogeneity are parameterized by D2 , D c , and D H . Figure 6 shows that in all three cases, the cortical neurons prefer the synchronous coding when the network is close to homogeneous, while the population rate coding is more efŽcient with increased heterogeneity. Figure 6D, in comparison with Figure 6C, shows that the conclusion remains the same when a stochastic instead of chaotic input is applied. The stochas-

Rate Coding and Temporal Coding

113

A

B 0.8

0.8

syn corr

0.6

0.6

0.4

0.4

0.2

0.2

0

0

0.5

1

1.5

2

syn corr

2.5

0

3

normalized heterogeneity in feedforward coupling C

0

0.05

0.1

0.15

0.2

normalized heterogeneity in membrane leak D

0.8

0.8

syn corr

0.6

0.6

0.4

0.4

0.2

0.2

0

0

0.05

0.1

0.15

syn corr

0.2

0.25

normalized heterogeneity in threshold

0

0

0.05

0.1

0.15

0.2

0.25

normalized heterogeneity in threshold

Figure 6: Change in syn and corr when the cortical neurons are heterogeneous. We consider heterogeneity in (A) the feedforward coupling strength 2 , (B) the membrane leak rate c , and (C, D) the Žring threshold H of the cortical neurons. The Lorenz and Žltered gaussian noise inputs are applied in A, B, C, and in D, respectively. We set n1 D 480, n01 D 240, n2 D 30, and 2 D 0.003. The horizontal axes show the width of the uniform distribution normalized by the average.

tic input signal is generated from the Ornstein-Uhlenbeck process, or the Žltered gaussian noise (Mainen & Sejnowski, 1995; van Rossum et al., 2002), deŽned by dx x C j ( t) , D ¡ tou dt S (t) D 2.25 £ 10¡2 C 4.00 £ 10¡4 x,

(3.1)

114

N. Masuda and K. Aihara

where tou D 8.0 ms and j ( t) is gaussian white noise applied at every time step dt D 0.02 ms. Heterogeneous resetting (Mar et al., 1999) and transmission delays of feedforward couplings (Gewaltig, Diesmann, & Aertsen, 2001) would introduce additional jitter. The biases in estimating S ( t) that heterogeneity could cause are negligible. Actually, corr grows to fairly large values (corr ¸ 0.6) for strong heterogeneity, compared to that when the network is homogeneous or when r or 2 is small. Although the Žring rates of the cortical neurons differ with heterogeneity, S ( t) can be estimated reliably from the collection of heterogeneous spike trains. The network states easily escape from constrained states such as full synchrony or clustered states (Mar et al., 1999; Guardiola et al., 2000). Homogeneous neurons tend to synchronize no matter how small r, 2 , and c are (Knight, 1972). Even when r D 0, two spike trains arriving at two cortical neurons still resemble each other because they encode the same stimulus, just with different phases. Synchronous Žring of coupled oscillators is considered important (Eckhorn et al., 1988; Gray, Konig, ¨ Engel, & Singer, 1989; Hatsopoulos et al., 1998; Brecht et al., 1999; Steinmetz et al., 2000), and mathematical studies have explored how much heterogeneity in, for example, the Žring thresholds and coupling strengths (Bottani, 1995) a stable synchronization can endure. However, it has been shown that even less than 10% heterogeneity in the Žring rates easily desynchronizes neurons (White et al., 1998; Tiesinga & JosÂe, 2000). Heterogeneous neurons do not necessarily synchronize all the time. Asynchronous Žring can provide accurate estimates of the instantaneous Žring rate (Knight, 1972; van Rossum et al., 2002).

4 Parameter UniŽcation

We have seen the effects of changing several model parameters on coding modes. Cortical neurons are likely to prefer the synchronous coding when the shared connectivity is large, strong feedbacks are present, the membrane leak is large, or the cortical neurons are fairly homogeneous. The performance as a population rate coder is highest in the opposite extremes except when the membrane leak is changed. The leak rate determines how inputs are Žltered, as well as the excitation dynamics. Selection of the coding scheme depends on the parameter values in a continuous manner. We deŽne an order parameter s that is essential in determining the preferred coding mode. The value of s measures the accumulated dispersion of the membrane potentials of two neurons after the duration of a typical interspike interval. We assume that the arrival of presynaptic spikes is a Poisson process with parameter l and that the input is approximated by a diffusion process (Lansky, 1984; Tuckwell, 1988). The membrane potential vi of the ith neuron

Rate Coding and Temporal Coding

115

follows the stochastic differential equation, dvi ( t) D (m ¡ c i vi ( t) ) dt C

p

DdBi (t ) ,

(4.1)

m D lE2 [2

i, j ],

(4.2)

D D lE2 [2

2 i, j ],

(4.3)

lD

220 N D , 40 ms T

where c i and 2 i, j are the leak rate of the ith cortical neuron and the amplitude of an incident spike from the sensory layer to the cortical neuron, respectively. The Brownian motion is denoted by Bi ( t) , E is the expectation, T is the typical interspike interval, and N is the number of incident spikes necessary for a cortical neuron to Žre. We assume that 2 i, ji ,c i , and also H i , the Žring threshold of the ith neuron, are taken from uniform distributions on [2 0 (1¡D2 ) , 2 0 (1 C D 2 ) ], [c 0 (1¡Dc ) , c 0 (1 C Dc )], and [H 0 (1¡D H ) , H 0 (1 C D H )], respectively. Consequently, equations 4.2 and 4.3 reduce to m D l2 0 , Á D D l 1C

D2

2

3

(4.4) ! .

(4.5)

Supposing vi (0) D 0, it follows from equations 4.1 and 4.4 that Z

vi (t) D

t 0

0

e ¡c i ( t¡t ) (l2 0 dt C

p

DdBi ( t) ) dt 0 .

(4.6)

We Žrst consider the uctuation of the interspike interval T caused mainly by heterogeneous H i and c i . We calculate T and its standard deviation s1 without considering input jitter. Substituting vi (t) D H i into equation 4.6 and ignoring the diffusive term, the interspike interval T is estimated by ³ ´ 1 c i Hi . (4.7) T D ¡ log 1 ¡ ci l2 0 Expansion of the right-hand side of equation 4.7 to the Žrst order in D H and D c yields T D A C (¡A C B )

ci ¡c 0 Hi ¡ H 0 CB , c0 H0

(4.8)

where ³ ´ 1 c 0H0 A D ¡ log 1 ¡ D E H,c [T], c0 l2 0

(4.9)

116

N. Masuda and K. Aihara

and BD¡

l2

0

H0 . ¡ c 0H 0

(4.10)

On the basis of equation 4.8 and the assumption that Hi and c i are uniformly distributed, we have s12 D s12 (c , D H , Dc ) ´ E H,c [(T¡A) 2 ] D

D c2 3

( A C B) 2 C

D 2H 3

B2 .

(4.11)

We next deŽne s2 , which estimates how the membrane potentials of two neurons starting from the same resting potential disperse when they come close to the threshold. To evaluate the difference between vi1 ( T ) and vi2 (T ) , it should be noted that Bi1 ( t) and Bi2 (t) are not independent of each other because of the shared connectivity. For any L2 ([0, T]) –bounded function f ( t) , it follows from the Itoˆ isometry that 2Á Z 4 Ev

!2 3

T

5D

f (t) dB ( t) 0

Z

T

f ( t) 2 dt,

(4.12)

0

where Ev [¢] denotes the average over the probability measure of the Brownian motion. We obtain "Z # Z Z T

Ev

0

T

f ( t) dBi1 ( t)

0

T

f ( t) dBi2 (t )

Dr

f ( t) 2 dt.

(4.13)

0

since the proportion r of the total incident spike inputs is shared by a pair of neurons (Shadlen & Newsome, 1998). We then normalize vi ( t) so that vi1 ( T ) and vi2 (T ) can be compared with the same effective threshold. By substituting vi (t ) D H i vQ i (t) / H 0 into equation 4.1, we obtain ³ dvQ i ( t) D

p ´ m H0 DH 0 ( ) ¡ c i vQ i t dt C dBi ( t) , Hi Hi

(4.14)

and the threshold for Žring is normalized to vQ D H 0 . Using E H,c ,v[vQ i1 ( t) ¡ vQ i2 ( t) ] D 0 and equations 4.5, 4.6, 4.12, and 4.13, the second-order approximation leads to s22 D s22 (r, c , D2 , D c , D H ) ´ E H,c ,v[ (vQ i1 (T ) ¡ vQ i2 (T ) ) 2 ] "ÁZ T e ¡c i1 ( T¡t ) H 0 (l2 D E H,c ,v H i1 0

0 dt C

p

DdBi1 ( t) )

Rate Coding and Temporal Coding

Z ¡

0

Z

0

0

e ¡c i1 ( T¡t) H 0 l2 Hi1

Z

T

T

¡r

Z 0

Á

1C

2Á

D2 2

¡l

!

3

ÁZ

T 0

Á ¡ r 1C

Z

T 0

5

e ¡2c i1 ( T¡t ) H 20D dt H2i1

e ¡c i2 (T¡t) H 0l2 H i2 3

T

dt 0

1 ¡ e ¡c i1 T 4 D 2l2 20 E c : 1 ¡ D 2H c i1 C

DdBi2 ( t) )

C

dt

!2 3

p

0

dt

e (c i1 Cc i2 ) ( T¡t ) H20 D dt5 H i1 Hi2

0

l

0 dt C

!2

e ¡c i1 ( T¡t ) H 0l2 H i1

T

¡

8 <

e ¡c i2 ( T¡t) H 0 (l2 Hi2

T 0

2Á Z 4 D 2E H,c ,v

117

1 1¡ D 2H

Z

T 0

!2 3 5 e ¡2c 0 (1¡D c ) t ¡ e ¡2c 0 (1CD c ) t dt 4c 0 Dc t

e ¡c 0 (1¡D c ) t ¡e ¡c 0 (1CD c ) t dt 2c 0 D c t

! D22 Z 3

T 0

³

!2

e ¡c 0 (1¡D c ) t ¡e ¡c 0 (1CD c ) t 2c 0D c t

´2

9 = dt . (4.15) ;

To combine s1 and s2 into the uniŽed parameter s, s1 should be transformed into a dispersion of membrane potentials. Using equation 4.14, the mean inŽnitesimal increase of vQ i at the Žring threshold is given by "³

Ec , H,v

#­ p ´ ­ m H0 DH 0 ­ ¡ c i vQ i ( t) dt C dBi ( t) ­ ­ Hi Hi

D (m ¡ c 0 H 0 ) dt D (l2 0 ¡ c 0 H 0 ) dt.

vQ i D H 0

(4.16)

Accordingly, we deŽne s by s D (l2

0

¡ c 0 H 0 )s1 C s2 .

(4.17)

We evaluate the integrals in equation 4.15 numerically. The relations between s and the coding performances are shown in Figure 7. The order parameter s parameterizes syn and corr well in spite of some deviations whose causes should be investigated in the future. Our method cannot incorporate 2 because its effect cannot be derived only by observing the dynamics of vi .

118

N. Masuda and K. Aihara

B 0.8

r g D Dg DQ

syn

0.6 0.4 0.2 0

0.01

0.1 degree of deviation

c o rr

A

0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

r g D Dg DQ 0.01

0.1 degree of deviation

Figure 7: (A) syn and (B) corr with respect to the order parameter s. The data points identiŽed by the symbols r, c , D 2 , D c , and D H are identical to those used for Figures 3, 5, 6A, 6B, and 6C, respectively.

5 Discussion 5.1 Dual Coding. The synchronous coding is more robust against disorders of Žring in each neuron or noise than the population rate coding (Masuda & Aihara, 2002b). The synchronous coding also consumes less power since downstream neurons Žre with a smaller number of incident spikes when they arrive synchronously than when asynchronously (Abeles, 1982, 1991; Bernander et al., 1991). It has also been shown numerically that a volley of synchronous Žring can stably propagate from layer to layer if the instantaneous synchronous population has at least 90 neurons (Diesmann et al., 1999; Cˆateau & Fukai, 2001; Gewaltig et al., 2001). However, the efŽciency of information transmission by the synchronous coding is limited by the duration of interspike or intersynchronization intervals. Signal information temporally Žner than this interval is discarded by time averaging over the interval. When the cortical neurons Žre asynchronously, the spike train from each cortical neuron encodes a different aspect of the signal. Thus, the Žner temporal waveform of S ( t) is restored by the population rate coding. Although the stimulus frequency can be much higher than the characteristic frequency of a single neuron, the cortical layer as a whole can preserve accurate information on signals (Knight, 1972; Mar et al., 1999). The population rate coding is slightly less robust than the synchronous coding in the sense that displacement in each spike timing directly affects the coding performance. This inuence is small, however, and the population rate coding in networks is sufŽciently stable with graceful degradation. We have shown that

Rate Coding and Temporal Coding

119

the functional role of asynchronous states is as important as that of synchronous states. The importance of analyses of complete asynchrony (Domany et al., 1994; van Vreeswijk & Sompolinsky, 1996; Bressloff & Coombes, 2000) should be recognized. Real neural networks may use the two coding mechanisms dually (Abeles, 1991; Araki & Aihara, 2001; Lu, Liang, & Wang, 2001; Tiesinga, Fellous, JosÂe, & Sejnowski, 2002). Actually, at the single-neuron level, neurotransmitter release probability mediates selection of the rate coding or the temporal coding by means of coincidence detection (Tsodyks & Markram, 1997). Selection between synchronous and asynchronous Žring is observed in experiments in which tasks or stimuli are switched (de Oliveira et al., 1997; Riehle, Grun, ¨ Diesmann, & Aertsen, 1997; Steinmetz et al., 2000). Animals may exploit sharply tuned population averaging under new or competitive circumstances so that they can gain as much external information as fast as possible. Once they adapt and get out of emergent situations, robust coding modes may be preferred even if they are not particularly efŽcient for reproducing the waveform of the external signal. Then the internal parameters may be modulated, or background activity may decrease, so that the synchronous coding replaces the population rate coding. The concept of dual coding also appears in different contexts in which more than one dynamic properties of Žring patterns are used simultaneously to encode different features of external stimuli or cognitive states. Dynamical cell assemblies may be linked to meanings while the Žring rate of a single neuron may represent the weight of its contribution to the meaning (Douglass & Martin, 1991; Kruger ¨ & Becker, 1991). The degree of synchrony may represent external or internal events, while the modulation of Žring rates may reect existence of external stimuli (Riehle et al., 1997; de Oliveira et al., 1997). The dual coding that we have proposed allows one of the two complementary codings selected appropriately for each situation (van Rossum et al., 2002). 5.2 Feedforward Networks with Many Layers. Our results can be generalized to feedforward neural networks with more than two layers. Feedback coupling among sensory neurons was ignored in this work, and the sensory layer worked as a population rate coder without synchronization. We have discussed only how the signal information passed through the population rate coder is processed in the second layer. We can consider another layer that receives spike trains from the cortical neurons of Figure 1. When the cortical layer works as a population rate coder, the arguments in the earlier sections apply to this additional layer, which works as a population rate coder or a synchronous coder, depending on the conŽguration or the value of s. Then how does the additional layer behave when the cortical neurons of Figure 1 Žre synchronously, or, equivalently, what information does the cortical layer encode when the sensory neurons Žre synchronously?

120

N. Masuda and K. Aihara

layer

A

B

C

D

E

F

(a)

R

R

R

R

R

S

(b)

R

R

S

S

S

S

(c)

R

S

S

S

R?

?

desynchronized Figure 8: Architectures of multilayered feedforward networks. The layers labeled by R and S work as population rate coders and synchronous coders, respectively. The mechanisms presented in a and b are realizable, while that in c is not.

If moderately strong noise is applied to each layer, then each layer works as an efŽcient population rate coder, and the information on the stimulus is stably transmitted through many layers (van Rossum et al., 2002). Once a layer Žres synchronously, the network loses the detailed information necessary for reliable signal estimation by population Žring rates (Abeles, 1991). Postsynaptic neurons Žre in unison when they receive inputs in the form of a synchronous volley (Abeles, 1991; MacLeod et al., 1998; Diesmann et al., 1999; Gewaltig et al., 2001; van Rossum et al., 2002), and successive layers would encode the signal by the synchronous coding. Although postsynaptic neurons could desynchronize given strong heterogeneity, for example, they cannot realize efŽcient population rate coding because accurate information on S ( t) is lost once the signal passes through a synchronous layer. The downstream layer receives signal information that is no more complete than that encoded by the synchronous coder in the previous layer. It is easy to switch from the population rate code to the synchronous code, whereas the reverse is difŽcult. Figure 8 shows how multilayered feedforward networks work. The network architectures shown in Figures 8a and 8b are easily realizable, with Figure 8b generalizing the synŽre chain model. A modal change from the popula-

Rate Coding and Temporal Coding

121

tion rate code to the synchronous code occurs at a certain stage. This transition does not have to be immediate; the networks can gradually shift the coding mode through multiple layers. Once the synchronous coding is reached, it is used in all the following layers. This is consistent with the Žnding that sensory neurons are more likely to use the rate coding, while cortical neurons tend to resort to temporal codings for, for example, feature binding (Domany et al., 1994; Fujii et al., 1996). The information coding scheme shown in Figure 8c is unlikely to encode the external signal faithfully because the last two layers represent nothing meaningful unless we use skillfully elaborated signal detectors that seem to be biologically implausible. Biological neural networks have global feedback connections among cortical layers. Although little is understood about the role of this feedback, it may return the global information stored in downstream or higher layers into upstream or lower layers that code local features of stimuli and promote further information processing. Model studies suggest that global feedback may enhance synchronization in upstream layers (Damasio, 1989). Here we rely on the general principle that correlated inputs help neurons synchronize. Global feedback from layer F to layer B in Figure 8a would compel the neurons in layer B to synchronize, and then synchronous Žring would propagate to layers C, D, E, and F. Even if the neurons in layer F are not synchronized, other factors such as high shared connectivity in the global feedback to layer B could trigger synchronous Žring in layer B, with layer F sending synthesized information on stimuli back to the lower layers. Global feedback from one layer to another, both of which do the population rate coding, may induce synchrony in the long run. The following speculation explains how synchrony is also stabilized by global feedback. Let us consider the global feedback from layer F to layer D in Figure 8b and assume that synchrony within each layer is not yet stable, possibly because of a paucity of the neurons in each layer (Diesmann, Gewaltig, & Aertsen, 1999; Cˆateau & Fukai, 2001; Gewaltig et al., 2001). Layer D receives partially synchronous volleys from layers C and F. If the synaptic delays are nearly homogeneous and the synchronous volleys are received with appropriate timing, synchrony is enhanced through layer D (Cˆateau & Fukai, 2001). Nonuniform delays inherent in real neural networks, however, will complicate the problem. 5.3 Learning and Its Timescale. Our work is motivated by experimental results showing the positive role of the desynchronization that occurs when cognitive states may be switched during tasks (Vaadia et al., 1995; de Oliveira et al., 1997; Riehle et al., 1997). The neuronal states switch from synchrony to asynchrony and vice versa within 1 to 10 ms in response to environmental changes. In addition to noise intensity (Masuda & Aihara, 2002b; van Rossum et al., 2002), threshold for Žring (Koch, 1999) is capable of rapid change to accommodate fast switching between two modes, and

122

N. Masuda and K. Aihara

the membrane leak rate is also predicted to change rapidly (Bernander et al., 1991). The shared connectivity and coupling strengths reect the wiring structure. They are mainly modulated through long-term potentiation (LTP) and long-term depression (LTD) that require tens of minutes or more (Koch, 1999). Synaptic learning on long timescales may be relevant to differentiation and long-term memory associated with selection of coding schemes as long-term strategies. Numerical studies have also shown that coupled LIF neurons can specialize in coincidence detection or synchronous coding after learning (Gerstner et al., 1996). Although much less is known about short-term synaptic dynamics, synaptic modulation that occurs in less than a second has been reported in the context of spike-timing-dependent plasticity (Markram, Lubke, ¨ Frotscher, & Sakmann, 1997; Abbott & Nelson, 2000), effective connectivity (Aertsen et al., 1989; Fujii et al., 1996), redistribution of synaptic efŽcacy (Markram & Tsodyks, 1996; Abbott & Nelson, 2000), and synaptic depression (Abbott, Varela, Sen, & Nelson, 1997; Tsodyks & Markram, 1997). Such short-term synaptic modulation may be capable of, for example, feature binding (von der Malsburg & Schneider, 1986), dynamic gain control (Tsodyks & Markram, 1997), and selection of the coding scheme (Tsodyks & Markram, 1997; Araki & Aihara, 2001; Tiesinga et al., 2002). In physiological experiments showing selection of synchrony and asynchrony during tasks, the population rate coding and the synchronous coding may have been properly switched as a result of short-term learning. It is important to conŽrm experimentally possible effects of synaptic modulation on neural coding. Acknowledgments

This work is supported by the Japan Society for the Promotion of Science and CREST, JST. References Abbott, L. F., & Nelson, S. B. (2000). Synaptic plasticity: Taming the beast. Nature Neuroscience (Suppl.), 3, 1178–1183. Abbott, L. F., Varela, J. A., Sen, K., & Nelson, S. B. (1997). Synaptic depression and cortical gain control. Science, 275, 220–224. Abeles, M. (1982).Role of the cortical neuron: Integrator or coincidence detector? Isr. J. Med. Sci., 18, 83–92. Abeles, M. (1991). Corticonics. Cambridge: Cambridge University Press. Aertsen, A. M. H. J., Gerstein, G. L., Habib, M. K., & Palm, G. (1989). Dynamics of neuronal Žring correlation: Modulation of “effective connectivity.” J. Neurophsiol., 61(5), 900–917. Aihara, K., & Tokuda, I. (2002). Possible neural coding with inter-event intervals of synchronous Žring. Physical Review E, 66, 026212.

Rate Coding and Temporal Coding

123

Araki, O., & Aihara, K. (2001). Dual information representation with stable Žring rates and chaotic spatiotemporal spike patterns in a neural network model. Neural Computation, 13, 2799–2822. Arieli, A., Sterkin, A., Grinvald, A., & Aertsen, A. (1996). Dynamics of ongoing activity: Explanation of the large variability in evoked cortical responses. Science, 273, 1868–1871. Azouz, R., & Gray, C. M. (1999). Cellular mechanisms contributing to response variability of cortical neurons in vivo. J. Neurosci., 19(6), 2209– 2223. ¨ Douglas, R. J., Martin, K. A. C., & Koch, C. (1991). Synaptic backBernander, O., ground activity inuences spatiotemporal integration in single pyramidal cells. Proc. Natl. Acad. Sci. USA, 88, 11569–11573. Bottani, S. (1995). Pulse-coupled relaxation oscillators: From biological synchronization to self-organized criticality. Phys. Rev. Lett., 74(21), 4189–4192. Brecht, M., Singer, W., & Engel, A. K. (1999). Patterns of synchronization in the superior colliculus of anesthetized cats. J. Neurosci., 19(9), 3567–3579. Bressloff, P. C., & Coombes, S. (2000). Dynamics of strongly coupled spiking neurons. Neural Computation, 12, 91–129. Caˆ teau, H., & Fukai, T. (2001). Fokker-Planck approach to the pulse packet propagation in synŽre chain. Neural Networks, 14, 675–685. Damasio, A. R. (1989). The brain binds entities and events by multiregional activation from convergence zones. Neural Computation, 1, 123–132. de Oliveira, S. C., Thiele, A., & Hoffmann, K-P. (1997). Synchronization of neuronal activity during stimulus expectation in a direction discrimination task. J. Neurosci., 17(23), 9248–9260. Diesmann, M., Gewaltig, M-O., & Aertsen, A. (1999). Stable propagation of synchronous spiking in cortical neural networks. Nature, 402, 529–533. Domany, E., van Hemmen, J. K., & Schulten, K. (Eds.). (1994). Models of neural networks II. New York: Springer-Verlag. Douglass, R. J., & Martin, K. A. C. (1991). Opening the grey box. Trends in Neurosciences, 14(7), 286–293. Eckhorn, R., Bauer, R., Jordan, W., Brosch, M., Kruse, W., Munk, M., & Reitboeck, H. J. (1988). Coherent oscillations: A mechanism of feature linking in the visual cortex? Biol. Cybern., 60, 121–130. Feng, J., Brown, D., & Li, G. (2000). Synchronization due to common pulsed input in Stein’s model. Phys. Rev. E, 61(3), 2987–2995. Fujii, H., Ito, H., Aihara, K., Ichinose, N., & Tsukada, M. (1996). Dynamical cell assembly hypothesis—theoretical possibility of spatio-temporal coding in the cortex. Neural Networks, 9(8), 1303–1350. Gerstner, W., Kempter, R., van Hemmen, J. L., & Wagner H. (1996). A neuronal learning rule for sub-millisecond temporal coding. Nature, 383, 76–78. Gewaltig, M-O., Diesmann, M., & Aertsen, A. (2001). Propagation of cortical synŽre activity: Survival probability in single trials and stability in the mean. Neural Networks, 14, 657–673. Gray, C. M., Konig, ¨ P., Engel, A. K., & Singer, W. (1989). Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reects global stimulus properties. Nature, 338, 334–337.

124

N. Masuda and K. Aihara

Guardiola, X., DÂõ az-Guilera, A., Llas, M., & PÂerez, C. J. (2000). Synchronization, diversity, and topology of integrate and Žre oscillators. Phys. Rev. E, 62(4), 5565–5570. Hatsopoulos, N. G., Ojakangas, C. L., Paninski, L., & Donoghue, J. P. (1998). Information about movement direction obtained from synchronous activity of motor cortical neurons. Proc. Natl. Acad. Sci. USA, 95, 15706– 15711. Knight, B. W. (1972). Dynamics of encoding in a population of neurons. Journal of General Physiology, 59, 734–766. Koch, C. (1999). Biophysics of computation. New York: Oxford University Press. Kruger, ¨ J., & Becker, D. (1991). Recognizing the visual stimulus from neuronal discharges. Trends in Neurosciences, 14(7), 282–286. Lansky, P. (1984). On approximations of Stein’s neuronal model. J. Theor. Biol., 107, 631–647. Lu, T., Liang, L., & Wang, X. (2001). Temporal and rate representations of timevarying signals in the auditory cortex of awake primates. Nature Neuroscience, 4(11), 1131–1138. MacLeod, K., B¨acker, A., & Laurent, G. (1998). Who reads temporal information contained across synchronized and oscillatory spike trains? Nature, 395, 693– 698. Mainen, Z. F., & Sejnowski, T. J. (1995). Reliability of spike timing in neocortical neurons. Science, 268, 1503–1506. Mar, D. J., Chow, C. C., Gerstner, W., Adams, R. W., & Collins, J. J. (1999). Noise shaping in populations of coupled model neurons. Proc. Natl. Acad. Sci. USA, 96, 10450–10455. Markram, H., Lubke, ¨ J., Frotscher, M., & Sakmann, B. (1997). Regulation of synaptic efŽcacy by coincidence of postsynaptic APs and EPSPs. Science, 275, 213–215. Markram, H., & Tsodyks, M. (1996). Redistribution of synaptic efŽcacy between neocortical pyramidal neurons. Nature, 382, 807–810. Masuda, N., & Aihara, K. (2001). Synchronization of pulse-coupled excitable neurons. Phys. Rev. E, 64, 051906. Masuda, N., & Aihara, K. (2002a). Spatio-temporal spike encoding of a continuous external signal. Neural Computation, 14, 1599–1628. Masuda, N., & Aihara, K. (2002b). Bridging rate coding and temporal spike coding by effect of noise. Phys. Rev. Lett., 88(24), 248101. Riehle, A., Grun, ¨ S., Diesmann, M., & Aertsen, A. (1997). Spike synchronization and rate modulation differently involved in motor cortical function. Science, 278, 1950–1953. Sauer, T. (1994). Reconstruction of dynamical systems from interspike intervals. Phys. Rev. Lett., 72(24), 3811–3814. Shadlen, M. N., Britten, K. H., Newsome, W. T., & Movshon, J. A. (1996). A computational analysis of the relationship between neuronal and behavioral responses to visual motion. J. Neurosci., 16(4), 1486–1510. Shadlen, M. N., & Newsome, W. T. (1998). The variable discharge of cortical neurons: Implications for connectivity, computation, and information coding. J. Neurosci., 18(10), 3870–3896.

Rate Coding and Temporal Coding

125

Softky, W. R., & Koch, C. (1993). The highly irregular Žring of cortical cells is inconsistent with temporal integration of random EPSPs. J. Neurosci., 13(1), 334–350. Stein, R. B. (1967). The information capacity of nerve cells using a frequency code. Biophysical Journal, 7, 797–826. Steinmetz, P. N., Roy, A., Fitzgerald, P. J., Hsiao, S. S., Johnson, K. O., & Niebur, E. (2000). Attention modulates synchronized neuronal Žring in primate somatosensory cortex. Nature, 404, 187–190. Thorpe, S., Fize, D., & Marlot, C. (1996). Speed of processing in the human visual system. Nature, 381, 520–522. Tiesinga, P. H. E., & JosÂe, J. V. (2000). Robust gamma oscillations in networks of inhibitory hippocampal interneurons. Network, 11, 1–23. Tiesinga, P. H. E., Fellous, J-M., JosÂe, J. V., & Sejnowski, T. J. (2002). Information transfer in entrained cortical neurons. Network, 13, 41–66. Tokuda, I., & Aihara, K. (2000). Inter-event interval reconstruction of chaotic dynamics. In Proc. of the Fifth International Symposium on ArtiŽcial Life and Robotics (AROB 5th’00) (pp. 177–180). Oita, Japan: Shubundo Insatsu. Tsodyks, M. V., & Markram, H. (1997). The neural code between neocortical pyramidal neurons depends on neurotransmitter release probability. Proc. Natl. Acad. Sci. USA, 94, 719–723. Tuckwell, H. C. (1988). Introduction to theoretical neurobiology (Vols. 1, 2). Cambridge: Cambridge University Press. Vaadia, E., Haalman, L., Abeles, M., Bergman, H., Prut, Y., Slovin, H., & Aertsen, A. (1995). Dynamics of neuronal interactions in monkey cortex in relation to behavioural events. Nature, 373, 515–518. van Rossum, M. C. W., Turrigiano, G. G., & Nelson, S. B. (2002). Fast propagation of Žring rates through layered networks of noisy neurons. J. Neurosci., 22(5), 1956–1966. van Vreeswijk, C., & Sompolinsky, H. (1996). Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science, 274, 1724–1726. von der Malsburg, Ch., & Schneider, W. (1986).A neural cocktail-party processor. Biol. Cybern., 54, 29–40. White, J. A., Chow, C. C., Ritt, J., Soto-Trevino, ˜ C., & Kopell, N. (1998). Synchronization and oscillatory dynamics in heterogeneous, mutually inhibited neurons. J. Comput. Neurosci., 5, 5–16. Zohary, E., Shadlen, M. N., & Newsome, W. T. (1994). Correlated neuronal discharge rate and its implications for psychophysical performance. Nature, 370, 140–143. Received January 14, 2002; accepted July 2, 2002.

LETTER

Communicated by Laura Martignon

Synchronous Firing and Higher-Order Interactions in Neuron Pool Shun-ichi Amari [email protected] Hiroyuki Nakahara [email protected] Laboratory for Mathematical Neuroscience, RIKEN Brain Science Institute, Wako-shi, Saitama, Japan Si Wu [email protected] Department of Computer Science, ShefŽeld University, ShefŽeld S1, 4DP, U.K. Yutaka Sakai [email protected] Department of Information and Computer Science, Saitama University, Saitama-shi, Saitama, Japan The stochastic mechanism of synchronous Žring in a population of neurons is studied from the point of view of information geometry. Higherorder interactions of neurons, which cannot be reduced to pairwise correlations, are proved to exist in synchronous Žring. In a neuron pool where each neuron Žres stochastically, the probability distribution q ( r ) of the activity r, which is the fraction of Žring neurons in the pool, is studied. When q ( r) has a widespread distribution, in particular, when q (r ) has two peaks, the neurons Žre synchronously at one time and are quiescent at other times. The mechanism of generating such a probability distribution is interesting because the activity r is concentrated on its mean value when each neuron Žres independently, because of the law of large numbers. Even when pairwise interactions, or third-order interactions, exist, the concentration is not resolved. This shows that higher-order interactions are necessary to generate widespread activity distributions. We analyze a simple model in which neurons receive common overlapping inputs and prove that such a model can have a widespread distribution of activity, generating higher-order stochastic interactions. 1 Introduction

Let us consider a pool of n binary neurons. Let xi be binary random variables, xi D 1 when the ith neuron Žres, and xi D 0 otherwise, and i D 1, . . . , n. The Neural Computation 15, 127–142 (2003)

° c 2002 Massachusetts Institute of Technology

128

S. Amari, H. Nakahara, S. Wu, and Y. Sakai

neurons Žre with a Žxed joint probability p ( x) of x D (x1 , . . . , xn ) , which may be correlated. The activity of this neuron pool is deŽned by rD

n 1X xi , n iD 1

(1.1)

which is the fraction of Žring neurons in the pool. This article studies the probability distribution of r called the joint Žring distribution. When n is large, we regard r as a continuous variable and denote its probability density function by q (r ). Then, q (r ) dr D

X P

p (x ) ,

(1.2)

xi D nr

P where the summation is taken over all x such that xi D nr and dr D 1 / n, reecting the fact that r is an integer multiple of 1 / n. When q ( r) is a unimodal distribution having a sharp peak at r0, the activity of the pool is mostly r0 , slightly uctuating around that. However, when q ( r) is widely distributed, in particular, when it has two peaks at r1 and r2 ( r1 < r2 ), the activity is sometimes around r2 and sometimes around r1 . When r2 is large and r1 is small, many neurons sometimes Žre simultaneously, while many neurons are quiescent at other times. This is the phenomenon called synchronous Žring of neurons (Abeles, 1991; Singer & Gray, 1995; Aertsen & Arndt, 1993). This was analyzed in Bohte, Spekreijse, and Roelfsema (2000) by using the maximum entropy principle. The higher-order interactions of neurons were also discussed in Martignon, Von Hasseln, Grun, ¨ Aertsen, & Palm (1995) and Bohte et al. (2000). See also Gerstein, Bedenbaugh, and Aertsen (1989) and Vaadia et al. (1995) for higher-order interactions in real neuron pools, and Amari (2001) and Nakahara and Amari (2002) for theoretical studies. We study the mechanism of synchronous Žring by using information geometry (Amari & Nagaoka, 2000) to show that higher-order interactions of neurons (Amari, 2001; Nakahara & Amari, 2002) are required to generate widespread synchronous Žring, and pairwise correlations cannot generate such a synchronous Žring distribution. We consider the case that n is large and that the distribution p ( x) is symmetric with respect to n variables x1 , . . . , xn , that is, the pool is homogeneous. The distribution q ( r) of activity (see Figure 1) reveals the inner structure of the neuron pool. We Žrst show two extreme cases. 1. Independent case: Each neuron Žres independently with probability a,

p ( x) D

n Y iD 1

fad ( xi D 1) C (1 ¡ a)d ( xi D 0) g,

(1.3)

Synchronous Firing and Higher-Order Interactions in Neuron Pool

129

A

40

q(r)

30 20 10 0 0.0

q(r)

3

0.2

0.4

0.2

0.4

0.2

0.4

B

r

0.6

0.8

1.0

0.6

0.8

1.0

0.6

0.8

1.0

2 1 0 0.0 8

C

r

q(r)

6 4 2 0 0.0

r

Figure 1: Typical joint Žring distributions: (A) concentrated, (B) widely spread, (C) bimodal.

where d is 1 when the event in the parentheses holds and 0 otherwise. In this case, xi is 1 with probability a, so that rD

1X xi n

(1.4)

converges to a as n ! 1. Hence, q ( r ) D d ( r ¡ a) ,

(1.5)

where d is the delta function, and the distribution is concentrated at r D a (see Figure 1A). This concentration is caused by the law of large numbers, when all the neurons Žre independently. Such concentration still holds even where there are pairwise correlations or triple-wise correlations among xi but no further higher-order correlations. Higher-order correlations are required for q ( r) to be widely distributed.

130

S. Amari, H. Nakahara, S. Wu, and Y. Sakai

2. Completely dependent case: All the neurons Žre or are silent at the same

time with probability a or 1 ¡ a, respectively. Then r D 1 with probability a and r D 0 with probability 1 ¡ a, q ( r ) D ad (r ¡ 1) C (1 ¡ a )d (r ).

(1.6)

The distribution has two concentrated peaks, at r D 0 and r D 1. This is the other extreme case where all the neurons synŽre completely. This article studies the intrinsic mechanism that generates a smooth distributed pattern q (r ) of Žring, as is seen in Figures 1B and 1C. Such a structure was observed in Bohte et al. (2000), which mentioned that higher-order interactions of neurons play a fundamental role. Information geometry (Amari & Nagaoka, 2000) is a useful tool for studying higher-order interactions and is applied to analysis of neuron populations (Amari, 2001; Nakahara & Amari, 2002). Here, we use a simple model to analyze the distributed Žring by using information geometry. We prove that weak but many higher-order interactions are necessary for generating a smooth distribution and a twopeak distribution. This elucidates intrinsic synchronous Žring mechanisms of neurons. We further show how to calculate the terms of higher-order interactions from a distribution of activity. The maximum entropy distribution is also studied, conŽrming the results of Bohte et al. (2000). 2 A Simple Model of Neuron Pool

A simple model we consider is a pool of neurons where the potentials ui of the neurons are subject to a joint gaussian distribution. Let us normalize the variance of ui equal to 1, and let a be the covariance of any two ui and uj . There are no higher-order correlations except for pairwise correlations in the gaussian case, but the nonlinear transformation, ( 1, ui > 0 (2.1) xi D 0, ui · 0, generates higher-order correlations among neural Žring x1 , . . . , xn . This distribution is realized when a pool receives common inputs s1 , . . . , sm . Let ui be the weighted sum of inputs to the ith neuron, ui D

m X jD 1

wij sj ¡ h,

(2.2)

where wij is the connection weight from the jth input to the ith neuron. The weights are randomly assigned, for example, ( 1, with probability c (2.3) wij D 0, with probability 1 ¡ c.

Synchronous Firing and Higher-Order Interactions in Neuron Pool

131

These ui are gaussian because of the central limit theorem. We normalize the scale of ui such that they are subject to the gaussian distribution N (¡h, 1) with mean ¡h and variance 1. Let the covariance of two variables be 1 ¸ 6 a ¸ 0 for any i D j. Then we may put ui D

p

1 ¡ avi C

p

ae ¡ h,

(2.4)

where v1 , . . . , vn and e are independent standard gaussian random variables subject to N (0, 1) . We prove that the activity distribution q (r ) D q ( r, a) has the following form under this model. Theorem 1.

The activity distribution q ( r, a) is given, in the limit n ! 1, by "

2a ¡ 1 q ( r, a) D C exp 2(1 ¡ a)

» F

¡1

p

a ( r) ¡ h 2a ¡ 1

where C is the normalization constant and Z 1 1 u2 ( ) e p F D e ¡ 2 du. p ae p 2p h¡ 1¡a

¼

2

#

,

(2.5)

(2.6)

Before giving a proof, we show the shape of q ( r) D q ( r, a) in terms of a 6 (see Figure 2). When a D 0, that is, ui are nearly independent, q (r, 0) is the delta function and is concentrated. (When a D 0 exactly, we cannot use this formula, because F ¡1 (r ) reduces to a constant.) In the range 0 < a

A probability distribution of a binary n-dimensional vector x is expanded in general as 8 9 <X = X hi xi C hij xi xj C ¢ ¢ ¢ C h1¢¢¢n x1 ¢ ¢ ¢ xn ¡ y , (3.1) p (x ) D exp : ; i< j

where hi , hij , . . . , h1¢¢¢n are parameters to specify p ( x) , and y is the normalizing factor. Such a model is called a full log-linear model in statistics. The

132

S. Amari, H. Nakahara, S. Wu, and Y. Sakai 8

q(r)

6

4

2

0 0.0

0.2

0.4

r

0.6

0.8

1.0

Figure 2: Joint Žring distributions of simple model for various a.

log-linear model is used in Gerstein, Bedenbaugh, and Aertsen (1989), Martignon et al. (1995), Vaadia et al. (1995), and others. Information geometry (Amari & Nagaoka, 2000; Amari, 2001; Nakahara & Amari, 2002) elucidates the meaning of this decomposition. The parameter fhi1 ¢¢¢ik g represents the kth-order interactions of k neurons i1 , . . . , ik, such that it is orthogonal to the parameters specifying the marginal distributions of m neurons ( m < k). When p ( x) is symmetric, all hi are equal, and we use h1 for representing 6 it. Also, all hij ( i D j) are equal, and we denote them by h2 , and so on. Hence, p ( x, µ ) D exp

nX

o hk Xk ¡ y ( µ ) ,

(3.2)

where µ D (h1 , . . . , hn ) is the new parameter vector to specify the symmetric distribution and Xk D

X

xi1 ¢ ¢ ¢ xik ,

(3.3)

with the summation taken over all combinations of k indices (i1 , . . . , ik ) . The set of all the probability distributions fp( x, µ )g is an exponential family, where the expectations of Xk are E[Xk] D n C kE[x1 , . . . , xk ], and E denotes expectation and n Ck is the binomial coefŽcients.

(3.4)

Synchronous Firing and Higher-Order Interactions in Neuron Pool

133

The probability q (r, µ ) of exactly k D nr neurons Žring is q ( r, µ ) dr D n CkProbfx1 D ¢ ¢ ¢ D x k D 1 I x kC 1 D ¢ ¢ ¢ D xn D 0g,

(3.5)

because there are n C k distinct patterns of exactly k neurons Žring whose probabilities are equal. When exactly k speciŽc neurons Žre, for example, x1 D ¢ ¢ ¢ D xk D 1 and all the others are 0, X (3.6) Xj D xi1 ,...,ij D k Cj , when j · k, when j > k,

Xj D 0,

(3.7)

so that Probfx1 D ¢ ¢ ¢ D x k D 1 I x kC 1 D ¢ ¢ ¢ D 0g 8 9 <X = k D exp k Cjhj ¡ y . : ;

(3.8)

jD 1

When n is large, we have an asymptotic formula, n Cnr

D p

1 expfnH ( r) g, 2p nr (1 ¡ r)

(3.9)

where H ( r) is the entropy, H ( r) D ¡r log r ¡ (1 ¡ r ) log(1 ¡ r) .

(3.10)

When k D nr, k X jD 1

k Cjhj

D nrh1 C

nr ( nr ¡ 1) nr ( nr ¡ 1) ( nr ¡ 2) h2 C h3 C ¢ ¢ ¢ 2 6

C nrhnr¡1 C hn ,

(3.11)

which is a polynomial in r. Taking these into account, we can rewrite equation 3.5 in the form q ( rI µ ) D exp[nfn (r, µ ) ¡ y ( µ ) ],

(3.12)

where fn ( r, µ ) D H ( r) C where kj,n ( r) D

nr Cj

1X hj kj,n ( r) , n

/ n is a polynomial in r.

(3.13)

134

S. Amari, H. Nakahara, S. Wu, and Y. Sakai

When fn ( r, µ ) has an isolated peak at r D r0 , there exists c > 0 such that fn ( r0 , µ ) ¡ fn (r, µ ) > c > 0

(3.14)

6 r 0, for |r ¡ r0 | > e for arbitrary small e > 0, and we have, for any r D

q ( r, µ ) D exp[nf fn ( r, µ ) ¡ fn ( r 0, µ )g] ! 0, q ( r 0, µ )

(3.15)

as n ! 1. In this case, q ( r, µ ) is concentrated at r0 . In order to have widely spread q ( r) , fn ( r, µ ) should be of order 1 / n. This gives the following theorem. When the activity distribution q ( r) is not concentrated, fn (r, µ ) D O ( n1 ) , and the interactions h2 , . . . , hn¡2 scale as Theorem 2.

h2 D O

³ ´ 1 , n

³ h3 D O ³ hk D O ³ hk D O

1 n2

´

1 nk¡1 1

,

...,

´ ,

k·

n 2

,

k>

n . 2

´

nn¡k

(3.16)

Moreover, µ satisŽes in the limit n ! 1, 1X hj kj,n ( r) . n!1 n

H ( r) D ¡ lim Proof.

(3.17)

Since fn ( r, µ ) is a polynomial, for a nonconcentrated distribution

q ( r) , lim n[ fn ( r, µ ) ¡ fn ( r0 , µ ) ] < 1

n!1

(3.18)

is required. This holds only when fn ( r, µ ) ¡ fn (r 0, µ ) is of order 1 / n, and hence all the hj scale as mentioned, and equation 3.16 holds. We now consider the case when there are no higher-order interactions. The simplest one is that all the hj D 0 except for h1 . The distribution p ( x, µ ) is independent, and q ( r) D d ( r ¡ r0 ) , where r0 D E[r], is easily determined from h1 . The second simplest case is that all hi are zero except for h1 and h2 , h3 D h4 D ¢ ¢ ¢ D 0. This is the case where only pairwise interactions exist.

Synchronous Firing and Higher-Order Interactions in Neuron Pool

135

The stationary distribution of the Boltzmann machine belongs to this class. The distribution is written as µ » ³ ´¼ ¶ 1 1 ¡y , (3.19) q ( r, µ ) D exp n H ( r) C h1 r C hN2 r r ¡ 2 n where hN2 D nh2 . The function fn ( r) D H ( r) C h1 r C hN2 r2

(3.20)

has a peak in the interval [0, 1] for hN2 < 0. Hence, we have a concentrated distribution. When hN2 is positive and large, there can be two concentrated peaks at r D 0 and r D 1, one of which dominates except for the symmetric case. By similar reasoning, we can prove that there is no widespread distribution q ( r) , when there is only a Žnite number of higher-order interactions. Weak higher-order interactions of almost all orders are required for realizing a widespread activity distribution. Theorem 3.

When h1 D nhN1 is of order n and h2 is of order 1, they dominate over the entropy term, and Remark.

µ q ( r, µ ) ¼ exp

¶ n2 fhN2 r2 ¡ hN1 r ¡ y g ¡ y . 2

(3.21)

Hence, for hN2 > 0, the distribution has two concentrated delta peaks at r D 0 and r D 1. This is the case of Bohte et al. (2000). For a symmetric distribution with E[xi ] D rN and Cov[xi , xj ] D r, the variance of r is "» ¼ 2# 1X 1 n¡1 ( xi ¡ rN ) r. (3.22) V[r] D E D V[xi ] C n n n Remark.

When r > 0, the distribution of r is not concentrated but is spread with variance r. This shows that whenever the covariance of xi and xj is a positive constant of order 1, q ( r) is widespread and higher-order interactions exist necessarily. 4 Calculations of Interactions

We calculate the higher-order interactions from a distribution q ( r) . To this Q ( ) Q ( ) Q ( ) end, we introduce P random variables X1 x , . . . , Xn x such that Xi x is equal to 1 when xj D i, that is, when exactly i neurons are excited, and is

136

S. Amari, H. Nakahara, S. Wu, and Y. Sakai

equal to 0 otherwise. This XQ i is different from Xi , and we have X XQ k D xi1 ,...,ik (1 ¡ xikC 1 ) , . . . , (1 ¡ xin ) ,

(4.1)

where the summation is taken over all permutations of indices. We put g k D E[Xk ],

(4.2)

gQ k D E[XQ k ].

(4.3)

We then have the following linear relations: gQ i D

n X jD 1

Theorem 4.

Aijgj I gi D

n X j D1

BijgQj .

(4.4)

The matrices A D ( Aij ) and B D (Bij ) are mutually inverse, and

Aij D (¡1) j¡ij Ci ,

(4.5)

Bij D j Ci .

(4.6)

Let us put

Proof.

xN i D 1 ¡ xi

(4.7)

and xi C xN i D 1. We then have the following relations: x1 , . . . , x k D x1 , . . . , x k ( x kC 1 C xN kC 1 ) , . . . , (xn C xN n ) ,

(4.8)

x1 , . . . , x k (1 ¡ xkC 1 ) , . . . , (1 ¡ xn ) D x1 , . . . , x kxN kC 1 , . . . , xN n .

(4.9)

By expanding the above relations and taking expectation, we have the relations that connect fE[x1 , . . . , x k]g and fE[x1 , . . . , x kxN kC 1 , . . . , xN n ]g. In order to obtain the explicit forms of A and B, we use the identity n C k n¡k Cj

D n CkC j kC j Ck .

Now we introduce new parameters hQ1 , . . . , hQn by X X hQi D hj D Bjihj , AijhQi .

(4.10)

(4.11)

Then we have another expression of probability distribution p (x ), p ( x, µQ ) D exp because

P

hi Xi D

nX

P

o hQj XQ j ¡ y ,

hQi XQ i .

(4.12)

Synchronous Firing and Higher-Order Interactions in Neuron Pool

137

The probability of exactly k neurons Žring is q ( k/ n ) dr D

X

x:

P

p ( x)

xi D k

D n Ck expfhQk ¡ y g,

(4.13)

because when exactly k neurons Žre, XQ i D 1 only when i D k, and all the other XQ i vanish, and there are n Ck such Žring patterns. From this, we have an explicit expression of hQk, hQk D ¡ log n Ck C log[q(k/ n ) / q (0) ],

(4.14)

because expf¡y g D

1 q (0) . n

(4.15)

When q ( r) is given, the corresponding higher-order interactions hk are given explicitly as hi D

X j

Aji f¡ log n Cj C log[q( j/ n ) / q (0) ]g.

(4.16)

5 Entropy Maximization

Bohte et al. (2000) treated the distribution q (r ) by maximizing the entropy under the condition that the Žrst-order and the second-order statistics are Žxed. We touch on this problem. The entropy of p ( x) is given by ¡

X

p ( x) log p ( x) X X gQ i log gQ i ¡ gQ 0 log gQ 0 C gQ i log n Ci , D ¡

(5.1)

where gQ 0 D 1 ¡

n X i D1

gQi .

(5.2)

This is the negative of the dual potential of the exponential family in information geometry (Amari & Nagaoka, 2000). Therefore, hQi are calculated by @ hQi D Q ( ´Q ) D log gQi ¡ log gQ 0 ¡ log n Ci . @gQ i

(5.3)

138

S. Amari, H. Nakahara, S. Wu, and Y. Sakai

The problem is written as: to maximize X SD¡ p (x ) log p (x )

(5.4)

under g1 D nE[x1 ] D const, g2 D n C2 E[x1 x2 ] D const.

(5.5) (5.6)

The solution is given by the distribution p ( x) D expfl1 X1 C l2 X2 ¡ y g,

(5.7)

which has only the pairwise correlations but no higher-order interactions. Now we rewrite the constraints: g1 D g2 D

n X jD 1 n X jD 2

Qj , j C1g

(5.8)

Qi. j C2g

(5.9)

The entropy is written in terms of ´Q as SD¡

n X iD 0

gQ i log gQ i C

X

gQ i log n Ci .

(5.10)

Now we Žx g1 and g2 . The maximal entropy distribution is given from @S @gQ i

D 0,

i D 3, 4, . . . , n.

The solution (Bohte et al., 2000) is ± i ² log Di D ¡ ¡ C 1 (i ¡ 1) log D 0 2 1 ¡ i (i ¡ 2) log D1 C i ( i ¡ 1) log D2 , 2

(5.11)

(5.12)

where Di D

1 gQ i . n Ci

(5.13)

The activity distribution has two peaks in general, but the bigger peak dominates as n ! 1.

Synchronous Firing and Higher-Order Interactions in Neuron Pool

139

6 Derivation of Activity Distribution

The probability that k D nr neurons Žre among n neurons is given by »

k q ( r) dr D Pr r D n

¼

D n Ck Prfx1 D x2 D ¢ ¢ ¢ D x k D 1, xk C 1 D ¢ ¢ ¢ D xn D 0g,

(6.1)

where dr D 1 / n. In order to calculate Prfx1 D ¢ ¢ ¢ D x k D 1, x kC 1 D ¢ ¢ ¢ D xn D 0g

D Prfu1 , . . . , u k > 0 I u k C 1 , . . . , un < 0g,

(6.2)

we Žrst Žx e in the expression ui in equation 2.4 and calculate the conditional probability. We then take the average over e . Since u1 , . . . , un are independent when e is Žxed, Prfx1 D ¢ ¢ ¢ D x k D 1, x kC 1 D ¢ ¢ ¢ D xn g

D Ee [Prfu1 , u2 , . . . , u k > 0, u k C 1 , . . . , un · 0 | e g] D Ee [n C k fPr(u > 0 | e )g k fPr(u · 0 | e ) gn¡k ],

(6.3) (6.4)

where Ee is the expectation taken with respect to a random variable e and Pr (¢ | e ) is the conditional probability conditioned on e —in other words, the probability for a Žxed e . In the above derivation, we used the homogeneity of neural activity, that is, symmetry of x1 , . . . , xn . Let us put F (e ) ´ Pr ( u > 0 | e ) p ¶ µ h ¡ ae D Pr ui > p 1¡a Z 1 t2 1 D p e ¡ 2 dt. p h¡ ae 2p p1¡a

(6.5)

We then have µ ¶ k Pr r D D Ee [n C k fF( e ) gk f1 ¡ F ( e ) gn¡k ] n ³ 2´ Z 1 1 e k n¡k ¢p exp ¡ D de. (6.6) n C k fF( e )g f1 ¡ F ( e ) g 2 2p ¡1

140

S. Amari, H. Nakahara, S. Wu, and Y. Sakai

To investigate further, by taking a large n limit, we approximate equation 6.6 as an equation taking a continuous variable r. In other words, we take µ ¶ k ¡¡¡¡! q ( r) dr, (6.7) P rD n n!1 where dr D we obtain

1 , n

and r on the right side is now a continuous variable. Then

µ ¶ k q (r ) ’ nPr r D n D nEe [n Cnr fF( e ) gnr f1 ¡ F ( e ) gn¡nr ]

r

» ³ ´¼ Z 1 1 ¡ F (e) n F (e) C (1 ¡ r ) log exp n r log D 2p r (1 ¡ r) ¡1 1¡r r ³ 2´ 1 e ¢p exp ¡ de 2 2p r ³ 2´ Z 1 1 e n expfnz (e ) g ¢ p exp ¡ (6.8) D de, 2p r (1 ¡ r) ¡1 2 2p where we used the following approximation, n Cnr

’ p

1 exp(¡nr log r ¡ n (1 ¡ r) log(1 ¡ r ) ) , 2p nr (1 ¡ r )

and deŽned z ( e ) ´ r log

1 ¡ F (e) F (e) C (1 ¡ r) log . 1 ¡r r

(6.9)

By Laplace’s approximation (also referred to as the saddle-point approximation), we get r q (r ) ’

s n ¢ 2p r (1 ¡ r )

Á ! e02 2p 1 exp(nz (e 0 ) ) ¢ p exp ¡ , (6.10) 2 n|z00 ( e0 ) | 2p

where e 0 is given by

e 0 ´ arg max z (e ) . Equation 6.11 is solved by setting e 0 as the solution of » ¼ 1 ¡r dz (e ) r ¡ D F0 ( e ) D 0, 1 ¡ F (e ) de F (e)

(6.11)

(6.12)

Synchronous Firing and Higher-Order Interactions in Neuron Pool

141

and the solution is

e 0 D F ¡1 ( r) .

(6.13)

For r ! 0, 1, we have e 0 ! § 1 or, equivalently, F ( e ) ! 0, 1. By some calculations, equation 6.10 is written by " q ( r) D C exp

# p ( h ¡ ae 0 ) 2 e 20 ¡ , 2(1 ¡ a) 2

(6.14)

where C is a normalization constant. Using equation 6.13, we Žnally obtain the required activity distribution. 7 Conclusion

The stochastic mechanism underlying the synchronous Žring of neurons is studied by using information geometry. When the probability distribution of the activity of neurons is concentrated on one value (the delta function), the fraction of Žring neurons is almost the same in each time. When the distribution has two peaks at r1 and r2 , the activity takes the higher value r2 where many neurons are synchronizing and the lower value r1 at other times. When the distribution is widespread and continuous, the neuron pool shows a variety of activities. It is a new Žnding that higher-order interactions are necessary for generating widespread activities, and independent, pairwise, or such a limited number of interactions cannot generate a widespread distribution. We have used a simple model of neuron pool that receives stimuli from overlapping common neurons to show that it can generate higher-order interactions and hence a widespread distribution of activity. Although the model is simple, it suggests many interesting features. One is the role of higher-order interactions in a neuron pool. This may be used to judge if there are such interactions in experimental data. We treated only symmetric and homogeneous interactions, but the essential point is the same even in unhomogeneous networks. It is also interesting to note that even when the neuronal potentials ui have no higher-order interactions, their nonlinear transforms can have them. We have not discussed dynamical aspects of synchronous Žring such as in synŽring chains, a subject for further research. References Abeles, M. (1991). Corticonics: Neural circuits of the cerebral cortex. Cambridge: Cambridge University Press. Aertsen, A., & Arndt, M. (1993). Response synchronization in the visual cortex. Current Opinion in Neurobiology, 3(4), 586–594.

142

S. Amari, H. Nakahara, S. Wu, and Y. Sakai

Amari, S. (2001).Information geometry on hierarchy of probability distributions. IEEE Transactions on Information Theory, 47(5), 1701–1711. Amari, S., & Nagaoka, H. (2000). Methods of information geometry. New York: Oxford University Press. Bohte, S. M., Spekreijse, H., & Roelfsema, P. R. (2000). The effects of pair-wise and higher-order correlations on the Žring rate of a postsynaptic neuron. Neural Computation, 12, 153–179. Gerstein, G. L., Bedenbaugh, P., & Aertsen, A. M. (1989). Neuronal assemblies. IEEE Transactions on Biomedical Engineering, 36(1), 4–14. Martignon, L., Von Hasseln, H., Grun, ¨ S., Aertsen, A., & Palm, G. (1995). Detecting higher-order interactions among the spiking events in a group of neurons. Biological Cybernetics, 73(1), 69–81. Nakahara, H., & Amari, S. (2002). Information geometric measure for neural spikes. Neural Computation, 14, 2269–2316. Singer, W., & Gray, C. M. (1995). Visual feature integration and the temporal correlation hypothesis. Annual Review of Neuroscience, 8, 555–586. Vaadia, E., Haalman, I., Abeles, M., Bergman, H., Prut, Y., Slovin, H., & Aertsen, A. (1995). Dynamics of neuronal interactions in monkey cortex in relation to behavioural events. Nature, 373, 515–518.

Received February 19, 2002; accepted July 3, 2002.

LETTER

Communicated by Bard Ermentrout

Determination of Firing Times for the Stochastic Fitzhugh-Nagumo Neuronal Model Henry C. Tuckwell [email protected] Department of Mathematics, University of California, Irvine, CA 92697, U.S.A., and Laboratory for Visual Neurocomputing, Riken Brain Science Institute, Wako-shi, Saitama 351-0198, Japan Roger Rodriguez [email protected] Centre de Physique ThÂeorique, CNRS, F13288 Marseille Cedex 9, France Frederic Y. M. Wan [email protected] Department of Mathematics, University of California, Irvine, CA 92697, U.S.A. We present for the Žrst time an analytical approach for determining the time of Žring of multicomponent nonlinear stochastic neuronal models. We apply the theory of Žrst exit times for Markov processes to the Fitzhugh-Nagumo system with a constant mean gaussian white noise input, representing stochastic excitation and inhibition. Partial differential equations are obtained for the moments of the time to Žrst spike. The observation that the recovery variable barely changes in the prespike trajectory leads to an accurate one-dimensional approximation. For the moments of the time to reach threshold, this leads to ordinary differential equations that may be easily solved. Several analytical approaches are explored that involve perturbation expansions for large and small values of the noise parameter. For ranges of the parameters appropriate for these asymptotic methods, the perturbation solutions are used to establish the validity of the one-dimensional approximation for both small and large values of the noise parameter. Additional veriŽcation is obtained with the excellent agreement between the mean and variance of the Žring time found by numerical solution of the differential equations for the one-dimensional approximation and those obtained by simulation of the solutions of the model stochastic differential equations. Such agreement extends to intermediate values of the noise parameter. For the mean time to threshold, we Žnd maxima at small noise values that constitute a form of stochastic resonance. We also investigate the dependence of the mean Žring time on the initial values of the voltage and recovery variables when the input current has zero mean. Neural Computation 15, 143–159 (2003)

° c 2002 Massachusetts Institute of Technology

144

H. Tuckwell, R. Rodriguez, and F. Wan

1 Introduction

Following the Žrst use of microelectrodes to observe the activity of individual neurons in the mammalian central nervous system, it was soon found that such activity was for the most part highly irregular and unpredictable (Burns, 1968). There have been proposed and analyzed many mathematical models whose aim is to provide quantitative theories of these phenomena (Softky & Koch, 1993; Stevens & Zador, 1998; Lee & Kim, 1999; Tuckwell, 1989, has a review of early modeling). In the past several years, there has been a vigorous renewed interest in such aspects of neuronal spiking activity, especially with regard to an understanding of information processing (Shadlen & Newsome, 1995; Konig, Engel, & Singer, 1996; Tiesinga, JosÂe, & Sejnowski, 2000). There have been many experimental studies of the spike trains emitted by cortical cells in response to natural and artiŽcial stimulation (Burns, 1968; Kreiter & Singer, 1996), but it is still uncertain which models reproduce experimental details (Softky & Koch, 1993; Plesser & Gerstner, 2000). Whereas many early studies of neuronal networks were based on the assumption that there are two possible states for each element of the network corresponding to a Žring or nonŽring nerve cell, models have since become increasingly realistic and have begun to include physiological details of neurons, including synaptic currents of various kinds. Methods of research have included phase analysis, simulation, or numerical integration of systems of ordinary differential equations, as well as direct analytical approaches to problems involving stochastic model neurons with synaptic input. We have recently reported a reductionist method for nonlinear noisy single neurons and networks in which differential equations are solved for the Žrst- and secondorder moments of the model dynamical variables (Tuckwell & Rodriguez, 1998). Although there has been considerable progress with mathematical analysis of the behaviors of model neurons, much of the research has involved simpliŽed models. In this direction, there have been advances with classical one-dimensional models, but these have mainly been for linear integrateand-Žre models such as lead to the subthreshold potential being represented as an Ornstein-Uhlenbeck process (Plesser & Tanaka, 1997; Plesser & Gerstner, 2000). There have been only a few analytical studies including noise in nonlinear models such as the Fitzhugh-Nagumo system and the HodgkinHuxley system (Rodriguez, 1995; Tanabe & Pakdaman, 2001). Although several software routines are available to implement simulation studies of model neurons with both linear and nonlinear dynamics, as, for example, in Liley, Alexander, Wright, and Aldous (1999), it is always desirable to have results obtained using analytical approaches. Using the theory of multidimensional (Markov) diffusion processes, we derive equations for the moments of random variables associated with the time to reach threshold in the Fitzhugh-Nagumo system. Although this sys-

Stochastic Neuronal Firing Times

145

tem does not have as Žrm a basis as conductance-based Hodgkin-Huxley type models, it is relatively simple and more directly amenable to analysis. We describe several methods of Žnding Žring times by solving associated ordinary or partial differential equations, including approximations suitable for small and large values of the noise parameter. Comparison will be made with results obtained by simulating the stochastic Fitzhugh-Nagumo system directly. In a subsequent article, we will conduct a similar analysis of the original Hodgkin-Huxley equations. The approach that we use can be employed for any conductance-based model. 2 The Stochastic Fitzhugh-Nagumo System

The Fitzhugh-Nagumo (FN) system of two differential equations has been employed to provide insight into the more complex Hodgkin-Huxley system of four equations. It shares with the latter the properties of subthreshold responses, action potentials or spikes in response to suitable stimuli, and repetitive activity (periodic solutions) in certain ranges of stimuli. There are associated Hopf bifurcations at the parameter values associated with the end points of these ranges. We study the following corresponding stochastic version of this model (Tuckwell, 1986, 1989; Feng & Brown, 2000) in which the components obey the following equations, dX D [ f ( X) ¡ Y C I]dt C sdW dY D b ( X ¡ c Y ) dt,

(2.1) (2.2)

where X D X ( t) is the “voltage” variable, Y D Y ( t) is the “recovery” variable, W is a standard (zero mean, variance t) Wiener process, s is a constant determining the overall input noise level, I D I ( t) is a deterministic input current (stimulus) that may be constant or time varying, and b and c are positive constants. The function f is a cubic f ( x ) D kx ( x ¡ a ) (1 ¡ x ) ,

(2.3)

where 0 < a < 1. Usually one takes a < 1 / 2 in order to obtain suitable supra-threshold responses. 2.1 Exact Equations for the Moments of the Time to a Threshold. Let T ( x, y ) be the Žrst exit time of the vector-valued process ( X, Y ) from an open set A in the phase-space for a starting point (x, y ) 2 A. Assuming that the stimulus I is constant so that the process is in fact temporally homogeneous, the theory of Markov diffusion processes (Tuckwell, 1976) yields the following partial differential equations (PDEs) for the Žrst moment

146

H. Tuckwell, R. Rodriguez, and F. Wan

F (x, y ) D E[T ( x, y )] and for the second moment G ( x, y) D E[T 2 (x, y )]: [ f ( x ) ¡y C I] [ f (x ) ¡y C I]

@F ( x, y ) @x

@G ( x, y ) @x

C b ( x¡c y)

C b ( x ¡ c y)

@F ( x, y) @y

@G ( x, y ) @y

C

s 2 @2 F (x, y ) D ¡1, 2 @x2

(2.4)

C

s 2 @2 G (x, y ) D ¡2F(x, y ) , 2 @x2

(2.5)

with suitable boundary conditions on the boundary of the set A. The set A will be chosen as an open rectangle Q containing ( x, y) . Since we are primarily interested in the time taken for the potential variable X to reach a threshold value h > 0 from the resting value X (0) D 0, the rectangle is taken to be (¡a, h ) £ ( y1 , y2 ) , where a is positive. The usual boundary conditions for Žrst exits from the open rectangle Q are that F and G vanish on its boundaries. In order to facilitate numerical solution, we modify these boundary conditions, but in such a way that accurate values are still obtained for the moments. The condition that F and G vanish along the line x D h is retained. For the parameter ranges and initial values of interest, most exits of ( X, Y ) from Q occur through the line x D h and, for the values of Y (0) considered here, away from the lines y D y1 and y D y2 . As a ! 1, the paths are less inuenced by the boundary condition at x D ¡a, with nearly all exits occurring at x D h rather than x D ¡a. Hence, one may use a reecting condition Fx (¡a, : ) D 0, where the subscript denotes partial differentiation, instead of F (¡a, : ) D 0. The zero-derivative conditions Fy ( : , y1 ) and Fy ( : , y2 ) may similarly be employed. These zero-derivative conditions facilitate numerical solution of the differential equations by avoiding sharp boundary layers at these three edges of Q and correspond to the Euler boundary conditions in a variational formulation of the problem. The time to Žrst threshold crossing or time to Žrst spike from an initial rest point (0, y 0 ) is not the same as the interspike interval in nonlinear models and has Žrst and second moments F (0, y 0 ) and G (0, y 0 ), respectively, where yo 2 ( y1 , y2 ) . 3 Methods of Solution

We have approached the solution of the moment equations in various ways. Approximate solutions can always be obtained by simulation, as described in the next section. However, accurate numerical solution of the moment equations is a useful and perhaps necessary check on simulation results, which are subject to sampling error. We have pursued the following methods of solution: (1) a simpliŽed approach, which considers the single component process X; (2) approximate solution of the PDEs 2.4 and 2.5 when s is small; and (3) the solution of the PDEs when s is large.

Stochastic Neuronal Firing Times

147

Figure 1: The value of the recovery variable is practically unchanged during the passage of X ( t) from 0 to the threshold value of h D 0.6. Here the input current is I D 1.5, and the noise parameter is s D 1. The values of the remaining parameters a D 0.1, b D 0.015, c D 0.2, k D 0.5, are the standard set throughout. 3.1 A SimpliŽed Approach. In this Žrst approach, we reduce the problem to a one-dimensional one. For the PDEs 2.4 and 2.5, this is tantamount to neglecting the Fy and Gy terms. The justiŽcation for this approach is the observation that during the initial stages of the interspike interval, the recovery variable Y (t ) is practically unchanged and may therefore be set as a constant equal to the initial value Y (0) D y0 . This is illustrated in Figure 1. We then consider the diffusion process satisfying

dX D [ f ( X) ¡ y 0 C I]dt C sdW.

(3.1)

Letting X (0) D x 2 (¡1, h ) , we consider the exit time Th ( x) , which is the time to reach the level h, identiŽed as the threshold for an action potential, given that the voltage started below this value. The Žrst and second moments, F (x ) and G ( x ) , respectively, of this exit time satisfy the differential equations s 2 d2 F ( x ) dF ( x) C [ f ( x) ¡ y0 C I] D ¡1, 2 2 dx dx

(3.2)

148

H. Tuckwell, R. Rodriguez, and F. Wan

and s 2 d2 G ( x) dG (x ) C [ f ( x ) ¡ y 0 C I] D ¡2F(x) , 2 dx2 dx

(3.3)

with the boundary conditions F (h ) D 0 and G (h ) D 0. For numerical solution, we make the interval of initial values Žnite with x 2 (¡a, h ) , where a > 0. As mentioned in the previous section, at the lower boundary, it is advantageous to choose zero-derivative conditions Fx (¡a) D 0 and Gx (¡a) D 0. The quantities of interest are the moments of the time to reach threshold from rest, F (0) and G (0). A further reduction is sometimes possible. The solutions of the secondorder ordinary differential equations 3.2 and 3.3 may be approximated when s is small by the solutions of the Žrst-order equations [ f ( x ) ¡ y 0 C I]

dF ( x ) D ¡1, dx

(3.4)

[ f ( x ) ¡ y 0 C I]

dG ( x) D ¡2F(x) . dx

(3.5)

and

These equations admit only a single boundary value, chosen as zero at x D h. A more accurate approach for small s is described in the next section. 3.2 Solution When s Is Small. When s is small, the PDEs 2.4 and 2.5 can be solved approximately using perturbation methods. Putting 2 D s 2 / 2, the equation satisŽed by the mean Žrst exit time can be written as 2 Fxx C L1 [F] D ¡1,

(3.6)

L1 [F] D P ( x, y ) Fx C Q ( x, y) Fy ,

(3.7)

with

and where P ( x, y ) D I ¡ y C kx ( x ¡ a) (1 ¡ x ) and Q ( x, y ) D b ( x ¡ c y) . The solution is obtained with F (h , y) D 0 and Fx (¡a, y) D 0. A regular perturbation expansion is sought in powers of 2 : F ( x, yI 2 ) D

1 X nD 0

Fn ( x, y) 2

n

.

(3.8)

Substitution in equation 3.6 gives Žrst-order PDEs for the coefŽcients fFn ( x, y ) g, which can be solved by the method of characteristics. For s ¿ 1, it is expected that the solution of equation 2.4 is F ( x, yI 2 ) ¼ F 0 ( x, y) C O (2 ) ,

Stochastic Neuronal Firing Times

149

where F 0 is the leading term in the regular perturbation expansion, equation 3.8. A similar development leads to G ( x, yI 2 ) ¼ G0 (x, y ) C O (2 ). For the leading terms F 0 and G 0, the relations 3.4 and 3.5 are applicable along with the characteristics of equations 2.4 and 2.5. Here, we have y D y0 ( x) , known from a separate determination of the characteristic projections for the linear PDEs for F and G. It follows that the vanishing slope conditions for F and G as x ! ¡1 are satisŽed by F 0 and G 0 . For s < 1, these leading term solutions (or the corresponding multiterm truncated perturbation series) differ by less than 5% from those obtained with the one-dimensional approach, validating the latter for such values of s. 3.3 Solution for Large s . It is convenient to rewrite equation 2.4 in diy s2 mensionless variables z D hx , l D 2h 2 , and u D h . The PDE becomes

lFzz C p ( z, u ) Fz C b ( z ¡ c u ) Fu D ¡1

(3.9)

for ¡a · z · 1, where p (z, u) D [I C kz (z ¡ a ) (1 ¡ hz) ¡ u] and where I D hI , k D kh, a D ha , a D F (1, u ) D 0 and Fz (¡a, u ) D 0. With Z

P ( z, u ) D

a h.

(3.10)

The boundary conditions are now

z

p ( w, u ) dw

µ ¶ h 4 1 1 2 3 ( ) (1 C ha) z ¡ az , D I¡u zC k ¡ z C 4 3 2

(3.11)

we multiply throughout by 1l eP/ l and write the PDE for F as [eP / l Fz ]z C

1 eP/ l [b(z ¡ c u ) eP/ l Fu ] D ¡ . l l

(3.12)

When l À 1, a regular perturbation solution is sought of the form F (z, uI l) D

1 X

Fn (z, u ) l ¡n .

(3.13)

nD 1

With the boundary conditions for the leading term given by F1 (1, u ) D 0 and F1,z (¡a, u ) D 0, the solution for F1 can be found as a double integral. The following expression is then obtained for the mean Žrst exit time: F (z, uI l) D

1 l

Z

1 z

e ¡P(g,u) / l

Z

g ¡a

eP (f ,u ) / l df dg[1 C O (l¡1 ) ].

(3.14)

150

H. Tuckwell, R. Rodriguez, and F. Wan

A similar analysis gives G ( z, uI l) D

1 l

Z

1

e ¡P(g,u) / l

Z

z

£ [1 C O (l ¡1 ) ].

g ¡a

( ) eP f ,u / l F1 (f , u ) df dg

(3.15)

In addition to yielding explicit accurate approximations for the moments of the Žrst passage time, the above leading term asymptotic solutions (or more accurate multiterm truncated series solutions) provide, for large s, validation of the one-dimensional approximation of section 3.1. The leading term of the perturbation solution for F is seen from equation 3.12 to be determined by equation 3.2. Similar remarks apply to G. 4 Results and Comparison with Simulation

We have determined the Žrst two moments of T (x, y ) by the above methods. We have additionally simulated the processes described by the evolution equations 2.1 and 2.2 using a standard strong Euler method. The use of this scheme is straightforward and is vindicated in the existing problem by virtue of the close agreement of the simulation results and those obtained using the analytical framework. To apply this method, with X ( kD t) and Y ( kD t) approximated by X k and Yk, respectively, the simulation scheme consists of the recursive relations p X kC 1 D Xk C [ f (X k ) ¡ Yk C I]D t C b D tN k Y kC 1 D Yk C b[Xk ¡ c Yk]D t, where k D 0, 1, 2, . . . and fNkg is a sequence of independent and identically distributed standard normal random variables, obtained from computer library routines. This was done principally with the set of parameter values a D 0.1, b D 0.015, c D 0.2, and k D 0.5 for various values of s and I. The threshold value h for spikes was estimated from sample paths with allowance for the discrete nature of simulation. The value of D t was pragmatically chosen between 0.00001 and .001, the value being determined in each case by requiring the minimum number of steps to threshold to be at least 20 and the maximum number several thousand. The initial value of X ( t) was set at 0 and that of Y ( t) usually set at 1.0 in the results to be reported. Other values of Y (0) were nevertheless considered for comparison, some being reported in the Žnal section. When I is less than about 1.3, the (expected) value of X ( t) rarely attains the designated threshold value of h D 0.6. For the chosen parameters, we therefore consider mainly values of I ¸ 1.3, although the value I D 0 is discussed later in the article. The values of the noise parameter s are chosen from 0.05 to 5.0. For small

Stochastic Neuronal Firing Times

151

Figure 2: Mean times to reach threshold versus noise parameter s. Parameter values are given in the text. Values are determined by simulation (circles) and by numerical solution of differential equation 2.4 (+).

values of s, the number of trials in simulations was 4500, whereas only 1500 trials were employed for larger values. Plots of the means of the times for X ( t) to reachh for various I as a function of s are shown in Figures 2 and 3. For a given value of I, results obtained by numerical solution are shown in the same Žgure for comparison. In Figure 2, the values of I are 1.3 and 1.5, whereas in Figure 3, the values of I are 2 and 3. The numerical results shown here were obtained using the simpliŽed approach from section 3.1. There is excellent agreement between the two sets of results, especially allowing for sampling error in the simulation results. The magnitudes of sampling errors ( § standard error) in the simulated means are summarized as follows, with best cases Žrst. When I D 1.3, for s D 0.05, § 0.17%; for s D 0.5, § 1.31%; for s D 0.75, §2.70%; and for s D 5, § 3.95%. When I D 1.5, for s D 0.05, § 0.13%; for s D 0.5, § 1.20%; for s D 0.75, §2.53%; and for s D 5, § 3.93%. When I D 2, for s D 5, § 0.10%; for s D 0.5, § 0.92%; for s D 0.75, §2.2%; and for s D 5, §4.50%. When I D 3, for s D 0.05, § 0.07%; for s D 0.5, § 0.66%; for s D 0.75, § 1.76%; and for s D 5, § 4.30%.

152

H. Tuckwell, R. Rodriguez, and F. Wan

Figure 3: Mean Žrst Žring time versus s when I D 2 and I D 3. Values are determined by simulation (circles) and by numerical solution of differential equation 2.4 (+).

In general, the means from simulation are slightly larger than those obtained by solution of the differential equation. It can be clearly seen in the top part of Figure 2, where I D 1.3, that as s increases away from zero, the mean Žrst passage time actually increases. After reaching a maximum when s is about equal to 0.25, the mean then declines, as is expected with increased variability. Similar behavior is observed for the larger values of the mean input current, except that the maximum of the mean interval occurs at larger values of s. This phenomenon is akin to stochastic resonance, well documented in nonlinear systems with noise, including neural systems such as that of Hodgkin-Huxley and Fitzhugh-Nagumo (Lee & Kim, 1999; Nozaki & Yamamoto, 1998; Massanes & Vicente, 1999). In Figures 4 and 5, we show the standard deviation of the time to reach threshold as determined by both simulation and solution of the differential equation 2.5 for the second moment. It can be seen that there is excellent agreement between the two sets of results for all four values of the input parameters I and s. In each of the four sets of results shown, the standard deviation attains a maximum as a function of s. The magnitudes of the sampling errors in the standard deviations in representative cases are as

Stochastic Neuronal Firing Times

153

Figure 4: Standard deviation of time to reach threshold versus noise parameter s. Parameter values are given in the text. Values are determined by simulation (circles) and by numerical solution (+) of the differential equation 2.4.

follows. When I D 1.3, for s D 0.05, § 2.0%; and for s D 5, § 2.6%. When I D 3, for s D 0.05, § 2.2%; and for s D 5, §4.8%. In Figure 6, we show the behavior of the coefŽcient of variation (CV, deŽned as standard deviation divided by the mean), calculated from computer simulations, as a function of the noise parameter s for the values I D 1.3 and I D 3. The results for I D 1.5 and I D 2 are similar to these, respectively. It is seen that the CV is greater than 1 over much of the range of parameters considered. 5 Discussion

Since interest Žrst arose in the theory of stochastic neuronal Žring over 30 years ago, analytical methods have been employed to determine properties of interspike intervals in many integrate-and-Žre model neurons in which the subthreshold membrane potential satisŽes a linear stochastic differential equation. In this regard, nonlinear models such as Fitzhugh-Nagumo and Hodgkin-Huxley have been studied mainly using computer simulations. We have used an analytical framework for the Žrst time in a nonlinear

154

H. Tuckwell, R. Rodriguez, and F. Wan

Figure 5: Standard deviation of Žrst Žring time versus s when I D 2 and I D 3. Values determined by simulation (circles) and by numerical solution of the differential equation 2.5 (+).

stochastic neural model. To this end, we have reduced the task of characterizing the Žring time to the solution of linear second-order partial differential equations. Some remarks about the nature of these equations and our methods of solution of them are in order when the noise term is small. In such singular perturbation problems, where the highest derivative term in the differential equation is multiplied by the small parameter, setting the small parameter equal to zero (or, equivalently, seeking the leading term of a regular perturbation solution in the small parameter) reduces the order of the equation and therefore leads to a loss of some of the components of the general solution of the equation. However, for problems with a certain combination of boundary conditions, such as prescribing the normal derivative equal to zero at negative inŽnity or at some large negative value of x and prescribing the value of the unknown (equal to zero in our case) at the opposite end of the interval, the missing solutions are irrelevant, as they would have been eliminated by the boundary condition at negative inŽnity anyway. That is, our less general solution, obtained after setting the small parameter s equal to zero, is made to satisfy all boundary conditions (in particular, the

Stochastic Neuronal Firing Times

155

Figure 6: CoefŽcient of variation of the time to threshold versus s for I D 1.3 and I D 3. Values are determined by simulation (circles) and by numerical solution of the differential equation 2.5 (+).

condition at negative inŽnity automatically), whereupon by a uniqueness theorem for linear PDEs, we have the correct solution of the problem without obtaining the missing solutions. A previous example of this technique is contained in our extensive investigation of the Ornstein-Uhlenbeck process representation of neuronal potential (Wan & Tuckwell, 1982). 5.1 Parameter and Initial Values: The Case I D 0. The parameters we have employed are necessarily in restricted particular ranges, and a brief consideration of other cases is warranted. The deterministic constant component of the input current I represents the drift or the mean synaptic drive, which depends on the relative magnitudes and frequencies of excitation and inhibition. The quantities we have determined using an analytical framework are the moments of the time for emergence of the Žrst spike. For values of I such as those employed in the calculations performed above, periodic Žring occurs; that is, subsequent to the Žrst spike, an approximately periodic spike train ensues. If the train is sufŽciently regular with a consistently deŽned apparent threshold, such as occurs when the noise amplitude is not excessive (see Figure 2 of Tuckwell & Rodriguez, 1998), then the methods

156

H. Tuckwell, R. Rodriguez, and F. Wan

Figure 7: Effects of changing the initial value of the voltage variable X (0) are illustrated for a Žxed value of Y (0) D 0. (Top) For the two indicated values of X (0) , the variation of the mean time to a spike as a function of noise parameter s is shown. (Bottom) Variations in mean time to a spike as the initial value X (0) changes for two Žxed values of the noise parameter.

employed here could be applied to the determination of the moments of the associated yet different random variable, namely, the interspike interval. Such a determination will be deferred to a later article. The initial value chosen above for the dynamical variables X to illustrate the usefulness of the analytical approach is X (0) D 0 because it corresponds to a natural resting or equilibrium value for the unperturbed system. The value Y (0) D 1 was based on previously published results as it was close to the values in sustained rhythmic Žring for chosen parameter values (Tuckwell & Rodriguez, 1998). However, other initial conditions are of interest and also the case where periodic Žring does not necessarily occur because the mean input current is not sufŽciently large. To study these aspects, we have determined, using simulation, the statistics of the time to Žrst spike in certain “excitable” cases when I D 0. First, we consider the effects of varying the initial value X (0) of the voltage variable for a Žxed value Y (0) D 0 of the recovery variable. Some results for the mean time to Žrst spike are shown in Figure 7. Comparison of the results

Stochastic Neuronal Firing Times

157

Figure 8: Effects of changing the initial value of the recovery variable Y (0) are illustrated for a Žxed value of X (0) D 0. (Top) For the indicated values of Y (0) , the variation of the mean time to a spike as a function of noise parameter s is shown. (Bottom) Variations in mean time to a spike as the initial value Y (0) changes for the two indicated values of the noise parameter.

in the top part of this Žgure with those in Figures 2 and 3 indicates that the variation in this mean as the noise parameter varies from 0.5 to 5.0 is similar for the three different values of X (0) even despite the different values of I and the fact that Figures 2 and 3 correspond to the periodic Žring mode with a different value for Y (0) . That the mean time to a spike is moderately sensitive to changes in X (0) for Žxed values of all the parameters and Y (0) can be seen in the lower part of Figure 7. Similar remarks apply to the effects of changing Y (0) , as can be gleaned from inspection of Figure 8. Noticeable here, however, is that when s D 4, the mean time to a spike increases only gradually as Y (0) increases from ¡0.4 to 1.0. In summary, we have investigated for the Žrst time the determination of times for a threshold crossing in a nonlinear stochastic neuronal model, that of Fitzhugh-Nagumo, with the addition of a white gaussian noise representing stochastic excitation and inhibition. We have found moments of threshold crossing times accurately by solving the associated partial differ-

158

H. Tuckwell, R. Rodriguez, and F. Wan

ential equations, obtained from the theory of Markov processes. The validity of the methods employed has been veriŽed by comparison with computer simulation results. An interesting structure in the mean Žrst passage time was noted at small values of the noise parameter. In future work, we will report our Žndings for the more complicated Hodgkin-Huxley system. Acknowledgments

H. C. T. appreciates support from the Riken Brain Science Institute, Saitama, Japan. References Burns, B. D. (1968). The uncertain nervous system. London: Arnold. Feng, J-F., & Brown, D. (2000). Integrate-and-Žre models with nonlinear leakage. Bulletin of Mathematical Biology, 62, 467–481. Konig, P., Engel, A. K., & Singer, W. (1996). Integrator or coincidence detector? The role of the cortical neuron revisited. Trends in Neurosciences, 19, 130–137. Kreiter, A. K., & Singer, W. (1996). Stimulus-dependent synchronization of neuronal responses in the visual cortex of the awake macaque monkey. Journal of Neuroscience, 16, 2381–2396. Lee, S-G., & Kim, S. (1999). Parameter dependence of stochastic resonance in the stochastic Hodgkin-Huxley neuron. Phys. Rev. E, 60, 826–830. Liley, D. T. J., Alexander, D. M., Wright, J. J., & Aldous, M. D. (1999). Alpha rhythm emerges from large-scale networks of realistically coupled multicompartmental model cortical neurons. Network: Comput. Neural Syst., 10, 79–92. Massanes, S. R., & Vicente, C. J. P. (1999). Nonadiabatic resonances in a noisy Fitzhugh-Nagumo neuron model. Phys. Rev. E, 59, 4490–4497. Nozaki, D., & Yamamoto, Y. (1998). Enhancement of stochastic resonance in a Fitzhugh-Nagumo neuronal model driven by colored noise. Phys. Lett. A, 243, 281–287. Plesser, H. E., & Gerstner, W. (2000). Noise in integrate-and-Žre neurons: From stochastic inputs to escape rates. Neural Computation, 12, 367–384. Plesser, H. E., & Tanaka, S. (1997). Stochastic resonance in a model neuron with reset. Physics Letters A, 225, 228–234. Rodriguez, R. (1995). Coupled Hodgkin-Huxley neurons with stochastic synaptic inputs. In J. Bertrand (Ed.), Modern group theoretical methods in physics (pp. 233–242). Norwood, MA: Kluwer. Shadlen, M. N., & Newsome, W. T. (1995). Noise, neural codes and cortical organization. Curr. Opin. Neurobiol., 4, 569–579. Softky, W. R., & Koch, C. (1993). The highly irregular Žring of cortical cells is inconsistent with temporal integration of random EPSPs. J. Neurosci., 13, 334–350. Stevens, C. F., & Zador, A. M. (1998). Input synchrony and the irregular Žring of cortical neurons. Nature Neuroscience, 1, 210–217.

Stochastic Neuronal Firing Times

159

Tanabe, S., & Pakdaman, K. (2001). Dynamics of moments of Fitzhugh-Nagumo neuronal models and stochastic bifurcations. Phys. Rev. E, 63, 031911. Tiesinga, P. H. E., JosÂe, J. V., & Sejnowski, T. J. (2000). Comparison of currentdriven and conductance-driven neocortical model neurons with HodgkinHuxley voltage-gated channels. Phys. Rev. E, 62, 8413–8419. Tuckwell, H. C. (1976). On the Žrst exit time problem for temporally homogeneous Markov processes. J. Appl. Prob., 13, 39–48. Tuckwell, H. C. (1986). Stochastic equations for nerve membrane potential. J. Theoret. Neurobiol., 5, 87–99. Tuckwell, H. C. (1989). Stochastic processes in the neurosciences. Philadelphia: SIAM. Tuckwell, H. C., & Rodriguez, R. (1998). Analytical and simulation results for stochastic Fitzhugh-Nagumo neurons and neural networks. J. Computational Neuroscience, 5, 91–113. Wan, F. Y. M., & Tuckwell, H. C. (1982). Neuronal Žring and input variability. J. Theoret. Neurobiol., 1, 197–218. Received August 24, 2001; accepted July 3, 2002.

LETTER

Communicated by Richard Zemel

Developmental Constraints Aid the Acquisition of Binocular Disparity Sensitivities Melissa Dominguez [email protected] Department of Computer Science, Universityof Rochester,Rochester,NY 14627,U.S.A. Robert A. Jacobs [email protected] Department of Brain and Cognitive Sciences, University of Rochester, Rochester, NY 14627, U.S.A. This article considers the hypothesis that systems learning aspects of visual perception may beneŽt from the use of suitably designed developmental progressions during training. We report the results of simulations in which four models were trained to detect binocular disparities in pairs of visual images. Three of the models were developmental models in the sense that the nature of their visual input changed during the course of training. These models received a relatively impoverished visual input early in training, and the quality of this input improved as training progressed. One model used a coarse-scale-to-multiscale developmental progression, another used a Žne-scale-to-multiscale progression, and the third used a random progression. The Žnal model was nondevelopmental in the sense that the nature of its input remained the same throughout the training period. The simulation results show that the two developmental models whose progressions were organized by spatial frequency content consistently outperformed the nondevelopmental and random developmental models. We speculate that the superior performance of these two models is due to two important features of their developmental progressions: (1) these models were exposed to visual inputs at a single scale early in training, and (2) the spatial scale of their inputs progressed in an orderly fashion from one scale to a neighboring scale during training. Simulation results consistent with these speculations are presented. We conclude that suitably designed developmental sequences can be useful to systems learning to detect binocular disparities. The idea that visual development can aid visual learning is a viable hypothesis in need of study. 1 Introduction

Human infants are born with limited perceptual, motor, and cognitive abilities relative to adults. Within the Želd of developmental psychology, there Neural Computation 15, 161–182 (2003)

° c 2002 Massachusetts Institute of Technology

162

M. Dominguez and R. Jacobs

are at least two perspectives regarding these limitations. The older and more commonplace view is that these limitations are barriers that must be overcome in order for a child to achieve adult function (Piaget, 1952). That is, they are deŽciencies or immaturities that serve no positive purpose. A newer view is that these apparent inadequacies are in fact helpful, perhaps necessary, stages in development. Limited mental abilities, according to this theory, reect simple neural representations that are useful stepping-stones or building blocks for the subsequent development of more complex representations (Turkewitz & Kenney, 1982). The idea that early developmental stages are useful or necessary precursors to more advanced stages is becoming increasingly studied in the cognitive neurosciences. The development of biological nervous systems is sometimes characterized as using a bootstrapping strategy. Greenough, Black, and Wallace (1987) speculated that asynchrony in brain development serves the useful function of stage setting. The developmental schedule for the maturation of different brain regions is staggered such that neural systems that develop relatively early provide a suitable framework for the development of later experience-sensitive systems. Harwerth, Smith, Duncan, Crawford, and von Noorden (1986) provided experimental evidence consistent with this hypothesis by performing behavioral studies of sensitive periods for visual development in monkeys. Their results suggest that these sensitive periods are organized into a hierarchy in which early visual functions requiring information processing in the earliest stages of the visual system have shorter sensitive periods than higher-level functions requiring more central neural processing. A second example is provided by the work of Shatz (1996). Within the lateral geniculate nucleus (LGN) of adult mammals, retinal ganglion cell axons from one eye are segregated from those arising from the other eye to form a series of alternating eye-speciŽc layers. These layers are not present initially in development. Moreover, they form during a period in which vision is not possible. Shatz argued that the development of eye-speciŽc layers is characterized by at least two important events. First, retinal ganglion cells spontaneously show waves of activity that sweep across the retina such that activity at nearby cells is more highly correlated than at distant cells. Second, LGN cells use a Hebb-style adaptation mechanism that sorts connections based on local correlations of activity in order to form eye-speciŽc layers. If so, then this is a clear example in which a developmental event at an earlier visual region (spontaneous waves of activity at the retina) sets the stage for an event at a later visual region (Hebb-style adaptation at the LGN). Newport (1990) hypothesized that children use a bootstrapping strategy when attempting to learn a language. Human languages are componential systems in which small linguistic components are systematically combined to form larger linguistic structures. According to Newport’s “less is more” hypothesis, the limited attentional and memorial abilities of children are useful when learning a language because they help children segment and

Acquisition of Binocular Developmental Sensitivities

163

identify the small components that comprise the language. Elman (1993) studied an implementation of this general idea and showed that a recurrent neural network whose memory capacity was initially limited but then gradually increased during the course of training learned aspects of an artiŽcial grammar better than a network whose memory capacity was never limited; that is, the second network’s memory capacity was always equal to that of the Žrst network at the end of training. Elman claimed that this outcome supports the idea that starting small is important to the subsequent acquisition of complex mental abilities. Rohde and Plaut (1999), however, were unable to replicate Elman’s simulation results, so it is difŽcult to know how to interpret these results. This article considers the hypothesis that systems learning aspects of visual perception may beneŽt from the use of suitably designed developmental progressions during training. We report the results of simulations in which four systems were trained to detect binocular disparities in pairs of visual images. Three of the systems were developmental models in the sense that the nature of their input changed during the course of training. These systems received a relatively impoverished visual input early in training, and the quality of this input improved as training progressed. The fourth system was a nondevelopmental model; the nature of its input remained constant during the course of training. The inputs to the systems were left and right retinal images Žltered with binocular energy Žlters tuned to various spatial frequencies. The training of the Žrst system, referred to as the coarse-scale-to-multiscale model (model C2M), included a developmental sequence such that the system was exposed only to lowspatial-frequency information at the start of training, and information at higher spatial frequencies was added to its input as training progressed. The training of the second system, referred to as the Žne-scale-to-multiscale model (model F2M), included an analogous developmental sequence with spatial frequency information added in the reverse order. This system received high-spatial-frequency information at the start of training, and information at lower spatial frequencies was added as training progressed. The third system, referred to as the random-developmental model (model RD), was similar to models C2M and F2M in the sense that its training included a developmental sequence. However, whereas the inputs received by models C2M and F2M at each developmental stage were organized by spatial frequency content, the inputs received by model RD at each stage were randomly selected. Finally, the fourth system, referred to as the nondevelopmental model (model ND), was not trained using a developmental sequence; it received information at all spatial frequencies throughout the training period. When comparing the three developmental models with the nondevelopmental model, there are at least two reasonable predictions that one could make about the simulation results. One prediction is that the nondevelopmental model should outperform the developmental models. The

164

M. Dominguez and R. Jacobs

nondevelopmental model received all input information throughout all stages of training, whereas the developmental models were deprived of portions of the input at certain training stages. If more information is better than less information, then the nondevelopmental model ought to perform best. This would be consistent with the traditional view of human infant development—that perceptual immaturities are barriers to be overcome. An alternative prediction, consistent with the general approach of the less-is-more hypothesis, is that the developmental models would show the better performance. If it is believed that too much information could lead a learning system in its early stages of training to form poor internal representations, then the developmental models ought to have an advantage. The nondevelopmental model had a greater number of inputs than the developmental models during the early stages of training and, thus, a greater number of modiŽable weights. Because learning in neural networks is a search in weight space, the nondevelopmental model needed to perform an unconstrained search in a high-dimensional weight space. Unconstrained searches frequently lead to the acquisition of poor representations. In contrast, the developmental models initially had fewer inputs and thus fewer modiŽable weights. During early stages of training, their searches in weight space were comparatively constrained. If the constraints were appropriate, this should have facilitated the acquisition of useful representations by the developmental models. When comparing the performances of the two developmental models whose stages are based on spatial frequency content (models C2M and F2M), there are at least three reasonable predictions one could make about the simulation results. One prediction is that model C2M should perform best. A motivation for this prediction comes from the Želd of computer vision. Consider the task of aligning two images of a scene where the images differ due to a small horizontal offset in their viewpoints. Roughly, this is known as the stereo correspondence problem. If this were something that you had never done before, you might try to align Žne details of each image. For example, you might pick a white dot in one image and repeatedly try aligning it with white dots in the other image. If the images are highly textured, such an approach would be inefŽcient because there is an intractable number of potential alignments that might need to be checked before the two images are properly aligned (see Figure 1A). If, however, the images are blurred, the Žne details that were the source of so much confusion would be removed, leaving a smaller number of larger features in each image. The problem of Žnding a good alignment is now signiŽcantly easier because there is a smaller number of potential alignments (see Figure 1B). After you have gained skill at aligning blurred images, you would then have a reliable foundation that you could use to learn to align clearer images. You might attempt to match the larger image features Žrst. Subsequent analysis of Žne details would seek to remove ambiguities that arise when aligning large features instead of being a starting point for locating correspondences.

Acquisition of Binocular Developmental Sensitivities

165

A

B Figure 1: (A) One image from a stereo pair depicting a textured object and background. Due to the Žne details in the stereo pair, solving the stereo correspondence problem is relatively difŽcult. (B) The image from A blurred so as to remove the Žne details. It is easier to solve the stereo correspondence problem when the stereo pair is blurred because image features tend to be larger, less numerous, and more robust to noise.

This style of processing is commonplace in the computer vision literature. Systems by Marr and Poggio (1979), Quam (1986), and Barnard (1987), among many others, initially search for correspondences within a pair of low-resolution images. Low-resolution images are used initially because these images contain fewer image features, larger image features, and image features that are relatively robust to noise. Next, these systems reŽne their estimates of corresponding image features using information from one or more higher-resolution pairs of images. The usefulness of a coarse-to-Žne processing strategy when searching for stereo correspondences suggests that a coarse-scale-to-multiscale developmental strategy might be useful for learning to detect binocular disparities. Before adopting this hypothesis, however, it is important to keep in mind the differences between these two strategies. Computer vision researchers use a coarse-to-Žne strategy when searching for correspondences in individual pairs of images. In contrast, we used the coarse-scale-to-multiscale developmental sequence while training a learning system to detect binocular disparities using many pairs

166

M. Dominguez and R. Jacobs

of images. It is not obvious that lessons from one situation can be applied to the other situation. If we assume that human intelligence provides a guide to creating machine intelligence, then a second motivation for the prediction that model C2M should perform best is the fact that human infants show a related developmental progression. Visual acuity is often measured using a grating, which is a visual pattern whose luminance values are sinusoidally modulated. Acuity is characterized by the highest-frequency grating, which is distinguishable from a solid gray pattern. Using this method, it has been found that newborns’ visual acuity is extremely poor. Whereas adults with normal vision (so-called 20 / 20 vision) can discriminate approximately 30 cycles per degree of arc, newborns can discriminate only 1 to 2 cycles per degree, giving them a visual acuity of about 20/ 400. Acuity improves approximately linearly from these low levels at birth to near adult levels by around eight months of age (Norcia & Tyler, 1985). Importantly for our purposes, infants are acquiring other visual abilities during this time period; in particular, sensitivity to binocular disparities appears at around four months of age (Atkinson & Braddick, 1976; Fox, Aslin, Shea, & Dumais, 1980; Held, Birch, & Gwiazda, 1980; Petrig, Julesz, Krop, Baumgartner, & Anliker, 1981). We speculate that the developments of visual acuity and binocular disparity sensitivity may be related in the sense that poor acuity at an early age aids in the acquisition of disparity sensitivity later in life. An alternative prediction is that model F2M should perform better than model C2M. A motivation for this prediction is the fact that computer vision researchers often Žnd it easier to solve the stereo correspondence problem by Žrst extracting edge information from left and right images, a form of highfrequency bandpass Žltering, and then searching for a good alignment of the images based on this information. This strategy is useful because edges can be sparse, large, and robust to noise relative to other image features. Analogous to computer vision systems’ initial use of high-frequency information, model F2M is initially trained solely with information extracted from highfrequency bandpass Žlters. If we are willing to assume that computer vision methods for Žnding stereo correspondences may provide lessons for how learning systems can learn to detect binocular disparities, then we might predict that model F2M has an advantage. A second motivation for the prediction that model F2M should perform best is the seemingly counterintuitive result that neural networks trained with input patterns that have been corrupted by noise frequently show better generalization than equivalent networks trained with input patterns that have not been corrupted (Sietsma & Dow, 1991). Training with noisy inputs has been shown analytically to be equivalent to a form of regularization (Bishop, 1995; Matsuoka, 1992; Webb, 1994). In other words, the learning process of networks trained with noisy inputs is more constrained than that of similar networks whose inputs are not corrupted by noise. If the retinal images contain noise and if model C2M tends to Žlter out the noise during

Acquisition of Binocular Developmental Sensitivities

167

early stages of training, then this model will not obtain the beneŽts of training with noisy inputs. We would therefore expect model F2M to perform best. Another possible prediction is that models C2M and F2M perform about equally well. If the relative advantages of model C2M (being able to use fewer image features, larger image features, and more noise-resistant image features at the start of training) and the relative advantages of model F2M (initial exposure to high-frequency bandpass information, being able to obtain the beneŽts of training with noisy inputs) are roughly balanced, then neither model would be expected to show superior performance. Furthermore, we outlined the logic of computer vision researchers who advocate a coarse-to-Žne processing strategy when analyzing stereo correspondences, and we tentatively speculated that the usefulness of this strategy suggests that the seemingly related coarse-scale-to-multiscale developmental strategy might be useful for learning to detect binocular disparities. If, however, we again make the assumption that human intelligence provides a guide to creating machine intelligence, then it is worth noting that humans often do not use a coarse-to-Žne strategy when analyzing stereo correspondences. Mallot, Gillner, and Arndt (1996) found that unambiguous information at a coarse scale is not always used by observers to disambiguate Žner-scale information and that observers can use unambiguous Žne-scale information to disambiguate coarse-scale information, meaning that observers are using a Žne-to-coarse processing strategy in these circumstances. Related Žndings have been reported by several other researchers (McKee & Mitchison, 1988; Mowforth, Mayhew, & Frisby, 1981; Smallman, 1995). Because human observers do not use a coarse-to-Žne strategy or a Žne-to-coarse strategy exclusively, we might not expect the exclusive use of a C2M developmental strategy or the exclusive use of a F2M strategy to yield a relative performance advantage. When comparing the performances of models C2M and F2M with that of model RD, the only reasonable prediction seems to be that model RD should perform no better than the other developmental models, and it is likely that it will perform worse. We have included model RD in this article in order to demonstrate that the use of a developmental sequence does not necessarily lead to performance advantages. Instead, the beneŽts of a developmental sequence are found only when the sequence incorporates constraints that are useful for learning the desired behavioral task. This article reports the results of computer simulations comparing the learning performances of the developmental and nondevelopmental models on the task of estimating binocular disparities in novel pairs of images. The results show that the developmental models whose stages are based on spatial frequency content (models C2M and F2M) consistently outperformed the nondevelopmental and random developmental models. We speculate that the superior performance of these models is due to two important features of their developmental progressions: (1) these

168

M. Dominguez and R. Jacobs

models were exposed to visual inputs at a single scale early in training, and (2) the spatial scale of their inputs progressed in an orderly fashion from one scale to a neighboring scale. Simulation results consistent with these speculations are presented. We conclude that suitably designed developmental sequences can be useful to systems learning to detect binocular disparities and that the general idea that visual development can aid visual learning is a viable hypothesis in need of future study. 2 Developmental and Nondevelopmental Models

Figure 2 illustrates the structure of the developmental and nondevelopmental models. This structure is based on a similar architecture studied by Gray, Pouget, Zemel, Nowlan, & Sejnowski (1998). The retinal layer consisted of two one-dimensional arrays 62 pixels in length for the left and right eye images. Each retina was treated as if it were shaped like a circle; the leftmost and right-most pixels were regarded as neighbors. This wraparound of the left and right edges was done to avoid edge artifacts. Although onedimensional retinas are a simpliŽcation, their use is justiŽed by the fact that the models were concerned only with horizontal disparities, as these are the ones that provide information about the three-dimensional conŽguration of the visual environment.1 The retinal inputs were Žltered using binocular energy Žlters. Based on neurophysiological studies, Ohzawa, DeAngelis, and Freeman (1990) proposed binocular energy Žlters as a way of modeling the binocular sensitivities of simple and complex cells in primary visual cortex. These Žlters are an extension of motion energy Žlters proposed by Adelson and Bergen (1985). A simple cell receives input from a pair of subunits, one for each retina. The receptive Želd proŽles of the subunits can be described mathematically as Gabor functions: ³ ´ x2 exp ¡ 2 sin(2p vx C w ) 2s 2p s ³ ´ 1 x2 exp ¡ 2 sin(2p vx C w C dw ) . gR ( x, w ) D p 2s 2p s gL ( x, w ) D p

1

(2.1) (2.2)

Each function is a sinusoid multiplied by a gaussian envelope. The parameter x is the distance to the center of the gaussian, s 2 is the variance of the 1

Vertical disparities provide information about viewing distance and angle of gaze but by themselves do not carry information about the three-dimensional structure of the visual environment. That is, by themselves, they cannot be used for making relative or absolute depth judgments. In contrast, horizontal disparities alone can be used for making relative depth judgments and, when scaled by viewing distance information (perhaps obtained via vertical disparities), for making absolute depth judgments.

Acquisition of Binocular Developmental Sensitivities

169

Output Layer Hidden Layer Input Layer

Binocular Energy Filters

Low SF

Med SF

Stage 1 Stage 2 Stage 3

High SF

Left Eye Retina

Right Eye Retina

Figure 2: The developmental and nondevelopmental models shared a common structure. The bottom portion of this structure is the left and right retinal images. These images are then Žltered by binocular energy Žlters. The outputs of these Žlters are the inputs to an artiŽcial neural network that is trained to estimate the disparity present in the images. The model illustrated here is model C2M in which low-spatial-frequency information was received during early stages of training, and information at higher frequencies was added as training progressed.

gaussian, v is the frequency of the sinusoid, and w and dw are referred to as the base phase and phase offset of the sinusoid, respectively. The Gabor functions associated with the left and right retinal subunits differ in that the phase of one is offset from the phase of the other. The output of a simple cell is formed in two stages. First, the convolution of the left retinal image with the left subunit Gabor is added to the convolution of the right retinal image with the right subunit Gabor; next, this sum is half-wave rectiŽed and squared (a negative sum is mapped to zero; a positive sum is mapped to its square). The magnitude of a simple cell’s output is related to the presence of a binocular disparity of a particular size in the retinal input. Simple cells formed from subunits with different phase offsets are sensitive to disparities of different sizes (Fleet, Wagner, & Heeger, 1996; Qian, 1994). The output of a complex cell is the sum of the outputs of four simple

170

M. Dominguez and R. Jacobs

cells. Because the base phases of these simple cells form quadrature pairs (the base phases are 0, p / 2, p , and 3p / 2), the complex cell’s output is relatively insensitive to the exact position of a disparity within its receptive Želd. In our simulations, there were 35 receptive Želd locations that received input from overlapping regions of the retina. At each location, there were 30 complex cells corresponding to three spatial frequencies and 10 phase offsets at each frequency. The three spatial frequencies were each separated by an octave: 0.25, 0.125, and 0.0625 cycles per pixel. The standard deviations of the Gabor functions were set to be inversely proportional to the frequency: 1.0 for 0.25 cycles per pixel, 2.0 for 0.125 cycles per pixel, and 4.0 for 0.0625 cycles per pixel. The 10 phase offsets were equally spaced over a range from 0 to p / 2. The outputs of the complex cells were normalized using a softmax nonlinearity, eEi ( x ) / t EO i (x ) D P E ( x ) / t , j je

(2.3)

where E i (x ) was the initial output of the complex cell, EO i ( x ) was the normalized output, t is a scaling parameter known as a temperature parameter (its value was set to 0.25), and j indexed the 10 complex cells with different phase offsets at a receptive Želd location within a single frequency band. As a result of this normalization, complex cells tended to respond to relative contrast in an image rather than absolute contrast. The normalized outputs of the complex cells were the inputs to an artiŽcial neural network. The network had 1050 input units (the complex cells had 35 receptive Želd locations, and there were 30 cells at each location). The hidden layer of the network contained 32 units organized into 8 groups of 4 units each. The connectivity to the hidden units was set so that each group had a limited receptive Želd; a group of hidden units received inputs from seven receptive Želd locations at the complex cell level. The hidden units used a logistic activation function. The output layer consisted of a single linear unit; this unit’s output was an estimate of the disparity present in the right and left retinal images. The weights of an artiŽcial neural network were initialized to small random values and were adjusted during the course of training to minimize a sum of squared error cost function using a conjugate gradient optimization procedure (Press, Teukolsky, Vetterling, & Flannery, 1992). This procedure was used because it tends to converge quickly and because it has no free parameters (e.g., no learning rate or momentum parameters). Weight sharing was implemented at the hidden unit level so that corresponding units within each group of hidden units had the same incoming and outgoing weight values and a hidden unit had the same set of weight values from each receptive Želd location at the complex unit level. This provided the

Acquisition of Binocular Developmental Sensitivities

171

network with a degree of translation invariance and also dramatically decreased the number of modiŽable weight values in the network. It therefore decreased the number of data items needed to train the network and the amount of time needed to train the network. Models were trained and tested using separate sets of training and test data items. Training sets contained 250 randomly generated data items; test sets contained 122 data items that were generated so as to cover the range of possible binocular disparities uniformly. Training was terminated after 35 iterations through the training set in order to minimize overŽtting of the training data. The results reported are based on the data items from the test set. Model C2M was trained using a coarse-scale-to-multiscale developmental sequence. This was implemented as follows. The training period was divided into three stages where the Žrst and second stages were each 10 iterations and the third stage was 15 iterations.2 During the Žrst stage, the neural network portion of the model received only the outputs of complex cells tuned to low spatial frequencies (the outputs of the other complex cells were set to zero). During the second stage, the network received the outputs of complex cells tuned to low and medium spatial frequencies; it received the outputs of all complex cells during the third stage. The training of model F2M was identical to that of model C2M except that its training used a Žne-scale-to-multiscale developmental sequence. During the Žrst stage of training, its network received the outputs of complex cells tuned to high spatial frequencies. This network received the outputs of complex cells tuned to high and medium frequencies during the second stage and received the outputs of all complex cells during the third stage. The training of model RD also used a developmental sequence, though this sequence was generated randomly and thus was not based on the spatial frequency tuning of the complex cells. The collection of complex cells was randomly partitioned into three equal-sized subsets with the constraint that each subset included all phase offsets at all receptive Želd locations. During the Žrst stage of training, the neural network portion of the model received the outputs of only the complex cells in the Žrst subset. It received the outputs of the cells in the Žrst and second subsets during the second stage of training and the outputs of all complex cells during the third stage. In contrast, the training period for the nondevelopmental model was not divided into separate stages; its neural network received the outputs of all complex cells throughout the training period.

2 The number of iterations in the training stages was roughly optimized using models C2M and F2M and the solid object data set described below. Models with stages of different sizes were tested, and the results suggest that models tended to show worse generalization performance if the number of iterations in the Žrst and second stages was decreased or increased.

172

M. Dominguez and R. Jacobs

3 Data Sets and Simulation Results

The performances of the four models were evaluated on three data sets. These data sets were based on related data sets used by Gray et al. (1998). In all cases the images were gray scale with luminance values between 0 and 1 and disparities with values between 0 and 3 pixels. Ten simulations of each model on each data set were conducted. In the solid object data set, images consisted of a single light or dark object on a gray background. The object’s gray-scale value was either between 0.0 and 0.1 or between 0.9 and 1.0, whereas the gray-scale value of the background was always 0.5. The location of the object was randomly chosen to be a real-valued location on the retina. The object’s disparity was randomly chosen to be a real value between 0 and 3 pixels. The object’s size was randomly chosen to be a real value between 10 and 25 pixels. Since the object’s size, location, and disparity were all real numbers, the ends of the object could fall at a real-valued location within a pixel. In these (common) cases, the value of the partially covered pixel was interpolated between the gray-scale value of the object and that of the background in proportion to the amount of the pixel covered by the object and background. An example of a right and left image is shown in the top panel of Figure 3. Given the right and left images, the task of a model was to estimate the object’s disparity. The upper left graph of Figure 4 illustrates the results. The horizontal axis gives the model, and the vertical axis gives the root mean squared error (RMSE) at the end of training on the data items from the test set (the error bars give the standard error of the mean). On average, developmental model C2M had a 16.5% smaller generalization error than the nondevelopmental model (the difference between the mean error rates is statistically signiŽcant; t D 3.77, p < 0.002 using a two-tailed t-test), and a 19.3% smaller error than the random developmental model (t D 4.60, p < 0.001). Developmental model F2M had a 12.2% smaller error than the nondevelopmental model (t D 23.74, p < 0.001) and a 15.2% smaller error than the random develop-

Figure 3: Examples of right and left images (top and bottom rows in each panel) from the solid object, noisy object, and planar data sets.

Acquisition of Binocular Developmental Sensitivities

1.2

solid object data set

1.0

1.2

173

noisy object data set

1.0

RMSE 0.8

0.6

0.8

C2M F2M

RD

1.2

0.6

ND

C2M F2M

RD

ND

planar data set

1.0 RMSE 0.8

0.6

C2M F2M

RD

ND

Figure 4: The four models’ RMSE on the test set data items after training on the three data sets (the error bars give the standard error of the mean).

mental model (t D 49.01, p < 0.001). Clearly, the two developmental models whose developmental progressions were organized by spatial frequency content outperformed both the nondevelopmental model and the random developmental model. A statistical comparison between models C2M and F2M shows that their performances were not signiŽcantly different. The variability in model C2M’s performance was notably greater than the variability in the performances of the other models. Although it is difŽcult to know why, it may have to do with the properties of binocular energy units. As discussed by Qian (1994), Zhu and Qian (1996), and Fleet et al. (1996), the responses of these units may peak even when the input disparity is outside the range of disparities to which the unit was thought to be responsive. In such cases, the large response is known as a false peak. False peaks occur for nearly all stimuli, including white noise. Moreover, some of the false peaks will be signiŽcantly larger than the peak at the disparity to which the unit

174

M. Dominguez and R. Jacobs

was thought to be selective. False peaks tend to occur at integer multiples of the wavelength to which the unit is tuned, meaning that units tuned to low frequencies tend to have false peaks that are relatively far from each other, and units tuned to high frequencies tend to have false peaks near to each other. Consequently, units tuned to low spatial frequencies tend to have false peaks at disparities that are signiŽcantly different from the disparity to which they were thought to be responsive, whereas units tuned to high frequencies tend to have false peaks at disparities that are closer to the disparity to which they were thought to be responsive (see Figure 5). The large variability in model C2M’s performance may be due to the fact that this model is sometimes misled early in training by false peaks at disparities that are far from the disparities to which units are thought to be selective. This is less likely to be as serious a problem for the other models because they are less likely than model C2M to emphasize information provided by units tuned to low spatial frequencies. The learning curves for the four models on the solid object data set are illustrated in the upper left graph of Figure 6. The horizontal axis gives the training time in epochs, and the vertical axis gives the RMSE on the test set data items. The solid black line is for model C2M, the solid gray line is for model F2M, the dashed black line is for model RD, and the dotted black line is for model ND. The learning curve for model ND falls quickly within the Žrst few epochs of training and then is relatively at for the remainder of the training period. In contrast, the developmental models learned relatively slowly, and models C2M and F2M eventually showed the best generalization performance. This result is consistent with the notion described above that apparent inadequacies in performance during early development are not necessarily bad; they sometimes suggest the use of internal representations, which are useful stepping-stones for the subsequent development of advanced behaviors. The images in the second data set, referred to as the noisy object data set, were meant to resemble random dot stereograms frequently used in behavioral experiments. Images contained a noisy object against a noisy background. The gray-scale values of the object pixels and the background pixels were set to random numbers between 0 and 1. The location of the object was randomly chosen to be a real-valued location on the retina. The object’s size was randomly chosen to be a real value between 10 and 25 pixels. The object’s disparity was a randomly chosen integer between 0 and 3 pixels. As before, the task was to map the left and right images to an estimate of the object’s disparity. An example of a left and right image is shown in the middle panel of Figure 3. The results are shown in the upper right panel of Figure 4. As before, the developmental models whose developmental progressions were organized by spatial frequency content performed best. On average, model C2M had a 7.1% smaller generalization error than model ND (t D 448.8, p < 0.001) and a 6.6% smaller error than model RD (t D 2.46, p < 0.05). Model F2M had a

Acquisition of Binocular Developmental Sensitivities

175

Low Spatial Frequency Left Eye

Right Eye

d d+l

High Spatial Frequency Left Eye

Right Eye

d d+l

Figure 5: The top two Žgures illustrate Gabor Žlters for the left-eye and righteye images that are tuned to a low spatial frequency (l is the wavelength). The right-eye Gabor is phase-shifted by d units relative to the left-eye Gabor. The sum of the Žlter responses tends to be large when the disparity in the images is d, d ¡ l, or d C l. Because the wavelength l is relatively large, the sum can peak at input disparities (e.g., d C l) that are far from the disparity (d) to which the sum was thought to be responsive. As illustrated in the bottom two Žgures, a similar situation holds for Žlters tuned to high spatial frequencies, but now the wavelength l is smaller and the false peaks occur at input disparities that are signiŽcantly closer to the disparity to which the sum was thought to be responsive. Consequently, false peaks are more misleading when using Žlters tuned to low frequencies than when using Žlters tuned to high frequencies.

176

M. Dominguez and R. Jacobs

solid object data set

2.2

C2M F2M RD ND

1.8

1.8

RMSE 1.4

1.4

1.0

1.0

0.6

noisy object data set

2.2

0.6 0

10 20 30 training time (epochs)

0

10 20 30 training time (epochs)

planar data set

2.2 1.8 RMSE 1.4 1.0 0.6 0

10 20 30 training time (epochs)

Figure 6: Learning curves for the four models on the three data sets.

4.3% smaller error than model ND (t D 1566.5, p < 0.001) and a 3.8% smaller error than model RD (this result is not statistically signiŽcant). Comparing the developmental models C2M and F2M, C2M had a 2.85% smaller error (t D 170.6, p < 0.001). The learning curves for the four models are shown in the upper right graph of Figure 6. A notable feature illustrated by this graph is that the random developmental model had highly unstable levels of performance during the course of training. The last data set, the planar data set, was different from the Žrst two data sets. Instead of an object in front of a background, the images depicted a frontoparallel plane. The values of the left-image pixels were randomly chosen to be either 0 or 1. The right image was generated by applying an integer shift to the left image of 0, 1, 2, or 3 pixels. Given the left and right images, the task was to estimate the size of the shift. An example of a left and right image is shown in the bottom panel of Figure 3.

Acquisition of Binocular Developmental Sensitivities

177

The results are shown in the bottom graph of Figure 4. Again, the developmental models whose developmental progressions were organized by spatial frequency content tended to outperform the other models. Model C2M had a 6.7% smaller generalization error than model ND (t D 2.27, p < 0.05) and a 1.3% smaller error than model RD (this result is not statistically signiŽcant). Model F2M had an 8.3% smaller error than model ND (t D 16.84, p < 0.001) and a 3.0% smaller error than model RD (t D 7.01, p < 0.001). The performances of models C2M and F2M were not statistically different. The bottom graph of Figure 6 gives the learning curves for the four models. In summary, the simulation results using the three data sets show that the developmental models whose progressions were organized by spatial frequency content (models C2M and F2M) signiŽcantly outperformed the nondevelopmental and random-developmental models. To understand the performances of models C2M and F2M better, we analyzed more carefully their behaviors using the solid object data set. We looked at how these models performed on images with disparities of different sizes (small, midsize, and large disparities) at the end of each developmental stage (stages 1, 2, and 3). These data are given in Figure 7. The three graphs in this Žgure correspond to the three disparity sizes. The horizontal axis of each graph gives the stage number for each model; the vertical axis gives each model’s RMSE at the end of the indicated stage on those test set data items with the indicated disparity size. For purposes of comparison, the graphs also include the data for the nondevelopmental model. Model ND did not signiŽcantly change its ability to detect disparities of a given size across training stages; its performance at detecting small, midsize, or large disparities remained nearly constant across the stages. Model C2M improved its ability to estimate disparities of all sizes as training progressed from one stage to the next. In contrast, model F2M appears to have learned about different-sized disparities in each of the different training stages. For example, it learned a great deal about detecting small disparities in stage 1; in stages 2 and 3, it learned more about detecting midsize and large disparities. Not surprisingly, its performance at detecting small disparities degraded as its performance at detecting midsize and large disparities improved. The question remains as to why models C2M and F2M showed superior performance. One logical possibility is that these models performed well due to the fact that they received relatively few inputs early in training. This possibility can be ruled out, however, because model RD received equally few inputs early in training, but this model did not perform well. We speculate that two important features of the developmental progressions of models C2M and F2M account for their superior performances. First, these models were exposed to visual inputs at a single scale early in training. Model C2M received only coarse-scale information at the start of training; model F2M received only Žne-scale information. In contrast, models RD and ND received information at all spatial scales at all stages of training.

178

M. Dominguez and R. Jacobs

small disparities (0-1 pixels)

mid-size disparities (1-2 pixels)

2.0

2.0

RMSE 1.0

1.0

0.0

123 C2M

123 F2M

0.0

123 ND

123 C2M

123 F2M

123 ND

large disparities (2-3 pixels) 2.0

RMSE 1.0

0.0

123 C2M

123 F2M

123 ND

Figure 7: Performance of models C2M, F2M, and ND on images with disparities of different sizes (small, midsize, and large disparities) at the end of each developmental stage (stages 1, 2, and 3). Training and test data items came from the solid object data set.

We conjecture that it might be advantageous for a learning system to receive inputs at a single scale early in training because this allows the system to combine and compare input features without the need to compensate for the fact that these features could be at different spatial scales. Second, models C2M and F2M might be at an advantage because the spatial scale of their inputs progressed in an orderly fashion from one scale to a neighboring scale. Consequently, when these models received inputs at a new spatial scale, this new scale was close to a scale with which the models were already familiar. If it is the case that this second feature of models C2M and F2M is important, then this leads to an interesting prediction. We predict that developmental models whose progressions do not proceed in an orderly manner from one scale to a neighboring scale ought to show poor performance. To test this prediction, two additional models were created and tested on the solid object data set. These models had developmental stages based

Acquisition of Binocular Developmental Sensitivities

179

solid object data set 1.4 RMSE 1.0

0.6

C2M

F2M

C-CF-CMF

F-CF-CMF

Figure 8: RMSE of models C2M, F2M, C-CF-CMF, and F-CF-CMF on the test items from the solid object data set.

on spatial frequency content (like models C2M and F2M), but the addition of new frequency bands to their inputs at each stage did not proceed in an orderly manner. The Žrst new model is referred to as model C-CF-CMF. It received the outputs of complex cells tuned to a low spatial frequency early in training. In stage 2, it received the outputs of complex cells tuned to low and high frequencies, and it received the outputs of cells tuned to low, medium, and high frequencies in stage 3. The second new model, referred to as model F-CF-CMF, was similar to model C-CF-CMF, but it started with high-frequency information. That is, it received the outputs of complex cells tuned to a high spatial frequency early in training. It received the outputs of cells tuned to low and high frequencies in stage 2, and the outputs of all complex cells in stage 3. Figure 8 shows the performances of models C2M, F2M, C-CF-CMF, and F-CF-CMF at the end of training on the solid object data set. In accord with our prediction, models C-CF-CMF and F-CF-CMF showed very poor performance. These data are consistent with the conjecture that it is advantageous to a developmental system for the spatial scale of its inputs to progress in an orderly fashion from one scale to a neighboring scale. 4 Summary and Conclusion

This article has considered the hypothesis that systems learning aspects of visual perception may beneŽt from the use of suitably designed develop-

180

M. Dominguez and R. Jacobs

mental progressions during training. We reported the results of simulations in which different systems were trained to detect binocular disparities in pairs of visual images. Three of the systems were developmental models in the sense that the nature of their input changed during the course of training. These systems received a relatively impoverished visual input early in training, and the quality of this input improved as training progressed. The fourth system was a nondevelopmental model; the nature of its input remained constant during the course of training. The results show that the developmental models whose stages were based on spatial frequency content, models C2M and F2M, consistently outperformed the nondevelopmental and random developmental models. We speculate that the superior performance of these models is due to two important features of their developmental progressions: (1) these models were exposed to visual inputs at a single scale early in training, and (2) the spatial scale of their inputs progressed in an orderly fashion from one scale to a neighboring scale. Simulation results consistent with these speculations were presented. We conclude that suitably designed developmental sequences can be useful to systems learning to detect binocular disparities, in accord with the less-is-more view of development. Moreover, the idea that visual development can aid visual learning is a viable hypothesis in need of study. With relatively few exceptions, the relationship between development and learning has been ignored by the neural computation community. We believe that this is unfortunate. It is well known that systems learn best when they are suitably constrained through the use of domain knowledge. Learning systems are inherently faced with the bias-variance dilemma (Geman, Bienenstock, & Doursat, 1995). Systems with little or no bias are often capable of learning many different sets of training items. Unfortunately, they tend to interpolate in unpredictable ways and thus generalize poorly to novel data items. In contrast, systems that are constrained through the use of domain knowledge and thus have large bias are not able to learn as wide a variety of training sets. However, they tend to show better generalization performance and less variable generalization performance when exposed to those training sets that they can adequately learn. The design of appropriate developmental progressions through the use of domain knowledge provides researchers with an effective means of biasing their learning systems so as to enhance their performances.

Acknowledgments

This work was supported by NSF graduate fellowship DGE9616170 and by NIH research grant R01-EY13149.

Acquisition of Binocular Developmental Sensitivities

181

References Adelson, E. H., & Bergen, J. R. (1985). Spatiotemporal energy models for the perception of motion. Journal of the Optical Society of America A, 2, 284– 299. Atkinson, J., & Braddick, O. (1976). Stereoscopic discrimination in infants. Perception, 5, 29–38. Barnard, S. T. (1987).Stereo matching by hierarchical,microcanonical annealing (Tech. Rep. 414). Menlo Park, CA: ArtiŽcial Intelligence Center, SRI International. Bishop, C. M. (1995). Training with noise is equivalent to Tikhonov regularization. Neural Computation, 7, 108–116. Elman, J. L. (1993). Learning and development in neural networks: The importance of starting small. Cognition, 43, 71–99. Fleet, D. J., Wagner, H., & Heeger, D. J. (1996). Neural encoding of binocular disparity: Energy models, position shifts, and phase shifts. Vision Research, 36, 1839–1857. Fox, R., Aslin, R. N., Shea, S. L., & Dumais, S. T. (1980). Stereopsis in human infants. Science, 207, 323–324. Geman, S., Bienenstock, E., & Doursat, R. (1995) Neural networks and the bias/variance dilemma. Neural Computation, 4, 1–58. Gray, M. S., Pouget, A., Zemel, R. S., Nowlan, S. J., & Sejnowski, T. J. (1998). Reliable disparity estimation through selective integration. Visual Neuroscience, 15, 511–528. Greenough, W. T., Black, J. E., & Wallace, C. S. (1987). Experience and brain development. Child Development, 58, 539–559. Harwerth, R. S., Smith, E. L., Duncan, G. C., Crawford, M. L. J., & von Noorden, G. K. (1986). Multiple sensitive periods in the development of the primate visual system. Science, 232, 235–238. Held, R., Birch, E., & Gwiazda, J. (1980). Stereoacuity in human infants. Proceedings of the National Academy of Sciences USA, 77, 5572–5574. Mallot, H. A., Gillner, S., & Arndt, P. A. (1996). Is correspondence search in human stereo vision a coarse-to-Žne process? Biological Cybernetics, 74, 95– 106. Marr, D., & Poggio, T. (1979). A computational theory of human stereo vision. Proceedings of the Royal Society of London B, 204, 301–328. Matsuoka, K. (1992). Noise injection into inputs in back-propagation learning. IEEE Transactions on Systems, Man, and Cybernetics, 22, 436–440. McKee, S. P., & Mitchison, G. J. (1988). The role of retinal correspondence in stereoscopic matching. Vision Research, 28, 1001–1012. Mowforth, P., Mayhew, J. E. W., & Frisby, J. P. (1981). Vergence eye movements made in response to spatial-frequency-Žltered random-dot stereograms. Perception, 10, 299–304. Newport, E. L. (1990). Maturational constraints on language learning. Cognitive Science, 14, 11–28. Norcia, A., & Tyler, C. (1985). Spatial frequency sweep VEP: Visual acuity during the Žrst year of life. Vision Research, 25, 1399–1408.

182

M. Dominguez and R. Jacobs

Ohzawa, I., DeAngelis, G. C., & Freeman, R. D. (1990). Stereoscopic depth discrimination in the visual cortex: Neurons ideally suited as disparity detectors. Science, 249, 1037–1041. Petrig, B., Julesz, B., Krop, W., Baumgartner, G., & Anliker, M. (1981). Development of stereopsis and cortical binocularity in human infants: Electrophysiological evidence. Science, 213, 1402–1405. Piaget, J. (1952). The origins of intelligence in children. New York: International Universities Press. Press, W. H., Teukolsky, S. A., Vetterling, W. T., & Flannery, B. P. (1992).Numerical recipes in C: The art of scientiŽc computing. Cambridge: Cambridge University Press. Qian, N. (1994). Computing stereo disparity and motion with known binocular cell properties. Neural Computation, 6, 390–404. Quam, L. H. (1986). Hierarchical warp stereo (Tech. Rep. 402). Menlo Park, CA: ArtiŽcial Intelligence Center, SRI International. Rohde, D. L. T., & Plaut, D. C. (1999). Language acquisition in the absence of explicit negative evidence: How important is starting small? Cognition, 72, 67–109. Shatz, C. J. (1996). Emergence of order in visual system development. Proceedings of the National Academy of Sciences USA, 93, 602–608. Sietsma, J., & Dow, R. J. F. (1991). Creating artiŽcial neural networks that generalize. Neural Networks, 4, 67–79. Smallman, H. S. (1995). Fine-to-coarse scale disambiguation in stereopsis. Vision Research, 34, 2971–2982. Turkewitz, G., & Kenney, P. A. (1982). Limitations on input as a basis for neural organization and perceptual development: A preliminary statement. Developmental Psychobiology, 15, 357–368. Webb, A. R. (1994). Functional approximation by feed-forward networks: A least-squares approach to generalization. IEEE Transactions on Neural Networks, 5, 363–371. Zhu, Y., & Qian, N. (1996). Binocular receptive Želd proŽles, disparity tuning, and characteristic disparity. Neural Computation, 8, 1611–1641. Received November 14, 2001; accepted June 4, 2002.

LETTER

Communicated by Terrence Sejnowski

A QuantiŽed Sensitivity Measure for Multilayer Perceptron to Input Perturbation Xiaoqin Zeng [email protected] Daniel S. Yeung [email protected] Department of Computing, Hong Kong Polytechnic University, Hong Kong The sensitivity of a neural network’s output to its input perturbation is an important issue with both theoretical and practical values. In this article, we propose an approach to quantify the sensitivity of the most popular and general feedforward network: multilayer perceptron (MLP). The sensitivity measure is deŽned as the mathematical expectation of output deviation due to expected input deviation with respect to overall input patterns in a continuous interval. Based on the structural characteristics of the MLP, a bottom-up approach is adopted. A single neuron is considered Žrst, and algorithms with approximately derived analytical expressions that are functions of expected input deviation are given for the computation of its sensitivity. Then another algorithm is given to compute the sensitivity of the entire MLP network. Computer simulations are used to verify the derived theoretical formulas. The agreement between theoretical and experimental results is quite good. The sensitivity measure can be used to evaluate the MLP’s performance. 1 Introduction

Theoretically, an artiŽcial neural network is aimed at realizing a mapping between its inputs and outputs, so the sensitivity of a neural network’s output to its input perturbation is an important measure for evaluating its performance, such as error tolerance and generalization ability. Obviously, a quantiŽed sensitivity measure will have both theoretical and practical value. How to deŽne and quantify an appropriate sensitivity for neural networks is still a problem in need of exploration. In the past decade, a variety of studies on the sensitivity of neural networks have emerged. For feedforward neural networks, there are two kinds of sensitivity that differ in the target function being studied: the sensitivity with respect to an objective (error) function and the sensitivity with respect to the neural networks’ output function (Engelbrecht, Fletcher, & Cloete, 1999). These two sensitivity measures seem in general to be different. The sensitivity of the error function is closely related to the process of training Neural Computation 15, 183–212 (2003)

° c 2002 Massachusetts Institute of Technology

184

X. Zeng and D. Yeung

and thus would be useful for training related issues, whereas the output sensitivity has been found useful in a number of problems both in and after training. This article focuses on the sensitivity with respect to the neural networks’ output function. Stevenson, Winter, and Widrow (1990) made use of a hypersphere as the mathematical model in analyzing the sensitivity of Adalines and Madalines. They deŽned sensitivity as the probability of erroneous output of Madalines under the assumption that input and weight perturbations are small and the number of Adalines per layer is sufŽciently large. Alippi, Piuri, and Sami (1995) generalized the hypersphere model by also considering the sensitivity as the probability of erroneous output of the feedforward multilayered neural networks with multiple-step symmetric activation functions. But their results are applicable only to networks with discrete neuron inputs and outputs and thus are not suitable for multilayer perceptrons (MLPs) with continuous inputs and outputs. Cheng and Yeung (1999) again used a geometrical technique to investigate the sensitivity of neocognitrons by modifying the hypersphere model. Unfortunately, the hypersphere model they used fails in the case of the MLP because its input vectors generally do not span a hypersphere surface or volume. PichÂe (1995) employed a statistical rather than a geometrical argument to analyze the effects of weight errors in Madalines. He made the assumption that inputs and weights as well as their errors are all independently and identically distributed with mean zero. Based on such a statistical model and under the condition that both input and weight errors are small enough, PichÂe derived an analytical expression for the sensitivity as the ratio of the variance of the output error to the variance of the output. Recently, Yeung and Sun (2002) generalized PichÂe’s method by deriving a universal expression of the MLP’s sensitivity for antisymmetric squashing activation functions, without any restriction on the magnitude of input and weight perturbations. However, this approach is applicable only to an ensemble of networks, not to any individual one, because the assumptions on the statistical model are too strong. There are basically two approaches to study the sensitivity of the MLP. The analytical approach deŽnes the sensitivity as a partial derivative of output to input. For example, Hashem (1992) and Fu and Chen (1993) computed the partial derivative one layer at a time, starting from the output layer and proceeding backward (using backward chaining) toward the input layer. But this approach is applicable only when the input deviation tends to zero. The other approach is statistically based. For example, Choi and Choi (1992) introduced a statistical sensitivity measure for the MLP with differentiable activation functions. The sensitivity is deŽned as the ratio of the standard deviation of output errors to the standard deviation of weight or input error under the condition that the latter tends to zero. They derived a formula for the sensitivity of a single-output MLP under a given input pattern. However, these two approaches are useful only to measure the output deviation with respect to a given input pattern. They are not

MLP’s Sensitivity to Input Perturbation

185

able to measure the expected output deviation with respect to the overall input patterns, an important factor affecting the performance measurement of networks like MLPs. For our study, instead of using a hypersphere as described in Stevenson et al. (1990), Alippi et al. (1995), and Cheng and Yeung (1999), we employed a hyperrectangle as the mathematical model to represent an MLP’s input space because it usually falls into a continuous interval. Our approach is different from those of the other studies and offers certain advantages over them. For example, it does not demand that the input deviation be very small as the derivative approach requests, and the mathematical expectation used in our approach reects the network’s output deviation more directly and exactly than the variance does; for example, some assumptions must be made for the expectation, as in Choi and Choi (1992), PichÂe (1995), and Yeung and Sun (2002). Moreover, our result is applicable to the MLP that deals with inŽnite input patterns, which is an advantage of the MLP over other discrete feedforward networks like Madalines. In Zeng and Yeung (2001) the sensitivity for an ensemble of MLPs due to input and weight perturbations was investigated. But this article is mainly concerned with an individual MLP with Žxed weights and thus Žxed network architecture. We note that the sensitivity analysis of an ensemble of MLPs with respect to weight perturbations provides some useful design guidelines prior to the hardware implementation of an MLP, whereas the sensitivity measure of a trained MLP to input perturbations is directly related to its application performance. The arrangement of this article is as follows. The MLP model is briey described in section 2. The deŽnitions of sensitivity are given in section 3 for different granules in the MLP from neuron to layer and Žnally to the entire network. Sections 4 and 5 deal with the computation of the sensitivity, in which algorithms based on some analytical expressions are given for the computation of the sensitivity of a single neuron and the sensitivity of the entire MLP. Simulation results that support the theoretical results are presented in section 6. Section 7 concludes the article.

2 The Multilayer Perceptron Model 2.1 Notation. Generally, an MLP can have L layers, and each layer l (1 · l · L ) has nl ( nl ¸ 1) neurons. The form n 0 ¡n1 ¡, . . . , ¡nL is used to represent an MLP with a certain structural conŽguration, in which each nl (0 · l · L ) not only indicates the number of neurons (n0 is an exception, which indicates the dimension of input), but also stands for a layer from left to right, including the input layer. n0 stands for the input layer and nL for the output layer. Since the number of neurons in layer l ¡ 1 is equal to the output dimension of that layer, which is also equal to the input dimension of layer l, the input dimension of layer l is nl¡1 . For a neuron i (1 · i · nl )

186

X. Zeng and D. Yeung

in layer l, its input vector is Xl D ( xl1 , . . . , xlnl¡1 ) T , its weight vector is W il D (wli1 , . . . , wl l¡1 ) T , and its output is yli D f ( Xl ¤W il ) , where f (¢) is an activation in function having the summation of weighted inputs as its variable. For each layer l, all of its neurons have the same input vector Xl . Its weight set is W l D fW 1l , . . . , Wnl l g, and its output vector is Yl D ( yl1 , . . . , ylnl ) T . For the entire MLP, its input is the vector X1 or Y 0 , its weight is W D W 1 [ ¢ ¢ ¢ [ W L , and its output is YL . Let D Xl D ( D xl1 , . . . , D xlnl¡1 ) T and D Yl D (D yl1 , . . . , D ylnl ) T be the corresponding deviations for input and output, respectively. 2.2 Structure. Neurons, as cells, in an MLP are organized into layers by linking them together. Links exist only between neurons of two adjacent layers; there is no link between neurons in the same layer or in any two nonadjacent layers. All neurons in a layer are fully linked from all the neurons in the immediately preceding layer and are fully linked to all the neurons in the immediately succeeding layer. At each layer except the input layer, the inputs of each neuron are the outputs of the neurons in the previous layer, Xl D Yl¡1 ( l ¸ 1). The commonly used logistic function is chosen as the activation function, that is,

f ( x ) D 1 / (1 C e¡x ) ,

(2.1)

with the following characteristics: 0 · f ( x) · 1

(2.2)

f ( x ) C f (¡x) D 1

(2.3)

f ( x0 ) < f ( x00 ) if x0 < x00 .

(2.4)

3 DeŽnition of the Sensitivity

The main issue addressed by the sensitivity study is what the effects of input deviation will be on the MLP network’s output. For the case of a single neuron, the most direct and natural way to express the output deviation arising from input deviation is the difference of its deviated and nondeviated output,

D yli

D g ( Xl , D Xl , W il ) D f ( ( Xl C

D Xl ) ¤Wil ) ¡ f (Xl ¤Wil ).

(3.1)

Since Wil is Žxed, D yli can be easily computed if the values of Xl and D Xl are known. But in most cases, especially in the performance evaluation of a network, it would not be appropriate to compute D yli for a speciŽc input. Rather, it would be more desirable to Žnd the relationship between

MLP’s Sensitivity to Input Perturbation

187

the output deviation D yli and its cause, the input deviation D Xl , in an ensemble sense by taking overall inputs into consideration. It is obvious that equation 3.1 reects the relationship between D yli and D Xl . The problem is to deŽne such a sensitivity that keeps the relationship without reference to Xl . Our approach is to consider the mathematical expectation of D yli with respect to all possible values of Xl , that is, to treat Xl as a statistical variable. Furthermore, since input deviation is usually randomly oriented, it is proper to replace D Xl in equation 3.1 by its expected absolute value, denoted as D Xl D (D xl1 , D xl2 , . . . , D xlnl¡1 ) D ( E (| D xl1 |) , E (| D xl2 | ) , . . . , E (| D xlnl¡1 |) ) . The following deŽnitions use a bottom-up way to deŽne the sensitivity of different granules in the MLP. First is the sensitivity of a single neuron, then the sensitivity of a layer, and Žnally the sensitivity of the entire MLP. For any input vector Xl 2 [a1 , b1 ] £[a2 , b2 ] £¢ ¢ ¢ £[anl¡1 , bnl¡1 ] µ [0, and its corresponding expected deviation D Xl and a given weight vector l¡1 Wil 2 Rn , the sensitivity of neuron i in layer l is deŽned as the mathematical expectation of the absolute value of D yli with respect to Xl , which is expressed as DeŽnition 1. l¡1 1]n

sli D E (| g ( Xl ) |) D E (| f ( (Xl C D Xl )¤Wil ) ¡ f ( Xl ¤Wil ) | ).

(3.2)

Obviously, sli is a function of D Xl , independent of any speciŽc Xl . Because of equation 2.2, the interval of the input of all layers except the input layer is l¡1 in [0, 1]n (l ¸ 2), and one can normalize the input of the Žrst layer to be in l¡1 0 [0, 1]n . So the assumption Xl 2 [a1 , b1 ]£[a2 , b2 ]£¢ ¢ ¢£[anl¡1 , bnl¡1 ] µ [0, 1]n in the sensitivity deŽnition is practically acceptable. For any input vector Xl 2 [a1 , b1 ] £[a2 , b2 ] £¢ ¢ ¢ £[anl¡1 , bnl¡1 ] µ [0, 1]n and its corresponding expected deviation D Xl and a given weight vector l¡1 set W l D fWil | Wil 2 Rn I 1 · i · nl g, the sensitivity of layer l is deŽned as DeŽnition 2. l¡1

Sl D ( sl1 , . . . , slnl ) T .

(3.3)

D Xl (l > 1) in fact can be regarded as the sensitivity of the previous layer, that is, D Xl D S 0 D ( E (| D x11 |) , E (| D x12 |) , . . . , E (| D x1n0 |) ) when l D 1 and D Xl D Sl¡1 when l > 1. It is noted that Sl is a vector, which consists

of the sensitivities of all neurons in layer l. Since sensitivity is generally agreed to be a scalar rather than a vector, one may choose, for instance, norm(Sl ) , max(Sl ) , avg(Sl ) , or min(Sl ) as the representation of a layer ’s sensitivity depending on the nature of the application involved. One popular strategy is to choose the one with a smaller value if the application requires a higher precision.

188

X. Zeng and D. Yeung

The sensitivity of an MLP is the sensitivity of its output layer,

DeŽnition 3.

that is, S D SL D ( sL1 , . . . , sLnL ) T .

(3.4)

4 Computation of the Sensitivity for a Single Neuron

In this section, we derive formulas for the computation of equation 3.2, i.e., the sensitivity of a single neuron. For simplicity, the superscript that marks the neuron’s layer and the subscript that marks the neuron’s order in the layer are omitted because only single neurons are considered and their positions in the network will be irrelevant. Hence, s, X, D X, and W are used to replace sli , Xl , D Xl , and Wil in this and the next section. Clearly equation 3.2 can be expressed as Z sD

Z V

Q (X ) | f ( (X C D X ) ¤W) ¡ f ( X¤W ) | dV,

¢¢¢

(4.1)

where Q (X ) is the density function and V is the region [a1 , b1 ] £ ¢ ¢ ¢ £ [an , bn ] spanned by X. By assuming that the components of X are independent from each other and thus each can be regarded as a coordinate axis, equation 4.1 can be further expressed as Z sD

b1 a1

Z

b2 a2

Z ¢¢¢

bn an

Q (X ) | f ( (X C D X ) ¤W) ¡ f ( X¤W ) | dx1 dx2 , . . . , dxn .

(4.2)

Furthermore, by equation 2.4, namely, f ( x) being a monotonic increasing function, and the fact that Q ( X ) ¸ 0, the absolute value sign can be moved to the outside of the integral. Thus, 8Z b Z b Z bn 1 2 > > > ¢ ¢ ¢ Q (X ) f ( ( X C D X ) ¤W) dx1 dx2 , . . . , dxn > > > a1 a2 an > > Z b1 Z b2 Z bn > > > > ¡ ¢¢¢ Q (X ) f ( X¤W ) dx1 dx2 , . . . , dxn , if D X¤W ¸ 0 > > > a1 a2 an < Z b1 Z b2 Z bn sD > ¢ ¢ ¢ Q (X ) f (X¤W ) dx1 dx2 , . . . , dxn > > > a1 a2 an > > Z b1 Z b2 Z bn > > > > ¡ ¢ ¢ ¢ Q ( X ) f ( ( X C D X ) ¤W) dx1 dx2 , . . . , dxn , > > > a1 a2 an > > : if D X¤W < 0

MLP’s Sensitivity to Input Perturbation

189

­ Z b Z b2 Z bn ­ ­1 ¢¢¢ Q ( X ) f ( ( X C D X )¤W ) dx1 dx2 , . . . , dxn D ­ ­a1 a2 an ­ Z b1 Z b2 Z bn ­ ­ ¡ ¢¢¢ Q ( X ) f ( X¤W ) dx1 dx2 , . . . , dxn ­ . ­ a1 a2 an

(4.3)

(¢) Now replacing the inner P product by a summation and f by the logistic function and let s D jnD1 D xj wj , equation 4.3 becomes ­ 0 1 ­ Z Z Z bn n X ­b1 b2 ¢¢¢ Q (X ) f @ (xj C D xj ) wj A dx1 dx2 , . . . , dxn sD ­ ­ a1 a2 an ­ j D1 ­ 0 1 ­ Z b1 Z b2 Z bn n X ­ ¡ ¢¢¢ Q (X) f @ xj wj A dx1 dx2 , . . . , dxn ­ ­ a1 a2 an ­ j D1 ­ ¿³ ´ Z b Z b2 Z bn Pn ­ ¡ ­1 j D1 xj wj ¡ s ¢¢¢ Q (X ) 1Ce D ­ dx1 dx2 , . . . , dxn ­ a1 a2 an Z ¡

b1

Z

a1

b2 a2

Z ¢¢¢

¿³

bn

Q (X ) an

1Ce

¡

Pn

j D1 xj wj

´

­ ­ ­ . dx1 dx2 , . . . , dxn ­ ­

(4.4)

Therefore, the computation of the sensitivity is essentially reduced to the calculation of the following integral, Z I ( A, B, W, s) D

b1 a1

Z

b2 a2

Z ¢¢¢

bn an

¿³ ´ Pn ¡ xj wj ¡s D 1 j Q (X) 1 1Ce dx1 dx2 , . . . , dxn ,

(4.5)

where A D fa1 , a2 , . . . , an g and B D fb1 , b2 , . . . , bn g are sets of lower and upper bounds of the integral, and W D fw1 , w2 , . . . , wn g is the set of the 6 weights, with bj > aj and wj D 0 for 1 · j · n. The following algorithm could be used to compute the sensitivity for a single neuron: Algorithm: NEURON SENSITIVITY ( A, B, Q ( X) , D X, W ) 1. s D D X¤W;

190

X. Zeng and D. Yeung

Z 2. I ( A, B, W, s) ¼

Z

a1

Z 3. I ( A, B, W, 0) ¼

b1

a2

b1 Z a1

b2

b2 a2

Z ¢¢¢

Q (X ) an

Z ¢¢¢

¿³

bn

1Ce

¡

Pn

j D1 xj wj

¡s

´

dx1 dx2 , . . . , dxn ; ¿³ ´ Pn bn ¡ j D 1 xj wj Q (X) 1Ce

an

dx1 dx2 , . . . , dxn ;

4. s D |I ( A, B, W, s) ¡ I ( A, B, W, 0) | is the sensitivity for a single neuron. The following sections present the derivation of an approximate solution for equation 4.5 under the assumption that Q (X ) D 1 (X being uniformly distributed in the hyperrectangle), and an algorithm for its computation. 4.1 Approximate Solution for Integration of the Logistic Function.

Since no primitive function can be found for the integrand in equation 4.5, the Taylor series,

1 / (1 C x ) ¼

p X

(¡1) kx k,

(4.6)

kD 0

under the convergent condition |x| < 1, is employed to approximate the p( ) integrand. Let i (X ) denote the integrand Pn and i X denote its Taylor expan¡ j D 1 xj wj ¡ s for x in equation 4.6, the sion of p terms. By substituting e integrand could be approximately expressed as

i ( X ) ¼ ip ( X ) D

p X

(¡1) ke

¡k

Pn

j D1 xj wj

¡ ks

,

(4.7)

kD 0

Pn Pn ¡ j D 1 xj wj ¡ s | < 1, namely, where |e j D 1 xj wj C s > 0, is required. But Pn how to deal with i ( X ) when jD 1 xj wj C s · 0? It is quite clear that when P X satisŽes jnD1 xj wj C s < 0, ip (X ) will certainly be divergent. Fortunately, (¡ ) ( ) the logistic function’s characteristic, expression 2.3, P f x D 1 ¡ f x , can be used to solve this problem. When X satisŽes jnD 1 xj wj C s D 0, ip (X ) oscillates between 0 and 1, that is, ip ( X ) D 0 when p is an odd number, say p1 , and ip (X ) D 1 when p is an even number, say p2 . But at X it actually has i (X ) D 1 / 2. PThus, it is necessary to amend the imprecision arising from X satisfying jnD 1 xj wj C s D 0 in V. This can be done by calculating equation 4.5 twice, once with p1 and once with p2 , and then taking the average of the two

MLP’s Sensitivity to Input Perturbation

191

results. Now equation 4.5 can be rewritten as Z I¼

Z V

¢¢¢

ip ( X ) dV

8Z ! Z ÁX Pn p > ¡k xj wj ¡ ks > k D 1 j > (¡1) e ¢¢¢ dV, > > > V > kD 0 > > > n > X > > > if (4.8a) xj wj C s > 0 > > > > D 1 j > > ³Z ´ Z Z Z > > 1 > p1 ( ) p2 ( ) > > C ¢ ¢ ¢ ¢ ¢ ¢ i X dV i X dV , > > 2 > V V > < n X D if (4.8b) xj wj C s D 0 > > > jD 1 > > > > > > > > ! >Z > Z Á Pn p > X > xj wj C ks > ¢¢¢ k k D 1 j > (¡1) 1¡ e dV, > > > V > kD 0 > > > n > X > > > if xj wj C s < 0. (4.8c) > : jD 1

PnFrom a geometric point of view, the hyperplane P that is deŽned by j D 1 xj wj C s D 0 can be regarded as a dividing plane. It divides the ndimensional space into three parts: the points P on P, the points below P, and the points above P. The points on P satisfy jnD 1 xj wj C s D 0, and each of P P the other two parts satisŽes either jnD1 xj wj C s > 0 or jnD 1 xj wj C s < 0. Pn They are two opposite cases. If one satisŽes jD 1 xj wj C s > 0, the other P must satisfy jnD1 xj wj C s < 0, and vice versa. Now the problem is how to Žnd out whether P cuts V. If it does, how can equations 4.8a and 4.8c be computed? A solution to this problem is proposed below. The idea is to locate the intersection points of P and those parallel lines that are the extensions of the edges of V. Each line must have one and only one intersection point 6 with P under the assumption wj D 0 for 1 · j · n. Then by comparing the coordinates on a given axis of those intersection points with the lower bound and the upper bound of V on that axis, it is easy to identify the intersection situations between P and V. It is known that V has 2n¡1 edges parallel to a given coordinate axis. Take the xn -axis as an example. Each edge parallel to the xn -axis can be determined by assigning the n ¡ 1 coordinates ( x1 , . . . , xn¡1 ) with either xj D aj or xj D bj . Thus, the xn -coordinate for a

192

X. Zeng and D. Yeung

given line’s intersection point can be calculated by 0 xn D @

1 n¡1 X jD 1

xj wj C s A / wn

(4.9)

with its n¡1 Žxed coordinates. This will yield a set En D fen,1 , en,2 , . . . , en,2n¡1 g representing 2n¡1 edges. Note that during the calculation of En , an order is set to select the 2n¡1 edges, which are represented by the second subscript of the elements of E n . This means that the second subscript uniquely corresponds to an edge and thus its n ¡ 1 Žxed coordinates, denoting as (x1 , . . . , xn¡1 ) or fx1 , . . . , xn¡1 g for en,i (1 · i · 2n¡1 ) . This property will be used later. Let en,s D minfen,1 , en,2 , . . . , en,2n¡1 g and en,l D maxfen,1 , en,2 , . . . , en,2n¡1 g, and compare the four values en,s , en,l , an , and bn . In the following we present Žve possible cases together with their corresponding formulas for calculating equation 4.8. Only three-dimensional Žgures are used for the illustration. 4.1.1 Case 1: en,s ¸ bn or en,l · an . In this case as illustrated by Figure 1, P does not cutP V. Hence, in V, only one convergent condition of equation 4.8 is satisŽed. If jnD 1 xj wj C s > 0, then Z I¼

V

Z D D D

b1

Z

a1 p X

p X

¡k (¡1) k

!

Pn

j D 1 xj wj ¡ ks

e

kD 0 b2

a2

Z

¢¢¢ Z

(¡1) k

bn

p X

an b1

a1

kD 0

Z

(¡1) ke

kD 0 b2 a2

¢¢¢

Z

¡k

bn

e

Pn

¡k

j D1 xj wj

Pn

j D 1 xj wj

dV ¡ ks

dx1 dx2 , . . . , dxn

¡ ks

dx1 dx2 , . . . , dxn

an

(¡1) kI1 (¡k, s, A, B, W ).

(4.10)

kD 0

Otherwise,

I¼

¢¢¢

Z ÁX p

Pn

p X

j D 1 xj wj C

s < 0, and it follows that

(¡1) kC 1 I1 ( k, s, A, B, W ) .

(4.11)

kD 1

4.1.2 Case 2:en,s ¸ an and en,l · bn . In this case, as illustrated by Figure 2, P cuts V by the side and divides V into two parts, V1 and V2 . Supposing

MLP’s Sensitivity to Input Perturbation

193

Figure 1: P does not cut V.

Pn

j D 1 xj wj C

s < 0 in V1 and

Z I¼

V1

D

kD 0

¢¢¢

Z X p Z

V1

p X kD 0

p X kD 1

C

(¡1) ke

kD 0

(¡1) kC 1

kD 1

C

D

V2

p X

j D 1 xj wj C

s > 0 in V2 , it follows that

! Z Á Pn p X k xj wj C ks k D 1 j (¡1) e ¢¢¢ 1¡ dx1 dx2 , . . . , dxn

Z C

Pn

Z

¢¢¢

Z

e

¡k

k

Z

(¡1) k V2

¢¢¢

e

Pn

j D1 xj wj

¡ ks

j D1 xj wj C

ks

Pn

¡k

Pn

j D 1 xj wj

dx1 dx2 , . . . , dxn

dx1 dx2 , . . . , dxn

¡ ks

dx1 dx2 , . . . , dxn

(¡1) kC 1 |I2 ( k, s, A, B ¡ fbn g, W ) |

p X kD 0

(¡1) k |I2 (¡k, s, B, A ¡ fan g, W ) |,

Figure 2: P cuts the four parallel sides of V.

(4.12)

194

X. Zeng and D. Yeung

where Z

Z V1

¢¢¢

e

Z D

b1

k

Pn

j D 1 xj wj C

Z

a1

b2 a2

Z ¢¢¢

bn¡1

ks

±P n¡1

Z

an¡1

dx1 dx2 , . . . , dxn

¡

jD 1

an

£e

k

² xj wj C s / wn

Pn

j D 1 xj wj C

ks

dx1 dx2 , . . . , dxn , (4.13)

and Z V2

¢¢¢

Z X p

Z D

(¡1) ke

kD 0 b1 a1

Z

b2 a2

Z ¢¢¢

¡k

Pn

bn¡1 an¡1

j D1 xj wj

Z

bn

p X kD 0

C

² xj wj C s / wn Pn ¡k j D1 xj wj ¡ ks jD 1

£e

I¼

dx1 dx2 , . . . , dxn

±P n¡1

¡

Similarly, the formula for I with in V2 is

¡ ks

Pn

j D 1 xj wj C s

dx1 dx2 , . . . , dxn . (4.14)

> 0 in V1 and

Pn

j D 1 xj wj C s

< 0

(¡1) k |I2 (¡k, s, A, B ¡ fbn g, W ) |

p X kD 1

(¡1) kC 1 |I2 ( k, s, B, A ¡ fan g, W ) |.

(4.15)

4.1.3 Case 3: en,s D an and en,l > bn or en,s < an and en,l D bn . In this case, as illustrated by Figure 3 , P cuts a prism from V, and there is a special intersection point that is a vertex of V. We assume that the point C(c1 , c2 , . . . , cn ) is the lower vertex of V on the edge corresponding to en,s and the point D(d1 , d2 , . . . , dn ) is the upper vertex of V on the edge corresponding to en,l . Then either C or D is the special intersection point, and it must be true that cn D an and dn D bn . The other n ¡ 1 coordinates of C and D can be determined by en,s and en,l , as mentioned above. The coordinates of these two points are critical in setting the boundary for the integral in calculating equation 4.8. For simplicity of expression, they are also regarded as two sets: C D fc1 , . . . , cn g and D D fd1 , . . . , dn g. Referring to the results in section 4.2, three regions V1 , V2 , and V3 are used to replace V, as indicated in Figures 4, 5, and 6. It is obvious that V1 is suitable for equation 4.24 and V2 for 4.23. But V3 , as Figure 7 illustrates, needs to be extended

MLP’s Sensitivity to Input Perturbation

Figure 3: P cuts a prism from V.

Figure 4: The region of V1 .

Figure 5: The region of V2 .

Figure 6: The region of V3 .

Figure 7: The extended region of V3 .

195

196

X. Zeng and D. Yeung

with 2n¡1 ¡ 2 small prisms in order to be suitable for equation 4.24. In fact, this situation may be true for V1 too. In order to give a general solution, equation 4.24 is Žrst used to calculate the integral by regarding V1 and V3 as prisms, and then the results are adjusted by subtracting the integral in each small prism under the condition that the integrand expansion in that small prism is convergent. The integrand expansion in each of the small prisms will diverge if V1 and V3 do not need such a prism for the extension. Thus, according to whether cn D en,s or dn D en,l , and the satisfaction of the convergent conditions of equations 4.8a and 4.8c in V1 , V2 , and V3 , four groups of formulas for different conditions could P be derived. Below, the derivation for only one group is given: cn D en,s , jnD 1 xj wj C s > 0 in V1 and Pn j D 1 xj wj C s < 0 in V 2 and V3 . Those for the other three groups can be found in Zeng (2001).

Z I¼

V1

¢¢¢

D

p X

¡ D

C

¡

¡ ks

dx1 dx2 , . . . , dxn

kD 0

kD 0

Z

Z

(¡1) k V1

¢¢¢

p X kD 1

kD 0

e

¡k

Z

Pn

j D1 xj wj

Z

(¡1) kC 1 V2

kD 1

kD 0

j D 1 xj wj

V3

p X

p X

Pn

! Z Á Pn p X k xj wj C ks k D 1 j (¡1) e ¢¢¢ 1¡ dx1 dx2 , . . . , dxn

kD 0

C

¡k

V2

Z ¡

(¡1) ke

! Z Á Pn p X k xj wj C ks k D 1 j (¡1) e ¢¢¢ 1¡ dx1 dx2 , . . . , dxn

Z C

Z X p

¢¢¢

Z

e Z

(¡1) kC 1 V3

¢¢¢

e

k

k

¡ ks

dx1 dx2 , . . . , dxn

Pn

j D1 xj wj C

ks

Pn

j D1 xj wj C

ks

dx1 dx2 , . . . , dxn

dx1 dx2 , . . . , dxn

0

(¡1) k @ |I3 (¡k, s, C ¡ fcn g) [ fdn g, W ) | ¡

p X kD 1 p X kD 1

2n¡1 X

1 dh A

6 s&h D 6 l hD 1&hD

(¡1) kC 1 |I2 ( k, s, C, D ¡ fdn g, W ) | 0 (¡1) kC 1 @|I

3 ( k,

s, D, W ) | ¡

n¡1 2X

6 s&hD 6 l h D1&hD

1

eh A ,

(4.16)

MLP’s Sensitivity to Input Perturbation

where Z V1

Z ¢¢¢

e

¡k

Pn

j D 1 xj wj

¡ ks

197

dx1 dx2 , . . . , dxn

² ­±P n¡1 ­ Z ­ j D 2 cj wj C dn wn C s / w1 D­ ­ c1 ­ ± ² Z x1 w1 C Pn¡1 cj wj C dn wn C s / w2 j D3 £ c2 ± ² Z Pn¡2 xj wj C dn wn C s / wn¡1 Z dn j D1 ±P ¢¢¢ n¡1

² xj wj C s / wn ­ ­ Pn ­ ¡k j D 1 xj wj ¡ ks dx dx , . . . , dx ­ £e 1 2 n­ ­

cn¡1

¡ Z ¢¢¢

(4.17)

e

k

Pn

j D 1 xj wj C

ks

dx1 dx2 , . . . , dxn

± ² ­ ­ Z b1 Z b2 Z bn¡1 Z ¡ Pn¡1 xj wj C s / wn p X ­ D 1 j kC 1 ¢¢¢ D­ ­ (¡1) a1 a2 an¡1 an ­ kD 1 ­ ­ Pn ­ C k x w ks j j D 1 j £e dx1 dx2 , . . . , dxn ­ , ­ ­

Z

Z V3

dh ,

6 s&hD 6 l h D1&h D

Z V2

and

n¡1 2 X

j D1

¢¢¢

e

k

Pn

j D 1 xj wj C

ks

(4.18)

dx1 dx2 , . . . , dxn

­ ­ Z Z d2 ­d1 ± ² ± ² D­ P Pn ­ n C C C s ¡ s d w w x w d w ­¡ / / w2 1 1 1 j j j j j D2 jD 3 ±P ² n¡1 Z dn¡1 Z C s x w / wn j j jD 1 ² ¢ ¢ ¢ ±Pn¡2 ¡ j D 1 xj wj C dn wn C s / wn¡1 dn ­ ­ n¡1 Pn 2 X ­ C k x w ks j j j D1 £e ¡ eh . (4.19) dx1 dx2 , . . . , dxn ­ ­ ­ h D 1&hD6 s&hD6 l

198

X. Zeng and D. Yeung

Further, 8­±P ² n¡1 ­ Z > C dh wh C dn wn C s / w1 c w > j j ­ 6 D 2&j D j h > > ­ > ¢¢¢ > ­ > > ­c1 > > > ± ² > > Z Ph¡1 xj wj C Pn¡1 cj wj C dn wn C s / wh > > D1 Dh C 1 j j > > > ¢¢¢ > > > dh > ± ² > P > n¡2 < Z Z dn j D1 xj wj C dn wn C s / wn¡1 dh D ±P ² > n¡1 > C ¡ s c x w / wn > n¡1 j j jD 1 > > > ­ > > ­ Pn > > ­ > ¡k > j D 1 xj wj ¡ ks dx , . . . , dx ­ > £ e , 1 h > ­ > > ­ > > > > > > if it is convergent for large p > > : 0, if it is divergent for large p 8 |I3 (¡k, s, ( C ¡ fch , cn g) [ fdh , dn g, W ) | > > > > p > X > > > (¡1) k |I3 (¡k, s, ( C ¡ fch , cn g) [ fdh , dn g, W ) | if < D

kD 0

> > > > > > > > :

< 0,

p X kD 0

(¡1) k |I

3 (¡k,

s, (C ¡ fcn g) [ fdn g, W ) |

otherwise

and 8­ ­ Z > > ­d1 > > ­± ² ¢¢¢ > Pn > ­ > > ­¡ ch wh C jD2&jD6 h dj wj C s / w1 > > > Z c > h > > > ±P ² ¢¢¢ > P > h¡1 > ¡ > x1 w1 C jnDh C 1 dj wj C s / wh > D 1 j > > ± ² > > Z Pn¡1 xj wj C s / wn < Z dn¡1 jD 1 eh D ±P ² n¡2 > > C C ¡ s x w d w / wn¡1 dn > n n j j jD 1 > > > ­ > > ­ Pn > > ­ > > £ ek jD1 xj wj C ks dx1 , . . . , dxn ­ , > > ­ > > ­ > > > > > if it is convergent for large p > > > :0, if it is divergent for large p

(4.20)

MLP’s Sensitivity to Input Perturbation

199

Figure 8: V is divided into two smaller rectangles.

Figure 9: V is extended to a bigger rectangle with two extra rectangles.

D

8 |I3 ( k, s, (D ¡ fdh g) [ fch g, W ) |, > > > > p > X > > > (¡1) kC 1 |I3 ( k, s, ( D ¡ fdh g) [ fch g, W ) | if > < > > > > > > > > > :

k D1

< 0,

p X

(4.21)

(¡1) k C 1 |I3 ( k, s, D, W ) |

kD 1

otherwise.

In a similar way, the other three groups of formulas could be derived. 4.1.4 Case 4: en,s > an and en,l > bn or en,s < an and en,l < bn . In this case, P cuts V as Figure 8 illustrates. It can be reduced to cases 1 and 3 by cutting V with the hyperplane xn D en,s or xn D en,l into two smaller hyperrectangles. 4.1.5 Case 5: en,s < an and en,l > bn . This case, as Figure 9 illustrates, can be reduced to cases 2 and 3 by considering a bigger hyperrectangle Žrst, as case 2 does, and then dealing with two extra hyperrectangles on the top and bottom, as case 3 does. 4.2 Three Integral Formulas Used in the Derivations. Below are three integration formulas that are used in the derivations of equations 4.10 to

200

X. Zeng and D. Yeung

4.21. These formulas can be established by mathematical induction on n. It is assumed that k is an integer Pn and s is a real number. These formulas all k j D 1 xj wj C ks , but different integration regions. have the same integrand; e Each region represents a closed subregion within the hyperrectangle V. Integration Formula Z b1 Z I1 ( k, s, A, B, W ) D a1

D

b2 a2

Z ¢¢¢

bn

e

k

Pn

j D 1 xj wj C

ks

an

8Y n > > ( bj ¡ aj ) > > < jD 1

dx1 dx2 , . . . , dxn

if k D 0 (4.22)

n Y > > ks > (e kbj wj ¡ e kaj wj ) / ( kwj ) , > e :

6 0 if k D

j D1

The integration region of this formula is the entire V, which is enclosed by 2n planes perpendicular to each other, xj D aj and xj D bj (1 · j · n ) , as Figure 10 illustrates. Integration Formula Z I2 ( k, s, A, B ¡ fbn g, W ) D

b1 a1

Z

b2 a2

Pn

Z ¢¢¢

bn¡1

Z

±P

¡

n¡1 j D 1 xj wj C

² s / wn

an¡1

j D1 xj wj C

ks dx1 dx2 , . . . , dxn 10 1 8 00 n¡1 n¡1 > X Y > > > ¡ @@ wj (bj C aj ) / 2A C an wn C s A @ ( bj ¡ aj ) A / wn , > > > > jD 1 jD 1 > > > < if k D 0 0 1 (4.23) D > n¡1 n¡1 n > Y Y Y > > > (bj ¡ aj ) / ( kwn ) ¡ ekan wn C ks ( ekbj wj ¡ ekaj wj ) / @kn wj A , > > > > D 1 D 1 D 1 j j j > > : 6 0. if k D £e 1

k

The integration P region of this formula is the lower part of V cut by the hyperplane P, jnD 1 xj wj C s D 0, through the side parallel to an axis, say xn axis, as Figure 11 illustrates. This formula is also suitable for the upper part of V by adjusting the integration limits. Furthermore, if one is interested only in the scalar of the integral, the condition bj > aj can be removed because the lower limit and the upper limit for each variable in the integral can be arbitrarily exchanged without affecting the absolute value of the integral.

MLP’s Sensitivity to Input Perturbation

201

Figure 10: The entire V is the integration region.

The previous formula, 4.22, and the next formula, 4.24, also possess this characteristic. Integration Formula ± Z Pn I3 ( k, s, A, W ) D

j D 2 aj wj C

² ± ² P s / w1 Z x1 w1 jnD 3 aj wj C s / w2

a1

a

¢¢¢

2 ± ² ± ² Z Pn¡2 xj wj C an wn C s / wn¡1 Z Pn¡1 xj wj C s / wn jD 1 jD 1

an¡1 k

Pn

an

j D 1 xj wj C ks

£e dx1 dx2 , . . . , dxn 8 0 1n ,0 1 n n > X Y > > > @n! (¡1) n @ aj wj C s A wj A , > > > > j D1 jD 1 > > > 0 1l ,0 1 > > <X n¡1 n n X Y @kn¡l l! D (¡1) n C 1 @ aj wj C s A wj A > > > lD 1 j D1 j D1 > > 0 1 > > ³ ´, Pn > n > Y > > n C 1 1 ¡ ek j D 1 aj wj C ks @ kn > wj A , > C (¡1) : jD 1

if k D 0 (4.24)

6 0. if k D

The integration region of this formula is a prism part of V, as Figure 12 illustrates, which is enclosed by P with the n planes perpendicular to each other: xj D aj (1 · j · n ) . This formula is suitable for any prism part of a

Figure 11: The lower part of V is the integration region.

202

X. Zeng and D. Yeung

Figure 12: The prism part of V is the integration region.

hyperrectangle cut by a hyperplane. The hyperplane part in the prism is its base, and the coordinate ( a1 , a2 , . . . , an ) indicates the vertex not on the base. 4.3 Algorithm for the Computation of the Integral. Based on the above derivations, the following algorithm outlines the computation of equation 4.5 with Q (X ) D 1:

Algorithm: I ( A, B, W, s) P n¡1 with either 1. Calculate en,i D ¡( jn¡1 D 1 xj wj C s) / wn for 1 · i · 2 xj D aj or xj D bj ; 2. Find s and l satisfying en,s D minfen,1 , . . . , en,2n¡1 g and en,l D maxfen,1 , . . . , en,2n¡1 g; 3. Branch to the following four cases according to the values of en,s , en,l , an , and bn : case1. en,s ¸ bn or en,l · an : If

n X jD 1

If

n X jD 1

xj wj C s > 0 then I ¼ xj wj C s < 0 then I ¼

p X

(¡1) kI1 (¡k, s, A, B, W ) ;

kD 0 p X

(¡1) kC 1 I1 ( k, s, A, B, W ) ;

kD 1

case2. en,s ¸ an and en,l · bn : If

n X jD 1

I¼

xj wj C s < 0 in V1 and p X kD 1

C

n X j D1

xj wj C s > 0 in V2 then

(¡1) kC 1 |I2 ( k, s, A, B ¡ fbn g, W ) |

p X kD 0

(¡1) k |I2 (¡k, s, B, A ¡ fan g, W ) |I

MLP’s Sensitivity to Input Perturbation

If

n X j D1

I¼

xj wj C s > 0 in V1 and p X kD 0

C

n X jD 1

203

xj wj C s < 0 in V2 then

(¡1) k |I2 (¡k, s, A, B ¡ fbn g, W ) |

p X kD 1

(¡1) kC 1 |I2 ( k, s, B, A ¡ fan g, W ) |I

case3. en,s D an and en,l > bn or en,s < an and en,l D bn : Let C D fc1 , . . . , cn g D fx1 , . . . , xn¡1 gs [ fan g and D D fd1 , . . . , dn g D fx1 , . . . , xn¡1 gl [ fbn g; If cn D en,s ,

I¼

p X kD 0

C

¡ where

dh D

and

eh D

n X j D1

xj wj C s > 0 in V1 , and 0

n X jD 1

xj wj C s < 0 in V2 and V3 then

(¡1) k @|I3 (¡k, s, (C ¡ fcn g) [ fdn g, W ) | ¡

p X kD 1 p X

2n¡1 X 6 s&hD 6 l hD 1&hD

(¡1) kC 1 |I2 ( k, s, C, D ¡ fdn g, W ) | 0 (¡1) kC 1 @ |I3 ( k, s, D, W ) |

kD 1

2n¡1 X

1

eh A ,

6 s&h D 6 l hD 1&h D

8 |I3 (¡k, s, ( C ¡ fch , cn g) [ fdh , dn g, W ) |, > > > > p > X > > > (¡1) k |I3 (¡k, s, ( C ¡ fch , cn g) [ fdh , dn g, W ) | if > < > > > > > > > > > :

kD 0

< 0,

p X kD 0

(¡1) k |I3 (¡k, s, (C ¡ fcn g) [ fdn g, W ) |

otherwise

8 |I3 ( k, s, (D ¡ fdh g) [ fch g, W ) | > > > > p > X > > > (¡1) kC 1 |I3 ( k, s, ( D ¡ fdh g) [ fch g, W ) | if > < > > > > > > > > > :

k D1

< 0,

p X kD 1

(¡1) kC 1 |I3 ( k, s, D, W ) |

otherwise

1 dh A

204

X. Zeng and D. Yeung

If cn D en,s ,

I¼

p X kD 1

C

¡ where

j D1

xj wj C s < 0 in V1 , 0

n X jD 1

xj wj C s > 0 in V2 and V3 then

(¡1) kC 1 @|I3 ( k, s, (C ¡ fcn g) [ fdn g, W ) | ¡

p X kD 0 p X kD 0

2n¡1 X

1 dh A

6 s&h D 6 l hD 1&h D

(¡1) kI2 (¡k, s, C, D ¡ fdn g, W ) 0 (¡1) k @|I3 (¡k, s, D, W ) | ¡

1

2n¡1 X

eh A ,

6 s&h D 6 l hD 1&h D

8 |I3 ( k, s, ( C ¡ fch , cn g) [ fdh , dn g, W ) |, > > > > p > X > > > (¡1) kC 1 |I3 ( k, s, ( C ¡ fch , cn g) [ fdh , dn g, W ) | if > <

dh D

kD 1

> > > > > > > > > :

and

eh D

<

p X kD 1

0,

(¡1) kC 1 |I3 ( k, s, (C ¡ fcn g) [ fdn g, W ) |

otherwise

8 |I (¡ s, ( D ¡ fdh g) [ fch g, W ) |, > > 3 k, > > p > X > > > (¡1) k |I3 (¡k, s, (D ¡ fdh g) [ fch g, W ) | if > < > > > > > > > > > :

kD 0

<

p X kD 0

C

¡

p X

(¡1) k |I3 (¡k, s, D, W ) |

kD 0

0,

If dn D en,l ,

I¼

n X

otherwise n X jD 1

xj wj C s > 0 in V1 , 0

n X jD 1

xj wj C s < 0 in V2 and V3 then

(¡1) k @ |I3 (¡k, s, ( D ¡ fdn g) [ fcn g, W ) | ¡

p X kD 1 p X kD 1

2n¡1 X 6 s&h D 6 l hD 1&hD

(¡1) kC 1 |I2 ( k, s, D, C ¡ fcn g, W ) | 0 (¡1) kC 1 @|I

3 ( k,

s, C, W ) |

2n¡1 X 6 s&h D 6 l h D1&h D

1

eh A ,

1 dh A

MLP’s Sensitivity to Input Perturbation

205

where 8 |I3 (¡k, s, ( D ¡ fdh , dn g) [ fch , cn g, W ) |, > > > > p > X > > > (¡1) k |I3 (¡k, s, ( D ¡ fdh , dn g) [ fch , cn g, W ) | if > <

dh D

> > > > > > > > > :

kD 0

<

p X kD 0

0,

(¡1) k |I3 (¡k, s, (D ¡ fdn g) [ fcn g, W ) |

otherwise

and

eh D

8 ( ) |, |I ( > > 3 k, s, C ¡ fch g) [ fdh g, W > > p > X > > > (¡1) kC 1 |I3 ( k, s, ( C ¡ fch g) [ fdh g, W ) | if > < > > > > > > > > > :

k D1

p X kD 1

C

¡

(¡1) kC 1 |I3 ( k, s, C, W ) |

kD 1

0,

If dn D en,l ,

I¼

p X

<

otherwise n X j D1

xj wj C s < 0 in V1 , and

n X j D1

xj wj C s > 0 in V2 and V3 then

0 (¡1) kC 1 @|I3 ( k, s, ( D ¡ fdn g) [ fcn g, W ) | ¡

p X kD 0 p X kD 0

n¡1 2 X

6 s&hD 6 l h D 1&hD

(¡1) k |I2 (¡k, s, D, C ¡ fcn g, W ) | 0 (¡1) k @ |I3 (¡k, s, C, W ) | ¡

2n¡1 X

1

eh A ,

6 s&h D 6 l hD 1&hD

where

dh D

8 |I3 ( k, s, (D ¡ fdh , dn g) [ fch , cn g, W ) |, > > > > p > X > > > (¡1) kC 1 |I3 ( k, s, ( D ¡ fdh , dn g) [ fch , cn g, W ) | if > < > > > > > > > > > :

kD 1

< 0,

p X kD 1

(¡1) kC 1 |I3 ( k, s, ( D ¡ fdn g) [ fcn g, W ) |

otherwise

1 dh A

206

X. Zeng and D. Yeung

and

eh D

8 |I (¡ s, ( C ¡ fch g) [ fdh g, W ) |, > > 3 k, > > p > X > > > (¡1) k |I3 (¡k, s, ( C ¡ fch g) [ fdh g, W ) | if > < > > > > > > > > > :

kD 0

< 0,

p X

(¡1) k |I3 (¡k, s, C, W ) |

kD 0

otherwise

case4. en,s > an and en,l > bn or en,s < an and en,l < bn : Reduce to case1 and case3; case5. en,s < an and en,l > bn : Reduce to case2 and case3. 5 Computation of the Sensitivity for the Entire MLP

This section discusses the computation of the sensitivity for an entire MLP. From a global point of view, the sensitivity of an MLP reects its output deviation, or exactly its output layer ’s output deviation due to the Žrst layer’s input deviation. As the inputs of each neuron in a layer are the outputs of all neurons in its immediately preceding layer, the Žrst layer ’s input deviation will be propagated through all internal layers to inuence the output layer’s output. The sensitivity of the output layer can be computed neuron by neuron from the Žrst layer to the last layer. In more detail, S1 can be computed with D X1 and W 1 by NEURON SENSITIVITY(A, B, Q ( X1 ) , D X1 , Wi1 ) (1 · i · n1 ) , then S2 with D X2 D S1 and W 2 , and so on to, Žnally, SL with D XL D SL¡1 and WL . Logically, since the input space and the distribution of a neuron in layer l (2 · l · L ) may be different from the neurons in its previous layers, the input bounds A and B and the density function Q ( Xl ) for each layer l (2 · l · L ) need to be adjusted in the computation. An algorithm to compute the MLP’s sensitivity is given below. The expected input deviation D X1 , that is, S 0 , and the computed sensitivity Sl (1 · l · L ) will form a two-dimensional array. By controlling the mapping between the iterative time and the arrays’ subscript, it would be easier to compute the sensitivity for an MLP. Algorithm: MLP SENSITIVITY(A, B, Q ( X1 ) , D X1 , W ) 1. Put the given expected input deviation D X1 (i.e., S 0 ) and the weight set W into array forms; 2. Compute the sensitivity neuron by neuron and layer by layer: For l D 1 step 1 to L

MLP’s Sensitivity to Input Perturbation

207

If l > 1 then adjusting A, B, and Q ( Xl ) ; For i = 1 step 1 to nl Call NEURON SENSITIVITY(A, B, Q (Xl ) , Sl¡1 , Wil ) and save result sli ; 3. SL is the computed sensitivity for an MLP. In this algorithm, the adjustment of the input bounds for layer l (2 · l · L ) can be easily done with the bounds and the weight information of layer l ¡1 because of the linear feature of the summation of weighted inputs and the monotonic increasing feature of the sigmoid function. But the adjustment of the density function is generally very complex. A reasonable distribution function to be employed as an approximation is the uniform distribution. Other distribution functions would most likely make the integration of equation 4.5 virtually impossible. Our experimental results presented in section 6 support the use of uniform distribution as being a reasonably good one. 6 Experimental VeriŽcation

To verify the derived theoretical results, a number of experiments have been conducted. Again, a single neuron is considered Žrst. In the experiments, for simplicity and without losing the generality of the spatial dimension and direction, eight neurons with four different input dimensions from n D 2 to n D 5 and different weight values are used. Each neuron has either all positive weight values or separate positive and negative weight values. As for the case of the neurons with all negative weight values, it could be reduced to the case of all positive elements because of equation 2.3. Some weight values for these neurons are given in Table 1. Both theoretical computations and the computer simulations are based on the data given by this table. The theoretical results, that is, the sensitivities, for each neuron are computed with the algorithm NEURON SENSITIVITY(A, B, Q (X ) , D X, W ) with the parameters A D f0g, B D f1g, Q ( X ) D 1, and the elements of D X being all identical and ranging from 0 to 0.1 with a step size of 0.01. The simulation results, that is, the average output deviations for each neuron, are obtained with respect to more than 100 million inputs selected randomly and uniformly from the interval [0, 1]n (2 · n · 5) . Theoretical and simulation results are shown in Figures 13 through 16. It is clear that the agreement between the two results is very good. The experiments on the networks were carried out for three MLPs: 2 ¡2 ¡1, 2 ¡3 ¡1, and 2 ¡2 ¡2 ¡1. Their weight values are given in Table 2. Similar to the veriŽcation of individual neurons, the theoretical computations and the computer simulations were run with the given weight sets, and the expected input deviations ranged from 0 to 0.1 with a step size of 0.01. The sensitivity for each network is approximately computed by the algorithm MLP SENSITIVITY(A, B, Q ( X1 ) , D X1 , W ) with

208

X. Zeng and D. Yeung

Table 1: Weight Dimension and Weight Values for Neurons. Parameter

n

w(1)

w(2)

Neuron 1 Neuron 2 Neuron 3 Neuron 4 Neuron 5 Neuron 6 Neuron 7 Neuron 8

2 2 3 3 4 4 5 5

1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0

2.0 ¡2.0 2.0 ¡2.0 2.0 ¡2.0 2.0 ¡2.0

w(3)

3.0 3.0 3.0 3.0 3.0 3.0

w(4)

4.0 ¡4.0 4.0 ¡4.0

w(5)

5.0 5.0

Figure 13: Sensitivity of the neurons with two-dimensional weight to expected input deviation.

A D f0g, B D f1g, Q (X1 ) D 1. They are not adjusted during the computations. The average output deviations for each network are with respect to more than 100 million inputs selected randomly and uniformly from the interval [0, 1]2 . The theoretical and simulation results are shown in Figure 17. 7 Conclusion

We have proposed a quantiŽed measure and its computation for the sensitivity of the MLP to its input perturbation. The theoretical results obtained

MLP’s Sensitivity to Input Perturbation

209

Figure 14: Sensitivity of the neurons with three-dimensional weight to expected input deviation.

Figure 15: Sensitivity of the neurons with four-dimensional weight to expected input deviation.

210

X. Zeng and D. Yeung

Figure 16: Sensitivity of the neurons with Žve-dimensional weight to expected input deviation. Table 2: Three Sets of Weight Values.

W

2 ¡2 ¡1 2 ¡3 ¡1 2 ¡2 ¡2 ¡1

w(1I 1, 1) w(2I 1, 1) w(3I 1, 1)

w(1I 1, 2) w(2I 1, 2) w(3I 1, 2)

w(1I 2, 1) w(2I 1, 3)

w(1I 2, 2) w(2I 2, 1)

¡5.0 ¡1.0

6.0 2.0

¡3.0

4.0

1.0 7.0

¡2.0 ¡8.0

3.0 9.0

¡4.0

10.0 ¡5.0 ¡1.0

¡9.0 6.0 2.0

8.0 ¡3.0

¡7.0 4.0

w(1I 3, 1) w(2I 2, 2)

w(1I 3, 2)

5.0

¡6.0

are in reasonable agreement with the simulation results. The sensitivity measure is something like an analog rule that network users can use as a tool to evaluate their networks’ performance. For example, Lee and Oh (1994) indicated that in pattern classiŽcation applications, their sensitivity (misclassiŽcation probability) result could be applied to select the optimal weight set among many weight sets acquired through a number of learning

MLP’s Sensitivity to Input Perturbation

211

Figure 17: Sensitivity of the three MLPs to expected input deviation.

trials. Among these weight sets, an MLP with the optimal set will have the best generalization capability and the best input noise immunity. Zurada, Malinowski, and Usui (1997) made use of analytical sensitivity to delete redundant inputs to an MLP for pruning its architecture. Engelbrecht and Cloete (1999) also used analytical sensitivity to dynamically select training patterns. Some theoretical issues as well as applications of the MLP, such as improving error tolerance, measuring generalization ability, and pruning the network architecture, would beneŽt from this study. Acknowledgments

This research work is supported by a Hong Kong CERG 5114/97E. References Alippi, C., Piuri, V., & Sami, M. (1995). Sensitivity to errors in artiŽcial neural networks: A behavioral approach. IEEE Transactions on Circuits and Systems-I: Fundamental Theory and Applications, 42(6), 358–361. Cheng, A. Y., & Yeung, D. S. (1999). Sensitivity analysis of neocognitron. IEEE Transactions on System, Man, and Cybernetics—Part C:Applications and Reviews, 29(2), 238–249.

212

X. Zeng and D. Yeung

Choi, J. Y., & Choi, C. H. (1992).Sensitivity analysis of multilayer perceptron with differentiable activation functions. IEEE Transactions on Neural Networks, 3(1), 101–107. Engelbrecht, A. P., & Cloete, I. (1999). Incremental learning using sensitivity analysis. In Proceedings of IEEE IJCNN’99 (Vol. 2, pp. 1350–1355). Washington, DC. Engelbrecht, A. P., Fletcher, L., & Cloete, I. (1999). Variance analysis of sensitivity information for pruning multilayer feedforward neural networks. In Proceedings of IEEE IJCNN’99 (Vol. 3, pp. 1829–1833). Washington, DC. Fu, L., & Chen, T. (1993). Sensitivity analysis for input vector in multilayer feedforward neural networks. In Proceedings of IEEE International Conference on Neural Networks (Vol. 1, pp. 215–218). San Francisco. Hashem, S. (1992). Sensitivity analysis for feedforward artiŽcial neural networks with differentiable activation functions. In Proceedings of IJCNN’92 (Vol. 1, pp. 419–424). Baltimore, MD. Lee, Y., & Oh, S. H. (1994). Input noise immunity of multilayer perceptrons. ETRI Journal, 16(1), 35–43. PichÂe, S. W. (1995). The selection of weight accuracies for madalines. IEEE Transactions on Neural Networks, 6(2), 432–445. Stevenson, M., Winter, R., & Widrow, B. (1990). Sensitivity of feed forward neural networks to weight errors. IEEE Transactions on Neural Networks, 1(1), 71–80. Yeung, D. S., & Sun, X. (2002). Using function approximation to analyze the sensitivity of MLP with antisymmetric squashing activation function. IEEE Transactions on Neural Networks, 13(1), 34–44. Zeng, X. (2001). Output sensitivity of MLPs derived from statistical expectation. Unpublished doctoral dissertation, Hong Kong Polytechnic University. Zeng, X., & Yeung, D. S. (2001). Sensitivity analysis of multilayer perceptron to input and weight perturbations. IEEE Transactions on Neural Networks, 12(6), 1358–1366. Zurada, J. M., Malinowski, A., & Usui, S. (1997). Perturbation method for deleting redundant inputs of perceptron networks. Neurocomputing, 14, 177–193. Received July 24, 2001; accepted May 30, 2002.

LETTER

Communicated by Hagai Attias

Variational Mixture of Bayesian Independent Component Analyzers R. A. Choudrey [email protected] S. J. Roberts [email protected] University of Oxford, Robotics Research, Oxford, U.K. There has been growing interest in subspace data modeling over the past few years. Methods such as principal component analysis, factor analysis, and independent component analysis have gained in popularity and have found many applications in image modeling, signal processing, and data compression, to name just a few. As applications and computing power grow, more and more sophisticated analyses and meaningful representations are sought. Mixture modeling methods have been proposed for principal and factor analyzers that exploit local gaussian features in the subspace manifolds. Meaningful representations may be lost, however, if these local features are nongaussian or discontinuous. In this article, we propose extending the gaussian analyzers mixture model to an independent component analyzers mixture model. We employ recent developments in variational Bayesian inference and structure determination to construct a novel approach for modeling nongaussian, discontinuous manifolds. We automatically determine the local dimensionality of each manifold and use variational inference to calculate the optimum number of ICA components needed in our mixture model. We demonstrate our framework on complex synthetic data and illustrate its application to real data by decomposing functional magnetic resonance images into meaningful—and medically useful—features. 1 Introduction

The goal of pattern analysis and recognition is to extract information from some data. In order for this information to be useful, the distribution of data must be represented in some meaningful way. In many cases, insight may be gained by dividing the data into self-similar areas and analyzing each of these clusters under some informative framework, for example, using some understanding of the assumed data-generating process. One such method is to model the data as being produced by a mixture of data generators (also called analyzers), where each component generator is responsible for generating a particular cluster. The problems to overcome in this mixture Neural Computation 15, 213–252 (2003)

° c 2002 Massachusetts Institute of Technology

214

R. Choudrey and S. Roberts

modeling are to decide how many generators are needed, where to place them, and how to adjust them to best represent the data. Mixtures of gaussians (MoG) are widely used throughout the Želds of machine learning and statistics for data modeling, where each generator is a gaussian density. Despite their popularity, however, MoGs suffer from two serious drawbacks. The Žrst is that as the dimensionality S of the problem space increases, the size of each covariance matrix, S2 , becomes prohibitively large. This can be dealt with by assuming isotropic gaussians (i.e., ignoring the covariance structure), but this greatly reduces the exibility of the model class. This problem has been solved by Tipping and Bishop (1999), who replaced each gaussian with a probabilistic principal component analyzer (PCA). This allowed the dimensionality of each covariance to be effectively reduced while maintaining the richness of the model class. This was formulated under a maximum-likelihood framework, which, although efŽcient, does not allow one to infer the optimum number of generators needed. This mixture was modiŽed into a mixture of factor analyzers (FA) by Ghahramani and Beal (2000), where Bayesian inference was used to infer the optimum number of analyzers. The second problem with MoGs is that each component is a gaussian, a strong assumption that is often violated in many natural clustering problems (Lee, Lewicki, & Sejnowski, 2000). Although MoGs are capable of modeling any distribution given enough components, the problem still remains of automatically grouping gaussians that together describe some larger-scale feature. It is this second problem that we address in this article. A solution is reached by extending the mixtures of probabilistic PCA / FA model to an independent component analysis (ICA) mixture model. We improve on previous work on mixtures of ICA by Lee et al. (2000) and Penny and Roberts (2001) by incorporating a very exible ICA model that can generate arbitrary densities using MoGs and by bringing the formalism into the Bayesian arena. We use Bayesian inference to infer the optimum number of ICAs needed and to determine their ideal dimensionalities automatically. 1.1 ICA. Independent component analysis seeks to extract salient features and structure from data assumed to be a linear mixture of independent underlying (hidden) features

x D As,

(1.1)

where x is an observed S-dimensional data vector, s is the L-dimensional source vector, and A is the SxL bases or mixing matrix.1 An ICA model formulated in this way is well posed only if L · S; otherwise, there are effectively more unknowns than equations in the model. The overcomplete 1 The term mixing refers to the linear mixing in the ICA model; mixture refers to the stochastic generating method in a mixture model.

215

S o u rc e 2

S e n so r 3

Variational Mixture of Bayesian Independent Component Analyzers

Source 1

Sensor 2

Sensor 1

Figure 1: The underlying and observed data densities.

problem (L ¸ S) is an area of ongoing research and will not be dealt with here. Which of A or s represents the features depends on one’s view of the data. If the data are considered to be constructed from varying amounts of L static features, then the columns of A represent the L static basis vectors and s represents the amount of each basis used for a given data vector. For example, if the data represent an image, that image may be considered a mixture of underlying (maybe fewer) independent edges, textures, and so forth. The aim of ICA in this case is to “unmix” the data set and recover these representative features. If the data are time varying, they can be seen as a linear mixing of underlying (possibly lower-dimensional) signals represented by s. For example, the data may be signals received at S microphones at a cocktail party. These signals are then simply mixings of L people talking. ICA is used to blindly (i.e., without access to s and A) separate the sensor signals into the underlying source signals s. This is known as blind source separation (BSS) in the signal processing literature. In either case, ICA can be recast as data density modeling. The data density p ( x) is a linear transform (i.e., scaling, rotation and possibly higher-dimensional projection) of an unknown, nongaussian manifold—the source density p (s ) (see Figure 1). The goal of ICA is to recover the source density and estimate the transform matrix. ICA has traditionally been performed in the noiseless limit (Bell & Sejnowski, 1995; Lee, Girolami, Bell, & Sejnowski, 1998), with limited exibility in the source density model and with noise often being dealt with as an extra source. More recently, however, Attias (1999b) extended ICA by using a more exible source model and incorporating full covariance noise explicitly into the ICA framework. The model, which Attias dubbed a independent factor analysis (IFA), was subsequently learned through a maximum likelihood expectation-maximization (EM) algorithm. Bayesian

216

R. Choudrey and S. Roberts

formalisms have also been introduced, most notably (under a variational framework) by Attias (1999a), Lappalainen (1999), Choudrey, Penny, and Roberts (2000), and Miskin and MacKay (2000). For a comprehensive review of ICA, see Hyv¨arinen (1999) or Roberts and Everson (2001). 1.2 ICA Mixture Model. ICA was originally developed as a solution to the BSS problem in auditory signal analysis by HÂerault and Jutten (1986). Since then, the BSS approach has been extended to human electroencephalograms (EEGs) (Makeig, Bell, Jung, & Sejnowski, 1996), Žnancial data analysis (Back & Weigend, 1998), and telecommunications (Ristaniemi & Joutsensalo, 1999), among many other applications. Recently, however, ICA’s remit has broadened to one of representation in general. The way data are presented very much inuences what patterns are found and how much information can be extracted from them. A primary goal in pattern recognition, then, is to Žnd some intrinsic coordinate system from which structure is easier to extract. In this sense, ICA is a higherorder extension of second-order methods such as PCA and FA. PCA/FA Žnd representations of data that preserve the maximum amount of variance by effectively Žnding an intrinsic coordinate system for the covariance matrix. Projections of the data on to the bases found lead to decorrelation. ICA seeks a loftier goal: a representation in which the data are maximimally statistically independent. In this way, ICA aims to repackage data, which may contain structure across its constituent dimensions, into independent signals that contain structure only within components, thereby making patterns (hopefully) more cogent. This application of ICA has led to interesting work in representation of text corpora (Isbell & Viola, 1999; Kolenda, Hansen, & Sigurdsson, 2000), of images such as face representation (Bartlett, Lades, & Sejnowski, 1998), and of natural scenes (Bell & Sejnowski, 1997). This, in turn, has hinted at possible links between ICA and coding schemes in the primary visual cortex (Olshausen & Field, 1997; Lewicki & Olshausen, 1999), although we stress that this connection has generated some controversy (van Hateren & van der Schaaf, 1998). It is this representational viewpoint that motivates our extension of ICA into a mixture of ICAs. Decomposing and representing data using ICA assumes the whole data distribution is adequately described by one coordinate frame. This may not be appropriate for many problems, however. In the BSS problem, ICA assumes each data signal carries a constant mixture of source signals. Consider the scenario proposed by Lee, Lewicki, and Sejnowski (1999). There are two people talking to each other, but never at the same time, and there is music in the background. This cacophany is picked up by two microphones. At any one time, the microphones pick up a mixture of one voice and the music or the other voice and the music, but never all three at the same time. Clearly in this case, standard ICA is an inadequate model. As Lee et al. (1999) showed, a mixture of ICAs is a more appropriate generative model. More generally, if the sensor density in Figure 1 consists of

Variational Mixture of Bayesian Independent Component Analyzers

217

various self-similar, nongaussian manifolds, enforcing a single global representation is not appropriate and will produce a suboptimal representation. A more intuitive way of representing the data is the use of a number of local coordinate frames, each constructed under local conditions. This leads to a mixture of independent component analyzers. ICA mixture models were Žrst formulated by Lee et al. (1999) using the extended Infomax algorithm (Lee, Girolami, & Sejnowski, 1999) to switch the source model between subgaussian and supergaussian regimes. The model was learned through a combination of maximum likelihood and gradient ascent. Although well demonstrated, the source model could switch only between Laplacian and bimodal densities and thus lacked exibility. Penny and Roberts (2001) relaxed this constraint by using generalized exponential sources that can model a wide variety of kurtoses by the adjustment of a parameter. A basic method of model selection was also incorporated using the Bayesian information criterion (BIC) to infer the number of hidden sources (i.e., the intrinsic dimensionality of the local manifold). Due to the problem formulation used, only two-dimensional or higher manifolds could be modeled. Although more exible, the densities could only be unimodal, and the learning scheme was also maximum likelihood. In this article, we present an ICA mixture model trained using Bayesian methods. In line with Attias (1999b), Lappalainen (1999), Choudrey et al. (2000), and Miskin and MacKay (2000), we choose a fully adaptable factorial MoG as the source model for our ICA components, allowing us to recover arbitrary source densities. Essentially, each ICA component will model selfsimilar areas (the features) as a mixture of gaussian subfeatures. To overcome the heavy computational load associated with Bayesian learning, we use the variational framework reviewed by Jaakkola and Jordan (2000) to make assumptions about the posterior, giving tractability to the Bayesian model. The variational framework is a whole class of methods for bounding intractable integrals. In particular, we use free-energy minimization (Hinton & van Camp, 1993) or ensemble learning (MacKay, 1995a), a variational method for Bayesian parameter estimation. We further extend the ensemble learning results of Choudrey et al. (2000) and Miskin and MacKay (2000) by using an MoG posterior density in a similar fashion to Attias (1999b)—rather than a single gaussian—letting us capture much richer posteriors. This, in turn, leads to greater robustness under uncertainty (Choudrey & Roberts, 2001). The variational Bayesian method is carried through to the ICA mixture model, allowing model comparison, incorporation of prior knowledge of the mixture process, and—by integrating out nuisance parameters— control of model complexity and thus overŽtting. By monitoring the variational free energy of our models, we can compare model assumptions, allowing us, for example, to infer the optimum number of components needed in our ICA mixture model (Attias, 1999a). We also employ automatic relevance determination (ARD) (MacKay, 1995b) to suppress unsupported sources and thus effectively infer the number of latent dimensions of each

218

R. Choudrey and S. Roberts

ICA component as part of the learning process. This leads to the variational Bayesian independent component analyzers mixture model (vbICA-MM). 2 The Model

The probability of generating a data vector xn from a C-component mixture model given assumptions M is p ( xn | M ) D

C X

p ( c | M 0 ) p (xn | Mc , c ).

(2.1)

cD 1

A data vector is generated by choosing one of the C components stochastically under p (c | M0 ) and then drawing from p ( xn | Mc , c) . M D fM 0 , M1 , . . . , MC g is the vector of component model assumptions, Mc , and assumptions about the mixture process, M 0 . The assumptions represent everything that essentially deŽnes the model: values of Žxed parameters, model structure, details of the component switching method, any prior information, and so forth. p ( xn | M ) is known as the evidence for model M and quantiŽes the likelihood of the observed data under model M. The variable c indicates which component of the mixture model is chosen to generate a given data vector x. If p (c | M 0 ) is a vector of probabilities and each component p ( xn | Mc , c) is a gaussian, then equation 2.1 simply describes an MoG. If the MoG is adapted through a maximum likelihood approach, then M represents a list of point estimates for the corresponding parameters. In our mixture model, however, each component has a nongaussian density derived from the ICA model presented in the next section, and M represents assumptions concerning the distribution of possible parameter values. Figure 2 shows a generative model for a data vector x. 2.1 The ICA Model. In common with ICA in the literature, we choose a generative model to work with. The observed variables, x, of dimension S are modeled as a linear combination of statistically independent latent variables, sc , of dimension Lc with added gaussian noise,

x D A c sc C y c C ec

(2.2)

where y c is an S-dimensional bias vector, ec is S-dimensional additive noise, and c represents the cth ICA model. In signal processing nomenclature, S is the number of (observed) sensors, and Lc is the number of latent (hidden) sources. Equation 2.2 acts as a complete description for cluster c in the data density. The bias vector, y c , deŽnes the position of the cluster in the S-dimensional data space, Ac describes its orientation, and sc describes the underlying manifold. Using ICA, data produced by mixing gaussian sources cannot be

Variational Mixture of Bayesian Independent Component Analyzers

M1

M2

MC

ICA 1

ICA 2

ICA C

x

c

219

M0

Figure 2: ICA mixture model.

separated as they only have 2-moment distributions, so can never be more than decorrelated. The noise, ec , is assumed to be zero-mean gaussian and isotropic, p (ec | 0, L c , c) D N

( ec I 0, lc I ) ,

(2.3)

where lc I is the precision.2 The noise essentially absorbs any (spherical) gaussianity present in the cluster. The probability of observing data vector xn under component c is then given by ³ p ( x n | hc , c ) D

lc 2p

´S 2

exp[¡Ec ],

(2.4)

where hc D fAc , snc , ¸c g and

lc n ( x ¡ Ac snc ¡ y c ) T ( xn ¡ Ac snc ¡ y c ) . 2 Since the sources sc D fsc,1 , .., sc,i , .., sc,Lc g are by deŽnition mutually independent, the distribution over sc for data point n can be written as Ec D

p (snc | Msc , c) D

Lc Y

p ( snc,i | Msc, i , c) ,

(2.5)

iD 1

2 Conventional notation speciŽes covariance rather than precision. We have found precisions easier to manipulate in the math, so have abused the notation somewhat to make subsequent equations easier to read.

220

R. Choudrey and S. Roberts

where the product runs over the Lc sources of component c and Msc is the vector of source model assumptions. p (snc | Msc , c) is the source model for ICA component c. Traditionally a reciprocal-cosh (Bell & Sejnowski, 1995) or unimodal density (Miskin & MacKay, 2000) has been used. These, however, have limited exibility in kurtosis representation or capturing multiple modes. Methods have been proposed for switching between super- and subGaussian regimes (Girolami, 1998; Lee et al., 2000; Lee, Lewicki et al., 1999), although these depend on heuristic stability analyses. More recently, Attias (1999b) introduced an MoG source model into his ICA formalism, potentially allowing any distribution to be modeled. This is the route we take. 2.2 ICA Source Model. The choice of a exible and mathematically attractive (tractable) source model is crucial if a wide variety of source distributions are to be modeled; in particular, the source model should be capable of encompassing both super- and subgaussian distributions (distributions with positive and negative kurtosis respectively) and complex multimodal distributions. One such distribution is a factorized mixture of one-dimensional gaussians with Lc factors (i.e., sources) and mi components per source (see Figure 3),

p ( snc | ’c , c) D D

Lc X mi Y i D 1 qi D 1 Lc X mi Y i D 1 qi D 1

p ( qni D qi | ¼ i , c) p ( snc,i | Qc,i , c) p i,qi N

(snc,i I m i,qi , bi,qi ) ,

(2.6)

where, for brevity, the ICA component subscript c has been dropped from parameters that can be seen to belong to ICA c from context. From now on, all subscripted parameters should be assumed to belong to the cth ICA model unless otherwise stated. Equation 2.6 essentially describes the local features of cluster c: m i,qi is the position of feature qi with regard to the cluster center, b i,qi is its size, and p i,qi its prominance with regard to other features. The mixture proportions p i,qi D p ( qni D qi | ¼ i ) are the prior probabilities of choosing component qi of the ith source (of the cth ICA model). qni is a variable indicating which component of the ith source is chosen for generating snc,i and takes on values of fqi D 1, . . . , qi D mi g (where mi , of course, depends on ICA model c). The mean and precision of gaussian qi in source i are m i,qi and bi,qi respectively. The parameters of source i are Qc,i D f¼ c,i , ¹c,i , ¯ c,i g where boldface type indicates the vector of mi parameters. The complete parameter set of the source model is ’c D fQc,1 , Qc,2 , . . . , Qc,Lc g.

Variational Mixture of Bayesian Independent Component Analyzers

M s,1

M s,2

Gaussian1

221

M s,m

Gaussian2

Gaussianm

s

q

M s,0

Figure 3: MoG source model.

The complete collection of possible Qsource states is denoted q c D fq c,1 , q c,2 , . . . , q c,m g and runs over all m D i mi possible combinations of source states. The probability of state q nc being chosen and generating source vector snc is p (snc , q nc | ’c , c) D

Lc Y iD 1

p ( qni D qi | ¼ i , c ) p ( snc,i | qi , m i,qi , bi,qi , c)

(2.7)

D p ( q nc | ¼ c , c ) p ( snc | q nc , ’c , c ) ,

where ¼ c D f¼ c,1 , ¼ c,2 , . . . , ¼ c,Lc g (similarly for ¹ and ¯). Note that the product of Lc one-dimensional MoGs in equation 2.6 is equivalent to a single MoG in Lc -dimensional space with m states. By integrating and summing over the hidden variables, fsc , q c g, the likelihood of the IID data X D fx1 , x2 , . . . , xN g given the model parameters H c D fAc , y c , lc , ’c g can now be written as p (X | H c , c) D

N X m Z Y

p ( xn , snc , q nc | Hc , c) dsc ,

(2.8)

n D 1 qD 1

Q where dsc D i ds c,i . If we stipulate a form for p (c | M 0 ) in equation 2.1, p (X | M ) D

C X c D1

p ( c | ·) p ( X | H c , c ) ,

(2.9)

222

R. Choudrey and S. Roberts

where p (c | ·) D fp(c D 1) D k1 , p ( c D 2) D k2 , . . . , p (c D C) D kC g, then equations 2.4 through 2.5 and 2.6 can be substituted into equation 2.9 to yield a maximum likelihood model. This can then be learned through an iterative process such as gradient descent (Lee et al., 2000) or the EM algorithm (Penny & Roberts, 2001). In this article, however, we take the Bayesian route by also integrating out the parameters f·, H c g in equation 2.9. 3 Bayesian Inference and Variational Learning

The maximum likelihood approach to learning the parameters of a model is well documented (for an introduction, see Pearlmutter & Parra, 1996; Cardoso, 1997; and Attias, 1999b), as are the pitfalls (e.g., overŽtting, no quantiŽcation of uncertainty in the model or model comparison). We choose to take the Bayesian approach and integrate out the parameters f·, H c g and hidden variables fsc , q c g. This allows us to take our assumptions one stage higher; rather than assume point estimates for the parameters (very harsh and speciŽc), we assume prior distributions over all possible parameter values. This is much more exibile because it allows us to tailor the distributions to avoid overŽtting the data, reduce the parameter search space to reasonable values, and code our inevitable uncertainty in the parameters into the model. First, we will state the prior distributions over the hidden variables and model parameters. 3.1 The Priors. The priors are chosen to be appropriately conjugate to allow tractibility. All boldface parameters without sub- or superscripts indicate the collection of C parameter vectors. The prior over the ICA mixture indicator variables, c D fc1 , c2 , . . . , cN g, simply factorizes over the N data vectors:

p ( c | ·) D

N Y

k cn .

(3.1)

nD 1

The prior over the ICA mixture coefŽcients k is a Dirichlet p ( ·) D

P C Y C ( c0 ic0 ) ic ¡1 kc . C (ic ) c D1

(3.2)

Because of source independence, it follows that the distribution over the MoG component indicator variables, p (q c | ¼c , c) , is a product over all p i,qni where i indexes the sources and n the data: p ( q c | ¼ c , c) D

Lc N Y Y nD 1 iD 1

p i,qni .

(3.3)

Variational Mixture of Bayesian Independent Component Analyzers

223

For a given state q , the distribution over the sources is p ( sc | q c , ’ c , c ) D

N Y Lc Y

( snc,i I m i,qi , bi,qi ) .

N

nD 1 iD 1

(3.4)

The prior over the source model (MoG) parameters is a product of priors over ¼ c , ¹c , ¯c : p (’) D

C Y

p ( ¼ c ) p (¹c ) p ( ¯ c ).

(3.5)

cD 1

The prior over the mixture proportions, ¼ c , for the ith source is a Dirichlet with parameter ri,qi : p (¼ ) D

C Y Lc Y mi C ( Y

P q0i

ri,q0i )

c D 1 i D1 qi D 1

C (r i,qi )

r i,q ¡1

p i,qi i

.

(3.6)

The prior over each MoG mean, m i,qi , is a gaussian with mean mi0 and precision ti0 , and the prior over the associated precision, bi,qi , is a gamma with width and scale parameters bi0 and ci0, respectively: p (¹ ) D p(¯ ) D

Lc Y mi C Y Y cD 1 i D 1 q i D 1 Lc Y mi C Y Y cD 1 i D 1 q i D 1

(m i,qi I mi0 , ti0 )

N

G (b i,qi I bi0 , ci0 ) ,

(3.7)

(3.8)

where we deŽne the gamma distribution as

G ( wI b, c) D

± w² 1 wc¡1 exp ¡ . C ( c) bc b

(3.9)

The prior over the bias vector, y c D fy1 , y2 , . . . , yS g, is a product over S zero-mean gaussians with precision tyj : p (y ) D

C Y S Y cD 1 jD 1

N

( yc, j I 0, tyj ) .

(3.10)

The prior over the sensor noise precision, lc , is a gamma distribution with width and scale parameters blc and clc : p (¸ ) D

C Y cD 1

G (lc I blc , clc ) .

(3.11)

224

R. Choudrey and S. Roberts

The prior over each element of the mixing matrix, Aji , is a zero-mean gaussian with precision ai for each column: p ( A) D

Lc Y C Y S Y

N

(aji | 0, ai ) .

(3.12)

c D 1 i D1 j D1

By monitoring the precisions ®c D fa1 , . . . , aLc g, the relevance of each source may be automatically determined (ARD). If ai is large, column i of Ac will be close to zero, indicating source i is irrelevant. Finally, the prior over each ai is a gamma(bai , cai ): p (®) D

Lc C Y Y cD 1 i D 1

D G (ac,i I bai , cai ) .

(3.13)

Figure 4 shows the graphical model for the variational Bayes ICA (vbICA) model. Bayesian inference in such a model is computationally intensive and often intractable. An important and efŽcient tool in approximating posterior distributions is the variational method (see Jordan, Ghahramani, Jaakkola, & Saul, 1999, for an excellent tutorial). In particular, we take the variational Bayes approach detailed by Attias (1999a) and Jaakkola and Jordan (2000). 3.2 Variational Bayesian Learning. Consider the log evidence for

data X, log p ( X ) D log

p (X , W ) , p (W | X )

(3.14)

which simply follows from Bayes’ rule and where the explicit dependence on model assumptions has been dropped for brevity. The term W is the vector of all hidden variables and unknown parameters. Because the log evidence does not depend on W , this can be rewritten as Z

p0 ( W ) p ( X , W ) dW p0 ( W ) p (W | X ) Z Z p(X , W ) p0 ( W ) C p0 ( W ) log D p0 ( W ) log d W dW p0 ( W ) p (W | X )

log p ( X ) D

p0 ( W ) log

0 D F[W ] C KL[p kp],

(3.15)

where p0 ( W ) is some approximation to the posterior p ( W | X ) and F[W ] D hlog p ( X , W ) ip0 ( W ) C H[p0 ( W ) ] Z p0 ( W ) 0 KL[p kp] D p0 ( W ) log dW . p (W | X )

(3.16) (3.17)

Variational Mixture of Bayesian Independent Component Analyzers

m10 t10 r1,q

m1,1

1

m1,m

q1

p1

mL0tL0

b10 c10 b1,1

1

b1,m

1

s1

rL,q

L

pL

mL,1

225

bL0 cL0

mL,m

L

qL

bL,m

bL,1

L

sL

ba1 ca1 a11

a1

l

x1

l

x2

l

xS

bl cl

y1

bl cl

y2

bl cl

yS

ty1

ty2

tyS

Figure 4: Graphical model of vbICA. Circles represent random variables, and boxes denote parameters not governed by distributions (initially set by hand).

H [p0 (W ) ] is the entropy of p0 ( W ) . The Žrst term in equation 3.15, F, is known as the mean-Želd bound to the log evidence. Alternatively, ¡F is called the Helmholtz energy or variational free energy. The second term is the Kullback-Leibler divergence (KL divergence), a pseudodistance that measures the difference between two densities. This term is strictly positive, which means that F is a strict lower bound on the log evidence. By maximizing F, not only do we minimize the KL divergence between the approximating and true posterior, we also implicitly integrate out the unknowns W . By choosing an appropriate form for the approximation p0 ( W ), we perform tractable Bayesian learning. As F is a strict lower bound to the model (log) evidence, a wide variety of models and assumptions can be compared and contrasted by calculating F for each model. The higher F is, the higher the likelihood of the data under that model, and therefore the better that model is at explaining the data. 3.3 Variational Learning for vbICA-MM. In our model, W D fc, s, q , ·, £ g. By choosing p0 ( W ) such that it factorizes, terms in each hidden vari-

226

R. Choudrey and S. Roberts

able can be maximized individually. We choose the following factorization, p0 ( W ) D p0 ( c) p0 ( sc | q c , c) p0 ( q c | c) p0 (· ) p0 (y ) £ p0 (¸ ) p0 ( A ) p0 ( ® ) p0 ( ’ ) ,

(3.18)

where p0 ( ’ ) D p0 (¼ ) p0 (¹ ) p0 (¯ ) and p0 ( a | b) is the approximating density of p ( a | b, X ). We will also stipulate that the posteriors over the sources factorize such that p0 ( sc , q c | c) D

Lc Y

p0 (qi | c) p0 (sc,i | qi , c).

(3.19)

i D1

This additional factorization allows efŽcient scaling of computation with the number of hidden sources, with little loss of accuracy (Miskin & MacKay, 2000). The term p0 ( sc,i | qi , c) in equation 3.19 implies a mixture posterior source density for source i in ICA component c, p0 ( snc,i | c) D D

mi X qi D 1 mi X qi D 1

p0 (qni D qi | c) p0 (snc,i | qni , c) n cOi,q N i

n ) ( snc,i I mO ni,qi , bOi,q , i

(3.20)

(3.21)

where a posterior MoG density follows from from Bayes’ rule. By substituting p ( X , W ) and equation 3.18 into 3.16, we obtain expressions for the bound, F, to our model Ftot D Fmix C

C X

FICAc ,

(3.22)

cD 1

where Fmix D F[c, ·] FICAc D F[sc , q c , ¸c , Ac , ®c , ’c ].

(3.23)

The terms in equation 3.23 can be further factorized into contributions from each parameter. By monitoring subsets of F, we can infer the effect of local assumptions, for example, FICAc can be used instead of ARD to infer the most likely number of sources in ICA component c. One may now proceed by specifying functional forms of each of the approximating posteriors and using these in equation 3.16, as shown by Lappalainen (1999). As shown by MacKay (1995a), however, there is no

Variational Mixture of Bayesian Independent Component Analyzers

227

need to specify functional forms for (all) the approximating posteriors. The functional forms “fall out” of the maximization process, helped by the factorized form of p0 ( W ) and the conjugacy of the priors. The optimal form for each posterior is simply given by p0 (W k ) / p ( W k ) exp[hlog p (X , W ) iQ

lD 6 k

p0 ( Wl )

],

(3.24)

where the index k refers to the kth parameter in W . If p0 ( sc | q c , c) D p0 ( sc ), free-form optimization gives the ICA algorithms Žrst presented by Choudrey et al. (2000). This factorization gives a gaussian posterior over s. Using equation 3.18, however, is less restrictive and, as discussed below, more robust for an ICA mixture model. The posterior expectations of the data likelihood, equation 2.3, under the mixture density, equation 3.21, are given by hsnc,i | ci D hsnc,i 2 | ci D

mi X qi D 1 mi X qi D 1

p0 ( qni D qi | c)hsc,i n | qi , ci

(3.25)

2 p0 ( qni D qi | c)hsnc,i | qi , ci,

(3.26)

where n p0 (qni D qi | c) D cOi,q i

(3.27)

hsnc,i | qni , ci D mO ni,qi

(3.28)

2 hsnc,i | qni , ci D (mO ni,qi ) 2 C

1 . n O b i,q i

(3.29)

The advantages of an MoG posterior over the sources instead of a single gaussian (Choudrey et al., 2000; Miskin & MacKay, 2000) are twofold. First, it allows us to capture arbitrary densities, something not possible under a gaussian posterior. Second, an MoG posterior is more robust under uncertainty, especially in the context of an ICA mixture model. To understand this more clearly, consider the update equations for p0 ( snc,i | c) and p0 (snc,i | qni , c ) presented below (see the appendix for notation), mO ni /

mi X qi D 1

n cOi,q hbi,qi ihm i,qi i C gO cn hlc i i

mO ni,qi / hbi,qi ihm i,qi i C gO cn hlc i

S X jD 1

S X jD 1

n | ci¡hyj i) haji i(xjn ¡hxOj,k 6 i D

n | q nk , ci ¡ hyj i) , haji i(xjn ¡ hxOj,k 6 i D

(3.30)

(3.31)

228

R. Choudrey and S. Roberts

where the explicit dependence on c has been dropped for brevity. Equation 3.30 is the update under a gaussian source posterior (posterior gaussian mean), and equation 3.31 is the update under an MoG posterior (posterior mean for component qi ). The important feature to note is that the updates consist of prior (i.e., current) information from the MoG source model plus new data information. This information is combined and fed back up to the MoG source models. The MoGs then use this information to update their parameters. In uncertain situations (i.e., high noise), hlc i tends toward 0, drastically downweighting the data term. In the gaussian posterior case, the MoG source model gets fed a weighted average across MoG components of the current component parameters; this leads the MoG components to update their parameters toward common values. If there are few data or little data support, the MoGs will evolve toward the centroids of their respective densities rather than staying static. Over a number of iterations, they will effectively become the same single gaussian. If the source posterior is itself an MoG, then each component of the posterior MoG is responsible for its equivalent in the source prior MoG. In equation 3.31, as hlc i ! 0, separate information is fed back to each component qi —essentially current parameter values. In this case, if there are few data or little data support, the source MoGs will remain static. The difference is more acute in an ICA mixture model context. As well as the noise estimate, hlc i, the data term is also weighted by the current ICA’s responsibility for the current datum, gO cn . If ICA model c has little or no responsibility, its source parameters should remain static until it observes data deemed under its jurisdiction. Again, this is possible only under a mixture posterior distribution for the sources. 4 Implementing vbICA-MM

The measure F is maximized using the free-form approach of equation 3.24. All the derived posteriors require solving a set of coupled parameter update equations (see the appendix). In practice, this is best achieved by Žrst initializing the posterior component responsibilities (p0 ( c )), using these to initialize each ICA component, then beginning learning on each ICA component. These components are then used to calculate the new posterior responsibilities, and the learning process is repeated until convergence. These steps may be conveniently implemented in the algorithm shown in pseudocode form in Table 1. Once trained, the model can be used to reconstruct hidden source signals (to within a scaling and permutation) given a data set and the (now Žxed) model parameter distributions by calculating hci, hq c i, and hsc i under their respective posteriors. 4.1 Priors and Initialization. Broad priors have little or no effect on the results because these allow the data to “do the talking.” Such priors allow pa-

Variational Mixture of Bayesian Independent Component Analyzers

Table 1: Pseudocode for vbICA-MM Updates.

initialize; WHILE (D F (ica ¡mm ) < tolerance ) FOR every ICA component WHILE (D F (ica) < tolerance) update ICA observation model by cycling through equations A.12–A.17 until convergence; update ICA source model by cycling through equations A.18–A.24 until convergence; update ICA noise model using equations A.25–A.28; calculate Fnew ; (ica ) . |; calculate D F (ica ) D |Fnew ¡ Fold (ica ) (ica ) END WHILE; END FOR;

update ICA-MM indicator probabilities and parameters using equations A.29–A.30; calculate Fnew ; (ica¡mm ) . |; calculate D F (ica ¡mm ) D |Fnew ¡ Fold (ica ¡mm ) (ica ¡mm ) END WHILE;

229

230

R. Choudrey and S. Roberts

rameters and hidden variables to remain within sensible boundaries while encoding very little knowledge of what values they might take. Allowing the priors to become too broad, however, can reduce their regularizing abilities, leading to implausible magnitudes and dangers of overŽtting. Narrow priors, on the other hand, encode strong assumptions about possible parameter and variable values and have a discernible effect on the inference process. If the priors are too narrow, the model becomes inexible, and its ability to learn is compromised. For the ICA mixture model presented in this article, we examined a wide variety of priors (10¡6 · b · 106 , 10¡6 · c · 106 for all gamma distributions, scale parameter = 5–5000 for all Dirichlets, and precision = 10¡6 ¡ 106 for gaussians). For gamma distributions, the variance is given by b2 c and the mean by bc. Large values of c give distributions that are highly peaked around the mode, tending toward gaussianity. Values of c < 1 have no mode and are more like exponentials. Consequently, they encourage small values for the random variate while curtailing large values beyond the mean. We found 1 · b · 1000 with bc ¼ 1 a useful range for gamma distributions. Gamma distributions govern scale parameters, in this case, precisions. Values of bc > 10 have a narrowing effect on the posteriors, making them overconŽdent. Values of bc < 10¡1 with c > 1 have a smoothing effect on the posteriors, losing detail in the process. Similarly, we have found precisions of the order of 100 ¡ 10¡3 a good range. The value of Dirichlet scale parameters is as a pseudocounts, so high values are very constraining. Appropriate values depend on the number of data vectors. If N is the number of data vectors, then values between 0.01N and 0.1N stop components dying while allowing the data to set the posterior values accurately. If the algorithm was recast as an on-line scheme, these values would have to be reexamined because the results would necessarily become more sensitive to the choice of priors. The choice of initialization is also important. As with any other ICA formalism, the vbICA-MM algorithm Žnds a local maximum in the parameter space, and results depend on initialization. There are many ways to initialize; we recommend that readers explore the different options available to work out which is most suitable for them. We have investigated two initialization schemes: random and singular value decomposition (SVD). We initialize the model by Žrst running Ccomponent k-means on the data. The class responsibilities generated are used to partition the data into C segments, which are used to initialize C ICA models either randomly or by using SVD on each segment. If SVD is used, then the Žrst Lc columns of the SVD data matrix are used to initialize Lc MoGs using k-means, while residual singular values are used to estimate the noise. The Žrst Lc columns of the SVD projection matrix are used to initialize the mixing matrix. We have found random initialization the most useful overall, particularly for modeling clusters that are best described by nonorthogonal axes. Initializing using SVD (tantamount to initializing at the PCA solution)

Variational Mixture of Bayesian Independent Component Analyzers

231

yields a quicker result, but vbICA-MM can have problems with resolving directions in clusters exhibiting high correlation, often staying at or close to the PCA result. This is discussed further in section 6.1. One must also choose the number of components in each source MoG. Although this can be determined by monitoring the contribution to F by each source (F[Qc,i ]) as a function of components, this is time-consuming. We have found that three components are enough for most data sets, although Žve should be used for image data. 5 Results

We Žrst demonstrate the versatility of the vbICA-MM algorithm on two- and three- dimensional synthetic data, each grouped into three classes. The algorithm’s ability to model intricate, highly nongaussian data is highlighted, as is its structure determination using ARD and the mean-Želd bound, F, to the log evidence. We then use vbICA-MM on real functional magnetic resonance imaging (fMRI) data to decompose the images into interpretable, independent features and compare the results with previous methods. We set vague priors (b D 1000, c D 1e ¡ 3 for all gamma distributions, scale parameter = 5 for all Dirichlets and precision = 1e-3 for gaussians) to encode poor prior knowledge for both the synthetic and real data sets. 5.1 Test Data. We tested the vbICA-MM algorithm on two-dimensional and three-dimensional synthetic test data drawn from three classes and with 10% added gaussian noise. We set three components per source MoG and commenced learning using the algorithm presented in Table 1. Training continued until F changed by less than 0.01% or the number of iterations reached 200. Typically, iterations of each ICA component start high (approximately 100–200 for the nonorthogonal cluster presented below and less than 100 for the orthogonal clusters) and end with only a handful of iterations. In the 2D case, the underlying source manifolds were two-dimensional. Figure 5a shows the three clusters, with one gaussian cluster, one nongaussian cluster described by orthogonal directions, and one highly correlated cluster made up of six subclusters. For this data set, vbICA-MM was intialized randomly. Training took nine iterations of the vbICA-MM algorithm. Figure 5b shows the partition by vbICA-MM trained on 3000 points (1000 from each cluster) compared with the original data; only 12 (0.4%) data were assigned incorrectly. The Žrst deviations of the underlying MoG components are shown as ellipses. Note how the gaussian source is deemed one-dimensional, while the two-dimensional structure of the other classes has been clearly captured. A gaussian source is unidentiŽable from gaussian noise in ICA. Its principal direction is captured by the single source, while its spread is absorbed by the (gaussian) noise variance. The vbICA-MM data density model in Figure 5e shows the multimodal structure captured within

232

R. Choudrey and S. Roberts

the clusters. In comparison, an MoG (see Figure 5c) and the generalized exponential decorrelating ICA mixture model (gedecICA-MM) presented by Penny and Roberts (2001) (see Figure 5d) show no structure within the classes. Subsequently, vbICA-MM gives a more efŽcient representation of the true data density, with little probability mass wasted in regions of low density within each cluster. The data density of the 3D test data is shown in Figure 6a. The three clusters are intrinsically one-, two- and three-dimensional. We initialized a vbICA-MM using SVD and trained on 1500 points, 500 drawn from each cluster. Training took Žve iterations of the vbIVA-MM algorithm. Figure 6b depicts the model captured by vbICA-MM showing accurate representation of the clusters and cluster structures. The Hinton diagrams (Hinton, 1989) of the mixing matrices in Figures 6c through 6e show how vbICA-MM has used ARD to infer the latent dimensionalities correctly. Unsupported columns in the matrices have been suppressed by small ARD coefŽcients (inverse ai ’s shown in Figure 7a), effectively switching off unnecessary sources in the source reconstructions below. Another way of inferring the latent dimensionality of each ICA component is to monitor the free energy of the model. As implied by equation 3.23, this is equivalent to monitoring the contribution each ICA component makes to the overall free energy of the model. Figure 7b is a plot of the mean-Želd bound (i.e., the negative of the free energy) for each ICA component, FICAc . Each curve has had its maximum subtracted such that the trends in Figure 7b are more apparent. These curves conŽrm the latent dimensionality inferred by ARD. Plotting the overall mean-Želd bound, F, across components in Figure 7c correctly infers three clusters. There is a further peak at Žve components (and, indeed, progressively shallower ones at seven and nine) as ICAs latch on to different subclusters in the 3D cube. It must be noted, however, that F is a bound to the log evidence, so these further peaks disappear when exponentiated to yield the evidence bound. In comparison, a Bayesian MoG infers 21 clusters, while the BIC in gedecICA-MM predicts 9. Figure 8 shows how well the supported sources’ multimodal densities have been modeled by the MoGs. The ability of vbICA-MM to capture undulating manifolds is fundamental in correctly inferring the true number of clusters. As a comparison, Figure 7c also plots the Bayesian information criterion (BIC) approximation found to the log evidence using MAP estimates (BIC is the negative of maximum description length modeling; see Rissanen, 1994, for an introduction). Interestingly, this also picks out the correct model order, with a higher score for the log evidence. Attias (1999a) showed that the BIC is equivalent to variational Bayes in the inŽnite data limit. The BIC achieves a higher score, presumably because it is a much simpler model, with no model averaging, although we must admit surprise at the difference. We must point out, however, that Penny, Roberts, and Everson (2001) have suggested that penalized likelihood is more applicable to ICA than other models due to the likely violation of the independence assumption in overcomplex models,

Variational Mixture of Bayesian Independent Component Analyzers

(a) Data density

(b) vbICA-MM partition

(c) Gaussian mixture model

(d) gedec-ICA mixture model

233

(e) vbICA mixture model

Figure 5: Two-dimensional test data results. Contours plot 0.001 · p ( x ) · 1 in 250 intervals.

234

R. Choudrey and S. Roberts

(a) Data density

(c) 1D cluster

(b) vbICA-MM isosurface of p ( x) D 0.01

(d) 2D cluster

(e) 3D cluster

Figure 6: Three-dimensional test data results.

leading to a penalizing effect on the data likelihood itself. This suggests that a full variational integration is not necessary in the high data–low noise limit and that penalized likelihood MAP learning is adequate. With more noisy data, however, Penny et al. (2001) showed that model order selction using penalized likelihood quickly breaks down, so in uncertain situations, a full Bayesian approach would be more robust.

Variational Mixture of Bayesian Independent Component Analyzers

ARD Coeff.

1

0.5

0 1 2

3

S o u rce N u m b e r

2

3 1

IC A C om po n e nt

(a) ARD CoefŽcients 0 ­ 2 00

F(ICA )

­ 4 00 ­ 6 00 ­ 8 00

1d 2d 3d

­ 10 00 ­ 12 00 ­ 14 00

1

2

3

N um b e r of S o u rc es

(b) FICA for each ICA component 4 000 2 000

F

0 ­ 2 000 ­ 4 000 ­ 6 000 ­ 8 000

1

2

3

4

5

Number of Components

6

(c) Solid: vbICA-MM negative free-energy Dotted: BIC using MAP estimates

Figure 7: Structure determination using ARD and negative free energy.

235

236

R. Choudrey and S. Roberts

(a) ICA 1

(b) ICA 2

(c) ICA 3

Figure 8: (Top) Original source distributions. (Bottom) Learned distributions.

5.2 Real Data. We used the vbICA, variational Bayesian mixture of factor analyzers (vbMFA), gedecICA-MM, and vbICA-MM algorithms on fMRI images of a slice through a tumor patient’s brain. Data were collected using both T2 and proton density (PD) spin sequences, which are used directly to form a two-dimensional feature space. Two 100 £ 100 pixel images were

Variational Mixture of Bayesian Independent Component Analyzers

237

vectorized, and 2000 samples were randomly drawn from the subsequent 2D data. Models with a range of latent dimensions were trained on the 2000 data vectors, and the most likely models were then used to unmix the complete data set of 10,000 points into the corresponding independent features. Figure 9 shows the two original images used, along with the feature extracted by vbICA and the gedecICA mixture model. Due to the inherent limitations of ICA, no more than two features could be extracted from the fMRI images. The vbICA algorithm favored a two-source ICA model, shown in Figure 9b. The left-hand feature is simply a copy of one of the original images. The right-hand image has separated out a local feature: cerebrospinal uid. The vbMFA algorithm infers an eight-component model with F D ¡14813.75, giving the features in Figure 9c. The BIC of the gedecICA-MM chose a four-component model with two sources per ICA component, giving the eight overall features in Figure 9d. Although the gedecICA-MM has managed to separate out the cerebrospinal uid, most features are simply scaled copies of each other, and therefore the gedecICA-MM overrepresents the fMRI images. We trained one-component to Žve-component models using vbICA-MM, with each ICA component having the maximum two allowed sources. The negative free energy (F) plot in Figure 10a shows that a two-component model is preferred,3 while the Hinton plots in Figures 10b and 10c imply one-source and two-source ICA components. Compared to vbMFA, vbICAMM had F D ¡3618.8, a substantial improvement. The three features extracted by the most likely model are presented in Figure 11, along with their learned distributions. The single source of the Žrst ICA component, shown in Figure 11a, is global “background” brain tissue detail. The second ICA component represents more local features, where the central part of Figure 11b is, once again, the cerebrospinal uid. The two-tone background demarcates white and gray brain matter and has not been separated out, unlike some of the features shown in Figure 9. The reasons for this are not yet clear. More interesting, however, the second source has extracted two dark blobs. These are the tumors, which neither vbICA, nor vbMFA, nor gedecICA-MM picked out. The features’ respective MoGs can be interpreted as the distribution of gray-scale values in each feature. The tumors’ distribution is heavily peaked around darker values, with the left-hand tail capturing the lighter gray–level information. The other distributions similarly represent lighter to darker gray values from left to right. The ability of vbICA-MM to capture multimodal feature distributions is pivotal in allowing the complex background distribution to be separated from the rest. This representation is thus much more interpretable and efŽcient than either simple ICA or the gedecICA-MM.

3

The BIC preferred a one-component model.

238

R. Choudrey and S. Roberts

(a) Original data

(b) ICA features

(c) vbMFA features

(d) gedecICA-MM features

Figure 9: fMRI images and ICA, vbMFA and gedecICA-MM extracted features.

Variational Mixture of Bayesian Independent Component Analyzers

­ 33 50 ­ 34 00 ­ 34 50

F

­ 35 00 ­ 35 50 ­ 36 00 ­ 36 50 ­ 37 00 ­ 37 50 1

2

3

4

Num ber of Com ponents

(a) vbICA-MM negative free-energy

(b) 1D cluster

(c) 2D cluster

Figure 10: Inferred latent structure for fMRI images.

5

239

240

R. Choudrey and S. Roberts

(a) Background

(d) ICA 1

(b) C-S uid

(c) Tumors

(e) ICA 2

Figure 11: vbICA-MM features extracted from fMRI images and respective ICA component source distributions.

Figure 12 illustrates how the original data density has been modeled and partitioned by both gedecICA-MM and vbICA-MM. The lighter cluster in Figure 12d is described by the 1D ICA component, while the darker area is the 2D ICA cluster. vbICA-MM has managed to partition the data density into two clusters that are not linearly separable. 6 Discussion

In this article, we have presented an algorithm for modeling multidimensional, nongaussian data distributions. The vbICA-MM algorithm splits the distribution into self-similar areas and uses ICA components to learn representations of these areas. The ICAs model these local cluster manifolds by forming independent directions in the underlying distribution. Automatic relevance determination selects the appropriate dimensionality of each manifold, and a Bayesian learning scheme allows the optimum number of ICA components to be inferred. We have demonstrated the algorithm on two-dimensional and three-dimensional data by faithfully modeling intricate, discontinuous clusters while simultaneously inferring their intrinsic dimensions, something not possible under previous ICA mixture model. We

Variational Mixture of Bayesian Independent Component Analyzers

241

Figure 12: vbICA-MM model for fMRI images.

have shown how it is possible to compare assumptions underlying various models, in particular, using the negative free energy, F, to ascertain the correct number of clusters. We have used vbICA-MM to decompose real fMRI images into interpretable, self-similar features, blindly and automatically. 6.1 Problems to Overcome. Although we have comprehensively demonstrated the versatility and robustness of the vbICA-MM algorithm, there are practical issues to address, in particular, initialization and speed. As discussed in section 4.1, the choice of initialization can have an effect on the Žnal solution. SVD initialization always starts at the same place for

242

R. Choudrey and S. Roberts

a given data set and always ends at the same solution. In practice, we have found that different random initializations generally lead to the same solution as well. However, if parts of the data density are highly correlated, the ICA component most responsible for that area will stay close to the PCA solution if initialized using SVD. Different random initializations may lead to different solutions or may converge onto the PCA / FA solution. ICA can be seen as a generalization of principal component analysis and factor analysis. As shown by Attias (1999b), noiseless ICA with gaussian sources is equivalent to PCA, while FA is the same with nonisotropic noise. This means the PCA / FA solution is a subset of all possible ICA solutions for a given data set. As a particular cluster becomes more and more correlated (or, equivalently, its independent directions become less orthogonal), the second moment starts to dominate so the cluster becomes easier to describe by a gaussian density. This means the PCA / FA maxima in the ICA solution space become more prominent, so more and more initialization points will lead to them. The nonorthogonal cluster in Figure 5 was chosen because empirically, it seems to be the limit of correlation and nonorthogonality that vbICA-MM can resolve into independent directions. This particular cluster has a correlation of ¡0.67, and we needed on average Žve random initializations before vbICA-MM latched on, as indicated by the signiŽcantly higher value for F after one iteration for this initialization. More investigation is needed into initialization strategies to overcome this. One possible solution is to decorrelate, or whiten, each cluster identiŽed by the k-means step and then commence learning on these whitened data. We have had encouraging results using this admittedly rather ad hoc method. Also, the factorial approximation in equation 3.19 is no longer a good approximation for highly correlated data, so dropping this will also solve much of the problem, albeit with a corresponding reduction in speed. Speed is the other problem. Bayesian learning is inherently slower than non-Bayesian methods due to the extra parameters that have to be estimated. In fact, Bayesian inference can be shown to be NP-hard (Cooper, 1990). The variational approximation brings tractability to the Bayesian formulation and is speedier than the nondeterministic sampling regime, but it still remains understandably slower than its non-Bayesian counterparts. Interesting work by Valpola and Pajunen (2000) on speeding up Bayesian ICA may provide a basis for a faster vbICA-MM algorithm. Also, modelorder selection for the number of ICA components can become prohibitive if a large number are required. Some progress may be made in this area if, for example, some sort of birth-death (Roberts, Holmes, & Denison, 2001) and split-merge (Ueda, Nakano, Ghahramani, & Hinton, 1999) criteria could be enforced allowing active generating, culling, splitting, and merging of components during the learning process. This is an area of ongoing research. 6.2 Further Applications. Interesting data are, almost by deŽnition, not gaussian distributed. For data distributions that consist of self-similar non-

Variational Mixture of Bayesian Independent Component Analyzers

243

gaussian clusters, we believe this model will provide a better representation over current methods, such as gaussian mixture modeling, mixtures of factor analyzers, and plain-vanilla ICA. The kinds of data that can beneŽt from such an analysis range from the prosaic to the provocative. The most obvious area is in nonstationary blind source separation, as alluded to in section 1.2. Standard ICA is now the de facto method for blindly separating mixtures of signals. It has shown to be very robust, effective, and, in a probabilistic formulation, highly interpretable. However, ICA is essentially limited to stationary mixings of independent sources, which number fewer than the sensors picking up the observations. ICA mixture models are an important step in overcoming these restrictions. Sudden changes in the mixing amounts will be encoded by vbICA-MM as a switching between ICA components. In the cocktail party problem, for example, this would include different voices cutting in and out over a background cacophony. Provided the number of voices (plus background) active at any one time does not exceed the total number of sensors, vbICA-MM can capture more sources than sensors. This was demonstrated in the fMRI example, where three features were extracted from 2D data. Furthermore, the independence assumption at the core of ICA holds within components but is not necessary between components. Sources that are not independent will be represented as sources within different ICA components by vbICA-MM. The application of vbICA-MM to enhanced blind source separation does not have to stop at auditory signals. The multimodal nature of the source distributions lends itself readily to image separation and other BSS problems. Clustering in general is another area that naturally comes under the vbICA-MMs umbrella. Mixture modeling is the deŽnitive formalism for Žnding clusters in data distributions. Interesting clusters are very often not gaussian, so mixture models based on gaussianity will generally not model such clusters well. The synthetic clusters in section 5 were nongaussian and also contained structure within themselves. This confused both a gaussian mixture model and the mixture of factor analyzers model. The former did not model the data accurately, while the latter failed to infer the correct number of clusters. Lee et al. (2000) have shown that their ICA mixture model performed better at clustering the Iris data of Fisher (1936) than those based on gaussian methods. The grouping of documents into semantic clusters is currently a very active area of research (see Landaur, Foltz, & Laham, 1998, for an introduction). It is highly unlikely that such man-made corpora are intrinsically gaussian in nature. The reader may have noticed that in the use of ICA as a representative tool, we have verged on the evangelical. This, we strongly believe, is where ICA and its extensions will prove to be indispensable. Lee, Lewicki et al. (1999) have already shown that ICA mixture models based on Laplacian source densities are over 20% more efŽcient than PCA at coding natural scenes and images of newspaper text, and more efŽcient than standard

244

R. Choudrey and S. Roberts

ICA. As ICA is only beginning to leave its BSS beginnings behind, this area is as yet largely unexplored. 6.3 Future Directions. There are a number of modiŽcations that could be made to vbICA-MM to take into account further information that one may have on the data. For example, the source models could be restricted to being the same dimensionality for all ICA components if it is expected that the data lie on some dimensionally consistent manifold. Indeed, the source models could be shared among a number, if not all, of the ICA components if different regions of the data space are thought to be different projections of some handful of underlying manifolds. Temporal information may be introduced if the mixture process prior is conditioned across samples as in, for example, a Markov process, leading to a Bayesian formulation of hidden Markov ICA (Penny, Roberts, & Everson, 2000). vbICA-MM leads to a whole family of ICA mixture models depending on which assumptions are coded into the model. Because of the Bayesian formulation, the effectiveness of these models can be quantitatively compared. Again, this is an area of active research. Appendix: Update Equations

The parameter posteriors are given by p0 ( sc | q c , c) D p0 ( A ) D p0 ( ® ) D p0 ( q c | c ) D 0

p (¼ ) D p0 ( ¹ ) D p0 ( ¯ ) D

Lc N Y Y nD 1 i D1

N

n ) ( snc,i I mO ni,qi , bOi,q i

Lc Y C Y S Y

( aji I m O aji , aO ji )

N

c D1 i D 1 j D 1 Lc C Y Y c D1 i D 1 Lc N Y Y nD 1 i D1

(A.1)

(A.2)

G (ai I bOai , cOai )

(A.3)

n cOi,q i

(A.4)

Lc Y mi ( C Y Y

c D1 i D 1 qi D1 Lc Y mi C Y Y c D1 i D 1 qi D1

q i0

lO i,qi0 )

C ( lO i,qi )

c D1 i D 1 qi D1 Lc Y mi C Y Y

P

lO i,q ¡1

p i,qi i

(m i,qi I m O i,qi , tOi,qi )

N

G (b i,qi I bOi,qi , cOi,qi )

(A.5)

(A.6)

(A.7)

Variational Mixture of Bayesian Independent Component Analyzers

p0 ( y ) D p0 ( ¸ ) D p0 ( c ) D p0 ( · ) D

C Y S Y

( yj I mO yj , tOyj )

N

c D1 j D 1 C Y c D1

(A.8)

G (lcI bOlc , cOlc )

N Y C Y n D1 c D1 C Y ( c D1

(A.9)

gOcn

P

245

(A.10)

0 c0 Oic )

C (Oic )

k cOic ¡1 .

(A.11)

Updated distribution parameters are hatted versions of the original parameters, hai are expectations taken with regard to p0 ( a) , and ha | bi are expectations with regard to p0 (a | b) . A.1 Observation Model.

A.1.1 p0 ( sc | qc , c) .

mO ni,qi

2 S X 1 4hb i,qi ihm i,qi i C gO cn hlc i haji i D n bOi,q D1 j

i

3

n | q nk , ci ¡ hyj i) 5 £ ( xjn ¡ hxOj,k 6 i D

n bOi,q D hb i,qi i C gO cn hlc i i

S X jD 1

ha2ji i

where

n xOj,k 6 i D D

hxOj,nkD6 i | ci D hxOj,nkD6 i | q nk , ci D

Lc X

ajksnc, k

6 i kD

Lc X 6 i kD

Lc X 6 i kD

hajk ihsnc, k | ci hajk ihsnc, k | q nk , ci,

(A.12)

(A.13)

246

R. Choudrey and S. Roberts

and deŽne xOjn D hxOjn | ci D hxOjn | q ni , ci D

Lc X

aji snc,i

iD 1 Lc X iD 1 Lc X iD 1

haji ihsnc,i | ci haji ihsnc,i | q ni , ci.

The expectations for the sources s are given by the expressions in section 3.3. In practice, equation A.12 has to be iterated for every i a number of times 6 i. until mO ni,qi converges as it depends on every other k D Note the intuitive form of equations A.12 and A.13. Source i “sees” data xjn at sensor j and works out what it can “explain away” given information n in the rest of the model, that is, xOj,k . The residual information is then used 6 i D to update its own parameters in a bid to explain what is “left over.” This is analogous to the message-passing algorithms of Pearl (1988), whereby a n ) using messages node (sc,i ) updates its belief (encapsulated in mO ni,qi and bOi,q i passed to it by its parents (the MoG expectations), its children (the sensor data), and all other parents of its children (i.e., all other sources and quantiŽed by xOj, kD6 i ). This use of information from a variable’s Markov blanket runs through all the update equations. A.1.2 p0 (A) . mO aji D

N hlc i X gO n hsn | ci ( xjn ¡ hxOj,nkD6 i | ci ¡ hyj i) a O ji nD 1 c c,i

aO ji D hai i C hlc i

N X nD1

gO cn hsnc,i 2 | ci

(A.14)

(A.15)

In practice, equation A.14 has to be iterated for every i a number of times until mO aji converges. A.1.3 p0 ( ®c ) . 0

bOai

1¡1 S X 1 1 C ha2 iA D @ 2 jD 1 ji bai

cOai D cai C

S 2

(A.16)

(A.17)

Variational Mixture of Bayesian Independent Component Analyzers

247

A.2 Source Model.

A.2.1 p0 (qc | c) . Á n c i,q i

D pQ i,qi

bQi,qi n bOi,qi

!1 2

µ

1 n n2 (bO mO ¡ hbi,qi ihm 2i,qi i) exp 2 i,qi i,qi

¶

n c i,q i n P cOi,q D n i 0 c qi i,q0

(A.18)

(A.19)

i

where 2

0

pQ i,qi D exp 4Y (rOi,qi ) ¡ Y @

13 X q0i

rOi,q0i A5

bQi,qi D bOi,qi exp[Y ( cOi,qi ) ] and where equation A.19 ensures that function.

P

n qi cOi,qi

D 1. Y (.) is the digamma

A.2.2 p0 ( ¼ ) . rOi,qi D r i,qi C

N X nD 1

n cOi,q i

(A.20)

A.2.3 p0 ( ¹) . Á O i,qi D m

1 tOi,qi

ti0 mi0 C hbi,qi i

tOi,qi D ti0 C hbi,qi i

N X nD 1

N X nD 1

! n cOi,q hsnc,i i

|

qni ,

ci

n cOi,q . i

A.2.4 p0 ( ¯ ) . ³ ´ ¡1 Obi,q D 1 C 1 sQ i,q i 2 i bi0 cOi,qi D ci0 C

N 1X cO n , 2 nD1 i,qi

where we deŽne the average variance of component qi in source i sQ i,qi D

N X nD 1

n (h n 2 | n cOi,q sc,i qi , ci ¡ 2hm i,qi ihsnc,i | qni , ci C hm 2i,qi i). i

(A.21)

(A.22)

(A.23)

(A.24)

248

R. Choudrey and S. Roberts

A.3 Noise Model.

A.3.1 p0 (y) . mO yj D

N hlc i X gO n ( xn ¡ hxOjn | ci) tOyj nD 1 c j

tOyj D tyj C hlc i

N X nD 1

(A.25)

gO nc .

(A.26)

A.3.2 p0 (l) . 2

bOlc

3¡1 S X N 1 1X C gO n h(xn ¡ xOjn ¡ yj ) 2 i5 D 4 2 j D1 n D1 c j blc

cOlc D clc C

N SX gO n . 2 n D1 c

(A.27)

(A.28)

A.4 ICA Mixture Update.

A.4.1 p0 (c) . gcn

S

Q c2 D kQ c l

µ

¶ lc n 2 n ) exp h(xj ¡ xOj ¡ yj i 2 j D1

S Y

gn gO cn D PC c

n c D 1 gc

,

(A.29)

(A.30)

where "

kQ c D exp Y (Oic ) ¡ Y

Á X c0

!# Oic0

lQ c D bOlc exp[Y ( cOlc ) ], where equation A.30 ensures that

P

O cn cg

D 1.

A.4.2 p0 ( ·) . iOc D ic C

N X nD 1

gO cn

(A.31)

Variational Mixture of Bayesian Independent Component Analyzers

249

The relevent moments are given by hai D mean(a)

(A.32)

ha2 i D mean(a) 2 C variance(a) . Note that for a mixture with only one ICA component gcn D 1 for all n, and equations A.12 through A.24 reduce to the update equations for a single ICA model. Acknowledgments

We thank Iead Rezek, Peter Sykacek, and Evangelos Roussos for perceptive comments and discussions. Thanks to Mike Gibbs for solving computerrelated problems. Thanks also to the anonymous referees for very helpful and constructive criticism. References Attias, H. (1999a). Learning parameters and structure of latent variable models by variational Bayes. In Electronic Proceedings of the Fifteenth Annual Conference on Uncertainty in ArtiŽcial Intelligence (VAI-1999). Available on-line at: http://www2.sis.pitt.edu/»dsl/UAI/uai.html. Attias, H. (1999b). Independent factor analysis. Neural Computation, 11, 803–851. Back, A., & Weigend, A. (1998). A Žrst application of independent component analysis to extracting structure from stock returns. International Journal of Neural Systems, 8(4), 473–484. Bartlett, M., Lades, H., & Sejnowski, T. (1998). Independent components representations for face recognition. In Proceedings of the SPIE Symposium on Electronic Imaging: Science and Technology: Conference on Human Vision and Electronic Imaging III (pp. 528–539). San Jose, CA: SPIE. Bell, A., & Sejnowski, T. (1995). An information maximization approach to blind separation and blind deconvolution. Neural Computation, 7, 1129–1159. Bell, A., & Sejnowski, T. (1997).The “independent components” of natural scenes are edge Žlters. Vision Research, 37(23), 3327–3338. Cardoso, J.-F. (1997). Infomax and maximum liklihood for blind source separation. IEEE Letters on Signal Processing, 4, 112–114. Choudrey, R., Penny, W., & Roberts, S. (2000). An ensemble learning approach to independent component analysis. In Proceedings of Neural Networks for Signal Processing X. Sydney, 2000. Choudrey, R., & Roberts, S. (2001). Variational Bayesian independent component analysis with exible sources (Tech. Rep. No. PARG-01-05). Available on-line at: http://www.robots.ox.ac.uk/sjrob/pubs.html. Oxford: Oxford University. Cooper, G. (1990). Computational complexity of probabilistic inference using Bayesian belief networks. ArtiŽcial Intelligence, 42(3), 393–405. Fisher, R. (1936). The use of multiple measurements in taxonomic problem. Annals of Eugenics, 7(2), 179–188.

250

R. Choudrey and S. Roberts

Ghahramani, Z., & Beal, M. (2000). Variational inference for Bayesian mixtures of factor analysers. In S. A. Solla, T. K. Leen, & K.-R. Muller ¨ (Eds.), Advances in neural information processing systems, 12 (pp. 449–455). Cambridge, MA: MIT Press. Girolami, M. (1998). An alternative perspective on adaptive independent component analysis algorithms. Neural Computation, 10(8), 2103–2114. HÂerault, J., & Jutten, C. (1986). Space or time adaptive signal processing by neural models. In J. Denker (Ed.), Proceedings AIP Conference: Neural Networks for Computing (Vol. 151, pp. 206–211). Snowbird, UT: AIP. Hinton, G. (1989). Connectionist learning procedures. ArtiŽcial Intelligence, 40, 185–234. Hinton, G., & van Camp, D. (1993). Keeping neural networks simple by minimising the description length of the weights. In Proceedings of COLT-93. Santa Cruz, CA. Hyva¨ rinen, A. (1999). Survey on independent component analysis. Neural Computing Surveys, 2, 94–128. Available on-line at: http://www.icsi.berkeley. edu/jagota/NCS. Isbell, B., & Viola, P. (1999). Restructuring sparse high dimensional data for effective retrieval. In M. Kearns, S. Solla, & D. Cohn (Eds.), Advances in neural information processing systems, 11 (pp. 480–486). Cambridge, MA: MIT Press. Jaakkola, T., & Jordan, M. (2000). Bayesian parameter estimation via variational methods. Statistics and Computing, 10, 25–37. Jordan, M., Ghahramani, Z., Jaakkola, T., & Saul, L. (1999). An introduction to variational methods for graphical models. In M. Jordan (Ed.), Learning in graphical models. Cambridge, MA: MIT Press. Kolenda, T., Hansen, L., & Sigurdsson, S. (2000). Independent component analysis in text. In M. Girolami (Ed.), Advances in independent component analysis. Berlin: Springer-Verlag. Landaur, T., Foltz, P., & Laham, D. (1998). An introduction to latent semantic analysis. Discourse Processes, 25, 295–284. Lappalainen, H. (1999).Ensemble learning for independent component analysis. In Proceedings of the First International Workshop on Independent Component Analyis (pp. 7–12). Aussois, France. Lee, T., Girolami, M., Bell, A., & Sejnowski, T. (1998). A unifying informationtheoretic framework for independent component analysis. International Journal of Mathematical and Computer Modelling, 39, 1–21. Lee, T., Girolami, M., & Sejnowski, T. (1999). Independent component analysis using an extended infomax algorithm for mixed sub-gaussian and supergaussian sources. Neural Computation, 11(2), 409–433. Lee, T., Lewicki, M., & Sejnowski, T. (1999). ICA mixture models for unsupervised classiŽcation and automatic context switching. In International Workshop on Independent Component Analysis (pp. 209–214). Aussois, France. Lee, T., Lewicki, M., & Sejnowski, T. (2000). ICA mixture models for unsupervised classiŽcation of non-gaussian classes and automatic context switching

Variational Mixture of Bayesian Independent Component Analyzers

251

in blind signal separation. IEEE Transactions on Pattern Recognition and Machine Intelligence, 22(10), 1–12. Lewicki, M., & Olshausen, B. (1999). A probabilistic framework for the adaption and comparison of image codes. Journal of the Optical Society of America, 16(7), 1587–1601. MacKay, D. (1995a). Developments in probabilistic modelling with neural networks—ensemble learning. In Proceedings of the Third Annual Symposium on Neural Networks (pp. 191–198). Nijmegen, Netherlands: SpringerVerlag. MacKay, D. (1995b). Probable networks and plausible predictions—a review of practical Bayesian methods for supervised neural networks. Network: Computation in Neural Systems, 6, 469–505. Makeig, S., Bell, A., Jung, T.-P., & Sejnowski, T. (1996). Independent component analysis for electroencephalographic data. In D. Touretzky, M. Mozer, & M. Hasselmo (Eds.), Advances in neural information processing systems, 8 (pp. 145–151). Cambridge, MA: MIT Press. Miskin, J., & MacKay, D. (2000). Application of ensemble learning to infra-red imaging. In Proceedings of ICA2000. Helsinki, Finland. Olshausen, B., & Field, D. (1997). Sparse coding with an overcomplete basis set: A strategy employed by V1? Vision Research, 37, 3311–3325. Pearl, J. (1988). Probabilistic reasoning in intelligent systems (2nd ed.). San Mateo, CA: Morgan Kaufmann. Pearlmutter, B., & Parra, L. (1996). A context-sensitive generalization of ICA. In ICONIP ’96 (pp. 151–157). Berlin: Springer-Verlag. Penny, W., & Roberts, S. (2001). Mixtures of independent component analysers. In ArtiŽcial Neural Networks—ICANN2001 (pp. 527–534). Vienna, Austria. Penny, W., Roberts, S., & Everson, R. (2000). Hidden Markov independent components analysis. In M. Girolami (Ed.), Advances in independent component analysis. Norwell, MA: Kluwer. Penny, W., Roberts, S., & Everson, R. (2001). ICA: Model order selection and dynamic source models. In S. Roberts & R. Everson (Eds.), Independent component analysis: Principles and practice (pp. 299–314). Cambridge: Cambridge University Press. Rissanen, J. (1994). MDL modeling—an introduction. In P. Grassberger & J.P. Nadal (Eds.), From statistical physics to statistical inference and back (pp. 95– 104). Norwell, MA: Kluwer. Ristaniemi, T., & Joutsensalo, J. (1999). On the performance of blind source separation in CDMA downlink. In Proceedings of ICA ’99 (pp. 437–442). Aussois, France. Roberts, S., & Everson, R. (Eds.). (2001).Independent component analysis:Principles and practice. Cambridge: Cambridge University Press. Roberts, S., Holmes, C., & Denison, D. (2001). Minimum entropy data partitioning using reversible jump Markov chain Monte Carlo. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(8), 909–915. Tipping, M., & Bishop, C. (1999). Mixtures of probabilistic principal component analyzers. Neural Computation, 11(2), 443–482.

252

R. Choudrey and S. Roberts

Ueda, N., Nakano, R., Ghahramani, Z., & Hinton, G. (1999). SMEM algorithm for mixture models. In M. S. Kearns, S. Solla, & D. Cohn (Eds.), Advances in neural information processing systems, 11. Cambridge, MA: MIT Press. Valpola, H., & Pajunen, P. (2000). Fast algorithms for Bayesian independent component analysis. In Proceedings of ICA2000 (pp. 233–237). Helsinki, Finland. van Hateren, J., & van der Schaaf, A. (1998). Independent component Žlters of natural images compared with simple cells in primary visual cortex. Proceedings of the Royal Society of London, Series B, 265, 359–366. Received October 15, 2001; accepted July 8, 2002.

Communicated by Paul Bressloff

LETTER

Interspike Interval Correlations, Memory, Adaptation, and Refractoriness in a Leaky Integrate-and-Fire Model with Threshold Fatigue Maurice J. Chacron [email protected] Department of Physics, University of Ottawa, Ottawa, Canada K1N 6N5

Khashayar Pakdaman [email protected] Inserm, U444, 75571 Paris Cedex 21, France

AndrÂe Longtin [email protected] Department of Physics, University of Ottawa, Ottawa, Canada K1N 6N5

Neuronal adaptation as well as interdischarge interval correlations have been shown to be functionally important properties of physiological neurons. We explore the dynamics of a modiŽed leaky integrate-and-Žre (LIF) neuron, referred to as the LIF with threshold fatigue, and show that it reproduces these properties. In this model, the postdischarge threshold reset depends on the preceding sequence of discharge times. We show that in response to various classes of stimuli, namely, constant currents, step currents, white gaussian noise, and sinusoidal currents, the model exhibits new behavior compared with the standard LIF neuron. More precisely, (1) step currents lead to adaptation, that is, a progressive decrease of the discharge rate following the stimulus onset, while in the standard LIF, no such patterns are possible; (2) a saturation in the Žring rate occurs in certain regimes, a behavior not seen in the LIF neuron; (3) interspike intervals of the noise-driven modiŽed LIF under constant current are correlated in a way reminiscent of experimental observations, while those of the standard LIF are independent of one another; (4) the magnitude of the correlation coefŽcients decreases as a function of noise intensity; and (5) the dynamics of the sinusoidally forced modiŽed LIF are described by iterates of an annulus map, an extension to the circle map dynamics displayed by the LIF model. Under certain conditions, this map can give rise to sensitivity to initial conditions and thus chaotic behavior.

Neural Computation 15, 253–278 (2003)

° c 2002 Massachusetts Institute of Technology

254

M. Chacron, K. Pakdaman, and A. Longtin

1 Introduction Since the seminal work of Adrian and Zotterman (1926), adaptation has been demonstrated to be a key property of neurons. For example, photoreceptor neurons have to cope with very different light intensities (e.g., day and night) and must adapt to each (Kufer, Fitzhugh, & Barlow, 1957). Moreover, this adaptation can sometimes give rise to interspike interval (ISI) correlations in experimental recordings (Kufer et al., 1957; Goldberg, Adrian, & Smith, 1964; Yamamoto & Nakahama, 1983; Teich & Lowen, 1994; Sch¨afer, Braun, Peters, & Bretschneider, 1995; Longtin & Racicot, 1998; Chacron, Longtin, St.Hilaire, & Maler, 2000; Neiman & Russell, 2001; Liu & Wang, 2001; Brandman & Nelson, 2002). Information-theoretic approaches have recently shown that neural information transfer can either increase or decrease as a result of correlations between spikes (Abbott & Dayan, 1999; Panzeri, Petersen, Schultz, Lebedev, & Diamond, 2001; Tiesinga, Fellous, JosÂe, & Sejnowski, 2002) or ISIs (Chacron, Longtin, & Maler, 2001a). Also, correlations between neuronal inputs can increase or decrease information transfer (Abbott & Dayan, 1999; Panzeri et al., 2001). One prototype sensory system in which the role of ISI correlations has been addressed is the electrosensory system of weakly electric Žsh. Negative ISI correlations have been found in P-type electroreceptors (Longtin & Racicot, 1998). A modeling study of these receptors has further shown that these correlations enhance not only information transfer but signal detection as well (Chacron et al., 2001a). It is clear from those studies that ISI correlations depend on various biophysical model parameters, as well as on noise and periodic forcing. The goal of this article is to provide a foundation for understanding how ISI correlations arise from the interplay of these factors in the context of a simple model. The leaky integrate-and-Žre model (LIF) is one of the most elementary spiking models and has been widely used to gain a better understanding of information processing in neurons (see, e.g., Tuckwell, 1988). One feature that makes this model particularly suitable for theoretical analysis is that it is analytically tractable and retains two key neuronal properties of type I membranes: the all-or-none response and the postdischarge refractoriness. The tractability of the standard LIF is in part due to the fact that following a discharge, the dynamics of the LIF depends on only the present and future inputs. In other words, after each Žring, the LIF has no memory of the past: inputs and discharge times prior to the last one bear no inuence on the next discharge times. This property has been a key ingredient in the analysis of the response of the standard LIF to inputs such as sinusoidal current or white gaussian noise. Indeed, it allows the standard LIF to be described by iterates of orientation-preserving circle maps (Rescigno, Stein, Purple, & Popele, 1970; Keener, Hoppensteadt, & Rinzel, 1981; Coombes & Bressloff, 1999; Pakdaman, 2001). Moreover, the discharge times evoked by

Memory and Adaptation in a Model Neuron

255

gaussian white noise stimulation form a renewal process (for recent reviews on the noisy LIF, see Lansky & Sato, 1999; Ricciardi, Di Crescenzo, Giornio, & Nobile, 1999; Pakdaman, Tanabe, & Shimokawa, 2001). The lack of memory about past discharges, which constitutes an advantage in the theoretical analysis of the standard LIF, is also one of the shortcomings of this model. Indeed, this property can eliminate the correlation between successive interspike intervals (ISIs) of the LIF. For instance, the ISIs evoked by additive gaussian white noise stimulation of this model are independent from one another. As mentioned, this is not the case experimentally for many neurons. Moreover, the LIF model assumes that the threshold to Žring is constant in time. There is experimental evidence that the threshold for Žring is not constant but depends on the past spiking history of the neuron (Azouz & Gray, 1999). Thus, a physiologically plausible modiŽcation to the LIF neuron would be to make the threshold a dynamical variable also. In this article, we study a modiŽed LIF neuron in which the threshold is a dynamical variable that models threshold fatigue after an action potential. Using step currents as stimuli, we show that the model can give rise to adaptation (i.e., a gradual change in Žring rate), a feature that is absent from the standard LIF model. Under certain conditions, we show that the threshold fatigue can lead to a saturation in the Žring rate of the neuron as the depolarizing current is increased. This feature is also absent from the LIF model, where the Žring rate diverges as a function of input current. We then show that the modiŽed LIF incorporating threshold fatigue can reproduce ISI correlations similar to those found in experimental recordings. The choice of the threshold dynamics is guided by previous observations that ISI correlations can result from the progressive accumulation of refractory effects (Stein, 1965; Weiss, 1966; Geisler & Goldberg, 1966; Holden, 1976; Gerstner & van Hemmen, 1992; Chacron et al., 2000, 2001a; Chacron, Longtin, & Maler, 2001b; Liu & Wang, 2001). If the neuron Žres two spikes in a relatively short time interval, then the next action potential usually occurs after a longer time interval. Implementing threshold fatigue, that is, decreasing excitability during high-frequency Žring, in the LIF should thus be a candidate for increasing the ISI correlations in a satisfactory way. Our investigations provide another conŽrmation of this point. Although more detailed ionic models of neurons such as the Hodgkin-Huxley model (Hodgkin & Huxley, 1952) are capable of adaptation and ISI correlations with the addition of suitable ionic channels, such models seldom are analytically tractable due to their complexity. In contrast, our model retains enough of the simplicity of the LIF model to remain analytically tractable for the most part. Various integrate-and-Žre-like models with mechanisms analogous to threshold fatigue have been previously used—for instance: • In the analysis of mutually inhibiting neuronal models (Reiss, 1962; Wilson & Waldron, 1968: for an account of these studies, see MacGregor & Lewis, 1977)

256

M. Chacron, K. Pakdaman, and A. Longtin

• A stochastic model where an inhibitory conductance is being reset to a value dependent on past history to mimic the possible summation of hyperpolarizing afterpotentials following each discharge (Geisler & Goldberg, 1966; Wehmier, Dong, Koch, & van Essen, 1989; Treves, 1996; Liu & Wang, 2001) • Periodically and irregularly forced neuronal models (Segundo, Perkel, Wyman, Hegstad, & Moore, 1968) • Self-exciting models and interacting excitatory assemblies (Vibert, Pakdaman, & Azmy, 1994; Pakdaman & Vibert, 1995; Pakdaman, Vibert, Boussard, & Azmy, 1996) • An LIF model in which the threshold decayed exponentially and was incremented by a Žxed amount immediately after an action potential (Geisler & Goldberg, 1966; Holden, 1976; Chacron et al., 2000, 2001a, 2001b; Liu & Wang, 2001) Although the model used in this study bears similarities with previous ones, one key difference resides in the introduction of a memory parameter to adjust the level of fatigue. Indeed, previous studies dealt with neuron models with a Žxed level of fatigue, whereas this one is concerned with a systematic analysis of the inuence of this factor. This program is carried out in three stages involving the description of the response of the modiŽed LIF to (1) constant and step current stimulation, (2) white gaussian noise, and (3) sinusoidal forcing. The Žrst two stages are concerned with adaptation and the ISI correlations due to fatigue. The last stage examines how these factors affect the response of the modiŽed model to other inputs. The particular selection of sinusoidal forcing is motivated by its relevance for understanding information processing in extrinsically forced neurons (Chacron et al., 2000). Furthermore, comparison of our results with the extensive studies of sinusoidally forced LIFs and their variants (Rescigno et al., 1970; Glass & Mackey, 1979; Keener et al., 1981; Alstrøm & Levinsen, 1988; Coombes, 1999; Coombes & Bressloff, 1999; Pakdaman, 2001) provides the basis for the determination of speciŽc contributions of threshold fatigue to the dynamics of the model. 2 The Model We consider the model given by the following equations and Žring rule: dv D ¡v / tv C I ( t) if v ( t) < s ( t) , dt ds sr ¡ s if v ( t) < s ( t) , D ts dt v ( t C ) D v0 C

if v ( t) D s (t)

s ( t ) D s0 C W ( s ( t) , a)

(2.1) (2.2) (2.3)

if v (t) D s ( t) ,

(2.4)

Memory and Adaptation in a Model Neuron

257

Figure 1: Voltage (black solid line) and threshold (gray solid line) time series obtained with the model. An action potential occurs when voltage and threshold are equal. The Žring times tn thus satisfy v ( tn ) D s ( tn ) . Immediately after an action potential, the voltage is reset to zero while the threshold is set to a value s ( tnC ) D s 0 C W ( s ( tn ) , a) .

where v is the voltage, s is the threshold, I (t ) is the stimulation current, tv and ts are the time constants for voltage and threshold, respectively, and sr is the value at which the threshold stabilizes in the absence of Žring. Firing occurs when the voltage reaches the threshold. Following this, the voltage is reset to v 0 (see equation 2.3), and the threshold is set to s ( t C ) D s0 C W ( s ( t) , a), where s 0 is a parameter and W is a monotically increasing function of s and a with W ( s, 0) D 0. a is a positive parameter controlling the memory in the model. We will mostly look at the case W ( s, a) D W 1 ( s, a) ´ as, but other nonlinear forms could be used for W. Throughout, we assume that v 0 · 0 < sr · s0 . The model dynamics are graphically illustrated in Figure 1. Equation 2.4 with W D W1 represents threshold fatigue, with a being the memory parameter. Indeed, when a D 0, the threshold value immediately after a discharge is independent of the threshold value at that discharge, so that future discharges bear no memory of the Žring history. The case where a D 0 and s0 D sr corresponds to the standard LIF with constant threshold. Conversely, when W D W1 and a D 1, one recovers the threshold fatigue

258

M. Chacron, K. Pakdaman, and A. Longtin

implemented in previous studies, that is, after each discharge, the threshold is raised by a Žxed amount s0 (Segundo et al., 1968; Vibert et al., 1994; Pakdaman & Vibert, 1995; Pakdaman et al., 1996; Chacron et al., 2000, 2001a, 2001b; Liu & Wang, 2001). Other models used a conductance that was decremented by a Žxed amount immediately after an action potential to achieve similar effects (Wehmeier et al., 1989; Treves, 1996; Liu & Wang, 2001). Liu and Wang (2001) have shown that the latter approach gave qualitatively similar results to our model with W D W 1 and a D 1. Previous studies were thus limited to the case a D 1. Generally, increasing a from zero corresponds to increasing the degree of fatigue in the model and, consequently, its dependence on its past. The following sections examine how this parameter affects the correlation between consecutive ISIs and the response of the model to various stimuli. 3 Results 3.1 Constant and Step Current Stimulation. Throughout this section, we assume W D W1 and that I (t ) D m is constant. Two situations arise in response to such stimuli. When m tv < sr , the stimulus is subthreshold, the membrane potential v (t ) stabilizes at m tv , and no Žring occurs. Conversely, when m tv > sr , the model generates sustained Žring. The remainder of this section analyzes the corresponding discharge pattern. This is done through the construction of a map relating consecutive postdischarge threshold values. Let v ( u ) be the solution to equation 2.1 with v (0) D v0 , and let s ( u, S ) be the solution to equation 2.2 with s (0, S ) D S. These are given by v ( u ) D ( v 0 ¡ m tv ) e¡u/ tv C m tv

(3.1)

s (u, S ) D ( S ¡ sr ) e¡u/ ts C sr .

(3.2)

Let us assume that a discharge occurred at time tn and resulted in the postdischarge threshold s ( tnC ) ´ SnC . The next Žring takes place after a time interval D n C 1 ´ tn C 1 ¡ tn such that s (D n C 1 , SnC ) D v ( D n C 1 ) and s ( u, SnC ) > v ( u ) for all 0 · u < D n C 1 .

(3.3)

This Žring yields the postdischarge threshold SnCC 1 at time tnCC 1 : SnCC 1 D s 0 C as( D n C 1 , SnC ) D s 0 C av( D n C 1 ) C

D s 0 C a[(Sn ¡ sr ) e

(3.4) (3.5)

¡D n C 1 / ts

C sr ].

(3.6)

Memory and Adaptation in a Model Neuron

259

From these equations, the relation between the nth and n C 1th ISIs D n and D n C 1 is readily derived as: 8 [(v 0 ¡ m tv ) e ¡D nC 1 / tv C m tv ¡ sr ] > > < £eD n C 1 / ts C sr ¡ s 0 ¡ am tv D n D ¡tv ln > a(v 0 ¡ m tv ) > :

9 > > = > > ;

.

(3.7)

While equation 3.7 can be used to discuss the relation between successive ISIs, we prefer to focus on the relation between successive values of the postdischarge threshold. Indeed, for constant stimulation, from the ISI D n one can unambiguously derive SnC , the nth postdischarge threshold, and conversely, given SnC , there is a unique ISI D n , so that using either the ISIs or the postdischarge thresholds yields the same information about the dynamics of the model. However, for the class of stimuli treated in section 3.2, this is not necessarily so; different values of Sn may yield the same D n . This in turn makes it more difŽcult to obtain the relation between D n and D n C 1 . For this reason, the following paragraphs describe the relation between SnC and SnCC 1 . Equation 3.3 uniquely deŽnes D n C 1 D D n C 1 ( SnC ) as a function of SnC , so that we can rewrite equation 3.4 as SnCC 1 D s0 C as( D n C 1 ( SnC ) , SnC ) ´ F ( SnC ) .

(3.8)

This relation implies that the postdischarge threshold SnC after the nth Žring entirely determines the next postdischarge Žring SnCC 1 . In other words, the dynamics of the LIF with threshold fatigue receiving a constant stimulation is determined by the iterates of the map F. This map is deŽned on the interval [s 0 C asr , C 1) . In general, D (S ) , and consequently F ( S ) cannot be explicitly written in closed form because equation 3.3 cannot be solved analytically for arbitrary ts . However, the geometrical properties of F that determine the behavior of the sequence fSnC g can be determined without such knowledge. For Žxed u, the function s (u, S ) is monotonic increasing, so that if S > S0 , then s ( u, S) cannot intersect the voltage prior to s ( u, S0 ) . In other words, S > S0 implies that D (S ) > D ( S0 ) , which means that the larger the postdischarge threshold is, the longer the following interspike interval. Given that v0 · 0 < m tv (the postdischarge potential is reset below the resting voltage) and that v ( u ) is monotonic increasing, we have that D (S ) > D (S0 ) implies v (D ( S ) ) > v (D ( S0 ) ) . This relation together with equation 3.5 and a > 0 yields that F ( S ) > F ( S0 ) whenever S > S0 , that is, F is monotonic increasing. This, combined with the observation that F (s 0 C asr ) > s0 C asr , and F (S ) ! s0 C am tv as S ! C 1, ensures that for any S1C 2 [s0 C asr , C 1) , the sequence SnC is either increasing or decreasing and converges to some value in [s0 C asr , s0 C am tv ]. Finally, all sequences converge to the same value, that is, F has a unique Žxed point, because F is concave. This property comes from the fact

260

M. Chacron, K. Pakdaman, and A. Longtin

that S ! s (u, S ) is contracting, that is, |s(u, S ) ¡ s (u, S0 ) | D exp(¡u / ts ) |S ¡ S0 | < |S ¡ S0 | for u > 0, and v ( u ) is concave, so that F0 , the derivative of F is monotonic decreasing. In summary, we have established that the dynamics of the modiŽed LIF with constant current is governed by SnCC 1 D F ( SnC ) ,

(3.9)

where F, which maps [s 0 C asr , C 1) onto [s 0 C asr , s0 C am tv ) , is a concave monotonic increasing function with a unique Žxed point that we denote by S¤ . Thus, for all S1 < S¤ , the sequence SnC increases and converges to S¤ , and, conversely, for all S1C > S¤ , the sequence SnC decreases toward S¤ . The Žrst consequence of the above result is that in response to a constant stimulation, the modiŽed LIF stabilizes at a periodic Žring with constant interspike intervals D ¤ given by µ

D¤

D tv ln

¶ a(v 0 ¡ m tv ) . S¤ ¡ s0 ¡ am tv

(3.10)

We remark that since D ( S ) is independent of a, the map F deŽned in equation 3.8 is an increasing function of a. A consequence of this observation is that the value of equilibrium postdischarge S¤ increases with a. Given that D ¤ D D ( S¤ ) , the Žring period also increases with a. In other words, the stationary Žring slows down as the factor representing threshold fatigue increases. We now study how D ¤ varies with the input current m . Although it is not possible in general to solve equation 3.7 to Žnd D ¤ as a function of m , it is possible to solve for m as a function of D ¤ . We get m (D ¤ ) D

¤ ¤ ¤ ¤ ¡s 0 C sr ¡ sr eD / ts C v 0eD / ts e ¡D / tv ¡ av0 e¡D / tv . ¤ ¤ ¤ ¤ tv (a ¡ eD / ts C eD / ts e¡D / tv ¡ ae ¡D / tv )

(3.11)

The derivative of m with respect to D ¤ is always negative under the assumptions v 0 · 0 < sr · s 0, a > 0, and D ¤ ¸ ts ln(a). We will now show that the ISI D ¤ can never be less than ts ln(a) . If a · 1, this condition is trivially satisŽed as ts ln(a) · 0 and D ¤ > 0. We thus concentrate on the case a > 1. The denominator of equation 3.11 is zero when D ¤ D ts ln(a) , so the function m ( D ¤ ) has a pole at this point. Since the function m ( D ¤ ) is monotonically decreasing and continuously differentiable on the range (ts ln(a) , C 1) and limD ¤ !C 1 m (D ¤ ) D sr / tv > 0, we have that limD ¤ !ts ln(a) m ( D ¤ ) D C 1. Thus, as we increase m from sr / tv to C 1, the ISI D ¤ decreases from 1 to ts ln(a) . The ISI D ¤ can thus never be lesser than ts ln(a) if a > 1. This has important implications for the model dynamics. In particular, it implies that the stationary ISI D ¤ remains greater than ts ln(a) as we increase m to arbitrarily large values.

Memory and Adaptation in a Model Neuron

261

Moreover, the fact that m ( D ¤ ) is a decreasing function of D ¤ implies that the Žring frequency 1 / D ¤ is always a monotonically increasing function of m . However, if a > 1, the Žring frequency will saturate to a Žnite value given by (ts ln(a) ) ¡1 . As mentioned above, this feature is absent from the LIF model (a D 0, sr D s 0 ) where the Žring rate diverges as a function of the input current. This type of behavior is commonly seen in experimental recordings and is usually associated with absolute refractoriness. Note, however, that the saturation in Žring rate does not imply that an absolute refractory period is present in the model. Indeed, the model can give rise to an ISI smaller than ts ln(a) following a sufŽciently high current pulse or step (see Figure 2C). Thus, we show that even in the absence of an absolute refractory period in the classical sense, threshold fatigue can give rise to a saturation in the Žring frequency under constant current input. This has important implications; it may be that the saturation in Žring rate seen in experimental recordings under constant depolarizing current is due to threshold fatigue rather than the absolute refractory period of the neuron. A modiŽcation dm of the value of m leads to a transient modiŽcation of the discharge rate of the model. Notably, when dm > 0, the sequence Sn increases from the previous value of S¤ toward the new value, resulting in a progressive lengthening of intervals to the new stationary ISI. Note that this ISI will be shorter than the old one. If initially m tv < sr is subthreshold and the new value is suprathreshold, the transient regime in which the ISIs are shorter than in the stationary regime corresponds to neuronal adaptation, that is, a transient frequency increase at the onset of stimulation. The above results hold for all a > 0. However, some distinctions exist between the different values of a that clarify the role of this parameter. One point is that for a · 1, the map F is contracting, that is, F0 (S ) < 1 for all S, but this is not necessarily so for a > 1, where the map may be expanding in some intervals of S. This means that the effects such as discharge rate adaptation tend to be more pronounced in maps with large a than in those with small a. Figure 2A shows the response of the model to a step current going from the subthreshold regime to the suprathreshold regime. Note that the ISIs progressively increase to the new equilibrium value as is seen in experimental recordings (Adrian & Zotterman, 1926). Figure 2B shows the response of the model when the value of the current before the step was set to be in the suprathreshold regime with all other parameters unchanged. Figure 2C shows the response of the model to the same step as in Figure 2B but with a increased. Note the faster rate of adaptation. The rate at which the model adapts to a step current can thus be varied by changing the parameter a. 3.2 Gaussian White Noise Stimulation. Throughout this section, we assume W D W1 and that I ( t) D m C sj ( t) where m is constant and j is white gaussian noise with unit intensity. In this situation, the voltage and threshold of the model are stochastic processes that depend on the particular noise

262

M. Chacron, K. Pakdaman, and A. Longtin

Figure 2: Response of the model to a step increase in current. (A) The current goes from subthreshold (m D 0.1) to suprathreshold (m D 0.9) at t D 50. Note the adaptation in the threshold and the progressive lengthening of ISIs to the new equilibrium value. Parameter values used were ts D 2, tv D 1, a D 1, sr D 0.2, s0 D 0.1, v 0 D 0. (B) The current goes from m D 0.4 to m D 0.9 with all other parameters unchanged. (C) Illustration of the effects of increased a. m goes from 0.4 to 0.9 but a D 10 with all other parameters unchanged. Note the increased rate of adaptation.

realization. The interspike intervals are deŽned as the Žrst passage times (FPTs) of the voltage through the threshold. When a D 0, that is, in the absence of threshold fatigue, the ISIs are independent and identically distributed random variables. We denote by g ( t | s0 ) the probability density function (pdf) of these variables, that is, the function g ( t | s0 ) is the FPT pdf of the Ornstein-Uhlenbeck process g, dg D ¡g / tv C m C sj ( t) with g(0) D g 0, dt

(3.12)

Memory and Adaptation in a Model Neuron

263

through the threshold s ( t) D ( s0 ¡sr ) exp(¡t / ts ) C sr . The FPT pdf g contains all information concerning the point process formed by the discharge times of the model. When a > 0, the description of the discharge times needs to take into account the variations in the postdischarge time threshold. In the same way as for constant stimulation, the key point to the description of the behavior of the model is in establishing the relation between consecutive postdischarge thresholds. In this case, given that the stimulation is noise, the description involves the construction of a Markov chain rather than a map. This is detailed in the following. The conditional pdf of SnCC 1 given SnC can be written as µ ¶ ts a(SnC ¡ sr ) C | Sn , P 1 ( u | Sn ) D g ts ln u ¡ s 0 ¡ asr u ¡ s0 ¡ asr C

(3.13)

where as before g represents the FPT pdf of the Ornstein-Uhlenbeck process, equation 3.12, through the threshold s (t, SnC ) D ( SnC ¡ sr ) exp(¡t / ts ) C sr . The pdf P 1 deŽnes the Žrst-order transition probabilities of an irreducible Markov chain. We denote by h¤ ( S ) the stationary distribution of this chain— in other words, in the long run, the probability for the postdischarge threshold to take a value within (S, S C dS ) is given by h¤ (S ) dS. The characteristics of the point process formed by the discharge times of the model are then deŽned in terms of this stationary distribution together with g and P 1 . For instance, the pdf of ISI distribution g¤ (t ) is given by Z ¤( ) (3.14) g t D g ( t | S ) h¤ (S ) dS. We are interested in the serial correlation coefŽcient of ISIs deŽned by rp D

hD n D n C p i ¡ hD n i2 , hD 2n i ¡ hD n i2

(3.15)

where hD n D n C p i, hD 2n i, and hD n i are the expectations of D n D n C p , D 2n , and D n , respectively. R For n large, the last two quantities are determined by hD n i D tg¤ ( t) dt R and hD 2n i D t2 g¤ ( t) dt. For the Žrst term—the expectation of the product of two intervals—we Žrst consider the case where n is large and p D 1. Given SnC , the postdischarge threshold after the nth discharge, the ISI between the nth and n C 1th Žrings, that is, D n , is distributed according to g (t | SnC ) . The knowledge of SnC and D n uniquely determines SnCC 1 as SnCC 1 D (SnC ¡ sr ) e¡D n / ts C sr D f (SnC , D n ).

(3.16)

This knowledge in turn yields that the ISI between the n C 1th and n C 2th Žrings, D n C 1 , has pdf g ( t | SnCC 1 ) D g ( t | f ( SnC , D n ) ). Combining these with

264

M. Chacron, K. Pakdaman, and A. Longtin

the fact that for n large, SnC is distributed according to h¤ yields Z Z Z hD n D n C 1 i D

S

D D0

DD 0 £ g[D 0 | f (S, D )] £ g[D | S] £ h¤ (S ) dD 0 dD dS.

(3.17)

In a similar way, we derive Z Z Z Z hD n D n C p i D

S

D S0 D 0

DD 0 £g[D 0 | S0 ]P p¡1 [S0 | f ( S, D ) ]£g[D | S] £ h¤ ( S ) dD 0 dS0 dD dS,

(3.18)

where Pp¡1 is the p ¡ 1th transition pdf, R that is, P 0 (u | S ) D d ( u ¡ S ) the Dirac function at S, and P kC 1 ( u | S ) D S0 P 1 ( u | S0 ) P k (S0 | S ) dS0 . In the absence of threshold fatigue, when a D 0, the postdischarge threshold is a Žxed value s 0, so that P 1 ( u | S ) D d (u ¡ s0 ) , h¤ ( S ) D d (S ¡ s0 ), and f ( S, D ) D s 0. Substituting these into equation 3.18 yields that for p ¸ 1, hD n D n C p i D hD n ihD n C p i and consequently that rp D 0. The analysis of the previous paragraphs establishes that the response to gaussian white noise of the modiŽed LIF with threshold fatigue is not described by a renewal process when a > 0. This is in contrast with the standard LIF whose response to white gaussian noise can be described by a renewal process. It also revealed that the modiŽed model with noisy forcing was characterized by a Markov chain relating consecutive postdischarge thresholds. Finally, it showed that the dependence of a postdischarge threshold on the previous value induces correlation between ISIs. Once it is theoretically established that ISIs of the modiŽed model are correlated with one another, we examine the nature of this dependence. For this purpose, rather than using the integral expressions obtained previously, it is appropriate to use numerical simulations of the model. Indeed, the integrals require the computation of the FPT pdf of the Ornstein-Uhlenbeck process through an exponential boundary. Given that no general analytical expression is available for this quantity, derivation of the correlation from the integrals can be computationally more demanding than estimating the same quantities from simulations. Figure 3A shows the ISI distribution g¤ ( t) obtained when the system is driven by gaussian white noise for a D 1, and Figure 3B shows the ISI correlation coefŽcients rp . The estimated r 1 is negative. Serial correlation coefŽcients at higher lags are close to zero, illustrating the rapid return of the system to equilibrium. The fact that r 1 is negative and that it is the only coefŽcient substantially different from zero implies that ISIs shorter (longer) than average will be followed by ISIs longer (shorter) than average. Figure 3C shows the ISI distribution obtained for a D 4. We can see that increasing a increases the mean ISI. Also, the magnitude of the coefŽcient r1

Memory and Adaptation in a Model Neuron

265

Figure 3: (A) ISI distribution obtained for a D 1 in the presence of gaussian white noise of standard deviation 0.1. (B) Correlation coefŽcients rj as a function of lag. Note that only r1 D ¡0.38 is negative and that all coefŽcients are zero for higher lags. (C) ISI distribution obtained for a D 4. (D) Correlation coefŽcients. Note that r1 D ¡0.48 is lower than for a D 1. Other parameter values were ts D 8, tv D 1, m D 1, sr D 0, s0 D 1.

increases with a, which is consistent with the predictions that more memory will give rise to more pronounced ISI correlations. The magnitude of the serial correlation coefŽcient at lag one depends on the slope of the map S ! F (S ) and on the strength of the noise. For high noise intensity, it is expected that the noise will wash out the correlations induced by the map. We thus expect an increase in r1 as a function of noise intensity. This is shown in Figure 4, where noise activates the deterministic properties of the map. The value of r1 has already been linked to the detectability of weak signals in neurons as well as their information transfer capabilities (Chacron et al., 2001a).

266

M. Chacron, K. Pakdaman, and A. Longtin

Figure 4: r1 as a function of the noise standard deviation. r1 exhibits a minimum for the noise intensity around 0.2. It is at this noise level that the noise is most effective at perturbing the map without itself destroying the ISI correlations. Parameter values were the same as in Figure 3 with a D 1. Twenty thousand ISIs were used in each case.

To understand how the deterministic properties of the map can give rise to ISI correlations in the presence of noise, we study the response of the model to perturbations without noise (s D 0). We assume that the stationary regime has been reached, that the model Žres periodically, and that a single sufŽciently large pulsatile stimulation is delivered a time lapse h < D after a discharge, to make the neuron Žre ahead of time. This results in a shortened ISI with length h , and consequently the postdischarge threshold S (h C ) is larger than S¤ . Following the perturbation, the sequence Sn progressively decreases toward S¤ . Given that the interspike interval D ( S) is a monotonic increasing function of S, the sudden increase in S at the time of the perturbation and its progressive decrease indicate that the shortened interval is followed by one that is longer than the period and that subsequent intervals progressively decrease toward the stationary value. If the system returns quasi-instantaneously to equilibrium, then only r 1 will be signiŽcantly negative. However, if the system takes a long time to return to equilibrium after the perturbation, then coefŽcients at higher lags can also be negative. We concentrate on the regime at which

Memory and Adaptation in a Model Neuron

267

the voltage has almost reached its asymptotic value m tv before a Žring: this occurs when ts À tv . We start from equation 3.3 in which we set v (D n C 1 ) D m tv : S (D n C 1 , SnC ) D m tv .

(3.19)

The stationary threshold value is then S¤ D s0 C am tv , and equation 3.10 gives us the stationary ISI D ¤ :

D¤

D ts ln

s 0 C am tv ¡ sr . m tv ¡ sr

(3.20)

Let us suppose that a voltage perturbation advances or delays an action potential, thus causing an ISI h that can be smaller or greater than D ¤ . Then the new value of the threshold immediately after the action potential is given by µ ³ ´¶ ¡h . Sh D s0 C a sr C ( s0 C am tv ¡ sr ) exp ts C

(3.21)

The next ISI, in the absence of further perturbations, will then be given by » hnext D ts ln

¼ ShC ¡ sr . m tv ¡ sr

(3.22)

Equations 3.21 and 3.22 give us the relation between hnext and h: ( hnext D ts ln

s0 ¡ sr C a[sr C ( s0 C am tv ¡ sr ) exp(¡ ths ) ] m tv ¡ sr

) .

(3.23)

The derivative of the map hnext D f (h ) is given by f 0 (h ) D

¡a(s 0 C am tv ¡ sr ) s0 ¡ sr C a[sr C ( s0 C am tv ¡ sr ) exp(¡ ths ) ]

(3.24)

and is negative for s0 > sr > 0 and am tv > 0. We now consider the effects of a on the ISI hnext . If a D 0, we have » ¼ s ¡s D ¤ D hnext D ts ln 0 r , (3.25) m ¡ sr that is, hnext is independent of h since we have f 0 (h ) D 0. This is because the system returns instantaneously to equilibrium and there is no memory extending beyond one ISI. We now consider the effects of a > 0. hnext increases with decreasing h. Thus, a perturbation that causes h larger (smaller)

268

M. Chacron, K. Pakdaman, and A. Longtin

Figure 5: Voltage (black solid line) and threshold (gray solid line) time series. A perturbation in voltage was applied at t D 1098. The perturbation caused an action potential to occur earlier than expected. As such, the ISIh was shorter than D ¤ , and the value of the threshold immediately after that action potential, shC , was higher than the threshold equilibrium value S¤ . Consequently, the threshold took a longer time to decay to m , and the next ISI hnext was longer than D ¤ . Parameter values were the same as in Figure 4.

than D ¤ will make hnext smaller (larger) than D ¤ . Furthermore, increasing a increases hnext. We illustrate this with numerical examples. Figure 5 shows a time series in which a perturbation in the voltage was applied at t D 1099. The perturbation increased voltage and caused an action potential to occur prematurely. The ISI h is thus shorter than D ¤ and consequently ShC (see equation 3.21) is greater than S¤ . Since the threshold starts from a higher value, it will take a longer time to decay to the value m tv . As such, the next ISI hnext will be greater than D ¤ . Similarly, had h been longer than D ¤ , shC would have been smaller than S¤ , and consequently hnext would have been smaller than D ¤ . The actual map hnext D f (h ) is shown in Figure 6A and has a negative slope as expected. Thus, the deterministic properties of the map lead to a perturbed ISI longer (shorter) than average being followed by an ISI shorter (longer) than average. Figure 6B illustrates the dependence of hnext on a for different values of h. As discussed, hnext increases with a. The rate at which hnext increases is, however, greater for low h than for higher h . This shows how the determin-

Memory and Adaptation in a Model Neuron

269

Figure 6: (A) ISI hnext as a function of h (solid black line). The slope of the map is negative as expected (see the text). (B) ISI hnext as a function of a for different values of h . Increasing a will increase hnext . Parameters have the same value as in Figure 5 except a.

istic properties of the map in response to perturbations can give rise to ISI correlations in the presence of noise. 3.3 Sinusoidal Stimulation. In this section, we consider the case where the stimulation is sinusoidal I ( t) D m C a sin(vt) , with a ¸ 0. We show that the response of the LIF with threshold fatigue to such inputs is described by iterates of an annulus map. Such maps can exhibit complex dynamics that can even be chaotic. Through numerical simulations, we illustrate the occurrence of such regimes in the periodically forced LIF with threshold fatigue. Assuming a Žring took place at time tn , we denote by v (t, tn ) and s (t, tn , SnC ) the voltage and threshold at time t, given the initial conditions v (tn , tn ) D v 0 and s (tn , tn , SnC ) D SnC . Provided t ¸ tn and no discharge takes place on

270

M. Chacron, K. Pakdaman, and A. Longtin

[tn , t], we have v (t, tn ) D m tv (1¡exp[(tn ¡t) / tv ]) C

atv [sin(wt) ¡wtv cos(wt) ] 1 C w2 tv2

atv exp[(tn ¡ t) / tv ] [sin(wtn ) ¡ wtv cos(wtn ) ] 1 C w2 tv2 ³ ´ tn ¡ t . s (t, tn , SnC ) D sr C ( SnC ¡ sr ) exp ts ¡

(3.26) (3.27)

p If m tv C atv / 1 C v2 tv2 · sr , then v ( t, tn ) < s (t, tn , SnC ) for all t ¸ tn (i.e., no substained Žring occurs), and, conversely, if q m tv C atv / 1 C v2 tv2 > sr ,

(3.28)

then there exists some Žnite tn C 1 , with tn C 1 > tn such that v ( tn C 1 , tn ) D s ( tn C 1 , tn , SnC ). In other words, the inequality 3.28 is a necessary and sufŽcient condition for sustained Žring of the model. When this inequality holds, the LIF with threshold fatigue will generate an inŽnite sequence of Žring times fti g1 i D 1 with tn ! 1 as n ! 1. We will assume that this is the case throughout the remainder of this section. The condition for sustained Žring is independent of the initial voltage and threshold values; however, the sequence of discharge times depends on these quantities. In the following, we describe the relation between successive discharge times and postdischarge thresholds through the construction of an annulus map. The Žring time tn C 1 is given by def

tn C 1 D F1 ( tn , SnC ) D infft: t > tn , v ( t, tn ) D s ( t, tn , SnC ) g,

(3.29)

and the threshold SnCC 1 immediately after the Žring is given by def

SnCC 1 D F2 ( tn , SnC ) D s0 C W ( sr C (SnC ¡ sr ) e

tn C 1 ¡tn ts

, a).

(3.30)

We deŽne the discharge map as F: ( tn , SnC ) ! ( tn C 1 D F1 ( tn , SnC ) , SnCC 1 D F2 ( tnp, SnC ) ) on [0, 1) £ [s0 C W ( sr , a) , s0 C W (VM , a)], where VM D m tv C atv / 1 C v2 tv2 . For neural computation, it is of interest to know where spikes occur in relation to periodic forcing (see HopŽeld, 1995). Given that the input current I is T-periodic with T D 2p / v, we thus deŽne the Žring phase w n 2 [0, 2p ) as » µ ¶¼ tn w n D w tn ¡ Tint , T

where the int[.] denote the integer part.

(3.31)

Memory and Adaptation in a Model Neuron

271

We can thus associate to F a map f D ( f1 , f2 ) on the annulus [0, 2p ) £ [s 0 C W ( sr , a) , s0 C W ( VM , a) ] as ³ ´ 2p T w n , SnC modulo T (3.32) f1 (w n , SnC ) D F1 2p T ³ ´ T C C w n , Sn . (3.33) f2 (w n , Sn ) D F2 2p The behavior of the periodically forced LIF with threshold fatigue is completely characterized by the iterates of the discharge map F and its associated annulus map f , in the sense that given the initial discharge time t1 and the corresponding postdischarge threshold S1C , the following discharge times and postdischarge thresholds are determined iteratively as ( tn C 1 , SnCC 1 ) D F (tn , SnC ) . We will concentrate on the annulus map f and the sequence of Žring phases and postdischarge thresholds (w i , SiC ) 1 i D 1 . When a D 0, the postdischarge threshold is always set to s0, so that the annulus map reduces to a map on the circle of radius s0 . Furthermore, when sr D s 0, that is, when dealing with the standard LIF without threshold fatigue, this circle map, when restricted to its range, is orientation preserving (Pakdaman, 2001). In such a situation, the system can mainly display phase locking and quasiperiodic behavior, as well as strange nonchaotic dynamics on a parameter set of zero measure. It cannot produce chaos (Coombes, 1999). However, when a > 0, no such restrictions hold. Although we did not observe chaotic behavior with W D W1 for the region of parameter space considered in this article, it is possible to observe sensitivity to initial conditions and chaotic dynamics when a nonlinear form is taken for W. We observed chaotic behavior when we chose W (s, a) D W2 (s, a) ´ exp(as) ¡1 (data not shown). Through systematic numerical simulations, we have observed that such dynamics can be easily observed when a is large and that the sensitivity to initial conditions is associated with a positive leading Lyapunov exponent. The details of this analysis will be described in another study (Chacron, Pakdaman, & Longtin, 2002). 4 Discussion In this study, we have shown that a simple single neuron model that accounts for the memory seen in several classes of neurons has surprisingly rich dynamical behavior. Previous studies involved either a D 0 (Geisler & Goldberg, 1966; Kretzberg, Egelhaaf, & Warcheza, 2001) or a D 1 (Geisler & Goldberg, 1966; Holden, 1976; Vibert et al., 1994; Pakdaman & Vibert, 1995; Pakdaman et al., 1996; Chacron et al., 2000, 2001a, 2001b; Liu & Wang, 2001). We instead varied a 2 [0, 1) continuously and studied its effects. Different values for the parameter a (measuring the amount of memory in the system) gave rise to qualitatively different regimes. The rate and degree of adaptation to a step in current were shown to vary with a. Adaptation is commonly seen in biological neurons; we have found that this interesting

272

M. Chacron, K. Pakdaman, and A. Longtin

feature could be reproduced by our model and that the rate of adaptation depended on the amount of memory. Another common feature of biological neurons is the saturation in Žring rate as the input current is increased. Our model reproduces this feature when a > 1, in contrast with the LIF neuron for which the Žring rate diverges as a function of the input current. In experimental recordings, this saturation is usually thought to be associated with an absolute refractory period or network effects. Our results show that an absolute refractory period is not necessary to obtain such an effect. In fact, this prediction could be veriŽed experimentally in vitro if the Žringrate saturation due to adaptation is lesser than the Žring-rate saturation due to the absolute refractory period. Under the inuence of perturbations and noise, we have shown that the model could give rise to negative ISI interval correlations. The value and rate of decay of these correlations depend on the parameter a and on the noise strength and can be varied to give rise to very different regimes. Expressions were derived for the ISI correlation coefŽcients. The model was also shown to give rise to ISI correlations similar to those seen in experimental data (Longtin & Racicot, 1998; Chacron et al., 2000). Further, the value of the correlation coefŽcient at lag 1 was shown to increase as a function of noise intensity. These correlations were shown to increase the detectability of weak signals in neurons (Chacron et al., 2001a). There thus might be an optimal noise intensity at which signal detection is maximal. ISI correlations have been observed in experimental data taken from neurons in different sensory systems. The advantages of a negative correlation coefŽcient at lag 1 for stimulus detection through long-term spike train regularization as well as stimulus encoding have been outlined in another study (Chacron et al., 2001a). Moreover, a similar Žring mechanism has already been used successfully to model electroreceptors of weakly electric Žsh that display this negative serial correlation coefŽcient at lag 1 (Longtin & Racicot, 1998; Chacron et al., 2000, 2001a, 2001b) and could potentially be used to model other neurons that also display negative ISI correlations (Kufer et al., 1957; Goldberg et al., 1964; Geisler & Goldberg, 1966). In fact, models similar to our own have already been used to model several classes of neurons in the visual system (Keat, Reinagel, Reid, & Meister, 2001), cortical neurons (Liu & Wang, 2001), and electroreceptor afferents (Brandman & Nelson, 2002). Results similar to our own can be obtained by having adaptation in the reset value of the membrane potential. Geisler and Goldberg (1966) have shown that an adapting reset could give rise to ISI correlations. However, as mentioned, there is experimental evidence for the threshold to Žring being dependent on the spiking history (Azouz & Gray, 1999), while there is, to our knowledge, no such evidence for the reset value. We thus believe a dynamic threshold to be more physiologically realistic than a dynamic reset.

Memory and Adaptation in a Model Neuron

273

Although our model contains two timescales, it cannot reproduce the bursting dynamics seen in many neurons with the parameter range considered in this study. This is because we considered only threshold fatigue after an action potential and not facilitation. Our model is thus different from the integrate-and-Žre-or-burst model considered by Coombes, Owen, & Smith (2001), which also has two timescales. It has, however, been shown that the addition of a facilitation current to our model with a D 1 could give rise to bursting dynamics (Chacron et al., 2001b). The response of the model to perturbations and noise will carry over in the presence of sinusoidal forcing (Chacron et al., 2000). However, it is then impossible to obtain analytical explicit expressions for the ISI under voltage perturbations. Furthermore, under sinusoidal forcing, we have shown that our model could be described by a map deŽned on an annulus. While this may seem a simple extension of the many studies that have described the dynamics of sinusoidally forced standard LIFs by circle maps (Rescigno et al., 1970; Keener et al., 1981; Coombes & Bressloff, 1999; Pakdaman, 2001), it has major implications in terms of the dynamics of the system. Indeed, the circle map associated with the periodically forced standard LIF is oneto-one and orientation preserving, so that it produces only one of the three following regimes: phase locked, quasi-periodic, or strange nonchaotic behavior (Keener, 1980). For Žxed parameters, it can produce neither a mix of these nor chaos. In contrast, the annulus map associated with the LIF model with threshold fatigue is not constrained by such limitations and may very well lead to more complex dynamics (Le Calvez, 2000), in agreement with our numerical investigations (details will be described in a future work). Also, introducing threshold fatigue as in our model yields an adapting sequence of ISIs in the presence of periodic forcing. In contrast, for a step increase in bias current, models in which periodic forcing is taken into account by sinusoidally modulating the threshold or the voltage reset value see an abrupt change in ISI. It should be noted that the annulus map is dissipative and not conservative in general. This system is thus different from the so-called standard map studied by various authors (e.g., Meiss, 1992). Our results suggest that neurons with strong adaptation are more prone to displaying complex dynamics than those that either do not adapt or adapt more slowly. Such sensitivity to initial conditions may confer some advantages, as chaotic systems have the potential to transmit much more information about time-varying stimuli than nonchaotic ones, depending on the Lyapunov exponents of the system (Abarbanel, 1996). Further studies are needed to investigate such possible functional roles of chaos in signal processing in nervous systems. In this study, we have considered how the addition of threshold fatigue affected the spike train of a single LIF neuron. However, it is clear that adaptation and ISI correlations could also be due to network or synaptic dynamics. To expand on this point, we now discuss the possible biological mechanisms that the threshold fatigue could model.

274

M. Chacron, K. Pakdaman, and A. Longtin

Cumulative inactivation of sodium channels (Mickus, Jung, & Spruston, 1999) could, for example, give rise to neural adaptation and ISI correlations. However, a fast spike-activated slowly inactivating negative current could also give rise to similar effects: a likely candidate would be members of the KV family of potassium currents (Wang, Gan, Forsythe, & Kaczmarek, 1998). Another candidate could be a calcium-activated potassium current such as Iahp . This current was shown to be present in cortical neurons (Madison & Nicoll, 1984). Liu and Wang (2001) have shown that a model with an Iahp current was qualitatively equivalent to our threshold fatigue model with W D W1 (i.e., a linear threshold reset) and a D 1. It is further known that most ionic currents have voltage-dependent conductances. The parameter s0 represents the amount of reset that is voltage independent. The parameter a in our model could then represent the degree of dependence on the past Žring time in the ionic channel that the dynamic threshold models. Increasing a would thus lead to a more history-dependent conductance. Long-term depression at a synapse could also give rise to relative refractoriness (Hausser & Roth, 1997). In fact, a depressing synapse produces negative spike train correlations (Goldman, Maldonado, & Abbott, 2002). Our model thus bears some similarity with models of short-term plasticity (Fuhrman, Segev, Markram, & Tsodyks, 2002; Goldman et al., 2002). Furthermore, it is known that many synapses are voltage dependent. The parameter a in our model could then represent the voltage dependence of synaptic depression. It should, however, be noted that the recovery time constant of the neurotransmitter at typical synapses is usually in the range of hundreds, if not thousands, of milliseconds (von Gernsdoff, Schneggenburger, Weis, & Neher, 1997). Although neural adaptation usually occurs on much shorter timescales, our model could in principle still be used to account for such phenomena if the threshold time constant is sufŽciently long. Moreover, it has been observed that neural adaptation was in many ways similar to recurrent inhibition (Ermentrout, Pascal, & Gutkin, 2001). Our single-neuron model could therefore describe network effects. It has thus been shown that threshold fatigue can be similar to recurrent inhibition, synaptic depression, or an intrinsic Iahp current responsible for adaptation. Acknowledgments We thank C. Laing, B. Doiron, J. Benda, B. Lindner, and L. Maler for useful discussions. This research was supported by NSERC (M. J. C. and A. L.). References Abarbanel, H. D. I. (1996). Analysis of observed chaotic data. New York: SpringerVerlag.

Memory and Adaptation in a Model Neuron

275

Abbott, L. F., & Dayan, P. (1999). The effect of correlated variability on the accuracy of a population code. Neural Comp., 11, 91–101. Adrian, E. D., & Zotterman, Y. (1926) The impulse produced by sensory nerve endings. Part 2: The response of a single end-organ. J. Physiol. (Lond.), 61, 151–171. Alstrøm, P., & Levinsen, M. T. (1988). Phase-locking structure of “integrate-andŽre” models with threshold modulation. Phys. Lett. A, 128, 187–192. Azouz, R., & Gray, C. M. (1999). Cellular mechanisms contributing to response variability of cortical neurons in vivo. J. Neurosci., 19, 2209–2223. Brandman, R., & Nelson, M. E. (2002). A simple model of long-term spike train regularization. Neural Comp., 14, 1507–1544. Chacron, M. J., Longtin, A., & Maler, L. (2001a).Negative interspike interval correlations increase the neuronal capacity for encoding time dependent stimuli. J. Neurosci., 21, 5328–5343. Chacron, M. J., Longtin, A., & Maler, L. (2001b). Simple models of bursting and non-bursting P-type electroreceptors. Neurocomputing, 38, 129–139. Chacron, M. J., Longtin, A., St.-Hilaire, M., & Maler, L. (2000). Suprathreshold stochastic Žring dynamics with memory in P-type electroreceptors. Phys. Rev. Lett., 85, 1576–1579. Chacron, M. J., Pakdaman, K., & Longtin, A. (2002). Chaotic Žring in the sinusoidally forced leaky integrate and Žre model with threshold fatigue. Unpublished manuscript. Coombes, S. (1999). Lyapunov exponents and mode-locked solutions for integrate-and-Žre dynamical systems. Phys. Lett. A., 255, 49–57. Coombes, S., & Bressloff, P. C. (1999). Mode locking and Arnold tongues in integrate-and-Žre neural oscillators. Phys. Rev. E., 60, 2086–2096. Coombes, S., Owen, M. R., & Smith, G. D. (2001). Mode locking in a periodically forced integrate-and-Žre-or-burst neuron model. Phys. Rev. E, 64, 041914. Ermentrout, B., Pascal, M., & Gutkin, B. (2001). The effects of spike frequency adaptation and negative feedback on the synchronization of neural oscillators. Neural Comp., 13, 1285–1310. Fuhrmann, G., Segev, I., Markram, H., & Tsodyks, M. (2002). Coding of temporal information by activity-dependent synapses. J. Neurophysiol., 87, 140–148. Geisler, C. D., & Goldberg, J. M. (1966). A stochastic model of the repetitive activity of neurons. Biophys. J., 7, 53–69. Gerstner, W., & van Hemmen, J. L. (1992). Associative memory in a network of “spiking” neurons. Network, 3, 139–164. Glass, L., & Mackey, M. C. (1979). A simple model for phase locking of biological oscillators. J. Math. Biol., 7, 339–352. Goldberg, J. M., Adrian, H. O., & Smith, F. D. (1964). Response of neurons of the superior olivary complex of the cat to acoustic stimuli of long duration. J. Neurophysiol., 27, 706–749. Goldman, M. S., Maldonado, P., & Abbott, L. F. (2002). Redundancy reduction and sustained Žring with stochastic depressing synapses. J. Neurosci., 22, 584–591.

276

M. Chacron, K. Pakdaman, and A. Longtin

Hausser, M., & Roth, A. (1997). Dendritic and somatic glutamate receptor channels in rat cerebellar Purkinje cells. J. Physiol. (Lond.), 501, 77–95. Hodgkin, A. L., & Huxley, A. F. (1952). A quantitative description of membrane current and its application to conduction and excitation in nerve. J. Physiol. (Lond.), 117, 500–544. Holden, A. V. (1976). Models of the stochastic activity of neurones. New York: Springer-Verlag. HopŽeld, J. J. (1995). Pattern recognition computation using action potential timing for stimulus representation. Nature, 376, 33–36. Keat, J., Reinagel, P., Reid, R. C., & Meister, M. (2001). Predicting every spike: A model for the response of visual neurons. Neuron, 30, 803– 817. Keener, J. P. (1980). Chaotic behavior in piecewise continuous difference equations. Trans. Amer. Math. Soc., 261, 589–604. Keener, J. P., Hoppensteadt, F. C., & Rinzel, J. (1981). Integrate-and-Žre models of nerve membrane response to oscillatory input. SIAM J. of Appl. Math., 41, 816–823. Kreztberg, J., Egelhaaf, M., & Warzecha, A. K. (2001). Membrane potential uctuations determine the precision of spike timing and synchronous activity: A model study. J. Comput. Neurosci., 10, 79–97. Kufer, S. W., Fitzhugh, R., & Barlow, H. B. (1957). Maintained activity in the cat’s retina in light and darkness. J. Gen. Physiol., 40, 683–702. Lansky, P., & Sato, S. (1999). The stochastic diffusion models of nerve membrane depolarization and interspike interval generation. J. Peripher. Nerv. Syst., 4, 27–42. Le Calvez, P. (2000) Dynamical properties of diffeomorphisms of the annulus and of the torus. Providence, RI: American Mathematical Society. Liu, Y. H., & Wang, X. J. (2001). Spike frequency adaptation of a generalized leaky integrate-and-Žre neuron. J. Comput. Neurosci., 10, 25–45. Longtin, A., & Racicot, D. M. (1998). Spike train patterning and forecastability. Biosystems, 40, 111–118. MacGregor, R. J., & Lewis, E. R. (1977). Neural modeling. New York: Plenum Press. Madison, D. V., & Nicoll, R. A. (1984). Control of the repetitive discharge of rat CA1 pyramidal neurones in vitro. J. Physiol., 354, 319–331. Mickus, T., Jung, H. Y., & Spruston, N. (1999). Properties of slow cumulative sodium channel inactivation in rat hippocampal CA1 pyramidal neurons. Biophys. J., 76, 846–860. Meiss, J. D. (1992). Symplectic maps, variational principles, and transport. Rev. Mod. Phys., 64, 795–848. Neiman, A., & Russell, D. F. (2001). Stochastic biperiodic oscillations in the electroreceptors of paddleŽsh. Phys. Rev. Lett., 86, 3443–3446. Pakdaman, K. (2001). The periodically forced leaky integrate-and-Žre model. Phys. Rev. E., 63, 1907–1912. Pakdaman, K., Tanabe, S., & Shimokawa, T. (2001). Coherence resonance and discharge time reliability in neurons and neuronal models. Neural Networks, 14, 895–905.

Memory and Adaptation in a Model Neuron

277

Pakdaman, K., & Vibert, J. F. (1995). Modeling excitatory networks. Journal of Sice, 34, 788–793. Pakdaman, K., Vibert, J. F., Boussard, E., & Azmy, N. (1996). Single neuron with recurrent excitation: Effect of the transmission delay. Neural Networks, 9, 797– 818. Panzeri, S., Petersen, R. S., Schultz, S. R., Lebedev, M., & Diamond, M. E. (2001). The role of spike timing in the coding of stimulus location in rat somatosensory cortex. Neuron, 29, 769–777. Reiss, R. F. (1962). A theory and simulation of rhythmic behavior due to reciprocal inhibition in small nerve nets. Proceedings of the 1962 AFIPS Spring Joint Computer Conference, 21, 171–194. Rescigno, A., Stein, R. B., Purple, R. L., and Popele, R. E. (1970).A neuronal model for the discharge patterns produced by cyclic inputs. Bull. Math. Biophys., 32, 337–353. Ricciardi, L. M., Di Crescenzo, D., Giorno, V., & Nobile, A. G. (1999). An outline of theoretical and algorithmic approaches to Žrst passage time problems with applications to biological modelling. Mathematica Japonica, 50, 247–322. Sch¨afer, K., Braun, H. A., Peters, C., & Bretschneider, F. (1995). Periodic Žring pattern in afferent discharges from electroreceptor organs of catŽsh. Eur. J. Physiol., 429, 378–385. Segundo, J. P., Perkel, D. H., Wyman, H., Hegstad, H., & Moore, G. P. (1968). Input output relations in computer simulated nerve cells. Kybernetik, 4, 157– 175. Stein, R. B. (1965). A theoretical analysis of neuronal variability. Biophys. J., 5, 173–194. Teich, M. C., & Lowen, S. B. (1994). Fractal patterns in auditory nerve spike trains. IEEE Eng. Med. Biol. Mag., 13, 197–202. Tiesinga, P. H. E., Fellous, J. M., JosÂe, J. V., & Sejnowski, T. J. (2002). Information transfer in entrained cortical neurons. Network, 13, 41–66. Treves, A. (1996). Mean-Želd analysis of neuronal spike dynamics. Network, 4, 259–284. Tuckwell, H. C. (1988). Introduction to theoreticalneurobiology:I. Cambridge: Cambridge University Press. Vibert, J. F., Pakdaman, K., & Azmy, N. (1994). Inter-neural delay modiŽcation synchronizes biologically plausible neural networks. Neural Networks, 7, 589– 607. von Gernsdoff, H., Schneggenburger, R., Weis, S., & Neher, E. (1997). Presynaptic depression at a calyx synapse: The small contribution of metabotropic glutamate receptors. J. Neurosci., 17, 8137–8146. Wang, L. Y., Gan, L., Forsythe, I. D., & Kaczmarek, L. K. (1998). Contribution of the Kv3.1 potassium channel to high-frequency Žring in mouse auditory neurones. J. Physiol. Lond., 509, 183–194. Weiss, T. F. (1966). A model of the peripheral auditory system. Kybernetik, 3, 153–175. Wehmeier, W., Dong, D., Koch, C., & van Essen, D. (1989). Modeling the mamalian visual system. In C. Koch & I. Segev (Eds.), Methods in neuronal modeling. Cambridge, MA: MIT Press.

278

M. Chacron, K. Pakdaman, and A. Longtin

Wilson, D. M., & Waldron, I. (1968). Models for the generation of the motor output pattern in ying locusts. Proceedings IEEE, 56, 1058–1064. Yamamoto, A., & Nakahama, H. (1983). Stochastic properties in somatosensory cortex and mesencephalic reticular formation during sleep-waking states. J. Neurophysiol., 49, 1182–1198. Received April 9, 2002; accepted July 8, 2002.

LETTER

Communicated by Roderick V. Jensen

Reliability of Spike Timing Is a General Property of Spiking Model Neurons Romain Brette [email protected] Centre de MathÂematiques et de Leurs Applications,Ecole Normale SupÂerieurede Cachan, 94230 Cachan, France, and INSERM U483, UniversitÂe Pierre et Marie Curie, 75005 Paris, France

Emmanuel Guigon INSERM U483, UniversitÂe Pierre et Marie Curie, 75005 Paris, France

The responses of neurons to time-varying injected currents are reproducible on a trial-by-trial basis in vitro, but when a constant current is injected, small variances in interspike intervals across trials add up, eventually leading to a high variance in spike timing. It is unclear whether this difference is due to the nature of the input currents or the intrinsic properties of the neurons. Neuron responses can fail to be reproducible in two ways: dynamical noise can accumulate over time and lead to a desynchronization over trials, or several stable responses can exist, depending on the initial condition. Here we show, through simulations and theoretical considerations, that for a general class of spiking neuron models, which includes, in particular, the leaky integrate-and-Ž re model as well as nonlinear spiking models, aperiodic currents, contrary to periodic currents, induce reproducible responses, which are stable under noise, change in initial conditions and deterministic perturbations of the input. We provide a theoretical explanation for aperiodic currents that cross the threshold. 1 Introduction The responses of neurons to dynamic stimuli have been shown to exhibit high reliability in vitro (Mainen & Sejnowski, 1995; Hunter, Milton, Thomas, & Cowan, 1998; Fellous et al., 2001; Beierholm, Nielsen, Ryge, Alstrom, & Kiehn, 2001) and in vivo (Berry, Warland, & Meister, 1997; Nowak, SanchezVives, & McCormick, 1997; Reich, Victor, Knight, Ozaki, & Kaplan, 1997; Berry & Meister, 1998; Buracas, Zador, DeWeese, & Albright, 1998; Bair & Koch, 1996). In this case, spike timing is reproducible on a trial-by-trial basis up to a precision of 1 ms or less, even a long time after stimulus onset (1 s in Mainen & Sejnowski, 1995). Consequently, in a variety of neurons and for time-varying stimuli, spike times convey more information about the stimulus than spike count alone does (Victor & Purpura, 1996; Neural Computation 15, 279–308 (2003)

° c 2002 Massachusetts Institute of Technology

280

R. Brette and E. Guigon

Figure 1: Three cases regarding the reproducibility of neuron responses to a given stimulus. (A) Small variances in ISIs accumulate, so that after some time, spikes are not synchronized between trials. (B) Spike timing is reliable even in the long run but depends on the initial state of the neuron—here, two stable solutions coexist. (C) Spike timing is reliable; noise does not accumulate over time.

de Ruyter van Steveninck, Lewen, Strong, Koberle, & Bialek, 1997; Buracas et al., 1998; Reich, Mechler, Purpura, & Victor, 2000; Reinagel & Reid, 2000). Rate-modulated Poisson processes do not account for these properties (Victor & Purpura, 1996; Baddeley et al., 1997; de Ruyter van Steveninck et al., 1997; Berry et al., 1997; Nowak et al., 1997; Reich, Victor, & Knight, 1998; Kara, Reinagel, & Reid, 2000; Reich et al., 2000). On the other hand, stationary stimuli elicit spike trains with very coarse precision (Mainen & Sejnowski, 1995; Berry et al., 1997; de Ruyter van Steveninck et al., 1997; Buracas et al., 1998). Mainen and Sejnowski (1995) injected a constant current into a neuron and compared the output spike trains over repeated trials. At the beginning, spikes occurred at the same time in all trials, but then small variances in interspike intervals (ISIs) began to accumulate, so that spikes eventually were totally desynchronized over trials. It is not clear why this does not happen with time-varying currents. What happens when the responses of a neuron to a given stimulus are compared over repeated trials, knowing that each trial is different due to intrinsic noise? Three cases may occur, as illustrated in Figure 1: Case 1: Small variances in ISIs accumulate, so that eventually spikes tend to desynchronize over trials.

Reliability of Spike Timing

281

Case 2: Spike times are reproducible in the long run but depend on the initial state of the neuron (e.g., on the membrane potential at the onset of the stimulus); that is, several stable responses exist. Case 3: Spike times are reproducible in the long run and do not depend on the initial state of the neuron. Experimentally, case 1 has been shown to occur for constant input currents, whereas time-varying stimuli induce reproducible responses (Mainen & Sejnowski, 1995), although the experimental procedures used in these studies do not allow us to distinguish between cases 2 and 3. Only in the last case can the responses be considered truly reproducible. In this article, we show that for a general class of spiking neuron models, which includes, in particular, the leaky integrate-and-Žre model (Lapicque, 1907; Knight, 1972) as well as nonlinear spiking models, all three cases can occur if the input current is periodic, while aperiodic currents induce reproducible responses. In addition to numerical simulations, we put forth a theoretical explanation of this property for aperiodic currents that oscillate around threshold. The conditions required for our explanation are not fulŽlled by the nonleaky integrate-and-Žre model—also called perfect integrator—which is never reliable. 2 Materials and Methods 2.1 What Is Reliability? A neuron is said to be reliable when its responses to a given stimulus are reproducible. Thus, reliability is a priori stimulus dependent. The experimental results of Mainen and Sejnowski (1995) show that with a constant current, the responses of a neuron in different trials are initially synchronous, but then small variances in ISIs add up and the spikes become desynchronized. If the intrinsic noise of neurons were lower, it would simply take the spikes longer to desynchronize, but in the end, the result would be the same. However, with nonconstant currents resembling synaptic inputs (low-pass Žltered gaussian noise), desynchronization does not happen. In both cases, intrinsic noise is low at the timescale of a single ISI, but it eventually leads to a desynchronization only in the case of a constant current. Thus, we propose the following deŽnition of reproducibility: responses to a stimulus are considered reproducible if the spike time jitter remains of the order of the intrinsic noise, however long we wait, or, equivalently, that any given precision can be reached if the level of noise is low enough. This deŽnition allows us to distinguish the two cases mentioned above. However, neuron responses can also fail to be reproducible if spike timing depends on the initial condition, that is, the membrane potential at stimulus onset. Therefore, we shall consider in the deŽnition the asymptotic spike time jitter induced not only by intrinsic noise but also by the dispersion of initial conditions.

282

R. Brette and E. Guigon

Our aim is to show that depending on the stimulus, all of the three cases mentioned in section 1 can occur. Furthermore, we shall show that a broad class of models has reproducible responses to aperiodic stimuli. We pay particular attention to leaky models—in a general sense, as deŽned below. However, we also introduce and study other types of models—the perfect integrator and nonleaky models—to show that the nonreliability of the perfect integrator is the exception rather than the rule. 2.2 Models. 2.2.1 The General Model. We consider spiking neuron models, where the membrane potential V evolves according to the following one-dimensional differential equation, dV D f ( V, t) , dt

(2.1)

with the spike modeled as follows: when the potential reaches a threshold Vt , it is reset to Vr . This general deŽnition includes many different models, such as the leaky integrator (Lapicque, 1907; Knight, 1972) and the perfect integrator (Knight, 1972). The equation needs not be linear. Such models are not always reliable, as the example of the perfect integrator shows. However we shall show that a very broad class of models of this kind exhibits reproducible responses. 2.2.2 Leaky Models. One example of such a model is the leaky integrator (Lapicque, 1907; Knight, 1972), where the membrane potential V evolves according to the following equation: t

dV D ¡V C RI ( t) , dt

(2.2)

where t is the membrane time constant, R is the resistance, and I ( t) the input current. We generalize the notion of leak by deŽning a leaky model as a model of type 2.1 satisfying the following condition: @f @V

< 0.

This includes the case when t and R vary in time and also includes nonlinear equations. Basic mathematical properties of leaky models are summarized in the article appendix. In our simulations, we used a leaky integrator with t D 33 ms, R D 200 MV, Vt D 15 mV, and Vr D ¡5 mV, and a nonlinear leaky model deŽned by t

dV D ¡aV3 C RI ( t) , dt

(2.3)

Reliability of Spike Timing

283

with the same values for t , R, Vt , and Vr , and a D 4444 V¡2 (this is for scaling reasons). This corresponds to a neuron model with rectiŽcation, that is, with a voltage-dependent leak. We chose these parameter values so as to obtain Žgures of the same magnitude as in experimental results. However, the actual value of parameters has no inuence on the results, since two sets of parameters actually deŽne the same model (change of variables). 2.2.3 Nonleaky Models. We shall show that the leak is not necessary to obtain reliable spike timing. We simulated an example of a nonleaky model, described by the following equation, t

dV D VI ( t) C k, dt

(2.4)

where I (t) is the input current, with Vt D 1, Vr D 0, t D 33 ms, and k D 1. This model is leaky only if I ( t) < 0 for all t. Otherwise, the solutions of equation 2.4 diverge exponentially in all intervals where I ( t) > 0. However, we shall see that it does not prevent the model from having reliable spike timing. Equation 2.4 does not correspond to a neuron model; we chose it only to illustrate our explanation of reliability. 2.2.4 Perfect Integrator. As an example of a nonreliable model, we have taken the common nonleaky integrate-and-Žre model, t

dV D f ( t) , dt

(2.5)

where f (t ) is an input function. We used t D 12.5 ms (in order to have comparable timescales), Vt D 1, and Vr D 0. We shall see in the simulations that this model is never reliable. Mathematically, the unreliability of this model results from the fact that the distance between two solutions is constant modulo 1, and thus dynamical noise always accumulates (Knight, 1972). 2.3 Stimuli. We tested the reproducibility of model responses to various periodic and aperiodic currents. We also assessed the inuence of current parameters, such as amplitude and mean, on spike timing. For this purpose, we used the following generic method: we deŽne a basis function B (t ) as one with values between ¡1 and 1, such as a sinusoidal wave, and then derive a family of currents by a parameterized afŽne change of variables, for example, I ( t) D Ith C 0.5 C pB ( t) with p 2 [0, 1], where Ith is the input current threshold (the minimum constant current eliciting a response). In this example, we obtain a family of currents with a Žxed DC part and whose amplitude varies from 0 to 1.

284

R. Brette and E. Guigon

Figure 2: Construction of a 2D spike density Žgure. (A) Basis function. A family of currents is deŽned with a parameter controlling the current mean. (B) Trials for a given parameter. Every short vertical line represents a spike. (C) PSTH computed from 2000 trials with the parameter used in B. (D) The PSTH is converted into gray rasters, with the higher the Žring rate, the darker the lines. (E) Gray rasters for all parameters are collected and arranged vertically.

In most simulations, we chose the change of variables so that the current is always above threshold for p 2 [0, 0.5] and oscillates around it for p 2 [0.5, 1]. Subsequently, we want to display the reliability of model responses for all the parameters of a given family of currents on a single Žgure. How this is done is illustrated by Figure 2. For a given basis function (see Figure 2A), we compute the outputs of a large number of trials for each parameter value (see Figure 2B) and then compute a poststimulus time histogram (PSTH) using small time bins (see Figure 2C), which we convert into a line of gray rasters (see Figure 2D). Arranging these lines vertically, we get a two-dimensional spike density Žgure (see Figure 2E), where histograms for all parameter values between 0 and 1 are represented simultaneously, with parameter on

Reliability of Spike Timing

285

the vertical axis and time on the horizontal axis. This allows us to visualize not only the reliability of spike timing for every parameter value, but also the effect of parameters on spike timing and reliability. Thus, we assess the stability of spike timing under noise as well as under deterministic perturbations. For periodic currents, we used sine waves, and for aperiodic currents, we used two types of basis functions: (1) a triangle-shaped function, with the slope of each line chosen according to a random uniform distribution between two values, and (2) a noisier function: a low-pass Žltered gaussian noise that we distorted so that its magnitude distribution over time was uniform between ¡1 and 1 (see, e.g., Figure 4.2.2). This was done to distinguish between the input current always above threshold from the input current oscillating around it. The speciŽc shape of input currents did not inuence the results. 2.4 Measure of Precision. Precision of spike timing was assessed from PSTHs. Most previous measures relied on the identiŽcation of “events” (Mainen & Sejnowski, 1995), deŽned as peaks in the PSTH, whose width is an estimate of the precision. However, assessing the precision only in events might lead to an overestimate, as observed in Tiesinga, Fellous, & Sejnowski (2002a), since the periods when the PSTH is not peaked enough are discarded. Other measures are based on the variance of the PSTH about the mean output frequency (Hunter et al., 1998), but this measure is misleading when the output frequency varies over time. The measure we propose here is based on the entropy of distributions (Borst & Theunissen, 1999). We cannot directly compute the entropy from a PSTH, because this is not a normalized probability distribution. We choose the successive ISIs of one trial to divide the PSTH in successive windows. The models we use have the property that for every other trial, the neuron spikes exactly once in each of these windows. Equivalently, we can split the PSTH in successive windows containing 2000 spikes (the number of trials). We can evaluate the precision of spike timing in any window from the PSTH using the following formula, X HD¡ p (i) log2 p (i) , i

where p ( i) is the number of spikes in time bin i divided by the number of trials. H is the entropy of the distribution of spike times in the window and represents the number of bits required on average to express in which time bin a neuron spikes. One advantage over event-based measures is that if there are m stable solutions, each PSTH window contains m peaks, which gives a higher entropy. However, this quantity depends on the size of the time bin used for the PSTH. We can translate it into a time measure that does not depend on the size of the time bin in the following way. An entropy of H bits is the entropy of a uniform distribution over 2H time bins, that is, over

286

R. Brette and E. Guigon

an interval of size dt2H , where dt is the length of the time bin, which gives the following formula: Á dt exp ¡

X

! p ( i) log p (i) .

(2.6)

i

As dt tends to 0, this quantity tends to ³ Z ´ exp ¡ p ( t) log p ( t) dt , where p ( t) represents the spike time density. Thus, if dt is small enough, the precision measure given by formula 2.6 does not depend on the length of the time bin. As an example, this measure gives a precision of D for a uniform distribution over an interval [t ¡ D / 2, t C D / 2] and a precision of 4s for a gaussian distribution with standard deviation s. We used formula 2.6 to compute the precision in each window and averaged over all successive windows, discarding the Žrst 2.5 s of the 10 s runs (to take the time of convergence into account). In the results, we shall say that neuron responses have a high precision if their measure of precision given by formula 2.6 is small. 2.5 Details of Simulations. Input currents were sampled at a rate depending on the required resolution—2000 Hz in most simulations with the leaky integrator. Equations for the linear leaky integrator and the perfect integrator were solved by exact integration. The equations of the two other models—the nonlinear and the nonleaky ones—were integrated by the Euler method (these were simulated mostly for illustration, and only on a short timescale). Spike timing was computed by bisection with a precision of 1 m s using the following observation: over a single current sampling step (1 / 2000 s), equation 2.1 reduces to an autonomous scalar equation because the input current is constant, which means that V ( t) is monotonous over this duration, and the time when the potential hits threshold can then be computed unambiguously and quickly by bisection. A small noise was added to the potential at each time step of integration. The type of noise used is irrelevant because it tends to a gaussian noise as the time step tends to 0. Therefore, for computational ease, we used a uniform noise with an amplitude ranging from 2.5 mV.s¡1/ 2 to 3.5 mV.s ¡1 / 2 . Reliability of spike timing was assessed by 2000 repeated trials for every tested model and input current over an extended period of time—10 s for the leaky integrator—and PSTHs were computed with a 0.5 ms time bin. Doing many trials avoided the problem of smoothing PSTHs.

Reliability of Spike Timing

287

3 Theoretical Predictions 3.1 Periodic Currents. Responses to periodic currents highlight the two difŽculties that may prevent the neuron responses from being reproducible: (1) a small noise may accumulate over time, or (2) several stable responses may coexist. Because periodically driven leaky models are related to the dynamical theory of homeomorphisms of the circle (Denjoy, 1932; Coddington & Levinson, 1955; Arnold, 1961; Herman, 1977; Keener, 1980; Veerman, 1989), their mathematical properties are well known (Knight, 1972; Keener, Hoppensteadt, & Rinzel, 1981). However, the reliability of periodically driven neuron models is a recent issue (Tiesinga, 2002; Tiesinga, Fellous, & Sejnowski, 2002b). We describe the available theoretical results and explain how they relate to our problem. The dynamics of leaky models depend on the value of the rotation number, deŽned as the ratio of the input frequency to the output spike rate: • It is rational if and only if the output pattern of spikes is (asymptotically) periodic, the period being a multiple of the input period. When this pattern is stable under perturbations, this is called phase locking, which has also been studied experimentally (Rescigno, Stein, Purple, & Poppele, 1970; Ascoli, Barbi, Chillemi, & Petracchi, 1977; Guttman, Feldman, & Jakobsson, 1980; Koppl, 1997). There is p : q phase locking when a stable pattern of p spikes is produced every q periods of the input. Thus when the model is phase-locked, a small noise does not accumulate over time. However, several stable responses may coexist. As noted in Tiesinga (2002), if a p : q phase-locked solution is shifted by a multiple of the input period, a new solution is obtained, so that at least q distinct stable solutions exist. Thus, regarding reliability, cases 2 and 3 can appear. • When it is irrational, under some regularity conditions, the dynamics of the model are topologically equivalent to the dynamics of a model with constant input, which means that a small noise always accumulates over time, however small it may be. Thus, case 1 occurs. When the input current is parameterized, for example, by its frequency and mean, the rotation number is a continuous function of the parameters, which implies that cases 1, 2, and 3 occur in any area of the parameter space where the rotation number is not constant. Thus, there is no systematic relationship between current parameters and reliability. However, while the set of parameters where case 1 occurs has positive measure if the current is always above threshold (Herman, 1977), it has measure zero, and thus is not observable, if the current oscillates around threshold (Keener, 1980; see Veerman, 1989, for a rigorous mathematical proof). To sum up, cases 2 and 3 occur in any area where the current oscillates around threshold; cases 1,

288

R. Brette and E. Guigon

Figure 3: A current below threshold results in an interval where no spiking is possible. Arrows represent whether the vector Želd points up or down at threshold.

2, and 3 occur in any area where the current is above threshold (unless the rotation number is constant). The (asymptotic) spike time jitter is an increasing function of the level of intrinsic noise, which tends to 0 as the noise variance tends to 0 when the model is phase-locked and to a positive number when it is not. Quantitatively, this function depends on various detailed aspects of the dynamics, such as the Lyapunov exponent (Tiesinga, 2002). 3.2 Aperiodic Currents. In the case of aperiodic currents, such as Žltered gaussian noise, we will see in the numerical results that neither case 1 nor case 2 arises. Other studies suggest that the responses are reliable when the current is above threshold (Pakdaman & Tanabe, 2001). Here, we explain why this should also be so when the input current oscillates around threshold. Our explanation relies on the construction of a set of possible spike times, which is done with no noise on the dynamics. In the appendix, we prove that this set is stable under noise and deterministic perturbations. Let t1 be a time when the input current goes below threshold. A solution arriving at threshold at time t1 cannot cross it and will hit threshold again only later, at time t2 , as shown in Figure 3. The area deŽned by this solution and the line V D Vt cannot be reached by any other solution, since trajectories cannot cross. Therefore, no spike can occur between t1 and t2 , which happens every time the current goes below threshold, and so such time intervals can be removed from the possible spike times. The area deŽned by the solutions starting at reset between t1 and t2 cannot be reached by any solution starting before t1 , since such a solution would spike between t1 and

Reliability of Spike Timing

289

Figure 4: Construction of unreachable time intervals. As in Figure 3, time is on the horizontal axis and membrane potential on the vertical axis. The color areas are unreachable by any solution. Thus, the sample solution, shown as a dashed line, always remains in the white area. The simulation was 500 ms long with a leaky integrator.

t2 . The spike times of these forbidden solutions deŽne a whole sequence of unreachable spike time intervals. Therefore, every time the input current goes below threshold, a whole sequence of intervals is removed from the possible spike times, as shown in Figure 4. We can see that the Žrst time the current goes below threshold, a red area that no solution can reach is created near threshold. The solutions starting from the resulting forbidden time interval are in red. The next time the current goes below threshold occurs outside the red area and creates a new sequence of yellow stripes. The third time occurs inside the red area, and the result is displayed in blue. Thus, a solution starting at time 0 always remains in the white area, as shown by the dashed black line. One can see that the white area gradually gets smaller and smaller. Based on the above considerations, we predict that when the current oscillates around threshold, the set of possible spike times eventually gets very narrow, so that spike timing should be reproducible in the long run, even if the potential is not Žxed at reset at stimulus onset. For the general model given by equation 2.1, “the current goes below threshold” means that the vector Želd points downward at threshold, that is, f (Vt , t) < 0. In the construction described above, we implicitly assumed that a solution is at reset potential at time t only when it spikes at time t, which is not necessarily so in the general case of equation 2.1. For example, in Figure 4, one might imagine a solution starting at time 0, going below reset, and then going up into the Žrst red stripe. This cannot happen if we assume f (Vr , t) > 0 for all t. Another sufŽcient condition to avoid this problem is to assume that the model is leaky, as discussed in the appendix. None of these two conditions is fulŽlled by the perfect integrator. Indeed, in this model, when the vector Želd points downward at threshold, it also does so at reset. The numerical simulations show that this model is never reliable. We briey describe the numerical construction of the set of reachable spike times, as illustrated in Figure 4. First, Žnd the Žrst time t1 when the input current goes below threshold. This is the starting point of the Žrst

290

R. Brette and E. Guigon

forbidden interval. Then run the solution starting from threshold at time t1 and stop when it hits threshold again at time t2 . The interval I D [t1 , t2 ] is the Žrst forbidden interval, where no spike can occur. Then run the solution starting from reset at time t1 (resp. t2 ) and stop when it hits threshold at time t3 (resp. t4 ), so as to obtain the next forbidden interval [t3 , t4 ]. By repeating the process, we get a whole sequence of intervals where no spike can occur. In the remaining set of times, the current goes again below threshold. So Žnd the Žrst time when it does so after t2 , and repeat the procedure. Thus, when the algorithm ends, we get the remaining set of reachable spike times, and we shall see that this set eventually gets very small. Since we chose to display the reliability of model responses for all parameters of a given family of currents (see Figure 5A) on a single Žgure, we computed the set of reachable spike times for each parameter (see Figures 5B and 5C) and displayed it in a two-dimensional white and black Žgure, with time on the horizontal axis and parameter on the vertical axis (see Figure 5D). Black dots represent reachable spike times. The explanation above does not apply to the case when the input current is always above threshold. However, in our numerical results, responses seem to be reproducible in both cases, although the required intrinsic noise for reproducibility is signiŽcantly lower than for currents crossing the threshold. Our explanation does not rule out the possibility of multiple stable solutions, but this does not seem to happen in our simulations. 4 Numerical Results 4.1 Periodic Currents. The results regarding periodic stimuli are displayed in Figure 6, which illustrates the theoretical results. A leaky integrator is driven by a 20 Hz sinusoidal current (see Figure 6A), whose mean is a decreasing linear function of the parameter. For parameters p < 0.5, the current is always above threshold, whereas for p > 0.5, the current oscillates around threshold. The spike density Žgure (see Figure 6B) was computed without noise, with random initial potential for every trial, so that phase locking appears clearly. Adding dynamical noise results only in blurring this Žgure; it has no qualitative inuence on the results. The model was run for 5 s. When phase locking occurs, all the stable solutions can be observed in the last 250 ms of the runs (see Figure 6B). Phase locking appears for a given parameter value when the density line for this value consists of a Žnite number of black dots only, that is, when the corresponding PSTH consists of a Žnite number of isolated peaks. Conversely, there is no phase locking when the density line consists of gray shades, that is, when the corresponding PSTH is a smooth distribution. As expected, responses are always phase-locked for p > 0.5, whereas there are not always for p < 0.5. Mean Žring rate and precision were computed for every parameter value with noise-free dynamics, as seen in Figure 6C (solid lines), where phase locking corresponds to plateaus in the Žring rate and to drops in precision

Reliability of Spike Timing

291

Figure 5: Construction of the set of possible spike times for all parameters of a given family. (A) Basis function. (B) For a given parameter, we compute the times when no spike may occur (dark areas are not reachable). (C) This is converted into a line of white and black rasters, with black ones corresponding to possible spike times. (D) Collecting all these lines for all parameters and arranging them vertically, we get a two-dimensional white and black Žgure of possible spike times, with time on the horizontal axis and parameter on the vertical axis.

measure. Precision was also computed with a small noise on the dynamics (1 mV.s ¡1 / 2 ), which results in a smoothed version of the noise-free precision plot, where only 1:1 phase locking appears clearly (see Figure 6C, dashed line). There is no trivial relationship between precision and parameter. The precision is better for p > 0.5, where the model is always phase-locked (the current oscillates around threshold), than for p < 0.5. The best precision is obtained for 1:1 phase locking, as observed in Hunter et al. (1998). We showed the results for three parameter values (see Figures 6D–6F), which illustrate the three possible cases of reliability. The simulations were done with a small dynamical noise. Case 1: Noise accumulates. For p D 0, responses were not reproducible. After 5 s, the PSTH was at, and individual trials were not synchro-

292

R. Brette and E. Guigon

Reliability of Spike Timing

293

nized (see Figure 6D). The model behaves in the same way as with a constant current. Case 2: Multiple stable solutions. For p D 1, the responses are phaselocked, as shown in the spike density Žgure (see Figure 6B). The spike rate is 13.3 Hz (see Figure 6C), which is two-thirds the input frequency. Thus, there is 2:3 phase locking; the model neuron spikes twice every three periods, as shown in the individual trials (see Figure 6F). The density Žgure (see Figure 6B) and the PSTH (see Figure 6F) indicate six spikes every three periods, which means there are three stable solutions, resulting also in a higher sensitivity to noise (see Figure 6C, dashed line). Case 3: One stable solution. For p D 0.5, the spike rate is 20 Hz (see Figure 6C): the model is 1:1 phase-locked. There is only one stable solution, as seen in the individual trials (see Figure 6E). Thus, responses are truly reproducible, and precision is very high (see Figure 6D). In these simulations, case 3 occurs in only about 30% of the parameters range (the 1:1 phase-locking area). 4.2 Aperiodic Currents. 4.2.1 Noise Does Not Accumulate over Time. In Figure 7, we assessed the reliability of a noisy leaky integrator in response to a random triangle wave, whose mean is constant and amplitude varies linearly with the parameter (the current was constant for p D 0). For all trials, membrane potential was at rest at stimulus onset; thus, we do not distinguish between cases 2 (several stable solutions) and 3 (one stable solution). The input current is always above threshold for p < 0.5 and oscillates around threshold for p > 0.5. Figure 6: Facing page. Reliability of a leaky integrator driven by a periodic input current. The model is a linear leaky integrator. (A) Input current is a 20 Hz sine wave I ( t) D 85 C 40(1 ¡ p ) C 30sin(40p t) pA, where p is the parameter. The results are shown from 4.75 s after the onset of the stimulus. (B) The model was run 2000 times for every parameter value (400 values between 0 and 1), with initial potential drawn uniformly between threshold and reset, but no noise on the dynamics, so that phase locking appears clearly. (Adding noise results in a blurred version of this Žgure.) The results are displayed as a spike density Žgure. (C) Firing rate was computed as a function of the parameter, with no noise on the dynamics. Precision was computed with no noise on the dynamics (solid line) and with a noise of 1 mV.s¡1 / 2 (dashed line). Phase locking appears for parameter values below 0.5 as drops in precision. (D–F) For three different parameter values (0, 0.5, 1), we show the results of 10 trials of the model with noise on the dynamics (2.5 mV.s¡1 / 2 ) and random initial potential, and also show the PSTH, computed from 2000 trials. Note the different scales for Žring rate in these graphs.

294

R. Brette and E. Guigon

Trials were 10 s long. The spike density Žgure (see Figure 7B) shows that when the input current oscillated around threshold (p > 0.5), spikes were synchronized in the long run, and the actual spike times corresponded with our computational prediction of possible spike times (see Figure 7C). Tight synchronization can be clearly seen in the 10 individual trials computed for parameter p D 0.8 in Figure 7E, and the PSTH is very peaked. Thus, noise did not accumulate (case 2 or 3). Spiking seems less precise when the input current is always above threshold (p < 0.5), as shown in the spike density Žgure (see Figure 7B) and the precision plot (see Figure 7F, solid line). Individual trials for parameter p D 0.1 (see Figure 7D) show synchronized spikes at the beginning of the simulation, but not in the last 500 ms of the 10 s runs. This is reected by the PSTH as a steady decrease in the height of the peaks during the Žrst 500 ms and as a at distribution during the last 500 ms. However, we also computed the precision of long runs (200 s) with a lower noise on the dynamics (0.7 mV.s ¡1/ 2 ) (see Figure 7F, dashed line). This duration was enough for noise to accumulate in the case of a constant current (p D 0), but it did not lead to a desynchronization of trials for parameters above 0.2. Thus, it seems that in contrast with periodic currents, responses to aperiodic currents are also reproducible (case 2 or 3) when the current is above threshold, as suggested by other studies (Jensen, 1998; Pakdaman & Tanabe, 2001), though the level of tolerable noise is lower than when the current oscillates around threshold. The type of current used (e.g., gaussian instead of triangle wave) did not alter the results. 4.2.2 There Is Only One Stable Solution. In Figure 8, we assessed the reliability of a noisy leaky integrator in response to a random triangle wave, Figure 7: Facing page. Reliability of a leaky integrator with initial potential at rest. The model is a linear leaky integrator. (A) Input current is I ( t) D 150 C 150pB(t) pA, where p is the parameter and B ( t) is a triangular wave taking values between ¡1 and 1, with rising and falling time drawn uniformly between 10 ms and 50 ms. We show the results for the Žrst and last 500 ms of the 10 s simulations. (B) The model was run 2000 times for every parameter value (400 values between 0 and 1), with membrane potential initially at rest value, and noise on the dynamics (3.5 mV.s¡1/ 2 ). The results are displayed as a spike density Žgure. (C) The set of possible spike times was computed for every parameter value. (D, E) For parameter values p D 0.1 (E) and p D 0.8 (F), 10 individual trials were extracted and displayed, along with the PSTH computed from 2000 trials. (F) Mean Žring rate does not vary much with the parameter. The precision was computed as a function of the parameter from the last 7.5 s of 10 s0 runs with a noise of 3.5 mV.s¡1/ 2 on the dynamics (solid line, 400 parameter values), and from the last 10 s of 200 s0 runs with a noise 0.7 mV.s¡1 / 2 (dashed line, 10 parameter values).

Reliability of Spike Timing

295

296

R. Brette and E. Guigon

whose mean is constant and amplitude varies linearly with the parameter. For every trial, membrane potential was drawn at random uniformly between rest and threshold, so that we could distinguish between cases 2 (several stable solutions) and 3 (one stable solution). When the input current oscillates around threshold (p > 0.5), we can see from the spike density Žgure (see Figure 8B) that spike timing becomes precise after some time. In the individual trials for parameter p D 0.95 (see Figure 8E), spikes initially occur at different times, depending on the membrane potential at onset, and eventually they become synchronized over trials, whatever the initial potential, which corresponds to case 3, and was predicted by the computation of possible spike times (see Figure 8C). The results are shown only for the Žrst 500 ms, but the behavior after 9.5 s is similar to the one shown in Figure 7. When the input current is always above threshold (p < 0.5), responses are not reproducible with this magnitude of intrinsic noise, as the results of individual trials for p D 0.1 show (see Figure 8D). However, we found that with a lower intrinsic noise, responses were reproducible in the long run for small parameter values even after 200 s (precision of less than 6 ms for p > 0.2). 4.2.3 Spike Times Are Stable Under Deterministic Perturbations. Do spike times in the long run change when the input current is increased? The spike times of a neuron driven by a constant current are not stable under deterministic perturbations: if the (constant) ISI is T and a slight decrease of the current makes it T C d, then the difference between the times of the nth spike is nd, leading to a desynchronization of the two patterns of spikes after enough time. In Figure 9, we simulated a noisy leaky integrator driven by a random triangle wave, whose mean and amplitude increased linearly with the parameter. The current oscillated around threshold for every parameter value. For every trial, membrane potential was drawn at random uniformly between rest and threshold. The mean Žring rate, computed over 10 s, increased from 22 to 35 Hz with the parameter (see Figure 9D), but spike times did not vary signiŽcantly with the parameter, even after 750 ms (see Figures 9B and 9C). Figure 8: Facing page. Reliability of a leaky integrator with random initial potential. The model is a linear leaky integrator. (A) Input current is I ( t ) D 150 C 150pB(t) pA, where p is the parameter and B ( t) is a normalized Žltered gaussian noise with time constant 10 ms. (B) The model was run 2000 times for every parameter value (400 values between 0 and 1), with initial potential drawn uniformly between rest and threshold, and noise on the dynamics (2.5 mV.s¡1 / 2 ). The results are displayed for the Žrst 500 ms as a spike density Žgure. (C) The set of possible spike times was computed for every parameter value. (D, E) For parameter values p D 0.1 (D) and p D 0.95 (E), 10 individual trials were extracted and displayed, along with the PSTH computed from 2000 trials.

Reliability of Spike Timing

297

298

R. Brette and E. Guigon

Thus, an increase in Žring rate creates additional spikes rather than a shift of all spikes. Precision was almost constant for all parameter values—around 3 ms (see Figure 9D). Thus, when the input current oscillates around threshold, responses are stable under signiŽcantly large deterministic perturbations. 4.2.4 Nonlinear and Nonleaky Models Are Also Reliable. In Figure 10, we assessed the reliability of a noisy leaky nonlinear model driven by equation 2.3 (see Figures 10B and 10C) and a noisy nonleaky model described by equation 2.4 (see Figures 10D and 10E) driven by a gaussian input current whose mean is constant and amplitude varies linearly with the parameter. For every trial, membrane potential was drawn at random uniformly between rest and threshold. Again, responses were reproducible (case 3) when the input current oscillated around threshold (p > 0.5). 4.2.5 The Perfect Integrator Is Never Reliable. As a counterexample, we simulated a noisy perfect integrator (Knight, 1972; see Figure 11), driven by a gaussian current whose mean is constant and amplitude varies linearly with the parameter. For all trials, membrane potential was at rest at stimulus onset. The hypotheses required for our theoretical explanation are not fulŽlled by this model. We can see in Figure 11B that whether the input current is above threshold (p < 0.5) or oscillates around it (p > 0.5), the noise accumulates and leads to a complete desynchronization of spikes over trials (case 1). Figure 11C shows that the set of possible spike times does not get smaller over time, and Figure 11D that the precision is always above 13 ms, which is half the average ISI. 5 Discussion We identiŽed two ways in which neuron responses can fail to be reproducible: dynamical noise can accumulate over time and lead to a desynchronization over trials, or several stable responses can exist, depending on the initial condition. These two cases can occur when the model neuron is driven by a periodic current, but the former can occur only if the current is above threshold (Keener et al., 1981). In contrast, we showed that responses of a general class of spiking neuron models to aperiodic currents oscillating around threshold are reproducible: spike timing in the long run is stable under noise and deterministic perturbation and does not depend on the initial condition. We provided a theoretical explanation based on a geometrical construction that shows that the set of possible spike times becomes smaller and smaller over time. Our numerical results suggest that responses to aperiodic currents above threshold are also reproducible, which conŽrms other studies (Pakdaman & Tanabe, 2001), but the level of tolerable noise seems lower in this case.

Reliability of Spike Timing

299

Figure 9: Stability of spike timing under deterministic perturbations. The model is a linear leaky integrator. (A) Input current is I ( t) D 112.5 C (75 C 37.5p) B ( t ) C 37.5p pA, where p is the parameter and B ( t ) is a triangular wave taking values between ¡1 and 1, with rising and falling time drawn uniformly between 10 ms and 50 ms. Thus, the mean input current increases with the parameter from 112.5 pA to 150 pA. (B) The model was run 2000 times for every parameter value (400 values between 0 and 1), with initial potential drawn uniformly between rest and threshold and noise on the dynamics (2.5 mV.s¡1 / 2 ). The results are displayed for the Žrst 750 ms as a spike density Žgure. (C) The set of possible spike times was computed for every parameter value. (D) Mean Žring rate and precision were computed from 10 s simulations.

300

R. Brette and E. Guigon

Figure 10: Reliability of a nonlinear and a nonleaky model. We assessed the reliability of two models that differ from the linear leaky integrator: a leaky nonlinear model driven by equation 2.3 (B, C) and a nonleaky model driven by equation 2.4 (D, E). (A) Input current is I ( t) D 150 C 150pB(t) pA for the leaky nonlinear model and I ( t) D 0.5 C 3pB(t) for the nonleaky model (unit is irrelevant), where p is the parameter and B ( t) is a normalized Žltered gaussian noise with time constant 10 ms. (B) The nonlinear leaky model was run 2000 times for every parameter value (400 values between 0 and 1), with initial potential drawn uniformly between rest and threshold, and noise on the dynamics (2.5 mV.s¡1/ 2 for the leaky model). The results are displayed for the Žrst 500 ms as a spike density Žgure. (C) The set of possible spike times was computed for every parameter value. (D, E) The same simulations were done for the nonleaky model, with noise on the dynamics 0.1 V.s¡1/ 2 .

Reliability of Spike Timing

301

Figure 11: The perfect integrator is not reliable. The model is a perfect integrator driven by equation 2.5. (A) Input current is I ( t) D .5 C pB ( t) (units are not signiŽcant here), where p is the parameter and B (t ) is a a normalized Žltered gaussian noise with time constant 10 ms. We show the results for the Žrst and last 500 ms of the 10 s simulations. (B) The model was run 2000 times for every parameter value (400values between 0 and 1), with membrane potential initially at rest value and noise on the dynamics (0.27 V.s¡1 / 2 ). The results displayed are displayed as a spike density Žgure. (C) The set of possible spike times was computed for every parameter value. (D) Mean Žring rate is almost constant, around 40 Hz, while precision is always above 13 ms, which is very high, knowing that this is about half the mean ISI.

302

R. Brette and E. Guigon

We showed that for aperiodic currents that cross the threshold, noise does not accumulate over time. In other words, in this case, the asymptotic spike time jitter is a function of the level of intrinsic noise that tends to 0 as the noise variance tends to 0, which is not generally so for periodic currents. This result is not a consequence of particular factors such as the leak, the amount of uctuations in the input current, or its frequency spectrum. However, quantitatively, the ratio of the asymptotic spike time jitter to the noise variance depends on various characteristics of the input current (and, more generally, of the differential equation; Hunter et al., 1998; Fellous et al., 2001; Beierholm et al., 2001; Tiesinga, 2002; Tiesinga et al., 2002b). Such a relationship is relevant only when responses are reproducible in the way deŽned above. In particular, it would be illusory to search for a formula describing the precision as a function of a parameter of the input in the case of periodic currents above threshold. Indeed, when varying the parameter, the model alternates between phase locking, where precision is close to 0, and structural unstability, where spike timing is imprecise even with an arbitrarily small intrinsic noise (besides being dependent on the initial condition). However, when the model is phase-locked, the (quantitative) precision depends on dynamical properties such as the Lyapunov exponent and the distance to the boundary of the Arnold tongue (Tiesinga, 2002). Appendix: Stability of the Set of Possible Spike Times A leaky model is described by a one-dimensional differential equation, dV D f ( V, t) dt

(A.1)

where V is the membrane potential, with the spike modeled as follows: when the potential reaches a threshold Vt , it is reset to Vr . The leak is described by the following condition on f : @f @V

< 0.

This inequality implies that the distance between any two solutions of equation A.1 (without spike) always decreases and tends to zero—hence, the term leak. Since we are interested in the sequence of spike times, it is natural to introduce the map Q: R ! R that gives the time of the Žrst spike following a given spike time, that is, Q ( t) is the time when a run starting from reset potential at time t hits threshold. We shall call this map the spike map. Thus, the sequence of spike times of a run with Žrst spike at time t is just t, Q (t) , Q2 (t) , . . . The spike map was Žrst introduced in Rescigno et al. (1970) in the case when the input current is periodic and studied further in Knight (1972) and Keener et al. (1981).

Reliability of Spike Timing

303

One can easily show that for every t, f (Vt , Q ( t) ) ¸ 0, which means there can be no spike if the vector Želd points downward at threshold, that is, in the case of the linear leaky integrator, if the input current is below the current threshold. This result is the Žrst cornerstone of the construction of the set of possible spike times, and it holds also for nonleaky spiking models, including the perfect integrator. It allows us to remove intervals from the set of possible spike times, as described in section 3.2. In the construction of this set, we also remove the image of these intervals by iterates of Q. However, this is correct only if Q is injective. For leaky models, one can show that Q is strictly increasing on its range (see Keener et al., 1981, for the case of periodic currents). This is also true if f ( Vr , t) > 0 for all t, which is the case of the nonleaky models we simulated. For periodic currents, that is, in the general case, when f is periodic with respect to t, the fact that Q is strictly increasing on its range means it is a lift of an orientation-preserving circle map. If Q is surjective, that is, if the current is always above threshold, then Q is a lift of a homeomorphism of the circle. The dynamics of both noninvertible orientation-preserving circle maps and homeomorphisms of the circle are well known, as we described in section 3.1. Our aim is to prove that the set of possible spike times constructed in section 3.2 is stable under noise and deterministic perturbations. We assume that the duration of ISIs is bounded, so that there is an M > 0 such that Q ( t) ¡ t < M for all t. For mathematical convenience, we consider here that a run may hit threshold without spiking, as long as it does not cross it. Consider runs deŽned on R (not only R C ), with inŽnitely many spikes on both R C and R ¡ . Note R ½ R the set of spike times of all these runs, that is, the set of possible spike times constructed from t D ¡1. Now consider a perturbation of this system deŽned in the following way: take 2 > 0, representing the amount of perturbation; the spike following a spike at time tn occurs at time tn C 1 such that |tn C 1 ¡ Q ( tn ) | · 2 . This accounts for an uncertainty that may be random or deterministic. Note R2 the set of spike times of all possible perturbed runs for a given 2 > 0. We shall prove the following stability theorem: Theorem 1. For any compact set K such that K \ R D ;, there is an 2 > 0 such that K \ R2 D ;. In other words, any compact set, such as an interval, that is not reachable under the original deterministic dynamical system is not reachable either if a small perturbation is added on the dynamics. Therefore, adding a small noise to the dynamics should have a small inuence on the set of spike times. In the same way, a slight change of the vector Želd should cause a small perturbation of spike times.

304

R. Brette and E. Guigon

First, let us formalize the considerations above. Considering that a run may hit threshold without spiking, as long as it does not cross it, means in the present case that if the spike map is discontinuous at t, then the spike following a spike at time t may occur at either time Q ( t¡ ) or Q ( t C ). Now choose 2 > 0 as the level of uncertainty; if there is a spike at time t, then the next spike will occur within distance 2 of either Q ( t¡ ) or Q (t C ) , that is, the set of possible times for next spike is [Q ( t¡ ) ¡ 2 , Q (t¡ ) C 2 ] [ [Q ( t C ) ¡ 2 , Q ( t C ) C 2 ]. Formally, this framework may be expressed in terms of a setvalued dynamical system. We refer the reader to Aubin and Frankowska (1990) for more details about set-valued analysis. A sequence ( tn ) of spike times agreeing with our considerations is one that satisŽes tn C 1 2 F2 ( tn ) ,

(A.2)

where F2 ( t) D [Q (t¡ ) ¡ 2 , Q ( t¡ ) C 2 ] [ [Q ( t C ) ¡ 2 , Q ( t C ) C 2 ]. Thus, F2 is a set-valued map that we write F2 : R ; R, as in Aubin and Frankowska (1990). We introduce a few deŽnitions from set-valued analysis: DeŽnition 1. • The graph of a set-valued map F: X ; Y is the set of points ( x, y ) 2 X £ Y such that y 2 F ( x) .

• The inverse of a set-valued map F: X ; Y is the set-valued map F ¡1 : Y ; X deŽned by x 2 F ¡1 ( y ) if and only if y 2 F ( x) . • If A ½ R, then [ F (A) D F ( t). t2A

• If F: R ; R and G: R ; R are two set-valued maps, then the composition G ± F: R ; R is the set-valued map deŽned by G ± F ( t) D G ( F (t ) ) . • The notation Fn stands for the n-fold composition F ± F ± ¢ ¢ ¢ ± F.

• The intersection of F: R ; R and G: R ; R is the set-valued map F \ G: R ; R deŽned by F \ G ( t) D F ( t) \ G ( t) . Thus, F2 n ( t) is the set of possible times for the nth spike following a spike at time t, given an uncertainty 2 . The set of possible times for the nth spike of any run is F2 n ( R ). Consider a run deŽned on R, with inŽnitely many spikes on R ¡ . Every spike of this run must occur within the set F2 n (R ) , for any integer n. Thus, the spike times are contained in the set \ R (F2 ) D F2 n ( R ) . n> 0

Conversely, for any time t 2 R ( F2 ) , we can Žnd a sequence of spike times satisfying equation A.2. Thus, R (F2 ) is the set of spike times of all such runs.

Reliability of Spike Timing

305

In order to prove theorem 1, we need another deŽnition: DeŽnition 2. Let F: X ; Y be a set-valued map. F is said to be compact on compact sets (CC) if for any sequence xn 2 X converging to x, any sequence yn 2 F ( xn ) has a cluster point y 2 F (x ) . Equivalently, F is CC if its graph is closed and it maps bounded sets to bounded sets. This is a generalization of continuity for set-valued maps. One may note that when F is a single-valued map, it is equivalent to continuity. Let us state a few basic results regarding CC maps: Proposition 1. 1. If F: X ; Y and G: Y ; Z is CC, then so is G ± F.

2. If Fl : X ; Y, l 2 I, is a family of CC set-valued maps, then the intersection \ Fl l2I

is also CC. 3. Let I £X ; Y

(l, x ) 7! Ft ( x )

a CC set-valued map, and let K ½ X be a compact set. Then l 7! Fl ( K) is CC. Theorem 1 follows from the following lemma: Lemma 1.

The set-valued map 2 2 R C 7! F2 n (R ) \ K is CC.

Proof of Theorem 1. It follows from lemma 1 and proposition 1 that the set-valued map 2 7! R ( F2 ) \ K is CC; hence, its domain is closed, that is, the 6 ; is closed. Therefore, if R ( F 0 ) \ K D ;, then set of 2 such that R ( F2 ) \ K D for 2 small enough, R (F2 ) \ K D ;. Proof of Lemma 1. Note that F2 n ( R ) \ K D F2 n ( F2 ¡n ( K ) ). It follows from proposition 1 that (2 , t) 7! F2 n ( t) is CC. Therefore, it is sufŽcient to prove that (2 , t) 7! F2 ¡n ( t) is CC, and since its graph is closed, we need to prove only that it maps bounded sets to bounded sets. That is, we must prove that

306

R. Brette and E. Guigon

for any A > 0 and any compact interval I, the set [ 2 ·A

¡n (I) F2 ¡n (I ) D FA

is bounded. Since the function t 7! Q ( t) ¡ t is bounded, we have lim Qn ( t) D ¡1.

t!¡1

Therefore, Qn maps an unbounded set of R ¡ to an unbounded set. Hence, ¡n (I ) must be bounded. FA Acknowledgments We thank K. Pakdaman and J.-P. Aubin for many helpful discussions and Christine Lienhart and Marc Maier for revising our English. References Arnold, V. I. (1961). Small denominators. I. Mapping the circle onto itself. Izv. Akad. Nauk. SSSR Ser. Mat., 25, 21–86. Ascoli, C., Barbi, M., Chillemi, S., & Petracchi, D. (1977). Phase-locked responses in the Limulus lateral eye: Theoretical and experimental investigation. Biophys. J., 19(3), 219–240. Aubin, J. P., & Frankowska, H. (1990). Set-valued analysis. Boston: Birkh¨auser. Baddeley, R., Abbott, L., Booth, M., Sengpiel, F., Freeman, T., Wakeman, E., & Rolls, E. (1997).Responses of neurons in primary and inferior temporal visual cortices to natural scenes. Proc. R. Soc. Lond. B Biol. Sci., 264(1389), 1775– 1783. Bair, W., & Koch, C. (1996). Temporal precision of spike trains in extrastriate cortex of the behaving macaque monkey. Neural Comp., 8(6), 1185–1202. Beierholm, U., Nielsen, C., Ryge, J., Alstrom, P., & Kiehn, O. (2001). Characterization of reliability of spike timing in spinal interneurons during oscillating inputs. J. Neurophysiol., 86(4), 1858–1868. Berry, M., & Meister, M. (1998). Refractoriness and neural precision. J. Neurosci., 18(6), 2200–2211. Berry, M., Warland, D., & Meister, M. (1997). The structure and precision of retinal spike trains. Proc. Natl. Acad. Sci. USA, 94(10), 5411–5416. Borst, A., & Theunissen, F. (1999). Information theory and neural coding. Nat. Neurosci., 2, 947–957. Buracas, G., Zador, A., DeWeese, M., & Albright, T. (1998). EfŽcient discrimination of temporal patterns by motion-sensitive neurons in primate visual cortex. Neuron, 20(5), 959–969. Coddington, E., & Levinson, N. (1955). Theory of ordinary differential equations. New York: McGraw-Hill.

Reliability of Spike Timing

307

de Ruyter van Steveninck, R., Lewen, G., Strong, S., Koberle, R., & Bialek, W. (1997). Reproducibility and variability in neural spike trains. Science, 275, 1805–1808. Denjoy, A. (1932). Sur les courbes dÂeŽnies par les e quations diffÂerentielles a` la surface du tore. J. Math. Pures. Appl., 9(11), 333–375. Fellous, J. M., Houweling, A., Modi, R., Pao, R., Tiesinga, P., & Sejnowski, T. (2001).Frequency dependence of spike timing reliability in cortical pyramidal cells and interneurons. J. Neurophysiol., 85, 1782–1787. Guttman, R., Feldman, L., & Jakobsson, E. (1980). Frequency entrainment of squid axon membrane. J. Membrane Biol., 56, 9–18. Herman, M. (1977). Mesure de Lebesgue et nombre de rotation. Berlin: SpringerVerlag. Hunter, J., Milton, J., Thomas, P., & Cowan, J. (1998). Resonance effect for neural spike time reliability. J. Neurophysiol., 80(3), 1427–1438. Jensen, R. (1998). Synchronization of randomly driven nonlinear oscillators. Phys. Rev. E, 58(6), R6907–R6910. Kara, P., Reinagel, P., & Reid, R. (2000). Low response variability in simultaneously recorded retinal, thalamic, and cortical neurons. Neuron, 27(3), 635–646. Keener, J. (1980). Chaotic behavior in piecewise continuous difference equations. Trans. Amer. Math. Soc., 261(2), 589–604. Keener, J., Hoppensteadt, F., & Rinzel, J. (1981). Integrate-and-Žre models of nerve membrane response to oscillatory input. SIAM J. Appl. Math., 41, 503– 517. Knight, B. (1972). Dynamics of encoding in a population of neurons. J. Gen. Physiol., 59, 734–766. Koppl, C. (1997). Phase locking to high frequencies in the auditory nerve and cochlear nucleus magnocellularis of the barn owl, Tyto alba. J. Neurosci., 17(9), 3312–3321. Lapicque, L. (1907).Recherches quantitatives sur l’excitation e lectrique des nerfs traitÂee comme une polarisation. J. Physiol. Pathol. Gen., 9, 620–635. Mainen, Z., & Sejnowski, T. (1995). Reliability of spike timing in neocortical neurons. Science, 268, 1503–1506. Nowak, L., Sanchez-Vives, M., & McCormick, D. (1997). Inuence of low and high frequency inputs on spike timing in visual cortical neurons. Cereb. Cortex, 7(6), 487–501. Pakdaman, K., & Tanabe, S. (2001). Random dynamics of the Hodgkin-Huxley neuron model. Phys. Rev. E, 64(5Pt1), 050902. Reich, D., Mechler, F., Purpura, K., & Victor, J. (2000). Interspike intervals, receptive Želds, and information encoding in primary visual cortex. J. Neurosci., 20(5), 1964–1974. Reich, D., Victor, J., & Knight, B. (1998). The power ratio and the interval map: Spiking models and extracellular recordings. J. Neurosci., 18, 10090–10104. Reich, D., Victor, J., Knight, B., Ozaki, T., & Kaplan, E. (1997).Response variability and timing precision of neuronal spike trains in vivo. J. Neurophysiol., 77(5), 2836–2841. Reinagel, P., & Reid, R. (2000). Temporal coding of visual information in the thalamus. J. Neurosci., 20(14), 5392–5400.

308

R. Brette and E. Guigon

Rescigno, A., Stein, R., Purple, R., & Poppele, R. (1970). A neuronal model for the discharge patterns produced by cyclic inputs. Bull. Math. Biophys., 32, 337–353. Tiesinga, P. (2002). Precision and reliability of periodically and quasiperiodically driven integrate-and-Žre neurons. Phys. Rev. E, 65, 041913. Tiesinga, P., Fellous, J. M., & Sejnowski, T. (2002a). Attractor reliability reveals deterministic structure in neuronal spike trains. Neural Comput., 14(7), 1629– 1650. Tiesinga, P., Fellous, J. M., & Sejnowski, T. (2002b). Spike-time reliability of periodically driven integrate-and-Žre neurons. Neurocomputing, 44–46: 195–200. Veerman, J. (1989). Irrational rotation numbers. Nonlinearity, 2(3), 419–428. Victor, J., & Purpura, K. (1996). Nature and precision of temporal coding in visual cortex: A metric-space analysis. J. Neurophysiol., 76(2), 1310–1326.

Received March 15, 2002; accepted July 5, 2002.

LETTER

Communicated by Terrence Sejnowski

The Dynamic Neural Filter: A Binary Model of Spatiotemporal Coding Brigitte Quenet [email protected] Laboratoire d’Electronique, Ecole Superieure de Physique et Chimie Industrielles, Paris 75005, France David Horn [email protected] School of Physics and Astronomy, Raymond and Beverly Sackler Faculty of Exact Sciences, Tel Aviv University, Tel Aviv 69978, Israel We describe and discuss the properties of a binary neural network that can serve as a dynamic neural Žlter (DNF), which maps regions of input space into spatiotemporal sequences of neuronal activity. Both deterministic and stochastic dynamics are studied, allowing the investigation of the stability of spatiotemporal sequences under noisy conditions. We deŽne a measure of the coding capacity of a DNF and develop an algorithm for constructing a DNF that can serve as a source of given codes. On the basis of this algorithm, we suggest using a minimal DNF capable of generating observed sequences as a measure of complexity of spatiotemporal data. This measure is applied to experimental observations in the locust olfactory system, whose reverberating local Želd potential provides a natural temporal scale allowing the use of a binary DNF. For random synaptic matrices, a DNF can generate very large cycles, thus becoming an efŽcient tool for producing spatiotemporal codes. The latter can be stabilized by applying to the parameters of the DNF a learning algorithm with suitable margins. 1 Introduction

Three types of neural network paradigms (Hertz, Krogh, & Palmer, 1991; Peretto, 1992) were developed two decades ago. Two of them were based on supervised learning, in order to construct feedback attractor networks (HopŽeld, 1982) for associative memory and feedforward multilayer perceptrons (Rumelhart, Hinton, & Williams, 1986) for functional representation. The third used unsupervised learning to produce self-organized maps (Kohonen, 1982). These three conventional paradigms of neural computation were formulated in terms of mathematical neurons that are very different from biological reality, and it is still unclear how, if at all, biological neural Neural Computation 15, 309–329 (2003)

° c 2002 Massachusetts Institute of Technology

310

B. Quenet and D. Horn

networks perform any of them. Over the past decade, interest in theoretical neuroscience (Dayan & Abbott, 2001) has shifted to analyzing neural networks of spiking neurons with dynamical synapses in the hope of getting closer to the natural roles and purposes of the relevant biological networks. We will argue that it is nevertheless timely to consider yet another model of binary neurons with discrete temporal dynamics and show its applicability to a biological system. An interesting issue in neuroscience is the question of spatiotemporal coding. Its existence has been demonstrated (Wehr & Laurent, 1996) in the locust olfactory system, where the spatiotemporal behavior of projection neurons encodes the odor presented to its receptor neurons. This transformation from odor input to spatiotemporal activity occurs in the antennal lobe, that is, the Žrst module of the olfactory system. This system may therefore be regarded as a dynamic neural Žlter that turns spatial information distributed over its many glomeruli, fed by the receptor neurons, to speciŽc spatiotemporal outputs. Although this is a complicated biological system, it has an important simplifying feature that allows it to be represented by a simpliŽed model of mathematical neurons: the fact that the activity of the projection neurons is limited to temporal bins deŽned by an oscillatory local Želd potential. Hence, a model with binary neurons obeying HopŽeld-Little dynamics (Peretto, 1992) can provide a valid Žrst-order approximation of the spatiotemporal behavior of that system. We study a recurrent model of this kind using an asymmetric coupling matrix that allows for the generation of large temporal sequences. The novelty of our approach is that we use this model as a dynamic neural Žlter (DNF), relating input space to spatiotemporal behavior of the recurrent network. As such, it does not correspond to any of the three paradigms noted above, it does not come to rest at Žxed points, and it is not necessarily based on supervised learning. We have presented this model in a previous work (Quenet, Horn, Dreyfus, & Dubois, 2001) and demonstrated how it can be used to generate the spatiotemporal behavior of projection neurons observed by Wehr and Laurent (1996). Here we discuss several fundamental issues concerning this model, such as its stability and coding capacity. Moreover, we provide an algorithm for constructing a DNF that can generate a given set of spatiotemporal data. This algorithm is used to deŽne and characterize the complexity of the data set of Wehr and Laurent (1996). It should be emphasized that the latter is used only as a relevant example, and we do not claim that the DNF is a realistic representation of the biological reality. Nonetheless, as in the conventional three paradigms, it can be used as a schematic model that will subserve further detailed modeling of biological systems. 2 The Model 2.1 Short Biological Motivation. The antennal lobe of insects serves as the Žrst stage of olfactory computation, using as input signals of chemical

The Dynamic Neural Filter

311

receptors and transforming them into outputs of projection neurons (PN) to the next stage of the olfactory tract (the mushroom body). In the locust, there are about 800 excitatory PN cells (Laurent and Naraghi, 1994) and a slightly smaller number of inhibitory local interneurons (LN) that form dendrodendritic interactions with the PNs and with themselves, and receive receptor information as well. The output of this system is that of PNs only; hence, we will consider our model as a rough sketch of PNs, with interactions that should be regarded as effective interactions mediated by the LNs. The antennal lobe exhibits a reverberating Želd potential (LFP) at a frequency of about 20 Hz. This LFP reverberates for a few cycles (up to tens of them) and then quiets down until it begins to repeat such behavior. (See Laurent et al., 2001, for details.) The PNs Žre within the up-phase of the LFP; hence the latter may be regarded as deŽning an effective clock, with time bins of, for example, 30 ms of activity and 10 ms of rest. This motivates us to use a simpliŽed model with discrete temporal dynamics. 2.2 The Recurrent Network. In our model, a binary neuron i, representing a PN, may either Žre, ni D 1, or be quiescent, ni D 0, in a given temporal bin of the LFP. There are N neurons in the model obeying the following HopŽeld-Little dynamics,

0 ni (t C 1) D H (hi ( t C 1) ) D H @

1 X j

wij nj ( t) C Ri ¡ hi A ,

(2.1)

where wij is the synaptic coupling matrix, Ri is the external constant input (specifying odor activation), and hi is the threshold. H is the Heaviside step function taking the values 0 for nonpositive arguments and 1 for positive ones. This model can be readily generalized to account for the presence of noise by replacing the deterministic rule by the stochastic one, Prob[ni ( t C 1) D 1] D

1 . 1 C e¡hi ( t C 1) /2

(2.2)

where 2 is the noise parameter. We will study the stability of results of the deterministic dynamics to the noise introduced by the stochastic dynamics. 2.3 Deterministic Dynamics. In order to analyze the one-step dynamics of equation 2.1, we deŽne a Lyapunov function L that describes just this onestep, in the sense that, considering all dynamical states at time t C 1, it will be minimal for the state that is a solution to these dynamics. Let us deŽne the initial and Žnal states,

nIi D ni (t )

nFi D ni ( t C 1) ,

(2.3)

312

B. Quenet and D. Horn

relevant to the one-step of equation 2.1. Clearly, I and F are just two of the 2N states that the system of neuron possesses. We can prove that L JI D ¡

X ij

J wij ni njI ¡

X i

J ni ( Ri ¡ hi )

(2.4)

obtains its unique minimum on the state J D F. Consider 0 1 X JI J J ki D ¡(2ni ¡ 1) hIi D ¡(2ni ¡ 1) @ wij njI C Ri ¡ hi A .

(2.5)

j

For J D F, this quantity is negative for every i, following the deterministic dynamics. For any other J, there will be elements i for which the sign will P JI JI ip. Hence, i ki will be minimal for J D F. We note in equation 2.5 that ki J JI contains terms that are dependent on ni , to be denoted by 2li where JI li

D

¡niJ

0 1 X @ wij njI C Ri ¡ hi A ,

(2.6)

j

J

and terms that are independent of ni . Since the latter are common for all J, P JI the minimum of i ki will be reached for the same J as the minimum of P JI L JI D i li , the Lyapunov function of equation 2.4. 2.4 Stochastic Dynamics. The one-step Lyapunov function, equation 2.4, plays an important role in the stochastic dynamics.1 As we will see, the probability of obtaining a state J after starting from a state I can be written in terms of this function:

e¡L /2 P ( J | I ) D P ¡LKI 2 . / Ke JI

(2.7)

To prove this result, we start from the probability of obtaining the state F, that is, the one following from the deterministic dynamics. It is straightforward to show that P (nFi | I ) D

1 FI 1 C eki /2

I

(2.8)

1 It was applied to a model with symmetric synaptic couplings by Quenet, Cerny, Dreyfus, and Lutz (1997).

The Dynamic Neural Filter

313

hence, P (F | I ) D P i P ( nFi | I ) .

(2.9)

Any other state J differs from the state F by some ips of neuronal states, J for example, nm D 1 ¡ nFm . In such a case, P ( J | I ) will be the probability of obtaining the corrupted state F, having the wrong digit in its mth place, FI which is P ( F | I ) e km /2 . But at this location m, kFI m D

1 FI JI JI ( k ¡ km ) D lFI m ¡ lm . 2 m

(2.10)

JI Since for all other neurons lFI i D li , it follows that FI

P ( J | I ) D P (F | I ) e ( L

¡L JI ) /2

.

(2.11)

This will be true for any state regardless of how many ips occur because of their independent nature. Hence, equation 2.7 is proved. The stochastic dynamics of equations 2.1 and 2.2 lead to a homogeneous and irreducible Markov process; the transition matrix is time invariant, and all elements of the transition matrix, T JI D P ( J | I ) ,

(2.12)

are strictly positive. Such a Markov process has a stationary probability distribution, p ( I ), that is, an eigenvector of the transition matrix with the highest eigenvalue, 1: X (2.13) T JI p ( I ) D p ( J ) . I

From this stationary probability distribution, obtained asymptotically in the evolution of the system, one can calculate the asymptotic probabilities of the transitions p ( I ! J ) : p (I ! J ) D P ( J | I ) p (I ) .

(2.14)

The stationary Markov process has an entropy rate (Cover & Thomas, 1991), X HD¡ (2.15) p (I ) T JI log2 T JI , IJ

that serves as a lower bound on the expected dimension of any binary code of the Markov process. Since in our case all states are described by vectors of length N, it follows that

H · N.

(2.16)

The upper limit is approached when the noise parameter 2 is high and all transition matrix elements tend to be equal.

314

B. Quenet and D. Horn

3 Numerical Examples

We propose to view the model described in the previous section as a Žlterrelating input space Ri to spatiotemporal sequences of the recurrent network. In this section, we will demonstrate numerically how the DNF acts. For simplicity, we choose both wij and Ri to have positive and negative integer values, while Žxing hi D 12 . This means that Ri will now effectively represent both input and threshold of neuron i. 3.1 Mapping of Input Space. Once the matrix w is given, there is a

restricted range of interest for R: ¡

X j

wij H (wij ) · Ri · ¡

X j

wij H (¡wij ) C 1.

(3.1)

Outside this range the dynamics becomes trivial because the input determines the neuronal values directly. As a Žrst example of deterministic dynamics, we study an N D 2 system because it can be easily mapped in an exhaustive manner. We choose the synaptic matrix ³ ´ 1 2 wD ¡2 ¡1 and use an initial null state ni (0) D 0. The dynamics generate 14 different sequences, including the four (2N ) Žxed points, several two-cycles, and one four-cycle. In Figure 1, we display the coding zones, that is, the ranges of R corresponding to a single temporal sequence. The overall range of R space is chosen to be somewhat larger than the range of equation 3.1. In its center, we Žnd that many different sequences are produced (small coding zones), while in the periphery, a single sequence dominates. Figure 2 shows the length of each sequence on the same R plane. We deŽne it as the length of the sequence the model generates before one of its states is being repeated. This deŽnition of length takes into account both the order of the cycle and the transition time it takes to reach it. The interesting (i.e., long) sequences appear near the center of R space. The periphery is dominated by Žxed points and two-cycles. Building on this experience, we next study an N D 5 system with the synaptic matrix 0 1 0 ¡2 ¡5 ¡3 0 B 6 2 8 ¡14 0C B C 1 1 0 ¡2 1C wDB B C. @¡4 6 1 1 3A 4 ¡1 2 ¡4 0

Label of the spatio-temporal sequence

The Dynamic Neural Filter

315

14 12 10 8 6 4 2 0 10 5 Input R2

0 -5

-6

-4

-2

0

2

4

6-

Input R1

Figure 1: Ranges of input space in a two-neuron problem that lead to identical spatiotemporal behavior, deŽned as coding zones. The label of each code, or spatiotemporal behavior, varies from 1 to 14 in this problem.

Now much larger sequences can be generated, and we would like to see the ensuing mapping of input space and test the stability of the sequences to noise. In order to be able to view the results, we choose R3 R4 R5 to lie in the center of their expected range, as speciŽed by equation 3.1, and study the system in the R1 R2 plane, limited to their relevant ranges. First, let us view, in Figure 3, the coding zones of different sequences in R space. Their average size is 10; that is, there is an order of magnitude of information compression from R space to the spatiotemporal sequences. The lengths of the latter are displayed in Figure 4. They include many cycles of lengths 4, 5, and 6. The coding zones are the analogs of basins of attraction, which signify domains of initial conditions that lead to the same attractor. Here, we stay with a Žxed initial condition (the null state) and search for domains in input space that lead to the same temporal sequence (transition into a cycle). Note the large number of different sequences obtained in this problem. It is exponentially larger than the number of Žxed points that we are accustomed to in attractor neural networks (describable by the same dynamics with symmetric synaptic matrices). We estimate, on the basis of simulations, that the total number of sequences in the Žve-dimensional R space is of the order of 3800. In the two-dimensional section of R space shown in Figure 3, we Žnd 38 different sequences, whose lengths are displayed in Figure 4.

316

B. Quenet and D. Horn

Sequence length

4

3

2

1 10 5 R2

0 -5

-8

-6

0

-2

-4

2

4

6

R1

Figure 2: Sequence length of the N D 2 problem mapped on the two-dimensional input space. Table 1: Six Neighboring Sequences in a System with Five Neurons, Displayed over Seven Time Steps. R2

DE

DH7

tD1

2

3

4

5

6

7

¡15 ¡12 ¡8 ¡3 2 8

0 2 4 5 6 7

0 1 2 10 11 7

17 17 17 17 25 25

22 22 22 30 30 30

6 14 14 16 16 16

8 8 16 3 3 11

3 3 3 17 17 3

17 17 17 30 30 17

22 22 22 16 16 30

3.2 Distances Between Codes. The mapping of R space into spatiotemporal codes can also serve as deŽning a distance between different codes. It is thus interesting to compare different sequences generated over a neighborhood of R space, as seen in Table 1. All of these sequences can be read off as we move along the R2 axis at Žxed R1 D 4. The sequences are ordered according to increasing value of R2 , where we indicate the central value of the coding zone, and the states for each time bin are written in the biP nary representation 1 C 5iD1 ni 25¡i . The columns DE and DH7 describe two distances, deŽned below, from the Žrst sequence.

Label of the spatio-temporal sequence

The Dynamic Neural Filter

317

40 35 30 25 20 15 10 5 0 20 10 R2

0 -10 -20

0

2

4

6 R1

8

10

12

Figure 3: Coding zones of spatiotemporal sequences of an N D 5 problem, plotted on a plane deŽned by two of the input variables. Other inputs are held constant at their central values.

As one moves along from the Žrst sequence to the second, and then to the third, only one state changes over the observed range of T D 7. Moreover, this change corresponds to the ipping of just one neuron (n2 at t D 3 between the Žrst and second sequence). Proximity in R may therefore be related to proximity in the identity of states appearing in two sequences, or proximity in the sense of Hamming distance between all neurons at all time steps. Thus, we deŽne three distances between sequences: DE is the “edit distance,” deŽned as the number of insertions and deletions needed to change one sequence into another, limiting oneself to the natural length of each sequence, that is, until one of the states reappears. DH is the Hamming difference between the two spatiotemporal patterns of neural activities. Clearly this grows with the length of the sequence. In the example of Table 1, where we limit ourselves to seven time steps, we list the corresponding seven-step Hamming distance, DH7 . Finally, we may deŽne DR , the Euclidean distance between the centers of the coding zones in R space. Table 1 lists the values of DE and DH7 distances from the Žrst sequence in this table. Using 18 sequences of this problem, including the six shown in Table 1, we Žnd a high correlation (0.87) between DE and DR and a lower cor-

318

B. Quenet and D. Horn

Sequence length

7 6 5 4 3 2 1

20 10

15 R2

0

10 -10

5 -20

0

R1

Figure 4: Sequence length of the 38 sequences mapped out in Figure 3.

relation (0.56) between DH7 and DR . The latter signiŽes the fact that whereas the Hamming distance correlates well with R distance for some sequences, there are cases where several neurons ip as one moves from one code to the next. In Table 1, this occurs between the third and the fourth sequences and between the Žfth and the sixth ones. Note, however, that in the fourth sequence, the state 30 replaces two states, 22 and 14, of the third sequence. Therefore, DE between the two will be only 3. This explains in what sense the two sequences should be regarded as close to one another, as also borne out by the short distance in R. These results can also be studied from the viewpoint of the individual neurons. Since the Žrst line has the lowest value of R2 , none of its states involves an active neuron number 2, that is, all have n2 D 0. Moving up to R2 D ¡12, n2 is activated in state 14. At the next stage, it is activated in state 16. Further increases of R2 lead to activation of neuron number 2 in the states 30 and 25. Thus, we observe a gradual increase in the number of states that contain an active neuron number 2. Since the volume of state-space is relatively moderate, 32, we Žnd that in spite of the change in the identity of the state, for example, a change of 6 to 14 in step t D 3 between the Žrst and the second row, the next state remains 8, and the rest of the sequence is reiterated. This leads to the good correlation with DE . We believe that this

The Dynamic Neural Filter

319

property may be useful in DNF applications to the study of problems such as gene sequences, where low N models may apply. The correlation between R distance and edit distance may be lost at large N. The reason is that the volume of state-space grows like 2N . Hence, once a change occurs in one state, the next transition may point to another state in this large space, and the ensuing sequence may change completely. Numerical trials in an N D 50 model have shown this to be the case. Thus, the correlation of nearby sequences, as deŽned by proximity in R space, may be lost in large neural networks. 3.3 Stability Under Stochastic Dynamics. Next, let us ask how robust these sequences are against noise. We choose 2 D 0.5, which is where considerable effects may be expected, since this is the lowest absolute value that the potentials hi can have. As an example, let us look at the probability of a correct four-step sequence,

P4 D P ( F4 | F3 ) P ( F3 | F2 ) P (F2 | F1 ) P ( F1 | 1),

(3.2)

where 1 designates the initial null state. This is the product of the probabilities of obtaining the correct Žrst four states of the appropriate deterministic sequence at a given point in R space. The results for the problem at hand are plotted in Figure 5 on the same grid of R space as in the two previous Žgures. We see that a few sequences are relatively stable, but others have a low probability of being correctly recovered during the Žrst four steps of this Markov process. To exemplify the origins of instability we compare two six-cycles, at locations (10, ¡10) and (10, 15) of Figures 4 and 5. In Figure 6 we plot the histograms of the hi values (over all neurons for the Žrst four steps of the dynamics) for these two cases. Clearly, at 2 D 12 , the occupancy of the bins of h D § 12 will determine the instability following from equation 2.2. Since the sequence at (10, ¡10) has three elements in these bins, it is reduced to P4 D 0.36. All other sequences in Figure 5 have three elements in these bins or more. The sequence at (10, 15) has Žve elements in these bins, as can be seen in Figure 6; hence, its retrieval probability is reduced to P4 D 0.18. Let us remark at this point on the Markov chain properties discussed in section 2.4 and, in particular, the stationary probability distribution p ( I) of equation 2.13. Using the R-space point (10, ¡10) as an example, we Žnd that the following states are the most probable ones: 17, 22, 30, 32. Their respective probabilities are 0.106, 0.175, 0.173, 0.169. In the problem at hand, we start with state 1 as the initial condition, looking for the sequence 1 ! 17 ! 22 ! 30 ! 32. The relevant transition probabilities for the terms in equation 3.2 are 0.98 £ 0.95 £ 0.73 £ 0.53, leading to P4 D 0.36. Obviously, the major reduction of the probability occurs at the Žrst two steps, from state 1 to 17 and from 17 to 22. This is also where the three appearances of

320

B. Quenet and D. Horn

0.4 0.35 0.3 P4

0.25 0.2 0.15 0.1 0.05 0 20 10 R2

0 -10 -20

0

2

4

6

8

10

12

R1

Figure 5: Probability of recovering correctly the Žrst four time steps in the sequences of Figures 3 and 4 when stochastic dynamics is being used with 2 D 0.5.

h D § 12 occur: two of them affect the transition of 1 to 17, and one occurs in the transition from 17 to 22. 3.4 Entropy of the Markov Chains. Finally, we show in Figure 7 an analysis of the entropy of the different Markov chains of this system. This is done for 2 D 0.5 and can be compared with the P4 results in Figure 5. The Markov chains with low entropy are the ordered ones, leading to a relatively high probability of correct retrieval of the Žrst four states. The chains with high entropy are the disordered ones, leading to a low probability of correct retrieval. 4 The Inverse Problem

The previous section exhibited examples of what may be called the direct DNF problem: given a matrix w and a set of R values, solve the dynamics of equation 2.1, and generate the resulting sequence. We now wish to pose an inverse problem: given a set of several sequences, Žnd a corresponding DNF, that is, values of w and R that can produce this set. There are two parts to this question. First, does a solution exist? Second,

The Dynamic Neural Filter

321

Figure 6: Histograms of hi values of two six-cycles in the N D 5 problem. Table 2: Six Spatiotemporal Sequences DeŽned for Four Time Steps in a System with Four Neurons. t/ k

1

2

3

4

5

6

1 2 3 4

1100 1110 1101 0001

1000 1100 1101 0001

1110 1111 0111 0 01 1

1000 1 01 0 0110 0111

1 01 1 1000 1110 1111

1000 1110 0111 0 0 01

if the problem is soluble, Žnd an example. This may be expanded into a search for all possible solutions, which lies outside the scope of this work. 4.1 The Existence Problem. Suppose a given data set comprises K different spatiotemporal sequences of length T each, presented as activities of N neurons. An example is shown in Table 2 for K D 6, T D 4 in a system with N D 4 neurons. We wish to Žnd out if there exists a DNF such that this set of sequences is elicited by K different inputs Rik, k D 1, . . . , K.

322

B. Quenet and D. Horn

1.8 1.6

Entropy

1.4 1.2 1 0.8 0.6 0.4 0.2 20 10 R2

0 -10 -20

0

2

4

6

8

10

12

R1

Figure 7: Entropy of the Markov chains corresponding to the sequences of Figures 3 and 4 when stochastic dynamics is being used with 2 D 0.5.

The neuronal values may be labeled nik ( t) , t D 1, . . . , T. Each time step, for each sequence, deŽnes a state of the system | k, ti D fnik ( t)giD 1,...,N .

(4.1)

Twenty-four such states can be seen in Table 2, some of them identical, yet no identical states appear in a single sequence. Each state |k, ti follows from the previous state |k, t ¡ 1i through deterministic dynamics. The initial state |k, 0i is chosen as the null state nik D 0 for all i D 1, . . . , N and k D 1, . . . , K. Consider a single neuron i in this system. We may then reduce the question posed here into a perceptron problem for this neuron. The neuron is given KT initial vectors (counting states from t D 0 to t D T ¡ 1) and its required output values nik ( t) (from t D 1 to t D T for all k D 1, . . . , K). It has to perform calculation 2.1 given a different input Rik for each of the K sequences. To take account of the K different inputs (biases), we extend the vector states of length N to new vectors of length N C K through the concatenation | k, t) D |k, ti £ fda,kgaD 1,...,K ,

(4.2)

where da,k is the Kronecker delta function obtaining the value 1 for a D k and 0 otherwise. The extension by K bias axes allows us to represent the

The Dynamic Neural Filter

323

deterministic dynamics, for each neuron i, as KT perceptron inequalities in an N C K dimensional system. These can be strictly met (Hertz et al., 1991) for N obeying

A:

KT · N C K

or

K (T ¡ 1) · N.

(4.3)

In the large N limit, one can apply the Cover result (Cover, 1965), saying that a solution may be found for

B:

KT · 2(N C K )

or

1 K (T ¡ 2) · N. 2

(4.4)

For the example of Table 1, the strict condition A implies N ¸ 18, and condition B leads to N ¸ 6. This could mean one should extend the system shown in Table 2 by at least two, if not more, neurons. Nonetheless, one can obtain a solution for N D 4. This is due to the fact that the states in this table were not chosen in a random and independent fashion. 2 Note also the example discussed in section 3.1. Thirty-three of the 38 sequences in Figure 4 have length 4 or more. Counting only the Žrst four states, we Žnd 22 different sequences, many more than conditions A and B would suggest. This is so because we did not ask in section 3.1 for the matrix that can produce an arbitrary choice of states appearing in K sequences of length 4, but counted how many different sequences were produced by a given matrix. These sequences turned out to be quite similar to one another, as described in section 3.2. Thus, conditions A and B should be regarded as sufŽciency conditions, assuring us A that a solution must exist and B that it may be found for lower N. With more effort, one can try to solve the inverse problem, that is, Žnd an explicit matrix w , for yet smaller N. 4.2 Perceptron Learning. From our approach to the existence problem, it should be quite evident that if a solution exists, it may be obtained using a straightforward extension of the Rosenblatt algorithm (Rosenblatt, 1962; Hertz et al., 1991). We describe it in this section, realizing explictly the abstract vectors in N C K dimensions described in the previous section. Let us start by deŽning, for each neuron i, an N C K–dimensional vector of perceptron weights w E i,

(w E i ) j D wij

for j D 1, . . . , N

(w E i ) N C k D Rik ¡ hi

for k D 1, . . . , K.

(4.5)

The sequence states of interest will be represented by vectors xE i, k ( t) , for neuron i, living in the same N C K–dimensional space but carrying further 2

The rationale for the choice of states in Table 2 is discussed in section 5.3.

324

B. Quenet and D. Horn

indices of k and t as follows: ( xE i,k (t) ) j D njk ( t) (2nik ( t C 1) ¡ 1)

for j D 1, . . . , N, t D 0, . . . , T ¡ 1

(xE i,k ( t) ) N C a D dak (2nik (t C 1) ¡ 1)

(4.6)

for a D 1, . . . , K.

In other words, the vectors xE i, k ( t) represent the states weighted by a positive or negative sign depending on whether the target neuron i at the next time step will be 1 or 0, respectively. With these deŽnitions, all constraints of the inverse problem can be written as KT perceptron inequalities: Ei ¢ x E i,k ( t) > 0 for all k D 1, . . . , K, t D 0, 1, . . . , T ¡ 1. w

(4.7)

We propose to use the perceptron learning rule (Hertz et al., 1991),

D wE i

E i, k ( t) H (¡w E i ¢ xE i,k ( t) ) , D gx

(4.8)

while iterating the system, time and again, over all KT states. The Heaviside function guarantees that w E i gets modiŽed at a given iteration by the vectors xE i, k (t ) that do not satisfy the inequality. This algorithm converges (Hertz et al., 1991) if the system of inequalities is soluble. Moreover, one can use the same algorithm to require stability under stochastic dynamics by insisting on a margin M in the perceptron inequalities: Ei ¢ x E i,k ( t) > M. w

(4.9)

This leads to the learning rule:

D wE i

E i, k ( t) H (¡w E i ¢ xE i,k ( t) C M ) . D gx

(4.10)

In section 3, where we used integer values of w and R and kept all h D 12 , we worked with an implicit margin of size 12 . This is the reason that testing the system with stochastic dynamics of 2 D 12 , we found considerable effects. Insisting on larger M, if the data allow it, will guarantee stability for larger ranges of 2 . Given a situation of the type encountered in Figure 5, one may want to improve the robustness of a possible code, such as the one located at (10,15) in the R plane. This can be tried by applying the algorithm presented by the previous equation for a suitably chosen M. The size of M is, of course, limited by the data. (For a general discussion of optimal margin selection, see Vapnik, 1995.) If the algorithms are implemented with g D 1, they lead to integer values of w and R ¡h. An arbitrary choice of g < 1 can lead to continuous h values. It is then advisable to keep a Žnite M to ensure some robustness to noise. Finally, we wish to comment that one may apply in a similar manner the Ho-Kashyap procedure (Duda, Hart, & Stork, 2001), indicating, if it does not converge, that a given set of data is not linearly separable.

The Dynamic Neural Filter

325

5 Capacity and Complexity 5.1 Capacity of Codes of Given Length. The DNF produces sets of spatiotemporal codes. Its capacity could be measured by the number of codes it can produce. This is a very large number and may not be very meaningful if we consider an application with stochastic dynamics. Moreover, what would be really interesting is a measure of capacity of random codes, expected to be very distant from one another. Clearly, two codes that are close will not constrain the system. If one is learned, some other close-by codes will be automatically produced for nearby R values. Suppose we look for the number of codes of length T such that the same state does not appear twice, either within the same code or among the different codes. This would be the case if the states are chosen randomly from all 2N possibilities of an N-neuron system. Following the reasoning of section 4.1, this number can be expected to reach

CT ¼

2N T ¡2

(5.1)

if criterion B is applied. CT may be viewed as a capacity measure of a DNF. Note that the last equation leads to CT D 1 for T D 2N C 2. This means that a DNF of order N may be expected to accommodate a cycle of length 2N C 2. The latter may then be used to construct the DNF. 5.2 Random Synaptic Weights. There exist synaptic matrices that do not have the ability to generate large cycles. It is well known that symmetric matrices w lead to Žxed points or two-cycles and antisymmetric matrices may lead up to four-cycles only (Peretto, 1992). In general, however, a system of N neurons may generate large cycles, limited by the total number of states, 2N . Gutfreund, Reger, and Young (1988) have shown that for Ri D 0, large cycles are generated when values of wij are chosen randomly from a gaussian P distribution centered at zero and thresholds hi D 12 j wij ¼ 0.3 This was further studied numerically by Nutzel ¨ (1991), who pointedPout that large

cycles can be obtained for some range of the asymmetry a D Pij ij

wij wji wij wij

around

the point 0 of random asymmetry.4 The average size of the cycles grows as an exponential in N, with an exponent that increases as the asymmetry 3 Related questions within the framework of Ising spin systems have been studied by Crisanti and Sompolinsky (1988). 4 This deŽnition of asymmetry differs from the parameter used by Gutfreund et al. (1988) and Nutzel ¨ (1991), but it shares the relevant characteristics of varying between 1 and ¡1, with the extremes characterizing the symmetric and antisymmetric cases. The completely asymmetric case corresponds to a D 0. The example of the Žve-dimensional matrix in section 3 has a D ¡0.4067.

326

B. Quenet and D. Horn

Table 3: Binary Odor Coding. t/ k

1

2

3

4

5

6

1 2 3 4

11 11 11 00

10 11 11 00

11 11 01 00

10 10 01 01

10 10 11 11

10 11 01 00

approaches zero. A system with a similar structure but slightly different dynamics was studied by McGuire, Littlewort, and Rafelski (1991). Working with nonzero R values in our DNF, we noted that the interesting large cycles occur around the center of the relevant range of R. This is where the average effect of the inputs cancels the average effect of the synaptic weights, in agreement with the observations of Gutfreund et al. (1988). Thus, we expect, for any given N, that matrices w that have large capacities CT for large T have a random-like distribution of synaptic values. We are, however, content even with cycles of order N, as is the case of our examples in section 3. Note that in the locust antennal lobe (see section 2.1), hundreds of neurons are participating in the real biological system. Hence, it is quite reasonable for it not to display cyclic behavior over the few LFP bins over which it is measured. 5.3 An Example: The Wehr-Laurent Experiment. We return now to the example that motivated our investigation. Wehr and Laurent (1996) display in Figure 3 of their article the response of two speciŽc neurons to nine mixtures of odors. SigniŽcant results were obtained during the Žrst four reverberations of the local Želd potential. They can be presented as six different binary codings, shown in Table 3, three of which appeared twice in the nine odor mixtures. Each of the six columns speciŽes the state of activity of the observed pair of neurons for one of the six spatiotemporal codes. Looking at the Žrst column, it is clear that it cannot be produced by a DNF, where states at time t are determined by states at t ¡ 1, unless it possesses at least two hidden neurons. Hence, we conclude that this should be treated as a T D 4, K D 6 problem. Looking for a system with C4 ¼ 6, we know we can do it with N D 18 (using condition A), but may expect to be able to do it with N D 6 (using condition B) or less. It turns out we can do it with N D 4. A set of states of four neurons, which accommodate the two observed neurons of Table 3, is given in Table 2. The latter was constructed by choosing states of hidden neurons by trial and error until all perceptron inequalities were satisŽed, ensuring the existence of DNFs with matrices w and inputs Rk that can generate this table. Thus, we are able to implement the binary odor coding of Wehr and Laurent (1996) in an N D 4 model.

The Dynamic Neural Filter

327

5.4 Minimal DNF as Measure of Complexity. The example of the WehrLaurent experiment raises the possibility of using the DNF model to deŽne the complexity of a spatiotemporal data set. This degree of complexity can be characterized by the minimal number of neurons N needed to accommodate the data within a DNF. From the discussion in the previous paragraph we conclude that the Wehr-Laurent problem of Table 3 has DNF complexity of degree 4.5 6 Discussion

The problem of odor spatiotemporal encoding was recently reviewed in detail by Laurent et al. (2001). As a theoretical model, these authors suggest “winnerless competition,” a concept that was expanded in Rabinovich et al. (2000, 2001). Basing their intuition on Lotka-Volterra equations they point out that for suitably chosen parameters, there exist heteroclinic orbits connecting all N attractors of the N-dimensional system, which are very sensitive to external inputs. They studied systems of FitzHugh-Nagumo neurons that exhibit spatiotemporal sensitivity to external inputs as a neural implementation, but made no attempt to Žt a particular data set such as the one of Wehr and Laurent (1996). In comparison, we use a binary model with discrete dynamics, but nevertheless we claim that it is relevant to the observed data. The reason is that the periodic LFP in the antennal lobe provides a temporal scale that accounts for speciŽc time bins, and allows discrete coding, as observed by Wehr and Laurent (1996). With a DNF model that Žts these data, one can envisage a model of spiking neurons capable of mimicking the experimental results (Quenet, Dubois, Sirapian, Dreyfus, & Horn, in press). This can be done by imposing overall periodic inhibition that allows for action potentials during periodic time windows and by Žtting synaptic delay parameters to match the same windows. Recently Friedrich & Laurent (2001) have observed spatiotemporal odor representations in the olfactory bulb of zebraŽsh. This system also has a reverberating local Želd potential and mitral cells that Žre in coincidence with it. The authors have studied the response of this system to similar odors, characterized by small changes in molecular structures of the relevant chemicals. One of their interesting results is that the correlation between temporal patterns of similar odors over the mitral cells reduces with time (between the Žrst and second 500 ms after odor presentation). This contradicts the expectation that short distances in R space lead to high correlations between the spatiotemporal sequences. The reason is the large value of N in

5 Obviously this should be regarded as a mathematical statement and may have nothing to do with the biological reality. In the antennal lobe, one Žnds excitation of hundreds of PNs during each odor presentation.

328

B. Quenet and D. Horn

this system. Since the volume of state-space is 2N , once a change in a sequence occurs, it may lead to a new sequence uncorrelated with the old one. Our model can be compared with recent work by Maass, Natschl a¨ ger, and Markram (in press), who propose a recurrent neural network that performs what they call “liquid computation.” Their network is composed of spiking neurons and is structured in two stages: (1) a Žlter transforming spatiotemporal input into spatiotemporal behavior of the network and (2) the activation of a readout map. Thus, this network projects spatiotemporal input into an output representation that is spatial in nature. Although these authors put special emphasis on real-time learning based on perturbations, the common feature of their approach and ours is the use of a Žlter projecting from an input to an output space. Whereas in our model the input space is spatial and the output space is spatiotemporal, their model goes in the opposite direction. However, in general, both models can connect spatiotemporal spaces to one another, and both use neural networks as Žlters, performing computations that are different from the conventional major paradigms. Our model is based on binary mathematical neurons, whereas Maass et al. (in press) deal with more realistic neural behavior. Nevertheless, the advantage of our DNF is that its structure can be made explicit, thus allowing full understanding of its operation and capabilities. Among other things, we can use this understanding to solve an inverse problem: construct a DNF that produces a given repertoire of spatiotemporal data. Acknowledgments

We thank G. Dror for advice and assistance. References Cover, T. M. (1965). Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition. IEEE Trans. Elect. Comp., 14, 326–334. Cover, T. M., & Thomas, J. A. (1991). Elements of information theory. New York: Wiley. Crisanti, A., & Sompolinsky, H. (1988). Dynamics of spin systems with randomly asymmetric bonds: Ising systems and Glauber dynamics. Phys. Rev. A, 37, 4865–4874. Dayan, P., & Abbott, L. F. (2001). Theoretical neuroscience. Cambridge, MA: MIT Press. Duda, R. O., Hart, P. E., & Stork, D. G. (2001). Pattern classiŽcation. New York: Wiley. Friedrich, R. W., & Laurent, G. (2001). Dynamic optimization of odor representations by slow temporal patterning of mitral cell activity. Science, 291, 889–894.

The Dynamic Neural Filter

329

Gutfreund, H., Reger, D. J., & Young, A. P. (1988). The nature of attractors in an asymmetric spin glass with deterministic dynamics. J. Phys. A: Math. Gen., 21, 2775–2797. Hertz, J., Krogh, A., & Palmer, R. G. (1991). Introduction to the theory of neural computation. Reading, MA: Addison-Wesley. HopŽeld, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proc. Nat. Acad. Sci. USA, 79, 2554–2558. Kohonen, T. (1982). Self-organized formation of topologically correct feature maps. Biol. Cyb., 43, 59–69. Laurent, G., & Naraghi, M. (1994). Odorant-induced oscillations in the mushroom bodies of the locust. J. Neurosciences, 14, 2993–3004. Laurent, G., Stopfer, M., Friedrich, R. W., Rabinovich, M. I., Volkovskii, A., & Abarbanel, H. D. I. (2001). Odor encoding as an active dynamical process: Experiments, computation and theory. Ann. Rev. Neurosci., 24, 263–297. Maass, W., Natschl¨ager, T., & Markram, H. (in press). Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural Computation. McGuire, P. C., Littlewort, G. C., & Rafelski, J. (1991). Brainwashing random asymmetric “neural” networks. Physics Letters A, 160, 255–260. Nutzel, ¨ K. (1991). The length of attractors in asymmetric random neural networks with deterministic dynamics. J. Phys. A: Math. Gen., 24, L151–L157. Peretto, P. (1992). An introduction to the modeling of neural networks. Cambridge: Cambridge University Press. Quenet, B., Cerny, V., Dreyfus, G., & Lutz, A. (1997). A dynamic model of key feature extraction: The example of olfaction. Unpublished manuscript, ESPCI. Quenet, B., Dubois, R., Sirapian, S., Dreyfus, G., & Horn, D. (in press). Two step modeling of spatiotemporal olfactory data: From binary to Hodgkin-Huxley neurons. Biosystems. Quenet, B., Horn, D., Dreyfus, G., & Dubois, R. (2001). Temporal coding in an olfactory oscillatory model. Neurocomputing, 38–40, 831–836. Rabinovich, M. I., Huerta, R., Volkovskii, A., Abarbanel, H. D. I., Stopfer, M., & Laurent, G. (2000). Dynamical coding of sensory information with competitive networks. J. Physiol. (Paris), 94, 465–471. Rabinovich, M. I., Volkovskii, A., Lecanda, P., Huerta, R., Abarbanel, H. D. I., & Laurent, G. (2001). Dynamical encoding by networks of competing neuron groups: Winnerless competition. Phys. Rev. Lett., 87, 068102. Rosenblatt, F. (1962). Principles of neurodynamics. New York: Spartan. Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning internal representation by back-propagating errors. Nature, 323, 533–536. Vapnik, V. (1995). The nature of statistical learning theory. New York: SpringerVerlag. Wehr, M., & Laurent, G. (1996). Odor encoding by temporal sequences of Žring in oscillating neural assemblies. Nature, 384, 162–166. Received November 20, 2001; accepted July 30, 2002.

LETTER

Communicated by Kwabena Boahen

Modeling Short-Term Synaptic Depression in Silicon Malte Boegerhausen [email protected] Pascal Suter [email protected] Shih-Chii Liu [email protected] Institute of Neuroinformatics, University of Zurich and ETH Zurich, Winterthurerstrasse 190, CH-8057 Zurich, Switzerland We describe a model of short-term synaptic depression that is derived from a circuit implementation. The dynamics of this circuit model is similar to the dynamics of some theoretical models of short-term depression except that the recovery dynamics of the variable describing the depression is nonlinear and it also depends on the presynaptic frequency. The equations describing the steady-state and transient responses of this synaptic model are compared to the experimental results obtained from a fabricated silicon network consisting of leaky integrate-and-Žre neurons and different types of short-term dynamic synapses. We also show experimental data demonstrating the possible computational roles of depression. One possible role of a depressing synapse is that the input can quickly bring the neuron up to threshold when the membrane potential is close to the resting potential. 1 Introduction

Short-term synaptic dynamics has been observed in different parts of the cortical system (Stratford, Tarczy-Hornoch, Martin, Bannister, & Jack, 1998; Varela et al., 1997; Markram, Wang, & Tsodyks, 1998; Reyes et al., 1998). The functionality of the short-term dynamics has been implicated in various cortical models (Senn, Segev, & Tsodyks, 1998; Chance, Nelson, & Abbott, 1998; Matveev & Wang, 2000). The properties of a network with dynamic synapses have been investigated by Tsodyks, Pawelzik, and Markram (1998), and Maass and Zador (1999) and the use of such a network in time series processing by Maass and Zador (1999) and Natschlager, Maass, and Zador (2001). The introduction of these dynamic synapses into hardware implementations of recurrent neuronal networks allows a wide range of operating regimes, especially in the case of time-varying inputs. In this work, we describe a model that was derived from a circuit implementation of short-term depression and facilitation. The dynamics of this Neural Computation 15, 331–348 (2003)

° c 2002 Massachusetts Institute of Technology

332

M. Boegerhausen, P. Suter, and S. Liu

circuit, initially proposed by Rasche and Hahnloser (2001), was not analyzed in their work. Our work is presented in two parts. We Žrst describe the model of this synaptic circuit and compare the differential equations describing the depression with the equations of one of the theoretical models frequently used in network simulations (Abbott, Sen, Varela, & Nelson, 1997; Varela et al., 1997). We analyze the transient and steady-state responses of this synaptic model in response to inputs of different statistical distributions. Next, we compare the theoretical results with experimental responses obtained from a silicon network of leaky integrate-and-Žre neurons together with different short-term dynamic synapses. We also show experimental data from the chip that demonstrate the possible computational roles of depression. From these experiments, we postulate that one possible role of depression is to bring the neuron’s response up to threshold quickly if the membrane potential of the neuron was close to the resting potential. We also mapped a proposed cortical model of direction selectivity that uses depressing synapses onto this chip. The results are qualitatively similar to the results obtained in the original work (Chance et al., 1998). The similarity of the circuit responses to the responses from Abbott and colleagues’ model means that we can use these VLSI networks of integrateand-Žre (IF) neurons as an alternative to computer simulations of dynamical networks composed of large numbers of integrate-and-Žre (IF) neurons using synapses with different time constants. The outputs of such networks can also be used as an interface with wetware. An infrastructure for a reprogrammable, reconŽgurable, multichip neuronal system is being developed along with a user-deŽned interface so that the system is easily accessible to a naive user. 2 Comparisons Between Models of Depression

Two theoretical models describing synaptic depression and facilitation used frequently in network simulations are the phenomenological model of Tsodyks and Markram (1997) and the model from Abbott et al. (1997). Because these models are equivalent if the inactivation synaptic time constant in Tsodyks and Markram’s model is small when compared with the recovery time constant of the depression, we compare the circuit model only with the model from Abbott and colleagues. Here, we describe only the circuit model for synaptic depression. The equivalent model for facilitation is described elsewhere (Liu, in press). 2.1 Theoretical Model for Depression Model. In the model from Abbott et al. (1997), the synaptic strength is described by gD (t) , where D is a variable between 0 and 1 that describes the amount of depression (D D 1 means no depression) and g is the maximum synaptic strength. The recovery

Modeling Short-Term Synaptic Depression in Silicon

333

dynamics of D is td

dD D 1 ¡ D, dt

(2.1)

where td is the recovery time of the depression. The update equation for D right after a spike at time t D tsp is C ¡ ) D dD ( tsp ), D ( tsp

(2.2)

where d (d < 1) is the amount by which D is decreased right after the spike and tsp is the time of the spike. The average steady-state value of depression for a regular spike train with a rate r is Dss D

1 ¡ e¡1/ ( rtd ) . 1 ¡ de ¡1 / (rtd )

(2.3)

2.2 Circuit Model of Depressing Synapse. In this model of synaptic depression derived from the circuit implementation, the equation that describes the recovery dynamics of the depressing variable, D, is nonlinear. This nonlinearity comes about because there is no compact circuitry to implement a linear resistor whose value can be adjusted easily after fabrication. A transistor can act as a linear resistor as long as the terminal voltages satisfy certain criteria. Additional circuitry would be needed to satisfy these criteria for a wide range of voltages, and it would increase the size of the circuit. One alternative is to replace the exponential dynamics in equation 2.1 with diode dynamics, which is easily obtained with a single diode-connected transistor. The current through the diode has a nonlinear decay instead of an exponential decay in the case of a linear resistor. Because of the different dynamics, the equation describing the recovery of D (derived from the circuit in the region where a transistor operates in the subthreshold region or the current is exponential in the gate voltage of the transistor) can be formulated as

dD 1k D M (1 ¡ D / ) , dt

(2.4)

where 1 / M is the equivalent of td in equation 2.1 and k (a transistor parameter) is less than 1. The maximum value of D is 1. The update equation remains as before: C ¡ ) D dD ( tsp ). D ( tsp

(2.5)

2.2.1 Circuit. Equations 2.4 and 2.5 are derived from the circuit in Figure 1. The operation of this circuit is described in the caption, and the detailed analysis leading to the differential equations for D is described in the

334

M. Boegerhausen, P. Suter, and S. Liu

Figure 1: Schematic for a depressing synapse circuit and responses to a regular input spike train. (a) Depressing synapse circuit. The voltage Va determines the synaptic conductance g, while the synaptic term gD or Isyn is exponential in the voltage, Vx . The subcircuit consisting of transistors, M1 , M2 , and M3 controls the dynamics of Isyn. The presynaptic input goes to the gate terminal of M3 , which acts like a switch. When there is a presynaptic spike, a quantity of charge (determined by Vd ) is removed from the node Vx . In between spikes, Vx recovers to the voltage, Va , through the diode-connected transistor, M1 . When there is no spike, Vx is around Va . When the presynaptic input comes from a regular spike train, Vx decreases with each spike and recovers between spikes. It reaches a steady-state value, as shown in b. During the spike, transistor M4 turns on, and the synaptic weight current Isyn charges up the membrane potential of the neuron through the current-mirror circuit consisting of M6 , M7 , and the capacitor C2 . We can convert the Isyn current source into a synaptic current Id with some gain and a “time constant” by adjusting the voltage Vgain . The decay dynamics of Id is given by Id ( t) D

Isyn (t D tsp ) 1C

AIsyn (tD tsp ) QT

k (V

where QT D C2 UT and A D e

) dd ¡Vgain / UT .

In a

normal synapse circuit (i.e., without short-term dynamics), Vx is controlled by an external bias voltage. (b) Input spike train at a frequency of 20 Hz (bottom curve) and corresponding response Vx (top curve) of the circuit for Vd D 0.3 V. The diode-connected transistor M1 has nonlinear dynamics. This dynamics is discussed in the text. The recovery time of the variable D varies depending on the distance of the present value of Vx from Va . The recovery rate of D increases for a larger difference between Vx and Va .

appendix. The voltage Vx in Figure 1 codes for gD. The conductance g is set by Va , while the dynamics of D is set by both Vd and Va . The time taken for the present value of Vx to return to the value of Va is determined by the current dynamics of the diode-connected transistor M1 and Va . As shown in the appendix, the recovery time constant (1 / M) of D is set by Va .

Modeling Short-Term Synaptic Depression in Silicon

335

The synaptic weight is described by the current, Isyn in Figure 1a, Isyn ( t) D Ion ek Vx / UT D gD ( t) , where g D Ion ek Va / UT is the synaptic strength, D ( t) is k ( Vdd ¡Vx ) / UT

Irf D Iop e

(2.6) k (V

e

dd ¡Va ) / UT Iop , Irf ( t)

and

. The recovery time constant (1 / M) of D is set by Va

Iopk ¡(1¡k ) ( V ¡Va ) / UT dd ). QT e

(M D The synaptic current, Id to the neuron, is then a current source Isyn , which lasts for the duration of the pulse width of the presynaptic spike. However, we can set a longer time constant for the synaptic current through Vgain . The equation describing this dependence (that is, the equation of the current in a current-mirror circuit) is given in the caption of Figure 1. It is difŽcult to compute a closed-form solution for equation 2.4 for any value of k (a transistor parameter which is less than 1). This value also changes under different operating conditions and between transistors fabricated in different processes. Hence, we solve for D (t) in the case of k D 0.5 given that the last spike occurred at t D t0 1 : dD / dt D M (1 ¡ D2 ) ) D ( t) D D

D (t 0 ) C D ( t0 ) e2Mt ¡ 1 C e2Mt ¡D(t0 ) C D ( t0 ) e2Mt C 1 C e2Mt

D ( t0 ) cosh ( Mt ) C sinh ( Mt ) . cosh ( Mt ) C D ( t0 ) sinh ( Mt )

(2.7)

When D is far from its recovered value of 1, we can approximate its recovery dynamics by dD / dt D M (irrespective of k ), and solving for D (t ), we get D ( t) D Mt C D (t 0 ) . In this regime, D ( t) follows a linear trajectory. Note that the same is true of equation 2.1 when t ¿ td . 3 Neuron Circuit

The dynamics of the neuron circuit of Figure 2 is similar to that of a leaky IF neuron with a constant leak. The circuit was previously described in Indiveri (2000) and Liu et al. (2001). It is a modiŽed version of previous designs (Mead, 1989; Van Schaik, 2001) and also includes the circuitry that models Žring-rate adaptation (Boahen, 1997a, 1997b) frequently seen in pyramidal 1

Note that if k =1, the equation reduces to equation 2.1.

336

M. Boegerhausen, P. Suter, and S. Liu C1

Vb V thresh Id V leak

Ileak Cm

Vo

Vm M5

V pw

M6

Vo

Vo

Vm

M1

Vo

M2

Vca

M4

M3

V refr

Ipw

Ca

Vt

Spike adaptation circuitry

Figure 2: Schematic of the leaky integrate-and-Žre neuron circuit. In this Žgure, Vrefr, sets the refractory period, Vthresh sets the threshold voltage, Vpw sets the pulse width of the spike, Vleak sets the leak current Ileak, and VCa and Vt set the output spike adaptation dynamics.

cells in the cortex. The equation describing the depolarization of the soma potential Vm is Cm

dV m (t) D i ( t ) ¡ Ileak ¡ Iahp , dt

Vm (t ) < Vthresh ,

(3.1)

where i ( t) is the synaptic current to the soma, Ileak is the leakage current, and Iahp is the after-hyperpolarization potassium (K) current, which causes the adaptation in the Žring rate of the cells. The synaptic current can be set to a point current source, that is, i ( t) D Idd ( t ¡tjk ) , or it can have a “time constant” and gain that is determined by the current-mirror circuit consisting of M6 , M7 , and C2 in Figure 1. The time constant and gain are controlled by the ( tD tsp ) I voltage Vgain . The decay dynamics of Id is of the form Id ( t) D syn AIsyn (tD tsp ) , kV

dd

¡V

1C

QT

gain

UT where QT D C2 UT and A D e . The operation of the circuit is as follows. When Vm (t ) increases above Vthresh at ts D t (ts is the time of the output spike), it increases by a step increment determined by the amplitude of the spike and the capacitive divider set by C1 and Cm . The output Vo becomes active at this time and turns on the discharging current path through transistors M5 and M6 . The time during which Vo remains high, TP , depends on the time taken for Vm to discharge below Vthresh . In this design, the pulse width TP is determined by the rate at which Vm is discharged, which in turn depends on the difference between the input current Id , the leak current Ileak, and the current Ipw . In other designs, Vm is reset immediately below Vthresh when Vo becomes active because either the input current is blocked from charging the membrane or

Modeling Short-Term Synaptic Depression in Silicon

337

the current Ipw is much larger than the input current. The refractory period, TR , is determined by Vrefr , which keeps Vo high so that Id cannot charge up the membrane. The spike output is taken from the node VNo . We ignore the Iahp current in determining the Žring rate dependence on the input current. The time needed for the neuron to charge up to threshold V

with the input current is TI D Cm i ( t)thresh . The spiking rate is given by ¡I rD

leak

1 . TI C TP C TR

4 Comparison Between Models

We compare the two models by looking at how D changes in response to a Poisson-distributed train whose frequency varied from 40 Hz to 1 Hz, as shown in Figure 3. We used a simple linear differential equation to describe the dynamics of the membrane potential Vm , tm

dV m (t) D Ri ( t) ¡ Vm ( t) , dt

where tm is the membrane time constant and i ( t) is the synaptic current. We ran an optimization algorithm on the parameters in the two models so that the least-square error between the excitatory postsynaptic potential (EPSP) outputs of both models was at a minimum. In this case, the EPSP responses were identical (see Figure 3b) and the corresponding D values (see Figure 3c) were almost identical except in the region where D was close to the maximum value. We performed the same comparison with Tsodyks and Markram’s model, and the results were similar. Hence, the circuit model can be used to describe short-term synaptic depression in a network simulation. However, the nonlinear recovery dynamics of the circuit model leads a different functional dependence of the average steady-state EPSP on the frequency of a regular input spike train. 5 Transient Response

The data in the Žgures in the remainder of this article are obtained from a fabricated silicon network of aVLSI IF neurons of the type described in section 3 together with different types of synapses. We Žrst measured the transient response of the neuron when stimulated by a 10 Hz spike train through the depressing synapse. We tuned the parameters of the synapse and the leak current so that the membrane potential did not build up to threshold. These data are shown in Figure 4a for a regular train and in Figure 4b for a Poisson-distributed train. The measured EPSP amplitude, D Vm can be Žtted with equation 2.6,

D Vm ( t)

D

dt d (Isyn (t) ¡ Ileak ) D t ( gD ( t) ¡ Ileak ) , Cm Cm

(5.1)

338

M. Boegerhausen, P. Suter, and S. Liu

Figure 3: Comparison between the outputs of the two models of depression. An optimization algorithm was used to determine the parameters of the models so that the least-square error between the EPSPs from the two models was at a minimum. The corresponding D distribution is shown in c. (a) Poissondistributed input with an initial frequency of 40 Hz and an end frequency of 1 Hz. (b) The EPSP responses of both models were identical. (c) The D values were almost identical except in the region when D is close to 1. Parameters used in the simulations: tm D 20 ms, d D 0.6, k D 0.7, M D 2.20 s ¡1 .

where Cm is the membrane capacitance, Ileak is the leak current, dt is the pulse width of the presynaptic spike, and D ( t) is computed using equation 2.7. Equation 5.1 Žts the measured data well (see Figure 4). 6 Steady-State Response

The equation describing the dependence of the steady-state values of D on the presynaptic frequency can easily be determined in the case of a regular spiking input of rate r by using equations 2.5 and 2.7. From these equations, we get p (¡1 C d) (1 C e2M/ r ) C 4d(¡1 C e2M/ r ) 2 C (1¡ d ) 2 (1 C e2M/ r ) 2 . (6.1) Dss D 2d(¡1 C e2M/ r )

Modeling Short-Term Synaptic Depression in Silicon

339

Figure 4: Transient responses to a (a) 10 Hz regular spike train and a (b) 10 Hz Poisson-distributed train. The input is the bottom curve of each plot. In a, the amplitude of the EPSP decreases with each incoming input spike, clearly showing the effect of synaptic depression. In b, the EPSP amplitude depends on the occurrence of the previous spike. The asterisks are the Žts of the circuit model to the peak value of each EPSP. The Žts give a d value of 0.79.

340

M. Boegerhausen, P. Suter, and S. Liu

If we use the reduced dynamics expression dD / dt D M, we replace equation 2.7 with equation 3.1 and obtain a simpler expression for Dss : Dss D

M . (1 ¡ d ) r

(6.2)

This equation shows that the steady-state D, and, hence, the steady-state EPSP amplitude, is inversely dependent on the presynaptic rate r. The form of the curve is similar to the results obtained by Abbott et al. (1997), where the data can be Žtted with equation 2.3. From the chip, we measured the steady-state EPSP amplitudes using the two-input spike distributions changing over a frequency range of 3 Hz to 50 Hz in steps of 1 Hz. Each frequency interval lasted 15 s, and the EPSP amplitude was averaged in the last 5 s to obtain the steady-state value. Four separate trials were performed; the resulting mean and the variance of the measurements are shown in Figure 5 for both the regular train and the Poisson-distributed train. We Žtted the experimental data using equations 5.1 and 6.1. The parameters from the Žts to the data in Figure 5a were then used to generate the Žtted curve to the data from the Poissondistributed train. The values from the Žts give recovery time constants from 1 to 3 s and Dss values varying between 0.02 and 0.04. 7 Role of Synaptic Depression

Different computational roles have been proposed for networks that incorporate synaptic depression. In this section, we describe some measurements that illustrate the postulated roles of depression. 7.1 Gain Control. Depressing synapses have been implicated in cortical gain control (Abbott et al., 1997). A depressing synapse acts like a transient detector to changes in frequency (or a Žrst derivative Žlter) similar to the way in which the photoreceptor responds with a higher gain to contrast and is less responsive to background illumination. A synapse with short-term depression responds equally to equal percentage rate changes in its input on different Žring rates. We demonstrate the gain control mechanism of short-term depression by measuring the neuron’s response to step changes in input frequency from 10 Hz to 20 Hz to 40 Hz. Each step change is the same rate change in input frequency. These results are shown in Figure 7.1a for a regular train and in Figure 6b for a Poisson-distributed train. Each frequency epoch lasted 3 s, so the synaptic strength should have reached steady state before the next step increase in input frequency. For both Žgures in Figure 7.1, the top curve shows the response of the neuron when stimulated by the input (bottom curve) through a depressing synapse (top curve) and a nondepressing synapse (middle curve). We tuned the parameters of

Modeling Short-Term Synaptic Depression in Silicon

341

Figure 5: Dependence of the steady-state EPSP amplitude on the input frequency for different values of depression. The data were measured from the fabricated circuit. The solid lines are Žts using equations 5.1 and 6.1. The parameters for the Žts are determined using the data in a. (a) Steady-state EPSP amplitude versus frequency for a regular train. From the retrieved values of d (between 0.2 and 0.6), we Žnd that Dss varies between 0.02 to 0.04. (b) Steadystate EPSP amplitude versus frequency for a Poisson-distributed train.

342

M. Boegerhausen, P. Suter, and S. Liu

both types of synapses in Figure 6b so that the steady-state Žring rate of the neuron was similar during the 20 Hz input period. Figure 6a shows clearly the transient increase in the Žring rate of a neuron when stimulated through a depressing synapse right after each step increase in input frequency and the subsequent adaptation of its Žring rate to a steady-state value. The steady-state Žring rate of the neuron with a depressing synapse is less dependent on the absolute input frequency when compared to the Žring rate of the neuron when stimulated through the nondepressing synapse. In the latter case, the Žring rate of the neuron is approximately linear in the input rate. The data shown in Figure 6a were obtained using a Poisson-distributed input. One obvious difference in the responses between the depressing and nondepressing synapse is that the neuron quickly reached threshold for a 10 Hz input for the depressing-synapse case, while the neuron remained subthreshold in the nondepressing synapse case until the input increased to 20 Hz. This suggests that a depressing synapse can be used to drive a neuron quickly to threshold when the membrane potential of the neuron is far from the threshold. 7.2 Phase Advance. Another feature of the depressing synapse is the phase advance of the membrane response of the neuron. This advance increases when the depression increases (see Figure 7). This feature was exploited in a model that described the direction-selective responses of visual cortical neurons (Chance et al., 1998). In this model, the neuron was driven by the output of a set of lateral geniculate nucleus (LGN) cells through depressing synapses and through an adjacent spatially shifted set of LGN cells through nondepressing synapses. We have attempted the same experiment using a neuron on our chip by using spikes recorded from an LGN

Figure 6: Facing page. Response of neuron to a step change in input frequency (bottom curve) when stimulated through a depressing synapse (top curve) and a nondepressing synapse (middle curve). The neuron was stimulated for three frequency intervals (10 Hz to 20 Hz to 40 Hz) lasting 3 s each. (a) Response of neuron using a regular spiking input. The steady-state Žring rate of the neuron increased almost linearly with the input frequency when stimulated through the nondepressing synapse. In the depressing synapse curve, there is a transient increase in the neuron’s Žring rate before the rate adapted to steady state. The steady-state output is less dependent on the absolute input frequency. (b) Response of neuron using a Poisson-distributed input. The parameters for both types of synapses were tuned so that the steady-state Žring rates were about the same at the end of each frequency interval for both synapses. Notice that during the 10 Hz interval, the neuron quickly built up to threshold if it was stimulated through the depressing synapse, but it stayed subthreshold when it was driven through to the nondepressing synapse until the input frequency reached 20 Hz.

Modeling Short-Term Synaptic Depression in Silicon

343

344

M. Boegerhausen, P. Suter, and S. Liu

Figure 7: Phase advance in the membrane response for different levels of depression in the synapse. The neuron was driven through the synapse by a Poisson-distributed input depicted at the bottom of the Žgure. The neuron shows the largest phase advance when the synapse has the largest depression value (Vd D 0.3V has the largest d value in the plot).

cell in the cat visual cortex during stimulation with a drifting sinusoidal grating (courtesy of K. Martin). An example of the direction-selective response is shown in Figure 8 (Liu, 2001). The direction-selective results were qualitatively similar to the data in Chance et al. (1998). 8 Conclusion

We described a model of synaptic depression derived from a circuit implementation. This circuit model has nonlinear recovery dynamics for the depression variable in contrast to current theoretical models of dynamic synapses. However, it gives qualitatively similar results when compared to the model of Abbott and colleagues (1997). Measured data from a chip with aVLSI IF neurons and dynamic synapses show that this network can be used to simulate the responses of dynamic networks with short-term dynamical synapses. Experimental results suggest that depressing synapses can be used to drive a neuron quickly up to threshold if its membrane potential is at the resting potential. The silicon networks provide an alternative to computer simulation of spike-based processing models with different time constant synapses because they run in real time and the computational time does not scale with the size of the neuronal network.

Modeling Short-Term Synaptic Depression in Silicon

345

Figure 8: Response to a 1 Hz drifting sinusoidal grating. The input spikes from the LGN cell are depicted at the bottom of the curve. The top curve shows the response of the neuron when the grating drifted in the preferred direction. The sharp excursions at the top of the potential are the output spikes of the neuron. The middle curve shows the response to the stimulus in the null direction. The membrane potential did not build up to threshold.

Appendix

The current equation for a p-type Želd-effect transistor (pFET) in subthreshold where the bulk of the transistor is at the voltage Vdd is Iop ek ( Vdd ¡Vg ) / UT (e ¡(Vdd ¡Vs ) / UT ¡ e ( Vdd ¡Vd ) / UT ) , where Vg is the voltage at the gate, and Vd and Vs are the voltages at the drain and source terminals of a transistor; k describes the efŽciency of the gate in controlling the current in the channel and is less than 1 for a transistor operating in subthreshold. The voltages in the exponent are normalized in terms of the thermal voltage, UT . The differential equation for the current in a diode-capacitor circuit (e.g., M1 and C in Figure 1) when there is no input current is given by Iop ek ( Vdd ¡Vx ) / UT ( e¡(Vdd ¡Va ) / UT ¡ e ¡(Vdd ¡Vx ) / UT ) D C

dV x . dt

(A.1)

346

M. Boegerhausen, P. Suter, and S. Liu

By introducing the current variable, Irf D Iop ek ( Vdd ¡Vx ) / UT , we solve for the differential, dI rf : dI rf D ¡

k UT

Irf dV x ) dV x D ¡

UT dI rf . k Irf

By changing variables from Vx to Irf in equation A.1, we get, Á

0

!

1 d k Irf

QT

"

) D dt @e¡(Vdd ¡Va / UT

k (V

By substituting D D

e

dd

¡Va ) / UT

Irf

Iop

Iop ¡ Irf

#1/k 1 A.

(A.2)

into equation A.2, we get

Iopk ¡(1¡k ) ( V ¡Va ) / UT dD dd (1 ¡ D1/k ) D M (1 ¡ D1 /k ) , D e dt QT

(A.3)

where M D QT e¡(1¡k ) ( Vdd ¡Va ) / UT . In steady state, D D 1. The update dynamics for D can be inferred from the dynamics of Ir , which are described by Iopk

Ir ( tnC ) D (1 C a) Ir ( tn¡ ) ,

(A.4)

where tnC is the time right after the nth spike and tn¡ is the time right before the nth spike. The a factor is determined by a voltage parameter and the pulse width of the presynaptic spike. The corresponding equation for D is D ( tnC ) D dD ( tn¡ ) , where d is

1 1C a

(A.5)

in equation 2.5. 2

Acknowledgments

This work was supported in part by the Swiss National Foundation Research SPP grant. We also acknowledge Kevan Martin, Pamela Baker, and Ora Ohana for many discussions on dynamic synapses. 2 In equation A.5, for d D 1, a has to be 0. But a is never zero due to leakage currents. To compute d, we Žrst solve for 1 C a D e (k Idt )/ QT , where I is set by the voltage parameter Vd , dt is the pulse width of the input spike, and k is a transistor parameter. The smallest I is the leakage current, so for this value, we approximate a D (k Ir0dt ) / QT .

Modeling Short-Term Synaptic Depression in Silicon

347

References Abbott, L., Sen, K., Varela, J., & Nelson, S. (1997).Synaptic depression and cortical gain control. Science, 275, 220–223. Boahen, K. A. (1997a). The retinomorphic approach: Pixel-parallel adaptive ampliŽcation, Žltering, and quantization. Analog Integrated Circuits and Signal Processing, 13, 53–68. Boahen, K. A. (1997b). Retinomorphic vision systems: Reverse engineering the vertebrate retina. Unpublished doctoral dissertation, California Institute of Technology, Pasadena, CA. Chance, F., Nelson, S., & Abbott, L. (1998). Synaptic depression and the temporal response characteristics of V1 cells. Journal of Neuroscience, 18, 4785–4799. Indiveri, G. (2000). Modeling selective attention using a neuromorphic aVLSI device. Neural Computation, 12, 2857–2880. Liu, S.-C. (2001, Oct. 19). Simple cortical modeling with aVLSI spiking neurons and dynamic synapses. Paper presented at the ZNZ Symposium, University of Zurich. Liu, S.-C. (in press). Analog VLSI circuits for short-term dynamic synapses. EURASIP Journal on Applied Signal Processing: Special Issue. Liu, S.-C., Kramer, J., Indiveri, G., Delbruck, ¨ T., Burg, T., & Douglas, R. (2001). Orientation-selective aVLSI spiking neurons. Neural Networks: Special Issue, 14(6/7), 629–643. Maass, W., & Zador, A. (1999). Computing and learning with dynamic synapses. In W. Maass & C. M. Bishop (Eds.), Pulsed neural networks (pp. 157–178). Cambridge, MA: MIT Press. Markram, H., Wang, Y., & Tsodyks, M. (1998). Differential signaling via the same axon of neocortical pyramidal neurons. Proc. Natl. Acad. Sci. USA, 95, 5323–5328. Matveev, V., & Wang, X. (2000). Differential short-term synaptic plasticity and transmission of complex spike trains: To depress or to facilitate? Cerebral Cortex, 10, 1143–1153. Mead, C. (1989). Analog VLSI and neural systems. Reading, MA: Addison-Wesley. Natschlager, T., Maass, W., & Zador, A. (2001). EfŽcient temporal processing with biologically realistic dynamic synapses. Network, 12, 75–87. Rasche, C., & Hahnloser, R. (2001). Silicon synaptic depression. Biological Cybernetics, 84, 57–62. Reyes, A., Lujan, R., Rozov, A., Burnashev, N., Somogyi, P., & Sakmann, B. (1998). Target-cell-speciŽc facilitation and depression in neocortical circuits. Nature Neuroscience, 1, 279–285. Senn, W., Segev, I., & Tsodyks, M. (1998). Reading neuronal synchrony with depressing synapses. Neural Computation, 10, 815–819. Stratford, K., Tarczy-Hornoch, K., Martin, K., Bannister, N., & Jack, J. (1998). Excitatory synaptic inputs to spiny stellate cells in cat visual cortex. Nature, 382, 258–261. Tsodyks, M., & Markram, H. (1997). The neural code between neocortical pyramidal neurons depends on neurotransmitter release probability. Proc. Natl. Acad. Sci. USA, 94(2), 719–723.

348

M. Boegerhausen, P. Suter, and S. Liu

Tsodyks, M., Pawelzik, K., & Markram, H. (1998). Neural networks with dynamic synapses. Neural Computation, 10, 821–835. Van Schaik, A. (2001). Building blocks for electronic spiking neural networks. Neural Networks, 14(6/7), 617–628. Varela, J., Sen, K., Gibson, J., Fost, J., Abbott, L., and Nelson, S. (1997). A quantitative description of short-term plasticity at excitatory synapses in layer 2/3 of rat primary visual cortex. Journal of Neuroscience, 17, 7926–7940. Received June 14, 2002; accepted July 24, 2002.

LETTER

Communicated by Hagai Attias

Dictionary Learning Algorithms for Sparse Representation Kenneth Kreutz-Delgado [email protected] Joseph F. Murray [email protected] Bhaskar D. Rao [email protected] Electrical and Computer Engineering, Jacobs School of Engineering, University of California, San Diego, La Jolla, California 92093-0407, U.S.A. Kjersti Engan [email protected] Stavanger University College, School of Science and Technology Ullandhaug, N-4091 Stavanger, Norway Te-Won Lee [email protected] Terrence J. Sejnowski [email protected] Howard Hughes Medical Institute, Computational Neurobiology Laboratory, Salk Institute, La Jolla, California 92037, U.S.A. Algorithms for data-driven learning of domain-speciŽc overcomplete dictionaries are developed to obtain maximum likelihood and maximum a posteriori dictionary estimates based on the use of Bayesian models with concave/Schur-concave (CSC) negative log priors. Such priors are appropriate for obtaining sparse representations of environmental signals within an appropriately chosen (environmentally matched) dictionary. The elements of the dictionary can be interpreted as concepts, features, or words capable of succinct expression of events encountered in the environment (the source of the measured signals). This is a generalization of vector quantization in that one is interested in a description involving a few dictionary entries (the proverbial “25 words or less”), but not necessarily as succinct as one entry. To learn an environmentally adapted dictionary capable of concise expression of signals generated by the environment, we develop algorithms that iterate between a representative set of sparse representations found by variants of FOCUSS and an update of the dictionary using these sparse representations. Experiments were performed using synthetic data and natural images. For complete dictionaries, we demonstrate that our algorithms have imNeural Computation 15, 349–396 (2003)

° c 2002 Massachusetts Institute of Technology

350

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

proved performance over other independent component analysis (ICA) methods, measured in terms of signal-to-noise ratios of separated sources. In the overcomplete case, we show that the true underlying dictionary and sparse sources can be accurately recovered. In tests with natural images, learned overcomplete dictionaries are shown to have higher coding efŽciency than complete dictionaries; that is, images encoded with an overcomplete dictionary have both higher compression (fewer bits per pixel) and higher accuracy (lower mean square error). 1 Introduction

FOCUSS, which stands for FOCal Underdetermined System Solver, is an algorithm designed to obtain suboptimally (and, at times, maximally) sparse solutions to the following m £ n, underdetermined linear inverse problem1 (Gorodnitsky, George, & Rao, 1995; Rao & Gorodnitsky, 1997; Gorodnitsky & Rao, 1997; Adler, Rao, & Kreutz-Delgado, 1996; Rao & Kreutz-Delgado, 1997; Rao, 1997, 1998), y D Ax,

(1.1)

for known A. The sparsity of a vector is the number of zero-valued elements (Donoho, 1994), and is related to the diversity, the number of nonzero elements, sparsity D #fx[i] D 0g 6 0g diversity D #fx[i] D

diversity D n ¡ sparsity. Since our initial investigations into the properties of FOCUSS as an algorithm for providing sparse solutions to linear inverse problems in relatively noise-free environments (Gorodnitsky et al., 1995; Rao, 1997; Rao & Gorodnitsky, 1997; Gorodnitsky & Rao, 1997; Adler et al., 1996; Rao & Kreutz-Delgado, 1997), we now better understand the behavior of FOCUSS in noisy environments (Rao & Kreutz-Delgado, 1998a, 1998b) and as an interior point-like optimization algorithm for optimizing concave functionals subject to linear constraints (Rao & Kreutz-Delgado, 1999; Kreutz-Delgado & Rao, 1997, 1998a, 1998b, 1998c, 1999; Kreutz-Delgado, Rao, Engan, Lee, & Sejnowski, 1999a; Engan, Rao, & Kreutz-Delgado, 2000; Rao, Engan, Cotter, & Kreutz-Delgado, 2002). In this article, we consider the use of the FOCUSS algorithm in the case where the matrix A is unknown and must be learned. Toward this end, we Žrst briey discuss how the use of concave (and 1

For notational simplicity, we consider the real case only.

Dictionary Learning Algorithms for Sparse Representation

351

Schur concave) functionals enforces sparse solutions to equation 1.1. We also discuss the choice of the matrix, A, in equation 1.1 and its relationship to the set of signal vectors y for which we hope to obtain sparse representations. Finally, we present algorithms capable of learning an environmentally adapted dictionary, A, given a sufŽciently large and statistically representative sample of signal vectors, y, building on ideas originally presented in Kreutz-Delgado, Rao, Engan, Lee, and Sejnowski (1999b), Kreutz-Delgado, Rao, and Engan (1999), and Engan, Rao, and Kreutz-Delgado (1999). We refer to the columns of the full row-rank m £ n matrix A, A D [a1 , . . . , an ] 2 Rm£n ,

n À m,

(1.2)

as a dictionary, and they are assumed to be a set of vectors capable of providing a highly succinct representation for most (and, ideally, all) statistically representative signal vectors y 2 Rm . Note that with the assumption that rank(A) D m, every vector y has a representation; the question at hand is whether this representation is likely to be sparse. We call the statistical generating mechanism for signals, y, the environment and a dictionary, A, within which such signals can be sparsely represented an environmentally adapted dictionary. Environmentally generated signals typically have signiŽcant statistical structure and can be represented by a set of basis vectors spanning a lowerdimensional submanifold of meaningful signals (Field, 1994; Ruderman, 1994). These environmentally meaningful representation vectors can be obtained by maximizing the mutual information between the set of these vectors (the dictionary) and the signals generated by the environment (Comon, 1994; Bell & Sejnowski, 1995; Deco & Obradovic, 1996; Olshausen & Field, 1996; Zhu, Wu, & Mumford, 1997; Wang, Lee, & Juang, 1997). This procedure can be viewed as a natural generalization of independent component analysis (ICA) (Comon, 1994; Deco & Obradovic, 1996). As initially developed, this procedure usually results in obtaining a minimal spanning set of linearly independent vectors (i.e., a true basis). More recently, the desirability of obtaining “overcomplete” sets of vectors (or “dictionaries”) has been noted (Olshausen & Field, 1996; Lewicki & Sejnowski, 2000; Coifman & Wickerhauser, 1992; Mallat & Zhang, 1993; Donoho, 1994; Rao & KreutzDelgado, 1997). For example, projecting measured noisy signals onto the signal submanifold spanned by a set of dictionary vectors results in noise reduction and data compression (Donoho, 1994, 1995). These dictionaries can be structured as a set of bases from which a single basis is to be selected to represent the measured signal(s) of interest (Coifman & Wickerhauser, 1992) or as a single, overcomplete set of individual vectors from within which a vector, y, is to be sparsely represented (Mallat & Zhang, 1993; Olshausen & Field, 1996; Lewicki & Sejnowski, 2000; Rao & Kreutz-Delgado, 1997). The problem of determining a representation from a full row-rank overcomplete dictionary, A D [a1 , . . . , an ], n À m, for a speciŽc signal mea-

352

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

surement, y, is equivalent to solving an underdetermined inverse problem, Ax D y, which is nonuniquely solvable for any y. The standard least-squares solution to this problem has the (at times) undesirable feature of involving all the dictionary vectors in the solution2 (the “spurious artifact” problem) and does not generally allow for the extraction of a categorically or physically meaningful solution. That is, it is not generally the case that a least-squares solution yields a concise representation allowing for a precise semantic meaning.3 If the dictionary is large and rich enough in representational power, a measured signal can be matched to a very few (perhaps even just one) dictionary words. In this manner, we can obtain concise semantic content about objects or situations encountered in natural environments (Field, 1994). Thus, there has been signiŽcant interest in Žnding sparse solutions, x (solutions having a minimum number of nonzero elements), to the signal representation problem. Interestingly, matching a speciŽc signal to a sparse set of dictionary words or vectors can be related to entropy minimization as a means of elucidating statistical structure (Watanabe, 1981). Finding a sparse representation (based on the use of a “few” code or dictionary words) can also be viewed as a generalization of vector quantization where a match to a single “code vector” (word) is always sought (taking “code book” = “dictionary”).4 Indeed, we can refer to a sparse solution, x, as a sparse coding of the signal instantiation, y. 1.1 Stochastic Models. It is well known (Basilevsky, 1994) that the sto-

chastic generative model (1.3)

y D Ax C º, m

can be used to develop algorithms enabling coding of y 2 R via solving the inverse problem for a sparse solution x 2 Rn for the undercomplete (n < m) and complete (n D m) cases. In recent years, there has been a great deal of interest in obtaining sparse codings of y with this procedure for the overcomplete (n > m) case (Mallat & Zhang, 1993; Field, 1994). In our earlier work, we have shown that given an overcomplete dictionary, A (with the columns of A comprising the dictionary vectors), a maximum a posteriori (MAP) estimate of the source vector, x, will yield a sparse coding of y in the low-noise limit if the negative log prior, ¡ log(P(x) ) , is concave/Schurconcave (CSC) (Rao, 1998; Kreutz-Delgado & Rao, 1999), as discussed below. 2 This fact comes as no surprise when the solution is interpreted within a Bayesian framework using a gaussian (maximum entropy) prior. 3 Taking “semantic” here to mean categorically or physically interpretable. 4 For example, n D 100 corresponds to 100 features encoded via vector quantization (“one column = one concept”). features using up to four ³ ´ If³we are ´ allowed ³ ´ to³represent ´ 100 100 100 100 C C C D 4,087,975concepts showing columns, we can encode 1 2 3 4

a combinatorial boost in expressive power.

Dictionary Learning Algorithms for Sparse Representation

353

For P (x ) factorizable into a product of marginal probabilities, the resulting code is also known to provide an independent component analysis (ICA) representation of y. More generally, a CSC prior results in a sparse representation even in the nonfactorizable case (with x then forming a dependent component analysis, or DCA, representation). Given independently and identically distributed (i.i.d.) data, Y D YN D ( y1 , . . . , yN ) , assumed to be generated by the model 1.3, a maximum likelibML , of the unknown (but nonrandom) dictionary A can be hood estimate, A determined as (Olshausen & Field, 1996; Lewicki & Sejnowski, 2000) bML D arg max P (YI A ) . A A

This requires integrating out the unobservable i.i.d. source vectors, X D XN D ( x1 , . . . , xN ) , in order to compute P ( YI A ) from the (assumed) known probabilities P ( x) and P (º). In essence, X is formally treated as a set of nuisance parameters that in principle can be removed by integration. However, because the prior P ( x ) is generally taken to be supergaussian, this integration is intractable or computationally unreasonable. Thus, approximations to this integration are performed that result in an approximation to P (YI A ), which is then maximized with respect to Y. A new, better approximation to the integration can then be made, and this process is iterated until the estimate of the dictionary A has (hopefully) converged (Olshausen & Field, 1996). We refer to the resulting estimate as an approximate maxibAML ). mum likelihood (AML) estimate of the dictionary A (denoted here by A No formal proof of the convergence of this algorithm to the true maxibML, has been given in the prior literature, but mum likelihood estimate, A it appears to perform well in various test cases (Olshausen & Field, 1996). Below, we discuss the problem of dictionary learning within the framework of our recently developed log-prior model-based sparse source vector learning approach that for a known overcomplete dictionary can be used to obtain sparse codes (Rao, 1998; Kreutz-Delgado & Rao, 1997, 1998b, 1998c, 1999; Rao & Kreutz-Delgado, 1999). Such sparse codes can be found using FOCUSS, an afŽne scaling transformation (AST)–like iterative algorithm that Žnds a sparse locally optimal MAP estimate of the source vector x for an observation y. Using these results, we can develop dictionary learning algorithms within the AML framework and for obtaining a MAP-like bMAP , of the (now assumed random) dictionary, A, assuming in estimate, A the latter case that the dictionary belongs to a compact submanifold corresponding to unit Frobenius norm. Under certain conditions, convergence to a local minimum of a MAP-loss function that combines functions of the discrepancy e D ( y ¡ Ax ) and the degree of sparsity in x can be rigorously proved. 1.2 Related Work. Previous work includes efforts to solve equation 1.3 in the overcomplete case within the maximum likelihood (ML) framework.

354

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

An algorithm for Žnding sparse codes was developed in Olshausen and Field (1997) and tested on small patches of natural images, resulting in Gabor-like receptive Želds. In Lewicki and Sejnowski (2000) another ML algorithm is presented, which uses the Laplacian prior to enforce sparsity. The values of the elements of x are found with a modiŽed conjugate gradient optimization (which has a rather complicated implementation) as opposed to the standard ICA (square mixing matrix) case where the coefŽcients are found by inverting the A matrix. The difŽculty that arises when using ML is that Žnding the estimate of the dictionary A requires integrating over all possible values of the joint density P ( y, xI A ) as a function of x. In Olshausen and Field (1997), this is handled by assuming the prior density of x is a delta function, while in Lewicki and Sejnowski (2000), it is approximated by a gaussian. The Žxed-point FastICA (Hyv¨arinen, Cristescu, & Oja, 1999) has also been extended to generate overcomplete representations. The FastICA algorithm can Žnd the basis functions (columns of the dictionary A) one at a time by imposing a quasi-orthogonality condition and can be thought of as a “greedy” algorithm. It also can be run in parallel, meaning that all columns of A are updated together. Other methods to solve equation 1.3 in the overcomplete case have been developed using a combination of the expectation-maximization (EM) algorithm and variational approximation techniques. Independent factor analysis (Attias, 1999) uses a mixture of gaussians to approximate the prior density of the sources, which avoids the difŽculty of integrating out the parameters X and allows different sources to have different densities. In another method (Girolami, 2001), the source priors are assumed to be supergaussian (heavy-tailed), and a variational lower bound is developed that is used in the EM estimation of the parameters A and X. It is noted in Girolami (2001) that the mixtures used in independent factor analysis are more general than may be needed for the sparse overcomplete case, and they can be computationally expensive as the dimension of the data vector and number of mixtures increases. In Zibulevsky and Pearlmutter (2001), the blind source separation problem is formulated in terms of a sparse source underlying each unmixed signal. These sparse sources are expanded into the unmixed signal with a predeŽned wavelet dictionary, which may be overcomplete. The unmixed signals are linearly combined by a different mixing matrix to create the observed sensor signals. The method is shown to give better separation performance than ICA techniques. The use of learned dictionaries (instead of being chosen a priori) is suggested.

2 FOCUSS: Sparse Solutions for Known Dictionaries 2.1 Known Dictionary Model. A Bayesian interpretation is obtained from the generative signal model, equation 1.3, by assuming that x has the

Dictionary Learning Algorithms for Sparse Representation

355

parameterized (generally nongaussian) probability density function (pdf), Z Pp ( x) D Zp¡1 e ¡c p dp ( x ) ,

Zp D

e ¡c p dp ( x ) dx,

(2.1)

with parameter vector p. Similarly, the noise º is assumed to have a parameterized (possibly nongaussian) density P q (º) of the same form as equation 2.1 with parameter vector q. It is assumed that x and º have zero means and that their densities obey the property d ( x ) D d (| x| ) , for | ¢ | deŽned component-wise. This is equivalent to assuming that the densities are symmetric with respect to sign changes in the components x, x[i] à ¡x[i] and therefore that the skews of these densities are zero. We also assume that d (0) D 0. With a slight abuse of notation, we allow the differing subscripts q and p to indicate that dq and dp may be functionally different as well as parametrically different. We refer to densities like equation 2.1 for suitable additional constraints on dp (x ) , as hypergeneralized gaussian distributions (Kreutz-Delgado & Rao, 1999; Kreutz-Delgado et al., 1999). If we treat A, p, and q as known parameters, then x and y are jointly distributed as P (x, y ) D P ( x, yI p, q, A ) . Bayes’ rule yields P (x | yI p, A ) D

1 1 P ( y | xI p, A ) ¢ P ( xI p, A ) D Pq ( y ¡ Ax ) ¢ Pp ( x ) (2.2) b b Z

b D P ( y ) D P ( yI p, q, A ) D

P ( y | x) ¢ Pp ( x) dx.

(2.3)

Usually the dependence on p and q is notationally suppressed, for example, b D P ( yI A) . Given an observation, y, maximizing equation 2.2 with respect to x yields a MAP estimate b x. This ideally results in a sparse coding of the observation, a requirement that places functional constraints on the pdfs, and particularly on dp . Note that b is independent of x and can be ignored when optimizing equation 2.2 with respect to the unknown source vector x. The MAP estimate equivalently is obtained from minimizing the negative logarithm of P ( x | y ), which is b x D arg min dq ( y ¡ Ax) C ldp ( x) , x

(2.4)

where l D c p /c q and dq ( y¡Ax ) D dq ( Ax¡y ) by our assumption of symmetry. The quantity l1 is interpretable as a signal-to-noise ratio (SNR). Furthermore, assuming that both dq and dp are concave/Schur–concave (CSC) as deŽned in section 2.4, then the term dq ( y ¡ Ax) in equation 2.4 will encourage sparse

356

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

residuals, e D y ¡ Ab x, while the term dp ( x ) encourages sparse source vector estimates, b x. A given value of l then determines a trade-off between residual and source vector sparseness. This most general formulation will not be used here. Although we are interested in obtaining sparse source vector estimates, we will not enforce sparsity on the residuals but instead, to simplify the development, will assume the q D 2 i.i.d. gaussian measurement noise case (º gaussian with known covariance s 2 ¢ I), which corresponds to taking, bx ) D c q dq ( y ¡ Ab

1 bxk2 . ky ¡ Ab 2s 2

(2.5)

In this case, problem 2.4 becomes 1 b x D arg min ky ¡ Axk2 C ldp ( x ). x 2

(2.6)

In either case, we note that l ! 0 as c p ! 0 which (consistent with the generative model, 1.3) we refer to as the low noise limit. Because the mapping A is assumed to be onto, in the low-noise limit, the optimization, equation 2.4, is equivalent to the linearly constrained problem, b x D arg min dp (x )

subject to

Ax D y.

(2.7)

In the low-noise limit, no sparseness constraint need be placed on the residuals, e D y ¡ Ab x, which are assumed to be zero. It is evident that the structure of dp (¢) is critical for obtaining a sparse coding, b x, of the observation y (Kreutz-Delgado & Rao, 1997; Rao & Kreutz-Delgado, 1999). Throughtout this article, the quantity dp ( x ) is always assumed to be CSC (enforcing sparse solutions to the inverse problem 1.3). As noted, and as will be evident during the development of dictionary learning algorithms below, we do not impose a sparsity constraint on the residuals; instead, the measurement noise º will be assumed to be gaussian (q D 2). 2.2 Independent Component Analysis and Sparsity Inducing Priors.

An important class of densities is given by the generalized gaussians for which p

dp ( x ) D kxkp D

n X

|x[k]| p ,

(2.8)

kD 1

for p > 0 (Kassam, 1982). This is a special case of the larger `p class (the p-class) of functions, which allows p to be negative in value (Rao & KreutzDelgado, 1999; Kreutz-Delgado & Rao, 1997). Note that this function has the special property of separability, dp ( x ) D

n X kD 1

dp ( x[k]) ,

Dictionary Learning Algorithms for Sparse Representation

357

which corresponds to factorizability of the density Pp ( x ) , Pp ( x) D

n Y

Pp ( x[k]) ,

kD 1

and hence to independence of the components of x. The assumption of independent components allows the problem of solving the generative model, equation 1.3, for x to be interpreted as an ICA problem (Comon, 1994; Pham, 1996; Olshausen & Field, 1996; Roberts, 1998). It is of interest, then, to consider the development of a large class of parameterizable separable functions dp ( x) consistent with the ICA assumption (Rao & Kreutz-Delgado, 1999; KreutzDelgado & Rao, 1997). Given such a class, it is natural to examine the issue of Žnding a best Žt within this class to the “true” underlying prior density of x. This is a problem of parametric density estimation of the true prior, where one attempts to Žnd an optimal choice of the model density Pp ( x ) by an optimization over the parameters p that deŽne the choice of a prior from within the class. This is, in general, a difŽcult problem, which may require the use of Monte Carlo, evolutionary programming, or stochastic search techniques. Can the belief that supergaussian priors, Pp ( x ), are appropriate for Žnding sparse solutions to equation 1.3 (Field, 1994; Olshausen & Field, 1996) be clariŽed or made rigorous? It is well known that the generalized gaussian distribution arising from the use of equation 2.8 yields supergaussian distributions (positive kurtosis) for p < 2 and subgaussian (negative kurtosis) for p > 2. However, one can argue (see section 2.5 below) that the condition for obtaining sparse solutions in the low-noise limit is the stronger requirement that p · 1, in which case the separable function dp (x ) is CSC. This indicates that supergaussianness (positive kurtosis) alone is necessary but not sufŽcient for inducing sparse solutions. Rather, sufŽciency is given by the requirement that ¡ log Pp ( x ) ¼ dp ( x) be CSC. We have seen that the function dp (x ) has an interpretation as a (negative logarithm of) a Bayesian prior or as a penalty function enforcing sparsity in equation 2.4, where dp ( x ) should serve as a “relaxed counting function” on the nonzero elements of x. Our perspective emphasizes that dp ( x ) serves both of these goals simultaneously. Thus, good regularizing functions, dp ( x ), should be exibly parameterizable so that Pp ( x) can be optimized over the parameter vector p to provide a good parametric Žt to the underlying environmental pdf, and the functions should also have analytical properties consistent with the goal of enforcing sparse solutions. Such properties are discussed in the next section. 2.3 Majorization and Schur-Concavity. In this section, we discuss functions that are both concave and Schur-concave (CSC functions; Marshall & Olkin, 1979). We will call functions, dp (¢) , which are CSC, diversity functions, anticoncentration functions or antisparsity functions. The larger the value of the

358

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

CSC function dp (x ) , the more diverse (i.e., the less concentrated or sparse) the elements of the vector x are. Thus, minimizing dp ( x ) with respect to x results in less diverse (more concentrated or sparse) vectors x. 2.3.1 Schur-Concave Functions. A measure of the sparsity of the elements of a solution vector x (or the lack thereof, which we refer to as the diversity of x) is given by a partial ordering on vectors known as the Lorentz order. For any vector in the positive orthant, x 2 RnC , deŽne the decreasing rearrangement . x D ( xb1c , . . . , xbnc ) , xb1c ¸ ¢ ¢ ¢ ¸ xbnc ¸ 0 and the partial sums (Marshall & Olkin, 1979; Wickerhauser, 1994), Sx [k] D

k X iD 1

xbnc ,

k D 1, . . . , n.

We say that y majorizes x, y  x, iff for k D 1, . . . , n, S y [k] ¸ Sx [k]I

S y [n] D Sx [n],

and the vector y is said to be more concentrated, or less diverse, than x. This partial order deŽned by majorization then deŽnes the Lorentz order. We are interested in scalar-valued functions of x that are consistent with majorization. Such functions are known as Schur-concave functions, d (¢) : RnC ! R. They are deŽned to be precisely the class of functions consistent with the Lorentz order, yÂx

)

d ( y ) < d ( x ).

In words, if y is less diverse than x (according to the Lorentz order) then d ( y) is less than d ( x) for d (¢) Schur-concave. We assume that Schur-concavity is a necessary condition for d (¢) to be a good measure of diversity (antisparsity). 2.3.2 Concavity Yields Sparse Solutions. Recall that a function d (¢) is concave on the positive orthant RnC iff (Rockafellar, 1970) d ( (1 ¡ c ) x C c y ) ¸ (1 ¡ c ) d (x ) C c d ( y ) , 8x, y 2 RnC , 8c , 0 · c · 1. In addition, a scalar function is said to be permutation invariant if its value is independent of rearrangements of its components. An important fact is that for permutation invariant functions, concavity is a sufŽcient condition for Schur-concavity: Concavity C permutation invariance ) Schur-concavity. Now consider the low-noise sparse inverse problem, 2.7. It is well known that subject to linear constraints, a concave function on RnC takes its minima

Dictionary Learning Algorithms for Sparse Representation

359

on the boundary of RnC (Rockafellar, 1970), and as a consequence these minima are therefore sparse. We take concavity to be a sufŽcient condition for a permutation invariant d (¢) to be a measure of diversity and obtain sparsity as constrained minima of d (¢) . More generally, a diversity measure should be somewhere between Schur-concave and concave. In this spirit, one can deŽne almost concave functions (Kreutz-Delgado & Rao, 1997), which are Schur-concave and (locally) concave in all n directions but one, which also are good measures of diversity. 2.3.3 Separability, Schur-Concavity, and ICA. The simplest way to ensure that d ( x ) be permutation invariant (a necessary condition for Schurconcavity) is to use functions that are separable. Recall that separability of dp ( x) corresponds to factorizability of Pp (x ) . Thus, separability of d ( x ) corresponds to the assumption of independent components of x under the model 1.3. We see that from a Bayesian perspective, separability of d ( x) corresponds to a generative model for y that assumes a source, x, with independent components. With this assumption, we are working within the framework of ICA (Nadal & Parga, 1994; Pham, 1996; Roberts, 1998). We have developed effective algorithms for solving the optimization problem 2.7 for sparse solutions when dp (x ) is separable and concave (Kreutz-Delgado & Rao, 1997; Rao & Kreutz-Delgado, 1999). It is now evident that relaxing the restriction of separability generalizes the generative model to the case where the source vector, x, has dependent components. We can reasonably call an approach based on a nonseparable diversity measure d ( x) a dependent component analysis (DCA). Unless care is taken, this relaxation can signiŽcantly complicate the analysis and development of optimization algorithms. However, one can solve the lownoise DCA problem, at least in principle, provided appropriate choices of nonseparable diversity functions are made. 2.4 Supergaussian Priors and Sparse Coding. The P-class of diversity measures for 0 < p · 1 result in sparse solutions to the low-noise coding problem, 2.7. These separable and concave (and thus Schur-concave) diversity measures correspond to supergaussian priors, consistent with the folk theorem that supergaussian priors are sparsity-enforcing priors. However, taking 1 · p < 2 results in supergaussian priors that are not sparsity enforcing. Taking p to be between 1 and 2 yields a dp ( x) that is convex and therefore not concave. This is consistent with the well-known fact that for this range of p, the pth-root of dp ( x ) is a norm. Minimizing dp (x ) in this case drives x toward the origin, favoring concentrated rather than sparse solutions. We see that if a sparse coding is to be found based on obtaining a MAP estimate to the low-noise generative model, 1.3, then, in a sense, supergaussianness is a necessary but not sufŽcient condition for a prior to be sparsity enforcing. A sufŽcient condition for obtaining a sparse MAP coding is that the negative log prior be CSC.

360

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

2.5 The FOCUSS Algorithm. Locally optimal solutions to the knowndictionary sparse inverse problems in gaussian noise, equations 2.6 and 2.7, are given by the FOCUSS algorithm. This is an afŽne-scaling transformation (AST)-like (interior point) algorithm originally proposed for the low-noise case 2.7 in Rao and Kreutz-Delgado (1997, 1999) and Kreutz-Delgado and Rao (1997); and extended by regularization to the nontrivial noise case, equation 2.6, in Rao and Kreutz-Delgado (1998a), Engan et al. (2000), and Rao et al. (2002). In these references, it is shown that the FOCUSS algorithm has excellent behavior for concave functions (which includes the the CSC concentration functions) dp (¢) . For such functions, FOCUSS quickly converges to a local minimum, yielding a sparse solution to problems 2.7 and 2.6. One can quickly motivate the development of the FOCUSS algorithm appropriate for solving the optimization problem 2.6 by considering the problem of obtaining the stationary points of the objective function. These are given as solutions, x¤ , to

AT ( Ax¤ ¡ y ) C lrx dp ( x¤ ) D 0.

(2.9)

In general, equation 2.9 is nonlinear and cannot be explicitly solved for a solution x¤ . However, we proceed by assuming the existence of a gradient factorization,

rx dp (x ) D a(x) P ( x ) x,

(2.10)

where a(x) is a positive scalar function and P (x ) is symmetric, positivedeŽnite, and diagonal. As discussed in Kreutz-Delgado and Rao (1997, 1998c) and Rao and Kreutz-Delgado (1999), this assumption is generally true for CSC sparsity functions dp (¢) and is key to understanding FOCUSS as a sparsity-inducing interior-point (AST-like) optimization algorithm.5 With the gradient factorization 2.10, the stationary points of equation 2.9 are readily shown to be solutions to the (equally nonlinear and implicit) system, x¤ D ( AT A C b ( x¤ )P ( x¤ ) ) ¡1 AT y D P

¡1

¤

(x ) A

T (b ( ¤ )

x I C AP

¡1

(2.11) ¤

T ) ¡1

(x ) A

y,

(2.12)

where b ( x ) D la(x) and the second equation follows from identity A.18. Although equation 2.12 is also not generally solvable in closed form, it does 5 This interpretation, which is not elaborated on here, follows from deŽning a diagonal positive-deŽnite afŽne scaling transformation matrix W (x) by the relation

P (x) D W ¡2 (x).

Dictionary Learning Algorithms for Sparse Representation

361

suggest the following relaxation algorithm, b x Ã

P ¡1 (b x ) AT (b (b x ) I C AP ¡1 (b x) AT ) ¡1 y,

(2.13)

which is to be repeatedly reiterated until convergence. Taking b ´ 0 in equation 2.13 yields the FOCUSS algorithm proved in Kreutz-Delgado and Rao (1997, 1998c) and Rao and Kreutz-Delgado (1999) to converge to a sparse solution of equation 2.7 for CSC sparsity functions 6 0 yields the regularized FOCUSS algorithm that will dp (¢). The case b D converge to a sparse solution of equation 2.6 (Rao, 1998; Engan et al., 2000; Rao et al., 2002). More computationally robust variants of equation 2.13 are discussed elsewhere (Gorodnitsky & Rao, 1997; Rao & Kreutz-Delgado, 1998a). Note that for the general regularized FOCUSS algorithm, 2.13, we have b (b x) D la(b x) , where l is the regularization parameter in equation 2.4. The function b ( x) is usually generalized to be a function of b x, y and the iteration number. Methods for choosing l include the quality-of-Žt criteria, the sparsity critera, and the L-curve (Engan, 2000; Engan et al., 2000; Rao et al., 2002). The quality-of-Žt criterion attempts to minimize the residual error y ¡ Ax (Rao, 1997), which can be shown to converge to a sparse solution (Rao & Kreutz-Delgado, 1999). The sparsity criterion requires that a certain number of elements of each x k be nonzero. The L-curve method adjusts l to optimize the trade-off between the residual and sparsity of x. The plot of dp (x ) versus dq ( y ¡ Ax) has an L shape, the corner of which provides the best trade-off. The corner of the L-curve is the point of maximum curvature and can be found by a one-dimensional maximization of the curvature function (Hansen & O’Leary, 1993). A hybrid approach known as the modiŽed L-curve method combines the Lcurve method on a linear scale and the quality-of-Žt criterion, which is used to place limits on the range of l that can be chosen by the L-curve (Engan, 2000). The modiŽed L-curve method was shown to have good performance, but it requires a one-dimensional numerical optimization step for each xk at each iteration, which can be computationally expensive for large vectors. 3 Dictionary Learning 3.1 Unknown, Nonrandom Dictionaries. The MLE framework treats parameters to be estimated as unknown but deterministic (nonrandom). In this spirit, we take the dictionary, A, to be the set of unknown but deterministic parameters to be estimated from the observation set Y D YN . In particular, bML is found from maximizing given YN , the maximum likelihood estimate A the likelihood function L ( A | YN ) D P (YN I A) . Under the assumption that

362

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

the observations are i.i.d., this corresponds to the optimization bML D arg max A A

N Y kD 1

P ( y kI A ) ,

Z P ( yk I A ) D

(3.1)

Z P ( y k, xI A) dx D

Z

P ( y k | xI A ) ¢ Pp ( x ) dx

Pq ( yk ¡ Ax ) ¢ Pp (x ) dx.

D

(3.2)

DeŽning the sample average of a function f ( y ) over the sample set YN D ( y1 , . . . , yN ) by h f ( y) iN D

N 1 X f ( yk ) , N kD 1

the optimization 3.1 can be equivalently written as bML D arg min ¡hlog(P ( yI A) ) iN . A A

(3.3)

Note that P ( y kI A ) is equal to the normalization factor b already encountered, but now with the dependence of b on A and the particular sample, y k, made explicit. The integration in equation 3.2 in general is intractable, and various approximations have been proposed to obtain an approximate bAML (Olshausen & Field, 1996; Lewicki & maximum likelihood estimate, A Sejnowski, 2000). In particular, the following approximation has been proposed (Olshausen & Field, 1996), b) ) , Pp (x ) ¼ d (x ¡ b xk ( A

(3.4)

where b) D arg max P ( yk , xI A b) , b xk ( A x

(3.5)

b for A. This approximation for k D 1, . . . , N, assuming a current estimate, A, corresponds to assuming that the source vector x k for which y k D Ax k is b) . With this approximation, the optimization 3.3 known and equal to b xk ( A becomes bAML D arg minhdq ( y ¡ Ab A x) N , A

(3.6)

Dictionary Learning Algorithms for Sparse Representation

363

which is an optimization over the sample average h¢iN of the functional 2.4 encountered earlier. Updating our estimate for the dictionary, bà A bAML , A

(3.7)

bAML has converged, hopefully we can iterate the procedure (3.5)–(3.6) until A bML D A bML (YN ) as the maximum likelihood (at least in the limit of large N) to A bML ( YN ) has well-known desirable asymptotic properties in the estimate A limit N ! 1. Performing the optimization in equation 3.6 for the q D 2 i.i.d. gaussian measurement noise case (º gaussian with known covariance s 2 ¢ I) corresponds to taking bb dq ( y ¡ A x) D

1 bb ky ¡ A xk2 , 2s 2

(3.8)

in equation 3.6. In appendix A, it is shown that we can readily obtain the unique batch solution, bAML D SyxO S ¡1 , A Ox xO

SyxO

D

N 1 X ykb xTk , N kD 1

(3.9)

SxO xO

D

N 1 X b x kb xTk . N kD 1

(3.10)

Appendix A derives the maximum likelihood estimate of A for the ideal case of known source vectors X D ( x1 , . . . , xN ) , ¡1 Known source vector case: AML D Syx Sxx ,

which is, of course, actually not computable since the actual source vectors are assumed to be hidden. As an alternative to using the explicit solution 3.9, which requires an often prohibitive n £ n inversion, we can obtain AAML iteratively by gradient descent on equations 3.6 and 3.8, N X bAML Ã A bAML ¡ m 1 A e kb xTk , N kD 1

bAMLb ek D A x k ¡ y k,

(3.11)

k D 1, . . . , N,

for an appropriate choice of the (possibly adaptive) positive step-size pabAML D A. b rameter m . The iteration 3.11 can be initialized as A A general iterative dictionary learning procedure is obtained by nesting the iteration 3.11 entirely within the iteration deŽned by repeatedly solvbAML , of the dictionary becomes ing equation 3.5 every time a new estimate, A

364

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

available. However, performing the optimization required in equation 3.5 is generally nontrivial (Olshausen & Field, 1996; Lewicki & Sejnowski, 2000). Recently, we have shown how the use of the FOCUSS algorithm results in an effective algorithm for performing the optimization required in equation 3.5 for the case when º is gaussian (Rao & Kreutz-Delgado, 1998a; Engan et al., 1999). This approach solves equation 3.5 using the afŽne-scaling transformation (AST)-like algorithms recently proposed for the low-noise case (Rao & Kreutz-Delgado, 1997, 1999; Kreutz-Delgado & Rao, 1997) and extended by regularization to the nontrivial noise case (Rao & Kreutz-Delgado, 1998a; Engan et al., 1999). As discussed in section 2.5, for the current dictionary b a solution to the optimization problem, 3.5, is provided by the estimate, A, repeated iteration, bT (b (b b ¡1 (b bT ) ¡1 y k, b x k à P ¡1 (b xk ) A x k ) I C AP xk ) A

(3.12)

k D 1, . . . , N, with P ( x ) deŽned as in equation 3.18, given below. This is the regularized FOCUSS algorithm (Rao, 1998; Engan et al., 1999), which has an interpretation as an AST-like concave function minimization algorithm. The proposed dictionary learning algorithm alternates between iteration 3.12 and iteration 3.11 (or the direct batch solution given by equation 3.9 if the inversion is tractable). Extensive simulations show the ability of the AST-based algorithm to recover an unknown 20 £ 30 dictionary matrix A completely (Engan et al., 1999). 3.2 Unknown, Random Dictionaries. We now generalize to the case where the dictionary A and the source vector set X D XN D ( x1 , . . . , xN ) are jointly random and unknown. We add the requirement that the dictionary is known to obey the constraint

A 2 A D compact submanifold of Rm£n . A compact submanifold of Rm£n is necessarily closed and bounded. On the constraint submanifold, the dictionary A has the prior probability density function P ( A) , which in the sequel we assume has the simple (uniform on A) form, ( A 2 A) ,

P ( A ) D cX

(3.13)

where X (¢) is the indicator function and c is a positive constant chosen to ensure that Z P (A) D P ( A ) dA D 1. A

The dictionary A and the elements of the set X are also all assumed to be mutually independent, P ( A, X ) D P ( A) P ( X ) D P ( A) Pp ( x1 ) , . . . , Pp ( xN ).

Dictionary Learning Algorithms for Sparse Representation

365

With the set of i.i.d. noise vectors, (º1 , . . . , ºN ) also taken to be jointly random with, and independent of, A and X, the observation set Y D YN D ( y1 , . . . , yN ) is assumed to be generated by the model 1.3. With these assumptions, we have P ( A, X | Y) D P (Y | A, X ) P ( A, X ) / P (Y ) D cX

(3.14)

( A 2 A ) P ( Y | A, X ) P ( X ) / P ( Y)

D

N cX ( A 2 A ) Y P ( y k | A, x k ) Pp ( xk ) P (Y ) kD 1

D

N cX ( A 2 A ) Y Pq ( y ¡ Ax k ) Pp ( x k ) , P (Y ) kD 1

using the facts that the observations are conditionally independent and P ( y k | A, X ) D P ( yk | A, xk ) . The jointly MAP estimates bMAP , X bMAP ) D ( A bMAP , b (A x1, MAP , . . . , b xN, MAP ) are found by maximizing a posteriori probability density P ( A, X | Y) simultaneously with respect to A 2 A and X. This is equivalent to minimizing the negative logarithm of P ( A, X | Y ), yielding the optimization problem, bMAP , X bMAP ) D arg min hdq ( y ¡ Ax) C ldp ( x) iN . (A A2A,X

(3.15)

Note that this is a joint minimization of the sample average of the functional 2.4, and as such is a natural generalization of the single (with respect to the set of source vectors) optimization previously encountered in equation 3.6. By Žnding joint MAP estimates of A and X, we obtain a problem that is much more tractable than the one of Žnding the single MAP estimate of A (which involves maximizing the marginal posterior density P ( A | Y )). The requirement that A 2 A, where A is a compact and hence bounded subset of Rm£n , is sufŽcient for the optimization problem 3.15 to avoid the degenerate solution,6 for k D 1, . . . , N, yk D Axk, with kAk ! 1 and kx kk ! 0.

(3.16)

This solution is possible for unbounded A because y D Ax is almost always solvable for x since learned overcomplete A’s are (generically) onto, and for any solution pair ( A, x) the pair ( a1 A, ax) is also a solution. This fact shows that the inverse problem of Žnding a solution pair ( A, x) is generally ill posed 6

kAk is any suitable matrix norm on A.

366

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

unless A is constrained to be bounded (as we have explicitly done here) or the cost functional is chosen to ensure that bounded A’s are learned (e.g., by adding a term monotonic in the matrix norm kAk to the cost function in equation 3.15). A variety of choices for the compact set A are available. Obviously, since different choices of A correspond to different a priori assumptions on the set of admissible matrices, A, the choice of this set can be expected to affect the performance of the resulting dictionary learning algorithm. We will consider two relatively simple forms for A. 3.3 Unit Frobenius–Norm Dictionary Prior. For the i.i.d. q D 2 gaussian measurement noise case of equation 3.8, algorithms that provably converge (in the low step-size limit) to a local minimum of equation 3.15 can be readily developed for the very simple choice,

A F D fA | kAkF D 1g ½ Rm£n ,

(3.17)

where kAkF denotes the Frobenius norm of the matrix A, kAk2F D tr(AT A ) , trace ( AT A ) , and it is assumed that the prior P ( A) is uniformly distributed on AF as per condition 3.13. As discussed in appendix A, AF is simply connected, and there exists a path in AF between any two matrices in AF. Following the gradient factorization procedure (Kreutz-Delgado & Rao, 1997; Rao & Kreutz-Delgado, 1999), we factor the gradient of d ( x) as

r d (x ) D a(x) P ( x ) x,

a(x) > 0,

(3.18)

where it is assumed that P (x ) is diagonal and positive deŽnite for all nonzero p x. For example, in the case where d ( x ) D kxkp , P ¡1 ( x ) D diag(|x[i]| 2¡p ).

(3.19)

Factorizations for other diversity measures d (x ) are given in Kreutz-Delgado and Rao (1997). We also deŽne b ( x) D la(x). As derived and proved in appendix A, a learning law that provably converges to a minimum of equation 3.15 on the manifold 3.17 is then given by d bT A b C b (b bT y kg, b x k D ¡V kf( A xk ) P (b x k ) )b xk ¡ A dt d b b ¡ tr( A bTd A b) A b) , m > 0, A D ¡m (d A dt

(3.20)

Dictionary Learning Algorithms for Sparse Representation

367

b is initialized to kAk b F D 1, V k are n £ n positive for k D 1, . . . , N, where A b deŽnite matrices, and the “error” d A is b D he(b dA x)b xT iN D

N 1 X e (b xk )b xTk , N kD 1

bb e (b xk ) D A x k ¡ y k,

(3.21)

which can be rewritten in the perhaps a more illuminating form (cf. equations 3.9 and 3.10), bD A bSxO xO ¡ SyxO . dA

(3.22)

A formal convergence proof of equation 3.20 is given in appendix A, where it is also shown that the right-hand side of the second equation in 3.20 correb onto the tangent space of A F , thereby sponds to projecting the error term d A b lies in the tangent space. Convergence of ensuring that the derivative of A the algorithm to a local optimum of equation 3.15 is formally proved by interpreting the loss functional as a Lyapunov function whose time derivabX b) is guaranteed tive along the trajectories of the adapted parameters ( A, to be negative deŽnite by the choice of parameter time derivatives shown in equation 3.20. As a consequence of the La Salle invariance principle, the loss functional will decrease in value, and the parameters will converge to the largest invariant set for which the time derivative of the loss functional is identically zero (Khalil, 1996). b and the vectors b Equation 3.20 is a set of coupled (between A x k) nonlinear differential equations that correspond to simultaneous, parallel updatb and b ing of the estimates A x k. This should be compared to the alternated separate (nonparallel) update rules 3.11 and 3.12 used in the AML algorithm described in section 3.1. Note also that (except for the trace term) the right-hand side of the dictionary learning update in equation 3.20 is of the same form as for the AML update law given in equation 3.11 (see also the discretized version of equation 3.20 given in equation 3.28 below). The key difference is the additional trace term in equation 3.20. This difference corresponds to a projection of the update onto the tangent space of the manifold 3.17, thereby ensuring a unit Frobenius norm (and hence boundedness) of the dictionary estimate at all times and avoiding the ill-posedness problem indicated in equation 3.16. It is also of interest to note that choosing V k to be the positive-deŽnite matrix, bT A b C b (b Vk D g k ( A x k ) P (b x k ) ) ¡1 , gk > 0,

(3.23)

in equation 3.20, followed by some matrix manipulations (see equation A.18 in appendix A), yields the alternative algorithm, d bT (b (b b ¡1 (b bT ) ¡1 y kg b xk D ¡gkfb x k ¡ P ¡1 (b xk ) A xk ) I C AP xk ) A dt

(3.24)

368

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

with g k > 0. In any event (regardless of the speciŽc choice of the positive deŽnite matrices V k as shown in appendix A), the proposed algorithm outb) , which satisŽes the implicit and lined here converges to a solution (b x k, A nonlinear relationships, bT (b (b b ¡1 (b bT ) ¡1 y, b x k D P ¡1 (b xk ) A x k ) I C AP xk ) A b D S T ( SxO xO ¡ cI ) ¡1 2 AF , A yxO

(3.25)

bTd A b) . for scalar c D tr( A To implement the algorithm 3.20 (or the variant using equation 3.24) in discrete time, a Žrst-order forward difference approximation at time t D tl can be used, b d x k (tl C1 ) ¡ b x k ( tl ) b x k ( tl ) ¼ dt tl C 1 ¡ tl

,

b x k[l C 1] ¡ b x k[l]

Dl

.

(3.26)

Applied to equation 3.24, this yields b [l] x k[l C 1] D (1 ¡ m l )b xk [l] C m lb xFOCUSS k

bT (b (b b ¡1 (b bT ) ¡1 y k b [l] D P ¡1 (b xFOCUSS xk ) A xk ) I C AP xk ) A k m l D g kD l ¸ 0.

(3.27)

b Similarly, discretizing the A-update equation yields the A-learning rule, equation 3.28, given below. More generally, taking m l to have a value between zero and one, 0 · m l · 1 yields an updated value b xk [l C 1], which is [l]. a linear interpolation between the previous value b x k[l] and b xFOCUSS k When implemented in discrete time, and setting m l D 1, the resulting Bayesian learning algorithm has the form of a combined iteration where we loop over the operations, bT (b (b b ¡1 (b bT ) ¡1 y k, b x k à P ¡1 (b xk ) A x k ) I C AP xk ) A k D 1, . . . , N

and

bà A b ¡ c (d A b ¡ tr( A bTd A b) A b) A

c > 0.

(3.28)

We call this FOCUSS-based, Frobenius-normalized dictionary-learning algorithm the FOCUSS-FDL algorithm. Again, this merged procedure should be compared to the separate iterations involved in the maximum likelihood b given by approach given in equations 3.11 and 3.12. Equation 3.28, with d A equation 3.21, corresponds to performing a Žnite step-size gradient descent

Dictionary Learning Algorithms for Sparse Representation

369

on the manifold AF . This projection in equation 3.28 of the dictionary update onto the tangent plane of AF (see the discussion in appendix A) ensures the well-behavedness of the MAP algorithm.7 The speciŽc step-size choice m l D 1, which results in the Žrst equation in equation 3.28, is discussed at length for the low-noise case in Rao and Kreutz-Delgado (1999). 3.4 Column-Normalized Dictionary Prior. Although mathematically very tractable, the unit-Frobenius norm prior, equation 3.17, appears to be somewhat too loose, judging from simulation results given below. In simulations with the Frobenius norm constraint AF , some columns of A can tend toward zero, a phenomenon that occurs more often in highly overcomplete A. This problem can be understood by remembering that we are using the 6 0 diversity measure, which penalizes columns associated with dp ( x) , p D terms in x with large magnitudes. If a column ai has a small relative magnitude, the weight of its coefŽcient x[i] can be large, and it will be penalized more than a column with a larger norm. This leads to certain columns being underused, which is especially problematic in the overcomplete case. An alternative, and more restrictive, form of the constraint set A is obtained by enforcing the requirement that the columns ai of A each be normalized (with respect to the Euclidean 2-norm) to the same constant value (Murray & Kreutz-Delgado, 2001). This constraint can be justiŽed by noting that Ax can be written as the nonunique weighted sum of the columns ai ,

Ax D

n X iD 1

ai x[i] D

n ³ ´ X ai i D1

ai

(ai x[i]) D A0 x0 ,

for any ai > 0, i D 1, . . . , n, showing that there is a column-wise ambiguity that remains even after the overall unit-Frobenius norm normalization has occurred, as one can now Frobenius-normalize the new matrix A0 . Therefore, consider the set of matrices on which has been imposed the column-wise constraint that » ­ ¼ ­ 2 1 T ­ AC D A ­ kai k D ai ai D , i D 1, . . . , n . (3.29) n The set AC is an mn ¡ n D n ( m ¡1) –dimensional submanifold of Rm£n . Note that every column of a matrix in A C has been normalized to the value p1n . 7 Because of the discrete-time approximation in equation 3.28,and even more generally because of numerical round-off effects in equation 3.20, a renormalization,

bà A b/ kAkF , A is usually performed at regular intervals.

370

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

In fact, any constant value for the column normalization can be used (including the unit normalization), but, as shown in appendix B, the particular normalization of p1n results in AC being a proper submanifold of the mn ¡ 1 dimensional unit Frobenius manifold AF ,

A C ½ AF , indicating that a tighter constraint on the matrix A is being imposed. Again, it is assumed that the prior P ( A ) is uniformly distributed on AC in the manner of equation 3.13. As shown in appendix B, AC is simply connected. A learning algorithm is derived for the constraint AC in appendix B, following much the same approach as in appendix A. Because the derivation of the b x k update to Žnd sparse solutions does not depend on the form of b update in algorithm 3.28 needs to be modiŽed. the constraint A, only the A Each column ai is now updated independently (see equation B.17), ai à ai ¡ c ( I ¡b aib aTi )d ai i D 1, . . . , n,

(3.30)

b in equation 3.21. We call the resulting where dai is the ith column of d A column-normalized dictionary-learning algorithm the FOCUSS-CNDL algorithm. The implementation details of the FOCUSS-CNDL algorithm are presented in section 4.2. 4 Algorithm Implementation

The dictionary learning algorithms derived above are an extension of the FOCUSS algorithm used for obtaining sparse solutions to the linear inverse problem y D Ax to the case where dictionary learning is now required. We refer to these algorithms generally as FOCUSS-DL algorithms, with the unit Frobenius-norm prior-based algorithm denoted by FOCUSS-FDL and the column-normalized prior-base algorithm by FOCUSS-CNDL. In this section, the algorithms are stated in the forms implemented in the experimental tests, where it is shown that the column normalization-based algorithm achieves higher performance in the overcomplete dictionary case. 4.1 Unit Frobenius-Norm Dictionary Learning Algorithm. We now summarize the FOCUSS-FDL algorithm derived in section 3.3. For each of the data vectors yk, we update the sparse source vectors x k using the FOCUSS algorithm:

P ¡1 (b x k ) D diag(|b x k[i]| 2¡p )

bT (lkI C AP b ¡1 (b bT ) ¡1 y k b x k à P ¡1 (b xk ) A xk ) A

(FOCUSS),

(4.1)

Dictionary Learning Algorithms for Sparse Representation

371

where lk D b (b xk ) is the regularization parameter. After updating the N b is reestimated, source vectors x k, k D 1, . . . , n, the dictionary A

SyxO

D

N 1 X y kb xTk N kD 1

SxO xO

D

N 1 X b x kb xTk N kD 1

bD A bSxO dA O x ¡ S yxO bà A b ¡ c (d A b ¡ tr( A bTd A b) A b) , A

c > 0,

(4.2)

where c controls the learning rate. For the experiments in section 5, the data block size is N D 100. During each iteration all training vectors are updated using equation 4.1, with a corresponding number of dictionary updates b it is renormalized using equation 4.2. After each update of the dictionary A, b to have unit Frobenius norm, k AkF D 1. The learning algorithm is a combined iteration, meaning that the FOCUSS algorithm is allowed to run for only one iteration (not until full convergence) before the A update step. This means that during early iterations, the b x k are in general not sparse. To facilitate learning A, the covariances S yOx and SxO xO are calculated with sparsiŽed b x k that have all but the rQ largest elements set to zero. The value of rQ is usually set to the largest desired number of nonzero elements, but this choice does not appear to be critical. The regularization parameter lk is taken to be a monotonically increasing function of the iteration number, lk D lmax tanh (10¡3 ¢ ( iter ¡ 1500) ) C 1) .

(4.3)

While this choice of l k does not have the optimality properties of the modiŽed L-curve method (see section 2.5), it does not require a one-dimensional optimization for each b x k and so is much less computationally expensive. This is further discussed below. 4.2 Column Normalized Dictionary Learning Algorithm. The improved version of the algorithm called FOCUSS-CNDL, which provides increased accuracy especially in the overcomplete case, was proposed in Murray and Kreutz-Delgado (2001). The three key improvements are colb an efŽcient way of adjusting umn normalization that restricts the learned A, the regularization parameter l k, and reinitialization to escape from local optima. The column-normalized learning algorithm discussed in section 3.4 and derived in appendix B is used. Because the b x k update does not depend on the constraint set A, the FOCUSS algorithm in equation 4.1 is used to

372

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

update N vectors, as discussed in section 4.1. After every N source vectors are updated, each column of the dictionary is then updated as ai à ai ¡ c ( I ¡ b aib aTi )d ai i D 1, . . . , n,

(4.4)

b which is found using equation 4.2. After where dai are the columns of d A, updating each ai , it is renormalized to kai k2 D 1 / n by ai à p

ai , nkai k

(4.5)

b F D 1 as shown in section B.1. which also ensures that kAk The regularization parameter lk may be set independently for each vector in the training set, and a number of methods have been suggested, including quality of Žt (which requires a certain level of reconstruction accuracy), sparsity (requiring a certain number of nonzero elements), and the L-curve which attempts to Žnd an optimal trade-off (Engan, 2000). The L-curve method works well, but it requires solving a one-dimensional optimization for each l k, which becomes computationally expensive for large problems. Alternatively, we use a heuristic method that allows the trade-off between error and sparsity to be tuned for each application, while letting each training vector y k have its own regularization parameter lk to improve the quality of the solution, Á l k D lmax

! bx kk kyk ¡ Ab 1¡ , ky kk

l k, lmax > 0.

(4.6)

For data vectors that are represented accurately, lk will be large, driving the algorithm to Žnd sparser solutions. If the SNR can be estimated, we can set lmax D (SNR) ¡1 . The optimization problem, 3.15, is concave when p · 1, so there will be multiple local minima. The FOCUSS algorithm is guaranteed to converge only to one of these local minima, but in some cases, it is possible to determine when that has happened by noticing if the sparsity is too low. Periodically (after a large number of iterations), the sparsity of the solutions b xk is checked, and if found too low, b x k is reinitialized randomly. The algorithm is also sensitive to initial conditions, and prior information may be incorporated into the initialization to help convergence to the global solution. 5 Experimental Results

Experiments were performed using complete dictionaries ( n D m) and overcomplete dictionaries ( n > m ) on both synthetically generated data and

Dictionary Learning Algorithms for Sparse Representation

373

natural images. Performance was measured in a number of ways. With synthetic data, performance measures include the SNR of the recovered sources x k compared to the true generating source and comparing the learned dictionary with the true dictionary. For images of natural scenes, the true underlying sources are not known, so the accuracy and efŽciency of the image coding are found. 5.1 Complete Dictionaries:Comparison with ICA. To test the FOCUSSFDL and FOCUSS-CNDL algorithms, simulated data were created following the method of Engan et al. (1999) and Engan (2000). The dictionary A of size 20 £20 was created by drawing each element aij from a normal distribution with m D 0, s 2 D 1 (written as N (0, 1) ) followed by a normalization to ensure that kAkF D 1. Sparse source vectors x k, k D 1, . . . , 1000 were created with r D 4 nonzero elements, where the r nonzero locations are selected at random (uniformly) from the 20 possible locations. The magnitudes of each nonzero element were also drawn from N (0, 1) and limited so that |x kl | > 0.1. The input data yk were generated using y D Ax (no noise was added). For the Žrst iteration of the algorithm, the columns of the initializabinit, were taken to be the Žrst n D 20 training vectors yk. tion estimate, A The initial xk estimates were then set to the pseudoinverse solution b xk D bT ( A binit A bT ) ¡1 y k. The constant parameters of the algorithm were set as A init init follows: p D 1.0, c D 1.0, and lmax D 2 £ 10¡3 (low noise, assumed SNR ¼ 27 dB). The algorithms were run for 200 iterations through the entire b was updated after updating 100 data data set, and during each iteration, A vectors b x k. To measure performance, the SNR between the recovered sources b x k and the the true sources x k were calculated. Each element xk [i] for Žxed i was considered as a time-series vector with 1000 elements, and SNRi for each was found using

³ SNRi D 10 log10

kx k[i]k2 kx k[i] ¡ b xk[i]k2

´ .

(5.1)

The Žnal SNR is found by averaging SNRi over the i D 1, . . . , 20 vectors and 20 trials of the algorithms. Because the dictionary A is learned only to within a scaling factor and column permutations, the learned sources must be matched with corresponding true sources and scaled to unit norm before the SNR calculation is done. The FOCUSS-FDL and FOCUSS-CNDL algorithms were compared with extended ICA (Lee, Girolami, & Sejnowski, 1999) and Fastica 8 (Hyv¨arinen 8 Matlab and C versions of extended ICA can be found on-line at http://www.cnl.salk. edu/»tewon/ICA/code.html. Matlab code for FastICA can be found at: http://www.cis. hut.Ž/projects/ica/fastica/.

374

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

Figure 1: Comparison between FOCUSS-FDL, FOCUSS-CNDL, extended ICA, and FastICA on synthetically generated data with a complete dictionary A, size 20 £ 20. The SNR was computed between the recovered sources b xk and the true sources xk . The mean SNR and standard deviation were computed over 20 trials.

et al., 1999). Figure 1 shows the SNR for the tested algorithms. The SNR for FOCUSS-FDL is 27.7 dB, which is a 4.7 dB improvement over extended ICA, and for FOCUSS-CNDL, the SNR is 28.3 dB. The average run time for FOCUSS-FDL/CNDL was 4.3 minutes, for FastICA 0.10 minute, and for extended ICA 0.19 minute on a 1.0 GHz Pentium III Xeon computer. 5.2 Overcomplete Dictionaries. To test the ability to recover the true A and x k solutions in the overcomplete case, dictionaries of size 20 £ 30 and 64 £ 128 were generated. Diversity r was set to Žxed values (4 and 7) and uniformly randomly (5–10 and 10–15). The elements of A and the sources x k were created as in section 5.1. The parameters were set as follows: p D 1.0, c D 1.0, lmax D 2 £ 10¡3 . The algorithms were run for 500 iterations through the entire data set, and b was updated after updating 100 data vectors b during each iteration, A xk . As a measure of performance, we Žnd the number of columns of A that were matched during learning. Because A can be learned only to within column permutations and sign and scale changes, the columns are normalized b is rearranged columnwise so that b so that kb ai k D kaj k D 1, and A aj is given the index of the closest match in A (in the minimum two-norm sense). A match is counted if

1 ¡ |aTib ai | < 0.01.

(5.2)

Dictionary Learning Algorithms for Sparse Representation

375

Table 1: Synthetic Data Results. Learned A Columns Algorithm

Size of A

Diversity, r

FOCUSS-FDL FOCUSS-CNDL FOCUSS-CNDL FOCUSS-CNDL FOCUSS-FDL FOCUSS-CNDL

20 £ 30 20 £ 30 64 £ 128 64 £ 128 64 £ 128 64 £ 128

7 7 7 5–10 10–15 10–15

Average SD 25.3 28.9 125.3 126.3 102.8 127.4

3.4 1.6 2.1 1.3 4.5 1.3

Learned x

%

Average

SD

%

84.2 96.2 97.9 98.6 80.3 99.5

675.9 846.8 9414.0 9505.5 4009.6 9463.4

141.0 97.6 406.5 263.8 499.6 330.3

67.6 84.7 94.1 95.1 40.1 94.6

Similarly, the number of matching b xk are counted (after rearranging the b elements in accordance with the indices of the rearranged A), 1 ¡ |xTib xi | < 0.05.

(5.3)

If the data are generated by an A that is not column normalized, other measures of performance need to be used to compare x k and b x k. The performance is summarized in Table 1, which compares the FOCUSSFDL with the column-normalized algorithm (FOCUSS-CNDL). For the 20 £ 30 dictionary, 1000 training vectors were used, and for the 64£128 dictionary 10,000 were used. Results are averaged over four or more trials. For the 64 £ 128 matrix and r D 10–15, FOCUSS-CNDL is able to recover 99.5% (127.4/128) of the columns of A and 94.6% (9463/10,000) of the solutions x k to within the tolerance given above. This shows a clear improvement over FOCUSS-FDL, which learns only 80.3% of the A columns and 40.1% of the solutions x k. Learning curves for one of the trials of this experiment (see Figure 2) show that most of the columns of A are learned quickly within the Žrst 100 iterations, and that the diversity of the solutions drops to the desired level. Figure 2b shows that it takes somewhat longer to learn the xk correctly and that reinitialization of the low sparsity solutions (at iterations 175 and 350) helps to learn additional solutions. Figure 2c shows the diversity at each iteration, measured as the average number of elements of each b x k that are larger than 1 £ 10¡4 . 5.3 Image Data Experiments. Previous work has shown that learned basis functions can be used to code data more efŽciently than traditional Fourier or wavelet bases (Lewicki & Olshausen, 1999). The algorithm for Žnding overcomplete bases in Lewicki and Olshausen (1999) is also designed to solve problem 1.1 but differs from our method in a number of ways, including using only the Laplacian prior (p D 1), and using conjugate gradient optimization for Žnding sparse solutions (whereas we use the FOCUSS algorithm). It is widely believed that overcomplete representations

376

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

Figure 2: Performance of the FOCUSS-CNDL algorithm with overcomplete dictionary A, size 64 £ 128. (a) Number of correctly learned columns of A at each iteration. (b) Number of sources xk learned. (c) Average diversity (n-sparsity) of the x k . The spikes in b and c indicate where some solutions b x k were reinitialized because they were not sparse enough.

Dictionary Learning Algorithms for Sparse Representation

377

are more efŽcient than complete bases, but in Lewicki and Olshausen (1999), the overcomplete code was less efŽcient for image data (measured in bits per pixel entropy), and it was suggested that different priors could be used to improve the efŽciency. Here, we show that our algorithm is able to learn more efŽcient overcomplete codes for priors with p < 1. The training data consisted of 10,000 8 £ 8 image patches drawn at random from black and white images of natural scenes. The parameter p was varied from 0.5–1.0, and the FOCUSS-CNDL algorithm was trained for 150 iterations. The complete dictionary (64 £ 64) was compared with the 2£ overcomplete dictionary (64 £ 128). Other parameters were set: c D 0.01, lmax D 2 £ 10¡3 . The coding efŽciency was measured using the entropy (bits per pixel) method described in Lewicki and Olshausen (1999). Figure 3 plots the entropy versus reconstruction error (root mean square error, RMSE) and shows that when p < 0.9, the entropy is less for the overcomplete representation at the same RMSE. An example of coding an entire image is shown in Figure 4. The original test image (see Figure 4a) of size 256 £ 256 was encoded using the learned dictionaries. Patches from the test image were not used during training. Table 2 gives results for low- and high-compression cases. In both cases, coding with the overcomplete dictionary (64£128) gives higher compression (lower bits per pixel) and lower error (RMSE). For the high-compression case (see

Figure 3: Comparing the coding efŽciency of complete and 2£ overcomplete representations on 8 £8 pixel patches drawn from natural images. The points on the curve are the results from different values of p, at the bottom right, p D 1.0, and at the top left, p D 0.5. For smaller p, the overcomplete case is more efŽcient at the same level of reconstruction error (RMSE).

378

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

Figure 4: Image compression using complete and overcomplete dictionaries. Coding with an overcomplete dictionary is more efŽcient (fewer bits per pixel) and more accurate (lower RMSE). (a) Original image of size 256 £ 256 pixels. (b) Compressed with 64 £ 64 complete dictionary to 0.826 bits per pixel at RMSE = 0.329. (c) Compressed with 64 £ 128 overcomplete dictionary to 0.777 bits per pixel at RMSE = 0.328.

Dictionary Learning Algorithms for Sparse Representation

379

Table 2: Image Compression Results. Dictionary Size 64 £ 64 64 £ 128 64 £ 64 64 £ 128

p

Compression (bits/pixel)

RMSE

0.5 0.6 0.5 0.6

2.450 2.410 0.826 0.777

0.148 0.141 0.329 0.328

Average Diversity 17.3 15.4 4.5 4.0

Figures 4b and 4c), the 64 £ 128 overcomplete dictionary gives compression of 0.777 bits per pixel at error 0.328, compared to the 64 £ 64 complete dictionary at 0.826 bits per pixel at error 0.329. The amount of compression was selected by adjusting lmax (the upper limit of the regularization parameter). For high compression, lmax D 0.02, and for low compression, lmax D 0.002. 6 Discussion and Conclusions

We have applied a variety of tools and perspectives (including ideas drawn from Bayesian estimation theory, nonlinear regularized optimization, and the theory of majorization and convex analysis) to the problem of developing algorithms capable of simultaneously learning overcomplete dictionaries and solving sparse source-vector inverse problems. The test experiment described in section 5.2 is a difŽcult problem designed to determine if the proposed learning algorithm can solve for the known true solutions for A and the spare source vectors xk to the underdetermined inverse problem y k D Axk . Such testing, which does not appear to be regularly done in the literature, shows how well an algorithm can extract stable and categorically meaningful solutions from synthetic data. The ability to perform well on such test inverse problems would appear to be at least a necessary condition for an algorithm to be trustworthy in domains where a physically or biologically meaningful sparse solution is sought, such as occurs in biomedical imaging, geophysical seismic sounding, and multitarget tracking. The experimental results presented in section 5.2 show that the FOCUSSDL algorithms can recover the dictionary and the sparse sources vectors. This is particularly gratifying when one considers that little or no optimization of the algorithm parameters has been done. Furthermore, the convergence proofs given in the appendixes show convergence only to local optima, whereas one expects that there will be many local optima in the cost function because of the concave prior and the generally multimodal form of the cost function. One should note that algorithm 3.28 was constructed precisely with the goal of solving inverse problems of the type considered here, and therefore

380

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

one must be careful when comparing the results given here with other algorithms reported in the literature. For instance, the mixture-of-gaussians prior used in Attias (1999) does not necessarily enforce sparsity. While other algorithms in the literature might perform well on this test experiment, to the best of our knowledge, possible comparably performing algorithms such as Attias (1999), Girolami (2001), Hyv¨arinen et al. (1999), and Lewicki and Olshausen (1999) have not been tested on large, overcomplete matrices to determine their accuracy in recovering A, and so any comparison along these lines would be premature. In section 5.1, the FOCUSS-DL algorithms were compared to the well-known extended ICA and FastICA algorithms in a more conventional test with complete dictionaries. Performance was measured in terms of the accuracy (SNR) of the recovered sources x k, and both FOCUSS-DL algorithms were found to have signiŽcantly better performance (albeit with longer run times). We have also shown that the FOCUSS-CNDL algorithm can learn an overcomplete representation, which can encode natural images more efŽciently than complete bases learned from data (which in turn are more efŽcient than standard nonadaptive bases, such as Fourier or wavelet bases; Lewicki & Olshausen, 1999). Studies of the human visual cortex have shown a higher degree of overrepresentation of the fovea compared to the other mammals, which suggests an interesting connection between overcomplete representations and visual acuity and recognition abilities (Popovic & Sjostrand, ¨ 2001). Because the coupled dictionary learning and sparse-inverse solving algorithms are merged and run in parallel, it should be possible to run the algorithms in real time to track dictionary evolution in quasistationary environments once the algorithm has essentially converged. One way to do this would be to constantly present randomly encountered new signals, y k, to the algorithm at each iteration instead of the original training set. One also has to ensure that the dictionary learning algorithm is sensitive to the new data so that dictionary tracking can occur. This would be done by an appropriate adaptive Žltering of the current dictionary estimate driven by the new-data derived corrections, similar to techniques used in the adaptive Žltering literature (Kalouptsidis & Theodoridis, 1993).

Appendix A: The Frobenius-Normalized Prior Learning Algorithm

Here we provide a derivation of the algorithm 3.20–3.21 and prove that it converges to a local minimum of equation 3.15 on the manifold AF D fA | kAkF D 1g ½ Rm£n deŽned in equation 3.17. Although we focus on the development of the learning algorithm on AF , the derivations in sections A.2 and A.3, and the beginning of subsection are done for a general constraint manifold A.

Dictionary Learning Algorithms for Sparse Representation

381

A.1 Admissible Matrix Derivatives.

A.1.1 The Constraint Manifold AF . In order to determine the structural d form of admissible derivatives, AP D dt A for matrices belonging to AF ,9 it is useful to view A F as embedded in the Žnite-dimensional Hilbert space of matrices, Rm£n , with inner product hA, Bi D tr(AT B ) D tr(BT A ) D tr(ABT ) D tr(BAT ) . The corresponding matrix norm is the Frobenius norm, p p kAk D kAkF D tr AT A D tr AAT . We will call this space the Frobenius space and the associated inner product the Frobenius inner product. It is useful to note the isometry, A 2 Rm£n () A D vec ( A ) 2 Rmn , where A is the mn-vector formed by stacking the columns of A. Henceforth, bold type represents the stacked version of a matrix (e.g., B D vec (B ) ). The stacked vector A belongs to the standard Hilbert space Rmn , which we shall henceforth refer to as the stacked space. This space has the standard Euclidean inner product and norm, p hA , B i D A T B , kA k D A T A . It is straightforward to show that hA, Bi D hA , B i and

kAk D kA k.

In particular, we have A 2 AF () kAk D kA k D 1. Thus, the manifold in equation 3.17 corresponds to the (mn¡1)–dimensional unit sphere in the stacked space, Rmn (which, with a slight abuse of notation, we will continue to denote by A F ). It is evident that AF is simply connected so that a path exists between any two elements of A F and, in particular, a path exists between any initial value for a dictionary, Ainit 2 AF , used to initialize a learning algorithm, and a desired target value, AŽnal 2 AF .10 9 Equivalently, we want to determine the structure of elements AP of the tangent space, TAF , to the smooth manifold AF at the point A. 10 For example, for 0 · t · 1, take the t–parameterized path,

A (t) D

(1 ¡ t)A init C tA Žnal . k(1 ¡ t)A init C tA Žnal k

382

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

A.1.2 Derivatives on AF : The Tangent Space T AF . Determining the form of admissible derivatives on equation 3.17 is equivalent to determining the form of admissible derivatives on the unit Rmn –sphere. On the unit sphere, we have the well-known fact that A 2 AF H)

d P D 0 H) A P ? A. kA k2 D 2A T A dt

P is A P D ¤ Q , where Q is arbitrary and This shows that the general form of A ³ D

¤

I¡

AA T

kA k2

´ D ( I ¡ AA T )

(A.1)

is the stacked space projection operator onto the tangent space of the unit Rmn –sphere at the point A (note that we used the fact that kAk D 1). The projection operator ¤ is necessarily idempotent, ¤ D ¤ 2 . ¤ is also self-adjoint, ¤ D ¤ ¤ , where the adjoint operator ¤ ¤ is deŽned by the requirement that h¤

¤

Q 1 , Q 2 i D hQ 1 , ¤ Q 2 i,

for all Q 1 , Q 2 2 Rmn ,

showing that ¤ is an orthogonal projection operator. In this case, ¤ ¤ D ¤ T , so that self-adjointness corresponds to ¤ being symmetric. One can readily show that an idempotent, self-adjoint operator is nonnegative, which in this case corresponds to the symmetric, idempotent operator ¤ being a positive semideŽnite matrix. This projection can be easily rewritten in the Frobenius space, P D ¤ Q D Q ¡ hA , Q iA () AP D L Q D Q ¡ hA, QiA A D Q ¡ tr(AT Q) A.

(A.2)

Of course, this result can be derived directly in the Frobenius space using the fact that A 2 AF H)

d P D 2 tr(AT AP ) D 0, kAk2 D 2hA, Ai dt

from which it is directly evident that AP 2 T A F at A 2 A F

,

P D tr AT AP D 0, hA, Ai

(A.3)

and therefore AP must be of the form11 tr(AT Q ) AP D L Q D Q ¡ A D Q ¡ tr(AT Q ) A. tr(AT A ) 11

It must be the case that L D I ¡

notation (Dirac, 1958).

| AihA| kAk 2

(A.4)

D I ¡ |AihA|, using the physicist’s “bra-ket”

Dictionary Learning Algorithms for Sparse Representation

383

One can verify that L is idempotent and self-adjoint and is therefore a nonnegative, orthogonal projection operator. It is the orthogonal projection operator from Rm£n onto the tangent space T AF . In the stacked space (with some additional abuse of notation), we represent the quadratic form for positive semideŽnite symmetric matrices W as kA k2W D A T WA . Note that this deŽnes a weighted norm if, and only if, W is positive deŽnite, which might not be the case by deŽnition. In particular, when W D ¤ , the quadratic form kA k2 is only positive semideŽnite. Finally, note from ¤ equation A.4 that 8A 2 A F , L Q D 0 () Q D cA, with c D tr(AT Q) .

(A.5)

A.2 Minimizing the Loss Function over a General Manifold A. Con-

sider the Lyapunov function, VN (X, A ) D hdq ( y ¡ Ax ) C ldp (x ) iN ,

A 2 A,

(A.6)

where A is some arbitrary but otherwise appropriately deŽned constraint manifold associated with the prior, equation 3.13. Note that this is precisely the loss function to be minimized in equation 3.15. If we can determine smooth parameter trajectories (i.e., a parameter-vector adaptation P AP ) such that along these trajectories VP ( X, A) · 0, then as a conserule) ( X, quence of the La Salle invariance principle (Khalil, 1996), the parameter values will converge to the largest invariant set (of the adaptation rule viewed as a nonlinear dynamical system) contained in the set C D f(X, A ) | VP N (X, A ) ´ 0 and A 2 A g.

(A.7)

The set C contains the local minima of VN . With some additional technical assumptions (generally dependent on the choice of adaptation rule), the elements of C will contain only local minima of VN . Assuming the i.i.d. q D 2 gaussian measurement noise case of equation 3.8, 12 the loss (Lyapunov) function to be minimized is then ½ ¾ 1 kAx ¡ yk2 C ldp ( x) , A 2 A, (A.8) VN (X, A ) D 2 N which is essentially the loss function to be minimized in equation 3.15. 13 12 Note that in the appendix, unlike the notation used in equations 3.8 et seq., the “hat” notation has been dropped. Nonetheless, it should be understood that the quantities A and X are unknown parameters to be optimized over in equation 3.15, while the measured signal vectors Y are known. 13 The factor 12 is added for notational convenience and does not materially affect the derivation.

384

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

Suppose for the moment, as in equations 3.4 through 3.9, that X is assumed to be known, and note that then (ignoring constant terms depending on X and Y) VN can be rewritten as VN ( A) D htr(Ax ¡ y ) ( Ax ¡ y ) T iN

D htr(AxxT AT ) ¡ 2 tr(AxyT ) C tr(yyT ) iN D tr A Sxx AT ¡ 2 tr A Sxy

VN ( A) D trfA Sxx AT ¡ 2A Sxyg, T deŽned as in equation 3.10. Using standard results for Sxx and S xy D Syx from matrix calculus (Dhrymes, 1984), we can show that VN ( A ) is minimized by the solution 3.9. This is done by setting

@ @A

b) D 0, VN ( A

and using the identities (valid for W symmetric) @ @A

tr AWAT D 2AW and

@ @A

tr AB D BT .

This yields (assuming that Sxx is invertible), @ @A

b) D 2A bSxx ¡ 2 S T D 2( A bSxx ¡ Syx ) D 0, VN ( A xy b D SyxO S ¡1 , )A xO xO

which is equation 3.9, as claimed. For Sxx nonsingular, the solution is unique and globally optimal. This is, of course, a well-known result. Now return to the general case, equation A.8, where both X and A are unknown. For the data indexed by k D 1, . . . , N, deŽne the quantities d k D dp ( xk ) , e ( x) D Ax ¡ y, and e k D Ax k ¡ yk. The loss function and its time derivative can be written, ½ VN ( X, A) D

1 T e ( x ) e (x ) C ldp (x ) 2

¾ N

P N VP N ( X, A) D heT ( x ) eP (x ) C lr T dp ( x ) xi

P ) C lr T dp (x ) xi P N. D heT ( x ) ( AxP C Ax

Then, to determine an appropriate adaptation rule, note that VP N D T1 C T2 ,

(A.9)

Dictionary Learning Algorithms for Sparse Representation

385

where P ND T1 D h(eT ( x ) A C lr T dp ( x ) ) xi

N 1 X ( eT A C lr T d k ) xP k N kD 1 k

(A.10)

and N X P ND 1 P k. T2 D heT ( x ) Axi eT Ax N kD 1 k

(A.11)

Enforcing the separate conditions T1 · 0 and T2 · 0,

(A.12)

(as well as the additional condition that A 2 A) will be sufŽcient to ensure that VP N · 0 on A. In this case, the solution-containing set C of equation A.7 is given by C D f(X, A ) | T1 (X, A ) ´ 0, T2 ( X, A) ´ 0 and A 2 Ag.

(A.13)

Note that if A is known and Žxed, then T 2 ´ 0, and only the Žrst condition of equation A.12 (which enforces learning of the source vectors, x k) is of concern. Contrawise, if source vectors x k, which ensure that e ( x k ) D 0 are Žxed and known, then T1 ´ 0, and the second condition of equation A.12 (which enforces learning of the dictionary matrix, A) is at issue. A.3 Obtaining the x k Solutions with the FOCUSS Algorithm. We now develop the gradient factorization-based derivation of the FOCUSS algorithm, which provides estimates of x k while satisfying the Žrst convergence condition of equation A.12. The constraint manifold A is still assumed to be arbitrary. To enforce the condition T1 · 0 and derive the Žrst adaptation rule given in equation 3.20, we note that we can factor r d k D r d ( x k ) as (Kreutz-Delgado & Rao, 1997; Rao & Kreutz-Delgado, 1999)

r d k D akP kx k with ak D axk > 0 and P xk D P k positive deŽnite and diagonal for all nonzero x k. Then, deŽning b k D lak > 0 and selecting an arbitrary set of (adaptable) symmetric positive-deŽnite matrices V k, we choose the learning rule xP k D ¡V kfAT ek C lr d kg D ¡Vkf(AT A C b kP k ) x k ¡ AT y kg, k D 1, . . . , N,

(A.14)

386

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

which is the adaptation rule for the state estimates x k D b xk given in the Žrst line of equation 3.20. With this choice, we obtain T 1 D ¡hkAT e (x ) C lr d ( x) k2V iN D ¡

N 1 X kAT ek C lr d kk2Vk N kD 1

D ¡

N 1 X k(AT A C b kP k ) xk ¡ AT y kk2Vk · 0, N kD 1

(A.15)

as desired. Assuming convergence to the set A.13 (which will be seen to be the case after we show below how to ensure that we also have T2 · 0), we will asymptotically obtain (reintroducing the “hat” notation to now denote converged parameter estimates) bT A b C b kP k )b bT y kk2 ´ 0, k D 1, . . . , N, k( A xk ¡ A Vk which is equivalent to bT A b C b kP k ) ¡1 A bT y k, k D 1, . . . , N, b xk D ( A

(A.16)

at convergence. This is also equivalent to the condition given in the Žrst line of equation 3.25, as shown below. Exploiting the fact that V k in equation A.14 are arbitrary (subject to the symmetry and positive-deŽniteness constraint), let us make the speciŽc choice shown in equation 3.23, V k D gk ( AT A C b kP k ) ¡1 , g k > 0, k D 1, . . . , N.

(A.17)

Also note the (trivial) identity, ( AT A C bP ) P ¡1 AT D AT ( AP ¡1 AT C bI) , which can be recast nontrivially as ( AT A C bP ) ¡1 AT D P ¡1 AT (b I C AP ¡1 AT ) ¡1 .

(A.18)

With equations A.17 and A.18, the learning rule, A.14, can be recast as T (b C T ) ¡1 xP k D ¡gkfx k ¡ P ¡1 AP ¡1 ykg, k D 1, . . . , N, kI k A k A

(A.19)

which is the alternative learning algorithm, 3.24. At convergence (when T1 ´ 0) we have the condition shown in the Žrst line of equation 3.25, bT (b kI C AP b ¡1 A bT ) ¡1 y k. b x k D P ¡1 A

(A.20)

Dictionary Learning Algorithms for Sparse Representation

387

This also follows from the convergence condition A.16 and the identity A.18, showing that the result, A.20, is independent of the speciŽc choice of V k > 0. Note from equations A.14 and A.15 that T1 ´ 0 also results in xP k ´ 0 for k D 1, . . . , N, so that we will have converged to constant values, b x k, which satisfy equation A.20. For the case of known, Žxed A, the learning rule derived here will converge to sparse solutions, x k, and when discretized as in section 3.3, yields equation 3.27, which is the known dictionary FOCUSS algorithm (Rao & Kreutz-Delgado, 1998a, 1999). Note that the derivation of the sparse source-vector learning algorithm here, which enforces the condition T1 · 0, is entirely independent of any constraints placed on A (such as, for example, the unit Frobenius-norm and column-norm constraints considered in this article) or of the form of the A-learning rule. Thus alternative choices of constraints placed on A, as considered in Murray and Kreutz-Delgado (2001) and described in appendix B, will not change the form of the x k-learning rule derived here. Of course, because the x k learning rule is strongly coupled to the A-learning rule, algorithmic performance and speed of convergence may well be highly sensitive to conditions placed on A and the speciŽc A learning algorithm used. A.4 Learning the Dictionary A.

A.4.1 General Results. We now turn to the enforcement of the second convergence condition, T2 · 0 and the development of the dictionary adaptation rule shown in equation 3.20. First, as in equation 3.21, we deŽne the error term dA as dA D he(x) xT iN D

N 1 X e ( xk ) xTk D A Sxx ¡ Syx , N kD 1

(A.21)

using the fact that e ( x) D Ax ¡ y. Then, from equation A.11, we have P N D htr(xe T ( x ) AP ) iN D tr(hxe T ( x ) iN AP ) T2 D heT ( x ) Axi P. D tr(dAT AP ) D d A T A

(A.22)

So far, these steps are independent of any speciŽc constraints that may be placed on A, other than that the manifold A be smooth and compact. With A constrained to lie on a speciŽed smooth, compact manifold, to ensure correct learning behavior, it is sufŽcient to impose the constraint that AP lies in the tangent space to the manifold and the condition that T2 · 0. A.4.2 Learning on the Unit Frobenius Sphere, AF . To ensure that T2 · 0 and that AP is in the tangent space of the unit sphere in the Frobenius space Rmn , we take P D ¡m ¤ d A () AP D ¡m LdA D ¡m (d A ¡ tr(ATdA) A ) , m > 0, A

(A.23)

388

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

which is the adaptation rule given in equation 3.20. With this choice, and using the positive semideŽniteness of ¤ , we have T 2 D ¡m kd A k2¤ · 0,

as required. Note that at convergence, the condition T2 ´ 0, yields AP ´ 0, so that we will have converged to constant values for the dictionary elements, and b D L (A bSxO xO ¡ S yxO ) H) d A b D (A bSxO xO ¡ SyxO ) D cA, b 0 D Ld A

(A.24)

bTd A b) . Thus, the steady-state solution is from equation A.5, where c D tr( A b D S y, xO ( SxO xO ¡ cI ) ¡1 2 AF . A

(A.25)

Note that equations A.20 and A.25 are the steady-state values given earlier in equation 3.25. Appendix B: Convergence of the Column–Normalized Learning Algorithm

The derivation and proof of convergence of the column-normalized learning algorithm, applicable to learning members of AC , is accomplished by appropriately modifying key steps of the development given in appendix A. As in appendix A, the standard Euclidean norm and inner product apply to column vectors, while the Frobenius norm and inner product apply to m £ n matrix elements of Rm£n . B.1 Admissible Matrix Derivatives.

B.1.1 The Constraint Manifold AC . Let ei D (0 ¢ ¢ ¢ 0 1 0 ¢ ¢ ¢ 0) T 2 Rn be the canonical unit vector whose components are all zero except for the value 1 in the ith location. Then n X ID ei eTi and ai D Aei . iD 1

Note that kai k2 D aTi ai D ( Aei ) T Aei D eTi AT Aei D tr ei eTi AT A D tr MTi A D hMi , Ai,

where

Mi , Aei eTi D ai eTi D [0 0 ¢ ¢ ¢ 0 ai 0 ¢ ¢ ¢ 0] 2 Rm£n .

(B.1)

Note that only the ith column of Mi is nonzero and is equal to ai D ith column of A. We therefore have that AD

n X i D1

Mi .

(B.2)

Dictionary Learning Algorithms for Sparse Representation

389

Also, for i, j D 1, . . . , n, hMi , Mj i D tr MTi Mj D tr ei eTi AT Aej ejT D tr eTi AT Aej ejT ei D kai k2di, j ,

(B.3)

where di, j is the Kronecker delta. Note, in particular, that kai k D kMi k. Let A 2 AC , where AC is the set of column-normalized matrices, as deŽned by equation 3.29. AC is an mn¡n D n ( m¡1 )–dimensional submanifold of the Frobenius space Rm£n as each of the n columns of A is normalized as kai k2 D kM i k2 ´

1 , n

and kAk2 D tr AT A D tr

n X iD 1

MTi

n X jD 1

Mj D

n X iD 1

kai k2 D

n D 1, n

using linearity of the trace function and property, B.3. It is evident, therefore, that

AC ½ AF , for AF deŽned by equation 3.17. We can readily show that AC is simply connected, so that a continuous path exists between any matrix A D Ainit 2 AC to any other columnnormalized matrix A0 D AFinal 2 AC . Indeed, let A and A0 be such that A D [a1 , . . . , an ], A0 D [a01 , . . . , a0n ], p kai k D kaj0 k D 1 / n, 8i, j D 1, . . . , n. There is obviously a continuous path entirely in A C from A 2 A C to the intermediate matrix [a01 , a2 , . . . , an ] 2 A C . Similarly, there is a continuous path entirely in AC from [a01 , a2 , . . . , an ] to [a01 , a02 , . . . , an ], and so on to [a01 , . . . , a0n ] D A0 . To summarize, AC is a simply connected ( nm ¡ n ) –dimensional submanifold of the simply connected ( nm ¡ 1) –dimensional manifold AF and they are both submanifolds of the (nm ) –dimensional Frobenius space Rm£n . B.1.2 Derivatives on AC : The Tangent Space T AC . i D 1, . . . , n deŽne b ai D

p ai D nai , kai k

kb ai k D 1,

and p bi D Mi D nMi D b M ai eTi , kMi k

bi k D 1. kM

For convenience, for

390

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

Note that equation B.3 yields bi , M bj i D tr M bT M bj D di, j . hM i

(B.4)

For any A 2 AC we have for each i D 1, . . . , n, d d d P i kai k2 D aTi ai D eTi AT Aei D 2eTi AT Ae dt dt dt T P T P D 2 tr ei eT i A A D 2 tr Mi A,

0D

or P D tr MT AP D 0 for A 2 AC ) hMi , Ai i

i D 1, . . . , n.

(B.5)

In fact, AP 2 T A C at A 2 AC

,

bi , Ai P D tr M bT AP D 0 hM i for

i D 1, . . . , n.

(B.6)

Note from equations B.2 and B.5 that * + n X P P hA, Ai D Mi , A D 0, iD 1

showing that (see equation A.3) T AC ½ T AF , as expected from the fact that AC ½ A F . An important and readily proven fact is that for each i D 1, . . . , n, bT AP D 0 tr M i

,

AP D Pi Qi

(B.7)

for some matrix Qi and projection operator, Pi , deŽned by b i hM bi , Qi D Q ¡ M bi tr M bT Q. Pi Q , Q ¡ M i

(B.8)

From equation B.4, it can be shown that the projection operators commute, P i Pj D Pj P i ,

6 iD j,

i, j D 1, . . . , n,

(B.9)

and are idempotent, P 2i D Pi ,

i D 1, . . . , n.

(B.10)

Indeed, it is straightforward to show from equation B.4 that for all Q, b i hM bi , Qi ¡ M bj h M bj , Qi, P i Pj Q D Pj Pi Q D Q ¡ M

for all

6 iD j, (B.11)

Dictionary Learning Algorithms for Sparse Representation

391

and b i hM bi , Qi D Pi Q, P2i Q D Q ¡ M

i D 1, . . . , n.

(B.12)

It can also be shown that hPi Q1 , Q2 i D hQ1 , Pi Q2 i,

showing that Pi is self–adjoint, Pi D P¤i . DeŽne the operator, P D P 1 , . . . , Pn .

(B.13)

Note that because of the commutativity of the Pi , the order of multiplication on the right-hand side of equation B.11 is immaterial, and it is easily determined that P is idempotent, P2 D P. By induction on equation B.11, it is readily shown that b1 h M b1 , Qi ¡¢ ¢ ¢ ¡ M b n hM bn , Qi D Q ¡ PQ D Q ¡ M

n X i D1

b i hM bi , Qi. M

(B.14)

Either from the self-adjointness and idempotency of each Pi , or directly from equation B.14, it can be shown that P itself is self–adjoint, P D P¤ , hPQ1 , Q2 i D hQ1 , PQ2 i.

Thus, P is the orthogonal projection operator from Rm£n onto T AC . As a consequence, we have the key result that AP 2 T AC at A

AP D PQ,

,

(B.15)

for some matrix Q 2 Rm£n . This follows from the fact that given Q, then the right-hand side of equation B.6 is true for AP D PQ. On the other hand, P we can take Q D AP if the right-hand side of equation B.6 is true for A, in equation B.15. Note that for the T AF –projection operator L given by equation A.2, we have that P D L P D PL , consistent with the fact that T AC ½ T AF . Let qi , i D 1, . . . , n be the columns of Q D [q1 , . . . , qn ]. The operation Q0 D Pj Q, corresponds to q0i D qi ,

6 iD j,

and

qj0 D (I ¡b ajb ajT ) qj ,

while the operation Q0 D PQ, corresponds to q0i D (I ¡ b aib aTi ) qi ,

i D 1, . . . , n.

392

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

B.2 Learning on the Manifold AC . The development of equations A.6 through A.22 is independent of the precise nature of the constraint manifold A and can be applied here to the speciŽc case of A D A C . To ensure that T2 · 0 in equation A.22 and that AP 2 T AC , we can use the learning rule, Á ! n X P b b (B.16) A D ¡m PdA D ¡m dA ¡ Mi tr MidA , i D1

for m > 0 and dA given by equation A.21. With the ith column of dA denoted by dai , this corresponds to the learning rule, aP i D ¡m ( I ¡b aib aTi )d ai ,

i D 1, . . . , n.

(B.17)

With rule B.16, we have P D ¡m hdA, PdAi D ¡m hdA, P2dAi T 2 D hdA, Ai 2 D ¡m hPdA, PdAi D ¡m kPdAk · 0,

where we have explicitly shown that the idempotency and self-adjointness of P correspond to its being a nonnegative operator. Thus, we will have convergence to the largest invariant set for which AP ´ 0, which from equation B.16 is equivalent to the set for which PdA D dA ¡

n X iD 1

bi h M bi , dAi ´ 0. M

(B.18)

This, in turn, is equivalent to dA D

n X iD 1

bi h M bi , dAi D M

n X i D1

nMi hMi , dAi D

n X

ci M i ,

(B.19)

iD 1

with ci , nhMi , dAi D naTidai ,

i D 1, . . . , n.

An equivalent statement to equation B.19 is dai D ci ai ,

i D 1, . . . , n.

DeŽning the diagonal matrix C D diag[ci , . . . , cn ] and recalling the deŽnitions A.21 and B.1, we obtain from equation B.19 that A Sxx ¡ Syx D dA D AC,

which can be solved as

A D Syx ( Sxx ¡ C) ¡1 .

(B.20)

This is the general form of the solution found by the AC –learning algorithm.

Dictionary Learning Algorithms for Sparse Representation

393

Acknowledgments

This research was partially supported by NSF grant CCR-9902961. K. K.-D. also acknowledges the support provided by the Computational Neurobiology Laboratory, Salk Institute, during a 1998–1999 sabbatical visit. References Adler, J., Rao, B., & Kreutz-Delgado, K. (1996). Comparison of basis selection methods. In Proceedings of the 30th Asilomar Conference on Signals, Systems, and Computers. PaciŽc Grove, CA: IEEE. Attias, H. (1999). Independent factor analysis. Neural Computation, 11(4), 803– 851. Basilevsky, A. (1994). Statistical factor analysis and related methods: Theory and applications. New York: Wiley. Bell, A., & Sejnowski, T. (1995). An information-maximization approach to blind separation and blind deconvolution. Neural Computation, 7, 1129–1159. Coifman, R., & Wickerhauser, M. (1992).Entropy-based algorithms for best basis selection. IEEE Transactions on Information Theory, IT-38(2), 713–718. Comon, P. (1994). Independent component analysis: A new concept? Signal Processing, 36, 287–314. Deco, G., & Obradovic, D. (1996). An information-theoreticapproach to neural computing. Berlin: Springer-Verlag. Dhrymes, P. (1984). Mathematics for econometrics (2nd ed.). New York: SpringerVerlag. Dirac, P. A. M. (1958). The principles of quantum mechanics. Oxford: Clarendon Press. Donoho, D. (1994). On minimum entropy segmentation. In C. Chui, L. Montefusco, & L. Puccio (Eds.), Wavelets:Theory, algorithms,and applications(pp. 233– 269). San Diego, CA: Academic Press. Donoho, D. (1995). De-noising by soft-thresholding. IEEE Trans. on Information Theory, 41, 613–627. Engan, K. (2000). Frame based signal representation and compression. Unpublished doctoral dissertation, Stavanger Universty College, Norway. Engan, K., Rao, B., & Kreutz-Delgado, K. (1999). Frame design using FOCUSS with method of optimal directions (MOD). In Proc. NORSIG-99. Tronheim, Norway: IEEE. Engan, K., Rao, B. D., & Kreutz-Delgado, K. (2000). Regularized FOCUSS for subset selection in noise. In Proc. NORSIG 2000, Sweden, June, 2000 (pp. 247– 250). Kolmarden, Sweden: IEEE. Field, D. (1994). What is the goal of sensory coding. Neural Computation, 6, 559– 599. Girolami, M. (2001). A variational method for learning sparse and overcomplete representations. Neural Computation, 13, 2517–2532. Gorodnitsky, I., George, J., & Rao, B. (1995). Neuromagnetic source imaging with FOCUSS: A recursive weighted minimum norm algorithm. Journal of Electroencephalographyand Clinical Neurophysiology, 95(4), 231–251.

394

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

Gorodnitsky, I., & Rao, B. (1997). Sparse signal reconstruction from limited data using FOCUSS: A re-weighted minimum norm algorithm. IEEE Trans. Signal Processing, 45(3), 600–616. Hansen, P. C., & O’Leary, D. P. (1993). The use of the L-curve in the regularization of discrete ill-posed problems. SIAM J. Sct. Comput., 14, 1487–1503. Hyva¨ rinen, A., Cristescu, R., & Oja, E. (1999). A fast algorithm for estimating overcomplete ICA bases for image windows. In Proc. Int. Joint Conf. on Neural Networks (IJCNN’99) (pp. 894–899). Washington, DC. Kalouptsidis, T., & Theodoridis, S. (1993). Adaptive system identiŽcation and signal processing algorithms. Upper Saddle River, NJ: Prentice Hall. Kassam, S. (1982). Signal detection in non-gaussian noise. New York: SpringerVerlag. Khalil, H. (1996). Nonlinear systems (2nd ed). Upper Saddle River, NJ: Prentice Hall. Kreutz-Delgado, K., & Rao, B. (1997). A general approach to sparse basis selection: Majorization, concavity, and afŽne Scaling (Full Report with Proofs) (Center for Information Engineering Report No. UCSD-CIE-97-7-1). San Diego: University of California. Available on-line at: http://dsp.ucsd.edu/»kreutz. Kreutz-Delgado, K., & Rao, B. (1998a). A new algorithm and entropy-like measures for sparse coding. In Proc. 5th Joint Symp. Neural Computation. La Jolla, CA. Kreutz-Delgado, K., & Rao, B. (1998b). Gradient factorization-based algorithm for best-basis selection. In Proceedings of the 8th IEEE Digital Signal Processing Workshop. New York: IEEE. Kreutz-Delgado, K., & Rao, B. (1998c). Measures and algorithms for best basis selection. In Proceedings of the 1998 ICASSP. New York: IEEE. Kreutz-Delgado, K., & Rao, B. (1999). Sparse basis selection, ICA, and majorization: Towards a uniŽed perspective. In Proc. 1999 ICASSP. Phoenix, AZ. Kreutz-Delgado, K., Rao, B., & Engan, K. (1999). Novel algorithms for learning overcomplete Dictionaries. In Proc. 1999 Asilomar Conference. PaciŽc Grove, CA: IEEE. Kreutz-Delgado, K., Rao, B., Engan, K., Lee, T.-W., & Sejnowski, T. (1999a). Convex/Schur–convex (CSC) log-priors and sparse coding. In Proc. 6th Joint Symposium on Neural Computation. Pasadena, CA. Kreutz-Delgado, K., Rao, B., Engan, K., Lee, T.-W., & Sejnowski, T. (1999b).Learning overcomplete dictionaries: Convex/Schur-convex (CSC) log-priors, factorial codes, and independent/dependent component analysis (I/DCA). In Proc. 6th Joint Symposium on Neural Computation. Pasadena, CA. Lee, T.-W., Girolami, M., & Sejnowski, T. (1999). Independent component analysis using an extended infomax algorithm for mixed sub-gaussian and supergaussian sources. Neural Computation, 11(2), 417–441. Lewicki, M. S., & Olshausen, B. A. (1999). A probabilistic framework for the adaptation and comparison of image codes. J. Opt. Soc. Am. A, 16(7), 1587– 1601. Lewicki, M. S., & Sejnowski, T. J. (2000). Learning overcomplete representations. Neural Computation, 12(2), 337–365. Mallat, S., & Zhang, Z. (1993). Matching pursuits with time-frequency dictionaries. IEEE Trans. ASSP, 41(12), 3397–3415.

Dictionary Learning Algorithms for Sparse Representation

395

Marshall, A., & Olkin, I. (1979). Inequalities: Theory of majorization and its applications. San Diego, CA: Academic Press. Murray, J. F., & Kreutz-Delgado, K. (2001).An improved FOCUSS-based learning algorithm for solving sparse linear inverse problems. In Conference Record of the 35th Asilomar Conference on Signals, Systems and Computers. PaciŽc Grove, CA: IEEE. Nadal, J.-P., & Parga, N. (1994). Nonlinear neurons in the low noise limit: A factorial code maximizes information transfer. Network, 5, 565–581. Olshausen, B., & Field, D. (1996). Emergence of simple-cell receptive Želd properties by learning a sparse code for natural images. Nature, 381, 607–609. Olshausen, B., & Field, D. (1997). Sparse coding with an overcomplete basis set: A strategy employed by V1? Vision Research, 37, 3311–3325. Pham, D. (1996). Blind separation of instantaneous mixture of sources via an independent component analysis. IEEE Trans. Signal Processing, 44(11), 2768– 2779. Popovic, Z., & Sj¨ostrand, J. (2001). Resolution, separation of retinal ganglion cells, and cortical magniŽcation in humans. Vision Research, 41, 1313–1319. Rao, B. (1997). Analysis and extensions of the FOCUSS algorithm. In Proceedings of the 1996 Asilomar Conference on Circuits, Systems, and Computers (Vol. 2, pp. 1218–1223). New York: IEEE. Rao, B. (1998). Signal processing with the sparseness constraint. In Proc. 1998 ICASSP. New York: IEEE. Rao, B. D., Engan, K., Cotter, S. F., & Kreutz-Delgado, K. (2002). Subset selection in noise based on diversity measure minimization. Manuscript submitted for publication. Rao, B., & Gorodnitsky, I. (1997). AfŽne scaling transformation based methods for computing low complexity sparse solutions. In Proc. 1996 ICASSP (Vol. 2, pp. 1783–1786). New York: IEEE. Rao, B., & Kreutz-Delgado, K. (1997). Deriving algorithms for computing sparse solutions to linear inverse problems. In J. Neyman (Ed.), Proceedingsof the 1997 Asilomar Conference on Circuits, Systems, and Computers. New York: IEEE. Rao, B., & Kreutz-Delgado, K. (1998a). Basis selection in the presence of noise. In Proceedings of the 1998 Asilomar Conference. New York: IEEE. Rao, B., & Kreutz-Delgado, K. (1998b). Sparse solutions to linear inverse problems with multiple measurement vectors. In Proceedings of the 8th IEEE Digital Signal Processing Workshop. New York: IEEE. Rao, B., & Kreutz-Delgado, K. (1999). An afŽne scaling methodology for best basis selection. IEEE Trans. Signal Processing, 1, 187–202. Roberts, S. (1998). Independent component analysis: Source assessment and separation, a Bayesian approach. IEE (Brit.) Proc. Vis. Image Signal Processing, 145(3), 149–154. Rockafellar, R. (1970). Convex analysis. Princeton, NJ: Princeton University Press. Ruderman, D. (1994). The statistics of natural images. Network: Computation in Neural Systems, 5, 517–548. Wang, K., Lee, C., & Juang, B. (1997). Selective feature extraction via signal decomposition. IEEE Signal Processing Letters, 4(1), 8–11. Watanabe, S. (1981). Pattern recognition as a quest for minimum entropy. Pattern Recognition, 13(5), 381–387.

396

K. Kreutz-Delgado, J. Murray, B. Rao, K. Engan, T. Lee, and T. Sejnowski

Wickerhauser, M. (1994).Adaptedwavelet analysis from theoryto software. Wellesley, MA: A. K. Peters. Zhu, S., Wu, Y., & Mumford, D. (1997). Minimax entropy principle and its application to texture modeling. Neural Computation, 9, 1627–1660. Zibulevsky, M., & Pearlmutter, B. A. (2001). Blind source separation by sparse decomposition in a signal dictionary. Neural Computation, 13(4), 863–882. Received August 6, 2001; accepted June 4, 2002.

LETTER

Communicated by Christian Wehrhahn

Spatiochromatic Receptive Field Properties Derived from Information-Theoretic Analyses of Cone Mosaic Responses to Natural Scenes Eizaburo Doi [email protected] Institute for Neural Computation, University of California, San Diego, La Jolla, CA 92093, U.S.A., and Graduate School of Informatics, Kyoto University, Kyoto 606-8501, Japan Toshio Inui [email protected] Graduate School of Informatics, Kyoto University, Kyoto 606-8501, Japan Te-Won Lee [email protected] Institute for Neural Computation, University of California, San Diego, La Jolla, CA 92093, U.S.A., and Howard Hughes Medical Institute, Computational Neurobiology Laboratory, Salk Institute, La Jolla, CA 92037, U.S.A. Thomas Wachtler [email protected] Neurobiology and Biophysics, Institute of Biology III, Albert-Ludwigs-Universit¨at, 79104 Freiburg, Germany Terrence J. Sejnowski [email protected] Institute for Neural Computation, University of California, San Diego, La Jolla, CA 92093, U.S.A., Howard Hughes Medical Institute, Computational Neurobiology Laboratory, Salk Institute, La Jolla, CA 92037, U.S.A., and Department of Biology, University of California, San Diego, La Jolla, CA 92093, U.S.A. Neurons in the early stages of processing in the primate visual system efŽciently encode natural scenes. In previous studies of the chromatic properties of natural images, the inputs were sampled on a regular array, with complete color information at every location. However, in the retina cone photoreceptors with different spectral sensitivities are arranged in a mosaic. We used an unsupervised neural network model to analyze the statistical structure of retinal cone mosaic responses to calibrated color natural images. The second-order statistical dependencies derived from the covariance matrix of the sensory signals were removed in the Žrst stage Neural Computation 15, 397–417 (2003)

° c 2002 Massachusetts Institute of Technology

398

E. Doi, T. Inui, T. Lee, T. Wachtler, and T. Sejnowski

of processing. These decorrelating Žlters were similar to type I receptive Želds in parvo- or konio-cellular LGN in both spatial and chromatic characteristics. In the subsequent stage, the decorrelated signals were linearly transformed to make the output as statistically independent as possible, using independent component analysis. The independent component Žlters showed luminance selectivity with simple-cell-like receptive Želds, or had strong color selectivity with large, often double-opponent, receptive Želds, both of which were found in the primary visual cortex (V1). These results show that the “form” and “color” channels of the early visual system can be derived from the statistics of sensory signals. 1 Introduction

New algorithms have recently been developed to test the hypothesis that sensory signals are transformed in the early stages of sensory processing to reduce the redundancy of the inputs (Barlow, 1961; Field, 1994). The spatial properties of ganglion cells in the retina and thalamocortical neurons in the lateral geniculate nucleus (LGN) are consistent with algorithms that use second-order statistics to decorrelate natural scenes (Atick, 1992; Atick & Redlich, 1993) and the localized and oriented simple cells in the primary visual cortex (V1) reduce higher-order statistics (Olshausen & Field, 1996; Bell & Sejnowski, 1997; van Hateren & van der Schaaf, 1998). These studies used gray-scale natural images. Natural color images have also been analyzed using RGB inputs (Tailor, Finkel, & Buchsbaum, 2000; Hoyer & Hyvarinen, 2000) and LMS inputs with the spectral sensitivities of the human L-, M-, and S-cone photoreceptors (Wachtler, Lee, & Sejnowski, 2001). In these studies, the images were sampled on a regular grid, with the color at each location represented as a vector of three elements. In the retina cone photoreceptors are arranged in a mosaic, with only one cone at each location. Thus, chromatic information in the retina is extracted by comparing the responses of different cone types at different locations, which requires spatial interaction. This is not the case for RGB or LMS color images, where the information for chromaticity is fully available at each pixel. Here, we consider a model that incorporates the cone mosaic found in the trichromatic foveal region of primates. We adopt a hierarchical model that consists of a decorrelating stage corresponding to LGN and a subsequent independent component analysis (ICA) stage corresponding to V1 (Bell & Sejnowski, 1997). Applying biological and computational constraints at each level of processing allows a better comparison to the properties of neurons in the early visual system. 2 Methods 2.1 Cone Mosaic Responses. A small cone mosaic patch was used to model the trichromatic foveal region of the primate retina (see Figure 1). To

Information-Theoretic Analyses of Cone Mosaic Responses

399

obtain estimates of the cone mosaic responses to natural scenes, we took natural images using a 3CCD camera and converted them into LMS images (see the appendix for details). By scanning the data set randomly with the cone mosaic, 507,904 samples were generated. 2.2 Model.

2.2.1 Computational Goal. cessing are linear,

We assume that the early stages of visual pro-

u D Wx ,

(2.1)

where x D ( x1 , . . . , xn ) T are the input signals to the visual system (namely, the cone mosaic responses), W D [w 1 , . . . , w n ]T is a square matrix consisting of receptive Želds w i , and u D ( u1 , . . . , un ) T is the output, corresponding to the neural activities (Olshausen & Field, 1996; Bell & Sejnowski, 1997). The computational goal of the system is assumed to reduce the redundancies or statistical dependencies among the output elements. This coding scheme is known as minimum entropy coding or factorial coding (Atick, 1992; Field, 1994) and can be implemented by ICA (Bell & Sejnowski, 1995; Lee, 1998; Girolami, 1999; Hyvarinen, Karhunen, & Oja, 2001). A linear generative model for the input data x is given by x D As D s1 a 1 C ¢ ¢ ¢ C sn a n ,

(2.2)

where A D [a1 , . . . , a n ] is a set of basis functions and s1 , . . . , sn are their coefŽcients, called sources or causes. Given sources s that are statistically independent of each other, equation 2.2 assumes that the input x is generated by a linear mixture of statistically independent causes s . The goal of ICA is to Žnd a matrix W that unmixes x so that u D s, namely, W D A ¡1 (up to permutation and scaling). The assumption of statistical independence among sources s and the general evidence of high redundancy in the sensory input x mean that the basis functions a i introduce statistical dependency in the input. Equation 2.2 means that when a source si is activated (i.e., taking nonzero value), more than one sensor will concurrently have nonzero values according to the pattern of a i , which leads to statistical dependency in the sensory signals x. 2.2.2 Hierarchical Implementation. We followed the approach of Bell and Sejnowski (1997) in decomposing the linear transformation W into decorrelation and ICA, W I W Z (see Figure 2): the Žrst operator W Z decorrelates the input x such that the covariance matrix of its output u Z satisŽes hu Z u Z T i D h( W Z x ) ( W Z x ) T i D I,

(2.3)

400

E. Doi, T. Inui, T. Lee, T. Wachtler, and T. Sejnowski

Information-Theoretic Analyses of Cone Mosaic Responses

401

Figure 2: Three-stage model of the early visual system. This is three-layer feedforward neural network. The Žrst stage corresponds to the photoreceptor layer in the retina, as shown in Figure 1. The subsequent stages are assumed to reduce redundancy in the sensory input in a sequential manner.

where I is the identity matrix. This decorrelation normalizing the output variance is also known as the whitening transformation or zero-phase Žlters (Bell & Sejnowski, 1997). The second operator W I makes the decorrelated signals W Z x to be statistically as independent as possible (ICA for whitened data). This decomposition is justiŽed as follows. First, the overlap of cone

Figure 1: Facing page. Properties of the input. (a) Cone mosaic model. This is based on the anatomical data from old world monkeys. It consists of 217 photoreceptors of L- (red), M- (green), and S-cone (blue). L- and M-cones are arranged randomly (Mollon & Bowmaker, 1992), whereas S-cones are arranged regularly (de Monasterio, McCrane, Newlander, & Schein, 1985; Szel, Diamantstein, & Rohlich, 1988). The cone ratio is L : M D 1 : 1 (Mollon & Bowmaker, 1992; Baylor et al., 1987), and S-cone occupies about 3% (de Monasterio et al., 1985; Mollon & Bowmaker, 1992). In this model L : M : S D 112 : 98 : 7. (b) Spectral sensitivities of cone photoreceptors. Here we used those of human cone photoreceptors (Stockman & Sharpe, 2000). (c–e) An example of cone mosaic output compared with RGB and gray-scale natural images. (c) RGB image. (d) Gray-scale image. M-cone responses are shown for reference. Larger responses correspond to brighter pixels. (e) Cone mosaic responses. The red hexagon shows the window of the cone mosaic shown in a; thus, the image inside is the response pattern of the cone mosaic. This image was prepared by concatenating several response patches of the same cone mosaic. The pixels are arranged in the hexagonal grid instead of a square grid to mimic the closely packed structure of the retinal cone mosaic as shown in a. They are generated by averaging four pixels of square-grid images with one pixel offset between each row. (f) Examples of natural images. Sixty-two natural images were used.

402

E. Doi, T. Inui, T. Lee, T. Wachtler, and T. Sejnowski

spectral sensitivities (especially between L- and M-cones) produces a strong correlation between elements of the input, as is the case of gray-scale natural images. This correlation could be removed as a Žrst step toward a statistically independent representation. This operation restricts the solution of the next stage W I to be in the orthogonal matrix manifold. Second, spatial properties of receptive Želds of retinal ganglion cells and LGN cells have been suggested to be explained by decorrelation, using gray-scale data sets (Atick, 1992; Atick & Redlich, 1993; Bell & Sejnowski, 1997). 2.2.3 Learning Algorithms. W Z is a whitening matrix that satisŽes equation 2.3 with an additional degree of freedom in the choice of whitening matrix: if W Z is a whitening matrix, UW Z is also a whitening matrix for arbitrary orthogonal matrices U . Here we assume that the whitening matrix is symmetric, W Z T D W Z , which makes it unique. For gray-scale natural images, this whitening yields a set of zero-phase Žlters, which atten the spatial-frequency (1/f) spectrum of shift-invariant data and show concentric center-surround receptive Želd organization (Atick, 1992; Atick & Redlich, 1993). Such a matrix can be derived by matrix square root of the inverse of the input covariance matrix (Golub & Loan, 1996; Bell & Sejnowski, 1997) or using a more biologically plausible learning algorithm (Atick & Redlich, 1993). Since both are equivalent, we used the former method for convenience. The ICA algorithm we used can be derived using information maximization or maximum likelihood estimation (Bell & Sejnowski, 1995; Cardoso, 1997) with the natural gradient (Amari, Cichocki, & Yang, 1996),

D W I / [I ¡ Q ( u ) u ] W I ,

(2.4)

( )

@ log p u where Q (u ) D ¡ @u , p ( u ) D P i p (ui ) is the probability density of u , and D W I is the change of weights computed iteratively until it converges to zero. Note that it requires a density model of p ( ui ) . We use the extended infomax algorithm for ICA that assumes different p (ui ) ’s for supergaussian and subgaussian density (Lee, Girolami, & Sejnowski, 1999). Thus, the results do not require the coefŽcients to have sparse distributions, unlike some previous methods (Olshausen & Field, 1996; Bell & Sejnowski, 1997).

3 Results 3.1 Decorrelating Stage. All units in the decorrelating stage showed concentric center-surround receptive Želd organization with the center region driven by a single cone (see Figure 3). Receptive Želds of type I neurons of parvo- or konio-cellular LGN (pLGN/kLGN) of the macaque have these properties (Lennie & D’Zmura, 1988; in their review, as in this study, the properties of retinal ganglion cells were not distinguished from those of LGN cells). They can be further classiŽed on the basis of the center-driving

Information-Theoretic Analyses of Cone Mosaic Responses

403

Figure 3: Receptive Želds of the decorrelating stage. Each panel shows a weighting pattern (receptive Želd organization) from cone mosaic to a certain decorrelating unit. Here we show two examples for each L-, M-, and S-center type. Bright and dark circles show positive and negative weights; red, green, and blue colors correspond to weights of L-, M-, and S-cone; and the circle size indicates the amplitude of the weight.

cone type, leading to three types of units that we refer to as L-center, Mcenter, and S-center. All units were classiŽed as color selective unit using the criterion in Hanazawa, Komatsu, and Murakami (2000): the Color Selectivity Index (CSI) greater than 0.5 (see Figure 4b). 1 This matches the lack of luminance or achromatic units in pLGN/kLGN (Lennie & D’Zmura, 1988; Hanazawa et al., 2000). Figure 4a illustrates color selectivity proŽles in comparison with those of pLGN/kLGN neurons, showing agreement between the model and the physiological data. Figure 4c is the histogram of their color tuning directions, segregated in three clusters as in the physiological data. The color tuning directions of the L-center and M-center types were almost completely opposite (¡6.3 § 13.5 and 180.8 § 1.9, respectively). This is also conŽrmed in Figure 4a, where contours of the responses were aligned in the same orientation but in the reverse order. This indicates that both the L- and M-center types were responsible for the same r-g channel, 1 (Color Selectivity Index) = [(max response) ¡ (min response)]/[max abs(response)]. In the physiological study of Hanazawa et al. (2000), the response is deŽned by [mean discharge ratio] ¡ [baseline activity]. Therefore, negative values are possible.

404

E. Doi, T. Inui, T. Lee, T. Wachtler, and T. Sejnowski

Information-Theoretic Analyses of Cone Mosaic Responses

405

while the S-center type was responsible for the y-b channel, in the color opponent stage. Regarding L-center and M-center type, there was no signiŽcant conetype speciŽcity in the receptive Želd organization. In fact, they showed close similarity to the whitening Žlters derived from a single cone-type cone mosaic (here we used M-cone, and as a result the data are gray-scale): w Z T w Zgray / kw Z kkw Zgray k D 0.995 § 0.005. This suggests that their receptive Želd organization is determined mainly by spatial factor, not by cone-type dependent chromatic factor. This also means that the signiŽcant color selectivity of L- and M-center type is mainly due to the weighting ratio between the center and the surround, itself mainly due to the spatial factor, not the cone-type speciŽc, antagonistic wiring. These results would be reasonable since the majority of the input signals come from L- and M-cones, of which spectral sensitivities are highly overlapping. Therefore, the statistics of cone mosaic data are close to those of gray-scale (only M-cone) data. S-center type shows S-cone speciŽcity with larger receptive Želds than Land M-center types (see Figure 3). This is consistent with the large dendritic arborization of the small bistratiŽed retinal ganglion cells, which are distinct from midget and parasol ganglion cells and belong to the konio-cellular (Scone) pathway (Dacey, 2000). Our results show that the S-cone weights in the surround are inhibitory. This is reasonable since the difference of correlated signals is oriented along the uncorrelated axis. The larger size of S-center

Figure 4: Facing page. Chromatic properties of the decorrelating stage. (a) Color selectivity. Model data (upper row) were derived from the same units illustrated in the lower row of Figure 3. Physiological data (lower row) are of pLGN/kLGN neurons. This diagram shows the responses of a certain unit to a set of color stimuli distributed in both MacLeod-Boynton (MB) and CIE 1931 xy (CIE-xy) chromaticity coordinates. For illustration purposes, a CIE-xy chromaticity diagram was used. Open and Žlled circles show positive and negative responses, respectively; their diameters indicate response amplitudes; C corresponds to the chromaticity of the color stimuli. It is important to note that all color stimuli were adjusted to have the same luminance (isoluminant stimuli), and thus any difference among the responses of a certain unit can be attributed to its color selectivity. Contours of the responses are also plotted: from thick to thin, 80%, 60%, 40%, and 20% of [maximum response] ¡ [minimum response]. Axes of color orientations deŽned in MB chromaticity coordinates are shown as well (0, 90, 180, 270 degrees). For the analysis of second-layer units, the stimuli were a uniform pattern to mimic diffuse light in the physiological experiment. (b) Distribution of color selectivity index. (c) Distribution of color tuning directions. This direction is deŽned by the maximum slope of the responses. Physiological data are of pLGN/kLGN. Color of the histogram is identical to that of b. All physiological data are from Hanazawa, Komatsu, and Murakami (2000).

406

E. Doi, T. Inui, T. Lee, T. Wachtler, and T. Sejnowski

type receptive Želds should be due to the sparse arrangement of S-cone in the cone mosaic, since in the case of L- and M-center type, the cones next to the center respond in a highly correlated way to the center cone. While the S-cone antagonism between the center and the surround is reasonable from the redundancy reduction point of view, physiological studies showed only the spatially coextensive yellow-blue cells in retinal ganglion cells (Dacey, 2000). 3.2 ICA Stage. In the ICA stage, two main types of receptive Želds emerged (see Figures 5 and 6): the majority (94.9%) of luminance type (achromatic, color nonselective) and the others (4.6%) of color-selective type. Note that this analysis was based on W , not W I . The luminance unit was similar to those derived using gray-scale samples, showing simple-cell-like receptive Želd with high spatial-frequency selectivity (see Figure 5a; Bell & Sejnowski, 1997; van Hateren & van der Schaaf, 1998). This suggests that the luminance units would represent spatial information (without color). Responses to isoluminant color stimuli are almost constant (see Figure 6a), and the color selectivity index was less than 0.5 for almost all luminance units (0.28 § 0.14; see Figure 6b), showing low color selectivity. The low value of this index means that the spectral selectivity was almost identical to the luminosity function Vl ,2 consistent with physiological data from simple cells in V1 (Lennie & D’Zmura, 1988; Lennie, Krauskopf, & Sclar, 1990). The color-selective units were characterized by cone-type speciŽc antagonistic regions in their receptive Želd, and classiŽed into two subtypes: Y/B type of yellow-blue selectivity and R/G type of red-green selectivity. The

Figure 5: Facing page. Receptive Želds of the ICA stage. (a) Receptive Želd organization. Here, each panel shows a weighting pattern from cone mosaic to a certain ICA unit. Two examples for each luminance, Y/B, and R/G type are shown. The optimal bar stimulus for the units is shown just below each receptive Želd. The optimal bar is deŽned by the bar that maximally matches the weighting pattern. A set of bars was prepared with 13 width, 1 » 3 length, and 22 orientation for arbitrary location. To Žnd the color-opponent region for color-selective type, weighting patterns were modiŽed as follows: C L C M ¡ S for Y/B type; C L ¡ M for R/G type (S-cone is neglected). (b) Basis function. These are the dual vectors of Žlters shown in a. 2

Due to the use of isoluminant color stimuli, the responses of luminosity function to the color stimuli are identical. The reverse is also true: any spectral sensitivity is written as the sum of luminosity function and R its residual, S (l) R D aVl (l) C D (l), R and the response to an isoluminant stimulus i(l) is S(l)i(l)dl D a Vl (l)i(l)dl C D (l)i (l)dl, where R R R a ´ S(l)i(l)dl / Vl (l)i(l)dl and a Vl (l)dl is constant by deŽnition. If the left side is constant over any isoluminant color stimuli, D (l) D 0; thus, S(l) D aVl (l).

Information-Theoretic Analyses of Cone Mosaic Responses

407

408

E. Doi, T. Inui, T. Lee, T. Wachtler, and T. Sejnowski

Figure 6: Chromatic properties of the ICA stage. (a) Color selectivity, derived from the units shown in the middle row of Figure 5a. For physiological data, see Figure 3A in Hanazawa et al. (2000) for R/G type; data corresponding to luminance or Y/B unit are not illustrated there, although they were reported. (b) Distribution of color selectivity index. (c) Distribution of color tuning direction. Here, those units of which CSI is greater than 0.5 are included in the analysis in accordance with the physiological study. Color of the histogram is identical to that of b. All physiological data are taken from Hanazawa et al. (2000).

Information-Theoretic Analyses of Cone Mosaic Responses

409

color selectivity index was much larger than 0.5 (larger than 1) for all colorselective units (1.43 § 0.16 for Y/B, 1.70 § 0.01 for R/G; see also Figure 6b), indicating that they are highly color selective. Note that CSI > 1 indicates the existence of color opponency, which yields positive response to one color and negative response to another. These color-selective subtypes may correspond to the two color channels suggested in psychophysical studies (Lennie & D’Zmura, 1988). However, it does not match the physiological studies that showed distributed tuning over the color axis (see the right panel in Figure 6c; see also Figure 2 in De Valois, Cottaris, Elfar, Mahon, & Wilson, 2000). Most color-selective units showed orientation selectivity. Although the orientation selectivity of color-selective cells in V1 is controversial (Michael, 1978; Lennie et al., 1990; Conway, 2001; Johnson, Hawken, & Shapley, 2001), our results suggest that they may be expected in redundancy reduction of cone mosaic responses. In addition to luminance units and color-selective units, we obtained a basis function showing a uniform pattern over the cone mosaic, referred to as the DC unit. Its receptive Želd organization was affected by the boundary effect. In a previous study, this component was excluded from the analysis by subtracting the average for each image patch (Hoyer & Hyvarinen, 2000). Color-selective units had a larger receptive Želd compared to luminance units: the size of the optimal bar is 4.42 § 3.41 (cone size) for luminance units and 36.30 § 6.73 and 74.62 § 25.52 for Y/B and R/G units. This is consistent with the low spatial-frequency selectivity of the color mechanism (Mullen, 1985), as expected, since the color information is derived from the comparison between at least two or three types of cones, but there is only a single cone at each retinal location. Thus, chromatic information requires averaging over several cones, leading to lower spatial-frequency selectivity. Note also that the percentage of the color-selective units was signiŽcantly lower than that of luminance units, indicating that the dimensionality assigned for chromatic information is much less than that for luminance information. Our analysis thus far of properties of receptive Želds w i suggests that ICA yields two types of units: luminance units responsible for encoding spatial information without color and color-selective units responsible for color information without Žne spatial information. We can rewrite the datagenerative model of the cone mosaic responses (see equation 2.2) as xD

X i2Form

ui a i C

X

uj aj ,

(3.1)

j2Color

where we assumed that Form consists of luminance and DC units and Color consists of color-selective units. Recall that the response ui represents the presence of the basis function a i in the input x. Some examples of the basis function are shown in Figure 5b. Regarding luminance units, the basis functions show cone-type irrespective edge pattern, exactly as the case of grayscale images. Note that even the locations of S-cone are Žlled in as if they

410

E. Doi, T. Inui, T. Lee, T. Wachtler, and T. Sejnowski

were part of the edge surface, while S-cone scarcely contributes their receptive Želd organization. Regarding color-selective units, the basis functions show cone-type speciŽc patterns, that is, color information. These suggest that the statistically independent patterns in the cone mosaic responses are a large number of cone-type unspeciŽc edge patterns and a small number of cone-type speciŽc edge patterns, along with a spatially uniform pattern for each type. In Figure 7, we visualize these different types of information by separate images (Color is divided into R/G and Y/B for convenience). It is clear that the form information is extracted by luminance and DC units (see Figure 7c), and color information is extracted by Y/B (see Figure 7d) and R/G units (see Figure 7e) from the cone mosaic responses (see Figure 7b). In this study, all resulting independent components had supergaussian distributions. When histogram equalization was used as a model for cone nonlinearity instead of the physiological model (see the appendix), one subgaussian component emerged whose basis function showed a DC component. This is consistent with other studies using gray-scale images (van Hateren & van der Schaaf, 1998) or LMS images (Wachtler et al., 2001), both of which applied the logarithm function as the cone nonlinearity. 3.3 PCA Comparison. The cone mosaic responses were also analyzed by principal component analysis (PCA) to compare the results with those derived from ICA. As shown in Figure 8, PCA yielded nonlocal Žlters as in the case of gray-scale images (Olshausen & Field, 1996; Bell & Sejnowski, 1997). There were only two components that showed cone-type speciŽcity; both showed S-cone speciŽcity. This means that the spatial variation (variance)

Figure 7: Facing page. Illustration of information content in Luminance, Y/B, or R/G channel. (a) RGB image. (b) Cone mosaic responses. This is the same as Figure 1e but for another natural scene. (c) Reconstruction from luminance and DC units. For each small patch of cone mosaic frame, the image xO was P reconstructed by xO D i ui a i , where i indicates luminance and DC units. These patches were tiled over the whole image. (d) Reconstruction from Y/B units. As in c above, but i indicates Y/B units. We also postprocessed the image to represent it using pseudocolor: L- and M-cones are regarded as contributing yellow direction (i.e., R- and G-pixels with like sign; B-pixel with opposite sign); S-cone is regarded as blue direction (B-pixel with like sign; R- and G-pixels with opposite sign). To avoid dark pixels, with color hard to see, the base color is set to be gray. (e) Reconstruction from R/G units. As in d, except that i indicates R/G units. In postprocessing, L- and S-cones are regarded as contributing red direction (i.e., R-pixel with like sign, G-pixel with opposite sign); M-cone is regarded as green direction (G-pixel with like sign; R-pixel with opposite sign). Any B-pixel is set as 0. As in d, the base color is set gray. (f) Superimposing luminance, R/G, and Y/B information above to see form and color information simultaneously.

Information-Theoretic Analyses of Cone Mosaic Responses

411

common with all cone-type dominates the chromatic variation speciŽc for each cone type (especially for L- and M-cone type). When LMS natural images are analyzed by PCA, Y/B and R/G components were as common as black-white components (Ruderman, Cronin, & Chiao, 1998). This demonstrates that it is not straightforward for the visual system to extract color information from the cone mosaic responses.

412

E. Doi, T. Inui, T. Lee, T. Wachtler, and T. Sejnowski

Figure 8: Principal components of cone mosaic responses. Here we show the 1st, 5th, 11th, 18th, 25th, and 120th principal component, as indicated for each panel. Here, cone type is not indicated by color, but otherwise these are the same illustrations as Figures 3 and 5a. Cone-type speciŽcity can be seen in only two units that show S-cone speciŽcity; the eighteenth component is one of them. The Žrst component is exactly the same as the DC type basis function in the ICA stage.

4 Discussion 4.1 Multiplexing and Separating Stages. It has been suggested that in the pLGN/kLGN, luminance information is conveyed with red-green information in the r-g opponent channel and that these “multiplexed” signals are “separated” into red-green and luminance information in the following stages, possibly in V1 (Ingling & Martinez-Uriegas, 1983; Lennie & D’Zmura, 1988; Lennie et al., 1990). Our results reproduce this two-stage model. We emphasize that this is an emergent property derived from redundancy reduction and not put in by hand as in the previous model studies (Billock, 1991; De Valois & De Valois, 1993; Kingdom & Mullen, 1995). These results suggest mechanisms by which the visual system could reduce redundancy in a sequence of stages. 4.2 Learning Visual Information Processing. It is possible that the color selectivity properties of neurons in the visual system are the end result of

Information-Theoretic Analyses of Cone Mosaic Responses

413

evolution, and during development they are implemented without visual experience. All of the properties of the cells in our model were obtained with unsupervised algorithms. This suggests that some of the properties of cells in both pLGN/kLGN and V1 could be acquired by unsupervised learning during development from visual input. Other properties of cortical neurons, such as orientation and direction selectivity, depend on the visual experience during development (Blakemore & Cooper, 1970; Blakemore & van Sluyters, 1975). Studies of natural scene statistics also support this idea (Olshausen & Field, 1996; Bell & Sejnowski, 1997; van Hateren & van der Schaaf, 1998). There are several psychophysical studies investigating the plasticity of color vision during development (Teller, 1998; Knoblauch, VitalDurand, & Barbur, 2001); however, the physical basis of plasticity in color vision has not yet been examined. Our results suggest that color selectivity, like orientation selectivity, may depend on the statistics of sensory signals. They also suggest that in general, the visual coding of attributes such as form and color (Livingstone & Hubel, 1988) can be understood as a result of redundancy reduction. 4.3 Relation to Previous Studies. Other studies applying ICA to RGB images (Tailor et al., 2000; Hoyer & Hyvarinen, 2000) used spectral sensitivities unlike those found in the human retina, which precludes direct comparisons to physiological or psychophysical data. We used spectral sensitivities of human cone photoreceptors, enabling us to obtain results under more realistic conditions. A similar approach was used earlier (Wachtler et al., 2001; Lee, Wachtler, & Sejnowski, 2002), but without sampling of a trichromatic cone mosaic. We examined the additional constraint that spatial separability of chromatic signals imposes on the human visual system and found that in contrast with the results of previous studies, (1) the cone-type speciŽcities for color vision can be learned under the realistic and difŽcult condition due to the cone mosaic sampling; (2) the spectral sensitivity of simple-cell-like units is along the luminance axis, not the black-white axis; (3) the percentage of luminance (simple-cell like) units is large ( > 90%), not low (about 1/3); and (4) the luminance units show higher spatial-frequency selectivity than color-selective units. 5 Conclusion

The goal of this study was to analyze the cone mosaic responses to natural scenes and to Žnd an efŽcient coding of spatiochromatic signals in the visual system. Our results suggest that a hierarchical redundancy reduction model explains the spatiochromatic properties of neurons in the early stages of processing in the visual system. Although our model used more realistic inputs than previous approaches (Tailor, et al., 2000; Hoyer & Hyvarinen, 2000; Wachtler et al., 2001), it still assumes a linear transformation. Thus, the current model cannot capture

414

E. Doi, T. Inui, T. Lee, T. Wachtler, and T. Sejnowski

complex properties such as nonlinear combination of cone signals (De Valois et al., 2000) or contextual interactions (Wachtler, Sejnowski, & Albright, 1999). Our model nevertheless approximates the early processing stages of the visual system. The similarities between our results and the known properties of the visual system further strengthen the hypothesis that redundancy reduction is an organizing principle in the visual system. Appendix: Preparation of Cone Responses to Natural Scenes

Images of 62 natural scenes was obtained using 3CCD digital still camera system (HC-2500, FujiŽlm, Japan). Each RGB image (1000 £ 1280 pixels, 10 bits for each R, G, B) were corrected for gamma calibration, effect of iris size, and exposure time. These linear RGB data, of which spectral sensitivities were also measured, were transformed into LMS cone responses using a 3£3 matrix. This matrix was derived by minimizing square errors for 170 surface reectance functions (Vrhel, Gershon, & Iwan, 1994) rendered by four kinds of CIE daylight functions (D50, D55, D65, and D75). The resulting errors were 0.23%, 0.14%, and 0.04% for L-, M-, and S-cone, respectively. Its approximation accuracy was also evaluated by a different set of reectance functions (ColorCheker, Macbeth, U.S.A.), and the estimation errors were reasonably small as 0.39%, 0.23%, and 0.03% for L-, M-, and S-cone, respectively. These linear cone response data were further transformed with an empirical cone nonlinear function (Baylor, Nunn, & Schnapf, 1987) given by rnl D 1 ¡ exp(¡k ¢ rl ) ,

(A.1)

where rl is linear cone responses, rnl is nonlinearly transformed cone responses, the range of which is [0,1], and the parameter k is determined so that the median of rl is 0.5 for each scene and for each cone type. We did not apply any preprocessing of the data such as low-pass Žltering (Olshausen & Field, 1996) or dimensionality reduction using PCA (van Hateren & van der Schaaf, 1998). Acknowledgments

E.D. was supported by JSPS Research Fellowships for Young Scientists (no. 200103140), T.I. was supported by Grant-in-Aid for ScientiŽc Research (A)(2) from MEXT (no. 11309007), and T.J.S. was supported by the Howard Hughes Medical Institute. We thank Akitoshi Hanazawa and Hidehiko Komatsu for allowing us to use their Žgures and for their helpful comments, Toshikazu Wada for help with the camera calibration, Kyoto Botanical Gardens for assistance with natural image collection, and Greg Horwitz for his helpful comments on the manuscript. Materials such as color natural images and corresponding cone responses, and the results of all units of decorrelation, ICA, and PCA are available on-line at http:/ / inc2.ucsd.edu/»edoi/pub/.

Information-Theoretic Analyses of Cone Mosaic Responses

415

References Amari, S., Cichocki, A., & Yang, H. (1996). A new learning algorithm for blind signal separation. In D. Touretzky, M. Mozer, & M. Hasselmo (Eds.), Advances in neural information processing systems, 8, (pp. 757–763). Cambridge, MA: MIT Press. Atick, J. J. (1992). Could information theory provide an ecological theory of sensory processing? Network, 3, 213–251. Atick, J. J., & Redlich, A. N. (1993). Convergent algorithm for sensory receptive Želd development. Neural Computation, 5, 45–60. Barlow, H. B. (1961). Possible principles underlying the transformation of sensory messages. In W. A. Rosenblith (Ed.), Sensory communication (pp. 217– 234). Cambridge, MA: MIT Press. Baylor, D. A., Nunn, B. J., & Schnapf, J. L. (1987). Spectral sensitivity of cones of the monkey macaca fascicularis. Journal of Physiology, 390, 145–160. Bell, A. J., & Sejnowski, T. J. (1995). An information-maximization approach to blind separation and blind deconvolution. Neural Computation, 7, 1129– 1159. Bell, A. J., & Sejnowski, T. J. (1997). The independent components of natural scenes are edge Žlters. Vision Research, 37, 3327–3338. Billock, V. A. (1991). The relationship between simple and double-opponent cells. Vision Research, 31, 33–42. Blakemore, C., & Cooper, G. F. (1970). Development of the brain depends on the visual environment. Nature, 228, 477–478. Blakemore, C., & van Sluyters, R. C. (1975). Innate and environmental factors in the development of the kitten’s visual cortex. Journal of Physiology, 248, 663–716. Cardoso, J. F. (1997). Infomax and maximum likelihood for source separation. IEEE letters on Signal Processing, 4, 112–114. Conway, B. R. (2001). Spatial structure of cone inputs to color cells in alert macaque primary visual cortex (v-1). Journal of Neuroscience, 21, 2768–2783. Dacey, D. M. (2000). Parallel pathways for spectral coding in primate retina. Annual Review of Neuroscience, 23, 743–775. de Monasterio, F. M., McCrane, E. P., Newlander, J. K., & Schein, S. J. (1985). Density proŽle of blue-sensitive cones along the horizontal meridian of macaque retina. Investigative Ophthalmology and Visual Science, 26, 289–302. De Valois, R. L., Cottaris, N. P., Elfar, S. D., Mahon, L. E., & Wilson, J. A. (2000). Some transformations of color information from lateral geniculate nucleus to striate cortex. PNAS, 97, 4997–5002. De Valois, R. L., & De Valois, K. K. (1993). A multi-stage color model. Vision Research, 33, 1053–1065. Field, D. J. (1994). What is the goal of sensory coding? Neural Computation, 6, 559–601. Girolami, M. (1999). Self-organizing neural networks: Independent component analysis and blind source separation. New York: Springer-Verlag. Golub, G. H., & Loan, C. F. (1996). Matrix computation (3rd ed.). Baltimore: Johns Hopkins University Press.

416

E. Doi, T. Inui, T. Lee, T. Wachtler, and T. Sejnowski

Hanazawa, A., Komatsu, H., & Murakami, I. (2000). Neural selectivity for hue and saturation of colour in the primary visual cortex of the monkey. European Journal of Neuroscience, 12, 1753–1763. Hoyer, P. O., & Hyvarinen, A. (2000). Independent component analysis applied to feature extraction from color and stereo images. Network, 11, 191–210. Hyvarinen, A., Karhunen, J., & Oja, E. (2001). Independent component analysis. New York: Wiley. Ingling, C. R. Jr., & Martinez-Uriegas, E. (1983). The relationship between spectral sensitivity and spatial sensitivity for the primate r-g X-channel. Vision Research, 23, 1495–1500. Johnson, E. N., Hawken, M. J., & Shapley, R. (2001). The spatial transformation of color in the primary visual cortex of the macaque monkey. Nature Neuroscience, 4, 409–416. Kingdom, F. A. A., & Mullen, K. T. (1995). Separating colour and luminance information in the visual system. Spatial Vision, 9, 191–219. Knoblauch, K., Vital-Durand, F., & Barbur, J. L. (2001). Variation of chromatic sensitivity across the life span. Vision Research, 41, 23–36. Lee, T.-W. (1998). Independent component analysis:Theory and applications. Norwell, MA: Kluwer. Lee, T.-W., Girolami, M., & Sejnowski, T. J. (1999). Independent component analysis using an extended infomax algorithm for mixed subgaussian and supergaussian sources. Neural Computation, 11, 417–441. Lee, T.-W., Wachtler, T., & Sejnowski, T. J. (2002). Color opponency is an efŽcient representation of spectral properties in natural scenes. Vision Research, 42, 2095-2103. Lennie, P., & D’Zmura, M. (1988). Mechanisms of color vision. CRC Critical Reviews on Neurobiology, 3, 333–400. Lennie, P., Krauskopf, J., & Sclar, G. (1990). Chromatic mechanisms in striate cortex of macaque. Journal of Neuroscience, 10, 649–669. Livingstone, M., & Hubel, D. (1988). Segregation of form, color, movement, and depth: Anatomy, physiology, and perception. Science, 240, 740–749. Michael, C. R. (1978). Color vision mechanisms in monkey striate cortex: Simple cells with dual opponent-color receptive Želds. Journal of Neurophysiology,41, 1233–1249. Mollon, J. D., & Bowmaker, J. K. (1992). The spatial arrangement of cones in the primate fovea. Nature, 360, 677–679. Mullen, K. T. (1985). The contrast sensitivity of human color vision to red-green and blue-yellow chromatic gratings. Journal of Physiology, 359, 381–400. Olshausen, B. A., & Field, D. J. (1996). Emergence of simple-cell receptive Želd properties by learning a sparse code for natural images. Nature, 381, 607–609. Ruderman, D. L., Cronin, T. W., & Chiao, C.-C. (1998). Statistics of cone responses to natural images: Implications for visual coding. Journal of the Optical Society of America A, 15, 2036–2045. Stockman, A., & Sharpe, L. T. (2000). Spectral sensitivities of the middle- and long-wavelength sensitive cone derived from measurements in observers of known genotype. Vision Research, 40, 1711–1737.

Information-Theoretic Analyses of Cone Mosaic Responses

417

Szel, A., Diamantstein, T., & Rohlich, P. (1988). IdentiŽcation of the blue-sensitive cones in the mammalian retina by anti-visual pigment antibody. Journal of Comparative Neurology, 273, 593–602. Tailor, D. R., Finkel, L. H., & Buchsbaum, G. (2000). Color-opponent receptive Želds derived from independent component analysis of natural images. Vision Research, 40, 2671–2676. Teller, D. Y. (1998). Spatial and temporal aspects of infant color vision. Vision Research, 38, 3275–3282. van Hateren, J. H., & van der Schaaf, A. (1998). Independent component Žlters of natural images compared with simple cells in primary visual cortex. Proceedings of Royal Society of London B, 265, 359–366. Vrhel, M. J., Gershon, R., & Iwan, L. S. (1994). Measurement and analysis of object reectance spectra. Color Research and Application, 19, 4–19. Wachtler, T., Lee, T.-W., & Sejnowski, T. J. (2001). Chromatic structure of natural scenes. Journal of Optical Society of America A, 18, 65–77. Wachtler, T., Sejnowski, T. J., & Albright, T. D. (1999). Interactions between stimulus and background chromaticities in macaque primary visual cortex. Investigative Ophthalmology and Visual Science, 40, 641. Received April 23, 2002; accepted July 23, 2002.

LETTER

Communicated by Erkki Oja

Linear Geometric ICA: Fundamentals and Algorithms Fabian J. Theis [email protected] Institute of Biophysics, University of Regensburg, Germany Andreas Jung [email protected] Institute for Theoretical Physics, University of Regensburg, Germany Carlos G. Puntonet [email protected] Elmar W. Lang elmar.lang@biologie,uni-regensburg.de Department of Architecture and Computer Technology, University of Granada, Spain Geometric algorithms for linear independent component analysis (ICA) have recently received some attention due to their pictorial description and their relative ease of implementation. The geometric approach to ICA was proposed Žrst by Puntonet and Prieto (1995). We will reconsider geometric ICA in a theoretic framework showing that Žxed points of geometric ICA fulŽll a geometric convergence condition (GCC), which the mixed images of the unit vectors satisfy too. This leads to a conjecture claiming that in the nongaussian unimodal symmetric case, there is only one stable Žxed point, implying the uniqueness of geometric ICA after convergence. Guided by the principles of ordinary geometric ICA, we then present a new approach to linear geometric ICA based on histograms observing a considerable improvement in separation quality of different distributions and a sizable reduction in computational cost, by a factor of 100, compared to the ordinary geometric approach. Furthermore, we explore the accuracy of the algorithm depending on the number of samples and the choice of the mixing matrix, and compare geometric algorithms with classical ICA algorithms, namely, Extended Infomax and FastICA. Finally, we discuss the problem of high-dimensional data sets within the realm of geometrical ICA algorithms. 1 Introduction

Given a random vector, independent component analysis (ICA) tries to Žnd its statistically independent components. This idea can also be used to solve the blind source separation (BSS) problem, which is, given only Neural Computation 15, 419–439 (2003)

° c 2002 Massachusetts Institute of Technology

420

F. Theis, A. Jung, C. Puntonet, and E. Lang

the mixtures of some underlying independent source signals, to separate the mixed signals—henceforth called sensor signals—thus recovering the original sources. In contrast to correlation-based transformations such as principal component analysis (PCA), ICA renders the output signals as statistically independent as possible by evaluating higher-order statistics. The idea of ICA was Žrst expressed by Jutten, HÂerault, Comon, and Sorouchiary (1991), while the term ICA was later coined by Comon (1994). However, the Želd became popular only with the seminal paper by Bell and Sejnowski (1995), who elaborated on the Infomax principle Žrst advocated by Linsker (1989, 1992). Many other ICA algorithms have been proposed, with the FastICA algorithm (Hyv¨arinen, 1999) being the most efŽcient among them. Recently, geometric ICA algorithms have received further attention due to their relative ease of implementation (Puntonet & Prieto, 1995). They have been applied successfully to the analysis of real-world biomedical data (Bauer, Puntonet, Rodriguez-Alvarez, & Lang, 2000) and have been extended as well to nonlinear ICA problems (Puntonet, Alvarez, Prieto, & Prieto, 1999). 2 Basics

In linear BSS, a random vector X: V ! Rn called sensor signal is given; it originates from an unknown independent random vector S: V ! Rn , which will be denoted as source signal, via a mixing process with an unknown mixing matrix A 2 Gl(n), that is, X D A ± S. Note that we assume as many sensors as sources. Here, V denotes a Žxed probability space and Gl(n) :D 6 fW 2 Mat(n £ nI R ) | det(W) D 0g the general linear group in Rn . Only the sensor signal is observable, and the task is to recover A and thereby S D A ¡1 ± X. Uniqueness of the solutions can be acertained if we allow at most one of the source variables Si :D p i ± S, where p i : Rn ! R denotes the projection on the ith coordinate, to be gaussian. Then any solution to the BSS problem, that is, any B 2 Gl(n) such that B ± X is independent, is equivalent to A ¡1 , where equivalent means that B can be written as B D LPA ¡1 with an invertible diagonal matrix (scaling matrix) L 2 Gl(n) and an invertible matrix with unit vectors in each row (permutation matrix) P 2 Gl(n) (Comon, 1994). Vice versa, any matrix B that is equivalent to A¡1 solves the BSS problem, since the transformed mutual information is invariant under scaling and permutation of coordinates. 3 Geometric Considerations

The basic idea of the geometric separation method lies in the fact that in the source space fs1 , . . . , sl g ½ Rn , where si represent a Žxed number of samples of the source vector S with zero mean, the data clusters along the axes of the coordinate system are transformed by A into data clusters along

Linear Geometric ICA

421

a2

a1

x

2

j=0

x1 Figure 1: A two-dimensional scatter plot of a mixture of two Laplacian signals with identical variance. The signals have been mixed by a matrix A mapping the unit vectors onto vectors inclined with angle ai to the x1 -axis. The ragged line along the circle is the histogram of the observations after projection onto the circle plotted in polar coordinates. Dashed lines show borders of the receptive Želds—the borders of the circle sections that lie closest to the angles ai or ¡ai , respectively.

transformed coordinate axes through the origin. The detection of these n new axes allows determining a demixing matrix B that is equivalent to A ¡1 (see Figure 1). We now consider the learning process to be terminated and describe precisely how to recover the matrix A then—after the axes, which span the observation space, have been extracted from the data successfully. Let 6 ig L :D f(x1 , . . . , xn ) 2 Rn | 9i xi > 0, xj D 0 for all j D

422

F. Theis, A. Jung, C. Puntonet, and E. Lang

be the set of positive coordinate axes, and denote with L 0 :D AL the image of this set under the transformation A. Note that due to A being bijective, L 0 intersects the unit (n ¡ 1) -sphere, Sn¡1 :D fx 2 Rn | |x| D 1g, in exactly n distinct points fp1 , . . . , pn g and that those pi ’s form a basis of Rn . DeŽne the matrix Mp1 ,...,pn 2 Gl(n) to be the linear mapping of ei onto pi for i D 1, . . . , n, that is, Mp1 ,...,pn D ( p1 | ¢ ¢ ¢ |pn ). This matrix thus effects the linear coordinate change from the standard coordinates ( ei ) to the new basis ( pi ). Note that for this coordinate transformation, the following lemma holds: For any permutation s 2 Sn , the two matrices Mp1 ,...,pn and Mps (1) ,...,ps(n) are equivalent.

Lemma 1.

Now we can state the following theorem: Theorem 1 (Uniqueness of the Geometric Method).

The matrix Mp1 ,...,pn is

equivalent to A. By construction of Mp1 ,...,pn , we have Mp1 ,...,pn ( ei ) D pi D f ( ei ) D so there exists a li 2 Rnf0g such that Mp1 ,...,pn (ei ) D li Aei . DeŽning L such that L ( ei ) :D li ei yields an invertible diagonal matrix L 2 Gl(n) such that Mp1 ,...,pn D LA. This shows the claim. Proof. Aei |Aei | ,

Corollary 1.

The matrix Mp¡1 solves the BSS problem. 1 ,...,pn

4 The Ordinary Geometric Algorithm

For now, we will restrict ourselves to the two-dimensional case for simplicity; extensions to higher dimensions are discussed in section 11. Let S: V ¡! R2 be an independent two-dimensional Lebesgue-continuous random vector describing the source pattern distribution; its density function is denoted by r: R2 ¡! R. As S is independent, r factorizes r ( x, y) D r1 ( x)r 2 ( y ) , with ri : R ¡! R denoting the corresponding marginal source density functions. As a further simpliŽcation, we will assume the source variables Si to have zero mean E (S ) D 0 and to be distributed symmetrically, that is, r i ( x) D r i (¡x ) for x 2 R and i D 1, . . . , n. To ensure stability of the geometric algorithm, we have to assume that the signal distributions are nongaussian and unimodal. In practice, these restrictions are often met at least approximately.

Linear Geometric ICA

423

As above, let X denote the sensor signal vector and A the invertible mixing matrix such that X D A ± S. Without loss of generality, assume that A is of the form Á ! cos a1 cos a2 AD , sin a1 sin a2 where ai 2 [0, p ) denote two angles. The ordinary geometric learning algorithm (Puntonet & Prieto, 1995) for symmetric distributions in its simplest form then is as follows: Pick four starting vectors w1 , w01 , w2 , and w02 on S1 such that wi and w0i are opposite each other (i.e., wi D ¡w0i for i D 1, 2) and w1 and w2 are linearly independent vectors in R2 . Usually one takes the unit vectors w1 D e1 and w2 D e2 . Furthermore, Žx P a learning rate g: N P¡! R with (Cottrell, Fort, & Pag`es, 1994) g(t) > 0, n2N g (n ) D 1 and n2N g (n ) 2 < 1. Then iterate the following step until an appropriate abort condition has been met: Choose a sample x ( t) 2 R2 according to the distribution of X. If x ( t) D 0, pick a new one. Note that this case happens with probability zero since the probability density function (pdf) r X of X is assumed to be continuous. x ( t) Project x ( t) onto the unit sphere to get y ( t) :D |x(t) | . Let i be in f1, 2g such that 0 wi or wi is the vector closest to y with respect to a Euclidean metric. Then update wi (t) according to the following update rule, ³ ´ y ( t) ¡ wi ( t) wi ( t C 1) :D pr wi ( t) C g ( t) , |y(t) ¡ wi ( t) | where pr: R2 nf0g ¡! S1 represents the projection onto the unit sphere, and w0i ( t C 1) :D ¡wi (t C 1). The other two w’s are not moved in this iteration. In Figures 2 and 3, the learning algorithm has been visualized on the sphere and after the projection onto [0, p ) . This weight update rule resembles unsupervised competitive learning rules used in many clustering algorithms like k-means, vector quantization, or Kohonen’s self-organizing maps, but with the modiŽcations that the step size along the direction of a sample does not depend on distance and the learning process takes place on S1 , not in R. 5 Formal Model of the Geometric Algorithm

Now we present a formal theoretical framework for geometric ICA that will be used in the next section to formulate a proper convergence condition. First, we show using the symmetry of S that it is in fact not necessary to have two vectors wi and w0i moving around on the same axis. Indeed, we should not speak of vectors but of lines in R2 , so the wi ’s would be in

424

F. Theis, A. Jung, C. Puntonet, and E. Lang

w2(0)

*

*

w2(too)

* w (t ) *w (0)

x2

1 oo

1

x1 Figure 2: Visualization of the geometric algorithm with starting points w1 (0) and w2 (0) and end points w1 (1) and w2 (1) . Dash-dotted lines mark receptive Želd borders.

the real projective space RP1 D S1 / », where » identiŽes antipodal points. This is the manifold of all one-dimensional subvector spaces of R2 . A metric is deŽned by setting d ([x], [y]) :D minf|x ¡ y|, |x C y|g for [x], [y] 2 RP1 . Alternatively, one can picture the w’s in S1C :D S1 \ f(x1 , x2 ) 2 R2 | x2 ¸ 0g / », where » identiŽes the two points (1, 0) and (¡1, 0) . Let f : S1 ¡! S1C represent the canonical projection. Furthermore, it is useful to introduce polar coordinates Q: S1C ¡! [0, p ) on S1C with the stratiŽcation Q0 : R ¡! S1C such that Q0 ± Q D id, where id denotes the identity. Let  :D Q ± Q0 : R ¡! [0, p ) be the modulo p map. We are interested in the projected random sensor signal vector pr ±X: V ¡! S1 , so after cutting open the circle S1 and identiŽcation of opposite points, we want to approximate the transformed

Linear Geometric ICA

425

0.01

Probability­ Density [1/Deg]

0.008

* w (t i

oo

)

0.006

0.004

50%

*

0.002

0

w (t) i

w (0) i

ai 0

10

20

50% 30

40 50 60 Angle in F [Deg]

70

80

* 90

i

Figure 3: Plot of the density r Y of a mixture of two Laplacian signals with identical variance. The weight adaptation by the geometric algorithm is also visualized. See also Figure 2.

random variable Y :D Q ± f ± pr ±X: V ¡! [0, p ) in a suitable manner. Note that using the symmetry of r, the density function r Y of the transformed sensor signal Y can be calculated from the density rX of the original sensor signal X by Z 1 rY (Q ) D 2|det A| ¡1 r ( A ¡1 ( r cos Q, r sin Q ) > ) r dr. 0

Then the geometric learning algorithm induces the following discrete Markov process W (t ) : V ¡! R2 , deŽned recursively by W (0) D ( w1 , w2 ) and W ( t C 1) D  2 ( W (t) C g ( t) q ( ( Y (t ) , Y (t) ) ¡ W (t ) ) ) , where ( (sgn (x ) , 0) |y| ¸ |x| q (x, y ) :D (0, sgn ( y) ) |x| > |y| and Y (0) , Y (1) , . . . is a sequence of independent and identically distributed random variables V ¡! R with the same distribution as Y. These random

426

F. Theis, A. Jung, C. Puntonet, and E. Lang

variables will be needed to represent the independence of the successive sampling experiments. Note that the modulo p map  guarantees that W (t C 1) 2 [0, p ) . Indeed, this is just winner-takes-all learning with a signum function in R, but taking into account the fact that we have to stay in [0, p ) . Note that the metric used here is the planar metric, which obviously is equivalent to the metric on S1C induced by the Euclidean metric on S1 ½ R2 . We furthermore can assume that after enough iterations, there is one point a 2 S1 that will not be transversed anymore, and without loss of generality, we assume a to be 0 (otherwise, cut S1 open at a and project along this resulting arc), so that the above algorithm simpliŽes to the planar case with the recursion rule W ( t C 1) D W ( t) C g (t )q ( (Y ( t) , Y ( t) ) ¡ W ( t) ). Without the sign function and the additional fact that the probability distribution of Y is log concave, it has been shown (Cottrell & Fort, 1987; Ritter & Schulten, 1988; Benaim, Fort, & Pag e` s, 1998) that the process W ( t) converges to a unique constant Žxed-point process W ´ w 2 R2 such that Z wi Z b2 ( Fi ) r Y (Q ) dQ D r Y (Q ) dQ b 1 ( Fi )

wi

for i D 1, 2, where Fi :D F ( wi ) :D fQ 2 [0, p ) |  (| Q ¡ wi | ) ·  (| Q ¡ wj |) 6 for all j D ig with Fi D  ([b1 (Fi ) , b2 (Fi ) ]) denotes the receptive Želd of wi and bj ( Fi ) designate the receptive Želd borders. However, it is not clear how to generalize the proof to the geometric case, especially because we do not have (and do not want) log concavity of Y, as this would lead to a unique Žxed point. Therefore, we will assume convergence in a sense stated in the following section. 6 Limit Points of the Geometric Algorithm

In this section, we study the end points of geometric ICA, so we will assume that the algorithm has already converged. The idea is to formulate a condition that the end points will have to satisfy and to show that the solutions are among them. DeŽnition 1 (Geometric Convergence Condition). Two angles l1 , l2 2 [0, p ) satisfy the geometric convergence condition (GCC) if they are the medians of Y restricted to their receptive Želds, that is, if li is the median of r Y | F ( li ) .

O ´ ( wO 1 , wO 2 ) 2 R2 is called Žxed A constant random vector W O (t) ) ) D 0. point of geometric ICA in the expectation if E (q ( Y ¡ W DeŽnition 2.

Hence, the expectation of a Markov process W ( t) starting at a Žxed point of geometric ICA will not be changed by the geometric update rule because E (W ( t C 1) ) D E ( W ( t) ) C g (t) E (q ( Y ( t) ¡ W (t) ) ) D E ( W (t ) ) .

Linear Geometric ICA

427

Given that the geometric algorithm converges to a constant random vector W (1) ´ ( w1 (1) , w2 (1) ) , then W (1) is a Žxed point of geometric ICA in the expectation, if and only if the wi (1) satisfy the GCC. Theorem 2.

Assume W (1) to be a Žxed point of geometric ICA in the expectation. Without loss of generality, let [b 1 , b2 ] be the receptive Želd of w1 (1) such that bi 2 [0, p ). Since W (1) is a Žxed point of geometric ICA in the expectation, we have E ( Â[b1 ,b2 ] ( Y ( t) ) sgn ( Y (t) ¡ w1 (1) ) ) D 0, where Â[b1 ,b2 ] denotes the characteristic function of that interval. But this means R w1 (1) R R (1) (¡1)rY (Q ) dQ C wb2 (1) 1r Y (Q ) dQ D 0 and therefore bw1 r Y (Q ) dQ D b1 1 1 R b2 ( ) (1) r Q dQ, so w1 satisŽes GCC. The same calculation for w2 (1) w1 (1) Y shows one direction of the claim. The other direction follows by simply reading the above proof backward, which completes the proof. Proof.

As before, let pi :D Aei be the transformed unit vectors, and let qi :D Q ± f ± pr(pi ) 2 [0, p ) be the corresponding angles for i D 1, 2. Theorem 3.

The transformed angles qi satisfy GCC.

Because of the symmetry of the claim, it is enough to show that q1 satisŽes GCC. Without loss of generality, let 0 < a1 < a2 < p using the symmetry of r. Then, due to construction, qi D ai . Let b1 :D a1 C2 a2 ¡ p2 and b2 :D b1 C p2 . Then the receptive Želd of q1 can be written (modulo p ) as F ( q1 ) D [b1 , b2 ]. Therefore, we have to show that q1 D a1 is the median of Ra Rb r Y restricted to F ( q1 ) , which means b11 rY (Q ) dQ D a12 rY (Q ) dQ. We will reduce this to the orthogonal standard case A D id by transforming the integral as follows: Z a1 Z a1 Z 1 r Y (Q ) dQ D 2|det A| ¡1 dQ r dr r ( A¡1 ( r cos Q , r sin Q ) > ) Proof.

b1

Z ¡1 D 2|det A|

b1

0

dx dy r ( A¡1 ( x, y) > ) , K

where K :D f(x, y ) 2 R2 | b1 · arctan(y / x ) · a1 g denotes the cone of opening angle a1 ¡ b1R starting from angle b1 . Using the transformation R a formula, we continue b11 rY (Q ) dQ D 2 A¡1 ( K ) dx dy r ( x, y ) . Now note that the transformed cone A¡1 (K ) is a cone ending at the x-axis of opening angle p / 4, because A is linear; therefore, we are left with the following integral: Z a1 Z 1 Z 0 Z 1 Z x r Y (Q ) dQ D 2 dx dy r (x, y ) D 2 dx dy r ( x, ¡y) b1

0

Z D 2

1 0

Z

¡x

0

Z

x

dx 0

dy r ( x, y ) D

b2

a1

0

rY (Q ) dQ,

428

F. Theis, A. Jung, C. Puntonet, and E. Lang

where we have used the same calculation for [a1 , b 2 ] as for [b1 , a1 ] at the last step. This completes the proof of the theorem. Combining both theorems, we have therefore shown: Let W be the set of Žxed points of geometric ICA in the expectation. ¡1 Then there exists ( wO 1 , wO 2 ) 2 W such that Mw O 1 , wO 2 solves the BSS problem. The stable Žxed points in W can be found by the geometric ICA algorithm. Theorem 4.

Furthermore, we believe that in the special case of unimodal, symmetric, and nongaussian signals, the set W consists of only two elements: a stable and an unstable Žxed point, where the stable Žxed point will be found by the algorithm: Assume that the sources Si are unimodal, symmetric, and nongaussian. Then there are only two Žxed points of geometric ICA in the expectation. Conjecture 1.

We can prove this conjecture for the special case of two sources with identical distributions that are nicely super- or subgaussian in the sense that r Y has only four extremal points. Theorem 5. Assume that the sources Si are unimodal and symmetric and that r1 D r2 . Assume that rY | [0, p ) with A D id has exactly two local maxima and two local minima Then there exist only two Žxed points of geometric ICA in the expectation.

The same calculation as in the proof of theorem 3 shows that we can assume A D id without loss of generality. Then by theorem 2 and by r1 D r2 , the two pairs f0, p2 g and f p4 , 3p 4 g satisfy GCC. We have to show that no other pair fulŽlls this condition. ) First, note that the symmetry of r1 and r2 shows that r Y ( np 2 ¡Q D np ) ( C rY ( np Q for 2 and Q 2 , and r r induces even r n Z R D 1 2 Y 2 4 ¡ ). C Q ) D rY ( np Q Due to assumption, r has only two maxima and two Y 4 minima in [0, p ) ; the above equations then show that those have to be at f0, p4 , p2 3p 4 g. np 6 | [b 1 , b1 C p2 ] Now we claim that for b1 D 4 , n 2 Z, the median of rY p does not equal b1 C 4 . Note that this shows theorem 5. For the proof of the claim, consider the smallest integer p 2 Z such that C p4 . Let b 2 :D b1 C p2 and a :D b1 C p4 . We have c 1 :D pp 2 > b 1 , and set c 2 :D c 1 to show that the median of r Y | [b1 , b 2 ] does not equal a. A visualization of these relations and the following deŽnitions is given in Figure 4. Using the symmetry noted above, we have for d1 :D c 1 C (c 1 ¡ b1 ) D Rc Rd 6 2c 1 ¡ b1 : C1 :D b11 r Y D c 11 rY . Note that d1 D a, or else pp D 2c 1 D Proof.

Linear Geometric ICA

429

Figure 4: Visualization of the proof of theorem 5. a, bi , c i , and di are deŽned in the text.

a C b1 D 2b 1 C p2 , which contradicts the assumption to b 1 . Without loss of generality, let a > d1 ; the other case can be proven similarly. Setting d2 :D c 2 C (c 2 ¡ a) D 2c 1 ¡ a, again symmetry shows that C2 :D R c2 R d2 Ra R b2 a r Y D c 2 rY and C3 :D d1 rY D d2 rY I the second equation follows because c 2 ¡ d1 D b2 ¡ c 2 and c 2 ¡ a D d2 ¡ c 2 . Now assume in contradiction to R aour claim R b that a is the median of rY | [b1 , b2 ]. Then we have 2C1 C C3 D b r Y D a 2 r Y D 2C2 C C3 and therefore 1 C1 D C3 . As shown above, r Y (c 1 ) and r Y (c 2 ) are the only extremal points of r Y | [b1 , b2 ]. Without loss of generality, let r Y (c 1 ) be R c a maximum; then r Y (c 2 ) has to be a minimum. But this means that C1 D b11 rY > (c 1 ¡ b1 )r Y (a) ¸ R c2 a r Y D C 2 which contradicts the above. This completes the proof of the theorem. Conjecture 1 states that there are only two Žxed points of geometric ICA. In fact, we claim that of those two, only one Žxed point is stable in the sense that slight perturbations of the initial conditions preserve the convergence. Then, depending on the kurtosis of the sources, either the stable (supergaussian case) or the unstable (subgaussian case) Žxed point represents the image of the unit vectors. This is stated in the following conjecture. Assume that the sources Si are unimodal, symmetric, and nongaussian. Then by conjecture 1, there are only two Žxed points (wO 1 , wO 2 ) and Conjecture 2.

430

F. Theis, A. Jung, C. Puntonet, and E. Lang

(wQ 1 , wQ 2 ) of geometric ICA in the expectation. We claim: i. There is only one stable Žxed point (wO 1 , wO 2 ) . ¡1 ii. If the sources are supergaussian, Mw O , wO solves the BSS problem. 1

iii. If the sources are subgaussian,

¡1 Mw Q 1 , wQ 2

2

solves the BSS problem.

7 Update Rules Without Sign Functions

We have shown that the geometric update step requires the signum function as follows: wi (t C 1) D wi (t) C g(t) sgn ( y ( t) ¡ wi ( t) ) . Then (normally), the wi converge to the medians in their receptive Želd. Note that the medians do not have to coincide with any maxima of the sensor signal density distribution on the sphere, as shown in Figure 5. Therefore, in general, any algorithm searching for the maxima of the distribution (Prieto, Prieto, Puntonet, Canas, & Martin-Smith, 1999) will not end at the medians, which are the correct images of the unit vectors under the given mixing transformation. Only given special restrictions of the sources (same supergaussian distribution of each component, as, for example, speech signals), the medians correspond to the maxima, and a maximum searching algorithm will converge to the correct Žxed points of geometric ICA. 8 Histogram-Based Algorithm: FastGeo

So far in geometric ICA, mostly on-line algorithms with competitive learning rules as in section 4 have been used (Puntonet & Prieto, 1995). As shown above, they search for points satisfying the GCC. In the following, we will establish a new geometric algorithm based on the GCC alone. For this, let n D 2 and Á cos(a1 ) A :D sin(a1 )

! cos(a2 ) . sin(a2 )

(8.1)

Theorem 3 shows that the vectors (cos(ai ) , sin(ai ) ) > satisfy the GCC. Therefore, the vectors wi will converge to the medians in their receptive Želds. This enables us to compute these positions directly using a search on the histogram of Y (see Figure 6), which reduces the computation time by a factor of about 100 or more. In the FastGeo algorithm, we scan through the different receptive Želds and test GCC. In practice, this means discretizing the distribution fY of Y using a given bin size b > 0 and then testing the p / b different receptive Želds. The algorithm will be formulated more precisely in the following.

Linear Geometric ICA

431

0.014

Probability­ Density [1/Deg]

0.012 0.01 0.008 0.006 0.004 0.002

a 0

0

a

1

45

2

90 Angle [Deg]

135

180

Figure 5: Projected density distribution rY of a mixture of two Laplacian signals with different variances, with the mixture matrix mapping the unit vectors ei to (cos ai , sin ai ) for i D 1, 2. Dark line = theoretical density function; gray line = histogram of a mixture of 10,000 samples.

For simplicity, let us assume that the cumulative distribution FY of Y is invertible; this means that FY is nowhere constant. DeŽne a function m : [0, p )

Q

¡! R p² l1 (Q ) C l2 ( Q ) ± 7¡! ¡ QC , 2 2

(8.2)

where ³ li (Q ) :D FY¡1

FY (Q C i p2 ) C FY (Q C ( i ¡ 1) p2 ) 2

´ (8.3)

is the median of Y | [Q C ( i ¡ 1) p2 , Q C i p2 ] in [Q C (i ¡ 1) p2 , Q C i p2 ] for i D 1, 2. Lemma 2.

Let Q be a zero of m in [0, p ) . Then the li (Q ) satisfy the GCC.

432

F. Theis, A. Jung, C. Puntonet, and E. Lang

0.01 F2

F1

Probability­ Density [1/Deg]

0.008

0.006

0.004

0.002

0

a1 0

a2 45

a1 +180°

90

135 180 225 Angle [Deg]

a2 +180° 270

315

360

Figure 6: Probability density function fY of Y from Figure 1 with the mixing angles ai and their receptive Želds Fi for i D 1, 2. ( ) ( ) ( ) ( ) By deŽnition, [ l1 Q C2 l2 Q ¡ p2 , l1 Q C2 l2 Q ] is the receptive Želd of l1 (Q ) . Since m (Q ) D 0, the starting point of the above interval is Q, because Q D l1 (Q ) C l 2 (Q ) ¡ p2 . Hence, we have shown that the receptive Želd of l1 (Q ) is 2 [Q, Q C p2 ], and by construction l1 (Q ) is the median of Y restricted to the above interval. The claim for l2 (Q ) then follows.

Proof.

Algorithm 1 (FastGeo).

Find the zeros of m .

m always has at least two zeros that represent the stable and the unstable Žxed point of the ordinary geometric algorithm. In practice, we extract the Žxed point, which then gives the proper demixing matrix A ¡1 by picking Q0 such that fY ( l1 (Q0 ) ) C fY ( l2 (Q0 ) ) is maximal. For unimodal and supergaussian source distributions, conjecture 2 claims that this results in a stable Žxed point. For subgaussian sources, choosing Q0 with fY ( l1 (Q0 ) ) C fY ( l2 (Q0 ) ) being minimal induces the corresponding demixing matrix. Hence, one advantage of this histogram-based algorithm is that without any modiŽcations, we can solve the ICA problem for subgaussian signals too. Furthermore, the

Linear Geometric ICA

433

sophisticated parameter choice of the ordinary geometric algorithm is not necessary any more; only one parameter, the bin size, has to be chosen. In practice, one sometimes notices that due to the discretization of the distribution, the approximated distribution has a rather noisy shape on small scales. This results from m having zeros being split up into multiple zeros close together. Therefore, a useful improvement of convergence can be established by smoothing this distribution with a kernel function with sufŽciently small halfwidth. This smoothing should be performed preferably during the discretization process of the original distribution. 9 Accuracy

In this section, we consider the dependence of the FastGeo ICA algorithm on the number of samples after the bin size b has been Žxed. As seen in the previous section, the accuracy of the histogram-based algorithm then depends only on the distribution of the samples X (resp. Y); that is, we can estimate the error made by approximating the mixing matrix A by a Žnite number of samples. In the following, we present some results of test runs made with this algorithm. When choosing two arbitrary angles ai 2 [0, p ) , i D 1, 2 for the mixing matrix A, we deŽne a as the distance between these two angles modulo p2 . This will give us an angle in the range between 0 and p2 . In Figure 7, the |a ¡arecovered |

relative accuracy of the recovered angles Daa D i ia is given as a function of the angle a for a Žxed number of samples. Obviously, the resulting graph is reasonably constant over a wide range of a, demonstrating that the estimate of the ai with respect to a is robust over a wide range of a (a > 10± ) and gets only slightly worse for small values of a. Note that the distortions around the origin are due to the Žnite bin size; increasing the number of bins increases the accuracy for small a’s, but also the computational effort. To investigate the relation between the error D a and the number of samples, we examine for different a the dependence of the standard deviation of D a on the number of samples (see Table 1), where we chose the following mixing matrix A: Á A :D

1 0.5

! 0.5 . 1

(9.1)

The table entry gives the standard deviation of the nondiagonal terms after normalizing each column of the mixing matrix, so that the diagonal elements are unity. For comparison, we also calculated the performance index E1 (Amari, Cichocki, & Yang, 1996).

434

F. Theis, A. Jung, C. Puntonet, and E. Lang

0.1

Mean Standard deviation 95% confidence interval

[°]

0.08 0.06 0.04 0.02 0

0

10

20

30

40

50

60

70

80

90

[°] Figure 7: Mixture of 1,000samples of two Laplacian source signals with identical variances. Plotted are the mean, standard deviation, and 95% conŽdence interval of D a/ a versus a calculated from 100 runs for each angle a. Table 1: Standard Deviations of the Nondiagonal Terms and the Performance Index E1 with Different Number of Samples. Number of Samples

Standard Deviation

Index E1

1000 10,000 100,000

0.033 0.013 0.007

0.18 0.07 0.038

10 Examples

In this section, we compare geometric ICA algorithms with other ICA algorithms: the Extended Infomax algorithm (Lee, Girolami, & Sejnowski, 1999), which is based on the classical ICA algorithm (Bell & Sejnowski, 1995), and the FastICA algorithm (Hyv¨arinen & Oja, 1997). As geometric algorithms, we use the ordinary geometric algorithm from section 4 (Puntonet & Prieto, 1995) and the FastGeo algorithm from above. Calculations were made on a PIII-850 PC using Matlab 6.0. In our Žrst example, we consider a mixture of two Laplacian signals. The results of the different algorithms are shown in Table 2. For each algorithm,

Linear Geometric ICA

435

Table 2: Comparison of Time per Run and Cross-talking Error of ICA Algorithms for a Random Mixture of two Laplacian Signals. Algorithm

Elapsed Time [s]

Index E1

Extended Infomax FastICA (pow3 D default) FastICA (tanh) FastICA (gauss) Ordinary Geometric FastGeo

11.1 0.068 0.11 0.12 > 60 0.84

0.072§ 0.002 0.076§ 0.004 0.052§ 0.001 0.048§ 0.001 0.18 § 0.10 0.110§ 0.071

Note: Means and standard deviations were taken over 1000 runs (100 runs for Extended Infomax and Ordinary Geo) with 10,000 samples and uniformly distributed mixing matrix elements.

Table 3: Comparison of Time per Run and Cross-Talking Error of ICA Algorithms for a Random Mixture of Two Sound Signals with 22,000 Samples. Algorithm

Elapsed Time [s]

Index E1

Extended Infomax FastICA (pow3 D default) FastICA (tanh) FastICA (tanh) Ordinary Geometric FastGeo

41.2 0.14 0.24 0.26 > 60 0.89

0.058§ 0.002 0.050§ 0.005 0.022§ 0.001 0.019§ 0.001 0.49 § 0.29 0.136§ 0.087

Note: Means and standard deviations were taken over 1000 runs (100 runs for Extended Infomax and Ordinary Geo) with uniformly distributed mixing matrix elements.

we measure the mean elapsed CPU time per run and the mean cross-talking error E1 with its standard deviation. Both the Extended Infomax and the FastICA algorithm perform best in terms of accuracy, and in terms of computational speed, FastICA lives up to its name, being followed by FastGeo, then InfoMax, and then the ordinary geometric algorithm, always by one scale difference. The last algorithm lacks accuracy and also shows some convergence problems, whereas FastGeo lies between the geometric algorithm and FastICA and Infomax regarding accuracy. The second example deals with real-world data: two audio signals (one speech and one music signal; see Table 3). The results are similar to the Laplacian toy example. FastICA outperforms the other algorithms in terms of speed. The accuracy of Extended Infomax and FastICA is comparable, and FastGeo is slightly (factor 4) worse but faster than the Extended Infomax. The ordinary geometric algorithm is both slower and less accurate, mainly because of convergence problems.

436

F. Theis, A. Jung, C. Puntonet, and E. Lang

11 Higher Dimensions

So far we have explicitly considered two-dimensional data sets only. In realworld problems, however, the sensor signals are usually high-dimensional data sets, for example, EEG data with 21 dimensions. Therefore, it would be satisfactory to generalize geometric algorithms to higher dimensions. The ordinary geometric algorithm can be easily translated to higher-dimensional cases, but one faces serious problems in the explicit calculations. In order to approximate higher-dimensional pdfs, it becomes necessary to have an exponentially growing number of samples available, as will be shown. The number of samples in a ball Bd¡1 of radius q on the unit sphere d¡1 ½ Rd divided by the number of samples on the whole Sd¡1 can be S calculated as follows if we assume a uniformly distributed random vector. Let Bd :D fx 2 Rd | |x| · 1g and Sd¡1 :D fx 2 Rd | |x| D 1g. The volume of Bd can be calculated by vol (Bd ) D

d

p2 ( d2 )!

D cd . It follows for d > 3: d¡1 )q d¡1

n vol(B Number of samples in ball vol(S d¡1 ) D n n

·

q d¡1 cd¡1 q d¡1 d . D p cd C 1

Obviously, the number of samples in the ball decreases by q d¡1 d if q < 1, which is the interesting case. To have the same accuracy when estimating the medians, the decrease must be compensated by an exponential growth in the number of samples. For three dimensions, we have found a good approximate of the demixing matrix by using 100,000 samples. In four dimensions, the reconstructed mixing matrix could not be reconstructed correctly, even with larger numbers of samples. A different approach for higher dimensions has been taken by Bauer et ) al. (2000), where A has been calculated using d ( d¡1 projections of X from 2 2 d R onto R along the different coordinate axes and reconstructed the multidimensional matrix from the two-dimensional solutions. However, this approach works satisfactorily only if the mixing matrix A is close to the unit matrix up to permutation and scaling. 12 Conclusion

The geometric ICA algorithm has been studied in a concise theoretical framework. The Žxed points of the geometric ICA learning algorithm have been examined in detail. We have introduced a GCC, which has to be fulŽlled by the Žxed points of the learning algorithm. We further showed that it is also fulŽlled by the mixed unit vectors spanning the sensor signal space. Hence, geometric ICA can solve the BSS problem. We then gave two conjectures for the unimodal case where the Žxed-point property is expected to be very rigid.

Linear Geometric ICA

437

We then presented a new algorithm for linear geometric ICA (FastGeo) based on histograms that is both robust and computationally much more efŽcient than the ordinary geometric ICA algorithm. The accuracy of the algorithm concerning the estimation of the relevant medians of the underlying data distributions, when varying both the mixing matrices and the sample numbers, has been explored quantitatively, showing a rather robust performance of the algorithm. When comparing FastGeo with classical ICA algorithms and the ordinary geometric one, we noticed that FastGeo performs only slightly worse than the classical ones in terms of accuracy and better than the ordinary geometric one. In terms of speed, FastGeo falls between FastICA and the Extended Infomax and is much faster than the Žrst geometric approaches, which also suffer from severe convergence problems. Furthermore, the fact that geometric algorithms and especially FastGeo are very easy to implement makes FastGeo a good choice even in comparison with the classical ICA algorithms in practical two-dimensional applications. We also considered the problem of high-dimensional data sets with respect to the geometrical algorithms and discussed how projections to lowdimensional subspaces could solve this problem for a special class of mixing matrices. Simulations with nonsymmetrical and nonunimodal distributions have shown promising results so far, indicating that the new algorithm will perform well with almost any distribution. This is the subject of ongoing research in our group. In future work, besides treating nonsymmetric sources S, the two conjectures will have to be proven in full, as well as the Kohonen proof of convergence to be translated into the above model. In addition, the histogrambased algorithm could be extended to the nonlinear case similar to Puntonet et al. (1999), using multiple centered spheres for projection on the surface on which the projected data histograms could then be evaluated. Finally, we are experimenting with the FastGeo algorithm for the overcomplete case, where we are currently able to detect three or more sources in only two mixtures. This can be useful for higher-dimensional cases as in section 11.

Acknowledgments

This research was supported by the Deutsche Forschungsgemeinschaft (DFG) in the Graduiertenkolleg “Nonlinearity and Nonequilibrium in Condensed Matter.” We thank Christoph Bauer and Tobias Westenhuber for suggestions and comments on the ordinary geometric algorithm and Michaela Lautenschlager and Stefanie Ulsamer for their helpful comments in trying to prove convergence.

438

F. Theis, A. Jung, C. Puntonet, and E. Lang

References Amari, S., Cichocki, A., & Yang, H. (1996). A new learning algorithm for blind signal separation. In D. Touretzky, M. Mozer, & M. Hasselmo (Eds.), Advances in neural information processing systems, 8 (pp. 757–763). Cambridge, MA: MIT Press. Bauer, C., Puntonet, C., Rodriguez-Alvarez, M., & Lang, E. (2000). Separation of EEG signals with geometric procedures. In C. Fyfe (Ed.), Engineering of intelligent systems (Proc. EIS’2000) (pp. 104–108). Millet, Alberta, Canada: ICSC Press. Bell, A. J., & Sejnowski, T. J. (1995). An information-maximization approach to blind separation and blind deconvolution. Neural Computation, 7, 1129–1159. Benaim, M., Fort, J.-C., & PagÂes, G. (1998). Convergence of the one-dimensional Kohonen algorithm. Adv. Appl. Prob., 30, 850–869. Comon, P. (1994). Independent component analysis—a new concept? Signal Processing, 36, 287–314. Cottrell, M., & Fort, J.-C. (1987). Etude d’un processus d’autoorganisation. Annales de l’Institut Henri PoincarÂe, 23(1), 1–20. (in French) Cottrell, M., Fort, J. C., & Pag`es, G. (1994). Two or three things that we know about the Kohonen algorithm. In M. Verleysen (Ed.), Proc. ESANN’94, European Symp. on ArtiŽcial Neural Networks (pp. 235–244).Brussels: D Facto Conference Services. Hyva¨ rinen, A. (1999). Fast and robust Žxed-point algorithms for independent component analysis. IEEE Transactions on Neural Networks, 10(3), 626–634. Hyva¨ rinen, A., & Oja, E. (1997). A fast Žxed-point algorithm for independent component analysis. Neural Computation, 9, 1483–1492. Jutten, C., HÂerault, J., Comon, P., & Sorouchiary, E. (1991). Blind separation of sources, parts I, II, and III. Signal Processing, 24, 1–29. Lee, T.-W., Girolami, M., & Sejnowski, T. (1999). Independent component analysis using an extended infomax algorithm for mixed sub-gaussian and supergaussian sources. Neural Computation, 11, 417–441. Linsker, R. (1989). An application of the principle of maximum information preservation to linear systems. In D. Touretzky (Ed.), Advances in neural information processing systems, 1. San Mateo, CA: Morgan Kaufmann. Linsker, R. (1992). Local synaptic learning rules sufŽce to maximize mutual information in a linear network. Neural Computation, 4, 691–702. Prieto, A., Prieto, B., Puntonet, C., Canas, A., & Martin-Smith, P. (1999). Geometric separation of linear mixtures of sources: Application to speech signals. In J. F. Cardoso, Ch. Jutten, & Ph. Loubaton (Eds.), Independent component analysis and signal separation (Proc. ICA’99) (pp. 295–300). Gi`eres, France: Imp. de Ecureuils. Puntonet, C., Alvarez, M., Prieto, A., & Prieto, B. (1999). Separation of speech signals for nonlinear mixtures. Lecture Notes in Computer Science, 1607, 665– 673. Puntonet, C., & Prieto, A. (1995). An adaptive geometrical procedure for blind separation of sources. Neural Processing Letters, 2.

Linear Geometric ICA

439

Ritter, H., & Schulten, K. (1988). Convergence porperties of Kohonen’s topology conserving maps: Fluctuations, stability, and dimension selection. Biological Cybernetics, 60, 59–71. Received December 14, 2001; accepted July 3, 2002.

LETTER

Communicated by Geoffrey Hinton

Equivalence of Backpropagation and Contrastive Hebbian Learning in a Layered Network Xiaohui Xie [email protected] Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, U.S.A. H. Sebastian Seung [email protected] Howard Hughes Medical Institute and Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, U.S.A. Backpropagation and contrastive Hebbian learning are two methods of training networks with hidden neurons. Backpropagation computes an error signal for the output neurons and spreads it over the hidden neurons. Contrastive Hebbian learning involves clamping the output neurons at desired values and letting the effect spread through feedback connections over the entire network. To investigate the relationship between these two forms of learning, we consider a special case in which they are identical: a multilayer perceptron with linear output units, to which weak feedback connections have been added. In this case, the change in network state caused by clamping the output neurons turns out to be the same as the error signal spread by backpropagation, except for a scalar prefactor. This suggests that the functionality of backpropagation can be realized alternatively by a Hebbian-type learning algorithm, which is suitable for implementation in biological networks. 1 Introduction

Backpropagation and contrastive Hebbian learning (CHL) are two supervised learning algorithms for training networks with hidden neurons. They are of interest, because they are generally applicable to wide classes of network architectures. In backpropagation (Rumelhart, Hinton, & Williams, 1986b, 1986a), an error signal for the output neurons is computed and propagated back into the hidden neurons through a separate teacher network. Synaptic weights are updated based on the product between the error signal and network activities. CHL updates the synaptic weights based on the steady states of neurons in two different phases: one with the output neurons clamped to the desired values and the other with the output neurons free (Movellan, 1990; Baldi & Pineda, 1991). Clamping the output neurons Neural Computation 15, 441–454 (2003)

° c 2002 Massachusetts Institute of Technology

442

X. Xie and H. Seung

causes the hidden neurons to change their activities, and this change constitutes the basis for the CHL update rule. CHL was originally formulated for the Boltzmann machine (Ackley, Hinton, & Sejnowski, 1985) and was extended later to deterministic networks (Peterson & Anderson, 1987; Hinton, 1989), in which case it can be interpreted as a mean-Želd approximation of the Boltzmann machine learning algorithm. However, this interpretation is not necessary, and CHL can be formulated purely for deterministic networks (Movellan, 1990; Baldi & Pineda, 1991). Compared to backpropagation, CHL appears to be quite different. Backpropagation is typically implemented in feedforward networks, whereas CHL is implemented in networks with feedback. Backpropagation is an algorithm driven by error, whereas CHL is a Hebbian-type algorithm, with update rules based on the correlation of pre- and postsynaptic activities. CHL has been shown to be equivalent to backpropagation for networks with only a single layer (Movellan, 1990), and there has been some other work to relate CHL to the general framework of backpropagation (HopŽeld, 1987; O’Reilly, 1996; Hinton & McClelland, 1988). However, a direct link between them for networks with hidden neurons has been lacking. To investigate the relationship between these two algorithms, we consider a special network for which CHL and backpropagation are equivalent. This is a multilayer perceptron to which weak feedback connections have been added and with output neurons that are linear. The equivalence holds because in CHL, clamping the output neurons at their desired values causes the hidden neurons to change their activities, and this change turns out to be equal to the error signal spread by backpropagation, except for a scalar factor. 2 The Learning Algorithms

In this section, we describe the backpropagation and CHL algorithms. Backpropagation is in the standard form, implemented in a multilayer perceptron (Rumelhart et al., 1986b). CHL is formulated in a layered network with feedback connections between neighboring layers of neurons. It is an extension of the typical CHL algorithm formulated for recurrent symmetric networks (Movellan, 1990). 2.1 Backpropagation. Consider a multilayer perceptron with L C 1 layers of neurons and L layers of synaptic weights (see Figure 1A). The activities of the kth layer of neurons are denoted by the vector xk, their biases by the vector b k, and the synaptic connections from layer k ¡ 1 to layer k by the matrix W k . All neurons in the kth layer are assumed to have the same transfer function fk, but this transfer function may vary from layer to layer. In particular, we will be interested in the case where fL is linear, though the other transfer functions may be nonlinear. In the basic deŽnition, fk acts on a scalar and returns a scalar. However, we will generally use fk to act on a

Backpropagation and Contrastive Hebbian Learning

443

Figure 1: Diagram on the network structures of the (A) multilayer perceptron and the (B) layered network with feedback connections. Layer 0 is the input, layer L is the output, and the others are hidden layers. The forward connections are the same for both networks. In B, there exist feedback connections between neighboring layers of neurons.

vector, in which case it returns a vector, operating component by component by fk. fk0 is the derivative of fk with respect to its argument. Similar to fk, when fk0 acts on a vector, it returns a vector as well, operated by fk0 in each component. We assume that fk is monotonically increasing. Backpropagation learning is implemented by repeating the following steps for each example in a training set of input-output pairs: 1. In the forward pass, x k D fk (W kxk¡1 C bk )

(2.1)

is evaluated for k D 1 to L, thereby mapping the input x0 to the output xL .

444

X. Xie and H. Seung

2. The desired output d of the network, provided by some teacher, is compared with the actual output xL to compute an error signal, yL D D L ( d ¡ xL ) .

(2.2)

The matrix D k ´ diagffk0 (W kxk¡1 C bk )g is deŽned by placing the components of the vector fk0 ( W kx k¡1 C b k ) in the diagonal entries of a matrix. 3. The error signal is propagated backward from the output layer by evaluating y k¡1 D Dk¡1 W Tk y k

(2.3)

for k D L to 2.

4. The weight update

D Wk

D gy k xTk¡1

(2.4)

is made for k D 1 to L, where g > 0 is a parameter controlling the learning rate. 2.2 Contrastive Hebbian Learning. To formulate CHL, we consider a modiŽed network in which, in addition to the feedforward connections from layer k ¡ 1 to layer k, there are also feedback connections between neighboring layers (see Figure 1B). The feedback connections are assumed to be symmetric with the feedforward connections, except that they are multiplied by a positive factor c . In other words, the matrix c W Tk contains the feedback connections from layer k back to layer k ¡ 1. CHL is implemented by repeating the following steps for each example of the training set:

1. The input layer x 0 is held Žxed, and the dynamical equations dx k ) C x k D f k ( W k x k¡1 C c W T kC 1 x kC 1 C b k dt

(2.5)

for k D 1 to L are run until convergence to a Žxed point. The case k D L is deŽned by setting xL C 1 D 0 and WL C 1 D 0. Convergence to a Žxed point is guaranteed under rather general conditions, to be shown later. This is called the free state of the network and is denoted by xL k for the kth layer neurons. 2. The anti-Hebbian update,

D Wk

D ¡gc k¡L xL k xL Tk¡1 ,

is made for k D 1, . . . , L.

(2.6)

Backpropagation and Contrastive Hebbian Learning

445

3. The output layer xL is clamped at the desired value d, and the dynamical equation 2.5 for k D 1 to L ¡ 1 is run until convergence to a Žxed point. This is called the clamped state and is denoted by xO k for the kth layer neurons. 4. The Hebbian update,

D Wk D gc

k¡L

xO kxO Tk¡1 ,

(2.7)

is made for k D 1, . . . , L. Alternatively, the weight updates could be combined and made after both clamped and free states are computed,

D W k D gc

k¡L ( O

x kxO Tk¡1 ¡ xL kxL Tk¡1 ).

(2.8)

This form is the one used in our analysis. This version of CHL should look familiar to anyone who knows the conventional version, implemented in symmetric networks. It will be derived in section 4, but Žrst we prove its equivalence to the backpropagation algorithm. 3 Equivalence in the Limit of Weak Feedback

Next, we prove that CHL in equation 2.8 is equivalent to the backpropagation algorithm in equation 2.4, provided that the feedback is sufŽciently weak and the output neurons are linear. In notation, x k, xO k, and xL k represent the kth layer activities of the feedforward network, the clamped state, and the free state, respectively. We consider the case of weak feedback connections, c ¿ 1, and use ¼ to mean that terms of higher order in c have been neglected and » to denote the order. The proof consists of the following four steps: 1. Show that the difference between the feedforward and free states is of order c in each component, d xL k ´ xL k ¡ x k » c ,

(3.1)

for all k D 1, . . . , L.

2. Show that in the limit of weak feedback, the difference between the clamped and free states satisŽes the following iterative relationship, dx k ´ xO k ¡ xL k D c DkW TkC 1dx kC 1 C O (c for k D 1, . . . , L ¡ 1, and dxL D d ¡ xL L .

L¡k C 1 )

,

(3.2)

446

X. Xie and H. Seung

3. Show that if the output neurons are linear, dxk is related to the error signal in backpropagation through dx k D c

L¡k

y k C O (c

L¡k C 1 )

.

(3.3)

4. Show that the CHL update can be approximated by

D Wk

D gy k xTk¡1 C O (c ) .

(3.4)

In the CHL algorithm, clamping the output layer causes changes in the output neurons to spread backward to the hidden layers because of the feedback connections. Hence, the new clamped state differs from the free state over the entire network, including the hidden neurons. Equation 3.2 states that dx k decays exponentially with distance from the output layer of the network. This is because the feedback is weak, so that dxk is reduced from dxkC 1 by a factor of c . Remarkably, as indicated in equation 3.3, the difference between the clamped and free states is equivalent to the error signal y k computed in the backward pass of backpropagation, except for a factor of c L¡k, when the output neurons are linear. Moreover, this factor annihilates the factor ofc k¡L in the CHL rule of equation 2.8, resulting in the update rule equation 3.4. 3.1 Proof. To prove the Žrst step, we start from the steady-state equation of the free phase,

xL k D fk ( W kxL k¡1 C bk C c W TkC 1 xL kC 1 ) ,

(3.5)

for k D 1, . . . , L ¡ 1. Subtracting this equation from equation 2.1 and performing Taylor expansion, we derive d xL k ´ xL k ¡ x k

D fk ( W k xL k¡1 C bk C c

(3.6) W TkC 1 xL kC 1 )

¡ fk ( W k xk¡1 C bk )

(3.7)

D D k W kd xL k¡1 C c D k W kTC 1 xL k C 1 C O (kW kd xL k¡1 C c W kTC 1 xL k C 1 k2 ) (3.8)

for all hidden layers, and d xL L D DL WLd xL L¡1 C O (kWLd xL L¡1 k2 ) for the output layer. In equation 3.7, the expansion is done around W kx k¡1 C bk. Since the zeroth layer is Žxed with the input, d xL 0 D 0, under the above iterative relationships, we must have d xL k » c in each component, that is, d xL k is in the order of c , for all k D 1, . . . , L. To prove equation 3.2, we compare the Žxed-point equations of the clamped and free states, f ¡1 ( xL k ) D W kxL k¡1 C bk C c W TkC 1 xL kC 1

(3.9)

f ¡1 ( xO k ) D W kxO k¡1 C bk C c W TkC 1 xO kC 1 ,

(3.10)

Backpropagation and Contrastive Hebbian Learning

447

for k D 1, . . . , L ¡ 1, where f ¡1 is the inverse function of f in each component. Subtract them, and perform Taylor expansion around xL k. Recall the deŽnition of dx k ´ xL k ¡ xO k . We have W kdxk¡1 C c W kTC 1dx kC 1 D f ¡1 (xO k ) ¡ f ¡1 ( xL k ) D Jkdx k C O (kd x k k2 ) ,

(3.11) (3.12)

where the matrix Jk ´ diagf@f ¡1 ( xL k ) / @xL kg. Since xL k ¡ xk » c , to the leading order inc , matrix Jk can be approximated by J k ¼ diagf@f ¡1 ( x k ) / @x kg D D ¡1 , k (c C and Jkdx k D D ¡1 dx O kdx k). Substituting this back to equation 3.12, k k k we get dx k D Dk (W kdx k¡1 C c W TkC 1dxkC 1 ) C O (c kdx kk) C O (kd x kk2 ) .

(3.13)

Let us start from k D 1. Since the input is Žxed (dx 0 D 0), we have dx1 D c D1 W 2Tdx2 C O (c kdx1 k) , and therefore, dx1 D c D1 W 2Tdx2 C O (c 2 kdx2 k). Next, we check for k D 2, in which case dx2 D D2 W2dx1 C c D2 W3Tdx3 C O (c kdx2 k) . Substituting dx1 , we have dx2 D c D2 W3Tdx3 C O (c kdx2 k), and therefore, dx2 D c D2 W3Tdx3 C O (c 2 kdx3 k). Following this iteratively, we have dx k D c D kW TkC 1dxkC 1 C O (c 2 kdx kC 1 k),

(3.14)

for all k D 1, . . . , L ¡ 1. Notice that dxL D d ¡ xL L D d ¡ xL C O (c ) is of order 1. Hence, dxL¡1 D c DL¡1 WLTdxL C O (c 2 ) , and iteratively, we have dx k D c D kW TkC 1dxkC 1 C O (c

L¡k C 1 )

,

(3.15)

for k D 1, . . . , L¡1. Therefore, equation 3.2 is proved. This equation indicates that dx k » c dx kC 1 and dx k » c L¡k in each component. If the output neurons are linear, then yL D dxL C O (c ) . Consequently, dxk D c L¡ky k C O (c L¡kC 1 ) for all k D 1, . . . , L. Finally, the weight update rule of CHL follows:

D Wk

D gc k¡L ( xO k xO Tk¡1 ¡ xL k xL Tk¡1 )

(3.16)

D gc k¡Ldx k xL Tk¡1 C gc k¡L xL kdxTk¡1 C gc k¡Ldx kdxTk¡1

(3.17)

D gc k¡Ldx k xL Tk¡1 C O (c )

(3.18)

D gy k xTk¡1 C O (c ).

(3.19)

The last approximation is made because xL k¡1 ¡ x k¡1 » c . This result shows that the CHL algorithm in the layered network with linear output neurons is identical to the backpropagation as c ! 0.

448

X. Xie and H. Seung

For nonlinear output neurons, dxL of CHL is different from yL in equation 2.2 computed in backpropagation. However, they are within 90 degrees if viewed as vectors. Moreover, if the activation function for the output neurons is taken to be the sigmoidal function, that is, fL ( z ) D 1 / (1 C exp(¡z) ) , the CHL algorithm is equivalent to backpropagation based on the cost function, Remark.

¡ dT log(xL ) ¡ (1 ¡ d ) T log(1 ¡ xL ) ,

(3.20)

since in this case, yL D d ¡ xL . In the above, function log acts on a vector and returns a vector as well. 4 Contrastive Function

The CHL algorithm stated in section 2.2 can be shown to perform gradient descent on a contrastive function that is deŽned as the difference of the network’s Lyapunov functions between clamped and free states (Movellan, 1990; Baldi & Pineda, 1991). Suppose E ( x) is a Lyapunov function of the dynamics in equation 2.5. Construct the contrastive function C (W ) ´ E ( xO ) ¡ E ( xL ) , where xO and xL are steady states of the whole network in the clamped and free phase, respectively, and W ´ fW1 , . . . , W L g. For simplicity, let us Žrst assume that E ( x) has a unique global minimum in the range of x and no local minima. According to the deŽnition of Lyapunov functions, xL is the global minimum O but under the extra constraints that the output neurons are of E and so is x, clamped at d. Therefore, C ( W ) D E ( xO ) ¡ E ( xL ) ¸ 0 and achieves zero if and O that is, when the output neurons reach the desired values. only if xL D x, Performing gradient descent on C ( W ) leads to the CHL algorithm. On the other hand, if E ( x) does not have a unique minimum, xL and xO may be only local minima. However, the above discussion still holds, provided that xO is in the basin of attraction of xL under the free phase dynamics. This imposes some constraints on how to reset the initial state of the network after each phase. One strategy is to let the clamped phase settle to the steady state Žrst and then run the free phase without resetting hidden neurons. This will guarantee that C (W ) is always nonnegative and constitutes a proper cost function. Next, we introduce a Lyapunov function for the network dynamics in equation 2.5, E (x ) D

L X kD 1

c

k¡L [1T

FN k ( x k ) ¡ xTk W kx k¡1 ¡ bTk x k],

(4.1)

where function FN k is deŽned so that FN0k ( x ) D fk¡1 (x ) . x ´ fx1 , . . . , xL g represents the states of all layers of the network. Equation 4.1 is extended from

Backpropagation and Contrastive Hebbian Learning

449

Lyapunov functions previously introduced for recurrent networks (HopŽeld, 1984; Cohen & Grossberg, 1983). For E ( x ) to be a Lyapunov function, it must be nonincreasing under the dynamics equation 2.5. This can be shown by EP D D D

´T L ³ X @E xk

kD 1 L X

c

xP k

k¡L [f ¡1 ( ) xk k

kD 1 L X kD 1

¡c

(4.2)

¡ W kx k¡1 ¡ c W TkC 1 x kC 1 ¡ bk ]T xP k

k¡L [f ¡1 ( ) xk k

¡ fk¡1 ( xP k C x k ) ]T [x k ¡ (xP k C x k ) ]

· 0,

(4.3)

(4.4) (4.5)

where the last inequality holds because fk is monotonic as we have assumed. Therefore, E (x ) is nonincreasing following the dynamics and stationary if and only if at the Žxed points. Furthermore, with appropriately chosen fk, such as sigmoid functions, E ( x) is also bounded below, in which case E ( x) is a Lyapunov function. Given the Lyapunov function, we can form the contrastive function C ( W ) and derive the gradient-descent algorithm on C accordingly. The derivative of E (xO ) with respect to W k is X @E @xO k dE ( xO ) @E C D dW k @W k @xO k @W k k D

(4.6)

@E

(4.7)

@W k

D ¡c k¡L xO k xO Tk¡1 ,

(4.8)

where the second equality holds because @E / @xO k D 0 for all k at the steady states. Similarly, we derive dE ( xL ) D ¡c dW k

k¡L L

x kxL Tk¡1 .

(4.9)

Combining equations 4.8 and 4.9, we Žnd the derivative of C ( W ) with respect to W k shall read dC dE ( xO ) dE ( xL ) ¡ D D ¡c dW k dW k dW k

k¡L (

xO k xO Tk¡1 ¡ xL kxL Tk¡1 ).

(4.10)

With a suitable learning rate, gradient descent on C ( W ) leads to the CHL algorithm in equation 2.8.

450

X. Xie and H. Seung

5 Equivalence of Cost Functions

In section 3, we proved that the CHL algorithm in the layered network with linear output neurons is equivalent to backpropagation in the weak feedback limit. Since both algorithms perform gradient descent on some cost function, the equivalence in the update rule implies that their cost functions should be equal, up to a multiplicative or an additive constant difference. Next, we demonstrate this directly by comparing the cost functions of these two algorithms. The backpropagation learning algorithm is gradient descent on the squared difference, kd ¡ xL k2 / 2, between the desired and actual outputs of the network. For the CHL algorithm, the cost function is the difference of Lyapunov functions between the clamped and free states, as shown in the previous section. After reordering, it can be written as CD

L X

c

k¡L

kD 1

[1T ( FN k ( xO k ) ¡ FN k (xL k ) ) ¡ dxTk (W kxL k¡1 C bk ) ¡ dxTk¡1 W kT xO k]. (5.1)

Recall that dx k » c L¡k. Therefore, the dxk term above multiplied by the factor c k¡L is of order 1, whereas the dx k¡1 multiplied by the same factor is of order c , and thus can be neglected in the leading-order approximation. After this, we get CD

L X

c

k¡L [1T (

kD 1

FN k ( xO k ) ¡ FN k (xL k ) ) ¡ dxTk (W kxL k¡1 C bk ) ] C O (c ) .

(5.2)

If the output neurons are linear (fL ( x) D x), then FNL ( x) D xT x / 2 and WL xL L¡1 C bL D xL L . Substituting them into C and separating terms of the output and hidden layers, we derive CD

1 T ( xO xO L ¡ xL TL xL L ¡ dxTL xL L ) 2 L C

L¡1 X kD 1

c

dxTk [fk¡1 ( xL k ) ¡ W kxL k¡1 ¡ bk ] C O (c )

k¡L

1 D kd ¡ xL k2 C O (c ) , 2

(5.3)

where the second term with the sum vanishes because of the Žxed-point equations. In conclusion, to the leading order in c , the contrastive function in CHL is equal to the squared error cost function of backpropagation. The demonstration on the equality of the cost functions provides another perspective on the equivalence between these two forms of learning algorithms.

Backpropagation and Contrastive Hebbian Learning

451

So far, we have always assumed that the output neurons are linear. If this is not true, how different is the cost function of CHL from that of backpropagation? Repeating the above derivation, we get the cost function of CHL for nonlinear output neurons, C D 1T FNL ( xO L ) ¡ 1T FNL ( xL L ) ¡ dxTL fL¡1 (xL L ) C O (c )

¡1 TN D ¡1T FNL ( xL ) ¡ dxT L fL ( xL ) C 1 FL ( d ) C O (c ).

(5.4) (5.5)

With sigmoidal-type activation function for output neurons, FNL ( z ) D x log(z) C (1 ¡ z ) log(1 ¡ z) . Substituting this into equation 5.5, we Žnd C D ¡dT log(xL ) ¡ (1 ¡ d) T log(1 ¡ xL ) C 1T FNL ( d) C O (c ) ,

(5.6)

which is the same as the cost function in equation 3.20 in the small c limit, except a constant difference. 6 Simulation

In this section, we use the backpropagation and the CHL algorithm to train a 784-10-10 three-layer network to perform handwritten digit recognition. The data we use are abridged from the MNIST database containing 5000 training examples and 1000 testing examples. We use the sigmoidal function fk ( x ) D 1 / (1 C exp(¡x) ) for the hidden and output layers. The backpropagation algorithm is based on the cost function in equation 3.20. We simulate the CHL algorithm in the layered network with three different feedback strengths: c D 0.05, 0.1, and 0.5. The label of an input example is determined by the index of the largest output. After each epoch of on-line training, the classiŽcation and squared error for both training and testing examples are computed and plotted in Figure 2. The classiŽcation error is deŽned as the percentage of examples classiŽed incorrectly. The squared error is deŽned as the mean squared difference between the actual and desired output for all training or testing examples. The desired output is 1 on the output neuron whose index is the same as the label and zero on the output neurons otherwise. The learning curve of CHL algorithm is very similar to those of backpropagation for c D 0.05 and 0.1, and it deviates from the learning curve of backpropagation for c D 0.5 (see Figure 2). In the simulation, we Žnd the overall simulation time of the CHL algorithm is not signiŽcantly longer than that of the backpropagation. This is because the layered network tends to converge to a steady state fast in the case of weak feedback. 7 Discussion

We have shown that backpropagation in multilayer perceptrons can be equivalently implemented by the CHL algorithm if weak feedback is added.

452

X. Xie and H. Seung

Figure 2: Comparison of performance between the backpropagation and the CHL algorithm. The two algorithms are used to train a 784-10-10 three-layer network to perform handwritten digit recognition. The CHL algorithm is used to train the three-layer network with feedback c D 0.05, 0.1, and 0.5 added. (A, B) ClassiŽcation and squared errors for training examples. (C, D) Test examples.

This is demonstrated from two perspectives: evaluating the two algorithms directly and comparing their cost functions. The essence behind this equivalence is that CHL effectively extracts the error signal of backpropagation from the difference between the clamped and free steady states. The equivalence between CHL and backpropagation in layered networks holds in the limit of weak feedback, which is true mathematically. This, however, does not imply that in engineering problem solving, we should substitute CHL for backpropagation to train neural networks. This is because in networks with many hidden layers, the difference between the clamped and free states in the Žrst few layers would become very small in the limit of weak feedback, and therefore CHL will not be robust against noise during training. In practice, when CHL is used for training the layered networks with many hidden layers, the feedback strength should not be chosen to be too small, in which case the approximation of CHL to backpropagation algorithm will be inaccurate.

Backpropagation and Contrastive Hebbian Learning

453

The investigation on the relationship between backpropagation and CHL is motivated by research looking for biologically plausible learning algorithms. It is believed by many that backpropagation is not biologically realistic. However, in an interesting study on coordinate transform in posterior parietal cortex of monkeys, Zipser and Anderson (1988) show that hidden neurons in a network model trained by backpropagation share very similar properties to real neurons recorded from that area. This work prompted the search for a learning algorithm, which has similar functionality as backpropagation (Crick, 1989; Mazzoni, Andersen, & Jordan, 1991) and at the same time is biologically plausible. CHL is a Hebbian-type learning algorithm, relying on only pre- and postsynaptic activities. The implementation of backpropagation equivalently by CHL suggests that CHL could be a possible solution to this problem. Mazzoni et al. (1991) also proposed a biologically plausible learning rule as an alternative to backpropagation. Their algorithm is a reinforcementtype learning algorithm, which is usually slow, has large variance, and depends on global signals. In contrast, the CHL algorithm is a deterministic algorithm, which could be much faster than reinforcement learning algorithms. However, a disadvantage of CHL is its dependence on special network structures, such as the layered network in our case. Whether either algorithm is used by biological systems is an important question that needs further investigation in experiments and theory. Acknowledgments

We acknowledge helpful discussions with J. J. HopŽeld and S. Roweis. We thank J. Movellan for suggesting the error function for nonlinear output neurons. References Ackley, D. H., Hinton, G. E., & Sejnowski, T. J. (1985). A learning algorithm for Boltzmann Machines. Cognitive Science, 9, 147–169. Baldi, P., & Pineda, F. (1991). Contrastive learning and neural oscillator. Neural Computation, 3, 526–545. Cohen, M. A., & Grossberg, S. (1983). Absolute stability of global pattern formation and parallel memory storage by competitive neural networks. IEEE Trans. on Systems, Man, and Cybernetics, 13, 815–826. Crick, F. (1989). The recent excitement about neural networks. Nature, 337, 129–132. Hinton, G. E. (1989). Deterministic Boltzmann learning performs steepest descent in weight-space. Neural Computation, 1, 143–150. Hinton, G. E., & McClelland, J. (1988). Learning representations by recirculation. In D. Z. Anderson (Ed.), Neural Information Processing Systems. New York: American Institute of Physics.

454

X. Xie and H. Seung

HopŽeld, J. J. (1984). Neurons with graded response have collective computational properties like those of two-state neurons. Proc. Natl. Acad. Sci. USA, 81, 3088–3092. HopŽeld, J. J. (1987). Learning algorithms and probability distributions in feedforward and feed-back networks. Proc. Natl. Acad. Sci. USA, 84, 8429–8433. Mazzoni, P., Andersen, R. A., & Jordan, M. I. (1991). A more biologically plausible learning rule for neural networks. Proc. Natl. Acad. Sci. USA, 88, 4433–4437. Movellan, J. (1990). Contrastive Hebbian learning in the continuous HopŽeld model. In D. Touretzky, J. Elman, T. Sejnowski, & G. Hinton (Eds.), Proceedings of the 1990 Connectionist Models Summer School (pp. 10–17). San Mateo, CA: Morgan Kaufmann. O’Reilly, R. (1996). Biologically plausible error-driven learning using local activation differences: The generalized recirculation algorithm. Neural Computation, 8, 895–938. Peterson, C., & Anderson, J. (1987). A mean Želd theory learning algorithm for neural networks. Complex Systems, 1, 995–1019. Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986a). Learning internal representations by error propagation. In D. E. Rumelhart, & J. L. McClelland (Eds.), Parallel distributed processing: Explorations in the microstructure of cognition. Vol. 1: Foundations (pp. 318–362). Cambridge, MA: MIT Press. Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986b). Learning representations by back–propagating errors. Nature, 323, 533–536. Zipser, D., & Andersen, R. A. (1988). A back-propagation programmed network that simulates response properties of a subset of posterior parietal neurons. Nature (London), 331, 679–684.

Received January 25, 2002; accepted August 1, 2002.

LETTER

Communicated by Maxwell Stinchcombe

Indexed Families of Functionals and Gaussian Radial Basis Functions Irwin W. Sandberg [email protected] Department of Electrical and Computer Engineering, University of Texas at Austin, Austin, TX 78712, U.S.A. We report on results concerning the capabilities of gaussian radial basis function networks in the setting of inner product spaces that need not be Žnite dimensional. SpeciŽcally, we show that important indexed families of functionals can be uniformly approximated, with the approximation uniform also with respect to the index. Applications are described concerning the classiŽcation of signals and the synthesis of reconŽgurable classiŽers. 1 Introduction

Radial basis functions are of interest in connection with a variety of approximation problems in the neural networks area and in other areas as well. Much is understood about the properties of these functions (Park & Sandberg, 1991, 1993; Hartman, Keeler, & Kowalski, 1990; Lo, 1972; Parzen, 1962). It is known (Park & Sandberg, 1993), for example, that arbitrarily good approximation in L1 ( Rn ) of a general f 2 L1 ( Rn ) is possible using uniform smoothing factors and radial basis functions generated in a certain natural way from a single g in L1 ( Rn ) if and only if g has a nonzero integral. As another example, Hartman et al. (1990) proved that gaussian radial basis functions can uniformly approximate arbitrarily well any continuous real functional deŽned on a compact convex subset of Rn . In Sandberg (2001), an approximation result is proved concerning gaussian radial basis functions in a general inner product space, and two applications are given. In particular, it is shown that gaussian radial basis functions deŽned on Rn can in fact uniformly approximate arbitrarily well over all of Rn any continuous real functional f on Rn that meets the condition that lim | f ( x ) | D 0.

kxk!1

This generalizes the result in Hartman et al. (1990) because, by the LebesgueUrysohn extension theorem (Stone, 1962), sometimes attributed to Tietze, any continuous real functional deŽned on a bounded closed subset of Rn Neural Computation 15, 455–468 (2003)

° c 2002 Massachusetts Institute of Technology

456

I. Sandberg

Figure 1: ClassiŽcation network.

can be extended so that it is deŽned and continuous on all of Rn and meets the above condition.1 The second application concerns the problem of classifying signals, which is of interest in several application areas (e.g., signal detection, computerassisted medical diagnosis, automatic target identiŽcation). Typically, we are given a Žnite number m of pairwise disjoint sets C1 , . . . , Cm of signals, and we would like to synthesize a system that maps the elements of each Cj into a real number aj so that the numbers a1 , . . . , am are distinct. Sandberg (1994) (see also Sandberg et al., 2001) showed that the structure displayed in Figure 1 can perform this classiŽcation for any prescribed distinct a1 , . . . , am , assuming only that the Cj are compact subsets of a real normed linear space. In the Žgure, x is the signal to be classiŽed, y1 , . . . , yk are functionals that can be taken to be linear,2 h denotes a continuous memoryless nonlinear mapping that, for example, can be implemented by a neural network having one hidden nonlinear layer, the blocklabeled Q is a quantizer that for each j maps numbers in the interval (aj ¡0.25r , aj C 0.25r) into aj , where r D miniD6 j |ai ¡aj |, and w is the output of the classiŽer.3 Sandberg (2001) showed that a similar classiŽcation problem can be solved in a very different way using just a generalized gaussian radial basis function network and a quantizer of the type in Figure 1. The network structure considered can also classify signals x whose critical property is that the norm of x is sufŽciently large.

1

For example, let a continuous f0 deŽned on a bounded closed subset A of Rn be given, and let A be contained in an open ball B centered at the origin of Rn . Then, since the complement C of B with respect to Rn is closed (and thus A [ C is closed), by the Lebesgue-Urysohn extension theorem there is a continuous extension f of f0 deŽned on Rn such that f (x) D 0, x 2 C. 2 Examples are given in Sandberg (1994) and Sandberg et al. (2001). 3 The proof given in Sandberg (1994) and Sandberg et al. (2001) expands on a brief remark in Sandberg (1991) that any continuous real functional on a compact subset of a real normed linear space can be uniformly approximated arbitrarily well using only a feedforward neural network with a linear-functional input layer and one memoryless hidden nonlinear (e.g., sigmoidal) layer.

Families of Functionals

457

This article too is concerned with the capabilities of gaussian radial basis functions. Here we show that important indexed families f f (a, ¢) : a 2 Ag of functionals of the type addressed in Sandberg (2001) can be uniformly approximated, with the approximation uniform also with respect to the index. More speciŽcally, we consider a setting in which A denotes, for example, the index set [0, 1], and X2 (we avoid the use of X or X1 because they are used differently in the following section) is taken to be Rn or a certain more general type of subset of an inner product space S2 that may be, for instance, a function space. We show that large families of continuous real functionals f on A £ X2 can be uniformly approximated arbitrarily well using gaussian radial basis functions, in the sense that given 2 > 0, there are a positive integer q, real functions c1 , . . . , cq deŽned on A, positive numbers b1 , . . . , bq , and elements u1 , . . . , uq of a certain subset of S2 such that ­ ­ q ­ ­ X ­ 2 ­ <2 (1.1) f (a, x) ¡ ck ( a) expf¡b kkx ¡ u kk g­ ­ ­ ­ kD 1

for all ( a, x ) 2 A £ X2 , in which k¢k is the norm in S2 . We show also that the coefŽcients ck (a ) can be taken to be given by ck ( a) D ak expf¡b k |a ¡ a k | 2 g,

(1.2)

in which a1 , . . . , aq are real numbers and a1 , . . . , aq belong to A. In particular, for X2 D S2 D Rn , approximation in the sense of inequality 1.1 holds whenever f is continuous on A £ X2 and lim | f ( a, x ) | D 0

kxk!1

uniformly in a (meaning that for each c > 0, there is a j > 0 such that | f ( a, x ) | < c for kxk > j and all a 2 A.). Such f ’s arise naturally. For example, let p be an arbitrary positive integer, and let f1 , f2 , . . . , fp be p continuous real functionals on Rn with the property that lim | fq ( x ) | D 0

kxk!1

for each q. Let I denote f1, 2, . . . , pg, and let A be any bounded closed real interval containing I. There are many functions f from A £ Rn to R such that f (q, x ) D fq ( x ) for all (q, x ) 2 I £ Rn . The collection of these functions is of interest with regard to the possibility of synthesizing an easily reconŽgurable network that can approximate any one of the p functionals, with “easily reconŽgurable” meaning essentially that one simply changes the point a at which the coefŽcients in equation 1.1 are evaluated. An example of such an f : A £ Rn ! R is given by the Lagrange-type interpolation relation, f ( a, x ) D

p X qD 1

c q ( a ) fq ( x ) ,

(1.3)

458

I. Sandberg

in which c q (a ) D

P `2I¡fqg ( a ¡ `) . P `2I¡fqg ( q ¡ `)

This f satisŽes the two conditions mentioned above for approximation in the sense of equation 1.1. To see this, note that since each c q is continuous on A and each fj is continuous on Rn , f is continuous on A £ Rn ,4 and that given c > 0, we may choose j so that for all j, max max |c q ( a) | ¢ | fj ( x ) | < c / p, kxk > j, q

a

in which case we have | f ( a, x ) | ·

p X q D1

|c q (a ) fq ( x ) | · max max |c q ( a) | q

a

p X

| fj (x ) | < c

jD 1

for kxk > j and all a 2 A. Motivated by the connection described in Sandberg (2001) between classiŽcation problems and the implementation of functionals as networks, this leads to the conclusion that there are classiŽers on Rn that are reconŽgurable in a simple manner. (See theorem 4 for an example of a precise statement along these lines, and see theorem 7 for a related proposition.) The results in section 2 show that this conclusion holds also for a large collection of classiŽcation problems in which the objects to be classiŽed are drawn from a function space, and a speciŽc result is given that addresses the case in which the objects are Lipschitz continuous functions deŽned on the n-dimensional interval [0, 1]n . There are applications of the material in section 2 to problems other than classiŽcation problems. In particular, because of the close relation between the problem of approximating shift-invariant nonlinear input-output maps, in which inputs and outputs depend on a time or space variable, and the problem of approximating functionals (see, for instance, Sandberg et al., 2001), the results in section 2 are of direct interest concerning the problem of approximating indexed families of input-output maps. In this connection, cases in which the elements of f f (a, ¢) : a 2 Ag are deŽned on a compact set are also addressed in section 2. The problem of approximating indexed families of maps arises also in connection with shift-varying nonlinear input-output maps, in which time or a space variable plays the role of the index (Sandberg et al., 2001; Sandberg, 1998). It comes up in addition in connection with studies of adaptive systems. 5 4 Regard eachc q and each fj as deŽned on A£Rn (with any of the usual product metrics; Sutherland, 1975), and use the proposition that the sum and product of two continuous R-valued functions on a metric space are continuous. 5 See, in particular, Lo and Bassu (2002) in which the focus is on approximation using sigmoidal functions and maps deŽned on compact subsets of Žnite-dimensional spaces.

Families of Functionals

459

2 Approximation of Indexed Families 2.1 Preliminaries. We Žrst describe a theorem in Sandberg (2001) that we will use. Let S be a real inner-product space6 (i.e., a real pre-Hilbert space) with inner product h¢, ¢i and norm k¢k derived in the usual way from h¢, ¢i. Let X be a metric space whose points are a subset of the points of S, and denote the metric in X by d. With V any convex subset of S such that fk¢ C vk: v 2 Vg separates the points of X (i.e., such that for x and y in X 6 6 with x D ky C vk), and with P any y, there is a v 2 V for which kx C vk D nonempty subset of (0, 1) that is closed under addition, let X 0 denote the set of functions g deŽned on X that have the representation

g ( x) D a expf¡bkx ¡ vk2 g

(2.1)

in which a 2 R, b 2 P, and v 2 V. Let X stand for the set of continuous functions f from X to the reals R with the property that for each 2 > 0, there is a compact subset X f,2 of X such that | f ( x ) ¡ f ( y) | < 2 for x, y 2/ X f,2 . And let X 1 denote the family of f ’s in X such that for each f and each 2 > 0, there is a compact subset X f,2 of X such that | f ( x ) | < 2 for x 2/ X f,2 . The result in Sandberg (2001) is the following: Assume that X is locally compact but not compact7 and that X is such that X 0 ½ X 1 . Then for each f 2 X 1 and each 2 > 0, there are a positive integer q, real numbers a1 , . . . , aq , numbers b 1 , . . . , bq belonging to P, and elements v1 , . . . , vq of V such that Theorem 1.

­ ­ q ­ ­ X ­ 2 ­ ak expf¡b kkx ¡ v kk g­ <2 f (x ) ¡ ­ ­ ­

(2.2)

kD 1

for all x 2 X. Because of, for example, the simple nature of the expression for the ck (a) in equation 1.2, the results in this article are of interest with regard to the approach discussed in Lo and Bassu (2002). For material related in a general sense concerning applications, see, for instance, Lainiotis (1976). In this context, see also material in the control systems literature concerning the topic of gain scheduling. 6 Theorems 1 and 2, which follow, hold also if all inner-product spaces considered are complex inner-product spaces. 7 X is locally compact if for each point x of X there is an open subset Ox of X such that x 2 Ox and the closure of Ox is compact. For example, Rn is locally compact.

460

I. Sandberg

6 Comments. Since the condition that kx ¡ vk D ky ¡ vk is equivalent to the

6 kxk2 ¡kyk2 , and assuming that x D 6 y in X implies condition that 2hx¡y, vi D 6 that x D y in S, we see that V can be taken to be, for instance, any convex subset of S that contains the points of an open ball in S. Of course, P can be taken to be (0, 1) or f1, 2, 3, . . .g. Since expf¡bkx ¡ vk2 g ! 0 as kxk ! 1 when b > 0, we have X 0 ½ X 1 when for each c > 0 there is a compact subset Xc of X such that kxk ¸ c for x 2 X with x 2/ Xc .

2.2 Product Spaces and Indexed Families. Let S1 be a real inner-product space with inner product h¢, ¢i1 and norm k¢k1 , and let S2 be a second real inner-product space with inner product h¢, ¢i2 and norm k¢k2 . The norms in S1 and S2 are derived from their respective inner products in the usual way. Take S (in section 2.1) to be the inner-product space S1 £ S2 with inner product h¢, ¢i given by

h(a1 , b1 ) ( a2 , b2 ) i D ha1 , a2 i1 C hb1 , b2 i2 . Let ( X1 , d1 ) and (X2 , d2 ) be two metric spaces whose points are subsets of the points of S1 and S2 , respectively. (The metrics d1 and d2 are not necessarily derived in the usual way from k¢k1 and k¢k2 .) Take X to be the metric space X1 £ X2 with the metric given by d[ (a1 , b1 ) , ( a2 , b2 )] D d1 (a1 , a2 ) C d2 ( b1 , b2 ) . Recall that V is a convex subset of S such that fk¢ C vk: v 2 Vg separates 6 the points of X (i.e., such that for x and y in X with x D y, there is a v 2 V 6 ky C vk), and that P is any nonempty subset of (0, 1) for which kx C vk D that is closed under addition. For v 2 S, we use v1 and v2 to denote the components of v belonging to S1 and S2 , respectively. One of our main results concerning the approximation of families of functionals is the following. Assume that ( X1 , d1 ) and ( X2 , d2 ) are locally compact but that (X1 , d1 ) or (X2 , d2 ) , or both, are not compact, and that X is such that X 0 ½ X 1 . Then for each f 2 X 1 and each 2 > 0, there are a positive integer q, real numbers a1 , . . . , aq , numbers b1 , . . . , bq belonging to P, and elements v1 , . . . , vq of V such that Theorem 2.

­ ­ q ­ ­ X ­ ­ 2 2 ak expf¡b kka ¡ v k1 k1 g expf¡b kkb ¡ vk2 k2 g­ <2 f ( a, b ) ¡ ­ ­ ­

(2.3)

kD 1

for all (a, b) 2 X.8 8 Any f 2 X can be similarly approximated by adding a real constant (that depends on f ) to the sum in inequality 2.3.

Families of Functionals

461

The proof follows from theorem 1 and the observation that for x D ( a, b ) 2 X, we have Proof.

expf¡b kkx ¡ v kk2 g D expf¡b kka ¡ v k1 k21 g expf¡b kkb ¡ v k2 k22 g. More speciŽcally, X is locally compact but not compact because ( X1 , d1 ) and ( X2 , d2 ) are locally compact and (X1 , d1 ) or ( X2 , d2 ) , or both, are not compact. The metric product X is locally compact (Mukherjea & Pothoven, 1978), and by the equivalence of compactness and sequential compactness in metric spaces, it easily follows that X is not compact.

In theorem 3, below, we refer to the following assumption. A.1: The metric d2 associated with X2 (recall that X2 is a subset of the points of a real innerproduct space S2 with inner product h¢, ¢i2 and norm k¢k2 ) is such that closed bounded subsets of (X2 , d2 ) are compact, ( X2 , d2 ) is unbounded, and such that for each c > 0, there is a compact subset Xc of ( X2 , d2 ) such that kxk2 ¸ c for x 2 X2 with x 2/ Xc . We give two examples. Assumption A.1 is met if X2 D S2 D Rn , in which Rn stands for the linear space of real n-vectors (as suggested earlier, n is an arbitrary positive integer), the inner product and corresponding norm k¢k2 on Rn are the usual ones, and the metric d2 associated with X2 is derived in the usual way from k¢k2 . For our other example, take S2 to be the inner-product space of real continuous functions deŽned on the n-dimensional interval [0, 1]n , with the inner product and corresponding norm on S2 speciŽed by Z hx, yi D

x (w ) y ( w ) dw. [0,1]n

With ` a positive number and with some norm on Rn , take X2 to be the set of all elements of S2 that are Lipschitz continuous with Lipschitz constant `, and let d2 be the usual uniform (i.e., max) metric. By a version of the Arzel`aAscoli theorem (Sutherland, 1975), closed bounded subsets of ( X2 , d2 ) are compact. In addition, (X2 , d2 ) is locally compact because the closure of any open ball in ( X2 , d2 ) is compact, and ( X2 , d2 ) is unbounded. By the equicontinuity of X2 , it follows that for each c > 0, there is a compact subset Xc of ( X2 , d2 ) such that kxk2 ¸ c for x 2 X2 with x 2/ Xc . More speciŽcally, let c be given and set d D sup a1 ,a2 2[0,1]n Ix2X2 |x(a1 ) ¡x(a2 ) |, which is Žnite. The subset Xc of (X2 , d2 ) given by fx 2 X2 : maxa |x(a) | · c C dg is compact, and we have Z

[0,1]n

|x(w) | 2 dw > c

2

for x 2 X2 with x 2/ Xc . This shows that A.1 is met.

462

I. Sandberg

With regard to slightly more general examples, we note that if A.1 is met, then A.1 is met also if ( X2 , d2 ) is replaced with ( XQ 2 , d2 ), where XQ 2 is any nonempty unbounded closed subset of ( X2 , d2 ) . Other examples can be given in which the elements of X2 are not necessarily continuous. Let A be a bounded closed subinterval of R. Suppose that A.1 holds, and let U be any convex subset of S2 such that fk¢ C uk2 : u 2 Ug separates the points of X2 . Let f be a continuous function from ( A, | ¢ |) £ ( X2 , d2 ) 9 to R such that for each c > 0, there is a compact subset Xc of (X2 , d2 ) for which Theorem 3.

6 Xc ) . | f ( a, x ) | < c ( a 2 A, x 2

(2.4)

Then for each 2 > 0 there are a positive integer q, real numbers a1 , . . . , aq , numbers b1 , . . . , bq belonging to P, elements a1 , . . . , aq of A, and elements u1 , . . . , uq of U such that ­ ­ q ­ ­ X ­ ­ ak expf¡b k |a ¡ a k | 2 g expf¡b kkx ¡ u kk22 g­ <2 f ( a, x ) ¡ ­ ­ ­

(2.5)

kD 1

for all (a, x) 2 A £ X2 . The theorem is a corollary of theorem 2 for S1 D R with the usual multiplication inner product, X1 D ( A, | ¢ |) , and the observation that A £ U is a convex subset of S D S1 £S2 for which fk¢ C vk: v 2 A £Ug separates the points of X. In particular, the space ( X2 , d2 ) is locally compact because each open ball in ( X2 , d2 ) is the interior of a compact subset of (X2 , d2 ) . (X2 , d2 ) , because it is unbounded, is thus locally compact but not compact, and of course A is compact and thus locally compact. We have f 2 X 1 because f is continuous, closed balls in ( A£X2 , d ) are compact (we show this below), and by condition 2.4 and the expression for d, in which maxa1 ,a2 2A |a1 ¡ a2 | < 1, for each 2 > 0 there is a closed ball in ( A £ X2 , d) outside of which | f (a, x) | < 2 . SpeciŽcally, let 2 be given, and select a compact subset X2 of ( X2 , d2 ) so 6 X2 and all a 2 A. Choose any ( a 0, x 0 ) 2 A £ X2 , that | f (a, x ) | < 2 for all x 2 and let r be the radius of a closed ball B 0 ( r) in ( X2 , d2 ) centered at x 0 such that B 0 (r ) contains the points of X2 . Observe that for ( a, x ) 2 A £ X2 , but outside the closed ball in ( A£X2 , d ) of radius r C maxa1 ,a2 2A |a1 ¡a2 | centered at ( a0 , x0 ) , we have x 2/ B 0 ( r) and thus x 2/ X2 . Now we make use of the hypothesis that for eachc > 0 there is a compact subset Xc of ( X2 , d2 ) such that kxk2 ¸ c for x 2 X2 with x 2/ Xc , and let Proof.

9 Here, as suggested, | ¢ | denotes the metric associated with the absolute-value norm. In accord with the material in section 2.2, the metric on (A, | ¢ | ) £ (X2 , d2 ) is the sum of the two metrics.

Families of Functionals

463

j > 0 be given. Select (a 0 , b0 ) 2 A £ X2 , and choose r > maxa2A |a ¡ a 0 | so that fx 2 Xj g ½ fb 2 X2 : d2 ( b, b0 ) · r ¡ max |a ¡ a 0 |g. a2A

Let B denote the closed ball in ( A £ X2 , d ) of radius r centered at ( a 0, b0 ). Then for ( a, b) 2 A £ X2 with (a, b) 2/ B, we have |a ¡ a0 | C d2 ( b, b0 ) > r and thus d2 ( b, b 0 ) > r ¡ maxa2A |a ¡ a 0 |, implying that b 2/ Xj . Therefore, for ( a, b) 2 A £ X2 with ( a, b) 2/ B, we have 1

k(a, b) k D (kak21 C kbk22 ) 2 ¸ kbk2 ¸ j, showing that X 0 ½ X 1 . Finally, let C denote a closed ball in (or any closed bounded subset of) ( A £ X2 , d) , and let f(a k, bk ) g1 be a sequence in C. By the compactness of A kD 0 and the compactness of closed bounded subsets of ( X2 , d2 ), there is a point ( a1 , b1 ) 2 A £ X2 and a subsequence f(a 0k, b0k ) g1 of f(ak, bk ) g1 such that kD 0 kD 0 |a 0k ¡ a1 | ! 0 as k ! 1 and d2 ( b0k, b1 ) ! 0 as k ! 1, and thus such that d[ ( a0k, b 0k ) , (a1 , b1 ) ] ! 0 as k ! 1. Since C is closed, (a1 , b1 ) 2 C. Thus, by the equivalence of compactness and sequential compactness in metric spaces, C is compact.10 In the case of either of the two examples given concerning the hypothesis A.1, U can be taken to be X2 , or even any subset of X2 that contains an open ball in ( X2 , d2 ) (see the comments following theorem 1). In theorem 3, a key assumption is that f is continuous on the product space ( A, |¢| ) £(X2 , d2 ). It is not enough that f be continuous separately in each of its two variables. A sufŽcient condition that is often useful is given in the appendix. Our next result concerns classiŽcation. Let U be any convex subset of S2 such that fk¢ C uk2 : u 2 Ug separates the points of X2 , suppose that A.1 holds, let d2 be derived in the usual way from a norm k¢kd2 , and let p be a positive integer. For each ` 2 f1, . . . , pg, let C1 ( `) , . . . , Cm (`) ( `) be pairwise disjoint compact subsets of ( X2 , d2 ) , in which m (`) is a positive integer, and let C0 ( `) D fx 2 X2 : kxkd2 ¸ j`g where j` > maxjD6 0 supfkxkd2 : x 2 Cj ( `) g. Also for each ` 2 f1, . . . , pg, let a0 ( `) , a1 ( `) , . . . , am ( `) (`) be distinct real numbers with a0 ( `) D 0 and, with r` D miniD6 j |ai ( `) ¡ aj (`) |, let Q`: [j ( aj (`) ¡ 0.25r `, aj ( `) C 0.25r `) ! R be speciŽed by Q (a ) ` D aj ( `) for a 2 ( aj ( `) ¡ 0.25r`, aj ( `) C 0.25r`) and each j. Then there are a positive integer Theorem 4.

10 Theorem 3 can be easily extended to cover the case in which A is instead a bounded closed n-dimensional subinterval of Rn , in which n is an arbitrary positive integer. In that case, | ¢ | in the theorem denotes the euclidean norm on Rn , and A D A1 £ ¢ ¢ ¢ £ An , in which the Aj are closed bounded subintervals of R. The proof is essentially the same.

464

I. Sandberg

q, real numbers a1 , . . . , aq , numbers b1 , . . . , bq belonging to P, elements a1 , . . . , aq of [1, p], and elements u1 , . . . , uq of U such that, for each ` 2 f1, . . . , pg, Á

q X

Q` k D1

! ak expf¡b k | ` ¡ ak | g expf¡b kkx 2

¡ u kk22 g

D aj ( `)

for x 2 Cj ( `) with j D 0, 1, . . . , m (`) . ( ) For each ` 2 f1, . . . , pg, let g`: [jmD 0` Cj ( `) ! R be deŽned by g` ( x ) D aj ( `) for x 2 Cj ( `) and each j. Observe that each [jmD 0 Cj ( `) is closed in (X2 , d2 ) , and that each g` is continuous (note that the distance between pairs of the Cj ( `) is positive). Using the Lebesgue-Urysohn extension theorem (Stone, 1962), let f` be a continuous extension of g` to (X2 , d2 ) for each `. Note that for every `, f`( x ) D 0 for kxkd2 ¸ j`. Proceeding as indicated in section 1, and with A D [1, p], construct a continuous function f from ( A, | ¢ |) £ ( X2 , d2 ) to R such that f (l, x ) D f` (x ) for each `, and f (a, x ) D 0 for a 2 A and x outside the compact set fx 2 X2 : kxkd2 · max` j`g. Thus, by theorem 3, for each 2 > 0 there are a positive integer q, real numbers a1 , . . . , aq , numbers b1 , . . . , bq belonging to P, elements a1 , . . . , aq of A, and elements u1 , . . . , uq of U such that ­ ­ q ­ ­ X ­ ­ 2 2 ak expf¡b k |a ¡ a k | g expf¡b kkx ¡ u kk2 g­ <2 (2.6) f ( a, x ) ¡ ­ ­ ­

Proof.

kD 1

for all ( a, x ) 2 A £ X2 . Since we may select 2 to be less than 0.25 min` r`, we have proved the theorem. Referring to the hypotheses of theorem 4, both examples given earlier (just before theorem 3) concerning A.1 are ones in which d2 is derived in the usual way from a norm. The interpretation of theorem 4 is that for each ` 2 f1, . . . , pg, the classiŽcation problem addressed by the theorem can be solved using a structure consisting of only a generalized RBF network followed by a quantizer, and that q and the ak, b k, a k, as well as the u k do not depend on `. Even for p D 1, this family of classiŽcation problems is more general than the one studied in Sandberg (1994) and Sandberg et al. (2001) in that here we also consider the problem of classifying signals x whose critical property is that the norm of x is sufŽciently large. This additional degree of generality is interesting from the viewpoint of understanding the capabilities of the structure considered, but it may not be of much signiŽcance from a practical viewpoint. A result similar to theorem 4, in which the additional class of signals is not considered and whose proof is along the same lines but is more direct, is given in the next section. 2.3 Related Results. The following theorem concerns cases in which (X1 , d1 ) and (X2 , d2 ) are compact.

Families of Functionals

465

Assume that ( X1 , d1 ) and ( X2 , d2 ) are compact. Then for each continuous f : X ! R, and each 2 > 0, there are a positive integer q, real numbers a1 , . . . , aq , numbers b1 , . . . , b q belonging to P, and elements v1 , . . . , vq of V such that ­ ­ q ­ ­ X ­ 2 2 ­ ak expf¡b kka ¡ vk1 k1 g expf¡b kkb ¡ v k2 k2 g­ <2 (2.7) f (a, b) ¡ ­ ­ ­ Theorem 5.

kD 1

for all ( a, b) 2 X. This theorem is a direct application of the following lemma given in Sandberg (2001). 11 Assume that X (of section 2.1) is compact, and let f : X ! R be continuous. Then for each 2 > 0, there are a positive integer q, real numbers a1 , . . . , aq , numbers b1 , . . . , b q belonging to P, and elements v1 , . . . , vq of V such that ­ ­ q ­ ­ X ­ ­ ak expf¡b kkx ¡ v kk2 g­ <2 (2.8) f (x ) ¡ ­ ­ ­ Lemma 1.

kD 1

for all x 2 X. Theorem 6, which follows, is a corollary of theorem 5 in which attention is focused on the approximation of a family of functionals indexed by a single parameter and deŽned on a compact set. Let A be a bounded closed subinterval of R. Suppose that (X2 , d2 ) is compact, and let U be any convex subset of S2 such that fk¢ C uk2 : u 2 Ug separates Theorem 6.

11 The lemma is an extension of the result in Hartman et al. (1990) concerning approximation on compact convex subsets of Rn . Theorems along the lines of theorem 5, involving, for example, sigmoidal functions or more general ridge functions instead of gaussian functions, can be proved using a theorem due to Dieudonne (see Stone, 1962), which asserts that if Y is the cartesian product of compact topological spaces Yc , c 2 C, then any continuous real functional on Y can be uniformly approximated by Žnite sums of Žnite products of continuous functions of one variable on Y. It is not difŽcult to check that for C a Žnite set and for each c , these continuous functions of one variable on Yc can be assumed to belong to any family of R-valued functions on Yc that is dense, in the sense of uniform approximation, in the set of continuous R-valued functions on Yc . However, this approach does not quite lead to theorem 5 in which the same constant bk appears in both exponential factors in the kth term in the sum in inequality 2.7. It is not difŽcult to see that theorem 5 can be generalized directly in that the exponential factors in inequality 2.7 can be replaced with other continuous functions drawn from corresponding dense sets. Also, in the light of a well-known one-point compactiŽcation proposition and a related continuous function extension idea (Stone, 1962) involving locally compact spaces, similar remarks hold in the context of theorem 2. In fact, the proof of theorem 1 given in Sandberg (2001) indirectly makes use of this material in Stone (1962).

466

I. Sandberg

the points of X2 . Let f be a continuous function from ( A, | ¢|) £ (X2 , d2 ) 12 to R. Then for each 2 > 0, there are a positive integer q, real numbers a1 , . . . , aq , numbers b1 , . . . , bq belonging to P, elements a1 , . . . , aq of A, and elements u1 , . . . , uq of U such that ­ ­ q ­ ­ X ­( ­ 2 2 ak expf¡b k |a ¡ ak | g expf¡b kkb ¡ u kk2 g­ <2 (2.9) f a, b ) ¡ ­ ­ ­ kD 1

for all (a, b) 2 A £ X2 . Theorem 7 is a version of theorem 4 whose proof is along the same lines but is more direct. Let U be a convex subset of S2 such that fk¢ C uk2 : u 2 Ug separates the points of X2 . For each ` 2 f1, . . . , pg, let C1 ( `) , . . . , Cm ( `) ( `) be pairwise disjoint compact subsets of ( X2 , d2 ) , in which m ( `) is a positive integer. Also for each ` 2 f1, . . . , pg, let a1 ( `) , . . . , am ( `) ( `) be distinct real numbers and, with r` D miniD6 j |ai (`) ¡ aj (`) |, let Q`: [j (aj ( `) ¡ 0.25r `, aj (`) C 0.25r `) ! R be speciŽed by Q (a ) ` D aj ( `) for a 2 ( aj ( `) ¡ 0.25r `, aj ( `) C 0.25r`) and each j. Then there are a positive integer q, real numbers a1 , . . . , aq , numbers b1 , . . . , bq belonging to P, elements a1 , . . . , aq of [1, p], and elements u1 , . . . , uq of U such that for each ` 2 f1, . . . , pg, Theorem 7.

Á Q`

q X k D1

! ak expf¡b k | ` ¡ ak | g expf¡b kkx 2

¡ u kk22 g

D aj ( `)

for x 2 Cj ( `) with j D 1, . . . , m ( `) . The proof is similar to the proof of theorem 4, so we give only an outline of it. ( ) For each ` 2 f1, . . . , pg, let g`: [jmD 1` Cj (`) ! R be deŽned by g` (x ) D Proof.

( )

m` aj ( `) for x 2 Cj ( `) and each j. Set D D [` [jD 1 Cj (`) . Observe that each [jmD 1 Cj (`) is closed in the metric space ( D, d2 ) , which is compact, and that each g` is continuous (note that the distance between pairs of the Cj ( `) is positive). Using the Lebesgue-Urysohn extension theorem, let f` be a continuous extension of g` to ( D, d2 ) for each `, and with A D [1, p], construct a continuous function f from ( A, | ¢ |) £ ( D, d2 ) to R such that f ( l, x) D f` ( x) for each `. The remainder of the proof makes use of theorem 6 instead of theorem 3 and uses the compactness of D.

12 Again, in accord with the material in section 2.2, the metric on (A, | ¢ |) £ (X2 , d2 ) is the sum of the two metrics.

Families of Functionals

467

For examples of spaces ( X2 , d2 ) of interest, see the examples concerning hypothesis A.1 in section 2.2. Appendix: On Continuity on Product Spaces

Here we record some observations concerning conditions for the continuity of a function deŽned on a product space. Let ( Y1 , d1 ) , ( Y2 , d2 ), and ( Y3 , d3 ) be three metric spaces, and with d deŽned on the product set Y :D (Y1 £ Y2 ) by d[ ( a1 , b1 ) , (a2 , b2 ) ] D d1 (a1 , a2 ) C d2 ( b1 , b2 ) (here d1 , d2 , and d are not necessarily the metrics of section 2.2), let f be a map from the metric space ( Y, d) into ( Y3 , d3 ) . Suppose that for each a 2 Y1 , the map f ( a, ¢) : (Y2 , d2 ) ! ( Y3 , d3 ) is continuous. Suppose also that f is continuous on (Y1 , d1 ) locally uniformly with respect to ( Y2 , d2 ) , in the sense that given ( a 0, b0 ) 2 Y1 £ Y2 and 2 > 0, there are positive constants d1 and d2 such that d3 [ f ( a, b) , f ( a0 , b )] < 2 for d1 ( a, a0 ) < d1 and d2 (b, b0 ) < d2 . Under these conditions, f is continuous on ( Y, d) . To see this, let ( a 0, b0 ) 2 Y1 £ Y2 and 2 1 > 0 be given. Choose d > 0 so that d3 [ f ( a 0, b) , f ( a0 , b0 )] < 2 1 / 2 for d2 ( b, b0 ) < d, and so that d3 [ f ( a, b ) , f ( a 0, b) ] < 2 1 / 2 for d1 ( a, a 0 ) < d and d2 ( b, b0 ) < d. Then observe that for d[ ( a, b) , ( a 0, b0 ) ] < d, we have d3 [ f ( a, b) , f ( a0 , b 0 ) ] · d3 [ f ( a, b) , f ( a0 , b )] C d3 [ f ( a0 , b ) , f (a 0, b0 ) ] < 2 1 , showing that f is continuous. (A simple example of an f that meets the conditions given here is f deŽned by equation 1.3.) We note also that if f is continuous on ( Y, d) and ( Y, d ) is compact, then f is uniformly continuous on (Y, d ) and therefore is both continuous separately in its second variable and continuous on ( Y1 , d1 ) locally uniformly with respect to (Y2 , d2 ) . References Hartman, E. J., Keeler, J. D., & Kowalski, J. M. (1990). Layered neural networks with gaussian hidden units as universal approximators. Neural Computation, 2, 210–215. Lainiotis, D. (1976). Partitioning: A uniform framework for adaptive systems (parts I and II). Proceedings of the IEEE, 64, 1126–1197. Lo, J. T. (1972). Finite dimensional sensor orbits and optimal nonlinear Žltering. IEEE Transactions on Information Theory, IT-18-15, 583–588. Lo, J. T., & Bassu, B. (2002). Adaptive multilayer perceptrons with long- and short-term memories. IEEE Transactions on Neural Networks, 13, 22–33. Mukherjea, A., & Pothoven, K. (1978). Real and functional analysis. New York: Plenum Press.

468

I. Sandberg

Park, J., & Sandberg, I. W. (1991). Universal approximation using radial basis function networks. Neural Computation, 3, 246–257. Park, J., & Sandberg, I. W. (1993). Approximation and radial-basis function networks. Neural Computation, 5, 305–316. Parzen, E. (1962). On estimation of a probability density function and mode. Annals of Mathematical Statistics, 33, 1065–1076. Sandberg, I. W. (1991). Structure theorems for nonlinear systems. Multidimensional Systems and Signal Processing, 2, 267–286. (Errata in vol. 3, no. 1, p. 101, 1992.) Sandberg, I. W. (1994). General structures for classiŽcation. IEEE Transactions on Circuits and Systems-I, 41, 372–376. Sandberg, I. W. (1998).Notes on uniform approximation of time-varying systems on Žnite time intervals. IEEE Transactions on Circuits and Systems-I, 45, 863– 865. Sandberg, I. W. (2001). Gaussian radial basis functions and inner-product spaces. Circuits, Systems, and Signal Processing, 20, 635–642. (Errata in vol. 21, no. 1, p. 123, 2002.) Sandberg, I. W., Lo, J. T., Francourt, C., Principe, J., Katagiri, S., & Haykin, S. (2001). Nonlinear dynamical systems: Feedforward neural network perspectives. New York: Wiley. Stone, M. H. (1962). A generalized Weierstrass approximation theorem. In R. C. Buck (Ed.), Studies in modern analysis. Englewood Cliffs, NJ: Prentice Hall. Sutherland, W. A. (1975). Introduction to metric and topological spaces. Oxford: Clarendon Press.

Received April 17, 2002; accepted July 17, 2002.

LETTER

Communicated by John Platt

EfŽcient Greedy Learning of Gaussian Mixture Models J. J. Verbeek [email protected] N. Vlassis [email protected] B. Krose ¨ [email protected] Informatics Institute, University of Amsterdam, 1098 SJ Amsterdam, The Netherlands This article concerns the greedy learning of gaussian mixtures. In the greedy approach, mixture components are inserted into the mixture one after the other. We propose a heuristic for searching for the optimal component to insert. In a randomized manner, a set of candidate new components is generated. For each of these candidates, we Žnd the locally optimal new component and insert it into the existing mixture. The resulting algorithm resolves the sensitivity to initialization of state-of-the-art methods, like expectation maximization, and has running time linear in the number of data points and quadratic in the (Žnal) number of mixture components. Due to its greedy nature, the algorithm can be particularly useful when the optimal number of mixture components is un known. Experimental results comparing the proposed algorithm to other methods on density estimation and texture segmentation are provided. 1 Introduction

This article concerns the learning (Žtting the parameters) of a mixture of gaussian distributions (McLachlan & Peel, 2000). Mixture models form an expressive class of models for density estimation. Applications in a wide range of Želds have emerged, used for density estimation in unsupervised problems, clustering purposes, estimating class-conditional densities in supervised learning settings, situations where the data are partially supervised, and situations where some observations have missing values. The most popular algorithm to learn mixture models is the expectationmaximization (EM) algorithm (Dempster, Laird, & Rubin, 1977). For a given Žnite data set X n of n observations and an initial mixture f 0 , the algorithm provides a means to generate a sequence of mixture models f f i g with nondecreasing log likelihood on X n . The EM algorithm is known to converge to a locally optimal solution. However, convergence to a globally optimal solution is not guaranteed. The log likelihood of the given data set under the found mixture distribution is highly dependent on the initial mixture f 0. Neural Computation 15, 469–485 (2003)

° c 2002 Massachusetts Institute of Technology

470

J. J. Verbeek, N. Vlassis, and B. Krose ¨

The standard procedure to overcome the high dependence on initialization is to start the EM algorithm for several random initializations and use the mixture yielding maximum likelihood on the data. In this article, we present a method to learn Žnite gaussian mixtures, based on the approximation result of Li and Barron (2000), which states that we can quickly approximate any target density with a Žnite mixture model, as compared to the best we can do with any (possibly nonŽnite) mixture from the same component class. This result also holds if we learn the mixture models in a greedy manner: start with one component and optimally add new components one after the other. However, locating the optimal new component boils down to Žnding the global maximum of a log-likelihood surface. Li (1999) proposed to grid the parameter space to locate the global maximum, but this is infeasible when learning mixtures in high-dimensional spaces. Here, we propose a search heuristic to locate the global maximum. Our search procedure selects the best member from a set of candidate new components that is dynamic in the sense that it depends on the mixture learned so far. Our search procedure for a new component has running time O (n ) . For learning a k component mixture, this results in a run time of O ( nk2 ) if the complete mixture is updated with EM after each insertion. If the mixture is not updated between the component insertions (the approximation result still holds in this case), the run time would be O ( nk). The article is organized as follows. In section 2, we recapitulate the definition and EM learning of gaussian mixtures. Section 3 discusses greedy learning of gaussian mixtures. Our component insertion procedure is discussed in section 4, the core of the article. In section 5, we present experimental results on two tasks: modeling of artiŽcially generated data drawn from several types of gaussian mixtures and texture segmentation. The experiments compare the performance of the new algorithm with the performance of several other methods to learn gaussian mixtures. Section 6 presents conclusions and a discussion. 2 Gaussian Mixtures and the EM Algorithm

A gaussian mixture (GM) is deŽned as a convex combination of gaussian densities. A gaussian density in a d-dimensional space, characterized by its mean m 2 Rd and d £ d covariance matrix C , is deŽned as w (x I h ) D (2p ) ¡d/ 2 det( C ) ¡1/ 2 exp (¡( x ¡ m ) > C ¡1 ( x ¡ m ) / 2),

(2.1)

where h denotes the parameters m and C . A k component GM is then deŽned as fk ( x) D and for

k X jD 1

p j w ( xI h j ) ,

with

j 2 f1, . . . , kg: p j ¸ 0.

k X jD 1

pj D 1 (2.2)

EfŽcient Greedy Learning of Gaussian Mixture Models

471

The p j are called the mixing weights and w (x I h j ) the components of the mixture. As compared to nonparametric density estimation, mixture density estimation (or semi-parametric density estimation) allows for efŽcient evaluation of the density, since only relatively few density functions have to be evaluated. Furthermore, by using few mixture components, a smoothness constraint on the resulting density is enforced, allowing for robust density estimates. The EM algorithm (Dempster et al., 1977) enables us to update the parameters of a given k-component mixture with respect to a data set X n D fx1 , . . . , x n g with all x i 2 Rd , such that the likelihood of X n is never smaller under the new mixture. The updates for a GM can be accomplished by iterative application of the following equations for all components j 2 f1, . . . , kg: P ( j | x i ) :D p j w ( x i I hj ) / fk (x i ) , p j :D m j :D C j :D

n X

(2.3)

P ( j | xi ) / n,

(2.4)

P ( j | xi ) x i / (np j ) ,

(2.5)

P ( j | xi ) ( x i ¡ mj ) (x i ¡ mj ) > / (np j ) .

(2.6)

iD 1 n X iD 1 n X iD 1

The EM algorithm is not guaranteed to lead us to the best solution, where “best” means the solution yielding maximal likelihood on X n among all maxima of the likelihood. (We implicitly assume that the likelihood is bounded by restricting the parameter space, and hence the maximum likelihood estimator is known to exist; (Lindsay, 1983.) The good thing is that if we are close to the global optimum of the parameter space, then it is very likely that by using EM, we obtain the globally optimal solution. 3 Greedy Learning of Gaussian Mixtures

In this section, we discuss the greedy approach to learning GMs. The basic idea is simple. Instead of starting with a (random) conŽguration of all components and improving on this conŽguration with EM, we build the mixture component-wise. We start with the optimal one-component mixture, whose parameters are trivially computed. Then we start repeating two steps until a stopping criterion is met: (1) insert a new component, and (2) apply EM until convergence. The stopping criterion can implement the choice for a prespeciŽed number of components, or it can be any model complexity selection criterion (like minimum description length, Akaike information criterion, or cross validation).

472

J. J. Verbeek, N. Vlassis, and B. Krose ¨

3.1 Motivation. The intuitive motivation for the greedy approach is that the optimal component insertion problem involves a factor k fewer parameters than the original problem of Žnding a near-optimal start conŽguration. We therefore expect it to be an easier problem. Of course, the intuitive assumption is that we can Žnd the optimal (with respect to log likelihood) ( k C 1) -component mixture by local search if we start the search from the mixture obtained by inserting optimally a new component in the optimal k-component mixture. We recently applied a similar strategy to the k-means clustering problem (Likas, Vlassis, & Verbeek, in press) and obtained encouraging results. Two recent theoretical results provide complementary motivation. The Kullback-Leibler (KL) divergence between two densities f and g is deŽned as

Z D ( f k g) D

V

f ( x ) log [ f ( x ) / g ( x )] dx,

(3.1)

where V is the domain of the densities f and g (see Cover & Thomas, 1991, for details). Li and Barron (2000) showed that for an arbitrary probability density function f , there exists a sequence f fi g of Žnite mixtures such that Pk ( ) ( ) ( ) fk ( x ) D i D 1 p i w xIRhi achieves KL divergence D f k f k · D f k gP C c / k for every gP D w ( xI h ) P (dh ) . Hence, the difference in KL divergence achievable by k-component mixtures and the KL divergence achievable by any (possibly nonŽnite) mixture from the same family of components tends to zero with a rate of c / k, where c is a constant dependent not on k but only on the component family. Furthermore, Li and Barron (2000) showed that this bound is achievable by employing the greedy scheme discussed in section 3.2. This tells us that we can quickly approximate any density by the greedy procedure. Therefore, we might expect the results of the greedy procedure—as compared to methods that initialize all at once—to differ more when Žtting mixtures with many components. The sequence of mixtures generated by the greedy learning method can conveniently be used to guide a model selection process in the case of an unknown number of components. Recently, a result of “almost” concavity of the log likelihood of a data set under the maximum likelihood k-component mixture, as a function of the number of components k, was presented (Cadez & Smyth, 2000). The result states that the Žrst-order Taylor approximation of the log likelihood of the maximum likelihood models as a function of k is concave under very general conditions. Hence, if we use a penalized loglikelihood model selection criterion based on a penalty term that is concave or linear in k, then the penalized log likelihood is almost concave. This implies that if the “almost” concave turns out to be concave, then there is only one peak in the penalized log likelihood, and hence this peak can be relatively easily identiŽed with a greedy construction method.

EfŽcient Greedy Learning of Gaussian Mixture Models

473

3.2 A General Scheme for Greedy Learning of Mixtures. Let L ( X n , f k ) P n i D 1 log f k ( xi ) (we will just write L k if no confusion arises) denote the log likelihood of the data set X n under the k-component mixture fk. The greedy learning procedure above can be summarized as follows: D

1. Compute the optimal (yielding maximal log likelihood) one-component mixture f1 . Set k :D 1. 2. Find the optimal new component w ( x Ih ¤ ) and corresponding mixing weight a¤ , fh ¤ , a¤ g D arg max fh,ag

n X iD 1

log [(1 ¡ a) fk (x i ) C aw (x i I h ) ],

(3.2)

while keeping fk Žxed. 3. Set fkC 1 ( x ) :D (1 ¡ a¤ ) fk ( x ) C a¤ w ( x Ih ¤ ) and k :D k C 1. 4. Optional: Update fk using EM until convergence.

5. If a stopping criterion is met, then quit; else go to step 2. Clearly, the crucial step is the component insertion (step 2), which will be the topic of the next section. Let us make a few remarks before we turn to that section. First, the stopping criterion in step 5 can be used to force the algorithm to Žnd a mixture of a prespeciŽed number of components. Step 5 may also implement any kind of model selection criterion. Second, note that step 4 may also be implemented by algorithms other than EM. Furthermore, step 4 is not needed to obtain the approximation result of Li and Barron (2000). Third, note the similarity to the vertex direction method (VDM) (Bohning, ¨ 1995; Lindsay, 1983; Wu, 1978) to learn GMs. Indeed, VDM may be regarded as a speciŽc choice for implementing the search needed for step 2, as we explain in the discussion in Section 6. 4 EfŽcient Search for New Components

Suppose we have obtained a k-component mixture fk , and the quest is to Žnd the component characterized by equation 3.2. It is easily shown that if we Žx fk and h , then L kC 1 is concave as a function of a only, allowing efŽcient optimization. For example, Vlassis and Likas (2002) used a secondorder Taylor approximation. However, L kC 1 as a function of h can have multiple maxima. Hence, we have to perform a global search among the new components in order to identify the optimum since no constructive method to Žnd the global maximum is known. In our new algorithm, the global search for the optimal new component is achieved by starting km partial EM searches from initial parameter pairs f(hi , ai ) g, 1 · i · km. By partial EM search, we mean that we Žx fk and optimize L kC 1 over h and a only. The use of partial EM searches is dictated

474

J. J. Verbeek, N. Vlassis, and B. Krose ¨

by the general scheme above, for which the results of Li and Barron (2000) hold. One may wonder why we would use such partial searches, since we might as well optimize over all parameters of the resulting ( k C 1) -component mixture. The answer is that if we updated all parameters, then the number of computations needed in each of the km individual searches would be O ( nk) instead of O ( n ). Each partial search starts with a different initial conŽguration. After these multiple partial searches, we end up with km candidate new components together with their mixing weights. We pick the candidate component w ( xI hO ) that maximizes the likelihood when mixed into the previous mixture by a factor a, O as in equation 3.2. Then, in step 3 of the general algorithm, in place O a). of the (unknown) optimal parameter pair (h ¤ , a¤ ), we use (h, O Note that a simple uniform quantization of the parameter space, as implemented for VDM for mixtures of univariate gaussian distributions in DerSimonian (1986), is not feasible in general since the number of quantization levels grows exponentially with the dimension of the parameter space. In the following section, we discuss step 2 as proposed in Vlassis and Likas (2002) and its drawbacks. In section 4.2, we present our improved search procedure. 4.1 Every Data Point Generates a Candidate. Vlassis and Likas (2002) proposed using n candidate components. (Henceforth, we refer to this method as VL.) Every data point is the mean of a corresponding candidate. All candidates have the same covariance matrix s 2 I, where s is taken at a theoretically justiŽable value. For each candidate component, the mixing weight a is set to the mixing weight maximizing the second-order Taylor approximation of L kC 1 around a D 1 / 2. The candidate yielding the highest log likelihood when inserted in the existing mixture fk is selected. The selected component is then updated using partial EM steps before it is actually inserted into fk to give fkC 1 . Note that for every component insertion that is performed, all point candidates are considered. There are two main drawbacks to this method. First, using n candidates in each step results in a time complexity O ( n2 ) for the search, which is unacceptable in most applications. O ( n2 ) computations are needed since the likelihood of every data point under every candidate component has to be evaluated. Second, by using Žxed small (referring to the size of the determinant of the covariance matrix) candidate components, the method keeps inserting small components in high-density areas, while in low-density areas, the density is modeled poorly. We observed this experimentally. Larger components that give greater improvement of the mixture are not among the candidates, nor are they found by the EM procedure following the component insertion. Working on improvements of the algorithm in Vlassis and Likas (2002), we noticed that both better and faster performance could be achieved by using a smaller set of candidates tailored to the existing mixture. In the

EfŽcient Greedy Learning of Gaussian Mixture Models

475

next section, we discuss how we can generate the candidates in a data- and mixture-dependent manner. 4.2 Generating Few Good Candidates. Based on our experience with the above method, we propose a new search heuristic for Žnding the optimal new mixture component. Two observations motivate the new search method:

1. The size (i.e., the determinant of the covariance matrix) of the new component should in general be smaller than the size of the existing components of the mixture. 2. As the existing mixture fk contains more components, the search for the optimal new component should become more thorough. In our search strategy, we account for both observations by (1) setting the size of the initial candidates to a value related to (and in general smaller than) the size of the components in the existing mixture and (2) increasing the number of candidates linearly with k. For each insertion problem, our method constructs a Žxed number of candidates per existing mixture component. Based on the posterior distributions, we partition the data set X n in k disjoint subsets Ai D fx 2 X n : i D arg maxj fP( j | x ) gg. If one is using EM for step 4 (of the general scheme given above), then the posteriors P (i | x ) are available directly since we have already computed them for the EM updates of the k-component mixture. For each set Ai (i D 1, . . . , k), we construct m candidate components. In our experiments we used m D 10 candidates, but more can be used, trading run time for better performance. In fact, as in Smola and Sch¨olkopf (2000), we can compute conŽdence bounds of the type: “With probability at least 1 ¡d the best split among m uniformly randomly selected splits is among the best 2 fraction of all splits, if m ¸ dlogd / log (1 ¡ 2 )e.” To generate new candidates from Ai , we pick uniformly randomly two data points x l and x r in Ai . Then we partition Ai into two disjoint subsets Ail and Air . For elements of Ail , the point x l is closer than x r and vice versa for Air . The mean and covariance of the sets Ail and Air are used as parameters for two candidate components. The initial mixing weights for candidates generated from Ai are set to p i / 2. To obtain the next two candidates, we draw new x l and x r , until the desired number of candidates is reached. The initial candidates can be replaced easily by candidates that yield higher likelihood when mixed into the existing mixture. To obtain these better candidates, we perform the partial EM searches, starting from the initial candidates. Each iteration of the km partial updates takes O ( mnk) computations, since we have to evaluate the likelihood of each datum under each of the mk candidate components. In the following section, we discuss how we can reduce the amount of computations needed by a factor k, resulting in only O ( mn) computations to perform one iteration of the partial updates.

476

J. J. Verbeek, N. Vlassis, and B. Krose ¨

Figure 1: The construction of a four-component mixture distribution.

We stop the partial updates if the change in log likelihood of the resulting ( k C 1) -component mixtures drops below some threshold or if some maximal number of iterations is reached. After these partial updates, we set the new component w ( x I hkC 1 ) as the candidate that maximizes the log likelihood when mixed into the existing mixture with its corresponding mixing weight. An example is given in Figure 1, which depicts the evolution of a solution for artiŽcially generated data. The mixtures f1 , . . . , f4 are depicted; each component is shown as an ellipse that has the eigenvectors of the covariance matrix as axes and radii of twice the square root of the corresponding eigenvalue. 4.3 Speeding up the Partial EM Searches. In order to achieve O ( mn) time complexity for the partial EM searches initiated at the O ( km ) initial candidates, we use a lower bound on the true log likelihood instead of the true log likelihood itself. The lower bound is obtained by assuming that the support (nonzero density points in X n ) for a candidate component w ( x , m kC 1 , C kC 1 ) originating from existing component i is fully contained in Ai . The lower bound, termed negative free energy (Neal & Hinton, 1998), is detailed in the appendix. Optimizing the lower bound allows us to base the

EfŽcient Greedy Learning of Gaussian Mixture Models

477

partial updates for a candidate from Ai purely on the data in Ai . The update equations are: aw ( xj I m kC 1 , C kC 1 ) (1 ¡ a) fk ( xj ) C aw ( xj I m kC 1 , C kC 1 ) P j2Ai p ( k C 1 | xj ) xj D P ( ) j2Ai p k C 1 | xj P ( )( )( )> j2Ai p k C 1 | xj xj ¡ m k C 1 xj ¡ m k C 1 P D j2Ai p ( k C 1 | xj ) P j2Ai p ( k C 1 | xj )

p ( k C 1 | xj ) D m kC 1 C kC 1

aD

N

(4.1) (4.2)

(4.3)

(4.4)

4.4 Time Complexity Analysis. The total run time of the algorithm is O ( k2 n ) (or O ( kmn) if k < m). This is due to the updates of the mixtures fi , which cost O ( ni ) computations each if we use EM for these updates. P Therefore, summing over all mixtures, we get O ( ikD 1 ni ) 2 O (nk2 ) . This is a factor k slower than the standard EM procedure. The run times of our experiments conŽrm this analysis; the new method performed on average about k / 2 times slower than standard EM. However, if we use EM to learn a sequence of mixtures consisting of 1, . . . , k components, then the run time would also be O ( nk2 ) . If we do not use the speed-up, the run time would become O ( k2 mn) since in that case, the component allocation step for the i C 1st mixture component would cost O ( imn) . This is a factor m more than the preceding EM steps; hence, the total run time goes up by a factor m. 5 Experimental Results

In this section, we present results of two experiments that compare the results obtained with k-means initialized EM, the method of VL discussed in section 4.1, split-and-merge EM (SMEM; Ueda, Nakano, Ghahramani, & Hinton, 2000), and the new greedy approach presented here. The SMEM algorithm checks after convergence of EM if the log likelihood can be improved by splitting one mixture component in two and merging two other mixture components. To initialize SMEM, we also used the k-means initialization, which makes use of the results of applying the k-means or generalized Lloyd algorithm to the data (Gersho & Gray, 1992). The means are initialized as the centroids of the Voronoi regions (VR) found and the covariance matrices as the covariance matrix corresponding to the data in the VRs. The mixing weights were initialized as the proportion of the data present in each VR. The centroids of the k-means algorithm were initialized as a uniformly random selected subset of the data.

478

J. J. Verbeek, N. Vlassis, and B. Krose ¨

5.1 ArtiŽcial Data. In this experiment, we generated data sets of 400 points in Rd with d 2 f2, 5g. The data were drawn from a gaussian mixture of k 2 f4, 6, 8, 10g components. The separation of the components was chosen: c 2 f1, 2, 3, 4g. A separation of c means (Dasgupta, 1999)

8iD6 j : k¹i ¡ ¹j k2 ¸ c ¢ maxftrace( C i ) , trace( Cj ) g. fi, jg

(5.1)

For each mixture conŽguration, we generated 50 data sets. We allowed a maximum eccentricity (i.e., the largest singular value of the covariance matrix over the smallest) of 15 for each component. Also, for each generated mixture, we generated a test set of 200 points not presented to the mixture learning algorithm. In the comparison below, we evaluate the difference in log likelihood of the test sets under the mixtures provided by the different methods. In Table 1, we show experimental results comparing our greedy approach to VL, SMEM, and several runs of normal EM initialized with k-means. The average of the difference in log likelihood on the test set is given for different characteristics of the generating mixture. The results show that the greedy algorithm performs comparably to SMEM and generally outperforms VL. Both greedy methods and SMEM outperform the k-means initialization by far. 5.2 Texture Segmentation. In this experiment, the task is to cluster a set of 16 £ 16 D 256 pixel image patches. The patches are extracted from 256£256 Brodatz texture images; four of these textures are shown in Figure 2. Because different patches from the same texture roughly differ from each other only in a limited number of degrees of freedom (translation, rotation, scaling, and lightness), patches from the same texture are assumed to form clusters in the 256-dimensional patch space. We conducted experiments where the number of textures from which patches were extracted ranged from k D f2, . . . , 6g. For each value of k, we created 100 data sets of 500k patches each by picking up 500 patches at random from each of k randomly selected textures. The experiment consisted of learning a gaussian mixture model with as many components as different textures involved. We compared our greedy approach to the same methods as in the previous experiment. To speed up the experiment, we projected the data into a linear subspace with principal component analysis. We used a 50-dimensional subspace or a lower than 50-dimensional space if that could capture at least 80% of the data variance. Once the mixture model was learned, we segmented the patches by assigning each patch to the mixture component with the highest posterior probability for this patch. To evaluate the quality of the different segmentations discovered by the different learning methods, we considered how informative the found segmentations are. We measure how predictable the texture label of a patch is given the cluster of the

EfŽcient Greedy Learning of Gaussian Mixture Models

479

Table 1: Detailed Exposition of Log-Likelihood Differences. Vlassis and Likas

dD2

cD1

2

3

4

dD5

cD1

2

3

4

kD4 6 8 10

0.03 0.01 0.00 0.01

0.00 0.03 0.04 0.06

0.01 0.05 0.06 0.07

0.01 0.02 0.03 0.03

kD4 6 8 10

0.05 0.04 0.03 0.04

0.02 0.04 0.07 0.05

¡0.01 0.13 0.13 0.11

0.04 0.13 0.04 0.16

Best of k runs of k-means initialization EM

dD2

cD1

2

3

4

dD5

cD1

2

3

4

kD4 6 8 10

0.03 0.03 0.02 0.06

0.14 0.16 0.23 0.27

0.13 0.34 0.38 0.45

0.43 0.52 0.55 0.62

kD4 6 8 10

0.02 0.09 0.12 0.13

0.28 0.35 0.45 0.49

0.36 0.48 0.75 0.75

0.37 0.54 0.82 0.83

dD2

cD1

2

3

4

dD5

cD1

2

3

4

kD4 6 8 10

¡0.01 ¡0.02 ¡0.02 ¡0.00

0.00 0.00 0.00 0.00

0.00 0.02 0.00 0.03

0.00 0.02 0.03 0.04

kD4 6 8 10

¡0.02 ¡0.06 ¡0.05 ¡0.01

0.01 ¡0.01 ¡0.04 0.04

0.00 ¡0.01 0.00 0.01

0.00 0.00 0.02 0.05

SMEM

Distance to generating mixture

dD2

cD1

2

3

4

dD5

cD1

2

3

4

kD4 6 8 10

0.04 0.07 0.10 0.13

0.03 0.06 0.07 0.12

0.03 0.05 0.07 0.10

0.02 0.04 0.09 0.12

kD4 6 8 10

0.16 0.28 0.45 0.58

0.13 0.22 0.33 0.50

0.14 0.19 0.32 0.45

0.11 0.18 0.42 0.51

Notes: The upper three tables give the log likelihood of a method subtracted from the log likelihood obtained using our greedy approach. The bottom table gives averages of the log likelihood under the generating mixture minus the log likelihood given by our method.

class. This can be done using the conditional entropy (Cover & Thomas, 1991) of the texture label given the cluster label. In Table 2, the average conditional entropies (over 50 experiments) are given. The results of the greedy method and SMEM are roughly comparable, although the greedy approach was much faster. Both provide generally more informative segmentations than using the k-means initialization. Note that VL fails to produce informative clusters in this experiment, probably because the high dimensionality of the data renders spherical candidate components rather unŽt.

480

J. J. Verbeek, N. Vlassis, and B. Krose ¨

Figure 2: Several Brodatz textures. The white squares indicate the patch size.

Table 2: Average Conditional Entropy for Different Values of k. k Greedy k-means SMEM VL Uniform

2

3

4

5

6

0.20

0.34 0.46

0.48 0.74

0.53

0.61

0.22

0.36

1.52 1.59

2.00 2

0.80 0.56 2.29 2.32

0.94 0.68 2.58 2.58

0.19

0.21 0.93 1

Notes: The bottom line gives the conditional entropy if the clusters are totally uninformative. Boldface is used to identify the method with minimum conditional entropy for each k.

EfŽcient Greedy Learning of Gaussian Mixture Models

481

6 Discussion and Conclusion 6.1 Discussion. Both VDM (Bohning, ¨ 1995; Lindsay, 1983; Wu, 1978) and the proposed method are instantiations of the more general greedy scheme given in section 3.2 (although VDM skips step 4 of the scheme). VDM makes use of the directional derivative D fk (w ) of the log likelihood for the current mixture fk, where

D fk (w ) D lim [L ( X n , (1 ¡ a) fk C aw ) ¡ L (X n , fk ) ] / a. a!0

VDM proceeds by picking the w ¤ that maximizes D fk (w ) and inserting this in the mixture with a factor a¤ such that a¤ D arg maxa fL ( X n , (1 ¡a) fk C aw ¤ g. Using the directional derivative at the current mixture can be seen as an instantiation of step 2 of the general scheme. The optimization over both w and a is replaced by an optimization over D fk (w ) (typically implemented by gridding the parameter space) and then an optimization over a. Note that by moving in the direction of maximum, D fk (w ) does not guarantee that we move in the direction of maximum improvement of log likelihood if we optimize over a subsequently. Recently, several novel methods to learn mixtures (of gaussian densities) were proposed (Dasgupta, 1999; Figueiredo & Jain, 2002; Ueda et al., 2000). The Žrst tries to overcome the difŽculties of learning mixtures of gaussian densities in high-dimensional spaces. By projecting the data to a lowerdimensional subspace (which is chosen uniformly randomly), Žnding the means in that space, and then projecting them back, the problems of high dimensionality are reduced. The last two methods try to overcome the dependence of EM on the initial conŽguration as does our method. In Ueda et al. (2000), split-and-merge operations are applied to local optima solutions found by applying EM. These operations constitute jumps in the parameter space that allow the algorithm to jump from a local optimum to a better region in the parameter space. By then reapplying EM, a better (hopefully global) optimum is found. The latter method and the greedy approach are complementary; they can be combined. The split-and-merge operations can be incorporated in the greedy algorithm by checking after each component insertion whether a component i can be removed such that the resulting log likelihood is greater than the likelihood before insertion. If so, the component i is removed and the algorithm continues. If not, the algorithm proceeds as usual. However, in experimental studies on artiŽcially generated data, we found that this combination gave little improvement over the individual methods. An important beneŽt of our new method over Ueda et al. (2000) is that the new algorithm produces a sequence of mixtures that can be used to perform model complexity selection as the mixtures are learned. For example, a kurtosis-based selection criterion, like the one in Vlassis and Likas (1999), can be used here. We also note that our algorithm executes faster

482

J. J. Verbeek, N. Vlassis, and B. Krose ¨

than SMEM, which has a run time of O ( nk3 ) . Figueiredo and Jain (2002) proposed starting with a large number of mixture components and successively annihilating components with small mixing weights. This approach can be characterized as pruning a given mixture, whereas our approach can be characterized as growing a mixture. As compared to both these methods, ours does not need an initialization scheme, since the optimal one-component mixture can be found analytically. Also, Bayesian methods have been used to learn gaussian mixtures. For example, in Richardson and Green (1997), a reversible-jump Markov chain Monte Carlo (MCMC) method is proposed. There, the MCMC allows jumps between parameter spaces of different dimensionality (i.e., parameter spaces for mixtures consisting of differing number of components). However, the experimental results that Richardson and Green (1997) reported indicate that such sampling methods are rather slow compared to constructive maximum likelihood algorithms. It is reported that about 160 sweeps per second are performed on a SUN Sparc 4 workstation. The experiments involve 200,000 sweeps, resulting in about 20 minutes of run time. Although the 200,000 sweeps are not needed for reliable results, this contrasts sharply with the 2.8 seconds and 5.7 seconds run time (allowing respectively about 480 and 960 sweeps) of the standard EM and our greedy EM in a similar experimental setting also executed on a SUN Sparc 4 workstation. 6.2 Conclusion. We proposed an efŽcient greedy method to learn mixtures of gaussian densities that have run time O ( k2 n C kmn) for n data points, a Žnal number of k mixture components, and m candidates per component. The experiments we conducted have shown that our greedy algorithm generally outperforms the k-means initialized EM. In experiments on artiŽcial data, we observed that SMEM performs in general comparable to the new method in terms of quality of the solutions found. The results on natural data were comparable to those obtained with SMEM if SMEM was initialized with the k-means algorithm. An important beneŽt of the greedy method, compared to both SMEM and normal EM, is the production of a sequence of mixtures, which obviates the need for initialization and facilitates model selection. As compared to VL, the O ( n2 k2 ) time complexity has been reduced by a factor n; the somewhat arbitrary choice of spherical candidate components with Žxed variance has been replaced by a search for candidate components that depends on the data and the current mixture; and experiments suggest that if the methods yield different performance, then the new method generally outperforms the old one. Appendix: A Lower Bound for Fast Partial EM

In the insertion step, we have a two-component mixture problem. The Žrst component is the mixture fk, which we keep Žxed in the partial EM steps.

EfŽcient Greedy Learning of Gaussian Mixture Models

483

The second component is the new component w , which has a mixing weight a. The initial k C 1 component mixture is given by fk C 1 D aw C (1 ¡ a) fk . Using the maximization-maximization view on EM (Neal & Hinton, 1998), we can deŽne a lower bound F on the actual log likelihood: X X log fkC 1 ( x i ) ¸ F ( Q, h ) D log fkC 1 ( x i ) ¡ DKL ( Qi k p (¢ | x i ) ) , (A.1) i

i

where DKL denotes Kullback-Leibler divergence and p (¢ | x i ) denotes the posterior probability on the mixture of components. The distributions Qi are the responsibilities used in EM. We write qi for the responsibility of the new component and (1 ¡qi ) for the responsibility of fk for data point x i . The above lower bound can be made to match the log likelihood by setting the responsibilities equal to the posteriors p ( s | x i ) . However, in our case, when considering a candidate based on component 6 Aj : qi D 0. It is easy j, we Žx the Qi for all points outside Aj such that 8xi 2 to see from equation A.1 that the Qi for which x i 2 Aj should match the posterior probability on the new component in order to maximize F. Note that the more the assumption that data points outside Aj have zero posterior on the new component is true, the tighter the lower bound becomes. If the data outside Aj indeed have zero posterior on the new component, the lower bound equals the log likelihood after the partial E-step. For the M-step, we use the alternative decomposition of F, X EQi log p ( x i , s) C H ( Qi ) , (A.2) F (Q, h ) D i

where we used the variable s to range over the two mixture components, fk and w . Note that the entropies H ( Qi ) remain Žxed in the M-step since they do not depend on h. The Žrst term can be written as: X (A.3) qi [log w ( x i ) C log a] C (1 ¡ qi ) [log fk ( xi ) C log (1 ¡ a) ] i

D

X i2Aj

qi [log w ( xi ) C log a] C

X i

(1 ¡ qi ) [log fk ( x i ) C log (1 ¡ a) ]. (A.4)

Note that in equation A.4, the new mixture component w occurs only in terms for data in Aj . Setting the derivative of equation A.5 to zero with regard to the parameters of the new component (mixing weight a, mean m and covariance matrix C ) gives the update equations in section 4.3. Note that from equation A.4, F can also be evaluated using only the points in Aj if we stored the complete data log likelihood under fk in advance. Acknowledgments

We thank Aristidis Likas for useful discussions and a referee for pointing out the conŽdence bounds used in Smola and Scholkopf ¨ (2000). This research

484

J. J. Verbeek, N. Vlassis, and B. Krose ¨

is supported by the Technology Foundation STW (project AIF4997) applied science division of NWO, and the technology program of the Dutch Ministry of Economic Affairs. Software implementing the algorithm in MATLAB is available from J. J. V. by e-mail.

References Bohning, ¨ D. (1995). A review of reliable maximum likelihood algorithms for semiparametric mixture models. J. Statist. Plann. Inference, 47, 5–28. Cadez, I. V., & Smyth, P. (2000). On model selection and concavity for Žnite mixture models. In Proc. of Int. Symp. on Information Theory (ISIT). Available on-line at: http://www.ics.uci.edu / »icadez / publications.html. Cover, T., & Thomas, J. (1991). Elements of information theory. New York: Wiley. Dasgupta, S. (1999). Learning mixtures of gaussians. In Proc. IEEE Symp. on Foundations of Computer Science. New York. Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, 39(1), 1–38. DerSimonian, R. (1986). Maximum likelihood estimation of a mixing distribution. J. Roy. Statist. Soc. C, 35, 302–309. Figueiredo, M. A. T., & Jain, A. K. (2002). Unsupervised learning of Žnite mixture models. IEEE Transactions on Pattern Analysis and Machine Intelligence,24, 381– 396. Gersho, A., & Gray, R. M. (1992). Vector quantization and signal compression. Norwell, MA: Kluwer. Li, J. Q. (1999). Estimation of mixture models. Unpublished doctoral dissertation, Yale University. Li, J. Q., & Barron, A. R. (2000). Mixture density estimation. In S. A. Solla, T. K. Leen, & K.-R. Muller ¨ (Eds.), Advances in neural information processing systems, 12. Cambridge, MA: MIT Press. Likas, A., Vlassis, N., & Verbeek, J. J. (in press). The global k-means clustering algorithm. Pattern Recognition. Lindsay, B. G. (1983). The geometry of mixture likelihoods: A general theory. Ann. Statist., 11(1), 86–94. McLachlan, G. J., & Peel, D. (2000). Finite mixture models. New York: Wiley. Neal, R. M., & Hinton, G. E. (1998). A view of the EM algorithm that justiŽes incremental, sparse, and other variants. In M. Jordan (Ed.), Learning in graphical models (pp. 355–368). Norwood, MA: Kluwer. Richardson, S., & Green, P. J. (1997). On Bayesian analysis of mixtures with an unknown number of components. J. Roy. Statist. Soc. B, 59(4), 731–792. Smola, A. J., & Sch¨olkopf, B. (2000). Sparse greedy matrix approximation for machine learning. In Proc. of Int. Conf. on Machine Learning (pp. 911–918). San Mateo, CA: Morgan Kaufmann. Ueda, N., Nakano, R., Ghahramani, Z., & Hinton, G. E. (2000). SMEM algorithm for mixture models. Neural Computation, 12, 2109–2128.

EfŽcient Greedy Learning of Gaussian Mixture Models

485

Vlassis, N., & Likas, A. (1999). A kurtosis-based dynamic approach to gaussian mixture modeling. IEEE Trans. Systems, Man, and Cybernetics, Part A, 29, 393–399. Vlassis, N., & Likas, A. (2002). A greedy EM algorithm for gaussian mixture learning. Neural Processing Letters, 15(1), 77–87. Wu, C. (1978). Some algorithmic aspects of the theory of optimal designs. Annals of Statistics, 6, 1286–1301. Received November 13, 2001; accepted June 27, 2002.

LETTER

Communicated by John Platt

SMO Algorithm for Least-Squares SVM Formulations S.S. Keerthi [email protected] Department of Mechanical Engineering, National University of Singapore, Singapore 117576 S.K. Shevade [email protected] Genome Institute of Singapore, National University of Singapore, Singapore 117528 This article extends the well-known SMO algorithm of support vector machines (SVMs) to least-squares SVM formulations that include LS-SVM classiŽcation, kernel ridge regression, and a particular form of regularized kernel Fisher discriminant. The algorithm is shown to be asymptotically convergent. It is also extremely easy to implement. Com putational experiments show that the algorithm is fast and scales efŽciently (quadratically) as a function of the number of examples. 1 Introduction

This article concerns least-squares formulations of support vector machines (SVMs). These formulations, which provide interesting and useful alternatives to SVMs, have been proposed for both regression and classiŽcation. Saunders, Gammerman, and Vovk (1998) proposed the kernel version of ridge regression and empirically demonstrated its usefulness. Suykens and Vandewalle (1999) gave an interesting and useful extension of the idea to classiŽcation and called their method least-squares SVM (LS-SVM). A careful empirical study by Van Gestel et al. (2000) has shown that LS-SVM is comparable to SVM in terms of generalization performance. Recently, Van Gestel et al. (2002) also established the equivalence of LS-SVM with a particular form of regularized kernel Fisher discriminant (KFD) method (Mika, Ratsch, ¨ & Muller, ¨ 2001). While SVMs are characterized by convex quadratic programming problems involving inequality constraints, kernel ridge regression and LS-SVMs are formulated using equality constraints only. Hence, their solution is obtained by solving a system of linear equations (of the order of the number of training examples) involving the dense kernel matrix. To tackle large-scale problems, Suykens, Lukas, Van Dooren, De Moor, and Vandewalle (1999) and Van Gestel et al. (2000) effectively employed the conjugate-gradient method. In this article, we propose a new iterative algorithm for these probNeural Computation 15, 487–507 (2003)

° c 2002 Massachusetts Institute of Technology

488

S.S. Keerthi and S.K. Shevade

lems. It is based on the solution of the Wolfe dual problem using ideas similar to those of the SMO (sequential minimal optimization) algorithm for SVMs (Platt, 1998; Keerthi, Shevade, Bhattacharyya, & Murthy, 2001). The algorithm is extremely easy to implement and scales efŽciently (quadratically) as a function of the number of examples. The scaling is much better than that of the CG algorithm. The article is organized as follows. In section 2, we formulate the problem and develop the Wolfe dual. Optimality conditions for the dual are derived in section 3. A useful condition for terminating numerical algorithms solving the least-squares formulations is also given. These conditions form the basis for the SMO algorithm developed in section 4, where we also prove that the algorithm is asymptotically convergent. In section 5, we discuss the conjugate gradient method of Suykens et al. (1999) and show how the termination criterion derived in section 3 can be easily applied to this method. Computational experiments describing the effectiveness of the algorithm are given in section 6. Section 7 briey discusses the solution of KFD. Section 8 extends the SMO algorithm to least-square formulations without the threshold parameter, followed by concluding remarks in section 9. The appendix gives a pseudocode for the SMO algorithm. 2 Problem Formulation

Throughout, we will use x to denote the input vector of the classiŽcationregression problem and z to denote the feature space vector related to x by the transformation, z D Q ( x) . As in all other kernel designs, we do not assume Q to be known; all computations will be done using only the kernel function, k ( x, xO ) D Q ( x ) ¢ Q ( xO ) ,

(2.1)

where “¢” denotes inner product in the z space. Let f(xi , yi )gm i D 1 denote the training set, where xi is the ith input pattern and yi is the corresponding target value. For classiŽcation problems (two classes), yi takes only two possible values (say, 1 if xi is in class 1 and ¡1 if xi is in class 2), while for regression, yi takes any real value. Let zi D Q ( xi ) . In kernel ridge regression1 and classiŽcation by LS-SVMs, the following optimization problem is solved: min E s.t.

D

1 2 C C 2 kwk 2

P

2 i ji

yi ¡ ( w ¢ zi ¡ b) D ji 8i.

(2.2)

1 Saunders et al. (1998) leave out b from their formulation. This actually leads to a simpler problem. We will discuss its solution in section 8.

SMO Algorithm for Least-Squares SVM Formulation

489

DeŽne Á

! w p I Cj

wQ D

Á zQ i D

zi

!

p1 ei C

8i,

(2.3)

where j is an m-dimensional vector containing the ji ’s and, for every i, ei is the m-dimensional vector in which the ith component is 1 and all other components are 0. DeŽne kQ ( xi , xj ) D zQ i ¢ zQj . By equations 2.1 and 2.3, 1 kQ ( xi , xj ) D k ( xi , xj ) C dij , C

(2.4)

where dij D 1 if i D j and 0 otherwise. It is easy to see that equation 2.2 transforms to min E D

1 kwk Q 2 s.t. yi ¡ ( wQ ¢ zQ i ¡ b ) D 0 8i. 2

(2.5)

The Lagrangian for this problem is LD

X 1 kwk Q 2C ai [yi ¡ ( wQ ¢ zQi ¡ b) ]. 2 i

(2.6)

The KKT optimality conditions are given by

rwQ L D wQ ¡ @L @b

D

X i

X i

ai zQ i D 0

(2.7)

ai D 0.

(2.8)

Note that because equation 2.5 has only equality constraints, the ai ’s are not constrained to be nonnegative. Using equation 2.7, wQ can be expressed as a function of the ai ’s: X ai zQ i . (2.9) wQ (a) D i

Let us now apply Wolfe duality theory to equation 2.5. The Wolfe dual corresponds to the maximization of L subject to equations 2.7 and 2.8, with Q b, and ai ’s as variables. Using equations 2.8 and 2.9, we can simplify the w, Wolfe dual as X 1 max f (a) D ¡ kwQ (a) k2 C ai yi 2 i

s.t.

X i

ai D 0.

(2.10)

490

S.S. Keerthi and S.K. Shevade

P P Note that kwk Q 2 D i j ai aj kQ ( xi , xj ) . Hence, equation 2.10 is a simple convex quadratic programming problem involving a single equality constraint and no nonnegativity constraints. Once the ai ’s are obtained by solving equation 2.10, the primal variables, w and ji ’s, can be determined using equations 2.9 and 2.3. The determination of b will be addressed in the next section. 3 Optimality Conditions for the Dual

To derive proper stopping conditions for algorithms that solve the dual and also determine the threshold parameter b, it is important to write down the optimality conditions for the dual. The Lagrangian for the problem in equation 2.10 is X X 1 ai yi C b ai . LN D ¡ kwQ (a) k2 C 2 i i

(3.1)

DeŽne Fi D ¡

@f @ai

X

Q (a) ¢ zQi ¡ yi D Dw

j

aj kQ ( xi , xj ) ¡ yi ,

(3.2)

where f is deŽned in equation 2.10. The KKT conditions for the dual problem are @LN

@ai

D (b ¡ Fi ) D 0 8 i.

(3.3)

DeŽne bup D max Fi

i up D arg max Fi

blow D min Fi

i low D arg min Fi .

i

i

i

and

i

(3.4)

Then optimality conditions will hold at a given a iff blow D bup.

(3.5)

In equations 3.2 and 3.4, note that Fi , bup, i up, blow , and i low are all functions of a. The functional dependencies have not been noted to avoid notational clutter. Using equations 3.3, 3.2, and 2.5, it is easy to see the close relationship between the threshold parameter b in the primal problem and the multiplier, b. In particular, at optimality, b and b are identical. Therefore, in the rest of the article, b and b will denote the same quantity.

SMO Algorithm for Least-Squares SVM Formulation

491

We will say that an index pair (i, j) deŽnes a violation at a if 6 Fj . Fi D

(3.6)

Thus, optimality conditions will hold at a iff there does not exist any index pair ( i, j) that deŽnes a violation. Suppose ( i, j) satisŽes equation 3.6 at some a. Then it is possible P to achieve an increase in f (while maintaining the equality constraint, ak D 0) by adjusting ai and aj only. To see this, let us deŽne the following: a Q i ( t) D ai ¡ tI

a Q j ( t) D aj C tI

and

6 i, j a Q k ( t) D ak 8k D

(3.7)

and w ( t) D f ( a(t) Q ).

(3.8)

Then it is easy to verify that w 0 ( t) D Fi ¡ Fj ,

(3.9)

6 where Fi and Fj are evaluated at a(t) 0 at Q . Since by equation 3.6, Fi ¡ Fj D t D 0, an increase in w is possible by choosing t suitably away from 0. Since in numerical solution, it takes too much computing time to compute the exact optimal solution, there is a need to deŽne approximate stopping conditions that guarantee a speciŽed closeness to the solution of the primal problem, equation 2.2. We consider a stopping condition based on duality gaps. This has been used effectively by Smola and Sch¨olkopf (1998) in SVM regression. P Suppose a is any vector that satisŽes i ai D 0, b is any real number, P w D i ai zi , and

ji D yi ¡ (w ¢ zi ¡ b ) D b C

ai ¡ Fi , C

(3.10)

where Fi is as deŽned by equation 3.2. Note that ( w, b, j ) satisŽes the constraints of the primal, equation 2.2, and a satisŽes the constraints of the dual, equation 2.10. Let E denote the primal cost corresponding to ( w, b, j ) , E? be the optimal primal cost of equation 2.2, f denote the dual objective function value given by a, and f ? be the optimal dual objective function value. By Wolfe duality (Fletcher, 1987), we have: E ¸ E ? D f ? ¸ f.

(3.11)

Although E ?( D f ?) is unknown, equation 3.11 allows us to obtain guaranteed bounds using duality gap (a computable quantity) deŽned by ¶ Xµ ± ai ² C 2 C ji . ai Fi ¡ (3.12) Dgap D E ¡ f D 2C 2 i

492

S.S. Keerthi and S.K. Shevade

Dgap is a nonnegative quantity that becomes zero iff optimality is reached. Thus, if far g is a sequence that converges to a?, the solution of equation 2.10, then the continuity of the Dgap function implies that fDgapr g goes to zero, where Dgapr is the value of Dgap for ar . Also, for any nontrivial problem, E? D f ? > 0. Thus, it is appropriate to choose 2 > 0 and use the stopping criterion, Dgap · 2 f.

(3.13)

More important, if equation 3.13 is satisŽed, then we can obtain a guaranteed bound for the primal problem using equation 3.11: E ¡ E? · Dgap · 2 f · 2 E.

(3.14)

Thus, if 2 D 10¡p and equation 3.13 is satisŽed, then the closeness of E to E? has more than p decimal digits of accuracy. These ideas are used in interior point methods of linear programming (Vanderbei, 1994) and have also been employed in SVMs (Smola & Sch¨olkopf, 1998). In our implementation of the SMO algorithm derived in the next section, we employ equation 3.13 as the stopping criterion. In section 5, we explain how to use equation 3.13 to efŽciently stop the conjugate gradient algorithm for LS-SVM proposed by Suykens et al. (1999). At a given a, we need a method for determining a good value of b for use in the above calculations as well as for use in the classiŽer function deŽned by o ( x) D w ¢ Q ( x) ¡ b D

X i

ai k ( x, xi ) ¡ b.

(3.15)

One possibility is to take bD

blow C bup , 2

(3.16)

where bup and blow are as deŽned in equation 3.4. An alternative method is to choose bPto minimize Dgap after Žxing w at w (a) . This is equivalent to minimizing i ji2 , which can be done in closed form to get bD

1 X± ai ² . Fi ¡ m i C

(3.17)

4 SMO Algorithm

In this section, we give the SMO algorithm for solving equation 2.10. A basic step consists of starting with a point a and optimizing only two variables,

SMO Algorithm for Least-Squares SVM Formulation

493

ai and aj , to form the new point anew . Consider equations 3.7 and 3.8. Given equation 3.9, the natural choice is i D i up and j D i low so as to make |w 0 (0) | as large as possible. Using the notations of section 3, we can write the optimization problem and the resulting solution as t? D arg max w ( t) t

and

anew D a(t Q ?) .

(4.1)

The function w ( t) is a simple quadratic function of t, w ( t) D w (0) C tw 0 (0) C

t2 00 w (0) , 2

(4.2)

where w 0 is given by equation 3.9, and w 00 (0) D g D 2 kQ ( xi , xj ) ¡ kQ ( xi , xi ) ¡ kQ (xj , xj ) .

(4.3)

Thus, equation 4.1 can be easily solved in closed form: t? D ¡

Fi ¡ Fj w 0 (0) . D¡ 00 w (0) g

(4.4)

The SMO algorithm can now be described. It is very much along the lines of the simpliŽed version of the SMO algorithm for SVMs given by Chang and Lin (2001). SMO Algorithm for Equation 2.10

1. Choose any a0 , and set r D 0.

2. If ar satisŽes equation 3.5, stop. If not, set a D ar , choose i D i up, j D i low, and solve equation 4.1. 3. Let ar C 1 D anew , r :D r C 1 and go back to step 2.

We now establish asymptotic convergence of the algorithm. The absence of hard boundaries in the optimization makes the proof of convergence much simpler than corresponding proofs for SMO algorithm for SVMs. Putting equation 4.4 in equation 4.2 and simplifying, we get f (ar C 1 ) ¡ f (ar ) D w ( t?) ¡ w (0) D ¡

(t?) 2 g. 2

(4.5)

p Using equation 2.4, we see that g < ¡2 / C. Also, |t?| D kar C 1 ¡ ar k / 2. Thus, equation 4.5 implies a sufŽcient increase in f in each basic step of the SMO algorithm: f (ar C 1 ) ¡ f (ar ) ¸

1 kar C 1 ¡ ar k2 . 2C

(4.6)

494

S.S. Keerthi and S.K. Shevade

Because of equation 2.4, ¡ f is a positive deŽnite quadratic form. Hence, the solution of equation 2.10 is unique. Call this point a?. We show the following result. Lemma 1.

far g converges to a?.

Proof. Since the algorithm increases f at each step and f is bounded above,

f f (ar ) g is a convergent sequence. By equation 4.6, we get that far C 1 ¡ ar g converges to 0. Because ¡ f is a positive deŽnite quadratic form, the set fa | f (a) ¸ f (a0g is a compact set. Since far g lies in this set, it is a bounded sequence and therefore has at least one limit point. Let far ( s) gs ¸ 0 denote a convergent subsequence and a N denote the limit point to which it converges. For any r ¸ 0, let i ( r) D i up (ar ) and j ( r) D i low (ar ), the two indices chosen for optimization at the rth step. Since w 0 ( t?) D 0 for t? given by equation 4.1, we get from equation 3.9 that Fi ( r ) (ar C 1 ) ¡ Fj ( r ) (ar C 1 ) D 0.

(4.7)

Since there is only a Žnite number of indices, there exists at least one pair (i1 , j1 ) such that i1 D i (r (s ) ) and j1 D j (r (s ) ) for inŽnitely many s. Let us restrict ourselves to this subsequence. To keep notations simple, we rename the subsequence and take that i1 D i (r (s ) ) D i up (ar ( s ) )

and

j1 D j (r (s ) ) D i low (ar ( s) )

8 s ¸ 0.

Since bup and blow are continuous functions of a, we also get bup (a) N ¡ blow ( a) N D lim [bup (ar ( s) ) ¡ blow (ar ( s) ) ] s!1

( ) ( ) D lim [Fi1 (ar s ) ¡ Fj1 (ar s ) ] s!1

(4.8)

D lim [P(s) C Q ( s ) C R ( s) ], s!1

where P ( s) D [Fi1 (ar ( s) ) ¡ Fi1 (ar ( s) C 1 ) ]

Q ( s) D [Fi1 (ar ( s) C 1 ) ¡ Fj1 (ar ( s) C 1 ) ]

(4.9)

( ) ( ) R ( s) D [Fj1 (ar s C 1 ) ¡ Fj1 (ar s ) ].

Since far ( s) C 1 ¡ ar ( s) g converges to 0, lims!1 P (s ) D 0 and lims!1 R ( s) D 0. By equation 4.7, Q ( s) D 0 8s. Thus, equation 4.8 yields bup (a) N ¡ blow ( a) N D 0. By equation 3.5, a N is a solution of equation 2.10, that is, aN D a?. Strict convexity of ¡ f implies that the sequence far g itself converges to a?. This completes the proof.

SMO Algorithm for Least-Squares SVM Formulation

495

The implementation of the SMO algorithm is extremely simple. A pseudocode is given in the appendix. For efŽciency, a cache is maintained for all Fi ’s. At each basic step, when only two indices, say, i1 and i2, change their old and aold to anew and anew , the values from ai1 Fi ’s can be updated as i2 i1 i2 new old Q new old Q Fnew D Fold i i C (ai1 ¡ ai1 ) k ( xi1 , xi ) C (ai2 ¡ ai2 ) k ( xi2 , xi ) 8i.

(4.10)

Also, the updating of the dual objective function can be written, using equations 4.4 and 4.5, as f new D f old ¡

g ?2 (t ) , 2

(4.11)

new ¡ aold ) old ¡ anew ) . (ai1 where t? D (ai2 i2 D i1 The algorithm can be initialized with a D 0. For this choice, Fi D ¡yi for all i and f (a) D 0. When the regularization parameter C is tuned (say, using k-fold cross validation) we usually come across a situation in which the solution of equation 2.10 is available for some C and 2.10 needs to be solved again for dC where d > 0. In such a situation, we can do alpha seeding (DeCoste & Wagstaff, 2000), that is, use the solution at C to initialize the variables for solving the problem at dC. We give the details below. Suppose a is the solution at C, and f , Fi 8 i are available. To start the algorithm for dC, we can use the same a. For this choice, the Fi for dC can be initialized using

F i :D F i ¡

(d ¡ 1) ai 8i. dC

(4.12)

After the Fi are computed, f can be obtained using f D

1X ai ( yi ¡ Fi ). 2 i

(4.13)

An alternative is to initialize with da for solving the problem at dC. Update formulas similar to equations 4.12 and 4.13 can be derived for this case too. Both initializations above work effectively and give similar performance. 5 Conjugate Gradient Algorithm

Suykens et al. (1999) devised an efŽcient method for obtaining the solution of LS-SVM using the conjugate gradient technique (Golub & Van Loan, 1996). This method is well suited for problems with a large number of examples. Let KQ be the m £m matrix whose (i, j)th element is kQ (xi , xj ). The steps of their method are as follows.2 2 The notations and description of the method given here are different from those given by Suykens et al. (1999).

496

S.S. Keerthi and S.K. Shevade

CG Method

1. Solve Q De Kg

and

Q Dy Kº

(5.1)

for g and º using the conjugate gradient method (twice), where e is a vector with all elements equal to 1 and y is a vector whose ith component is yi . 2. Compute b D ¡g ¢ y /g ¢ e, and set a D º C bg. Although the conjugate gradient algorithm solves a system of m linear equations in m steps, for efŽciency, it is important to stop the algorithm much earlier than m steps. Hamers, Suykens, and De Moor (2001) suggest stopping the algorithm when one of three criteria is satisŽed; for details, see their report. These criteria are not directly related to the solution of the problem in equation 2.2 and hence cannot be used efŽciently to stop the algorithm with a prescribed guarantee on the primal solution, such as satisfying equation 3.14. Below we explain how Dgap can be computed easily in the CG method so that equation 3.14 can be satisŽed by using equation 3.13. Although the linear systems in equation 5.1 are decoupled, for efŽcient use of kernel computations, they are solved together. At any stage of application of the conjugate gradient method to solve them, g and º, which approxQ imately satisfy equation 5.1, are available. The residue vectors, rg D e ¡ Kg Q and rº D y ¡ Kº, play a crucial role in the computations and are also available. Step 2 of the CG method can be used to construct approximate b and a. Using that a, it is possible to compute F (the vector containing the Fi ’s) as Q ¡ y D b ( e ¡ rg ) ¡ rº. F D Ka

(5.2)

Then Dgap can be obtained using equations 3.10 and 3.12. However, there is a difŽculty associated with the use of Dgap computed in this way for ensuring equation 3.14. Recall from section 3 that P to derive equation 3.14, an important requirement is that the constraint i ai D 0 has to be satisŽed. Since g and º are only approximate solutions of equation 5.1, this condition is usually not satisŽed. Fortunately, this issue is easily taken care of if we replace step 2 of the CG method with this alternate implementation: Compute b D ¡º ¢ e /g ¢ e and set a D º C bg.

(5.3)

What we have done is replace g ¢ y with º ¢ e in the calculation of b. Note that when g and º satisfy equation 5.1, we have g ¢ y D º ¢ e. Once step 2 is replaced by equation 5.3, application of equations 5.2, 3.10, 3.12, and 3.13 guarantees equation 3.14.

SMO Algorithm for Least-Squares SVM Formulation

497

Table 1: Computational Costs for SMO and CG Algorithms (a D 0 Initialization).

log 10 C ¡4 ¡3 ¡2 ¡1 0 1 2 3 4

Banana s 2 D 1.8221

Image s 2 D 2.7183

Splice s 2 D 29.9641

Waveform s 2 D 24.5325

SMO

CG

SMO

CG

SMO

CG

SMO

CG

0.90 0.83 0.53 0.50 0.70 3.53 31.58 320.42 3317.90

0.32 0.48 0.64 1.28 2.24 4.00 7.84 14.40 32.32

9.30 7.48 5.17 4.84 7.05 33.65 254.13 1908.62 11825.40

3.38 5.07 8.05 20.28 49.01 131.82 410.67 1275.95 4096.56

4.73 4.35 2.94 3.13 3.07 5.33 11.45 29.88 99.90

1.00 2.00 3.00 6.00 13.00 30.00 65.00 89.00 109.00

0.93 0.91 0.52 0.42 0.54 2.39 10.51 21.62 28.33

0.32 0.48 0.64 1.12 2.40 5.76 14.56 23.36 27.36

Note: Each unit corresponds to 106 kernel evaluations.

As in the SMO algorithm, alpha seeding can be done for the CG method also, as pointed out by Hamers et al. (2001). EfŽcient update formulas for rº, rg , and the dual objective function can be written down similar to equations 4.12 and 4.13.

6 Numerical Experiments on ClassiŽcation Problems

The SMO and CG methods were implemented in Fortran and executed on a Pentium IV 1.5 GHz machine running on Windows platform. The gaussian xk2 ) was used. For both methods we used the kernel K ( x, xN ) D exp(¡ kx¡N 2s 2 termination criterion in equation 3.13 with 2 D 10¡6 . We empirically compared the two methods in terms of computational cost, measured using the number of kernel evaluations since the bulk of the effort in both algorithms resides in computing kernels. Four benchmark data sets were used for this purpose: Banana, Image, Splice, and Waveform. Detailed information about these data sets can be found in Ra¨ tsch (1999). For each data set, we Žxed s at a suitable value that gives good generalization performance and varied C over a wide range. In the Žrst experiment, we tried the following nine C values: 10i , i D ¡4, ¡3, . . . , 3, 4. For each C, the methods were initialized with a D 0. The computational costs associated with the four data sets are given in Table 1 as functions of C. In the second experiment, we tested the effect of alpha seeding on the computational cost. C was sequentially varied from 2 ¡10 to 210 in multiples of 2. Thus, there were 21 C values. For C D 2 ¡10, the methods were ini-

498

S.S. Keerthi and S.K. Shevade

Table 2: Computational Costs for SMO and CG Algorithms with Alpha Seeding. Banana s 2 D 1.8221

Image s 2 D 2.7183

Splice s 2 D 29.9641

Waveform s 2 D 24.5325

log 2 C

SMO

CG

SMO

CG

SMO

CG

SMO

CG

¡10 ¡9 ¡8 ¡7 ¡6 ¡5 ¡4 ¡3 ¡2 ¡1 0 1 2 3 4 5 6 7 8 9 10

0.83 0.62 0.47 0.51 0.44 0.44 0.42 0.42 0.43 0.45 0.60 0.90 1.47 2.61 4.84 9.31 18.42 36.56 72.40 146.95 297.37

0.48 0.32 0.32 0.48 0.64 0.80 0.96 1.12 1.28 1.60 2.08 2.24 2.88 3.04 4.00 5.12 5.44 7.04 8.64 9.76 12.16

7.44 6.15 5.54 4.56 4.69 4.09 3.98 3.79 3.82 4.38 5.83 8.71 13.95 24.16 43.70 81.07 149.06 278.62 518.92 955.57 1717.04

5.07 3.38 5.07 5.07 6.76 10.14 13.52 18.59 21.97 28.73 38.87 54.08 72.67 98.02 138.58 197.73 265.33 378.56 549.25 763.88 1098.50

4.37 3.50 3.34 2.92 2.96 2.75 2.56 2.70 2.40 2.32 2.43 2.74 3.19 3.62 3.91 4.09 4.04 4.17 4.18 4.04 3.77

2.00 1.00 2.00 2.00 3.00 3.00 4.00 5.00 6.00 8.00 10.00 14.00 17.00 23.00 32.00 37.00 52.00 65.00 73.00 82.00 88.00

0.91 0.78 0.70 0.50 0.39 0.40 0.42 0.37 0.38 0.39 0.49 0.73 1.09 1.82 2.99 4.70 7.09 9.92 12.78 14.91 15.83

0.48 0.32 0.32 0.48 0.48 0.64 0.80 0.96 1.12 1.44 1.92 2.56 3.68 4.96 6.40 8.96 11.84 14.72 17.44 20.64 23.36

Note: Each unit corresponds to 106 kernel evaluations.

tialized with a D 0. For all other C values, alpha seeding was used. 3 The computational costs associated with the four data sets are given in Table 2 as functions of C. For the SMO algorithm, the increase in cost at large C values is sharp. See, for instance, the growth in cost of SMO after C D 102 for the Banana and Image data sets. In the case of the Image data set, while SMO was doing quite better than CG at low and medium C values, it became very expensive at large C values. In most practical problems, the optimal value of C where best generalization occurs (say, the value at which the k-fold cross-validation error is smallest) is usually not high. For instance, in the case of Image data set, Žve-fold cross-validation error was smallest around C D 24.16. Thus, when searching for the optimal C, if large C values can be carefully avoided, then the SMO algorithm will work very efŽciently. It is also worth noting that for both SMO and CG methods, the effectiveness of the alpha seeding diminishes at large C values. Since large C values correspond to large costs 3 In section 4 we mentioned two alpha seedings: (1) set a to the optimal a of the previous C value and (2) set a to d times the optimal a of the previous C value. Our experiment showed that both initializations give nearly the same cost.

SMO Algorithm for Least-Squares SVM Formulation

499

Table 3: Ratio of the Cross-Validation Cost of CG and the SMO Algorithm. Banana (s 2 D 1.8221)

Image (s 2 D 2.7183)

Splice (s 2 D 29.9641)

Waveform (s 2 D 24.5325)

1.89

2.62

5.77

1.34

for both methods, such values should be avoided (unless really needed) in the C tuning procedure to maintain efŽciency. On the Image and Splice data sets, SMO was quite faster than CG at the middle C values. Since the “optimal” C usually occurs in this range, this is an advantage in favor of the SMO method. On the Banana and Waveform data sets, the difference between the two methods was not much in this range of C values. When C is tuned using cross validation, we usually start with a low C value, say, 2 ¡10, increase C in steps (say, in multiples of 2), and stop when the cross-validation error clearly shows no further decrease. We compared the CG and the SMO methods with respect to the total cost associated with this process. To do this, we used the same four data sets of Table 2, while also Žxing s 2 at the values given in that table. Fivefold cross validation was employed. Table 3 gives the ratio of the computational efforts associated with the two methods. The SMO method is clearly faster in all cases, so the SMO algorithm proposed here can be viewed as an excellent alternative method for solving LS-SVMs. To see how the SMO algorithm for LS-SVM scales to large-size problems, we did a scaling experiment on the Adult and Web data sets by gradually increasing the training set size in eight steps and observing the training cost. (These data sets are available on-line from Platt’s SMO page: http://www.research.microsoft.com/˜jplatt/smo.html.) The Adult data set was trained with C D 1.0 and s 2 D 10, while the Web data set was trained with C D 5.0 and s 2 D 10. Figures 1 and 2 give plots of the computational costs of the two algorithms as functions of the problem size. The SMO algorithm for LS-SVM scales well on both data sets, with scaling exponents of 2.017 and 2.006, respectively, for the Adult and Web data sets. For the CG algorithm, the corresponding scaling exponents were 2.385 and 2.496. This is due to the fact that the number of conjugate gradient iterations needed to achieve equation 3.13 slowly rises as the number of examples increases. The results indicate that the SMO algorithm is better suited for large problems. Next, we briey compared the generalization performance of SVM and LS-SVM. Fivefold cross validation was used to tune the hyperparameters involved in the problem formulations (that is, C and s), and the test set error was obtained using the optimal hyperparameter values for each formulation. The search for optimal hyperparameters was done on a 20 £ 20 uniform grid in the (log C, log s) space. The fractions of test set errors (TErr) are given in Table 4. It is clear that the generalization capabilities of both

500

S.S. Keerthi and S.K. Shevade

1e+11

N u m b e r o f K e rn e l E v a lu a tio n s

CG SMO 1e+10

1e+09

1e+08

1e+07 1000

10000 Training set size

Figure 1: Variation of computational cost with training set size for the Adult data set. Table 4: Generalization Performances of LS-SVM and SVM on Benchmark Data Sets. Data Set

LS-SVM

SVM

Banana Image Splice Waveform

.1161 .0208 .1012 .1089

.1180 .0228 .0989 .1076

methods are comparable. This observation is consistent with that made by Van Gestel et al. (2000). 7 Solution of KFD

Mika, Ra¨ tsch, Weston, Sch¨olkopf, and Muller ¨ (1999) Žrst proposed the idea of (regularized) KFD. Mika et al. (2001) gave a quadratic programming

SMO Algorithm for Least-Squares SVM Formulation

501

1e+11

N u m b e r o f K e r n e l E v a lu a tio n s

CG SMO 1e+10

1e+09

1e+08

1000

10000 Training set size

Figure 2: Variation of computational cost with training set size for the Web data set.

formulation of KFD employing general regularizers. For the case of Kregularizer, that is, kwk2 , Van Gestel et al. (2002) established the equivalence 4 between LS-SVM and KFD. In this section, we give an alternative to their way of establishing the equivalence between the KFD and LS-SVM formulations. Our approach clearly brings out the relationship between the dual variables and the threshold parameter associated with the two formulations. The quadratic programming formulation of Mika et al. (2001) is: min s.t.

1 O 2 C C 2 kwk 2

P

O2 i ji

yi ¡ (wO ¢ zi ¡ bO ) D jOi 8i P P jOi D 0 and i2IC

(7.1) O i2I¡ ji D 0,

4 By equivalence, we mean that the normal directions of the optimal hyperplanes associated with the two formulations are the same.

502

S.S. Keerthi and S.K. Shevade

where I C and I¡ are the index sets corresponding to the two classes and, as usual, yi takes the value 1 if example i is in class 1 and ¡1 otherwise. The Lagrangian for this problem is 1 C X O2 X O 2C j C a O i [yi ¡ ( wO ¢ zi ¡ bO ) ¡ jOi ] LO D kwk 2 2 i i i X X C mC jOi C m ¡ jOi , i2IC

i2I¡

where the a O i ’s, m C and m ¡ are Lagrange multipliers. Setting the gradients O O bO and jOi ’s to zero and using them together with the of L with respect to w, equalities in equation 7.1 yields the following square system of equations O m C , and m ¡ : for solving the a O i ’s, b, P P i

P

aO i D 0,

j

Oj C Kij a

i2IC

1 a O C i

m D bO C Ci C yi 8 i

aO i D m C m C

and

P

i2I¡

aO i D m¡m ¡ ,

(7.2)

where m i is m C if yi D 1 and m ¡ if yi D ¡1; m C is the number of examples in class 1 (positive class); m ¡ is the number of examples in class 2 (negative class); and m D m C C m ¡. These equations imply that m C m C C m¡m ¡ D 0. Let us deŽne m using the equation m D m C m C D ¡m¡m ¡ .

(7.3)

If we deŽne aD 1C

m 2C

³

1 1 C m C m¡

´ and

bD

µ ³ ´¶ 1 O m 1 1 ¡ bC , 2C m C m ¡ a

(7.4)

then the right-hand side of each of the Žrst-level equations in equation 7.2, that is, bO C mCi C yi , can be rewritten as a ( b C yi ) . If we also deŽne ai D

a Oi a

8i,

(7.5)

it is easy to see that the ai and b are obtained by solving P j

Kij aj C C1 ai ¡ b D yi 8 i P i ai D 0.

(7.6)

Using the ai ’s, we can get m by solving X i2IC

µ ai D m / a D m /

m 1C 2C

³

1 1 C mC m¡

´¶ .

(7.7)

SMO Algorithm for Least-Squares SVM Formulation

503

O and the a Then equations 7.3, 7.4, and 7.5 yield m C , m ¡ , a, b, O i ’s. Since equation 7.6 is nothing but the solution of the instance of equation 2.2 corresponding to LS-SVM, it follows that the solution of equation 7.1 can be obtained by solving LS-SVM; this can be numerically done using the SMO algorithm of section 4. The two equality constraints in the last row of equation 7.1 contribute only to a scaling of w and a shift in b given by the LS-SVM solution. Thus, if b and bO associated with equations 2.2 and 7.1 are determined in a uniform way (say, using the class posterior probabilities obtained in a post-processing step; Platt, 1999), then the classiŽers obtained by using these two formulations are identical. 8 Primal Problem Without b

In kernel ridge regression (Saunders et al., 1998) and Bayesian designs (Van Gestel et al., 2002), it is usual to consider the primal problem, equation 2.2, P without the b parameter.5 When b is absent, the equality constraint i ai D 0 disappears from the dual, equation 2.10, and the optimality conditions for the dual problem become Fi D 0 8i.

(8.1)

Because of the absence of the equality constraint, the SMO algorithm can simply modify one ai at a time. If the indices are chosen sequentially for updating, then the SMO algorithm is precisely the SOR algorithm (Bertsekas, 1999; Hamers et al., 2001). The efŽciency of this SOR algorithm can be signiŽcantly improved if we modify it so that at any given instance, the index that gives the best increase in f value is chosen. This can be neatly done by maintaining a cache for the Fi ’s. Like the sequential SOR algorithm, the modiŽed algorithm also requires only O ( m ) computations and kernel evaluations in each step. However, the modiŽed algorithm requires many fewer steps to achieve the termination criterion. Let us now work out the details of the modiŽed algorithm. Suppose at a given a, an index i is considered for updating. DeŽne a Q i ( t) D ai C tI

6 iI a Q j ( t) D aj 8j D

and

w ( t) D f (a(t) Q ).

(8.2)

With t? and anew as deŽned in equation 4.1, it is easy to check that t? D

Fi kQ ( xi , xi )

and

f (anew ) ¡ f (a) D

F2i 2 kQ (xi , xi )

.

(8.3)

5 If b is present and the term c b2 / 2 is added to the cost function in equation 2.2, that is equivalent to a primal problem without b if the constant 1 /c is added to the kernel function k.

504

S.S. Keerthi and S.K. Shevade

Thus, at each step of the SMO algorithm, we can choose i that has the F2 largest value of Q i . Since a cache is maintained for the Fi ’s and kQ ( xi , xi ) , 2 k ( xi ,xi )

i D 1, . . . , m can be precomputed and stored, the above step can be efŽciently done. Convergence of the resulting SMO algorithm can be established along the lines given in section 4. The proofs actually become much simpler because of the simpler problem and algorithm associated with the absence of b. The solution of the problem in equation 2.10 without the equality conQ D y, so the solution can also be straint corresponds to the solution of Ka obtained directly using the conjugate gradient method. Duality gap ideas discussed in section 3 can be easily adapted (simply omit b in all calculations) to give efŽcient stopping criteria for both SMO and CG methods. 9 Conclusion

We have given a new algorithm for least-squares formulations of SVMs (LSSVMs and kernel ridge regression), proved its convergence, and discussed implementation aspects. Our algorithm also applies to a particular form of regularized kernel Fisher discriminant method. The algorithm is robust in the sense that on the many complex data sets we have tried, there was not a single case of failure. It is also fast and scales well to large-size problems. It provides a better alternative to the conjugate gradient algorithm developed by Hamers et al. (2001), especially for large problems. Appendix: Pseudocode for the SMO Algorithm

Notes for the Pseudocode 1. kernel(i,j) refers to kQ ( xi , xj ) given by equation 2.4. 2. The ai are stored in alpha array, while the Fi are stored in Fcache array. 3. 2 in equation 3.13 is referred to as epsilon . 10¡6 is a suitable value for it. 4. eps is used to check if sufŽcient change has occurred in each step. It can be set to 10¡12. 5. The function fabs returns the absolute value of a real number. 6. Any line in the pseudocode that begins with the character % is treated as a comment.

procedure takeStep(i1, i2) alph1 = alpha{i1}; alph2 = alpha{i2}; y1 = y{i1}; y2 = y{i2}; F1 = Fcache{i1}; F2 = Fcache{i2}; gamma = alph1 + alph2; k11 = kernel(i1,i1); k22 = kernel(i2,i2); k12 = kernel(i1,i2);

SMO Algorithm for Least-Squares SVM Formulation

505

eta = 2*k12-k11-k22; a2 = alph2 - (F1-F2)/eta; if (fabs(a2-alph2) < eps*(a2+alpha2+eps)) return 0; a1 = gamma-a2; alpha{i1} = a1; alpha{i2} = a2; Update the DualObjectiveFunction using (4.11). Update Fcache{i} for all i in I using (4.10). Compute (iÇ low, bÇ low) and (iÇ up, bÇ up) using (3.4). return 1; endprocedure procedure ComputeDualityGap() DualityGap = 0; set b using (3.16) or (3.17). loop over all i xi = b + (alpha{i}/C) - Fcache{i}; DualityGap += alpha{i}(Fcache{i}-(0.5*alpha{i}/C)) + C*xi*xi/2; return DualityGap; endprocedure procedure InitializeVariables(delta) if delta is non-zero % delta denotes the factor by which C is changed. for every i, reset Fcache{i} using (4.12). compute DualObjectiveFunction using (4.13). else initialize alpha{i}=0 for all i. set Fcache{i}=-y{i} for all i and DualObjectiveFunction to 0. Compute (iÇ low, bÇ low) and (iÇ up, bÇ up) using (3.4). endprocedure procedure SMO() % If solution is available at some value of C=C1 % and solution is to be % found at C=C2 then set delta := C2/C1. Otherwise, % set delta := 0. InitializeVariables(delta); numChanged = 1; DualityGap = ComputeDualityGap(); while (DualityGap > epsilon*DualObjectiveFunction && numChanged != 0) numChanged = takeStep(iÇ up, iÇ low); DualityGap = ComputeDualityGap(); % If the procedure stops with numChanged=0, it means that % epsilon is too small. endprocedure

506

S.S. Keerthi and S.K. Shevade

References Bertsekas, D. P. (1999). Nonlinear programming (2nd ed.). Belmont, MA: Athena ScientiŽc. Chang, C. C., & Lin, C. J. (2001). LIBSVM: A library for support vector machines. Available on-line at: http://www.csie.ntu.edu.tw/»cjlin/libsvm. DeCoste, D., & Wagstaff, K. (2000).Alpha seeding for support vector machines. Paper presented at the International Conference on Knowledge Discovery and Data Mining, Boston. Fletcher, R. (1987). Practical methods of optimizations (2nd ed.). New York: Wiley. Golub, G. H., & Van Loan, C. F. (1996). Matrix computations (3rd ed.). Baltimore: Johns Hopkins University Press. Hamers, B., Suykens, J., & De Moor, B. (2001). A comparison of iterative methods for least squares support vector machine classiŽers (Internal Rep. 01-110). Leuven, Belgium: ESAT-SISTA, K. U. Leuven. Keerthi, S. S., Shevade, S. K., Bhattacharyya, C., & Murthy, K. R. K. (2001). Improvements to Platt’s SMO algorithm for SVM design. Neural Computation, 13, 637–649. Mika, S., R¨atsch, G., & Muller, ¨ K. R. (2001). A mathematical programming approach to the kernel Fisher algorithm. In T. K. Leen, T. G. Dietterich, & V. Tresp (Eds.), Advances in neural informationprocessingsystems,13 (pp. 591–597).Cambridge, MA: MIT Press. Mika, S., R¨atsch, G., Weston, J., Sch¨olkopf, B., & Muller, ¨ K. R. (1999). Fisher discriminant analysis with kernels. In Y. H. Hu, J. Larsen, E. Wilson, & S. Douglas (Eds.), Neural networks for signal processing IX (pp. 41–48). New York: IEEE. Platt, J. (1998). Sequential minimal optimization: A fast algorithm for training support vector machines (Tech. Rep. MSR-TR-98-14). Redmond, WA: Microsoft Research. Platt, J. (1999). Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In A. Smola, B. Sch¨olkopf, & D. Schuurmans (Eds.), Advances in large margin classiŽers. Cambridge, MA: MIT Press. R¨atsch, G. (1999). [Benchmark repository.] Available on-line at: http://ida.Žrst. gmd.de/»raetsch/data/benchmarks.htm. Saunders, C., Gammerman, A., & Vovk, V. (1998). Ridge regression learning algorithm in dual variables. In Proceedingsof the Fifteenth InternationalConference on Machine Learning (ICML-98). Madison, WI. Smola, A., & Sch¨olkopf, B. (1998) A tutorial on support vector regression (NeuroColt2 Tech. Rep. NC2-TR-1998-030).Berlin: ESPRIT Working Group in Neural and Computational Learning II. Suykens, J., Lukas, L., Van Dooren, P., De Moor, B., & Vandewalle, J. (1999). Least squares support vector machine classiŽers: A large scale algorithm. In Proceedings of the European Conference on Circuit Theory and Design (ECCTD’99) (pp. 839–842). Stresa, Italy. Suykens, J., & Vandewalle, J. (1999). Least squares support vector machine classiŽers. Neural Processing Letters, 9, 293–300.

SMO Algorithm for Least-Squares SVM Formulation

507

Vanderbei, R. J. (1994). LOQO: An interior point code for quadratic programming (Tech. Rep. SOR-94-15). Princeton, NJ: Statistics and Operations Research, Princeton University. Van Gestel, T., Suykens, J., Baesens, B., Viaene, S., Vanthienen, J., Dedene, G., De Moor, B., & Vandewalle, J. (2000). Benchmarking least squares support vector machine classiŽers (Internal Rep. 00-37). Leuven, Belgium: ESAT-SISTA, K. U. Leuven. Van Gestel, T., Suykens, J., Lanckriet, G., Lambrechts, A., De Moor, B., & Vandewalle, J. (2002). A Bayesian framework for least squares support vector machine classiŽers. Neural Computation, 14, 1115–1147. Received May 9, 2002; accepted July 8, 2002.

ARTICLE

Communicated by Bard Ermentrout

Synchronization in Networks of Excitatory and Inhibitory Neurons with Sparse, Random Connectivity Christoph Borgers ¨ [email protected] Department of Mathematics, Tufts University, Medford, MA 02155, U.S.A.

Nancy Kopell [email protected] Department of Mathematics and Center for BioDynamics, Boston University, Boston, MA 02215, U.S.A.

In model networks of E-cells and I-cells (excitatory and inhibitory neurons, respectively), synchronous rhythmic spiking often comes about from the interplay between the two cell groups: the E-cells synchronize the I-cells and vice versa. Under ideal conditions—homogeneity in relevant network parameters and all-to-all connectivity, for instance—this mechanism can yield perfect synchronization. We Žnd that approximate, imperfect synchronization is possible even with very sparse, random connectivity. The crucial quantity is the expected number of inputs per cell. As long as it is large enough (more precisely, as long as the variance of the total number of synaptic inputs per cell is small enough), tight synchronization is possible. The desynchronizing effect of random connectivity can be reduced by strengthening the E!I synapses. More surprising, it cannot be reduced by strengthening the I!E synapses. However, the decay time constant of inhibition plays an important role. Faster decay yields tighter synchrony. In particular, in models in which the inhibitory synapses are assumed to be instantaneous, the effects of sparse, random connectivity cannot be seen. 1 Introduction In networks of E-cells (excitatory neurons) and I-cells (inhibitory neurons), synchronous, rhythmic spiking often results from the interplay between the two cell groups, with the E-cells synchronizing the I-cells and vice versa. This mechanism, called PING (pyramidal interneuronal network gamma; Whittington, Traub, Kopell, Ermentrout, & Buhl, 2000) or ° -II (Tiesinga, Fellous, Jos´e, & Sejnowski, 2001), has been observed in modeling studies, and there are reasons to believe that some experimentally observed gamma rhythms are in fact based on this mechanism (Traub, Jefferys, & Whittington, 1999; Whittington et al., 2000; Tiesinga et al., 2001). Neural Computation 15, 509–538 (2003)

c 2003 Massachusetts Institute of Technology °

510

C. Borgers ¨ and N. Kopell

A

B E­ cells

400

E­ cells

400 200

200 0 100

I­ cells

I­ cells

0 100 50 0 0

100 t

200

50 0 0

100 t

200

Figure 1: PING in E-I networks with (A) all-to-all connectivity and (B) sparse connectivity.

The conditions under which PING occurs are not completely understood, though we discuss them heuristically in section 6. We focus here on the effects of random connectivity on PING. This effect is illustrated by Figure 1. Details about this Žgure will be given later. For now, it sufŽces to say that Figure 1A shows the emergence of PING in a model E=I network with all-to-all connectivity and without any kind of heterogeneity in network parameters. The Žgure indicates spike times. Both cell groups (E and I) synchronize tightly. If the connectivity is made sparse and random, Figure 1A turns into Figure 1B. The two cell groups now Žre spike volleys of brief but positive duration. (The spike volleys of the I-cells are so brief that they appear to have zero duration in the plot.) The main goal of this article is to analyze the durations and shapes of these volleys and their dependence on network parameters. Synchronization in the presence of random connectivity has been studied previously for excitatory networks (Barkai, Kanter, & Sompolinsky, 1990), inhibitory networks (Brunel & Hakim, 1999; Wang & Buzs a´ ki, 1996), and E=I networks (Brunel, 2000; Bush & Sejnowski, 1996; Golomb & Hansel, 2000; Hansel & Mato, 2001; van Vreeswijk & Sompolinsky, 1996, 1998; Wang, Golomb, & Rinzel, 1995). This literature is aimed at understanding either the stability of the asynchronous state or transitions from asynchrony to rhythms and vice versa. We do not consider these issues here, but focus instead on a detailed understanding of the near-synchronous state. In section 2, we review the theta model (Ermentrout & Kopell, 1986; Gutkin & Ermentrout, 1998; Hoppensteadt & Izhikevich, 1997), an idealization of a large class of conductance-based neuronal models. Our arguments and simulations in this article are based on this model. We also introduce

Synchronization in Sparse, Random Networks

511

our model of synapses in section 2 and describe the connectivity of our model networks. In section 3, we present numerical experiments demonstrating that the desynchronizing effect of sparseness and randomness in the connectivity primarily originates from the variance in the number of inputs per cell, not from the randomness per se. The synchronization of a population of cells by an inhibitory input pulse is analyzed in section 4, Žrst assuming that all cells receive the same input pulse, and then, motivated by the result of section 3, assuming that different cells receive input pulses of different strengths. Similarly, the synchronization of a population of cells by an excitatory input pulse is analyzed in section 5, Žrst assuming that all cells receive the same input pulse, then assuming that different cells receive input pulses of different strengths. The results of sections 4 and 5 are combined in section 6 to analyze PING in E=I networks. In section 7, we summarize our results and put them into the context of other recent work on the same subject. 2 Review of Theta Neurons 2.1 Equation of a Single Theta Neuron. In the Hodgkin-Huxley model, a periodically spiking space-clamped neuron is represented by a point moving on a limit cycle in a four-dimensional phase space. Analogously, in the theta model (Ermentrout & Kopell, 1986; Gutkin & Ermentrout, 1998; Hoppensteadt & Izhikevich, 1997), a neuron is represented by a point P D .cos µ; sin µ / moving on the unit circle S1 . In the absence of synaptic coupling, the differential equation governing the motion is 1 dµ D .1 ¡ cos µ/ C I.1 C cos µ/: dt ¿

(2.1)

Here, I should be thought of as an input “current,” measured in radians per unit time. The time constant ¿ > 0 is needed to make equation 2.1 dimensionally correct; it is analogous to a membrane time constant. When I < 0, equation 2.1 has the two Žxed points: µ0§ D §2 arccos p

1 1 ¡ ¿I

:

µ0¡ is stable, and µ0C is unstable. For I < 0, the vector Želd on the circle is shown in the left panel of Figure 2A. If µ is perturbed slightly from µ0¡ , it returns to µ0¡ . However, if µ is raised beyond µ0C , a large excursion occurs, with the point P moving around the entire circle while µ increases to µ0¡ C2¼. The stable Žxed point µ0¡ is the analog of the stable equilibrium of a neuron. The unstable Žxed point µ0C is the analog of a spiking threshold. As I approaches 0, the Žxed points approach each other. A saddle-node bifurcation occurs when I D 0. The two Žxed points come together at µ D 0 (see the center panel of Figure 2A). When I > 0, dµ=dt > 0 for all t, so there is no Žxed point (see the right panel of Figure 2A).

512

C. Borgers ¨ and N. Kopell

A:

B:

sin q

1 0

­1 0

20

40

t

60

80

100

Figure 2: Theta model. (A) Vector Želd on the circle for I < 0, I D 0, and I > 0. (B) Sin µ as a function of time.

The transition from I < 0 to I > 0 is the analog of the transition from excitability to spiking in a neuron. Neuronal models are called of type I if this transition involves a saddle-node bifurcation on a limit cycle and of type II if it involves a subcritical Hopf bifurcation (Ermentrout, 1996; Gutkin & Ermentrout, 1998; Rinzel & Ermentrout, 1998). This classiŽcation goes back to Hodgkin (1948). Thus, the theta model is a type I neuronal model. It has been shown to be canonical, in the sense that other type I models can be reduced to it by coordinate transformations (Ermentrout & Kopell, 1986; Hoppensteadt & Izhikevich, 1997). If 0 < I ¿ 1=¿ , the motion is much slower near µ D 0, that is, near the “ghost” of the Žxed point annihilated in the saddle-node bifurcation, than elswhere. This is illustrated in Figure 2B, which shows sin µ as a function of t for ¿ D 1, I D 0:02. As the point P D .cos µ; sin µ/ moves slowly past .1; 0/, sin µ changes slowly. As P moves rapidly around the circle, sin µ rapidly rises to 1, then falls to ¡1, then returns to values slightly below 0. To some extent the graph of sin µ resembles the voltage trace of a spiking neuron.1 1 Some authors think of ¡ cos µ , not sin µ , as the “voltage-like” quantity in the theta model. Which of the two we call voltage-like is of no consequence for this article, and neither is voltage-like in any precise sense. However, we prefer to think of sin µ as the voltage-like quantity for the following aesthetic reason. Consider a theta neuron driven slightly above threshold. Near the ghost of the equilibrium point (µ D 0), sin µ is slowly increasing, while ¡ cos µ has a local minimum. On the other hand, for a Hodgkin-Huxleytype neuron of type I driven slightly above threshold, the membrane potential V is slowly

Synchronization in Sparse, Random Networks

513

When µ crosses .2l ¡ 1/¼ , l integer, with dµ=dt > 0, we say that the neuron spikes. If I > 0 and ¡¼ · µ1 · µ2 · ¼, the time it takes for µ to rise from µ1 to µ2 equals Z

µ2 µ1

dµ D .1 ¡ cos µ/=¿ C I.1 C cos µ/

r µ ¶ tan.µ =2/ µ2 ¿ arctan p : I ¿I µ1

(2.2)

Setting µ1 D ¡¼ and µ2 D ¼ in this formula, we Žnd that the period equals r

PD¼

¿ : I

(2.3)

We denote the time it takes for µ to rise from ¼=2 to 3¼=2 by W and call it the spike width. Applying formula 2.2 with .µ1 ; µ2 / D .¼=2; ¼ / and .µ1 ; µ2 / D .¡¼; ¡¼=2/ and adding the results, we Žnd µ ¶r 1 ¿ W D ¼ ¡ 2 arctan p : I ¿I

(2.4)

For physiological realism, we wish to ensure W=P ¿ 1. By equations 2.3 and 2.4, 2 1 W D 1 ¡ arctan p : P ¼ ¿I Therefore, W=P ¿ 1 means the same as ¿ I ¿ 1. Since arctan.1=²/ D ¼=2 ¡ ² C O.² 2 / as ² ! 0, equation 2.4 implies W ¼ 2¿ when ¿ I ¿ 1. Thus, in the parameter regime of interest to us, ¿ is approximately half the spike width. Motivated by this discussion and by the fact that spike widths in real neurons are on the order of milliseconds, we set ¿ D1 for the remainder of this article, think of time as measured in milliseconds, and always consider input currents I ¿ 1. 2.2 Synapses Between Theta Neurons. We model synapses by adding time-dependent input currents to equation 2.1. When the presynaptic neuron spikes, the postsynaptic neuron receives an input current (positive or negative, depending on whether the synapse is excitatory or inhibitory), which jumps to its maximum value instantaneously and then decays exponentially. This is the model of section 2.1.5 of Izhikevich (2000) (see also Ermentrout, 1996, for a discussion of synapses between theta neurons). increasing near the ghost of the equilibrium point. In this sense, V is more similar to sin µ than to ¡ cos µ .

514

C. Borgers ¨ and N. Kopell

Thus, a network of N coupled theta neurons is described by a system of differential equations of the form Á ! N X dµj D 1 ¡ cos µj C Ij C ®i gij sij .1 C cos µj /; 1 · j · N: dt iD1 Ij denotes the external input to the jth neuron, which can be positive or negative. The constant ®i equals C1 or ¡1, depending on whether neuron i is excitatory (E) or inhibitory (I). The constant gij ¸ 0 measures the strength of the synapse from neuron i to neuron j, and sij D sij .t/ is the synaptic gating variable associated with this synapse. The value of sij always lies between 0 and 1. It jumps to 1 when neuron i spikes. Between spikes of neuron i, it decays exponentially, following the differential equation dsij sij D¡ : dt ¿ij The decay time constants ¿ij are positive. The jumps of sij occurring when neuron i spikes cause difŽculties in the numerical simulation of the network and are not physiologically realistic. In our simulations, we therefore replace the jumps by rapid but smooth rises, letting sij be governed by a differential equation of the form dsij dt

D¡

sij ¿ij

C e¡´.1Ccos µi /

1 ¡ sij ¿R

;

with ¿R D 0:1 and ´ D 5. The term e¡´.1Ccos µi / .1 ¡ sij /=¿R is very close to zero unless µi ¼ .2l ¡ 1/¼ , l integer, and drives sij toward 1 rapidly when µi ¼ .2l ¡ 1/¼ . The parameter ¿R is reminiscent of a synaptic rise time. Figure 3 shows sij for ¿ij D 2 and 10, with the input of the presynaptic neuron chosen so that its period equals 25. 2.3 Sparse, Random E=I Networks. Throughout this article, the decay time constants ¿ij are assumed to depend on the type of i only (excitatory or inhibitory), so there are two distinct values of ¿ij , denoted ¿E and ¿I . We also assume that all excitatory neurons receive the same constant external input drive IE , and all inhibitory neurons receive the same constant external drive II . In later sections, we consider networks of coupled theta neurons, including NE D 4N=5 excitatory and NI D N=5 inhibitory neurons. These proportions are motivated by the fact that in large portions of the cortex, there are about four times more excitatory than inhibitory neurons (Braitenberg & Schuz, ¨ 1998). The strengths gij of the synapses are chosen at random. For a given network, they are chosen once and for all; they do not depend on time. To de-

Synchronization in Sparse, Random Networks

515

1

2:

0.5

s

t

0 0

20

40

20

40

60

80

100

60

80

100

1

10 :

0.5

s

t

0 0

t

Figure 3: Synaptic gating variable s as a function of time, with decay time constants ¿ D 2 and ¿ D 10.

scribe the choice of the gij , the following notation is useful. Let E µ f1; : : : ; Ng denote the set of all indices of excitatory neurons, and similarly I the set of all indices of inhibitory neurons. For i 2 E and j 2 I , we deŽne gij D

gEI wij ; pEI NE

(2.5)

with ( wij D

1 0

with probability pEI , otherwise,

(2.6)

where gEI ¸ 0 and pEI 2 .0; 1] are constants. For j 2 I , we deŽne wEj D

X

(2.7)

wij :

i2E

Note that wEj is a binomially distributed random variable. We also deŽne gEj D

X i2E

gij D

gEI wEj : pEI NE

(2.8)

516

C. Borgers ¨ and N. Kopell

From the formulas for the mean and standard deviation of binomially distributed random variables, we see that gEj has mean gEI and standard deviation s 1 ¡ pEI (2.9) ¾EI D gEI : pEI NE By the central limit theorem, gEj is approximately normally distributed if pEI NE is large. Assuming pEI ¿ 1, as is physiologically realistic (Braitenberg & Schuz, ¨ 1998), equation 2.9 shows that 1 ¾ EI ¼p : gEI pEI NE

(2.10)

The left-hand side of equation 2.10 is the coefŽcient of variation (the standard deviation divided by the mean) of gEj , j 2 I . The expression pEI NE appearing on the right-hand side is the expected number of excitatory inputs per I-cell. Formulas analogous to 2.5 through 2.10 apply to the I ! E, E ! E, and I ! I synapses. In particular, s

¾IE D gIE

1 ¡ pIE ; pIE NI

(2.11)

and therefore 1 ¾ IE ¼ p gIE pIE NI

(2.12)

for pIE ¿ 1. The left-hand side of equation 2.12 is the coefŽcient of variation of gIj , j 2 E . The expression pIE NI appearing on the right-hand side is the expected number of inhibitory inputs per E-cell. Several authors have pointed out that pEI NE and pIE NI are much more important than pEI , NE , pIE , and NI in isolation (Golomb & Hansel, 2000; Tiesinga, Fellous, Jos´e, & Sejnowski, 2002; Wang et al., 1995). 3 Loss of Tight Synchrony Is Attributable to Variance in the Number of Inputs per Cell Before considering networks with sparse, random connectivity, we consider one with all-to-all connectivity (pEI D pIE D 1:0). Figure 1A shows an example of a simulation with IE D 0:1; II D 0; gEI D gIE D 0:25; gEE D gII D 0; ¿E D 2; ¿I D 10:

(3.1)

Synchronization in Sparse, Random Networks

517

Figure 1A shows spike times, with the horizontal axis indicating time and the vertical axis cell index. Each of the two cell groups (E and I) synchronizes very rapidly, with the synchronous population spikes of the I-cells slightly lagging behind those of the E-cells. The synchronization mechanism seen in Figure 1A is PING, briey described in section 1 and discussed in more detail in section 6.1. We comment briey on our parameter choices. A neuron driven with p IE D 0:1 spikes periodically with an interspike interval equal to ¼= I ¼ 9:93. Since we think of time as measured in milliseconds, this corresponds to a frequency of .1000=9:93/ H ¼ 100 Hz. Thus, the E-cells are driven so hard that they would spike above gamma frequency if they were not subject to any inhibition. The I-cells are driven at threshold; they do not spike without additional excitatory input, but any excitatory input, regardless how weak, will make them spike. The values of gEI and gIE can be varied considerably without any qualitative change in Figure 1A. However, for small values of gEI (roughly < 0:1), two or more population spikes of the E-cells occur before the I-cells respond. For large values of gEI (roughly > 0:7), two or more population spikes of the I-cells occur in response to a population spike of the E-cells. For small values of gIE (roughly < 0:1), the E-cells are not synchronized. For large values of gIE , the rhythm is slow but not qualitatively different from that in Figure 1A. For simplicity, we assume here that there are no E ! E or I ! I synapses— gEE D gII D 0. In our experience, E ! E synapses (with a brief synaptic decay time such as ¿E D 2) do not affect PING rhythms much. However, I ! I synapses are crucial in some parameter regimes; this will be discussed in section 6.1. Our choices of ¿E D 2 and ¿I D 10 are motivated by the decay time constants of excitatory synapses involving AMPA receptors (approximately 2 ms) and inhibitory synapses involving GABAA receptors (approximately 10 ms); recall that we think of time as measured in milliseconds. When pEI and pIE are reduced from 1:0 to 0:5, Figure 1A turns into Figure 1B. The two cell groups now Žre spike volleys of positive durations. As stated in section 1, the main goal of this article is to understand the durations and shapes of these volleys. In the network underlying Figure 1B, each E-cell receives input from a random number of I-cells. The expected value of this number is 50. Similarly, each I-cell receives input from a random number of E-cells, with expectation 200. We now repeat the simulation of Figure 1B with a network in which the connectivity is still sparse and random, but the variance in the number of inputs per cell has been eliminated. That is, each E-cell receives input from a random set of exactly 50 I-cells, and each I-cell receives input from a random set of exactly 200 E-cells. The network is like that of Figure 1B in all other regards. The result is shown in Figure 4A; tight synchrony is restored. When the number of inputs per cell is Žxed, tight synchronization is possible even for much sparser networks. Figure 4B shows a simulation

518

C. Borgers ¨ and N. Kopell

A

B E­ cells

400

E­ cells

400 200

200 0 100

I­ cells

I­ cells

0 100 50 0 0

100 t

50 0 0

200

200

D 400

E­ cells

E­ cells

C

100 t

1000

200

I­ cells

0 100

I­ cells

0 400 200 0 0

100 t

200

50 0 0

100 t

200

Figure 4: Simulation of E-I networks with sparse, random connectivity but without variance in the number of inputs per cell. (A) Parameters as in Figure 1B, but with variance in numbers of inputs per cell eliminated. (B) Only four excitatory inputs per I-cell and one inhibitory input per E-cell. (C) Same as B in a larger network. (D) Same as B, but with only one excitatory input per I-cell.

in which each E-cell receives input from one I-cell, and each I-cell receives input from four E-cells. Although synchronization is achieved a little less rapidly than in Figure 4A, it becomes tight within a few oscillation periods. A four times larger network, again with one inhibitory input into each Ecell and four excitatory inputs into each I-cell, shows similar behavior (see Figure 4C).

Synchronization in Sparse, Random Networks

519

If the number of excitatory inputs per I-cell is reduced again, from four to one, there appears to be no more synchronization (see Figure 4D). Thus, there appears to be a minimum number of inputs per cell needed for synchronization—a “percolation threshold.” However, this number is very small in our simulations. In previous work on different models, similar thresholds have been found (Golomb & Hansel, 2000; Wang & Buzs a´ ki, 1996; Wang et al., 1995). In all three of these references, asynchrony was found to become unstable when the number of inputs per cell exceeded a threshold value independent of network size. A cell in a human brain typically receives input from thousands of other cells (Braitenberg & Schuz, ¨ 1998). Our numerical experiments suggest that with so many synapses, the impact of sparse, random connectivity on synchronization is attributable to the variance in the number of inputs per cell, not to the percolation threshold. We will assume this to be the case from now on. 4 Synchronization of a Population of Theta Neurons by a Single Strong Inhibitory Pulse 4.1 Synchronization by an Inhibitory Pulse of Uniform Strength. We consider a population of N identical, uncoupled neurons with common constant external drive above threshold, receiving a common inhibitory synaptic input pulse at time 0. Following section 2, we model this population by the equations dµj D .1 ¡ cos µj / C .I ¡ gs.t//.1 C cos µj /; dt

1 · j · N;

(4.1)

with I > 0; g > 0, and ( e¡t=¿I if t > 0 s.t/ D 0 if t · 0 with ¿I > 0. If the inhibitory pulse is strong, it brings the population close to synchrony. An example is shown in Figure 5A, which displays results of a simulation with N D 100;

I D 0:05;

g D 0:25;

¿I D 10:

(4.2)

The synchronization brought about by the inhibitory pulse at time 0 is immediate and nearly perfect. To understand the synchronization shown in Figure 5A, consider the initial value problem, dµ D 1 ¡ cos µ C .I ¡ ge¡t=¿I /.1 C cos µ/; dt µ.0/ D µ0 ;

t > 0;

(4.3) (4.4)

520

C. Borgers ¨ and N. Kopell

Figure 5: Synchronization by a single inhibitory pulse. (A) Inhibitory pulse of uniform strength. (B) Inhibitory pulse of nonuniform strength. (C) Phase portrait for equations 4.5 and 4.6 with I D 0:05, ¿I D 10, with the stable river indicated boldly. (D) Distribution of spikes within the Žrst volley following t D 0, in a simulation identical to that in B, but with 1000 neurons: predicted (solid line) and actual (bars).

with ¡¼ < µ0 < ¼ . We deŽne J.t/ D I ¡ ge¡t=¿I : Equations 4.3 and 4.4 can then be rewritten in the form dµ D 1 ¡ cos µ C J.1 C cos µ/ dt dJ J¡I D¡ dt ¿I µ .0/ D µ0

J.0/ D I ¡ g:

(4.5) (4.6) (4.7) (4.8)

Synchronization in Sparse, Random Networks

521

The phase portrait for the two-dimensional dynamical system 4.5 and 4.6 is shown in Figure 5C for I D 0:05 and ¿I D 10. The dashed line in Figure 5C is the nullcline dµ=dt D 0. The nullcline dJ=dt D 0 is the horizontal line J D I, the upper edge of the window shown in Figure 5C. The Žgure should be extended periodically in µ with period 2¼. The ow is upward, in the direction of increasing J. The most striking feature of Figure 5C is the existence of strongly attracting and strongly repelling trajectories. Trajectories of this kind exist in many systems of ordinary differential equations and are called rivers (Diener, 1985a, 1985b). The Žgure reveals a stable river, that is, a trajectory .µs ; Js / that is attracting in forward time, indicated as a bold line in Figure 5C, with .µs ; Js / ! .¡¼; ¡1/ as t ! ¡1, dµs =dt > 0, and Js .t/ D I ¡ ge ¡t=¿I

(4.9)

for all t. We denote by T the time when µs .T/ D ¼, and deŽne J¤ D Js .T/ 2 .0; I/:

(4.10)

Equations 4.9 and 4.10 imply T D ¿I ln g ¡ ¿I ln.I ¡ J¤ /:

(4.11)

Note that the phase portrait depends on the parameters I and ¿I but not on g. Therefore, the value of J¤ depends on I and ¿I but not on g. The synchronization seen in Figure 5A can be understood from Figure 5C in the following way. For g sufŽciently large (that is, J.0/ sufŽciently negative), and for µ0 sufŽciently far from ¼ , .µ .t/; J.t// is rapidly attracted to .µs .t/; Js .t//. At the time when µ D ¼, we therefore have J ¼ J¤ or t ¼ T. Thus, the Žrst spike after time zero occurs approximately at time T. For µ0 sufŽciently close to ¼ , µ.t/ quickly passes through ¼ and is then rapidly attracted to .µ s .t/ C 2¼; Js .t//. When µ.t/ reaches 3¼, J ¼ J¤ and therefore t ¼ T. Thus, a spike occurs soon after time zero, followed by a spike approximately at time T. Only for values of µ0 in a narrow transition regime is .µ .t/; J.t// attracted to neither .µ s .t/; Js .t// nor .µs .t/ C 2¼; Js .t//. Equation 4.11 gives the time of the Žrst approximately synchronous population spike. Since we have no explicit formula for J¤ , equation 4.11 is not an explicit formula for T. However, since J¤ does not depend on g, equation 4.11 does give the precise dependence of T on g. This dependence will be of primary interest to us in the remainder of section 4. For later reference, we note how equation 4.11 is modiŽed when the synchronizing inhibitory pulse arrives not at time 0 but at some time T0 : T D T0 C ¿I ln g ¡ ¿I ln.I ¡ J¤ /:

(4.12)

522

C. Borgers ¨ and N. Kopell

4.2 Approximate Synchronization by an Inhibitory Pulse of Nonuniform Strength. Motivated by section 3, we are interested in the effects of variable synaptic strengths. We therefore let the constant g in equation 4.1 depend on j: dµj

D .1 ¡ cos µj / C .I ¡ gj s.t//.1 C cos µj /; 1 · j · N: dt That is, different neurons receive inhibitory pulses of different strengths. We assume that the gj are independent, normally distributed random variables, with mean g > 0 and standard deviation ¾ g > 0. If g is large enough and ¾ g is small enough, the population is still synchronized approximately. This is illustrated in Figure 5B, which shows results of a simulation similar to that of Figure 5A, with N D 100;

I D 0:05;

g D 0:25;

¾ g D 0:025;

¿I D 10:

(4.13)

(Compare these parameters with those in equations 4.2.) Instead of the nearly synchronous population spikes of Figure 5A, we now see spike volleys of brief but positive durations. To analyze the durations of these spike volleys, let us consider the initial value problem 4.3 and 4.4 with a random g > 0. Let ½ g D ½g .° /, ° > 0 be the probability density of g, and let g > 0 and ¾ g > 0 be its mean and standard deviation. Let X D ln g:

(4.14)

Combining equations 4.11 and 4.14, T D ¿I X ¡ ¿I ln.I ¡ J¤ /:

(4.15)

The only random quantity on the right-hand side of equation 4.15 is X. We will discuss its distribution Žrst. Let ½X D ½X .» /, ¡1 < » < 1 be the probability density of X. For ¡1 < a < b < 1, Z

b a

½X .» / d» D P.X 2 .a; b// D P.ln g 2 .a; b// D P.g 2 .ea ; eb // Z D

Z

eb ea

½ g .° / d° D

b

e» ½ g .e» / d»:

a

Therefore, ½X .» / D e» ½g .e» /

(4.16)

for all » . For small ¾ g , the standard deviation of X is ¾X ¼ ln0 .g/¾ g D

¾g : g

(4.17)

Synchronization in Sparse, Random Networks

523

From equations 4.15 and 4.16, we see that the probability density function of T is ½T .t/ D

1 .tC¿I ln.I¡J¤ //=¿I ¤ e ½g .e.tC¿I ln.I¡J //=¿I /: ¿I

(4.18)

For later reference, we note how formula 4.18 changes if the approximately synchronizing inhibitory pulse arrives not at time 0 but at some time T0 : ½T .t/ D

1 .t¡T0 C¿I ln.I¡J¤ //=¿I ¤ e ½ g .e.t¡T0 C¿I ln.I¡J //=¿I /: ¿I

(4.19)

Using equations 4.15 and 4.17, we see that the standard deviation of T is ¾ T D ¿I ¾ X ¼ ¿I

¾g g

(4.20)

for small ¾ g . We think of ¾T as a measure of the duration of the spike volleys. Thus, the duration of the spike volleys is proportional to the product of ¿I , the decay time constant of inhibition, and the coefŽcient of variation ¾ g =g of g. To verify these results computationally, we return to the example of Figure 5B. Strictly speaking, the preceding discussion does not apply to this example, since g, which is assumed to be normally distributed, is not guaranteed to be positive. However, formulas 4.16, 4.18, and 4.20 are well deŽned if ½ g is a normal density. Since g D 0:25 and ¾ g D 0:025, the probability of g · 0 is extremely small. We therefore expect equations 4.16, 4.18, and 4.20 to hold with good accuracy. We deŽne T .j/ to be the time of the spike of the jth neuron within the Žrst nearly synchronous spike volley. We set

O TD

PN

jD1 T

N

.j/

and

¾O T D

v uP u N .T .j/ ¡ T/ O 2 t jD1 N¡1

:

(4.21)

Here and for the remainder of this article, hats indicate results obtained from numerical simulations. In the example of Figure 5B, we Žnd ¾O T ¼ 1:02: We see that ¾O T is indeed close to ¿I ¾ g =g, which equals 1:0 in this example. If we double ¿I in this experiment, ¾O T rises from 1:02 to 2:04, in agreement with equation 4.20. To verify equation 4.18 numerically, we must know the value of ¿I ln.I ¡ J ¤ /. Taking expectations on both sides of equation 4.15, we Žnd ¿I ln.I ¡ J ¤ / D ¿I E.X/ ¡ E.T/ D ¿I E.ln g/ ¡ E.T/:

524

C. Borgers ¨ and N. Kopell

For sufŽciently small ¾ g , this implies ¿I ln.I ¡ J¤ / ¼ ¿I ln g ¡ E.T/; suggesting the approximation O ¿I ln.I ¡ J¤ / ¼ ¿I ln g ¡ T:

(4.22)

Figure 5D shows the density ½T , as deŽned in equation 4.18, using the approximation 4.22, with ¿I D 10, and assuming that ½ g is a normal density with g D 0:25 and ¾ g D 0:025. The histogram in Figure 5D indicates the actual spike time density, determined from the numerical simulation. The agreement between the theoretical prediction and the actual spike time distribution is excellent. For later reference, we note how equation 4.22 changes when the inhibitory pulse arrives not at time 0 but at some time T0 . From equation 4.12, we then obtain O ¿I ln.I ¡ J¤ / ¼ ¿I ln g C T0 ¡ T:

(4.23)

5 Synchronization of a Population of Theta Neurons by a Single Strong Excitatory Pulse 5.1 The Synchronous Population Spike Triggered by an Excitatory Pulse of Uniform Strength. We next consider a population of N identical, uncoupled neurons with common constant external drive below or at threshold, receiving a common excitatory synaptic input pulse at time 0. We model this situation by the equations dµj D .1 ¡ cos µj / C .I C gs.t//.1 C cos µj /; dt

1 · j · N;

(5.1)

with I · 0, g > 0, and ( e¡t=¿E if t > 0 s.t/ D 0 if t · 0 with ¿E > 0. If the excitatory pulse is strong, it triggers a nearly synchronous population spike soon after time 0. To analyze this in more detail, we consider the initial value problem: dµ D 1 ¡ cos µ C .I C ge¡t=¿E /.1 C cos µ/; dt 1 µ .0/ D ¡2 arccos p : 1¡I

t > 0;

(5.2) (5.3)

Synchronization in Sparse, Random Networks

525

cell index

100

A:

50 0 ­ 10

0 t

10

B:

T

10 5 0 0

0.2

0.4

0.2

0.4

g

0.6

0.8

1

0.6

0.8

1

C:

¶ T/¶ g

0 ­5

­ 10 0

g

Figure 6: (A) Approximately synchronous population spike triggered by a single nonuniform excitatory pulse. (B) Time T between arrival of excitatory pulse and spike triggered by it, as a function of the strength g of the pulse, for .I; ¿E /D.0; 1/ (solid line), .0; 5/ (dashed line), .0; 2/ (dash-dotted line), .¡0:01; 2/ (circles). (C) @T=@g as a function of g, for the parameter values of B.

Recall from section 2 that for I < 0, the right-hand side of equation 5.3 represents the stable Žxed point of the equation dµ=dt D 1 ¡ cos µ C I.1 C cos µ /. Thus, we are considering the response of a neuron at rest to an excitatory synaptic pulse. We denote by T the Žrst time at which the neuron spikes, with T D 1 if there is no spike at all. We have no general analytic expression for T as a function of I, ¿E , and g. However, it is easy to see that for Žxed I and ¿E , T is a strictly decreasing function of g, with limg!1 T D 0 and limg!gc C T D 1 for some gc ¸ 0 (see Figure 6B). For ¿E D 1, T can be computed using formula 2.2, with I

526

C. Borgers ¨ and N. Kopell

replaced by I C g and ¿ D 1. The formula becomes particularly simple for I D 0; in that case, TD

¼ 1 p : 2 g

(5.4)

The approximation ¿E D 1 is accurate as long as e¡T=¿E ¼ 1, since then the exponential decay in equation 5.2 can be neglected over the time interval [0; T]. Since T ! 0 as g ! 1, this means that the assumption ¿E D 1 is accurate for sufŽciently large g. The assumption I D 0 is accurate when jIj=g is sufŽciently small. So this assumption too is accurate for sufŽciently large g. Figure 6B shows T as a function of g, for various values of I and ¿E , demonstrating that equation 5.4 approximates T reasonably well over a large range of parameter values. 5.2 The Approximately Synchronous Population Spike Triggered by an Excitatory Pulse of Nonuniform Strength. If the synaptic strength in equation 5.1 depends on j, that is, if different neurons receive excitatory pulses of different strengths, the equations are dµj

D .1 ¡ cos µj / C .I C gj s.t//.1 C cos µj /; 1 · j · N: dt Figure 6A shows that the resulting population spike is not perfectly synchronous. (But notice that Figure 6A shows a brief time window only; the synchronization is not perfect but fairly tight.) In the simulation underlying this Žgure, N D 100;

I D 0;

g D 0:25;

¾ g D 0:025;

¿E D 2:

(5.5)

To analyze the duration of the spike volley triggered by an excitatory pulse of nonuniform strength, we consider the initial value problem 5.2, 5.3 with a random g > gc . If ¾ g is small, the standard deviation of T is ­ ­ ­ @T ­ ­ (5.6) ¾T ¼ ­ ¾g : ­ @g ­ For ¿E D 1 and I D 0, @T ¼ D g¡3=2 4 @g

(5.7)

by equation 5.4. Figure 6C shows @ T=@g as a function of g, for various values of I and ¿E . The Žgure conŽrms that the right-hand side of equation 5.7 approximates @T=@g reasonably well for large values of g. Combining equations 5.6 and 5.7 yields ¾T ¼

¼ 1 ¾g p : 4 g g

(5.8)

Synchronization in Sparse, Random Networks

527

For illustration, we return to the example of Figure 6A (I D 0, ¿E D 2, g D 0:25, ¾ g D 0:025). We deŽne T .j/ to be the time of the spike of the jth neuron, and deŽne TO and ¾O as in equation 4.21. Numerically, we Žnd T

¾O T ¼ 0:270: To evaluate the right-hand side of equation 5.6, we approximate @T=@g numerically. We Žnd @T=@g ¼ ¡10:30 for the parameter values of Figure 6A. The approximation of equation 5.6, based solely on the assumption that ¾ g is so small that the relation between ¾T and ¾ g is approximately linear, proves fairly accurate here; it yields ¾T ¼ 0:256: The assumption ¿E D 1, which underlies equation 5.8, degrades the accuracy, but by less than a factor of two: ¾T ¼ 0:157: 6 The PING Synchronization Mechanism 6.1 PING in Fully Connected E-I Networks. We return to the example of Figure 1A. Each of the cell groups (E and I) synchronizes rapidly, with the population spikes of the inhibitory neurons slightly lagging behind those of the excitatory ones. We state, in a nonrigorous way, based on numerical experience and heuristics, conditions that are sufŽcient to induce Žring patterns as in Figure 1A: Condition 1: The E-cells receive external input signiŽcantly above their spiking threshold. Condition 2: The E ! I synapses are so strong and have so short a rise time that a surge in spiking of the E-cells quickly triggers a surge in spiking of the I-cells. Condition 3: The I-cells spike only in response to the E-cells. Condition 4: The I ! E synapses are so strong that a population spike of the I-cells approximately synchronizes the E-cells. If these four conditions are satisŽed, synchronous rhythmic spiking develops as follows (Whittington et al., 2000; Tiesinga et al., 2001). Initial activity in the E-cells triggers activity in the I-cells. This inhibits activity in the E-cells, thereby removing the drive to the I-cells. A period of low activity in both E- and I-cells results. When the inhibition wears off and the E-cells spike again, they are closer to synchrony than previously because of the mechanism described in section 4.1. The spiking of the E-cells causes spiking of the I-cells, closer to synchrony than previously because of the mechanism of section 5.1. The cycle now repeats.

528

C. Borgers ¨ and N. Kopell

A

B E­ cells

400

E­ cells

400 200

200 0 100

I­ cells

I­ cells

0 100 50 0 0

100 t

50 0 0

200

C E­ cells

E­ cells

400

200

200 0 100

I­ cells

0 100

I­ cells

200

D

400

50 0 0

100 t

100 t

200

50 0 0

100 t

200

Figure 7: Illustration of conditions 2–4 from section 6.1. (A) PING is lost when E ! I synapses become too weak. (B) PING is lost as a result of too much drive to the I-cells. (C) Rhythm is restored by adding I ! I synapses. (D) PING is lost when I ! E synapses become too weak.

Condition 1 is evidently needed to drive activity. We discuss conditions 2 through 4 in more detail and present numerical results illustrating what happens when they are violated. When condition 2 is violated, that is, when the E ! I synapses are weak, a pattern such as the one shown in Figure 7A often develops: the E-cells and the I-cells still synchronize, but the E-cells spike several times between population spikes of the I-cells. The parameters in Figure 7A are as in Figure 1A, except that gEI has been reduced from 0:25 to 0:05.

Synchronization in Sparse, Random Networks

529

Condition 3 is violated if the drive to the I-cells becomes too strong, but can be restored by introducing I ! I-synapses. We illustrate this with the following numerical experiment. In Figure 1A, II D 0. If we raise II to 0:05, the Žgure changes dramatically, as shown in Figure 7B. Here, II is strong enough to drive asynchronous activity in the I-cells that suppresses the Ecells altogether. Condition 3, and with it the rhythm, is restored by setting gII D 0:25 (see Figure 7C). We note that in the example of Figure 7C, the I-cells would spike synchronously even without the E-cells. Thus, the role of the I-I synapses is to synchronize the I-cells, replacing nearly constant inhibition by phasic inhibition, which allows the E-cells to Žre. One might therefore consider the rhythm in Figure 7C an “ING” or “° -I” rhythm (see section 7). The parameter regime investigated here is similar to that of Figure 7d of Tiesinga et al. (2001). There as here, asynchronous activity of the I-cells suppresses activity in the E-cells altogether, while synchronous activity of the I-cells permits Žring of the E-cells. However, in Figure 7C, the E-cells do play an important role in setting the frequency of the rhythm. In the absence of the E-cells, the rhythm would be much slower. Thus, the Žring of the I-cells in Figure 7C does come in response to the Žring of the E-cells, as in PING. Figure 7D shows what may happen when condition 4 is violated, that is, when the I ! E synapses are weak. The parameters in Figure 7D are as in Figure 1A, except that gIE has been reduced from 0:25 to 0:05. The inhibitory synapses no longer sufŽce to synchronize the E-cells. As a result, the I-cells, which were synchronized by the E-cells in Figure 1A, are no longer synchronized either. A mathematical examination of conditions 1 through 4 will be the subject of future publications. 6.2 PING in Sparsely, Randomly Connected E-I Networks. When pEI and pIE are reduced from 1:0 to 0:5, Figure 1A turns into Figure 1B. In analyzing the population spikes of the E-cells in Figure 1B, we make the simplifying assumption that the population spikes of the I-cells are perfectly synchronous. Since the I-cells are in fact fairly tightly synchronized in Figure 1B, this is a good approximation at least for the parameters used in Figure 1B. When the I-cells spike, all E-cells receive inhibitory pulses. However, different neurons receive inputs of different strengths because of the random connectivity. The resulting approximately synchronous spike volley of the E-cells can be analyzed using section 4.2. We focus on one particular spike volley of the E-cells, say, the Žrst one .j/ following t D 100 in Figure 1B. We deŽne TE to be the time of the spike of the jth E-cell during this volley. We deŽne v u P .j/ P .j/ u .T ¡ TO /2 T t j E E OT D j E and ¾O E D : E NE NE ¡ 1

530

C. Borgers ¨ and N. Kopell

From equations 2.11 and 4.20, we Žnd the prediction s ¾O E ¼ ¿I

1 ¡ pIE : pIE NI

(6.1)

For the Žrst spike volley of the E-cells following t D 100 in Figure 1B, we Žnd numerically ¾O E D 1:18: The prediction of equation 6.1 is remarkably accurate: s 1 ¡ pIE ¼ 1:22: ¿I pIE NI We expect the shape of the spike volley to be approximately described by equation 4.19. To evaluate equation 4.19, we must evaluate T0 ¡ ¿I ln.I ¡ J¤ /. (Recall that T0 is the time at which the synchronizing inhibitory pulse arrives, here the time of the inhibitory population spike immediately preceding the excitatory population spike under consideration.) Following equation 4.23, we use the approximation T 0 ¡ ¿I ln.I ¡ J¤ / ¼ TO E ¡ ¿I ln gIE : The predicted and actual spike time distributions are shown in Figure 8. The agreement is good. We now consider the Žrst spike volley of the I-cells following t D 100 in .j/

Figure 1B. We deŽne TI to be the time of the spike of the jth I-cell during this volley, and v u P .j/ P .j/ u .T ¡ TO /2 t j I I OT D j TI and O D ¾ : I I NI NI ¡ 1 From equations 2.9 and 5.8, we Žnd the prediction s ¼ 1 ¾O I ¼ p 4 gEI

1 ¡ pEI : pEI NE

(6.2)

For the Žrst spike volley of the I-cells following t D 100 in Figure 1B, we Žnd numerically ¾O I D 0:151 : The prediction of equation 6.2 is somewhat inaccurate, as was to be expected, because it is based on three rather substantial idealizations: ¿E D 1, perfect

Synchronization in Sparse, Random Networks

531

Figure 8: Distribution of spikes within the Žrst spike volley following t D 100 in Figure 1B: predicted (solid line) and actual (bars).

synchrony of the E-cells, and the assumption that the I-cells return to rest between the spike volleys of the E-cells (see equation 5.3). However, the discrepancy is still not greater than a factor of two: s ¼ 1 p 4 gEI

1 ¡ pEI ¼ 0:0785: pEI NE

We note that it would be possible to relax the assumption of perfect synchrony in the E-cells, since we have fairly precise information about the durations and even the shapes of the spike volleys of the E-cells. We do not pursue this here, since our goal here is qualitative insight, not precise quantitative information. 7 Discussion We have analyzed the effects of sparse, random connectivity on the PING synchronization mechanism. In particular, we have derived approximate formulas for the durations of the spike volleys, equations 6.1 and 6.2. To make these formulas as transparent as possible, let us use the approximations 1 ¡ pIE ¼ 1 and 1 ¡ pEI ¼ 1, and write M EI D pEI NE

532

C. Borgers ¨ and N. Kopell

for the expected number of excitatory inputs per inhibitory cell and MIE D pIE NI for the expected number of inhibitory inputs per excitatory cell. Equations 6.1 and 6.2 then become ¾O E ¼ ¿I p

1

(7.1)

M IE

and 1 ¼ p ¾O I ¼ p : 4 gEI MEI

(7.2)

An interesting feature of these formulas is their lack of symmetry. The time constant ¿I appears in equation 7.1, but the time constant ¿E does not appear in equation 7.2. Similarly, gEI appears in equation 7.2, but gIE does not appear in equation 7.1. If a theta neuron were driven with a constant drive equal to gEI , it would spike periodically with a period which we call PEI . From equation 2.3, ¼ P EI D p : gEI Using this in equation 7.2, we Žnd ¾O I ¼

1 PEI p : 4 M EI

(7.3)

Let P denote the period of the PING rhythm. In the simulations of this article, P EI ¿ ¿I < P;

M EI > MIE À 1:

(7.4)

The inequality MIE À 1 is certainly realistic (Braitenberg & Schuz, ¨ 1998). The inequality ¿I < P holds for a gamma rhythm if the inhibitory synapses are mediated by GABAA , since the typical period of gamma oscillations is about 25 ms and the decay time constant of GABA A synapses is about 10 ms. It appears that PING without PEI ¿ ¿I is not possible with realistic values of ¿I ; more will be said on this point. Combining 7.1, 7.3, and 7.4, we Žnd ¾O I ¿ ¾O E ¿ P: Thus, our theory predicts that for PING in realistic parameter regimes, sparse and random connectivity will not prevent either E- or I-cells from synchronizing fairly tightly, but the I-cells will synchronize much more tightly

Synchronization in Sparse, Random Networks

533

E­ cells

400 200

I­ cells

0 100 50 0 0

100 t

200

Figure 9: PING in an E-I network with sparse, random connectivity, with nearly instantaneous synapses.

than the E-cells. Our computational results are in agreement with these conclusions (see Figure 1B). It is possible to obtain PING without the condition PEI ¿ ¿I if ¿I is made unrealistically small. To compensate, one must then also make gIE unrealistically large. In fact, in many modeling studies, synapses have been assumed to act instantaneously, that is, with zero rise and decay times (Brunel, 2000; Brunel & Hakim, 1999; Izhikevich, 1999; Mirollo & Strogatz, 1990; Tiesinga & Sejnowski, 2001); this amounts to taking a limit in which simultaneously ¿I ! 0 and gIE ! 1. (In Brunel, 2000, and Brunel & Hakim, 1999, there is a delay between the spiking of the presynaptic neuron and its effect on the postsynaptic neuron; however, this delay is not relevant for this discussion.) In some regard, the behavior of instantaneous synapses is not very different from that of synapses with more realistic time courses. However, equations 7.1 and 7.2 predict that the effects of random connectivity will be reduced dramatically by making the synapses instantaneous. To verify this numerically, we repeat the simulation of Figure 1B, with ¿I reduced by a factor of 50 and gIE raised by a factor of 50. This closely mimics instantaneous synapses but is computationally simpler. The result is shown in Figure 9. As predicted, synchronization is much tighter than in Figure 1B, even though the randomness in the connectivity is the same in Figure 9 as in Figure 1B. The rhythm is also accelerated; this is a result of the reduction in ¿I .

534

C. Borgers ¨ and N. Kopell

Our theory for the spike volleys of the E-cells is precise enough to allow accurate predictions not just of the durations of the volleys but even of their shapes, as shown in Figure 8. A reŽnement of the theory could be obtained by taking into account the positive durations of the spike volleys of the I-cells, perhaps using ideas similar to those of section 3.2 of Tiesinga and Sejnowski (2001). (Recall that in section 4, the duration of the spike volleys of the I-cells was assumed to be negligible.) However, our numerical results indicate that in our parameter regime, neglecting the durations of the spike volleys of the I-cells gives an excellent approximation. Tiesinga et al. (2002) discuss a related problem, using numerical simulation primarily, and with an emphasis on information-theoretic ideas. They also present experimental results. For instance, Figure 8b of their article illustrates how the jitter in the spike times of a rat hippocampal neuron decreases when the quantity npre (our pIE NI ) increases, in rough qualitative agreement with equation 6.1. (Rough qualitative agreement is the best that can be expected here, in view of the idealized nature of our theory.) Our theory for the spike volleys of the I-cells is less accurate than that for the excitatory ones. However, since the synchronization of the I-cells is typically quite tight and brought about by a crude mechanism (a burst of excitation triggers an almost immediate, and therefore almost synchronous, response of the I-cells), a precise theory for the spike volleys of the I-cells is of less interest here. To create such a theory, we would need to study how the duration of an excitatory input spike volley is related to the duration of an output spike volley triggered by it. (Recall that in section 5, the duration of the spike volleys of the E-cells was assumed to be negligible.) This issue is centrally important in the study of synŽre chains (Diesmann, Gewaltig, & Aertsen, 1999). Figure 3c of Diesmann et al. (1999) shows that the strength of the input spike volley is crucial. Strong, loosely synchronous input spike volleys can trigger tightly synchronous output spike volleys. Using a different way of measuring synchrony, the relation between input synchronization and output synchronization was also studied by Burkitt and Clark (2001). Combining ideas from the preceding two paragraphs, a more accurate overall theory of PING may be created as follows. First, the approximate spike time distribution within the spike volleys of the E-cells, computed based on section 4, is taken into account in approximating the spike time distribution within the spike volleys of the I-cells. This yields a reŽnement of section 5, which in turn can be taken into account in approximating the spike time distribution within the spike volleys of the E-cells, leading to a reŽnement of section 4. Iterating this process, one may obtain increasingly accurate approximations to the spike time distributions within spike volleys. However, such an improved theory of PING is beyond the scope of this article. While working on this project, we carried out far more simulations than have been presented here. Our conclusions hold over a wide range of pa-

Synchronization in Sparse, Random Networks

535

E­ cells

400 200

I­ cells

0 100 50 0 0

100 t

200

Figure 10: ING in an E-I network with sparse, random connectivity.

rameters. For instance, E!E synapses with gEE D 0:25 have little effect on Figure 1B. Weak I!I synapses have little effect, except for the point made at the end of section 6.1: such synapses can make gamma rhythms possible in cases when the external drive to the I-cells is fairly strong. Strengthening I!I synapses often leads to a transition to a different synchronization mechanism, called ING by Whittington et al. (2000) and ° -I by Tiesinga et al. (2001). In ING, the I-cells are driven externally, not by the E-cells, and synchronize not only the E-cells but also themselves. For ING, the widths and shapes of the spike volleys of both E- and I-cells can be approximated using the ideas of section 4.1. Figure 10 shows a simulation with gIE D gII D 0:25;

gEI D gEE D 0;

IE D II D 0:1;

¿I D 10;

pIE D pII D 0:5:

As our theory would predict, the volleys of the E- and I-cells are now of equal duration. In future work, we will investigate ING in sparse, random networks in more detail, including in particular the effects of E ! I-synapses. Throughout this article, we have used the theta model. As remarked in section 2.1, the theta model is canonical for type I neuronal models in the sense that other type I models can be reduced to it by coordinate transformations (Ermentrout & Kopell, 1986; Hoppensteadt & Izhikevich, 1997). We therefore expect the picture to be qualitatively similar for all type I models,

536

C. Borgers ¨ and N. Kopell

even though the details of our calculations, and in particular the details of the centrally important Figure 5C, do depend on our choice of model. Whether and how our results generalize to type II models remains to be explored. It would also be interesting to explore the effects of spike adaptation on our analysis. Synchronization in sparsely, randomly connected networks with spike adapation has been studied by van Vreeswijk and Hansel (2001). They discuss synchronization of bursts (not individual spikes) via adaptation (not inhibition) and observe that strong I ! E synapses desynchronize bursts, in contrast with our regime, in which strong I ! E synapses synchronize spikes. The model and analysis are so different from ours that a detailed comparison would be a major endeavor, but the article certainly suggests studying effects of adaptation in our model in the future. We have analyzed synchronization by common input for the purpose of better understanding PING and ING. We remark, however, that synchronization by common input is also of neurobiological interest by itself (Usrey & Reid, 1999). We have shown that even very sparse input can synchronize and have analyzed the desynchronizing effect of heterogeneity in input strength. Acknowledgments We are grateful to Marcelo Camperi for stimulating discussions during the early stages of this work and to Steve Epstein for carefully reading the manuscript and providing helpful criticism. We thank the referees for interesting and useful comments. C. B. thanks the Center for BioDynamics at Boston University for a productive and pleasant visit during the spring of 2001, when this project was begun. His work was also supported by an equipment grant from Tufts University’s Marshall Fund for Biomedical Research. N. K. received support from NSF grant DMS-9706694 and NIH grant MH47150. References Barkai, E., Kanter, I., & Sompolinsky, H. (1990). Properties of sparsely connected excitatory neural networks. Phys. Rev. A, 41, 590–597. Braitenberg, V., & Schuz, ¨ A. (1998). Cortex: Statistics and geometry of neuronal connectivity (2nd ed.). New York: Springer-Verlag. Brunel, N. (2000). Dynamics of sparsely connected networks of excitatory and inhibitory spiking neurons. J. Comp. Neurosci., 8, 183–208. Brunel, N., & Hakim, V. (1999). Fast global oscillations in networks of integrateand-Žre neurons with low Žring rates. Neural Comp., 11, 1621–1671. Burkitt, A. N., & Clark, G. M. (2001). Synchronization of the neural response to noisy periodic synaptic input. Neural Computation, 13, 2639–2672.

Synchronization in Sparse, Random Networks

537

Bush, P., & Sejnowski, T. (1996). Inhibition synchronizes sparsely connected cortical neurons within and between columns in realistic network models. J. Comp. Neurosci., 3, 91–110. Diener, F. (1985a). Propri´et´es asymptotiques des euves. C. R. Acad. Sci. Paris, 302, 55–58. Diener, M. (1985b). D´etermination et existence des euves en dimension 2. C. R. Acad. Sci. Paris, 301, 899–902. Diesmann, M., Gewaltig, M.-O., & Aertsen, A. (1999). Stable propagation of synchronous spiking in cortical neural networks. Nature, 402, 529–533. Ermentrout, G. B. (1996). Type I membranes, phase resetting curves, and synchrony. Neural Comp., 8, 879–1001. Ermentrout, G. B., & Kopell, N. (1986). Parabolic bursting in an excitable system coupled with a slow oscillation. SIAM J. Appl. Math, 46, 233–253. Golomb, D., & Hansel, D. (2000). The number of synaptic inputs and the synchrony of large sparse neuronal networks. Neural Comp., 12, 1095–1139. Gutkin, B. S., & Ermentrout, G. B. (1998). Dynamics of membrane excitability determine interspike interval variability: A link between spike generation mechanisms and cortical spike train statistics. Neural Comp., 10, 1047–1065. Hansel, D., & Mato, G. (2001). Existence and stability of persistent states in large neuronal networks. Phys. Rev. Lett., 86, 4175–4178. Hodgkin, A. L. (1948). The local changes associated with repetitive action in a non-medullated axon. J. Physiol. (London), 107, 165–181. Hoppensteadt, F. C., & Izhikevich, E. M. (1997). Weakly connected neural networks. New York: Springer-Verlag. Izhikevich, E. M. (1999). Class 1 neural excitability, conventional synapses, weakly connected networks, and mathematical foundations of pulse-coupled models. IEEE Transactions on Neural Networks, 10, 499–507. Izhikevich, E. M. (2000). Neural excitability, spiking, and bursting. Int. J. Bifurcation and Chaos, 10, 1171–1266. Mirollo, R. E., & Strogatz, S. H. (1990). Synchronization of pulse-coupled biological oscillators. SIAM J. Appl. Math., 50, 1645–1662. Rinzel, J., & Ermentrout, G. B. (1998). Analysis of neural excitability and oscillations. In C. Koch & I. Segev (Eds.), Methods in neuronal modeling (pp. 251–292). Cambridge, MA: MIT Press. Tiesinga, P. H. E., Fellous, J.-M., Jos´e, J. V., & Sejnowski, T. J. (2001). Computational model of carbachol-induced delta, theta, and gamma oscillations in the hippocampus. Hippocampus, 11, 251–274. Tiesinga, P. H. E., Fellous, J.-M., Jos´e, J. V., & Sejnowski, T. J. (2002). Information transfer in entrained cortical neurons. Network: Comput. Neural Syst., 13, 41–66. Tiesinga, P. H. E., & Sejnowski, T. J. (2001). Precision of pulse-coupled networks of integrate-and-Žre neurons. Network: Comput. Neural Syst., 12, 215– 233. Traub, R. D., Jefferys, J. G. R., & Whittington, M. A. (1999). Fast oscillations in cortical circuits. Cambridge, MA: MIT Press. Usrey, W. M., & Reid, R. C. (1999). Synchronous activity in the visual system. Annual Review of Physiology, 61, 435–456.

538

C. Borgers ¨ and N. Kopell

van Vreeswijk, C., & Hansel, D. (2001). Patterns of synchrony in neural networks with spike adaptation. Neural Comp., 13, 959–992. van Vreeswijk, C. A., & Sompolinsky, H. (1996). Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science, 274, 1724–1726. van Vreeswijk, C. A., & Sompolinsky, H. (1998). Chaotic balanced state in a model of cortical circuits. Neural Comp., 10, 1321–1372. Wang, X.-J., & Buzs´aki, G. (1996). Gamma oscillation by synaptic inhibition in a hippocampal interneuronal network model. J. Neurosci., 16, 6402–6413. Wang, X.-J., Golomb, D., & Rinzel, J. (1995). Emergent spindle oscillations and intermittent burst Žring in a thalamic model: SpeciŽc neuronal mechanisms. Proc. Natl. Acad. Sci. USA, 2, 5577–5581. Whittington, M. A., Traub, R. D., Kopell, N., Ermentrout, B., & Buhl, E. H. (2000). Inhibition-based rhythms: Experimental and mathematical observations on network dynamics. Int. J. of Psychophysiol., 38, 315–336. Received February 11, 2002; accepted July 30, 2002.

NOTE

Communicated by Emilio Salinas

The Effects of Input Rate and Synchrony on a Coincidence Detector: Analytical Solution Shawn Mikula [email protected]

Ernst Niebur [email protected] Zanvyl Krieger Mind=Brain Institute and Department of Neuroscience, Johns Hopkins University, Baltimore, MD 21218, U.S.A.

We derive analytically the solution for the output rate of the ideal coincidence detector. The solution is for an arbitrary number of input spike trains with identical binomial count distributions (which includes Poisson statistics as a special case) and identical arbitrary pairwise crosscorrelations, from zero correlation (independent processes) to complete correlation (identical processes). 1 Introduction Individual neurons typically receive thousands of inputs that vary temporally with respect to both mean rate and cross-correlation. Much experimental evidence underscores the importance of mean input rate and cross-correlation as determinants of neural Žring, though the relative contributions of each are not so clear (Abeles, 1982, 1991; Softky & Koch, 1992, 1993; Shadlen & Newsome, 1994; Alonso, Usrey, & Reid, 1996; Decharms & Merzenich, 1996; Konig, ¨ Engel, & Singer, 1996; de Oliveira, Thiele, & Hoffmann, 1997; Riehle, Grun, ¨ Diesmann, & Aertsen, 1997; Steinmetz et al., 2000; Salinas & Sejnowski, 2000; Niebur, Hsiao, & Johnson, 2002; Williams & Stuart, 2002). Given the importance of rate and cross-correlation in neural coding, the determination of how inputs modulated with respect to both affect neural Žring assumes signiŽcance if the impact of these different neural codes is to be fully appreciated. Employing simple probabilistic and combinatorial methods, we derive the exact analytical solution for the output rate of a coincidence detector receiving an arbitrary number of inputs modulated with respect to both mean rate and cross-correlation 2 Binomial Spike Trains with SpeciŽc Cross-Correlation In this section, we introduce a systematic method for the generation of an arbitrary number of spike trains with speciŽed pair-wise mean crossNeural Computation 15, 539–547 (2003)

c 2003 Massachusetts Institute of Technology °

540

S. Mikula and E. Niebur

correlations and Žring rates. 1 Spikes are distributed according to binomial counting statistics in each spike train. Mean Žring rates and crosscorrelations are the same for all spike trains (or all pairs of spike trains, respectively), but they can be chosen independently of each other. The algorithm that we describe here is as suitable for the analytical computations in this report as for the implementation of such spike trains in numerical simulations. We emphasize that although we use this particular algorithm to illustrate the derivation of the output rate of the coincidence detector, our results are of general validity since the properties of the spike trains are entirely speciŽed by their correlation coefŽcient and the statistics of the individual spike trains. However, no higher-order effects (correlations of order 3 and higher; see, e.g., Bohte, Spekreijse, & Roelfsema, 2000) are included. First, we establish some notation. Let m be the number of input spike trains, each one having n time bins. All bins are of equal length 1t, chosen sufŽciently small so that each contains a maximum of one spike, that is, each bin is guaranteed to contain either one or zero spikes. The probability that a spike is found in any given time bin is p; no spike is therefore found with probability 1 ¡ p. Bins in any given spike train are independent, which implies that all the following analysis can be limited to a single time bin. Note that the physiologically important Poisson statistics are the special case of low rates and long spike trains. Generation of m spike trains with speciŽc correlation starts by generating m C 1 independent spike trains with the desired mean Žring rate (which is just p=1t). Let us assume we want a correlation coefŽcient q (with 0 · q · 1) between each and any pair of the m spike trains. We designate spike train number m C 1 as the reference spike train. In order to introduce the correlation coefŽcient q between spike trains 1; : : : ; m, we will switch, with p a probability q, the state of a time bin in each of these spike trains to that found in the reference spike train. This yields a mean correlation coefŽcient of q between any two of the spike trains 1; : : : ; m without changing the mean Žring rate.2 No further use is made of the reference spike train.

1 The method proposed here is not the only possible way to introduce correlations between spike trains. For instance, one could add spikes generated by a common Poisson process to each individual spike train (Stroeve & Gielen, 2001), or start from a common spike train and generate two different spike trains by removing spikes independently. The method presented here has the advantage of being straightforward and efŽcient and of generating spike trains with controlled rates and correlation coefŽcients, both determined in a direct way. 2 Since all spike trains, including the reference spike train, have the same mean Žring rate, changing the state of any bin to the state of any bin in another spike train does not change the probability of this bin’s containing a spike. Therefore, this manipulation does not change the mean Žring rate of the spike train.

Coincidence Detector: Analytical Solution

541

3 Coincidence Detection for Vanishing Input Correlation The coincidence detector is a computational unit that Žres if the number of input spikes received within a given time bin equals or exceeds the threshold, µ. It is instructive to Žrst derive a solution for independent inputs (q D 0) with mean rate p=1t. Using the binomial distribution and elementary combinatorics, we Žnd that the probability of obtaining exactly j coincident input spikes from a set of m input spike trains with probability p for a spike (to exist within each time bin) is

P.j/ D

³ ´ m j p .1 ¡ p/m¡j : j

(3.1)

Since we are interested only in cases where the coincidence detector receives at least µ coincident input spikes, the probability for the coincidence detector to produce an output spike is

Pout .p; m; µ I q D 0/ D

m ³ ´ X m jDµ

j

p j .1 ¡ p/m¡j :

(3.2)

Dividing equation 3.2 by 1t, we obtain the mean output rate of a coincidence detector in the case of vanishing input correlation:

Nout .p; m; µI q D 0/ D

m ³ ´ 1 X m j p .1 ¡ p/m¡j : 1t jDµ j

(3.3)

4 Finite Correlation Let us continue to the case of input spikes that vary with respect to both mean rate and cross-correlation. In the notation introduced in section 3, we have to compute Pout .p; m; µ; q/, the probability that the coincidence detector produces an output spike (in this time bin) when it receives m input spike trains with cross-correlation q. Conceptually, it is helpful to start with independent input spike trains, q D 0, and then apply the modiŽcation of the result as the procedure described in section 2 is applied to generate input spike trains with Žnite q. Thus, the generation of our correlated input spike trains has two parts: (1) the generation of independent spike trains and (2) the imposition of our cross-correlation procedure to introduce crosscorrelation between the input spike trains.

542

S. Mikula and E. Niebur

Without loss of generality, we treat Pout .p; m; µ; q/ as the following joint probability, again derived from the binomial distribution: m ³ ´ X m j P out .p; m; µ; q/ D p .1 ¡ p/m¡j C1 .j; q/ j jDµ C

µ¡1 ³ ´ X m jD0

j

p j .1 ¡ p/m¡j C2 .j; q/:

(4.1)

In this equation, both C1 .j; q/ and C2 .j; q/ are conditional probabilities; in addition to j and q, they depend on µ and m, but these arguments are suppressed to alleviate the notation. SpeciŽcally, C1 .j; q/ is the probability of obtaining at least µ coincident input spikes given that initially (i.e., before applying our cross-correlation procedure), there were j ¸ µ coincident input spikes. The factor C2 .j; q/ is the probability of obtaining at least µ coincident input spikes given that initially there were j < µ. From their deŽnition, it is obvious that C1 .j; 0/ ´ 1 and C2 .j; 0/ ´ 0, which will later be found to be the case explicitly (see equations 4.2 and 4.4). As it must be, equation 3.2 is thus just the special case of q D 0. Equation 4.1 expresses the joint probabilities for a two-part experiment: the generation of independent spike trains and the correlation procedure applied to these spike trains that is described in section 2. Let us Žrst consider C2 , the conditional probability for a suprathreshold event (that is, among the m input neurons, µ or more have a spike) occurring in the (whole) experiment given that initially (after completion of only the Žrst part) it was subthreshold. This can happen only if the reference spike train has a spike in the time bin under consideration since otherwise, a subthreshold event can never be converted to a suprathreshold event (if the state of any neuron will be switched, it will be switched to that of the reference spike train). This occurs with a probability p.3 Since by assumption j < µ neurons already have a spike (and their state will not be inuenced even if a switch should occur since the reference spike by assumption also has a spike), we have to compute the probability that a sufŽcient number of the remaining .m ¡ j/ neurons that do not spike before application of the correlation procedure will be switched. Since p all switches are independent and occur with a probability q, we can expect i switches among the available .m ¡ j/ neurons with a probability ¡m¡j¢p i p q .1 ¡ q/.m¡j/¡i . This gives rise to a suprathreshold event if i ¸ µ ¡ j i 3

Formally, the procedure could also be viewed as a three-part experiment: (1) the generation of spike trains, (2) the correlation procedure described in section 3 for any reference spike train, and (3) the drawing of either a spike or a nonspike in the reference spike train, with probabilities p and .1 ¡ p/, respectively. We have simpliŽed the analysis somewhat by taking only those parts in equations 4.2 and 4.3 that actually contribute to the output of the coincidence detector.

Coincidence Detector: Analytical Solution

543

since in this case, the total number of spikes is j C i ¸ µ . We therefore have to sum the probabilities for all cases with i ¸ µ ¡ j and obtain C2 .j; q/ in equation 4.1 as C2 .j; q/ D

³

m¡j X iDµ ¡j

´ m¡ j p i p q .1 ¡ q/m¡j¡i p: i

(4.2)

Now we turn to the derivation of C1 .j; q/ in equation 4.1, which is the probability that an initially (i.e., at q D 0) suprathreshold event remains suprathreshold after cross-correlation. This is more difŽcult to compute directly, and therefore we compute its complement, that is, one minus this probability. The latter is the probability of switching a suprathreshold event to a subthreshold event. Assuming now j ¸ µ, this probability is found by an argument completely analogous to that leading to equation 4.2 as ³´ j X j p i p q .1 ¡ q/ j¡i .1 ¡ p/: i iDj¡µ C1

(4.3)

It can be seen that this is the probability that i (with i > j ¡ µ) input spikes get switched to the value in the reference spike train during the generation of correlated spike trains. This expression also takes into account the requirement that the bin in the reference train contains no spike (from which the .1 ¡ p/ term results). Therefore, we have the following: C1 .j; q/ D 1 ¡

³´ j X j p i p q .1 ¡ q/ j¡i .1 ¡ p/: i iDj¡µC1

(4.4)

This completes the computation of equation 4.1 for Pout .q/. To compute the mean output rate of the coincidence detector Nout .p; q; µ/ as a function of input rate p=1t, cross-correlation q, and threshold µ, we collect terms and multiply equation 4.1 by the inverse of the time bin width, 1t, to yield the following: Nout .p; m; µ; q/ 2 0 1 ³´ j m ³ ´ X 1 X m j p p 4 D p j .1¡p/m¡j @1¡ qi .1¡ q/ j¡i .1¡p/A 1t jDµ j i iDj¡µ C1 C

3 ³ ´ m¡j p i p p j .1 ¡ p/m¡j q .1 ¡ q/m¡j¡i 5 : (4.5) j i iDµ¡j

µ¡1 ³ ´ X m jD0

m¡j X

We have thus obtained our main result: the exact analytical solution for the output rate of a coincidence detector receiving an arbitrary number of

544

S. Mikula and E. Niebur

inputs with identical binomial statistics and modulated with respect to both mean rate and cross-correlation. The validity of equation 4.5 is conŽrmed by simulation (not shown). 5 Example An illustration of equation 4.1 for a coincidence detector receiving m D 100 binomial input spike trains and with a threshold µ D 15 is shown in Figure 1a. For low values of cross-correlation, the output rate of the coincidence detector is sigmoidal as a function of the input rate, and it rapidly saturates as the input rate increases. For increasing cross-correlation, this behavior approaches more and more a linear increase with input rate, which is, of course, nothing but the solution in which each synchronous volley of incoming action potentials leads to exactly one output spike. In Figure 1b, we show a cut through Figure 1a for the case of constant probability p D 0:1, corresponding to a Žring rate of 50 Hz for 1t D 2 ms, which is physiologically realistic in many cases. For this low probability, the binomial distribution of input spike trains can be reasonably well approximated by a Poisson distribution. Bernander, Koch, and Usher (1994) emphasized that increasing synchrony does not necessarily lead to an increased output rate since synchronous volleys with more spikes than required to overcome the threshold are wasted. They demonstrated by way of simulation of different neuron models that the output Žring rate as a function of input correlation in this case takes on the shape of an inverted U. Figure 1b shows an example of this behavior in the case of our exact solution. While synchrony in this example always increases the efŽcacy of the incoming Žring rates beyond that obtained for independent spike trains, this increase reaches a maximum for small q, after which the output rate decreases again. For this value of p, the trivial solution in which one input volley generates exactly one output spike (i.e., output rate ´ p) is obtained for all q »> 0:1.

Figure 1: Facing page. The plots illustrate the main equation (4.1) derived in this article for a coincidence detector receiving m D 100 binomial input spike trains with a threshold µ D 15. (a) The probability for a coincidence detector to Žre an action potential per unit time (see equation 4.1). (b) An iso-frequency (p = .1) slice through the surface of a, showing the output probability of the coincidence detector as a function of input cross-correlation. At this relatively low input frequency (corresponding to 50 Hz for 1t D 2 ms), the input spike trains are approximately Poisson. Note the inverted U shape of the plot, whereby higher cross-correlated inputs result in a lower output due to inefŽcient allocation of input spikes.The horizontal dashed line shows p D 0:1.

Coincidence Detector: Analytical Solution

545

546

S. Mikula and E. Niebur

6 Discussion In this article, we have derived analytically the solution for the output Žring rate of an ideal coincidence detector as a function of the input rate and correlation between input spike trains. Our derivation was based on a simple model and it is important to make explicit the key assumptions of our model: (1) the bins in any given input spike train are independent, (2) the coincidence detector “neuron” integrates over a very short time window, and (3) there is no inhibition. The Žrst condition means that there might be additional effects that depend on the structure of the individual spike trains; the second condition makes this model memoryless, which serves to differentiate it from integrator models with long time constants; and the last condition means that the applicability of our model to neurons that receive strong, structured inhibitory input may be limited. All of these conditions are signiŽcant simpliŽcations; for instance, it is well known (for a recent example, see Tiesinga, Fellous, & Sejnowski, 2002) that neurons are not memoryless processes (assumption 1 for incoming spike trains and assumption 2 for the neural processing). We feel that these simpliŽcations are justiŽed in the interest of obtaining an analytical solution. Others have noted that output frequency varies with the correlation of input spike trains in a nontrivial and frequently nonmonotonic way. In particular, the existence of a maximum, as shown in Figure 1b, has been predicted by Bernander et al. (1994), based on simulations. We can compute the location of this maximum from our solution by differentiating equation 4.5 with respect to the cross-correlation q and setting the right-hand side equal to zero. The computation is straightforward but not particularly illuminating. Instead, we plotted iso-frequency curves through the surface given by equation 4.5 to obtain the position of this maximum (not shown). We observed that the prominence of this peak increases with increasing number of inputs and that it is conŽned to low input rates (the Poisson regime), disappearing altogether above a certain input rate where monotonically decreasing iso-frequency curves are obtained. This general behavior can be seen in Figure 1a. These observations lead us to infer that correlations modulate Žring rate best when the number of inputs is high and the input rates are low, which corresponds to the most neurally plausible scenario. Thus, our results may have relevance for synchrony-based models of neural computing. In sum, we believe our derivation will prove useful both as a theoretical example highlighting the power of combinatorial methods and for explaining the relative contributions of rate versus synchrony in neural coding. Acknowledgments This work was supported by the National Science Foundation under a CAREER grant to E.N. (grant 9876271).

Coincidence Detector: Analytical Solution

547

References Abeles, M. (1982). Local cortical circuits. New York: Springer-Verlag. Abeles, M. (1991). Corticonics—Neural circuits of the cerebral cortex. Cambridge: Cambridge University Press. Alonso, J. M., Usrey, W. M., & Reid, R. C. (1996). Precisely correlated Žring in cells of the lateral geniculate nucleus. Nature, 383 (6603), 815–819. Bernander, O., Koch, C., & Usher, M. (1994). The effect of synchronized inputs at the single neuron level. Neural Computation, 6(4), 622–641. Bohte, S. M., Spekreijse, H., & Roelfsema, P. R. (2000). The effects of pair-wise and higher-order correlations on the Žring rate of a postsynaptic neuron. Neural Computation, 12, 153–179. Decharms, R. C., & Merzenich, M. M. (1996). Primary cortical representation of sounds by the coordination of action potential timing. Nature, 381, 610–613. de Oliveira, S. C., Thiele, A., & Hoffmann, K. P. (1997). Synchronization of neuronal activity during stimulus expectation in a direction discrimination task. J. Neurosci., 17(23), 9248–9260. Konig, ¨ P., Engel, A. K., & Singer, W. (1996). Integrator or coincidence detector? The role of the cortical neuron revisited. Trends in Neurosciences, 719(4), 130– 137. Niebur, E., Hsiao, S. S., & Johnson, K. O. (2002). Synchrony: A neuronal mechanism for attentional selection? Current Opinion in Neurobiology,12(2), 190–194. Riehle, A., Grun, ¨ S., Diesmann, M., & Aertsen, A. (1997). Spike synchronization and rate modulation differentially involved in motor cortical function. Science, 278, 1950–1953. Salinas, E., & Sejnowski, T. J. (2000). Impact of correlated synaptic input on output Žring rate and variability in simple neuronal models. Journal of Neuroscience, 20(16), 6193–6209. Shadlen, M. N., & Newsome, W. T. (1994). Noise, neural codes and cortical organization. Current Opinion in Neurobiology, 4, 569–579. Softky, W., & Koch, C. (1992). Cortical cells should Žre regularly but do not. Neural Computation, 4(5), 643–646. Softky, W., & Koch, C. (1993). The highly irregular Žring of cortical cells is inconsistent with temporal integration of random EPSPs. J. Neurosci., 13(1), 334–350. Steinmetz, P. N., Roy, A., Fitzgerald, P., Hsiao, S. S., Johnson, K. O., & Niebur, E. (2000). Attention modulates synchronized neuronal Žring in primate somatosensory cortex. Nature, 404, 187–190. Stroeve, S., & Gielen, S. (2001). Correlation between uncoupled conductancebased integrate-and-Žre neurons due to common and synchronous presynaptic Žring. Neural Computation, 13, 2005–2029. Tiesinga, P. H. E., Fellous, J.-M., & Sejnowski, T. J. (2002). Attractor reliability reveals deterministic structure in neuronal spike trains. Neural Computation, 14, 1629–1650. Williams, S. R., & Stuart, G. J. (2002). Dependence of EPSP efŽcacy on synapse location in neocortical pyramidal neurons. Science, 295, 1907–1910. Received May 2, 2002; accepted July 30, 2002.

NOTE

Communicated by Arjen van Ooyen

A Theoretical Model of Axon Guidance by the Robo Code Geoffrey J. Goodhill [email protected] Department of Neuroscience, Georgetown University Medical Center, Washington, D.C. 20007, U.S.A.

After crossing the midline, different populations of commissural axons in Drosophila target speciŽc longitudinal pathways at different distances from the midline. It has recently been shown that this choice of lateral position is governed by the particular combination of Robo receptors expressed by these axons, presumably in response to a gradient of Slit released by the midline. Here we propose a simple theoretical model of this combinatorial coding scheme. The principal results of the model are that purely quantitative rather than qualitative differences between the different Robo receptors are sufŽcient to account for the effects observed following removal or ectopic expression of speciŽc Robo receptors, and that the steepness of the Slit gradient in vivo must exceed a certain minimum for the results observed experimentally to be consistent. 1 Introduction A crucial process in the construction of a nervous system is the establishment of appropriate connectivity. This consists of two stages: the guidance of axons to target regions largely independent of neural activity, followed by the activity-dependent reŽnement of synaptic connections. Much progress has recently been made in understanding the mechanisms of axon guidance in response to molecular cues in the developing nervous system (TessierLavigne & Goodman, 1996; Mueller, 1999; Wilkinson, 2001; Song & Poo, 2001). An emerging theme is that a limited number of evolutionarily conserved axon guidance molecules are reused in many different contexts. When it is necessary to generate several different outcomes in the same region of space and time, an obvious strategy is combinatorial coding. For instance, two guidance molecules could simultaneously attract three populations of axons to different targets if population 1 expressed only one type of receptor, population 2 expressed only the other type, and population 3 expressed both types. Since axons can also be attracted or repelled depending on their internal state (Song, Ming, & Poo, 1997), for N guidance molecules, the number of separate populations that could in principle simultaneously be guided to different locations is of order 3N . About 100 guidance molecules are now known, allowing many more potential outcomes than there are total Neural Computation 15, 549–564 (2003)

c 2003 Massachusetts Institute of Technology °

550

G. Goodhill

connections in any biological nervous system. Combinatorial coding is thus a very efŽcient coding strategy and is used by nervous systems in contexts ranging from axon guidance (Shirasaki & Pfaff, 2002) to information coding by spike trains (Diesman, Gewaltig, & Aertsen, 1999) and odor coding in the olfactory system (Malnic, Hirono, Sato, & Buck, 1999). A particularly well-studied model system in axon guidance is the growth of commissural axons to, across, and beyond the midline (Kaprielian, Imondi, & Runko, 2000). Commissural axons are initially attracted toward the midline (e.g., the insect nerve cord or the vertebrate spinal cord) by a gradient of Netrin released by midline glia (SeraŽni et al., 1994; Kennedy, SeraŽni, de la Torre, & Tessier-Lavigne, 1994; Mitchell et al., 1996). On reaching the midline, they lose their responsiveness to Netrin but become responsive instead to the repellent molecule Slit, which binds to the Robo family of receptors (Brose et al., 1999; Kidd, Bland, & Goodman, 1999). In Drosophila, this currently consists of the three members Robo, Robo2, and Robo3, each of which plays a particular role in enabling commissural axons to cross the midline and not then recross (Rajagopalan, Nicolas, Vivancos, Berger, & Dickson, 2000; Simpson, Kidd, Bland, & Goodman, 2000). After crossing, axons project to speciŽc lateral pathways before turning longitudinally. Recent results from Drosophila (Rajagopalan, Vivancos, Nicolas, & Dickson, 2000; Simpson, Bland, Fetter, & Goodman, 2000) have focused on three speciŽc longitudinal Fas-II positive tracts, referred to as medial, intermediate, and lateral. In this case, the choice of pathway depends on a combinatorial code of Robo receptors expressed by axons after they have crossed the midline—the so-called Robo code. Axons that normally turn in the medial tract express only Robo, axons that normally turn in the intermediate tract express Robo and Robo3, and axons that normally turn in the lateral tract express Robo, Robo2, and Robo3 (see Figure 1A). Although a gradient of Slit emanating from the midline has not yet been directly observed, it has been hypothesized that this gradient drives axons to speciŽc lateral positions based on their combinatorial code of Robo receptors (Rajagopalan, Vivancos, Nicolas, & Dickson, 2000; Simpson, Bland, Fetter, & Goodman, 2000). This hypothesis is supported by loss- and gain-of-function experiments (Rajagopalan, Vivancos, Nicolas, & Dickson, 2000; Simpson, Bland, Fetter, & Goodman, 2000). When particular Robo receptors are deleted, axons often project to a more medial fascicle than normal, whereas when particular Robo receptors are ectopically expressed, axons often project to a more lateral fascicle than normal. The shifts induced by these genetic perturbations are always discrete jumps between fascicles: these axons do not generally turn between fascicles. Both Rajagopalan, Vivancos, Nicolas, and Dickson (2000) and Simpson, Bland, Fetter, and Goodman, (2000) argue that the Robo code targets axons to a general region; then local cues take over to target a particular tract within that region. Although these different Robo receptors all mediate repulsion by Slit, the speciŽc signal transduction pathways they activate inside the growth

Axon Guidance by the Robo Code

A

551

B M

I

L S(x)

robo robo3 robo robo3 robo2

MIDLINE

robo

Slit gradient S(M) S(I) S(L) x0

M

d

M

d

d

I

d

L

X

Figure 1: (A) Schematic diagram of the Robo code. Three different populations of commissural axons (dashed lines) are shown projecting to three different lateral fascicles: M (medial), I (intermediate), and L (lateral). (B) Parameters of the model. x0 D distance of the medial fascicle from the midline, 2d D spacing of the fascicles. S(M), S(I), and S(L) refer to the Slit concentrations at the medial, intermediate, and lateral fascicles, respectively.

cone may be different (discussed in Rajagopalan, Vivancos, Nicolas, & Dickson, 2000). One possibility is that each type of Robo receptor signals qualitatively different information regarding levels of Slit, which would be a combinatorial code in the sense most analogous to the other examples cited above. However, an alternative possibility is that the differences between types of Robo receptors are simply quantitative; each generates the same type of repulsive signal to particular levels of Slit, differing only in degree. Roughly speaking, this is consistent with the experimental results that the more Robo receptors an axon expresses, the more repulsion and thus the further its lateral projection. By analogy with levels of measurement in statistics (Krzanowski, 1988), we refer to the Žrst possibility as a categorical model and the second as a ratio model. In the categorical model, only the categories of Robo receptors expressed by a given axon determine where it projects to, whereas in the ratio model, only the relative proportions matter. The categorical model is consistent with many of the experimental results. For instance, overexpressing Robo does not generally cause a lateral shift (Simpson, Bland, Fetter, & Goodman, 2000). However, a purely categorical model is insufŽcient to explain some results: for instance, axons that normally project laterally still do so in the robo=robo3 double mutant (Simpson, Bland, Fetter, & Goodman, 2000), and medially projecting axons that project to the intermediate zone when robo3 is added can be “supershifted” to the lateral zone by the addition of two copies of the robo3 gene (Rajagopalan, Vivancos, Nicolas, & Dickson, 2000). Results such as these led both Simpson, Bland, Fetter, and Goodman (2000) and Rajagopalan, Vi-

552

G. Goodhill

vancos, Nicolas, and Dickson (2000) to propose a mixed categorical-ratio model, where Robo acts as a category (only the presence of Robo is important, not its precise amount) and Robo2 and Robo3 act in a more graded way. To explore these coding issues in a more quantitative way, we have developed a simple mathematical model of the Robo code based on a pure ratio and a mixed categorical-ratio approach. We show that a pure ratio model could in fact be consistent with the data if the Slit gradient is steep enough. This leads to experimentally testable predictions regarding how the outcome of Robo overexpression experiments might depend on the steepness of the Slit gradient. In addition, for the mixed categorical-ratio model, we show that even this version is consistent with the data only when the Slit gradient is sufŽciently steep. 2 Mathematical Model 2.1 Form of the Slit Gradient. There is no direct evidence concerning the shape of the Slit gradient in vivo. Theoretically, one approach would be to model it as free diffusion from a line source in an inŽnite three-dimensional volume (Crank, 1975; Carslaw & Jaeger, 1959), analogous to a recent model of axon guidance by a target-derived diffusible factor (Goodhill, 1997, 1998). However, data from other systems suggest the diffusion might not be free (Hiramoto, Hiromi, Giniger, & Hotta, 2000). Given this uncertainty, it seems sensible to choose a very generic shape with a parameterized steepness for the Slit gradient. We therefore assume it to be a decaying exponential, S.x/ D Ce¡x=® , where C gives the concentration of Slit at the midline, x is distance from the midline, and ® gives the rate of decrease moving away from the midline. For small ®, Slit is tightly concentrated around the midline, whereas for larger ®, it decays more gradually. 2.2 Generation of Repulsion: Linear Model. In the linear version of the model, we assume that the amount of repulsion induced in growth cones by Slit is given by S.x/ £ R, where R is just a sum of the amount of repulsion transduced by each Robo receptor separately (Nakamoto et al., 1996; Goodhill & Richards, 1999). That is, R D r for axons normally projecting to the medial fascicle, R D r C r3 for intermediate axons, and R D r C r3 C r2 for lateral axons, where r, r3 , and r2 give the “net” amount of repulsion due to each Robo receptor (for the mixed categorical-ratio model, R is assumed to depend on only r2 and r3 ). The values of the r’s that are consistent with the data come out of the model. For experiments where a second copy of a gene is inserted, this net repulsion is assumed to be doubled. That is, the net repulsion is assumed to be equal to the product of the number of receptors times the strength of repulsive signaling through a single receptor of that type. For Robo overexpression experiments, where the degree of ampliŽcation of r is uncertain, we introduce a multiplication

Axon Guidance by the Robo Code

553

parameter n. The values of n that are consistent with the data emerge from the model. Each axon is assumed to be repelled by Slit until the amount of repulsion S.x/£R reaches a certain threshold T that is the same for all axons. Axons are then assumed to be attracted by local cues toward the nearest of the medial, intermediate, or lateral fascicles. The model presented here concerns only how axons target a particular general area based on their Robo code and does not explicitly address how local cues reŽne their positions within these general areas (we return to this in section 4). This means that the positions that axons reach deŽned purely by Slit repulsion can be anywhere in the region of a particular fascicle, provided they are nearer to that fascicle than neighboring fascicles. Fascicles are assumed to be evenly spaced, distance 2d apart, with the medial fascicle distance x0 from the midline (see Figure 1B). We also assume that for axons to project to the medial fascicle, their turning position must be more than x0 ¡ d from the midline, and for axons to project to the lateral fascicle, their turning position must be less than x0 C 5d from the midline. Based on data such as Figure 5A of Simpson, Bland, Fetter, and Goodman (2000), we chose x0 D 4, d D 1 (in arbitrary units), so that the three fascicles run at distances 4, 6, and 8 from the midline (absolute distances are not indicated in Simpson, Bland, Fetter, & Goodman, 2000, or Rajagopalan, Vivancos, Nicolas, & Dickson, 2000). These units set the scale for ®. The constraints imposed by the experimental data can now be expressed as a set of inequalities that must all be simultaneously satisŽed. As an example, consider the normal targeting of axons projecting to the intermediate fascicle. The equation for targeting to position x is S.x/.r C r3 / D T:

(2.1)

Substituting the exponential form for S gives e¡x=® .r C r3 / D T=C: The ratio T=C sets the absolute scale for the r’s. However, since we are interested only in the relative values of the r’s, we can without loss of generality set T=C D 1. This yields a value for x of x D ® log.r C r3 /:

(2.2)

This target position must be closer to the intermediate tract than either the medial or lateral tracts. Thus we require (cf. Figure 1) that x0 C d < ® log.r C r3 / < x0 C 3d or equivalently e

x0 Cd ®

< r C r3 < e

x0 C3d ®

:

554

G. Goodhill

Table 1 shows a set of inequalities that can be derived by this approach from the experimental data of Rajagopalan, Vivancos, Nicolas, and Dickson (2000) and Simpson, Bland, Fetter, and Goodman (2000) (see particularly the schematic Figure 4 in Simpson, Bland, Fetter, & Goodman, 2000). Cases labeled “crossing defect” are assumed to represent problems with crossing rather than lateral termination zone and are not considered further in our analysis. In case 9 in Table 1 (axons that normally project to the lateral fascicle in the robo2 mutant), the axons seem undecided between the intermediate and lateral fascicles. This was not included in our analysis, and we return to this issue in section 4. Not all of these inequalities are independent. For instance, if ® log.r3 / > x0 C d (index 5 in Table 1), then it is certainly true that ® log.r C r3 / > x0 C d (index 2 in Table 1), and the latter constraint is superuous. However, we explicitly include all the constraints in Table 1 since it makes it easier to see which piece of datum each is based on and what would happen if each were deleted or altered. 2.3 Generation of Repulsion: Nonlinear Model. A more realistic picture of binding kinetics than the linear version presented above is [RL] D

[R][L] ; [L] C Kd

where [R] is the receptor concentration, [L] is the ligand concentration, [RL] is the concentration of the product, and Kd is the dissociation constant (Gutfreund, 1995). When [L] ¿ Kd , this simpliŽes to the linear version (with Kd absorbed into the threshold T). The three Robo receptors have similar Kd s for binding Slit, in the range 10 to 40 nM (Simpson, Kidd, Bland, & Goodman, 2000), and we consider these as equal. The analog of equation 2.1 is then S.x/ .r C r3 / D T: S.x/ C Kd Substituting the exponential form for S.x/ and dividing by C gives e¡x=® .r C r3 / D T; C 1=Nc

e¡x=®

(2.3)

where cN D C=Kd , the normalized Slit concentration at the midline. Again, T simply sets the overall scale of the r’s, so we can set it to 1 without loss of generality. The analog to equation 2.2 is now x D ® log[Nc.r C r3 ¡ 1/]; and an analogous set of constraints to Table 1 can be generated (not shown).

WT WT WT r¡ r¡ r¡ r2 ¡ r2 ¡ r2 ¡ r3 ¡ r3 ¡ r3 ¡ r¡, r2 ¡ r¡, r2 ¡ r¡, r2 ¡ r¡, r3 ¡ r¡, r3 ¡ r¡, r3 ¡ r2 ¡, r3 ¡ r2 ¡, r3 ¡ r2 ¡, r3 ¡ nr nr nr r2 C r3 C r2 C, r3 C 2r3 C 2r2 C, r3 C

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29

Medial Intermediate Lateral Medial Intermediate Lateral Medial Intermediate Lateral Medial Intermediate Lateral Medial Intermediate Lateral Medial Intermediate Lateral Medial Intermediate Lateral Medial Intermediate Lateral Medial Medial Medial Medial Medial

Population

r2 r r r nr nr, r3 nr, r3 , r2 r, r2 r, r3 r, r3 , r2 r, 2r3 r, r3 , 2r2 < < < < < < < < < < < <

< < <

x0 ¡ d x0 ¡ d x0 C 3d

x0 C 3d x0 ¡ ¡d x0 ¡ ¡d x0 ¡ d x0 ¡ d x0 C d x0 C 3d x0 C 3d x0 C d x0 C 3d x0 C 3d x0 C 3d

< < < <

x0 C d x0 C 3d x0 ¡ d x0 C d

r3 r2 , r3 r r, r3 r, r3 r r r, r2 r3 r3

< < <

x0 ¡ d x0 C d x0 C 3d

r r, r3 r, r3 , r2

Robo Code ® log.r/ ® log.r C r3 / ® log.r C r2 C r3 / Crossing defect ® log.r3 / ® log.r2 C r3 / ®log.r/ ® log.r C r3 / See text ® log.r/ ® log.r/ ® log.r C r2 / Crossing defect Crossing defect Crossing defect Crossing defect Crossing defect ® log.r2 / ® log.r/ ® log.r/ ® log.r/ ® log.nr/ ® log.nr C r3 / ® log.nr C r3 C r2 / ® log.r C r2 / ® log.r C r3 / ® log.r C r3 C r2 / ® log.r C 2r3 / ® log.r C r3 C 2r2 /

Equation

< < < < < < < < < < < <

< < <

< < < <

< < <

x0 C 5d x0 C d x0 C d x0 C d x0 C d x0 C 3d x0 C 5d x0 C 5d x0 C 3d x0 C 5d x0 C 5d x0 C 5d

x0 C d x0 C d x0 C 5d

x0 C 3d x0 C 5d x0 C d x0 C 3d

x0 C d x0 C 3d x0 C 5d

Notes: r¡ indicates null mutant for Robo and so on; r2 C indicates ectopic expression for Robo2 and so on; 2r2 indicates two copies of Robo2. “Population” identiŽes the axons involved in each case by where they would normally project to in the wild type (WT).

Mutant

Index

Table 1: Complete Set of Constraints for the Linear Model Derived from Wild Type and Mutants.

Axon Guidance by the Robo Code 555

556

G. Goodhill

3 Results 3.1 Linear Model. In the mixed categorical-ratio model, it is assumed that only r2 and r3 act in a quantitative way: the set of constraints used is as in Table 1, except that r is ignored (i.e., set to zero). For instance, constraint 2 now becomes x0 C d < ® log.r3 / < x0 C 3d (or e.x0 Cd/=® < r3 < e.x0 C3d/=® ). Since there are now only two variables, r2 and r3 , each constraint can be represented as a straight line in the r2 , r3 plane. Figure 2 shows how the constraints conspire to limit the allowable combinations of r2 and r3 that satisfy the inequalities. This is illustrated for two gradient steepnesses, ® D 1 and ® D 2. It is immediately obvious from Figure 2 that for ® & 2, there can be no more solutions. For the pure ratio model, there are the additional parameters r and n, giving four dimensions in total, which is hard to represent graphically. To investigate which sets f®; r; r2 ; r3 g satisfy all the inequalities, an exhaustive search was performed over the parameter space ri 2 [0; rmax ] in increments rinc for various values of ®. For certain values of ®, we found many different combinations of r, r3 , and r2 , which satisfy the constraints. The precise number of solutions per value of ® depends on rinc , which was varied for different ®. The total number of solutions is not important here, providing solutions exist, but we ensured that there were always at least 100 so that averaging over them would be meaningful. Since only the ratios of the r’s are of interest, they were normalized by r3 . The results are shown in Table 2. Means and standard deviations for the r’s are quoted: the standard deviations give a measure of the size of the box in the four-dimensional space in which the solutions exist. Several trends are apparent: ² As for the mixed categorical-ratio model, there are no solutions for ® & 2. ² The values of the r’s (i.e., the relative strength of signaling of the different Robo receptors) are always ordered r2 > r3 > r. ² The ratios of the r values become more extreme as ® decreases (i.e., the gradient becomes steeper). For instance, for n D 2, the ratios r:r3 :r2 go from 0.4:1:2.2 for ® D 2:0 to 0.007:1:20.1 for ® D 0:5.

² For a very steep gradient (® D 0:5), values of n (the ampliŽcation factor for Robo overexpression experiments) up to 25 are allowable. As ® decreases further, even larger values of n are allowed (results not shown). As the gradient becomes shallower, n becomes more restricted, until Žnally for ® D 2, solutions exist only for n D 2. 3.2 Nonlinear Model. Comparing equations 2.2 and 2.3, it can be seen that the nonlinear model with S.x/ D e¡x=® is formally equivalent to a linear e¡x=® model with S.x/ D e¡x=® . Thus, we may think of the nonlinear version C1=Nc

Axon Guidance by the Robo Code

557 120

10000

100

8000

Robo2

Robo2

80 6000

4000

40

2000

0 0

60

20

2000

4000

6000

Robo3

8000

0 0

10000

20

40

20

40

20

40

20

40

60

80

100

120

60

80

100

120

60

80

100

120

60

80

100

120

Robo3

120

10000

100

8000

Robo2

Robo2

80 6000

4000

40

2000

0 0

60

20

2000

4000

6000

Robo3

8000

0 0

10000

Robo3

120

10000

100

8000

Robo2

Robo2

80 6000

4000

40

2000

0 0

60

20

2000

4000

6000

Robo3

8000

0 0

10000

Robo3

120

10000

100

8000

Robo2

Robo2

80 6000

4000

40

2000

0 0

60

20

2000

4000

6000

Robo3

8000

10000

0 0

Robo3

Figure 2: Interaction of constraints for the mixed categorical-ratio model (linear version). (Left) ® D 1:0. (Right) ® D 2:0 (x0 D 4, d D 1). The shaded box in each case represents the region of r2 –r3 space allowed by the constraints. First row: cases 2 and 12 from Table 1. Second row: case 28 is added. Third row: case 6 is added. Fourth row: case 29 is added. It can be seen that the allowable region of space shrinks as ® increases (the Slit gradient becomes shallower).

558

G. Goodhill

Table 2: Values of r, r2 , and r3 Obtained for the Pure Ratio Model (Linear Version). n

® D 0:5

® D 1:0

® D 2:0

2

0:0067 § 0:0038 20 § 11 0:0047 § 0:0026 20 § 11 0:0035 § 0:0018 20 § 11 0:0030 § 0:0015 20 § 11 0:0025 § 0:0011 20 § 11 0:0021 § 0:0008 20 § 11 0:0017 § 0:0007 20 § 11 0:001 § 0:0002 20 § 11 No solutions

0:065 § 0:026 3:3 § 1:2 0:048 § 0:015 3:3 § 1:2 0:039 § 0:010 3:3 § 1:2 0:035 § 0:007 3:3 § 1:2 0:031 § 0:005 3:3 § 1:2 No solutions

0:35 § 0:04 2:2 § 0:2 No solutions

No solutions

No solutions

No solutions

No solutions

No solutions

No solutions

3 4 5 6 7 10 25 30

No solutions No solutions No solutions No solutions

Notes: n is the increase in repulsion compared to wild type in Robo ectopic expression experiments; ® is the steepness of the Slit gradient. Shown are the mean and standard deviation of the values obtained for r (above) and r2 (below) normalized by r3 (i.e., r3 D 1).

as simply the linear model with a different Slit gradient. In comparing different gradients, the overall scaling is unimportant since in the model, we are interested only in the ratio of the r’s. Thus, signiŽcant differences are expected only between the results for the linear and nonlinear models when the effective gradients have signiŽcantly different shapes in the region between x D 4 and x D 8. For the range of ® considered above, this occurs only when cN À 1 (see Figure 3). Table 3 shows results for the pure ratio model for cN D 100, that is, a midline Slit concentration 100Kd (® D 1). This implies Slit concentrations at the medial, intermediate, and lateral fascicles of 65Kd , 20Kd , and 3Kd , respectively (650 nM, 200 nM, and 30 nM for Kd D 10 nM). There are slight differences from the linear model, but overall the general trends are the same. The numerical differences are smaller for smaller cN (data not shown). 4 Discussion The model presented here is a way of simplifying a varied and complex set of experimental results so as to attempt to extract certain key pieces of quantitative information about the system. A number of biologically relevant conclusions can be drawn. Perhaps the most important of these is that a pure ratio model can be consistent with the data, despite the fact

Axon Guidance by the Robo Code

559

A

B 1

1

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

0 0

2

4

6

8

10

0 0

2

4

6

8

10

Figure 3: Comparison of an exponential Slit gradient with the effective gradient shape in the nonlinear model. The horizontal axis is distance from the midline; the vertical axis is Slit concentration (arbitrary units). (A) From left to right, effective gradient in nonlinear model for cN D 1, exponential gradient, effective gradient in nonlinear model for cN D 10, 100. (B) Same gradients, but normalized so that the Slit concentration at distance 4 is always 1. From left to right, exponential gradient, effective gradient in nonlinear model for cN D 1 (indistinguishable from exponential), cN D 10, 100. Note that the effective gradient shape diverges signiŽcantly from an exponential only when cN À 1. For all curves, ® D 1.

that Simpson, Bland, Fetter, and Goodman (2000) and Rajagopalan, Vivancos, Nicolas, and Dickson (2000) did not generally Žnd lateral shifts when Robo was overexpressed. The model predicts that arbitrarily large degrees of overexpression of Robo may indeed not cause shifts in a ratio model, provided the Slit gradient is sufŽciently steep. The reason for this is actually quite intuitive. The ratio of Slit concentration at the intermediate compared ¡6=® to medial fascicles in the linear model is ee¡4=® D e¡2=® . As ® becomes smaller, this ratio becomes smaller; that is, for steeper gradients, there is a larger disparity between the concentrations at the intermediate and medial fascicles. Since in the linear model the repulsion due to Robo induced by Slit is given by r £ S.x/, a larger Slit disparity means that a larger increase in Robo expression is possible before a shift to the intermediate fascicle is induced. Thus, by precisely measuring the actual level of Robo in overexpression experiments where no shifts occur, it should be possible to predict based on the model a minimum steepness for the Slit gradient in vivo. Similarly, the pure ratio model predicts that there should be a certain level of overexpression of Robo at which lateral shifts can be made to occur. Interestingly, Simpson, Bland, Fetter, and Goodman (2000) did in fact Žnd one GAL4 line (15J2) where increased Robo can cause a lateral shift of the dMP2 and vMP2 axons. A possible explanation suggested by the model is that it is only in this line that Robo levels exceed that required for a lateral shift. A robust result of both the pure ratio and mixed categorical-ratio model is that the experimental data are consistent only when the Slit gradient is

560

G. Goodhill

Table 3: Values of r, r2 , and r3 Obtained for the Pure Ratio Model (Nonlinear Version). n

® D 0:5

® D 1:0

® D 2:0

2

0:0067 § 0:0038 20 § 11 0:0046 § 0:0025 20 § 11 0:0036 § 0:0019 20 § 11 0:0030 § 0:0015 20 § 11 0:0025 § 0:0012 20 § 11 0:0022 § 0:0010 20 § 11 0:0016 § 0:0007 20 § 11 0:00083 § 0:00016 20 § 11 0:00083 § 0:00016 20 § 11 No solutions

0:17 § 0:028 3:4 § 1:2 No solutions

No solutions

No solutions

No solutions

No solutions

No solutions

No solutions

No solutions

No solutions

No solutions

No solutions

No solutions

No solutions

No solutions

No solutions

No solutions

No solutions

No solutions

3 4 5 6 7 10 25 30 35

No solutions

Notes: n is the increase in repulsion compared to wild type in Robo ectopic expression experiments; ® is the steepness of the Slit gradient. Shown are the mean and standard deviation of the values obtained for r (above) and r2 (below) normalized by r3 (i.e., r3 D 1).

sufŽciently steep. For the linear model, the cut-off is about ® & 2. For the nonlinear model, the cut-off depends on cN, the midline concentration compared to Kd . When cN . 1, the cut-off is similar to the linear model; for higher values of cN, a slightly steeper gradient is required. According to the linear model, the shallowest allowable Slit gradient consistent with the data has a concentration at the medial fascicle (position x0 D 4) of approximately e¡4=2 D 0:14 times its concentration at the midline, and corresponding values 0:05 and 0:03 at the intermediate and lateral fascicles, respectively. The exact numbers are dependent on the assumption of an exponential gradient shape; while there is no direct experimental evidence for this shape, there is no direct evidence for any other shape either. The exponential function provides a convenient way of exploring gradients varying from very steep (small ®) to very shallow (large ®) and has been used to model the gradient in previous theoretical analyses of axon guidance such as Gierer (1987). When experimental data about Slit gradients in vivo become available, these could be used in the model instead. Although changing the gradient function will likely change the precise numerical results, it is unlikely to have a strong effect on the general conclusions of the model.

Axon Guidance by the Robo Code

561

What implications does this work have for our understanding of the coding strategies employed in axon guidance? In a strict sense, the pure ratio model is not a combinatorial code, since the receptors are interchangeable with appropriate scaling. In this case, why have three receptors when in principle one would do? One possibility is that more than one receptor type is already required by commissural axons for midline crossing (Rajagolapalan, Vivancos, Nicolas, & Dickson, 2000; Simpson, Bland, Fetter, & Goodman, 2000), and since this diversity is available, it is simply reused. However, an alternative possibility is that it might not be possible to robustly achieve the widely but precisely different levels of expression that would be required of a single receptor. From Table 2, for ® D 0:5, the relative levels of expression needed for the putative single receptor would be 0:0067 for axons projecting to the medial tract, 1:0067 for axons projecting to the intermediate tract, and 21:0067 for axons projecting to the lateral tract. These differ by a factor of about 3000. Since the number of receptors expressed on the surface of a growth cone is probably no more than about 10,000, the number of receptors present on the growth cones of medially projecting axons would need to be about 3. Even if it was possible to achieve this precisely, the response of these axons to a Slit gradient would then be overwhelmed by noise (Berg & Purcell, 1977; Goodhill & Urbach, 1999). While it is true that for the shallowest gradients allowable by the model, these demands are more reasonable (e.g., for ® D 2:0, the difference in expression required would be about a factor of ¼ 10), using multiple receptors with different strengths of signaling is clearly a more robust strategy overall. The model presented here does not consider the precise timing of robo expression, that is, the way the expression of each robo receptor varies as axons traverse the midline and then project laterally. It also does not consider how local cues reŽne axonal targeting within a general region. One hypothesis is that each speciŽc Fas II tract releases an attractive factor, and it is the balance of this attractive force with the Slit repulsion that guides axons to a speciŽc tract. To include such a mechanism in the model would, however, require introducing additional parameters whose values are not currently constrained by experimental data. Also, there is a general problem with this hypothesis in that such an attractive force would necessarily decay symmetrically on each side of the Fas II tract, whereas the Slit repulsion is asymmetric on each side of the tract. The current model instead remains agnostic about the nature of this local focusing mechanism. The model also does not take into account the fact that many of the mutant and misexpression results listed in Table 1 are in reality just statistical tendencies and do not occur in 100% of cases (this is particularly true in case 9). A possible extension of the model would include probabilistic rules, for instance, that there is noise in the measurement of the level of Slit present (Berg & Purcell, 1977; Goodhill & Urbach 1999). Although we have shown that qualitative differences between robo receptors are not required to account for the lateral targeting of commis-

562

G. Goodhill

sural axons in Drosophila, it does not rule out the possibility that qualitative differences play a role. There are several other indications that qualitative differences between Robo and Robo2=Robo3 do exist (Simpson, Kidd, Bland, & Goodman, 2000). Robo2 and Robo3 are positioned close by chromosomally and have closely related sequences. Their intracellular capabilities are different from Robo; for instance, unlike Robo, they lack an Enabled binding site. The different Robos certainly play qualitatively different roles in the midline crossing of commissural axons (Simpson, Kidd, Bland, & Goodman, 2000) and in the growth and guidance of motor neuron dendrites (Godenschwege et al., 2002). Rajagopalan, Vivancos, Nicolas, and Dickson (2000) report that experiments are now under way to generate chimeric Robo receptors, where, for example, the extracellular domain of Robo is coupled to the intracellular domain of Robo2. This will give information about the relative importance of the different signal transduction pathways that Robo and Robo2=Robo3 activate. The model we have presented so far combines extra- and intracellular effects into a single effective strength of signaling. An extension would be to split each r value into the product re £ ri , representing extra- and intracellular inuences, respectively. When available, the chimeric receptor data could then be used to constrain the additional three parameters and determine whether there are any sets of values that are still consistent with a purely quantitative model. Acknowledgments I thank the anonymous referees for many helpful suggestions for improving this article. References Berg, H. C., & Purcell, E. M. (1977). Physics of chemoreception. Biophysical Journal, 20, 193–219. Brose, K., Bland, K. S., Wang, K. H., Arnott, D., Henzel, W., Goodman, C. S., Tessier-Lavigne, M., & Kidd, T. (1999). Slit proteins bind robo receptors and have an evolutionarily conserved role in repulsive axon guidance. Cell, 96, 795–806. Carslaw, H. S., & Jaeger, J. C. (1959). Conduction of heat in solids (2nd ed.). Oxford: Clarendon Press. Crank, J. (1975). The mathematics of diffusion (2nd ed.). Oxford: Clarendon Press. Diesmann, M., Gewaltig, M. O., & Aertsen, A. (1999). Stable propagation of synchronous spiking in cortical neural networks. Nature, 402, 529–533. Gierer, A. (1987). Directional cues for growing axons forming the retinotectal projection. Development, 101, 479–489. Godenschwege, T. A., Simpson, J. H., Shan, Y., Bashaw, G. J., Goodman, C. S., & Murphey, R. K. (2002). Ectopic expression in the giant Žber system of

Axon Guidance by the Robo Code

563

Drosophila reveals distinct roles for Rounabout (Robo) Robo2, and Robo3 in dendritic guidance and synaptic connectivity. J. Neurosci., 22, 3117–3129. Goodhill, G. J. (1997). Diffusion in axon guidance. European Journal of Neuroscience, 9, 1414–1421. Goodhill, G. J. (1998). Mathematical guidance for axons. Trends in Neurosciences, 21, 226–231. Goodhill, G. J., & Richards, L. J. (1999). Retinotectal maps: Molecules, models, and misplaced data. Trends in Neurosciences, 22, 529–534. Goodhill, G. J., & Urbach, J. S. (1999). Theoretical analysis of gradient detection by growth cones. Journal of Neurobiology, 41, 230–241. Gutfreund, H. (1995). Kinetics for the life sciences. Cambridge: Cambridge University Press. Hiramoto, M., Hiromi, Y., Giniger, E., & Hotta, Y. (2000). The Drosophila Netrin receptor Frazzled guides axons by controlling Netrin distribution. Nature, 406, 886–889. Kaprielian, Z., Imondi, R., & Runko, E. (2000). Axon guidance at the midline of the developing CNS. Anat. Rec., 261, 176–197. Kennedy, T. E., SeraŽni, T., de la Torre, J. R., & Tessier-Lavigne, M. (1994). Netrins are diffusible chemotropic factors for commissural axons in the embryonic spinal cord. Cell, 78, 425–435. Kidd, T., Bland, K. S., & Goodman, C. S. (1999). Slit is the midline repellent for the robo receptor in Drosophila. Cell, 96, 785–794. Krzanowski, W. J. (1988). Principles of multivariate analysis: A user’s perspective. New York: Oxford University Press. Malnic, B., Hirono, J., Sato, T., & Buck, L. B. (1999). Combinatorial receptor codes for odors. Cell, 96, 713–723. Mitchell, K. J., Doyle, J. L., SeraŽni, T., Kennedy, T. E., Tessier-Lavigne, M., Goodman, C. S., & Dickson, B. J. (1996). Genetic analysis of Netrin genes in Drosophila: Netrins guide CNS commissural axons and peripheral motor axons. Neuron, 17, 203–215. Mueller, B. K. (1999). Growth cone guidance: First steps towards a deeper understanding. Annu. Rev. Neurosci., 22, 351–388. Nakamoto, M., Cheng H. J., Friedman, G. C., Mclaughlin, T., Hansen, M. J., Yoon, C. H., O’Leary, D. D. M., & Flanagan, J. G. (1996). Topographically speciŽc effects of ELF–1 on retinal axon guidance in–vitro and retinal axon mapping in–vivo. Cell, 86, 755–766. Rajagopalan, S., Nicolas, E., Vivancos, V., Berger, J., & Dickson, B. J. (2000). Crossing the midline: Roles and regulation of Robo receptors. Neuron, 28, 767–777. Rajagopalan, S., Vivancos, V., Nicolas, E., & Dickson, B. J. (2000). Selecting a longitudinal pathway: Robo receptors specify the lateral position of axons in the Drosophila CNS. Cell, 103, 1033–1045. SeraŽni, T., Kennedy, T. E., Galko, M. J., Mirzayan, C., Jessell, T. M., & TessierLavigne, M. (1994). The netrins deŽne a family of axon outgrowth–promoting proteins homologous to C. elegans UNC–6. Cell, 78, 409–424. Shirasaki, R., & Pfaff, S. L. (2002). Transcriptional codes and the control of neuronal identity. Annu. Rev. Neurosci., 25, 251–281.

564

G. Goodhill

Simpson, J. H., Kidd, T., Bland, K. S., & Goodman, C. S. (2000). Short-range and long-range guidance by slit and its Robo receptors: Robo and Robo2 play distinct roles in midline guidance. Neuron, 28, 753–766. Simpson, J. H., Bland, K. S., Fetter, R. D., & Goodman, C. S. (2000). Short range and long range guidance by Slit and its Robo receptors: A combinatorial code of Robo Receptors controls lateral position. Cell, 103, 1019–1032. Song, H., Ming, G. L., & Poo, M.-M. (1997). CAMP-induced switching in turning direction of nerve growth cones. Nature, 388, 275–279. Song, H., & Poo, M.-M. (2001). The cell biology of neuronal navigation. Nat. Cell. Biol., 3, E81–88. Tessier-Lavigne, M., & Goodman, C. S. (1996). The molecular biology of axon guidance. Science, 274, 1123–1133. Wilkinson, D. G. (2001). Multiple roles of Eph receptors and ephrins in neural development. Nat. Rev. Neurosci., 2, 155–164.

Received April 25, 2002; accepted August 5, 2002.

LETTER

Communicated by Misha Tsodyks

Spike-Driven Synaptic Dynamics Generating Working Memory States Daniel J. Amit [email protected]

Gianluigi Mongillo [email protected] Dipartimento di Fisica, Universita’ di Roma, La Sapienza, 00185 Rome, Italy, and Racah Institute of Physics, Hebrew University, Jerusalem, Israel

The collective behavior of a network, modeling a cortical module of spiking neurons connected by plastic synapses is studied. A detailed spikedriven synaptic dynamics is simulated in a large network of spiking neurons, implementing the full double dynamics of neurons and synapses. The repeated presentation of a set of external stimuli is shown to structure the network to the point of sustaining working memory (selective delay activity). When the synaptic dynamics is analyzed as a function of pre- and postsynaptic spike rates in functionally deŽned populations, it reveals a novel variation of the Hebbian plasticity paradigm: in any functional set of synapses between pairs of neurons (e.g., stimulated– stimulated, stimulated–delay, stimulated–spontaneous), there is a Žnite probability of potentiation as well as of depression. This leads to a saturation of potentiation or depression at the level of the ratio of the two probabilities. When one of the two probabilities is very high relative to the other, the familiar Hebbian mechanism is recovered. But where correlated working memory is formed, it prevents overlearning. Constraints relevant to the stability of the acquired synaptic structure and the regimes of global activity allowing for structuring are expressed in terms of the parameters describing the single-synapse dynamics. The synaptic dynamics is discussed in the light of experiments observing precise spike timing effects and related issues of biological plausibility. 1 Introduction Long-term modiŽcations of the synaptic efŽcacies are believed to affect information processing in the brain. The occurrence of such modiŽcations would be manifest as the appearance of new patterns of neural activity. In both IT and PF cortex of monkeys, trained to perform a delayed-matchto-sample (DMS) task, small neural assemblies have been found to exhibit selective, persistent enhanced spike rates during the delay interval between successive stimulations (Miyashita, 1988; Miller, Erickson, & DesNeural Computation 15, 565–596 (2003)

c 2003 Massachusetts Institute of Technology °

566

D. Amit and G. Mongillo

imone, 1996). This kind of activity is related to the ability of the monkey to hold an item in memory (Amit, 1995); indeed, it reects the last stimulus seen. It will be referred to as working memory (WM) activity. WM appears only after a substantial training stage, during which, presumably, the local synaptic structure is modiŽed by the incoming stimuli.1 Then new patterns of neural activity appear. Spiking neural network models indicate that the phenomenology observed in DMS tasks can be reproduced by coherent modiŽcations of the synaptic efŽcacies (Amit & Brunel, 1997a; Amit, 1998). SpeciŽcally, the mean excitatory synaptic efŽcacy among neurons belonging to the selective population must increase, while the efŽcacies in synaptic population connecting selective to other neurons must decrease. Hence, one expects that the neural activity evoked by presentation of stimuli during the learning stage produces such modiŽcations so that WM can appear. This raises the question of how the synaptic dynamics is related to the neural activity. A simulation like the one presented here has multiple utility. First, it calls into focus certain features of the single synapse that should be clariŽed in biological investigation. Second, it provides a more complete benchmark (relative to simulations with Žxed synapses), in the bridge between experiment and theory (Amit, 1998). And once the simulation clariŽes the essential features of the microscopic synaptic dynamics, simpliŽed (and hence faster) learning algorithms can be identiŽed, to be used in simulating complex behavioral situations. We Žrst review the experimental protocols used to induce long-term synaptic modiŽcations. Then we recapitulate a general model of spikedriven synaptic plasticity, proposed by Fusi, Annunziato, Badoni, Salamon, and Amit (2000) and discuss the extent to which this dynamics satisŽes general desiderata and concords with experimental Žndings. The plastic synapse is then embedded in a full-scale simulation of a large network of spiking neurons. The synaptic dynamics is propelled purely by the actual spikes emitted by the neurons, as a consequence of a preassigned protocol of stimulus presentation, mimicking those used in the DMS tasks. It is shown that WM is actually formed in the process and that its slow formation can be qualitatively understood. 2 Experimental Protocols for LTP and LTD Long-term potentiation (LTP) and long-term depression (LTD) are persistent changes in synaptic efŽcacy that are expected to be induced by patterns of pre- and postsynaptic activity. Hebbian tradition had these changes re-

1 There are zones in cortex where the synaptic structure giving rise to delay activity may be built in (Kritzer & Goldman-Rakic, 1995; Compte, Brunel, Goldman-Rakic, & Wang, 2000). Those are outside the scope of our discussion.

Spike-Driven Synaptic Dynamics Generating Working Memory States

567

lated to pre- and postsynaptic spike rates. A bridge between spikes and plasticity has been recently conŽrmed by detailed experimental studies. SpeciŽc, well-controlled stimulation protocols have been developed that reliably induce LTP and LTD in hippocampal and cortical synapses. However, certain questions remain open. Most important, we consider the passage from spike-induced synaptic plasticity to rate-dependent plasticity, so essential for the learning of working memory. The difŽculty in providing such a bridge, from spikes to rates, opens up questions concerning the dependence of the effects observed on the speciŽc details of the protocols used (see section 7.1). This calls for a closer identiŽcation of the neural activity parameters that control the synaptic transitions and determine the type of modiŽcation that occurs: LTP or LTD. Some protocols for the induction of LTP and LTD involve repetitive presynaptic stimulation by electrical pulses (Dudek & Bear, 1992). The induction consists of a Žxed number of several hundred (e.g., 600–900) pulses delivered at various frequencies. No experimental control is exercised on the postsynaptic activity, which is determined by the presynaptic activation. The experiments are carried out in slices from the CA1 region of the hippocampus, stimulating the Schaffer collateral afferents. Depending on the stimulation frequency, LTP or LTD is induced. The same protocols also induce reliably synaptic changes in the neocortex. High-frequency stimulation is particularly effective in inducing LTP. On the other hand, presynaptic activation at low frequencies is likely to trigger homosynaptic LTD. More precisely, no plasticity is observed at frequencies less than 0:1 Hz, LTD is observed using 1 Hz stimulation, and LTP is observed using stimulation frequencies greater than 10 Hz (Bear, 1996). This phenomenology is explained in the following way (Bear, 1996): Upand down-regulation of synaptic strength depends on the postsynaptic intracellular concentration of calcium, which in turn depends on the level of postsynaptic membrane potential. If the level of [Ca2C ] exceeds a threshold, the synapse tends to be potentiated. Otherwise, it tends to be depressed. High-frequency stimulation of a cell produces a higher level of postsynaptic depolarization, due to the fast temporal summation of excitatory postsynaptic potentials (EPSPs) and, hence, higher levels of [Ca2C ]. Stimulation at 10 Hz produces, on average, neither LTD nor LTP, which may indicate a critical level of postsynaptic [Ca2C ] corresponding to the presynaptic activation. This hypothesis seems corroborated by other experiments (Steele & Mauk, 1999) in which the level of recurrent inhibition during the stimulation is pharmacologically controlled by using either agonist or antagonist of the GABA receptor. When antagonist is applied, the threshold frequency for LTP is decreased; if agonist is present, it is increased. When GABA antagonist is present, the recurrent inhibition is less effective. Accordingly, the depolarization level allowing for LTP can be reached with lower frequencies. The reasoning is analogous in the presence of the GABA agonist.

568

D. Amit and G. Mongillo

More recently, other stimulation protocols have been developed to induce long-term synaptic modiŽcations in which postsynaptic activity also is experimentally controlled (Markram, Lubke, ¨ Frotscher, & Sakmann, 1997; Bi & Poo, 1998). In this experimental setup, both pre- and postsynaptic events are evoked by injecting current pulses into the cell body. Events are then coupled according to various protocols, varying either the temporal order between the pre- and postsynaptic events or the number and/or the frequency of pairing. The role of the number of pre- and postsynaptic spike pairings is studied in Markram et al. (1997). A postsynaptic spike is evoked several milliseconds after presynaptic emission. Such pairing is repeated 2, 5, and 10 times, and the sequence is repeated 10 times every 4 seconds. The postsynaptic spikes are evoked at 20 Hz. In all these cases, LTP is observed. Subsequently, the frequency dependence has been studied according to the following protocol: Žve presynaptic spikes are paired, as previously described, with postsynaptic spikes at various frequencies. Again, the pairing is repeated 10 times every 4 seconds. It was found that no potentiation occurs if the frequency is lower than 10 Hz. For the dependence on the relative timing between pre- and postsynaptic spikes, it is found that LTP occurs if the presynaptic spike precedes the postsynaptic one, while if the presynaptic spike follows the postsynaptic one, the synapse undergoes LTD. It is also noted that if the temporal interval between the two events is too large, no synaptic modiŽcations occur. Bi and Poo (1998) determined the exact temporal window in which the occurrence of both pre- and postsynaptic emission is effective to induce long-term synaptic modiŽcations. They coupled 60 presynaptic spikes delivered at 1 Hz with postsynaptic spikes by varying the temporal order of occurrence of the events (pre-post or post-pre) and the time difference between them. Synapses are strengthened if the presynaptic spikes precede postsynaptic spikes by less than 20 milliseconds and are weakened if instead presynaptic spikes follow postsynaptic spikes within the same interval. Markram et al. (1997) did not Žnd potentiation for presynaptic activation rates lower than 10 Hz, while Bi and Poo (1998) did Žnd both potentiation and depression at 1 Hz. The seemingly inconsistent behavior could be due to the fact that while Markram et al.’s experiment was carried out with cortical cells, Bi and Poo worked with hippocampal synapses. An alternative explanation is based on the difference in the number of pairings in the two cases. In other words, 5 pairings at 1 Hz do not provoke LTP, whereas 60 at the same frequency increase the synaptic efŽcacy. This fact could be understood in the following way: Each pairing of pre- and postsynaptic spikes tends to modify the synaptic efŽcacy, but a single event does not trigger long-term modiŽcation. The effect of a single pairing should therefore decay with time, and it is the accumulation of the effects of several pairings that is required to trigger a long-lasting modiŽcation. To provoke LTP or LTD, a speciŽc number of couplings must occur in a given time interval,

Spike-Driven Synaptic Dynamics Generating Working Memory States

569

so the individual effects are not totally forgotten. The number of pairings with respect to the time interval could in principle be experimentally determined, furnishing information about the effect of a single pairing and its decay time. Despite the somewhat contrasting conclusions reached using different protocols, as to the event that ultimately triggers the modiŽcations, other properties of synaptic plasticity are widely accepted. Experiments in vitro and in vivo have shown that a synapse is bidirectionally modiŽable (Dudek & Bear, 1993). In other words, the same synapse can undergo both LTP and LTD. Moreover, the same synapse once depressed can be newly potentiated by a suitable stimulation, and vice versa. However, the modiŽcation of the synaptic strength is prevented when NMDA receptors are blocked, regardless of the stimulation used (Dudek & Bear, 1992; Markram et al., 1997). An immediate action of the NMDA release is the opening of voltage-gated Ca2C channels and the consequent inux of the calcium ions. When this is prevented, the expression of the synaptic plasticity is impaired; no transitions are observed. This fact seems to support the model in which intracellular calcium concentration is a fundamental parameter controlling LTP and LTD. 3 From Experiments to the Model We now proceed to recapitulate the model of synaptic dynamics (Fusi et al., 2000) to be used as the learning element in the simulation. A plausible synaptic device would be able to maintain only a small number of discrete stable efŽcacies on long timescales. We choose the synapses to have two such states and to move between them stochastically. There is also some experimental evidence that synapses may in fact have only two stable values on a long timescale (Petersen, Malenka, Nicoll, & HopŽeld, 1998) . In section 7.1, we show that our synapse is not inconsistent with the wealth of Žndings concerning synaptic plasticity at the individual synaptic level described in section 2. We have not opted for a detailed agreement with all these Žndings because quite a few of them are still rather tentative and because our objective has been to have a reasonable synapse for a Žrst study of the formation of selective delay activity by the spikes produced in the course of the natural behavior of the neurons in a network. 3.1 A Model of Plastic Synapse. The plastic synapse is characterized by an internal analog variable X and a two-state value for its stable efŽcacy Jd ; Jp (depressed and potentiated, respectively). The stable efŽcacy is in turn determined by X (Fusi et al., 2000; Del Giudice & Mattia, 2001). When X is above a threshold µX , the synapse is in its potentiated state of efŽcacy Jp . Otherwise, the synapse is in its depressed state Jd .< Jp /. A transition occurs when X crosses the synaptic threshold: if X crosses from below to above, the result is LTP (Jd ! Jp ); if X crosses from above to below, the result is LTD (Jp ! Jd ). X is restricted to the interval (0 and 1), whose end points are

570

D. Amit and G. Mongillo

reecting barriers for the dynamics of X, and it obeys P D R.t/ C H.t/; X.t/

(3.1)

where R.t/ is a refresh term, responsible for long-term state preservation. It drives X toward 0.1/, depending on whether it is below (above) the synaptic threshold µX . This term mimics the biochemical mechanisms that keep synaptic efŽcacy stable in the absence of stimulation, against erasure by spontaneous activity. The dynamics of the drift we choose to be linear, R.t/ D ¡®2.¡X C µX / C ¯2.X ¡ µX /;

(3.2)

where 2.¢/ is the Heaviside function, 2.x/ D 1 for x > 0 and 0 otherwise. This bistability is analogous to that of a computer DRAM memory bit. H.t/ relates the synaptic dynamics to the pre- and postsynaptic neural activities and is responsible for synaptic transitions. The synaptic efŽcacy is modiŽable only when the synapse is activated by a presynaptic action potential. Hence, H.t/ should be different from zero only upon arrival of a presynaptic spike, which we express as X pre pre (3.3) H.t/ D F.tk /±.t ¡ tk /; k

that is, a sum over all presynaptic spike emission times. A priori, the time of the synaptic activation does not coincide with the presynaptic emission. In fact, once an action potential is generated in the soma, it travels along the axon, reaching the synaptic boutons (synaptic activations). Here, the spike provokes the release of the neurotransmitter, which in turn provokes the current inux into the postsynaptic cell. The time elapsed from the presynaptic spike emission to the postsynaptic variation of the depolarization is the synaptic delay and is of the order of a couple of milliseconds. However, most of this time is associated with postsynaptic ionic ow along dendrites. The axonal delay is very short, and the synaptic activation coincides with the presynaptic emission. Each spike induces a jump in X whose value is determined by F.¢/, which in turn depends on the state of the postsynaptic neuron at the time of the presynaptic emission. The complete speciŽcation of H.t/ requires the introduction of a neural model, which we assume to be that of an integrateand-Žre neuron (see section 4.1). One possibility is that F is determined by the intracellular concentration of calcium, as proposed by Bear (1996). F.¢/ would be positive when intracellular [Ca2C ] exceeds some critical level and negative if it is below another critical level. We model this complex mechanism taking F.¢/ to be a function of the postsynaptic depolarization.2 This 2 This assumption seems to be corroborated by recent experimental evidences (Sj¨ostr¨om, Turrigiano, & Nelson, 2001). See also section 7.1.

Spike-Driven Synaptic Dynamics Generating Working Memory States

571

may not be the Žnal answer, but it can be judged by its performance for the special protocols used in experiments on plasticity and with respect to the distributed learning dynamics it generates in a network. The synaptic dynamics is then related to the neural dynamics by postulating 8 if Vpost > µ1 > : 0 otherwise; with µ2 · µ1 < µ , where µ is the neural threshold for spike emission. Vpost is the membrane depolarization of the postsynaptic neuron. Examples of synaptic dynamics for LTP and LTD are depicted in Figures 1 and 2, respectively. 3.2 Transition Probabilities. The temporal evolution of H.t/, and hence the dynamics of the synaptic internal variable X.t/, could in principle be determined by specifying the pre- and postsynaptic neural activity. The situation we are interested in is neurons embedded in a large recurrent network. In this case, neural Žring is stochastic, principally due to the random pattern of connectivity (van Vreeswijk & Sompolinsky, 1996). As a consequence, LTP TRANSITION Pre­ synaptic spikes

NO TRANSITION Pre­ synaptic spikes

Synaptic internal variable X(t)

1 q

Synaptic internal variable X(t)

X

0

Post­ synaptic depolarization V(t)

Post­ synaptic depolarization V(t)

1 q 0

50

100 time (ms)

150

200

V

0

0

50

100 time (ms)

150

200

Figure 1: Synaptic dynamics. The time evolution of X.t/ (middle panel); following presynaptic emission (top), X.t/ is regulated up (down) if the postsynaptic depolarization V.t/ is greater (smaller) than µV .D µ1 ´ µ2 ) (bottom). In the time intervals between spikes, X.t/ drifts linearly up or down, according to equation 3.2 (see text for details). (Left) LTP. The synapse starts from its depressed value Jd , (since X < µX ); by the end of the interval, it is potentiated to Jp , (since X > µX ). (Right) No transition. At the end of the interval, X returns to its initial value. The evolution of V.t/ is that of an integrate-and-Žre neuron with a linear leakage current. For details see section 4.1. (Reproduced from Fusi et al., 2000.)

572

D. Amit and G. Mongillo

LTD TRANSITION Pre­ synaptic spikes

NO TRANSITION Pre­ synaptic spikes

Synaptic internal variable X(t)

1 q

X

0

Post­ synaptic depolarization V(t)

Synaptic internal variable X(t)

Post­ synaptic depolarization V(t)

1 q

V

0

50

100 time (ms)

150

200

0

0

50

100 time (ms)

150

200

Figure 2: Synaptic dynamics. (Left) LTD. A presynaptic burst allows X to cross the threshold µX from above, provoking downregulation of synaptic efŽcacy .Jp ! Jd /. (Right) No transition. Conventions as in Figure 1. (Reproduced from Fusi et al., 2000.)

long-term synaptic transitions are themselves stochastic, and it is possible to characterize them by a frequency-dependent transition probability. We deŽne PLTP .ºpre; ºpost ; T/ as the probability that a depressed synapse undergoes LTP during a time interval T of constant external afferents. ºpre and ºpost are, respectively, the mean emission rates of the pre- and postsynaptic neuron. The probability of LTD, PLTD .ºpre ; ºpost ; T/, is deŽned analogously. PLTP .ºpre; ºpost ; T/ and PLTD .ºpre ; ºpost ; T/ contain the information relevant for the unsupervised learning dynamics. They can be estimated by simulating the coupled neural and synaptic dynamics as described in Fusi et al. (2000). Once they are known, it becomes possible to determine the regimes of network activity of plasticity (i.e., signiŽcant modiŽcations of synaptic structure) and of overall structural stability (i.e., negligible modiŽcations). It also becomes possible to estimate the number of synaptic transitions in a given synaptic population per stimulus presentation or, equivalently, the probability of transition per presentation. When these probabilities are known, the structuring process can be described in a compact form, obtaining the fraction of potentiated synapses in each population as a function of the number of stimulus presentations. A simpliŽed classiŽcation of the synaptic dynamics can be obtained in terms of PT ´ PLTP C PLTD and PR ´ PLTP =PT (see also Fusi et al., 2000). PT is deŽned as the total transition probability, that is, the probability that a long-term transition occurs during T (either potentiation or depression), and P R is deŽned as the relative probability of LTP with respect to LTD. When PR » 1, LTP is likely to occur.

Spike-Driven Synaptic Dynamics Generating Working Memory States

573

Suppose a < µX and b < .1 ¡ µX /, so that several jumps are required to cross synaptic threshold. When the presynaptic neuron Žres, X is modiŽed according to the value of the postsynaptic depolarization. If a further spike does not arrive to push X toward the threshold, the synaptic internal state is reset by the refresh current, to either 0 or 1, depending on whether X is above or below its threshold (see equation 3.2). The time to forget completely a given jump in X is the ratio between the amplitude of the jump and the synaptic refresh current. In the synaptic device described here, we have a reset time for potentiation ¿p .D a=®/ and one for depression ¿d .D b=¯/. To simplify the reasoning, we suppose ¿p D ¿d . Transitions typically would be provoked by bursts of presynaptic spikes. A burst can cause a transition if the time intervals between successive spikes (ISI) are on average shorter than ¿p . In this case (the high presynaptic rate regime), a long-term synaptic transition, either up or down, can occur. When the mean ISI is greater than ¿p (the low presynaptic rate regime), the probability of a synaptic transition is negligible. Consequently, PT is zero if ºpre ¿ 1=¿p and is an increasing function of the presynaptic emission rates for ºpre > 1=¿p . The direction of a jump in the synaptic internal variable depends on the value of the postsynaptic depolarization (see equation 3.4). The relative frequency of upregulation of X (X ! X C a) with respect to that of downregulation (X ! X ¡ b) increases with postsynaptic emission rates. Indeed, to emit at high rates, the postsynaptic depolarization must often be near the threshold and, hence, V > µ1 . On the other hand, when the postsynaptic neuron is emitting at a low rate, its depolarization is uctuating around the rest potential, and hence below µ2 , so P R would be an increasing function of ºpost . We have argued that the synaptic device behaves in a Hebbian way. In other words, a synapse between two neurons, both emitting at a high rate, tends to be potentiated (PT > 0, PR » 1), while a synapse connecting a high-rate presynaptic neuron to a low-rate postsynaptic one tends to be depressed (PT > 0, PR » 0). In fact, as we shall see, a dynamic synapse naturally extends the plasticity scenario: synapses in neural populations where one would expect LTP may undergo LTD, and vice versa. This may be due to uctuations or to the width of the emission rate distribution in a functionally uniform neural population. When the average rates are high and close to each other in the pre- and postsynaptic cells or if there is a very large difference between the two, deviations are few, and Hebb is maintained to a very good accuracy. But if the average rates are closer, as would, for example, be the case when one population of neurons is stimulated and another is in enhanced delay activity (the substrate for the generation of contextcorrelations; Miyashita, 1988; Brunel, 1996), or of pair-associate representations (Sakay & Miyashita, 1991; Erickson & Desimone, 1999), LTP and LTD probabilities for the synapses connecting the two populations would both be signiŽcant. The one for LTP would be higher to allow the association.

574

D. Amit and G. Mongillo

The structuring in that population will not go on indeŽnitely (creating representation collapse), but will saturate at a level determined by the ratio of the two probabilities, a very welcome modiŽcation that will be explored in depth elsewhere (Mongillo & Amit, 2001a). 4 Simulations 4.1 Single Cell Model. As a model of the spiking neuron we use the linear integrate-and-Žre (LIF) model of Fusi and Mattia (1999) because it has been found to be consistent with the behavior of real cells in noisy conditions (Rauch, La Camera, Luscher, ¨ Senn, & Fusi, 2002) and also because of its analytical tractability. Despite its simplicity, this neural model is able to reproduce much of the phenomenology of networks of RC integrate-and-Žre neurons (Fusi & Mattia, 1999; Mongillo & Amit, 2001b). It is characterized by a Žring threshold µ, a (postspike) reset potential Vr , a constant leakage current ¯I , and a refractory period ¿arp . The evolution of the membrane depolarization below threshold is ( ¡¯I C I.t/ if V.t/ > 0 (4.1) VP D 0 if V.t/ D 0 and VP < 0: I.t/ is the afferent current charging the cell’s membrane. When V crosses the threshold, a spike is emitted, and the depolarization is reset to Vr and kept constant for ¿arp milliseconds. If I.t/ is stationary, gaussian, and independent at different times (delta correlated), the distribution of the depolarization P.V/ and the mean emission rate º are calculated to be (Fusi & Mattia, 1999) ± ²i º h ¹ 1 ¡ exp ¡2 2 .µ ¡ V/ for V 2 [0; µ] (4.2) P.V/ D ¹ ¾ µ ³ ´¶¡1 ¾ 2 2¹µ ¡ 2¹µ ¾2 ¡ 1 C (4.3) º ´ 8.¹; ¾ 2 / D ¿arp C e ; 2¹2 ¾ 2 for Vr D 0. The hypotheses on the statistics of the afferent current are quite well satisŽed in a large spiking network (Amit & Brunel, 1997b). This model does not include adaptation effects, essential for Žtting neural response dynamics of real cells (Rauch et al., 2002), limiting the increase in neural emission rates during sustained stimulation. In our simulations, this simpliŽcation is corrected by hand (see section 4.2.5). 4.2 The Learning Process. 4.2.1 The Network Architecture. The network (modeling a cortical module) is composed of NE pyramidal cells and NI interneurons. Each neuron receives, on average, CE synaptic contacts from excitatory and CI from inhibitory neurons inside the network and Cext excitatory synaptic contacts

Spike-Driven Synaptic Dynamics Generating Working Memory States

575

representing external afferents (Amit & Brunel, 1997b). The current resulting from the activation of the external synapses represents noise from the rest of the cortex and selective afferents due to stimuli. When the network is set up, the afferent presynaptic neighbors of a given neuron are selected as follows: for each (postsynaptic) neuron (excitatory or inhibitory), one selects the presynaptic neighbors, independently and at random, by a binary process in which a presynaptic excitatory (inhibitory) neuron is a neighbor with probability CE =NE (CI =NI ). The existing inhibitory as well as the excitatory synapses onto interneurons are assigned continuous values, each drawn from a gaussian distribution with its preassigned mean and variance. These synapses remain Žxed throughout the simulation. The plastic excitatory-excitatory synapses (prior to training) are distributed independently and randomly, with P .Jij .0/ D Jp / D Cp0 D 0:25. This is equivalent to a distribution for excitatory-excitatory synapses with mean JEE =µ D Cp0 Jp C .1 ¡ Cp0 /Jd D 0:022 and variance 12 JEE =µ 2 D Cp0 .1 ¡ Cp0 /.Jp2 C Jd2 / D 0:00063. Numerical values correspond to the parameters in Table 1. Furthermore, to each synapse is associated a transmission delay ±, which represents the time needed by the presynaptic spike to affect the postsynaptic depolarization. In our simulations, we chose ± D 1 ms for all the synapses. This is the unstructured state of the network, which is supposed to sustain spontaneous activity. The complete list of parameters used in the simulations is reported in Table 1. 4.2.2 The Dynamics. The simulation consists of a numerical integration of the discretized dynamical equations of both neurons and plastic synapses. The temporal step 1t is chosen to be shorter than the time between two successive afferent spikes. In our simulations, we take 1t D 0:05 ms. The initial distribution of depolarization in the network is set uniform. Spikes begin to be emitted due to external excitatory afferents. All neurons are receiving a nonselective external current Ii.ext/ D Jext ½.t/, where ½.t/ is a Poisson process with mean Cext ºext per unit time. ºext is taken of the order of the rate of excitatory neurons in the spontaneous state and is kept Žxed. At each 1t, the afferent external current is Jext ½.t/1t. The initial distribution of the neural depolarization is found not to be important. The network reaches its stationary state (depending on the synaptic matrix and level of external signal) within short relaxation times (see Figure 6 and the discussion in section 5.1). The network is allowed to evolve freely (i.e., without stimulus presentation) for about 100 ms, to reach stationary spontaneous activity, and then the training stage begins. The depolarization of all neurons is sequentially updated according to equation 4.1. If Vj .tC1t/ > µ, a spike is delivered to all postsynaptic neurons, and the depolarization is reset to Vj D Vr . The spike adds to the value of the depolarization of postsynaptic neuron i, at time t C 1t C ±, the value of the synaptic efŽcacy connecting the neuron j to i, if they are connected.

576

D. Amit and G. Mongillo Table 1: Parameters Used in the Simulations. Network parameters NE Number of pyramidal cells NI Number of interneurons CE Number of recurrent E connections per cell CI Number of recurrent I connections per cell Cext Number of connections from outside ºext [Hz] Spike rate at external synapses p Number of selective populations f Fraction of cells responding to a stimulus Single cell parameters E µ[µ ] Spike emission threshold 1 Vr =µ Postspike reset potential 0 ¯I =µ [ms¡1 ] Leakage current .011 ¿arp [ms] Absolute refractory period 2 Synaptic parameters Jext=µ EPSP produced by external afferent JEI =µ IPSP amplitude on pyramidal cells JIE =µ EPSP amplitude on interneurons JII =µ IPSP amplitude on interneurons Jp =µ EPSP produced by potentiated synapse Jd =µ EPSP produced by depressed synapse Cp0 Fraction of Jp in unstructured network ± [ms] Synaptic delay Synaptic dynamics parameters µX Threshold for synaptic transition µ1 =µ Threshold for upregulation of X µ2 =µ Threshold for downregulation of X ®[ms¡1 ] Drift toward 0 ¯[ms¡1 ] Drift toward 1 a Amplitude of up jump b Amplitude of down jump

5000 1250 380 120 380 5 5 0.1 I 1 0 .0113 2 .019 .063 .025 .055 .057 .01 .25 1 .374 .7 .5 .0067 .01 .17 .14

Notes: Units are given in brackets. Before structuring, the mean and variance of the excitatory efŽcacy in the excitatory population are given by JEE =µ D Cp0 Jp C.1¡ Cp0 /Jd D 0:022 and 12 JEE =µ 2 D Cp0 .1 ¡ Cp0 /.Jp2 C Jd2 / D 0:00063.

Moreover, if the emitting neuron is excitatory, all plastic synapses connecting it to other excitatory neurons are updated according to equations 3.1, 3.2, and 3.4. The level of the postsynaptic depolarization, determining the direction of the jump in X.t/ according to equation 3.4, is read at the time of the presynaptic emission (see section 3.1). If Xij .t C 1t/ (the internal variable of the plastic synapse connecting neuron j to i) crosses the threshold µX , the efŽcacy Jij .t C 1t/ is suitably modiŽed, producing either LTP or LTD. 4.2.3 Statistics of Stimuli. The p stimuli to be used in training are set up when the simulation is initialized and is kept Žxed. Each stimulus cor-

Spike-Driven Synaptic Dynamics Generating Working Memory States

577

responds to a pool of fN e , visually responsive excitatory neurons. f is the coding level of the stimuli and is chosen low ( f ¿ 1). These pools are selected nonoverlapping, that is, the neurons have a perfectly sharp tuning curve, and pf < 1. They are p consecutive groups of fNe neurons. Although this is not a realistic constraint, it is very useful in allowing the monitoring of the complex double dynamics by mean-Želd theory and in simplifying greatly the analysis of the learning dynamics. This fact, however, does not render the process trivial (see section 5.1). Moreover, for low memory loading levels, p ¿ 1=f , it is a good approximation to the case of unconstrained random choice of stimuli (Amit & Brunel, 1997b). At higher loading levels, one must confront the full issue of the network memory capacity, whose details are beyond this study. The presentation of a stimulus is expressed by an increase in the rates of the external afferents to the selective cells (the corresponding pool). The rate of spikes arriving at these neurons is increased by a factor ge > 1. ½k .t/ is still a Poisson process, but with mean ge Cext ºext dt, where the index k runs over the visually responsive population. External currents to the other excitatory neurons are unaltered. Similarly, the afferents to all interneurons increase their activation rates by a factor gi > 1 (Miller et al., 1996). Accordingly, the neurons selective to the stimulus presented emit at elevated rates. In other words, a stimulus elicits a visual response in the same subset of cells whenever it is presented. There is no noise in the process of presentation. In a more realistic situation, the subset of cells excited may be slightly different on different presentations of the same stimulus. Accordingly, we carried out simulations with a noisy version of the stimuli. In these simulations, each stimulus was considered a prototype of speciŽc f NE cells. In each presentation, a fraction 1 ¡ f1 of the prototype was excited, as well as f2 of all other .1 ¡ f /NE excitatory cells. f1 is the noise level, and f1 D f2 D 0 is the noiseless case. To have a constant (on average) number of stimulated cells in each presentation, we chose f2 so that f2 D f1 f=.1 ¡ f /: 4.2.4 Training Protocol. For the training protocol, the set of stimuli is repeatedly presented to the network. Each stimulus appears for Tstim milliseconds. Then it is removed, and following a delay period of Tdelay, during which none of the populations is stimulated, another stimulus is presented. The presentation sequence of the stimuli is either kept Žxed (1 ! ¢ ¢ ¢ ! p ! 1) or the sequence is generated by choosing each stimulus to be presented, independently and randomly, with probability 1=p. All along the simulation, during and between stimulus presentation, the neural and synaptic dynamics are free and are described by equations 3.4 and 4.1. 4.2.5 Control of Visual Response. As structuring takes place, the increasing average recurrent excitatory synaptic efŽcacy causes a signiŽcant in-

578

D. Amit and G. Mongillo

crease of neural emission rates, at parity of external signal (contrast, ge ). Such an increase is doubly undesirable—Žrst, because it is not observed experimentally (Erickson & Desimone, 1999), and second, and more relevant for the present purposes, because it distorts the learning process. As things stand, it seems to be an artifact of the simplicity of the single cell dynamics as well as of the synaptic transmission model rather than of the learning process. In a more realistic situation, this problem is resolved by the adaptation features of the neurons and of the synaptic transmission (Tsodyks & Markram, 1997). This tendency of increased visual response during training is partially balanced by the stimulation of the inhibitory neurons during stimulus presentation. The enhanced Žring of interneurons limits the visual response of the pyramidal cells by the hyperpolarizing currents. Moreover, it enhances the synaptic depression process between stimulated and unstimulated neurons because the stronger hyperpolarizing currents, arising from stimulated interneurons, decrease the mean depolarization level of neurons of the unstimulated population. This enhances the depression of synapses afferent on them. The stimulation of inhibitory neurons, and its effects, appears to be consistent with experimental Žndings (Steele & Mauk, 1999). However, the activity of the inhibitory population cannot grow beyond a certain level because the interneurons also inhibit each other. On the other hand, inhibitory contacts among interneurons cannot be weakened too much, or the emission rate of the inhibitory population becomes so high as to suppress the activity of the excitatory cells completely. When inhibition was no longer sufŽcient, we artiŽcially kept the rate of stimulated neurons approximately constant during the learning process. The emission rates during stimulus presentation are suggested by the properties of the single synapse (see section 3.2) and are chosen at the start of the simulation. When, as a consequence of synaptic structuring, these rates exceed the initial level by more than 15%, stimulus presentation is interrupted. The synaptic structuring reached is used to calculate, by mean-Želd analysis (see the appendix), a new level of contrast (ge ) for the stimuli, keeping gi Žxed. ge is calculated to produce the original rate of visual response, and the simulation resumes. 4.2.6 Observables Monitored. During the simulation, various observables related to the collective neural activity as well as the synaptic dynamics are sampled. To describe the evolution of the structuring among excitatory neurons in the network, we deŽne functional neural populations and functional synaptic populations. The neural populations are of excitatory neurons, such as the population corresponding to one of the p stimuli; when a particular stimulus is presented, the union of the p ¡ 1 populations corresponding to the other stimuli, not now recalled; and the population of background cells. In every phase of the simulation, the neurons in each of these populations receive the same average external input and see, on average, the same synaptic structure.

Spike-Driven Synaptic Dynamics Generating Working Memory States

579

The functional synaptic populations are the synapses connecting pairs of neurons within a functional neural population or between two different functional neural populations. Therefore, a synaptic population is fully deŽned once the functional properties of its pre- and postsynaptic neurons are speciŽed. The state of any synaptic population (its structuring) is quantiŽed by the fraction, Cp , of potentiated synapses in that population. In practice, we monitor only two structuring parameters, Cp . One of them is CpHH , between neurons in one of the p selective populations. It is one parameter and not p since we observe a very small variability of structuring corresponding to the different stimuli, and so we monitor their average. The other is CpHL , between neurons belonging to a selective population and those in the background or in other, unstimulated populations. Again, the average over p such populations is monitored. Recall that only synapses with high-rate presynaptic neurons have a signiŽcant transition probability. We do not distinguish between postsynaptic neurons in the background and neurons of nonselected populations because their rates are rather close. This is obvious before the network is signiŽcantly structured. It remains so after structuring due to the enhanced inhibitory activity. Most of the (nonstimulated) neurons emit at very low rates, and the distributions of frequencies in those populations are quite similar. This was checked by carrying out a set of simulations and comparing the structuring of the synaptic population between visually responsive and background neurons with the structuring between visually responsive and neurons responsive to a different stimulus. Moreover, we do not consider the structuring between background, or unselected, neurons and other populations, because the presynaptic rates are low, and so structuring is negligible, as conŽrmed by the simulations. Because the selective neural populations are nonoverlapping, the functional synaptic populations deŽned above are also nonoverlapping. This is true also for the synaptic populations between selected neurons and the rest, because even when the synapses of the two populations share postsynaptic neurons, the presynaptic neurons are different. The synaptic structuring reached in one synaptic population is not disrupted by the presentation of a different stimulus, and this fact simpliŽes the analysis of synaptic structuring. The level of synaptic structuring required for WM states may be estimated by mean-Želd analysis (see the appendix). When synaptic structuring reaches the level theoretically estimated, the training stage is interrupted. Each stimulus is presented for 150 ms and then removed and the network evolves for 1 second in the absence of further stimulations. If, following the removal, the network exhibits a WM state for all stimuli, the learning stage is terminated. Otherwise, it is resumed. If stimulus presentation continues after WM activity appeared, the structuring reaches an asymptotic level and further presentations do not affect the synaptic matrix. The asymptotic

580

D. Amit and G. Mongillo

structuring level expresses a detailed balance between potentiating and depressing processes (see section 6.1). 5 Results 5.1 The Structuring Process. The simulations presented here are for nonoverlapping stimuli. The parameters are given in Table 1. Even in this basic case, the process of synaptic structuring, and the consequent appearance of WM states, is not trivial. Indeed, various instabilities tend to appear during learning. The most common instabilities encountered in our simulations were oscillatory behavior, uncontrolled growth of the network global activity, or depression where one expects potentiation or vice versa. The essential underlying reason is that to reach stable, selective WM states, synapses must not only be potentiated among visually selective neurons, but this potentiation must be accompanied by adequate depression from selective to nonselective cells (Brunel, 2000). On the other hand, with the synaptic dynamics proposed here, in any population of synapses, one Žnds potentiations as well as depressions, as we explain below. Depressing synapses that should be potentiated can prevent the formation of WM states, because the fraction of potentiated synapses remains too low, at Žxed Jp and Jd . Often these effects appear when the level of visual response becomes too high. As a consequence, the rate of synaptic modiŽcations increases appreciably, and hence, the effects of unwanted synaptic transitions, occurring due to the wide distribution of rates inside the neural populations, are ampliŽed. Alternatively, the ratio between the probabilities of potentiation and depression could vary, again provoking unwanted synaptic modiŽcations. Another source of unwanted plasticity is the correlation between the stimulated and the unstimulated pyramidal cells. Due to the increased inhibitory activity, the activity in the background population is quite low. The spike emission of a neuron belonging to the background population is provoked principally by the spikes afferent from the visually responsive population. As a consequence, spike emission in a presynaptic neuron of the selective population consistently tends to precede the emission of the postsynaptic neuron if it belongs to the background population (Rubin, Lee, & Sompolinsky, 2001). Thus, despite the low emission rate, the postsynaptic depolarization could be found often at a high level when the synapse is activated by presynaptic emission. This could provoke LTP instead of the appropriate LTD. However, this effect is negligible when the network is sufŽciently large and the single EPSP is small with respect to the threshold for spike emission. Hence, in a way, the results we report represent a demonstration that in the rich and complex space of parameters of the synapses and the neurons exists a zone in which the fundamental structuring for working memory can take place. It would have been more satisfactory had the scenario been

Spike-Driven Synaptic Dynamics Generating Working Memory States

581

more robust, in the sense of self-organization, so that learning would induce neural activity conducive to structuring. But the complexity of the situation limits us to one step at a time. Figure 3 reports the average fraction of potentiated synapses in both the HH and HL synaptic populations as a function of the number of presentations per stimulus. Cp increases monotonically in the HH populations with stimulus presentation, while the fraction of potentiated synapses decreases with stimulus presentation (in the HL populations). Figure 4 reports the average number (averaged over stimuli) of synaptic potentiations and depressions per presentation in both the HH and HL synaptic populations. Note that there are depressions in the HH populations as well as potentiations in the HL populations, due to the wide distribution of rates in each

0.9 0.8

HH

0.7 0.6

C

p

0.5 0.4 0.3 0.2 HL

0.1 0 0

20

40 60 80 # of presentations/stimulus

100

120

Figure 3: Synaptic structuring: Cp in the two monitored synaptic populations as a function of the number of presentations (nonoverlapping populations). Horizontal line: The unstructured initial state (Cp0 D 0:25). CpHH (between visually responsive neurons) increases monotonically with stimulus presentation; CpHL (between visually responsive to nonresponsive neurons) decreases monotonically. The values of CpHH and CpHL are averaged over all functionally equivalent synaptic populations. The interpopulation variability is included in the circles representing the points. Synaptic structuring allowing for WM is reached after 105 presentations of each stimulus. See also Figure 7. The parameters are given in Table 1.

582

D. Amit and G. Mongillo

1500

400

1000

200

500

# of transitions

600

0 0

10 20 # of presentation

0 0

10 20 # of presentation

Figure 4: Number of synaptic transitions (stimulus average) per presentation versus presentation number for the Žrst 20 presentations. Squares represent number of potentiations and circles the number of depressions. (Left) HH population: number of potentiations greater than the number of depressions. (Right) HL population: number of depressions greater than the number of potentiations. The difference between the absolute numbers of transitions in the two Žgures is due to the greater number of synapses in the HL population. Parameters are as in Figure 3.

population (see Figure 5), which produces a Žnite probability that synapses within a selective population of neurons Žnd a high-rate presynaptic neuron and a low-rate postsynaptic neuron. Similarly, there is a Žnite probability that synapses between selective and nonselective populations Žnd two high-rate neurons. However, with the parameters of Table 1 and the relevant rates, the number of “correct” transitions is overwhelmingly greater than the number of “wrong” transitions. The outcome is an increase of CpHH and a strengthening of the mean synaptic efŽcacy among selective neurons and a monotonic decrease of CpHL and a depression of the mean synaptic efŽcacy from selective to nonselective neurons. In this situation, we recover the classical Hebb rule. As a control, we have veriŽed that similar results are obtained when the set of stimuli is noisy. We carried out simulations with f1 D 0:1 and f1 D 0:2. Recall that 1 ¡ f1 is the probability that a neuron of the prototype is effectively activated by its presentation in the presence of noise. The only noticeable, and expected, effect was that WM states require more presentations to appear: 140 to 150 presentations per stimulus instead of 100 to 110 required in the noiseless case. 5.1.1 Appearance of Working Memory. Working memory (selective delay activity) states appear for CpHH D 0:875 and CpHL D 0:0045. The appearance of WM states is related not only to the level of synaptic structuring, that is, the values of Cp in the various synaptic populations, but also to the other

Spike-Driven Synaptic Dynamics Generating Working Memory States 0.07

583

0.1 0.09

0.06

0.08 0.05

0.07 0.06

P(n)

0.04

0.05 0.03

0.04 0.03

0.02

0.02 0.01 0 20

0.01 40

60 n [Hz]

80

100

0 0

20 n [Hz]

40

Figure 5: Population distribution of spike rates during stimulation. (Left) Stimulated population. The mean emission rate is ºstim D 60 Hz. (Right) Unstimulated pyramidal cells; the mean emission rate is ºbg D 0:9 Hz. Despite the low mean emission rate, neurons are found with high rate. Note that the high-frequency tail of the background distribution overlaps with the stimulated distribution. This could impair the synaptic structuring. Recurrent inhibition is effective in reducing this overlap. See the text for details.

parameters of the network, principally the ratio Jp =Jd between potentiated and depressed synaptic efŽcacy. For example, decreasing Jp once the structuring level allowing for WM states is reached prevents the appearance of WM states. On the other hand, increasing Jp could destabilize spontaneous activity. All network parameters, as connectivity, mean synaptic efŽcacies, level of synaptic structuring for WM states, and so forth should be chosen to lie in a biologically plausible range. Starting from an unstructured synaptic matrix (Cp0 D 0:25), 100 to 150 presentations per stimulus are required to reach stable WM, that is, selective delay states. Figure 6 shows the neural activity in a visually responsive population before, during, and following the presentation of a stimulus at various level of synaptic structuring. What is plotted is the number of spikes emitted per neuron, in bins of 5 ms, averaged over the selective populations and normalized to Hz (dividing by the bin size). The transients into the various stationary states, spontaneous activity, WM, or stimulated state are

584

D. Amit and G. Mongillo

100

100

n [Hz]

A

B 50

50

0 0 ­

500

­ T [ms] 100

0 0 ­

­ T [ms]

500

n [Hz]

C 50

0 0­

­

500 T [ms]

1000

Figure 6: Selective population activity during trial at three structuring stages: population-averaged neural activity (in 5 ms bins) converted to Hz (see text), before, during, and following stimulation. Arrows indicate 150 ms stimulation interval. Note the short transients from one stationary state to another. Mean Želd predictions of mean emission rates, ºth , during stimulation (ST), spontaneous (SA), and working memory (WM) activity are compared with the results of the simulation. (A) Fifty-Žve presentations per stimulus; CpHH D 0:755; CpHL D 0:0245. ST: ºth D 51:5 Hz ºsim D 44:5 § 1:5 Hz; SA: ºth D 1:3 Hz ºsim D 1:21 § 0:21 Hz. (B) Seventy presentations per stimulus; CpHH D 0:8; CpHL D 0:016. ST: ºth D 51:5 Hz ºsim D 45:7 § 2:9 Hz; SA: ºth D 1:3 Hz ºsim D 1:17 § 0:18 Hz. Horizontal dashed line: Average emission rate during stimulus presentation and SA. In both A and B no WM: population returns to spontaneous state after removal of stimulus. (C) Appearance of WM state: 105 presentations per stimulus; CpHH D 0:875 and CpHL D 0:0045. WM: ºth D 27 Hz ºsim D 24:3 § 4:25 Hz. Horizontal dashed line: Average emission rate during WM activity. Large uctuations are due to strong Žnite-size effects. See text for details.

Spike-Driven Synaptic Dynamics Generating Working Memory States

585

short, on the order of 50 to 100 ms. In Figures 6A and 6B, the level of synaptic structuring is still too low, and the stimulated population returns to spontaneous activity when the stimulus is removed. In Figure 6C, one observes a WM state in which the cells of the selective populations emit at elevated rates (24.3 Hz) following the removal of the stimulus. Mean Želd predictions for the mean emission rates during spontaneous and WM activity (dashed horizontal lines) are in good agreement with the simulations. During WM activity, we observed very large uctuations of the average emission rate. This is due to Žnite size effects; to reduce the duration of the simulations with the double dynamics, the selective populations were taken relatively small and, given the low connectivity, a neuron receives only about 38 connections from other neurons in the same selective population (see Table 1). Due to the small number of connections from other selective cells and the stochastic nature of spike emission,pthe current afferent on a cell has large uctuations (of relative order 1= N, N is the number of pre-synaptic neurons) around its mean which produce uctuations in the emission rate. 6 Estimates of the Structuring Process 6.1 Population Dynamics of the Structuring Level. We deŽne qC as the probability that a depressed synapse undergoes LTP during one stimulus presentation and q¡ as the probability of depression of a potentiated synapse per presentation. It is expected that qC > q¡ in an HH synaptic population and vice versa in an HL population. The fraction of potentiated synapses follows a population evolution as a function of the number of stimulus presentations, Cp .n C 1/ D Cp .n/[1 ¡ q¡ .n/] C qC .n/[1 ¡ Cp .n/];

(6.1)

where Cp .n/ represents the fraction of potentiated synapses in a given synaptic population after n presentations of the same stimulus. The dependence of qC and q¡ on n takes into account the possibility that they could vary along the learning process. If both the frequencies of visual response and the statistics of the emission process do not vary appreciably during learning, qC and q¡ would remain approximately constant. In this case, for nonoverlapping populations, after a large number of presentations, (

HH HH CpHH D qHH C =.qC C q¡ /

CpHL

D

HL qHL C =.qC

C qHL ¡ /

for HH populations for HL populations:

(6.2)

Note that when the selective populations overlap, the structuring of a given population could change even if its preferred stimulus is not presented. In

586

D. Amit and G. Mongillo

this case, equation 6.1 becomes more complicated, and one must also take into account the structuring due to the presentation of different stimuli. We do not expand on this issue here. In our case (see, e.g., Figure 4), the number of wrong transitions—depressions instead of potentiations in the selective population and potentiation between selective and nonselective—is rather small. We neglect these transitions, and equation 6.1 becomes (

Cp .n C 1/ D Cp .n/ C qC .n/[1 ¡ Cp .n/] Cp .n C 1/ D Cp .n/[1 ¡ q¡ .n/]

for HH populations for HL populations:

(6.3)

The dashed curves in Figure 7 are a least-square Žt of Cp in the HH and HL populations measured in the simulation by the solution of equation 6.3 with constant qC and q¡ , during the Žrst 20 stimulations. The degree of agreement between the curves indicates that these probabilities are approximately constant during learning, partly because the rates under stimulation are kept approximately constant. 6.2 Population Transition Probabilities and Transition Numbers. The probability of an LTP transition in a homogeneous population of synapses, per stimulus presentation of duration T, could be estimated as follows.3 Let c be the probability of a synaptic contact from a neuron in the presynaptic population to a neuron in the postsynaptic one. The population of synapses connects neurons in two populations: one of Npre presynaptic neurons and one of Npost postsynaptic ones (the physical neurons in the two populations can, of course, be the same). The number of potentiations in such a synaptic population, per presentation of a stimulus, is a random variable, X Npot D cij ¢ °ij ; i;j

where the index j runs over the presynaptic neurons and i over the postsynaptic ones. If there exists a synapse between the two neurons, cij D 1, otherwise D 0. °ij D 1 if the synapse was potentiated during the stimulation, otherwise D 0. Hence, °ij D 1 with probability PLTP .ºi ; ºj ; T/ ¢ ±.Jij ¡ Jd /, where Jij is the synaptic efŽcacy before the stimulation and ±.¢/ is 1 if its argument is zero, and zero otherwise. The mean number of potentiations is Npot D

X i;j

cij ±.Jij ¡ Jd / ¢ PLTP .ºi ; ºj ; T/:

(6.4)

The sum on the right-hand side can be converted to a sum over pairs of activity .ºi ; ºj /. Let Ppost.pre/ .º/ be the probability of Žnding a neuron with 3

The reasoning is analogous for LTD.

Cp

Spike-Driven Synaptic Dynamics Generating Working Memory States

0.6

0.3

0.4

0.2

587

0.2 0.1 0 10 20 0 10 20 # of presentations/stimulus # of presentations/stimulus Figure 7: Population dynamics description of synaptic structuring during initial 20 presentations per stimulus. (Left) Average Cp in HH populations. (Right) Average Cp in HL populations. Dashed lines represent the solution of equation 6.3, with qC D 0:0233 and q¡ D 0:0426, constant (independent of n). Error bars are included in the circles. Horizontal lines: unstructured state of the network.

rate º in the post-(pre-)synaptic neural neural population, respectively. The probability of having a depressed synapse with a presynaptic neuron at rate ºj afferent on a postsynaptic neuron at rate ºi is c.1 ¡ Cp /Ppre .ºj /Ppost .ºi /. Hence, when the number of neurons is large, equation 6.4 becomes Z Z Npot ’ c.1¡Cp /NpreNpost dºi dºj PLTP .ºi ; ºj ; T/Ppre.ºj /Ppost .ºi /: (6.5) Correspondingly, the mean number of synaptic depressions is Z Z Ndep ’ cCp NpreNpost dºi dºj PLTD .ºi ; ºj ; T/Ppre.ºj /Ppost .ºi /:

(6.6)

If during learning, the integrals on the right-hand side of equations 6.5 and 6.6 do not vary signiŽcantly, they can be identiŽed, respectively, with qC and q¡ . In that case, it becomes straightforward to obtain equations 6.3 from equations 6.5 and 6.6. In order to compare the theoretical estimates with the values of qC and q¡ obtained from simulations, we have to estimate PLTP and PLTD over the range of frequencies observed in the simulation. For PLTP , the rate intervals of both pre- and postsynaptic activity cover the distribution in the stimulated population. Similarly, for PLTD , ºpre varies over the emission rates of stimulated neurons, while ºpost varies over the rates of unstimulated neurons. (See, e.g., Figure 5.) To obtain the currents giving rise to the observed emission rates, we proceeded as follows. In the stimulated population, the neurons operate in a signal-dominated regime (Fusi & Mattia, 1999), with small uctuations. We

588

D. Amit and G. Mongillo

assume that the variance of the current is constant within the population and is given by mean-Želd analysis (see the appendix). The mean current corresponding to a given frequency is simply obtained (for constant ¾ ) by inverting the transduction function, equation 4.3, which is a monotonic function of ¹. In the unstimulated population, the neurons operate in the noise-dominated regime (Fusi & Mattia, 1999); spike emission is due to sporadic large uctuations in the afferent current, which tends to hyperpolarize the membrane. Mean emission rates are very low, and the corresponding population distribution of rates is peaked around the mean-Želd result. We assume that all neurons in the population are in this state of activity. Mean Želd then provides the mean and variance of the currents. We proceed by simulating the coupled neural and synaptic dynamics, as described in Fusi et al. (2000), to obtain PLTP and PLTD . The extent to which the assumptions used are justiŽed in the real network can be judged by comparing the theoretical estimates, equations 6.5 and 6.6, and the probabilities sim observed during the simulation: qth C D 0:017 and qC D 0:023 § 0:005, while th sim for LTD we have q¡ D 0:038 and q¡ D 0:043 § 0:009. 6.3 Estimating Lifetime of Synaptic Structure. The synaptic structure may be affected by synaptic transitions provoked by spontaneous or selective delay activity. This may cause progressive erasure of the stored memories in the absence of stimulation. The mean lifetime of the acquired synaptic structure depends on the probability of having such synaptic transitions. After stable WM states appeared, a separate set of simulations was carried out in order to check the stability of the synaptic structure against both spontaneous and selective delay activity. First, we checked the stability of the synaptic structure when the network exhibits spontaneous activity. Simulations as long as 60 seconds were carried out, with no stimulation. There were no modiŽcations in synaptic matrix. Similarly, a WM state was elicited by presenting the corresponding stimulus for 150 ms, and the network evolved freely for 4 seconds. The time difference with respect to the case of the spontaneous activity is due to the shorter lifetime of the WM state as a consequence of Žnite-size effects. Again, there were no synaptic modiŽcations. The probability of a synaptic transition during spontaneous activity is very small, because the average rate of about 1 Hz ¿ 1=¿p . Thus, the number of repetitions needed to obtain a reasonable estimate of these probabilities becomes enormous. To obtain a rough estimate of q in spontaneous activity, we ran simulations to obtain the P LTP for neurons at a rate of 20 Hz and then rescaled this probability to rates of 1 Hz. To obtain the Žrst, we ran 106 repetitions of the single synapse simulation, of T D 400 ms each, with ºpre ´ ºpost D 20 Hz. No transition occurred. The default estimate of the transition probability is then P LTP .20; 20; T D 400/ » 10¡6 . In other words, the mean time one has to wait until the synapse makes a spontaneous transition (at 20 Hz) is about 4 days. Next we approximately rescale this probability to 1 Hz.

Spike-Driven Synaptic Dynamics Generating Working Memory States

589

For n up-jumps to provoke LTP within a time interval T 0 , one must have, na ¡ ¯T 0 ¸ µX ;

(6.7)

where a is the single up-jump amplitude and ¯ is the synaptic refresh current (see equations 3.2 and 3.4). For n Žxed, the longest time interval within which the presynaptic spikes must arrive and yet be able to produce LTP is Tmax .n/ D .na ¡ µX /=¯. For the parameters of Table 1, three spikes, all provoking up-jump, within 20 ms is the most probable burst that can produce LTP, considering the low emission rates of both pre- and postsynaptic neurons. Hence, to provoke the transition, such a burst must occur within the 400 milliseconds over which we have measured the transition probabilities above. During spontaneous activity, the statistics of the emission process of the neurons is well approximated by a Poisson process (Amit & Brunel, 1997b), then P3 .T 0 D 20I º D 20/ · 203 ¢ P3 .T 0 D 20I º D 1/; where Pn .T 0 I º/ is the probability that n presynaptic spikes occur within a time interval T 0 when the emission rate is º. Neglecting other factors, as the dependence of the probability of an up-jump on the postsynaptic emission rate, we obtain qSA .400/ D PLTP .1; 1; 400/ » 20¡3 PLTP .20; 20; 400/; leading to a mean lifetime of the order of 10 years. 7 Discussion The principal result of this study is a feasibility test. The spike-driven synaptic dynamics, introduced in Fusi et al. (2000), for a suitable choice of the parameters implements rate-dependent plasticity and exhibits both longterm potentiation and long-term homosynaptic depression under diverse experimental stimulation protocols. 4 It is not to be excluded that a synaptic device in natural conditions behaves more like the synapse discussed here than as in the special protocols in which precise timing is observed. Moreover, leaving from an unstructured synaptic state, the synaptic dynamics is able to drive the network into a structured state, sustaining selective delay activity, or working memory. The generated synaptic structure is robust against spontaneous and delay activity in the absence of stimulation. The question of stability of the acquired synaptic structure, and hence the related neural activity, is also addressed. The appearance of WM states, following the repetitive presentation of a set of stimuli, is not simply a direct consequence of the rate-dependent 4

This synaptic dynamics does not generate long-term heterosynaptic depression.

590

D. Amit and G. Mongillo

plasticity, implemented by the synaptic model. Several constraints must be met during the structuring stage: ² The effects of LTP should be adequately balanced by LTD to avoid instabilities along the learning process. This balance between the probabilities of LTP and LTD, imposes constraints on the synaptic and neural parameters. Further work is needed to relate such constraints to the neural and synaptic parameters in a simple form. ² The frequency distributions in the stimulated and unstimulated populations should not overlap signiŽcantly. Alternatively, the synapse should be highly sensitive to small variations of the frequencies in the overlap region (Del Giudice & Mattia, 2001). Indeed, this could be a source of unwanted plasticity that in turn impairs the structuring process. We have found that recurrent inhibition could play a signiŽcant role in separating the distributions, reducing the high-rate tail of the frequency distribution of the unstimulated populations. A signiŽcant by-product, mentioned here only briey, is the resulting extension of the Hebbian plasticity rule. A realistic synaptic model would generate both LTP as well as LTD in any population of synapses. In the particular situation (parameters) considered in detail here, the difference between “right” and “wrong” transitions is so large that the classical Hebbian picture ensues. But when one deals with the emergent coupling between two neural populations whose mean rates are not so different (as between stimulus driven cells and delay activity cells), the extension of the Hebbian scenario promises saturation in structuring, which is an essential ingredient in maintaining independent representations alongside WM. 7.1 Back to Timing-Dependent Plasticity. The sharp cutoff in the time difference between the arrival of pre- and postsynaptic spike, observed in experiment (Bi & Poo, 1998), could be a consequence of the experimental procedure. In the model presented here, the two thresholds (see equation 3.4) can be chosen to reproduce the results of Markram et al. (1997) when the two neurons connected by the synapse operate in deterministic conditions. This may, in fact, be the case in the experiments mentioned. In a deterministic situation, one can consider the two neurons stimulated by a constant noiseless current. The evolution of the depolarization is deterministic. Thus, there is a relation between the value of V at a given instant and the time elapsed from the emission of the last spike. If one chooses the two synaptic thresholds µ1 and µ2 of equation 3.4 to be µ1 D µ ¡ T1 ¹;

µ2 D T2 ¹;

(7.1)

where ¹ is the total current, the following picture emerges. Upon the emission of a presynaptic spike, the postsynaptic neuron has V < µ2 only if it

Spike-Driven Synaptic Dynamics Generating Working Memory States

591

emitted a spike at most T2 before, and the synaptic activation causes downregulation. Similarly, the postsynaptic neuron has V > µ1 only if the time of the presynaptic spike emission precedes the postsynaptic spike by at most T1 . It will then be followed by an upregulation of the synapse. If this is the case, one should conclude that up- and downregulation depend on the level of postsynaptic depolarization and the timing effect is only a consequence of the experimental setup, as argued above. Recent experimental Žndings seem to corroborate this hypothesis (Sj¨ostrom ¨ et al., 2001). Recently, Abbott and Song (1999) and Rubin et al. (2001) studied the behavior of a synaptic model into which the exact temporal relation, observed experimentally by Bi & Poo (1998), is built in. EfŽcacy is modiŽed according to the temporal interval 1t D tpost ¡ tpre , where tpre and tpost are the times of the pre- and postsynaptic spike emission. When 1t > 0, efŽcacy is upregulated, and the efŽcacy is downregulated for 1t < 0. Such a synaptic mechanism tends to potentiate synapses connecting correlated neurons, for example, a presynaptic neuron that consistently Žres before the postsynaptic one. On the other hand, presynaptic inputs that are not causally correlated with postsynaptic Žring are weakened. The overall effect is the convergence of the synaptic distribution to an asymptotic stationary distribution. Depending on the update rule, the equilibrium distribution can be either unimodal or bimodal (Rubin et al., 2001). However, in both cases, the asymptotic distribution is largely insensitive to the Žring rates of pre- and postsynaptic cells. Therefore, such a model of plastic synapse is unable to structure the synaptic matrix to sustain selective delay activity. 7.2 Open Issues. The collective behavior of coupled homogeneous neural and synaptic populations has been described in a compact form, in terms of probabilities of potentiation PLTP and depression P LTD , functions of the pre- and postsynaptic emission rates. Here, these functions were obtained from the detailed model of the plastic synapse. It is not clear whether such a description could be more general, so as to be qualitatively independent of the detailed neural and synaptic dynamics. A priori, this seems plausible as a general consequence of the stochasticity of the neural activity. If so, we could characterize the properties of P LTP and PLTD , allowing for successful structuring, regardless of the speciŽc model of the synapse. Of course, the construction of a detailed synaptic model, which matches both available experimental data and theoretical desiderata, should be the natural conclusion of such a study. The constraints on the transition probabilities become tighter when the issue of network capacity is considered. If stimuli are sparsely coded ( f ¿ 1) and the probabilities of transition are low (slow learning), a necessary condition to recover the optimal storage capacity is that the number of potentiations approximately balances the number of depressions in each presentation, requiring q¡ » fqC (Amit & Fusi, 1994; Brunel, Carusi, & Fusi,

592

D. Amit and G. Mongillo

1998). This ensures that the difference between Cp in HH populations and Cp in HL populations, reached asymptotically, is maximal, depending on the number of stimuli to be learned. However, the appearance of WM states is not guaranteed. It depends not only on the level of structuring but also on the other network parameters. It remains to be seen whether the network exhibits WM activity in a biologically plausible range of parameters. The behavior of the synaptic device is manipulable. By choosing suitably the parameters a, b, ®, ¯, and µX , the balance constraint may be satisŽed. Simulations should be carried out to check the behavior of large networks at high loading level. Those require very efŽcient algorithms, such as Mattia and Del Giudice (2000). But little is guaranteed; the results reported in Amit and Fusi (1994) and Brunel et al. (1998) are obtained under a simplifying hypothesis: the neurons are binary, and the existence of the attractors is checked by using a signal-to-noise ratio analysis. A further step toward the networks of spiking neurons is made by Herz and colleagues (see for a review Herz, 1996). They studied the capacity of network with analog neurons (no spiking) but with a symmetric connectivity matrix. The situation may be quite different when one deals with a recurrent network of spiking neurons, like that studied here. Appendix: Mean Field for Nonoverlapping Populations The mean-Želd analysis developed allows calculating the mean Žring frequencies in each of the populations in stationary (asynchronous) states of the network as a function of the instantaneous synaptic structure and the level of the external signal. The instantaneous synaptic structuring is described by the fraction of potentiated synapses in the various synaptic populations. Plastic synapses are divided by an incoming stimulus in three homogeneous populations, according to pre- and postsynaptic activities: ² High pre- and postsynaptic activity. It is expected that in this population, the fraction of potentiated synapses increases to CpHH .> Cp0 /. ² High presynaptic and low postsynaptic activity. The fraction of potentiated synapses decreases to CpHL .< Cp0 /. ² Low presynaptic activity. The probability of a synaptic transition is negligible; thus, the fraction of potentiated synapses is unaltered. Following the presentation of a stimulus that has been previously “learned,” the network divides in four functionally different populations of cells: cells belonging to the population activated by the stimulus (denoted sel); cells representing other stimuli, not activated (denoted +); cells not responsive to any stimulus (denoted bg); and interneurons (denoted I).

Spike-Driven Synaptic Dynamics Generating Working Memory States

593

The mean Žring rates in each in the four neural populations are given by mean-Želd equations (Amit & Brunel, 1997a), 2 2 ºsel D 8.¹sel ; ¾sel /; ºC D 8.¹C ; ¾C2 /; ºbg D 8.¹bg ; ¾bg /; ºI D 8.¹I ; ¾I2 /:

The statistics of the afferent currents is calculated as function of synaptic structuring. The recurrent currents are .e/ ¹sel D ¡¯I C CE [ f JC ºsel C f .p ¡ 1/J¡ ºC C .1 ¡ fp/J0 ºbg ] ¡ CI JEI ºI

¹C D ¡¯I.e/ CCE [ f J¡ ºsel C f [JC C .p ¡ 2/J¡ ]ºC C.1¡fp/J0 ºbg]¡CI JEI ºI .e/ ¹bg D ¡¯I C CE [ f J¡ ºsel C f .p ¡ 1/J¡ ºC C .1 ¡ fp/J0 ºbg ] ¡ CI JEI ºI

¹I D ¡¯I.i/ C CE JIE [ f ºsel C f .p ¡ 1/ºC C .1 ¡ fp/ºbg] ¡ CI JII ºI ;

(A.1)

where JC D CpHH Jp C .1 ¡ CpHH /Jd J¡ D CpHL Jp C .1 ¡ CpHL /Jd J0 D Cp0 Jp C .1 ¡ Cp0 /Jd :

(A.2)

The variances of the recurrent currents are 2 2 D CE [ f 1JC ºsel C f .p ¡ 1/1J¡ ºC C .1 ¡ fp/1J0 ºbg ] C CI JEI ¾sel ºI 2 ¾C2 D CE [ f 1J ¡ºsel C f [1JC C.p¡2/1J¡ ]ºC C.1¡fp/1J0 ºbg ]CCI JEI ºI 2 2 D CE [ f 1J¡ ºsel C f .p ¡ 1/1J¡ ºC C .1 ¡ fp/1J0 ºbg ] C CI JEI ¾bg ºI 2 [ f ºsel C f .p ¡ 1/ºC C .1 ¡ fp/ºbg] C CI JII2 ºI ¾I2 D CE JIE

where 1JC D CpHH Jp2 C .1 ¡ CpHH /Jd2 and analogous for 1J¡ , 1J0 . The external currents depend on the level of the external signal: .ext/

¹sel .ext/

¹C

2

.ext/ 2 D ge Cext Jext ºext; ¾sel D ge Cext Jext ºext 2

2

.ext/ 2 ´ ¹.ext/ D Cext Jext ºext ; ¾ C.ext/ ´ ¾bg D Cext Jext ºext bg 2

2 D gi Cext Jext ºext ; ¾I.ext/ D gi Cext Jext ¹.ext/ ºext : I

In a steady state, the density function of V will be also stationary. It is given by equation 4.2. Both the mean and the variance of the afferent

594

D. Amit and G. Mongillo

currents are a linear function of º; hence, P.V/ is parameterized only by the mean emission rates. In other words, there is a one-to-one correspondence between the emission rates and the distribution of the depolarization. Here we use the theory to determine (1) the range of CpHH and CpHL for which the network is able to sustain selective delay activity states and (2) the mean emission rate of stimulated populations as a function of the instantaneous level of synaptic structuring (3) and to compare simulation results in stationary states with theory. Acknowledgments We thank Stefano Fusi for having brought to our attention Sjostr ¨ om ¨ et al. (2001). This study was supported by a Center of Excellence grant, “Changing Your Mind,” from the Israel Science Foundation and by a Center of Excellence grant, “Statistical Mechanics and Complexity,” of the INFM, Rome. References Abbott, L. F., & Song, S. (1999). Temporally asymmetric Hebbian learning and neuronal response variability. In M. S. Kearns, S. A. Solla, & D. A. Cohn (Eds.), Advances in neural information processing systems, 11. Cambridge, MA: MIT Press. Amit, D. J. (1995). The Hebbian paradigm reintegrated: Local reverberations as internal representations. Behavioral and Brain Sciences, 18, 617–626. Amit, D. J. (1998). Simulation in neurobiology—Theory or experiment? TINS, 21, 231–237. Amit, D. J., & Brunel, N. (1997a). Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex. Cerebral Cortex, 7, 237–252. Amit, D. J., & Brunel, N. (1997b). Dynamics of a recurrent network of spiking neurons before and following learning. Network: Computation in Neural Systems, 8, 373–404. Amit, D. J., & Fusi, S. (1994).Learning in neural networks with material synapses. Neural Computation, 6, 957–982. Bear, M. F. (1996). A synaptic basis for memory storage in the cerebral cortex. Proc. Natl. Sci. USA, 93, 13453–13459. Bi, G. Q., & Poo, M. M. (1998). Synaptic modiŽcations in cultured hippocampal neurons: Dependence on spike timing, synaptic strength and cell type. Journal of Neuroscience, 18, 10464–10472. Brunel, N. (1996). Hebbian learning of context in recurrent neural networks. Neural Computation, 8, 1677–1710. Brunel, N. (2000). Persistent activity and the single cell f-I curve in a cortical network model. Network, 11, 261–280. Brunel, N., Carusi, F., & Fusi, S. (1998). Slow stochastic Hebbian learning of classes of stimuli in a recurrent neural network. Network, 9, 123–152.

Spike-Driven Synaptic Dynamics Generating Working Memory States

595

Compte, A., Brunel, N., Goldman-Rakic, P. S., & Wang, X. J. (2000). Synaptic mechanisms and neural dynamics underlying spatial working memory in a cortical network model. Cerebral Cortex, 10, 910–923. Del Giudice, P., & Mattia, M. (2001). Long and short-term synaptic plasticity and the formation of working memory: A case study. Neurocomputing, 38, 1175–1180. Dudek, S. M., & Bear, M. F. (1992). Homosynaptic long-term depression in area CA1 of hippocampus and effects of NMDA receptor blockade. Proc. Natl. Sci. USA, 89, 4363–4367. Dudek, S. M., & Bear, M. F. (1993).Bidirectional long-term modiŽcation of synaptic effectiveness in the adult and immature hippocampus. Journal of Neuroscience, 13, 2910–2918. Erickson, C. A., & Desimone, R. (1999).Responses of macaque perirhinal neurons during and after visual stimulus association learning. Journal of Neuroscience, 23, 10404–10416. Fusi, S., & Mattia, M. (1999). Collective behavior of networks with linear (VLSI) integrate-and-Žre neurons. Neural Computation, 11, 643–652. Fusi, S., Annunziato, M., Badoni, D., Salamon, A., & Amit, D. J. (2000). Spikedriven synaptic plasticity: Theory, simulation, VLSI implementation. Neural Computation, 12, 2227–2258. Herz, A. V. M. (1996). Global analysis of recurrent neural networks. In E. Domany, J. L. van Hemm, & K. Schulten (Eds.), Models of neural networks III. New York: Springer-Verlag. Kritzer, M. F., & Goldman-Rakic, P. S. (1995). Intrinsic circuit organization of the major layers and sublayers of the dorsolateral prefrontal cortex in the rhesus monkey. Journal of Computational Neurobiology, 359, 131–143. Markram, H., Lubke, ¨ J., Frotscher, M., & Sakmann, B. (1997). Regulation of synaptic efŽcacy by coincidence of postsynaptic APs and EPSPs. Science, 275, 213–215. Mattia, M., & Del Giudice, P. (2000). EfŽcient event-driven simulation of large networks of spiking neurons and dynamical synapses. Neural Computation, 12, 2305–2329. Miller, E. K., Erickson, C., & Desimone, R. (1996). Neural mechanism of working memory in prefrontal cortex of macaque. Journal of Neuroscience, 16, 5154– 5167. Miyashita, Y. (1988). Neural correlate of visual associate long-term memory in the primate temporal cortex. Nature, 335, 817–820. Mongillo, G., & Amit, D. J. (2001a). Spike driven synaptic dynamics—a selfsaturating Hebbian paradigm. Neural Plasticity, 8, 188. Mongillo, G., & Amit, D. J. (2001b). Oscillations and irregular emission in networks of linear spiking neurons. Journal of Computational Neuroscience, 11, 249–261. Peter