No title

Neural Computation, Volume 12, Number 1

CONTENTS Articles Correctness of Local Probability Propagation in Graphical Models with Loops Yair Weiss

1

Population Dynamics of Spiking Neurons: Fast Transients, Asynchronous States, and Locking Wulfram Gerstner

43

Dynamics of Strongly Coupled Spiking Neurons Paul C. Bressloff and S. Coombes

91

Notes On Connectedness: A Solution Based on Oscillatory Correlation DeLiang L. Wang

131

Practical IdentiŽability of Finite Mixtures of Multivariate Bernoulli Distributions ´ Carreira-Perpi˜n´an and Steve Renals Miguel A.

141

Letters The Effects of Pair-wise and Higher-order Correlations on the Firing Rate of a Postsynaptic Neuron S. M. Bohte, H. Spekreijse, and P. R. Roelfsema

153

Effects of Spike Timing on Winner-Take-All Competition in Model Cortical Circuits Erik D. Lumer

181

Model Dependence in QuantiŽcation of Spike Interdependence by Joint Peri-Stimulus Time Histogram Hiroyuki Ito and Satoshi Tsuji

195

Reinforcement Learning in Continuous Time and Space Kenji Doya

219

ARTICLE

Communicated by Stuart Russell

Correctness of Local Probability Propagation in Graphical Models with Loops Yair Weiss Department of Brain and Cognitive Sciences, MIT, Cambridge, MA 02139, U.S.A.

Graphical models, such as Bayesian networks and Markov networks, represent joint distributions over a set of variables by means of a graph. When the graph is singly connected, local propagation rules of the sort proposed by Pearl (1988) are guaranteed to converge to the correct posterior probabilities. Recently a number of researchers have empirically demonstrated good performance of these same local propagation schemes on graphs with loops, but a theoretical understanding of this performance has yet to be achieved. For graphical models with a single loop, we derive an analytical relationship between the probabilities computed using local propagation and the correct marginals. Using this relationship we show a category of graphical models with loops for which local propagation gives rise to provably optimal maximum a posteriori assignments (although the computed marginals will be incorrect). We also show how nodes can use local information in the messages they receive in order to correct their computed marginals. We discuss how these results can be extended to graphical models with multiple loops and show simulation results suggesting that some properties of propagation on single-loop graphs may hold for a larger class of graphs. SpeciŽcally we discuss the implication of our results for understanding a class of recently proposed error-correcting codes known as turbo codes. 1 Introduction Problems involving probabilistic belief propagation arise in a wide variety of applications, including error-correcting codes, speech recognition, and medical diagnosis. Typically, a probability distribution is assumed over a set of variables, and the task is to infer the values of the unobserved variables given the observed ones. The assumed probability distribution is described using a graphical model (Lauritzen, 1996); the qualitative aspects of the distribution are speciŽed by a graph structure. Figure 1 shows two examples of such graphical models: a Bayesian network (Pearl, 1988; Jensen, 1996) (see Figure 1a) and Markov networks (Pearl, 1988; Geman & Geman, 1984) (see Figures 1b–c). Neural Computation 12, 1–41 (2000)

c 2000 Massachusetts Institute of Technology °

2

Yair Weiss

Figure 1: Examples of graphical models. Nodes represent variables, and the qualitative aspects of the joint probability function are represented by properties of the graph. Shaded nodes represent observed variables. (a) Singly connected Bayesian network. (b) Singly connected Markov network (A and F are observed). (c) Markov network with a loop. This article analyzes the behavior of local propagation rules in graphical models with a loop.

If the graph is singly connected—there is only one path between any two given nodes—then there exist efŽcient local message-passing schemes to calculate the posterior probability of an unobserved variable given the observed variables. Pearl (1988) derived such a scheme for singly connected Bayesian networks and showed that this “belief propagation” algorithm is guaranteed to converge to the correct posterior probabilities (or “beliefs”). However, as Pearl noted, the same algorithm will not give the correct beliefs for multiply connected networks: When loops are present, the network is no longer singly connected and local propagation schemes will invariably run into trouble. . . . If we ignore the existence of loops and permit the nodes to continue communicating with each other as if the network were singly connected, messages may circulate indeŽnitely around the loops and the process may not converge to a stable equilibrium. . . . Such oscillations do not normally occur in probabilistic networks. . . . which tend to bring all messages to some stable equilibrium as time goes on. However, this asymptotic equilibrium is not coherent, in the sense that it does not represent the posterior probabilities of all nodes of the network. (p. 195) Despite these reservations, Pearl advocated the use of belief propagation in loopy networks as an approximation scheme (J. Pearl, personal communication), and one of the exercises in Pearl (1988) investigates the quality of the approximation when it is applied to a particular loopy belief network.

Correctness of Local Probability Propagation

3

Several groups (Frey, 1998; MacKay & Neal, 1995; Weiss, 1996) have recently reported excellent experimental results by running algorithms equivalent to Pearl’s algorithm on networks with loops. Perhaps the most dramatic instance of this performance is in an error-correcting code scheme known as turbo codes (Berrou, Glavieux, Thitimajshima, 1993), described as “the most exciting and potentially important development in coding theory in many years” (McEliece, Rodemich, & Cheng, 1995) and have recently been shown to use an algorithm equivalent to belief propagation in a network with loops (Wiberg, 1996; Kschischang & Frey, 1998; McEliece, MacKay, & Cheng, 1998). Although there is widespread agreement in the coding community that these codes “represent a genuine, and perhaps historic, breakthrough” (McEliece et al., 1995), a theoretical understanding of their performance has yet to be achieved. Here we lay a foundation for an understanding of belief propagation in networks with loops. For networks with a single loop, we derive an analytic expression relating the steady-state beliefs to the correct posterior probabilities. We discuss how these results can be extended to networks with multiple loops and show simulation results for networks with multiple loops. SpeciŽcally we discuss the implication of our results for understanding the performance of turbo codes. 2 Formulation: Bayes Nets, Markov Nets, and Belief Propagation Bayesian networks and Markov networks are graphs that represent the qualitative nature of a distribution over a set of variables; the structure of the graph implies a product form for the distribution. See Pearl (1988, chap. 3) for a more exhaustive treatment. A Bayesian network is a directed acyclic graph in which the nodes represent variables, the arcs signify the existence of direct inuences among the linked variables, and the strength of these inuences is expressed by forward conditional probabilities. Using the chain rule, the full probability distribution over the variables can be expressed as a product of conditional and prior probabilities. Thus, for example, for the network in Figure 1a, we can write: P( ABCDEF) = P( A) P( B | A) P( C) P( D | B, C) P(E | C).

(2.1)

A Markov network is an undirected graph in which the nodes represent variables, and arcs represent compatability constraints between them. Assuming that all probabilities are nonzero, the Hammersley-Clifford theorem (Pearl, 1988) guarantees that the probability distribution will factorize into a product of functions of the maximal cliques of the graph. Thus, for example, in the network in Figure 1c we have: P( ABCD) =

1 Y(AB)Y(AC)Y(CD)Y(DB). Z

(2.2)

4

Yair Weiss

Different communities tend to prefer different graphical models (see Smyth, 1997, for a recent review). Directed graphs are more commonin artiŽcial intelligence, medical diagnosis, and statistics, and undirected graphs are more common in image processing, statistical physics, and error-correcting codes. In both cases, however, a full speciŽcation of the probability distribution involves specifying the graph and numerical values for the conditional probabilities (in Bayes nets) or the clique potentials (in Markov nets). Given a full speciŽcation of the probability, the inference task in graphical models is simply to infer the values of the unobserved variables given the observed ones. Figure 1b shows an example. The observed nodes A, F are represented by shaded circles in the graph. We use O to denote the set of observed variables. There are actually three distinct subtasks for the inference problem: • Marginalization: Calculating the marginal probability of a variable given the observed variables—for example P( C = c | O ). • Maximum a posteriori (MAP) assignment: Finding assignments to the unobserved variables that are most probable given the observed variables— for example, Žnding ( b, c, d, e) such that P( B = b, C = c, D = d, E = e | O ) is maximized. • Maximum marginal (MM) assignment: Finding assignments to the unobserved variables that maximize the marginal probability of the assignment—for example, Žnding ( b, c, d, e) such that P( B = b | O ),P( C = c | O ),P( D = d | O ) ,P(E = e | O ) are all maximized. 2.1 Message-Passing Algorithms for Graphical Models. For singly connected graphical models, there exist various message-passing schemes for performing inference (see Smyth, Heckerman, & Jordan, 1997 for a review). Here we present such a scheme for pairwise Markov nets—ones in which the maximal cliques are pairs of units. This choice is not as restrictive as it may seem. First, any singly connected Markov net is a pairwise Markov network, and a Markov network with larger cliques can be converted into a pairwise Markov network by merging larger cliques into cluster nodes. Furthermore, as we show in the appendix, any Bayesian network can be converted into a pairwise Markov net. When this conversion is performed, the update rules given here reduce to Pearl’s original algorithm for inference in Bayesian networks. The main advantage of the pairwise Markov net formulation is that it enables us to write the message-passing scheme in terms of matrix and vector operations, and this makes the subsequent analysis simpler. To derive the message-passing algorithm, consider Žrst the naive way of performing inference: by exhaustive enumeration. Referring again to

Correctness of Local Probability Propagation

5

Figure 1b, calculating the marginal of C merely involves computing: P(C = c | A = a, F = f ) = a

X

P( a, b, c, d, e, f ),

(2.3)

b,d,e

where a is a normalizing factor. This exhaustive enumeration is, of course, exponential in the number of unobserved nodes, but for the particular factorized distributions of Markov nets, we can write: P(C = c | A = a, F = f ) XXX = a Y(ab)Y(bc)Y(cd)Y(df )Y( de) = a

(

b

d

X

e

!

Y(ab)Y(bc)

b

(

X d

Y(cd)Y(df )

X

(2.4) !

Y(de) .

(2.5)

e

Note that the exponential enumeration is now converted into a series of enumerations over each variable separately and that many of the terms P (e.g., b Y(ab)Y(bc)) are equivalent to matrix multiplication. This forms the basis for the message-passing algorithm. The messages that nodes transmit to each other are vectors, and we denote by vEXY the message that node X sends to node Y. We deŽne the transition matrix of an edge in the graph by def

M XY (i, j) = Y(X = i, Y = j).

(2.6)

Note that the matrix going in one direction on an edge is equal by deŽnition to the transpose of the matrix going in the other direction: MXY = M TYX . We denote by bEX the belief vector at node X. Following the terminology of Pearl (1988) we call the message-passing scheme for calculating marginals belief update. In belief update, the message that node X sends to node Y is updated as follows: • Combine all messages coming into X except for that coming from Y into a vector vE. The combination is done by multiplying all the message vectors element by element. • Multiply vE by the matrix M XY corresponding to the link from X to Y.

• Normalize the product MXY vE so it sums to 1. The normalized vector is sent to Y. The belief vector for a node X is obtained by combining all incoming messages to X (again by multiplying the message vectors element by element) and normalizing.

6

Yair Weiss

By introducing a symbol ¯ for the componentwise multiplication operator, the updates can be rewritten: vEXY Ã aMXY bEX Ã a

¯

¯

Z2N( X)nY

Z2N( X)

(2.7)

vEZX

vEZX ,

(2.8)

where Ez = xE ¯ yE $ Ez( i) = xE(i) yE( i). Throughout this article aE v denotes normalizing the components of vE so they sum to 1. The procedure is initialized with all message vectors set to (1, 1, . . . , 1). Observed nodes do not receive messages, and they always transmit the same vector. If X is observed to be in state x, then vEXY (i) = Y(X = x, Y = i). The normalization of vEXY in equation 2.7 is not theoretically necessary; whether the message are normalized or not, the belief vector bEX will be identical. Thus one could use an equivalent algorithm where the messages are not normalized. However, as Pearl (1988) has pointed out, normalizing the messages avoids numerical underow and adds to the stability of the algorithm. Equation 2.7 does not specify the order in which the messages are updated. For simplicity, we assume throughout this article that all nodes simultaneously update their messages in parallel. For a singly connected network, the belief update equations (equations 2.7–2.8) solve two of the three inference tasks mentioned earlier. It can be shown that the the belief vectors reach a steady state after a number of iterations equal to the length of the longest path in the graph. Furthermore, at convergence, the belief vectors are equal to the posterior marginal probabilities. Thus for singly connected networks, bEX ( j) = P( X = j | O ). We deŽne the belief update (BU) assignment by assigning each variable X to the state j that maximizes bEX ( j). Since bEX converges to the correct posteriors for a singly connected network, the BU assignment is guaranteed to give the MM assignment. Thus two of the three inference tasks mentioned earlier can be solved using the belief update procedure. To calculate the MAP assignment, however, calculating the posterior marginals is insufŽcient. In order to calculate the MAP assignment, the nodes need to calculate the posterior when the other unobserved nodes are maximized rather than summed. In Figure 1 this corresponds to calculating P ¤ (C = c | A = a, F = f ) = a max P (a, b, c, d, e, f ) b,d,e

(2.9)

(we use the notation P¤ to denote the fact that P¤ does not correspond to a marginal probability). In complete analogy to equation 2.4 the maximization over the unobserved variables can be replaced by a series of maximizations over individual variables. This leads to an alternative message-passing scheme that we call, following Pearl (1988), belief revision.

Correctness of Local Probability Propagation

7

The procedure is almost identical to that described earlier for belief update. The only difference is that the matrix multiplication in equation 2.7 needs to be replaced with a nonlinear matrix operator. For a matrix M and 1

1

a vector xE, we deŽne the operator M such that yE = M xE if yE(i) = max M( i, j) xE( j).

(2.10)

j

1

Note that the M operator is similar to regular matrix multiplication, with the sum replaced with the maximum operator. The belief revision update rules are 1

vEXY Ã a M XY bEX Ã a

¯

¯

Z2N ( X)nY

Z2N( X )

vEZX ,

vEZX

(2.11) (2.12)

with initialization identical to belief update. Again, it can be shown that for singly connected networks, the belief vector bEX ( i) at a node will converge to the modiŽed belief P¤ ( X = i). We deŽne the belief revision (BR) assignment as assigning each node X the value j that maximizes bEX ( j) after belief revision has converged. From the deŽnition of P¤ , it follows that the BR assignment is equal to the MAP assignment for singly connected networks. Thus, all three inference tasks can be accomplished by local message passing. Update rules such as equations 2.7–2.8 and Equations 2.11–2.12 have appeared in many areas of applied mathematics and optimization. In the hidden Markov model literature, the belief update equations (2.7–2.8) are equivalent to the forward-backward algorithm, and the BR equations (2.11– 2.12) are equivalent to the Viterbi algorithm (Rabiner, 1989). In a general optimization context, equations 2.11–2.12 are a distributed implementation of standard dynamic programming (Bertsekas, 1987). If the nodes represent continuous gaussian random variables, equations 2.7–2.8 are equivalent to the Kalman Žlter and optimal smoothing (Gelb, 1974). In error-correcting codes, both belief revision and belief update can be thought of as special cases of a general class of decoding algorithms for codes deŽned on graphs (Kschischang & Frey, 1998; Forney, 1997; Aji & McEliece, in press). In nearly all the contexts surveyed above, belief propagation has been analyzed only for singly connected graphs. Note, however, that these procedures are perfectly well deŽned for any pairwise Markov network. This raises questions, including: • How far is the steady-state belief from the correct posterior when the update rules (equations 2.7–2.8) are applied in a loopy network?

8

Yair Weiss

Figure 2: Intuition behind loopy belief propagation. In singly connected networks (a), the local update rules avoid double counting of evidence. In multiply connected networks (b), double counting is unavoidable. However, as we show in the text, certain structures lead to equal double counting.

• What are the conditions under which the BU assignment equals the MM assignment when the update rules are applied in a loopy network? • What are the conditions under which the BR assignment equals the MAP assignment when the update rules are applied in a loopy network? Answering these questions is the goal of this article. 3 Intuition: Why Does Loopy Propagation Work? Before launching into the details of the analysis, let us consider the examples in Figure 2. In both graphs, there is an observed node connected to every unobserved nodes; we will refer to the observed node connected to X as the local evidence at X. Intuitively, the message-passing algorithm can be thought of as a way of communicating local evidence between nodes such that all nodes calculate their beliefs given all the evidence. As Pearl (1988) has pointed out, in order for a message-passing scheme to be successful, it needs to avoid double counting—a situation in which the same evidence is passed around the network multiple times and mistaken for new evidence. In a singly connected network (such as Figure 2a) this is accomplished by update rules such as those in equation 2.7. Thus in Figure 2a, node B will receive from A a message that involves the local evidence at A and send that information to C. The message it sends to A will involve the local evidence at C but not the local evidence at A. Thus, A never receives its own evidence back again, and double counting is avoided.

Correctness of Local Probability Propagation

9

In a loopy graph (such as Figure 2b), double counting cannot be avoided. Thus, B will send A’s evidence to C, but in the next iteration, C will send that same information back to A. Thus, it seems that belief propagation in such a net will invariably give the wrong answer. How, then, can we explain the good performance reported experimentally? Intuitively, the explanation is that double counting may still lead to correct inference if all evidence is double counted in equal amounts. Because nodes mistake existing evidence for new evidence, they are overly conŽdent in their beliefs regarding which values should be assigned to the unobserved variables. However, if all evidence is equally double counted, the assignment, although based on overly conŽdent beliefs, may still be correct. Indeed, as we show in this article, this intuition can be formalized for belief revision. For all networks with a single loop, the BR assignment gives the maximum a posteriori (MAP) assignment even though the numerical values of the beliefs are wrong. The notion of equal double counting can be formalized by means of what we call the unwrapped network corresponding to a loopy network. The unwrapped network is a singly connected network constructed such that performing belief propagation in the unwrapped network is equivalent to performing belief propagation in the loopy network. The exact construction method of the unwrapped network is discussed in section 5, but the basic idea is to replicate the nodes and the transition matrices as shown in the examples in Figure 3. As we show, (see also Weiss, 1996; MacKay & Neal, 1995; Wiberg, 1996; Frey, Koetter, & Vardy, 1998), for any number of iterations of belief propagation in the loopy network, there exists an unwrapped network such that the Žnal messages received by a node in the unwrapped network are equivalent to those that would be received by a corresponding node in the loopy network. This concept is illustrated in Figure 3. The messages received by node B after one, two, and three iterations of propagation in the loopy network are identical to the Žnal messages received by node B in the unwrapped networks shown in Figure 3. It seems that all we have done is to convert a Žnite, loopy problem into an inŽnite network without loops. What have we gained? The importance of the unwrapped network is that since it is singly connected, belief propagation on it is guaranteed to give the correct beliefs. Thus, belief propagation on the loopy network gives the correct answer for the unwrapped network. The usefulness of this estimate now depends on the similarity between the probability distribution induced by the unwrapped problem and the original loopy problem. In subsequent sections we formally compare the probability distribution induced by the loopy network to that induced by the original problem. Roughly speaking, if we denote the original probability induced on the hidden nodes in Figure 1b by P(a, b, c) = ae¡J(a,b,c) ,

(3.1)

10

Yair Weiss

Figure 3: (a) Simple loopy network. (b–d) Unwrapped networks corresponding to the loopy network. The unwrapped networks are constructed by replicating the evidence and the transition matrices while preserving the local connectivity of the loopy network. They are constructed so that the messages received by node B after t iterations in the loopy network are equivalent to those that would be received by B in the unwrapped network. The unwrapped networks for the Žrst three iterations are shown.

Correctness of Local Probability Propagation

11

then MAP assignment for the unwrapped problem maximizes PQ(a, b, c) = ae¡kJ(a,b,c)

(3.2)

for some positive constant k. That is, if we are considering the probability of assignment, the unwrapped problem induces a probability that is a monotonic transformation of the true probability. In statistical physics terms, the unwrapped network has the same energy function but at different temperature. This is the intuitive reason that belief revision, which Žnds the MAP assignment in the unwrapped network, also Žnds the MAP assignment in the loopy network. However, belief update, which Žnds marginal distributions of particular nodes, may not Žnd the MM assignment; the marginals of PQ are not necessarily a monotonic transformation of the marginals of P. Since every iteration of loopy propagation gives the correct beliefs for a different problem, it is not immediately clear why this scheme should ever converge. Note, however, that the unwrapped network of iteration t C 1 is simply the unwrapped network of size t with an additional Žnite number of nodes added at the boundary. Thus, loopy belief propagation will converge when the addition of these nodes at the boundary will not alter the posterior probability of the node in the center. In other words, convergence is equivalent to the independence of center nodes and boundary nodes in the unwrapped network. In the subsequent sections, we formalize this intuition. 4 Belief Update in Networks with a Single Loop Consider a network composed of N unobserved nodes arranged in a loop and N observed nodes, one attached to each unobserved node (see Figure 4). We denote by U1 , U2 , . . . , UN the unobserved variables, and O1 , O2 , . . . , ON the corresponding observed variables. In this section we derive a relationship between the belief at a given node bEU1 and the correct posterior probability, which we denote by pEU1 . First, let us Žnd what the messages received by node U1 converge to. By the belief update equations (see equation 2.7), the message that UN sends to U1 depends on the message UN receives from UN¡1 ¡ ¢ (4.1) vEUN U1 à aMUN U1 vEON UN ¯ vEUN¡1 UN , and similarly the message UN¡1 sends to UN depends on the message UN¡1 receives from UN¡2 : ¡ ¢ (4.2) vEUN¡1 UN à aMUN¡1 UN vEON¡1 UN¡1 ¯ vEUN¡2 UN¡1 We can continue expressing each message in terms of the one received from the neighbor until we go back in the loop to U1 : ¡ ¢ (4.3) vEU1 U2 à aMU1 U2 vEO1 U1 ¯ vEUN U1 .

12

Yair Weiss

Figure 4: Simple network with a single loop. We label the unobserved nodes U1 , . . . , UN and the observed nodes O1 , . . . , ON . We derive an analytical relationship between the belief vector at each node in the loop (e.g., EbU 1 ) and its correct marginal probability (e.g., pE U1 ).

Thus, the message UN sends to U1 at a given time step depends on the message UN sent to U1 at a previous time step (N time steps ago), (

)

()

tCN t vEUN U1 = aCN1 vEUN U1 ,

(4.4)

where the matrix CN1 summarizes all the operations performed on the message as it is passed around the loop, CN1 = MUN UN¡1 DN MUN¡1 UN¡2 DN¡1 , . . . , M U1 U2 D1 ,

(4.5)

and we deŽne the matrix Di to be a diagonal matrix whose elements are the constant messages sent from observed node Oi to unobserved Ui . A diagonal matrix is used because it enables us to rewrite the componentwise multiplication of two vectors as a multiplication of a matrix and a vector. For any vector xE, Di xE = vEOi Ui ¯ xE. Using the matrix CN1 we can formally state the claims of this section. Claims. Consider a pairwise Markov network consisting of a single loop of unobserved variables fUi gN i= 1 , each of which is connected to an observed variable Oi . DeŽne CN1 as in equation 4.5 where Di a diagonal matrix whose entries are the messages vEOi Ui . If all elements of CN1 are nonzero, then: 1. vEUN U1 converges to the principal eigenvector of CN1 .

T 2. vEU2 U1 converges to the principal eigenvector of D¡1 1 C N1 D1 .

3. The convergence rate of the messages is governed by the ratio of the largest eigenvalue of CN1 to the second-largest eigenvalue.

Correctness of Local Probability Propagation

13

4. The diagonal elements of CN1 give the correct posteriors: pEU1 ( i) = aCN1 (i, i). 5. The steady-state belief bEU1 is related to the correct posterior marginal pEU1 by: bEU1 = b pEU1 C (1 ¡ b)EqU1 , where b is the ratio of the largest eigenvalue of CN1 to the sum of all eigenvalues and qEU1 depends on the eigenvectors of CN1 . The Žrst claim follows from the recursion (see equation 4.4). Recursions of this type form the basis of the power method for Žnding eigenvalues of matrices (Strang, 1986). The method is based on the following lemma (Strang, 1986). Power method lemma. Let C be a matrix with eigenvalues l 1 , . . . , l n sorted by decreasing magnitude and eigenvectors uE1 , . . . , uEn . If |l 1 | > |l 2 |, then the E( tC1) = aCE recursion x( t) converges to a multiple of uE1 from any initial vector Px (0) Ei s.t. b1 6 = 0. The convergence factor r is given by r = |l 2 /l 1 | (the xE = i bi u distance to the steady-state vector decreases by r% at every iteration). Proof. If vE is a Žxed point for the recursion (see equation 4.4), then vE = aCN1 vE or CN1 vE = 1/ aE v so that vE is an eigenvector of CN1 . The power method lemma guarantees that in general, the method will converge to the principal eigenvector—the one with the largest eigenvalue. It is not trivial to check for a given matrix and initial condition whether the assumptions of the power method lemma are satisŽed. A sufŽcient (but not necessary) condition is that all elements of the matrix C and the initial vector xE(0) be positive. The Perron-Frobenius theorem (Minc, 1988) guarantees that for such matricesl 1 > l 2 and that C will have only one eigenvector whose elements are nonnegative: uE 1 . Thus for C with positive elements, the iterations will converge to auE1 for any initial vector xE(0) with positive elements. If the matrix contains nonnegative values, conditions for guaranteed convergence from arbitrary initial condition are more complicated (Minc, 1988). In the context of graphical models, the matrix CN1 is a product of matrices with nonnegative elements (the elements are either conditional probabilities or potential functions), but it may contain zeros. Indeed, it is possible to construct examples of graphical models for which the messages do not converge. These examples, however, are extremely rare (e.g., when all the transition matrices are deterministic). To summarize, if the matrix CN1 has only positive elements, then vEUN U1 converges to the principal eigenvector of CN1 . Proof.

To prove the second claim, we deŽne

C21 = MU2 U1 D2 MU3 U2 D3 , . . . , MU1 UN D1 ,

(4.6)

14

Yair Weiss

and in complete analogy to the previous proof, this deŽnes a recursion for vEU2 U1 and therefore the steady-state message is the principal eigenvector of C21. Furthermore, C21 can be expressed as T C21 = D¡1 1 C N1 D1 ,

(4.7)

where we have used the fact that for any two nodes X, Y MXY = MTYX . Proof. The third claim, regarding the convergence rate of the messages, also follows from the power method lemma. Thus the convergence rate of vEUN U1 is governed by the ratio of the largest eigenvalue to the second largest eigenvalue of CN1 and that of vEU2 U1 by the ratio of eigenvalues of C21. Furthermore, by equation 4.7, C21 and CN1 have the same eigenvalues. Proof. To prove the fourth claim, we denote by eEi the vector that is zero everywhere except for a 1 at the ith component; then: pEU1 (i) = a

X

P (U1 = i, U2 , . . . , UN )

(4.8)

Y(i, U2 )Y(i, O1 )Y(U2 , U3 )

(4.9)

U2 ,...,UN

X

= a

U2 ,...,UN

Y(U2 , O2 ), . . . , Y(UN , i)Y( UN , ON ) X X Y(i, U2 )Y(U2 , O2 ) Y(U2 , U3 )

= a

U2

¢¢¢

X

(4.10)

U3

Y(UN , ON )Y( UN , i)Y( i, O1 )

UN

= aEeTi M U2 U1 D2 MU3 U2 D3 , . . . , MU1 UN D1 Eei = aEeTi C21 eEi = aC21( i, i).

(4.11) (4.12) (4.13)

Note that we can also calculate the normalizing factor a in terms of the matrix C21 ; thus: pEU1 (i) =

C21 ( i, i) . trace( C21 )

(4.14)

Again, by equation 4.7, C21 and CN1 have the same diagonal elements so pEU1 ( i) = aCN1 ( i, i). E i , l i g the eigenvectors and Proof. To prove the Žfth claim, we denote by fw eigenvalues of the matrix CN1 and by fEsi , l i g the eigenvectors and eigenvalues of matrix C21 . Now, recall from the belief update rules equation 2.7, that: bEU1 = aE vU2 U1 ¯ vEUN U1 ¯ vEO1 U1 ,

(4.15)

Correctness of Local Probability Propagation

15

Using the Žrst two claims, we can rewrite this in terms of the diagonal matrix D1 and the principal eigenvectors of C21 and CN1 , bEU1 = aD1 sE1 ¯ wE1 ,

(4.16)

or, in component notation, E 1 (i). bEU1 ( i) = aD1 (i, i)sE1 ( i) w

(4.17)

However, to obtain an expression analogous to equation 4.14 we want to rewrite bEU1 solely in terms of C21. First we write C21 = SL S¡1 where the columns of S contain the eigenvectors of C21 and the diagonal elements of L contain the corresponding eigenvalues. We order S such that the principal eigenvector is in the Žrst column; thus S( i, 1) = bEs1 (i). Surprisingly, the Žrst row of S¡1 is related to E 1 , where w E 1 is the principal eigenvector of CN1 : the product D1 w C21 = SL S¡1

(4.18)

¡1 ¡1 T T ( ) L ST D 1 . CN1 = D¡1 1 C21 D1 = D1 S

(4.19)

Thus the Žrst row of S¡1 D¡1 1 gives the principal eigenvector of CN1 , or E 1 ( i) = c S¡1 (1, i), D1 ( i, i) w

(4.20)

where the constant c is independent of i. Substituting into equation 4.17 gives: bEU1 ( i) = aS( i, 1)S¡1 (1, i),

(4.21)

where a is again a normalizing constant. In fact, this constant is equal to unity since for any invertible matrix S: X (4.22) S( i, 1)S¡1 (1, i) = 1. i

Finally, we can express the relationship between pEU1 and bEU1 : pEU1 ( i) =

EeTi C21 Eei trace( C21)

EeTi SL S¡1 eEi P j lj P ( )l ¡1 ( j, i) j S i, j j P P = j lj P l 1 bEU1 ( i) C j= 2 S( i, j)l j S¡1 ( j, i) P = . jlj =

(4.23) (4.24) (4.25) (4.26)

16

Yair Weiss

Thus pEU1 can be written as a weighted average of bEU1 and a second term, which we will denote by qEU1 , ! l1 E l1 (4.27) pEU1 = P bU1 C 1 ¡ P qEU1 j lj j lj

(

with: P

j= 2 S( i,

qEU1 =

P

j)l j S¡1 ( j, i)

j= 2 l j

.

(4.28)

The weight given to bEU1 is proportional to the maximum eigenvalue l 1 . Thus the error in loopy belief propagation is small when the maximum eigenvalue dominates the eigenvalue spectrum: ! l 1 (EqU1 C pEU1 ). (4.29) pEU1 ¡ bEU1 = 1 ¡ P j lj

(

Note again the importance of the ratio between the subdominant eigenvalue and the dominant one. When this ratio is small, loopy belief propagation converges rapidly, and furthermore the approximation error is small. The preceding discussion has made no assumption about the range of possible values that variables in the networks can take. If we now assume that the variables are binary, we can show that the belief update assignment is guaranteed to give the maximum marginal assignment. Claim. Consider a pairwise Markov network consisting of a single loop of unobserved variables Ui , each connected to an observed variable Oi . Assume the unobserved variables are binary; then the belief update assignment gives the maximum marginal assignment. Proof.

Note that in this case, l 1 S(i, 1)S¡1 (1, i) Cl 2 S(i, 2)S¡1 (2, i) l 1 Cl 2 E l 1 bU1 ( i) Cl 2 (1 ¡ bEU1 ( i)) = . l 1 Cl 2

pEU1 (i) =

(4.30) (4.31)

Thus: pEU1 (i) ¡ pEU1 ( j) =

l1 ¡ l2 E (bU1 ( i) ¡ bEU1 ( j)). l 1 Cl 2

(4.32)

Thus pEU1 (i) ¡ pEU1 ( j) will be positive if and only if bEU1 ( i) ¡ bEU1 ( j) is positive (l 1 > l 2 , l 1 > 0 and trace( C12) > 0). In other words the calculated beliefs may be wrong but are guaranteed to be on the correct side of 0.5.

Correctness of Local Probability Propagation

17

4.1 Correcting the Beliefs Using Locally Available Information. Equation 4.26 suggests that if node U1 was able to estimate the higher-order eigenvalues and eigenvectors of the matrix C21 from the messages it receives, it would be able to correct the belief vector. Indeed, there exist several numerical algorithms to estimate all eigenvectors of a matrix using recursions similar to equation 4.4 (Strang,1986). Claim. Consider a pairwise Markov network consisting of a single loop of unobserved binary variables fUi gN i= 1 , each of which is connected to an observed variable Oi . For node Ui deŽne the temporal difference () () ( ) () ( ) , and the ratio r as the solution to dEi t = rdEi t¡N . Then dEi t = vEUt 2 U1 ¡ vEUt¡N 2 U1 1 E r (1 ¡ bEUi ). pEUi = 1Cr bUi C 1Cr The proof is based on the following lemma: Temporal difference lemma. Let C be a matrix with eigenvalues l 1 , . . . , l n sorted by decreasing magnitude and eigenvectors uE1 , . . . , uEn . Assume that the eigenvectors are normalized so they sum to 1. DeŽne the recursion xE( tC1) = aCE x( t) ( t) ( t) ( t¡1) E and the temporal difference d = xE ¡ xE . Assume that the conditions of the power method lemma hold, and in addition, |l 2 | > |l 3 |, then dE( t) approaches a multiple of uE 2 ¡ uE 1 , and furthermore dE(t) approaches ll 2 dE( t¡1) . 1

Proof. This lemma can be proved by linearizing the relationship between xE( tC1) and xE( t) around the steady-state value xE1 and then applying the powermethod lemma. The temporal difference lemma guarantees that each component of dE in the claim will decrease geometrically in magnitude, and the rate of decrease gives the ratio r = l 2 / l 1 . Substituting r = l 2 /l 1 into equation 4.31 gives: pEU1 =

1 E r (1 ¡ bEU1 ). bU C 1 Cr 1 1 Cr

(4.33)

More complicated correction schemes are possible. In the case of ternary nodes, the nodes can use the direction of the vector dE to calculate the second eigenvector and obtain a better estimate of the correct probability. In general, all eigenvectors and eigenvalues can be calculated by performing operations on the stream of messages a node receives. 4.2 Illustrative Examples. Figure 5a shows a network of four nodes, connected in a single loop. Figure 5b shows the local probabilities—the probability of a node’s being in state 1 or 2 given only its local evidence. The transition matrices were set to: M=

¡ 0.9 0.1

¢

0.1 . 0.9

(4.34)

18

Yair Weiss

Figure 5: (a) Simple loop structure. (b) Local evidence probabilities (EvOi U i ) used in the simulations. (c) Differences between messages at four iterations (t) (t¡4) (| vE U2 U 1 ¡ vE U2 U1 |). Note the geometric convergence. (d) Log of the magnitude of the differences plotted in c. The slope of this line determines the ratio of the second and Žrst eigenvalues.

Figure 5c shows the convergence rate of one of the messages as a function of iteration. We plot kdE( t) k as a function of iteration, and the magnitude can be seen to decrease geometrically (or linearly in the log plot in Figure 5d). The rate at which the components of dE(t) decrease gives the value of r that the nodes can use to correct the beliefs. Figure 6a shows the correct posteriors (calculated by exhaustive enumeration) and the steady-state beliefs estimated by loopy propagation. Note that the estimated beliefs are on the correct side of 0.5 as guaranteed by the theory but are numerically wrong. In particular, the estimated beliefs are overly conŽdent (closer to 1 and 0, and further from 0.5). Intuitively, this is due to the fact that the loopy network involves counting each evidence multiple

Correctness of Local Probability Propagation

19

Figure 6: (a) Correct marginals (calculated by exhaustive enumeration) for the data in Figure 5. (b) Beliefs estimated at convergence by the loopy belief propagation algorithm. Note that the estimated beliefs are on the correct side of 0.5 as guaranteed by the theory but are numerically wrong. In particular, the estimated beliefs are overly conŽdent (closer to 1 and 0, and further from 0.5). (c) Estimated beliefs after correction based on the convergence rates of the messages (see equation 4.33). The beliefs are identical to the correct beliefs up to machine precision.

times rather than a single time, thus leading to an overly conŽdent estimate. More formally, this is a result of l 2 being positive (see equation 4.31). Figure 6c shows the estimated beliefs after the correction of equation 4.33 has been added. The beliefs are identical to the correct beliefs up to machine precision. Figure 7 illustrates the relationship between the asymptotic convergence ratio | r| = kdE(tC4) k/ kdE( t) k and the approximation error kbEX ¡ pEX k at a given node. Each data point represents the results on a simple loop network (see

20

Yair Weiss

Figure 7: Maximal error between the beliefs estimated using loopy belief update and the true beliefs, plotted as a function of the convergence ratio. As predicted by the theory, the error is small when convergence is rapid.

Figure 5a) with randomly selected transition matrices. Consistent with equation 4.26 the error is small when convergence is rapid and large when convergence is slow. 4.3 Networks Containing a Single Loop and Additional Trees. The results of the previous section can be extended to networks that contain a single loop and additional trees. An example is shown in Figure 8a. These networks include arbitrary combinations of chains and trees, except for a single loop. The nodes in these networks can be divided into two categories: those in the loop and those not in the loop. For example in Figure 8, nodes 1–4 are part of the loop, while nodes 5–7 are not. Let us focus Žrst on nodes inside the loop. If all we are interested in is obtaining the correct beliefs for these nodes, then the situation is equivalent to a graph that has only the loop and evidence nodes. That is, after a Žnite number of iterations, the messages coming into the loop nodes from the nonloop nodes will converge to a steady-state value. These messages can be thought of as messages from observed nodes, and the analysis of the previous section holds. For nodes outside the loop, however, the relationship derived in the previous section between pEX and bEX does not hold. In particular, there is no

Correctness of Local Probability Propagation

21

Figure 8: (a) A network including a tree and a loop. (b) The probability of each node given its local evidence used in the simulations.

guarantee that the BU assignment will give the MM assignment, even in the case of binary nodes. Consider node 5 in the Žgure. Its posterior probability can be factored into three independent sources: the probability given the evidence at node 6, at node 7, and the evidence in the loop. The messages that node 5 receives from nodes 6 and 7 in the belief update algorithm will correctly give the probability given the evidence at these nodes. However, the message it receives from node 4 will be based on the messages passed around the loop multiple times; that is, evidence in the loop will be counted more times than evidence outside the loop. Nevertheless, if nodes in the loop use the convergence rate to correct their beliefs, they can also correct the messages they send to nodes outside the loop in a similar way. One way to do this is to calculate outgoing messages vEXY by dividing the belief vector bEX componentwise by the vector vEYX and multiplying the result by M XY . If bEX is as deŽned by equation 2.8 then this procedure gives the exact same messages as equation 2.7. However, if bEX represents the corrected beliefs, then the outgoing messages will also be corrected, and the beliefs in all nodes will be correct. To illustrate these ideas, consider Figures 8 and 9. Figure 8b shows the local probabilities for each of the seven nodes. The transition matrix was identical to that used in the previous example (see equation 4.33). Figure 9a shows the correct posterior probabilities calculated using exhaustive enumeration and Figure 9b the beliefs estimated using regular belief update on the loopy network. Note that the beliefs for the nodes inside the loop are on the correct side of 0.5 but overly conŽdent. But the belief of node 6 is not on

22

Yair Weiss

Figure 9: (a) Correct posterior probabilities calculated using exhaustive enumeration. (b) Beliefs estimated using regular belief update on the loopy network. Note that the beliefs for the nodes inside the loop are on the correct side of 0.5 but overly conŽdent. The belief of node 6 is not on the correct side of 0.5. (c) Results of the corrected belief propagation algorithm, modiŽed by using equation 4.33 for nodes in the loop. The results are identical to the correct posteriors up to machine precision.

the correct side of 0.5. Finally, Figure 9c shows the results of the corrected belief propagation algorithm, by using equation 4.33 for nodes in the loop. The results are identical to the correct posteriors up to machine precision. 5 Belief Revision in Networks with a Single Loop For belief revision the situation is simpler than belief update. In belief update we had to distinguish between binary nodes versus nonbinary nodes, and between networks consisting of a single loop versus networks with a loop

Correctness of Local Probability Propagation

23

and additional trees. For belief revision we can prove the optimality of the BR assignment for any net with a single loop. Claim. Consider a pairwise Markov net with a single loop. If all potentials Y are positive and belief revision converges, then the BR assignment gives the MAP assignment. The proof uses the notion of the “unwrapped” network introduced in section 3. Recall that for every loopy net, node X, and iteration t, there exists an unwrapped network such that the messages received by node X in the loopy net after t iterations are identical to the Žnal messages received by the corresponding node in the unwrapped network after convergence. This means that the BR assignment at node X after t iterations corresponds to the MAP assignment for the corresponding X in the unwrapped network. It does not, however, tell us anything directly about the MAP assignment for other replicas of X in the unwrapped network. When belief revision converges, however, the BR assignment at X gives us the MAP assignment at nearly all replicas of X in the unwrapped network. This is shown in the following two lemmas. Unwrapped network lemma. For every node X in a loopy network and itera( t) tion time t, we can construct an unwrapped network such that all messages vEYX that X receives from a neighboring node Y at time t in the loopy network are equal to the Žnal messages vEYX that X receives from the corresponding node in the unwrapped network. Proof. The proof is by construction. We construct an unwrapped tree by setting X to be the root node and then iterating the following procedure t times: • Find all leafs of the tree (nodes that have no children). • For each leaf, Žnd all k nodes in the loopy graph that neighbor the node corresponding to this leaf. • Add k¡1 nodes as children to each leaf, corresponding to all neighbors except the parent node. The transition matrices are set to be identical to those in the loopy network. Likewise, if a node X in the loopy network is observed to be in state i, then all replicas of X are also set to be observed in state i. Thus, any node in the interior of the unwrapped tree has the same neighbors as the corresponding node in the loopy network and the same transition matrices. It is easy to verify that the messages that the root node receives in the unwrapped tree are identical to those received by the corresponding node in the loopy network after t iterations.

24

Yair Weiss

Figure 10: Illustration of the procedure for generating an unwrapped network. At every iteration, we examine all leaf nodes in the tree and Žnd the neighbors of the corresponding node in the loopy network. We add nodes as children to each leaf corresponding to all neighbors except the parent node.

Figure 10 illustrates the construction of the unwrapped network for a network with a single loop. Although we have used a tree structure to generate the unwrapped network, for the case of a single-loop network, the unwrapped network is equivalent to an increasingly long chain (corresponding to the loop nodes) and additional tree nodes connected to the chain at the corresponding nodes. Figure 11a shows the same unwrapped network as in Figure 10 but drawn as a long chain. We deŽne the end point nodes of the unwrapped network as the two loop nodes that are most distant from the root. Thus, in Figure 11a, U200 and U003 are end point nodes. Any unwrapped network deŽnes a joint probability distribution, and we can therefore deŽne the MAP assignment of the unwrapped network. Although the unwrapped network is constructed so that all replicas of X have

Correctness of Local Probability Propagation

25

identical neighbors, there is no constraint that their MAP assignment is identical. Thus in Figure 11a, the MAP assignment in the unwrapped network requires assigning values to 11 variables. There is no guarantee that replicas of the same node (e.g., U1 , U10 ) will have the same assignment. However, as we show in the following lemma, if belief revision converges the MAP assignment for the unwrapped network is guaranteed to be semiperiodic. All replicas of a given node in the unwrapped network will have the same MAP assignment as long as they are far away from the end point nodes. Periodic assignment lemma. Assume all messages in a network with a single ( t) = vE1 loop converge to their steady-state values after tc iterations: vEXY XY for t > tc . If Y is a node in the unwrapped network such that the distance of Y to both end points is greater than tc , then the MAP assignment at Y is equal to the BR assignment of the corresponding node in the loopy network. Proof. We Žrst show that all messages transmitted in the unwrapped network are equal to the steady-state messages in the loopy network. To show this, we divide the nodes in the unwrapped network into two categories: chain nodes, which form part of the inŽnite chain (corresponding to nodes in the loop in the loopy network), and nonchain nodes (corresponding to nodes outside the loop in the loopy network). This gives four types of messages in the unwrapped network. Leftward and rightward messages are the two directions of propagation inside the chain, outward are propagated in the nonchain nodes away from the chain nodes, and inward messages go toward the chain nodes. Thus in Figure 11a, vEU2 U1 is a rightward message, vEU3 U1 is a leftward message, vEU3 U4 is an outward message, and vEU4 U3 is an inward message. The inward messages depend on only other inward messages and the evidence at the nonchain nodes. Thus, they are identical at all replicas of the nonchain nodes. Since the transition matrices and evidences are replicated from the loopy network, the inward messages are identical to the corresponding messages in the loopy network. The leftward messages depend on only other leftward messages to the right of them and the inward message. By construction of the unwrapped network, if X is a node in the unwrapped network whose distance from the ( d) right-most end point is d and Y is its neighbor on the left, then vEXY = vEXY ; that is, the message is identical to the message that X sends to Y in the loopy network at time d. Since d > tc vEXY = vE1 XY . The proof for the rightward messages is analogous to that of the leftward messages. Finally, the outward messages that a chain node sends to a nonchain node depend on only the leftward and rightward messages. Thus if both leftward and rightward messages that it receives are equal to the steady-state messages in the loopy network, so will the outward message it sends out.

26

Yair Weiss

Figure 11: (a) For networks with a single loop, the unwrapped network is equivalent to an increasingly long chain (corresponding to the loop nodes) and additional trees connected at the corresponding nodes. (b) When belief revision converges, the MAP assignment of the unwrapped network contains periodic replicas of the MAP assignment in the loopy network.

If X is a node in the unwrapped tree whose distance from the end point is more than tc , all incoming messages to X are identical to the steady-state messages in the loopy network in the corresponding node. Therefore, the belief vector at X is identical to the corresponding steady-state belief in the loopy network, and the MAP assignment at X is equal to the BR assignment in the corresponding node. The periodic assignment lemma guarantees that if belief revision converges, then the MAP assignment in the unwrapped tree will have replicas of the BR assignment for all nodes whose distance from the end points is greater than tc . The importance of this result is that we can now relate optimality in the unwrapped tree to optimality in the loopy network. It is easier here to work with log probabilities than with probabilities. We deŽne J( X, Y) = ¡ log Y(X, Y). For any Markov network with observed variables, we refer to the network cost function J as the negative log posterior of the unobserved variables. Thus in Figure 10a, J(fui g) = J(u1 , u2 ) C J(u2 , u3 ) C J (u3 , u1 ) C J( u3 , u4 ) C J( u4 , o4 ), where o4 is the observed value of O4 .

(5.1)

Correctness of Local Probability Propagation

27

Optimality relationship lemma. Assume that belief revision converges in a network with a single loop and u¤i is the BR assignment in the loopy network. Assume furthermore that all potentials Y(X, Y) are nonzero, and deŽne J as the negative log posterior of the loopy network. For any n, u¤i must minimize Jn (fui g) = nJ (fui g) C J,Q

(5.2)

where JQ is Žnite and independent of n. Proof. To prove this, note that for any n there exists a t such that the unwrapped network at iteration t contains a subnetwork with n copies of all links in the loopy network. Furthermore, by choosing a sufŽciently large t, the distance of all nodes in the subnetwork from the end points can be made sufŽciently large. By the periodic assignment lemma, the MAP assignment will have replicas of the BR assignment for all nodes within the subnetwork. Furthermore, by the deŽnition of the MAP assignment, the assignment within the subnetwork maximizes the posterior probability when nodes outside the subnetwork are clamped to their MAP assignment values. We denote by Jn the cost function that the assignment of the subnetwork minimizes. The cost function decomposes into a sum of pairwise costs, one for each link in the subnetwork. For a periodic assignment, the cost function can be rewritten as Jn (fui g) = nJ (fui g) C J,Q

(5.3)

where the JQ term captures the existence of links between the boundaries of the chain and its observed neighbors. This term is independent of n (the distance of the neighboring nodes to the end points is also greater than tc ). Since periodic copies of fu¤i g form the MAP assignment for the subnetwork, fu¤i g must minimize Jn . Figure 12 illustrates the optimality relationship lemma for n = 2. The subnetwork encircled by the dotted line includes two copies of all links in the original loopy network. Therefore the subnetwork cost Jn is given by Jn (fui g) = 2J(fui g) C JQ(fui g),

(5.4)

with JQ(fui g) a sum of the two links that connect the subnetwork to the larger unwrapped network: J( u¤2 , u1 ) C J (u1 , u¤3 ). Proof. The optimality relationship lemma essentially proves the main claim of this section. Since the BR assignment minimizes Jn for arbitrarily large n and JQ is independent of n, the BR assignment must minimize J and hence is the MAP assignment for the loopy network. To illustrate this analysis, Figure 13 shows two network structures for which there is no guarantee that the BU assignment will coincide with the

28

Yair Weiss

Figure 12: Proof of the optimality relationship lemma for networks with a single loop. For any n there exists a t such that the unwrapped network of size t includes a subnetwork that contains exactly n copies of all links in the original loopy network.

MM assignment. The loop structure has ternary nodes, while the added tree on the right means that the BU assignment even for binary variables will not necessarily give the MM assignment. However, due to the claim proved above, the BR assignment should always coincide with the MAP assignment. We selected 5000 random transition matrices for the two structures and calculated the correct beliefs using exhaustive enumeration. We then counted the number of correct assignments using belief update and belief revision by comparing the BU assignment to the MM assignment and the BR assignment to the MAP assignment. We considered assignment only when belief revision converged (about 95% of the simulations). The results are shown in Figure 13. In both of these structures, the BU assignment is not guaranteed to give the MM assignment, and indeed the wrong assignment is sometimes observed. However, an incorrect BR assignment is never observed. 6 Discussion: Single Loop The analysis enables us to categorize a given network with a loop into one of two categories: (1) structures for which both belief update and belief revision are guaranteed to give the correct assignment or (2) those for which only belief revision is guaranteed to give the correct assignment.

Correctness of Local Probability Propagation

29

Figure 13: (a–b) Two structures for which the BU assignment need not coincide with the MM assignment, but the BR assignment is guaranteed to give the MAP assignment. (a) Simple loop structure with ternary nodes. (b) Loop adjoined to a tree. (c–d) Number of correct assignments using belief update and belief revision in 5000 networks of these structure with randomly generated transition matrices. Consistent with the analysis, the BU assignment may be incorrect, but an incorrect BR assignment is never observed.

The crucial element determining the performance of belief update is the eigenspectrum of the matrix CN1 . Note that this matrix depends on both the transition matrices and the observed data. Thus belief update may give very exact estimates for certain observed data even in structures for which it is not guaranteed to give the correct assignment. We have primarily dealt here with discrete state spaces, but the analysis can also be carried out for continuous spaces. For the case of gaussian posterior probabilities, it is easy to show that belief update assignment will give the correct MAP assignment for all gaussian Markov networks with single

30

Yair Weiss

loop (this is due to the fact that belief update and revision are identical for a gaussian). We emphasize again that our choice of Markov nets with pairwise potentials as the primary object of analysis was based mainly on notational convenience. Due to the equivalence between Pearl’s algorithm on Bayesian nets and our update rules on Markov nets, our results also hold for applications of Pearl’s original update rules on Bayesian nets with a single loop. In particular, this means that one can perform exact inference on a Bayesian net with a single loop by running Pearl’s propagation algorithm and then using a correction analogous to equation 4.33. More generally, if the Bayesian network can be converted into a Markov network with a single loop, our results show how to perform exact inference. Exact inference in networks containing a single loop, however, can also be performed using cut-set conditioning (Pearl, 1988). Our goal in analyzing belief propagation in networks with a single loop is primarily to understand loopy belief propagation in general, with single-loop networks being the simplest special case. Independent of our work, several groups working in the context of errorcorrecting codes have recently obtained results on probability propagation in networks with a single loop. Aji, Horn, and McEliece (1998) have shown that iterative decoding (equivalent to belief update) of a single-loop code will converge to a correct decoding for binary nodes. They also showed that the messages will converge to the principal eigenvectors of a matrix analogous to our matrix C21. Forney, Kschischang, and Marcus (1998) have also shown the convergence of the messages to the principal eigenvectors and have additionally analyzed the “max-sum” decoding algorithm (analogous to belief revision) on a network consisting of a single loop, showing that it will converge to the dominant pseudocodeword—a periodic assignment on the unwrapped network such that the average cost per period is minimal. To the best of our knowledge, the analytical relationship between the correct beliefs and the steady-state ones, the analysis of networks including a single loop and arbitrary trees, and the correction of beliefs using locally available information have not been discussed elsewhere. One of the important distinctions that arise from analyzing networks containing a single loop and arbitrary trees is the difference in performance between belief update and belief revision when both converge. The BR assignment is always guaranteed to give the MAP assignment, while the BU assignment is guaranteed to give the MM assignment only when there are no additional trees attached to the loop and the nodes are binary. As we show in subsequent sections, this difference in performance is also apparent in networks with multiple loops. 7 Networks with Multiple Loops The basic intuition that explains why belief propagation works in networks with a single loop is the notion of equal double counting. Essentially, while

Correctness of Local Probability Propagation

31

evidence is double counted, all evidence is double counted equally, as can be seen in the unwrapped network, and hence belief revision is guaranteed to give the correct assignment. Furthermore, by using recursion expressions for the messages, we could quantify the “amount” of double counting, and derive a relationship between the estimated beliefs and the true ones. For networks with multiple loops, the situation is more complex. In particular, the relationship between the messages sent at time t and at time t C n is not as simple as in the single loop case, and tools such as the power method lemma or the temporal difference lemma are of no immediate use. However, the notion of the unwrapped tree is equally well applied to networks with multiple loops. Figure 14 shows an example. The unwrapped network lemma of the previous section holds; that is, the messages received by node A in the loopy network after four iterations are identical to the Žnal messages received by node A in the unwrapped network. Furthermore, a version of the periodic assignment lemma of the previous section holds for any network; if belief revision converges, then the MAP assignment for arbitrarily large unwrapped networks contains periodic replicas of the BR assignment. Thus for a network with multiple loops, if BR converges, then one can construct arbitrarily large problems such that periodic copies of the BR assignment give the optimal assignment for that problem. For example, for the network in Figure 14, the periodic assignment lemma guarantees that if fu¤i g is the BR assignment, then it minimizes Jn (fui g) = nJ (fui g) C JQn

(7.1)

for arbitrarily large n. As in the single-loop case, JQn is a boundary cost reecting the constraints imposed on the end points of the unwrapped network. Note that unlike the single-loop case, the boundary cost here may increase with increasing n. Because the boundary cost may grow with n, we cannot automatically assume that if an assignment minimizes Jn , it also minimizes J; that is, the assignment may be optimal only because of the boundary cost. We have not been able to determine analytically the conditions under which the steadystate assignment is due to JQ as opposed to J. However, the fact that equation 7.1 holds for arbitrarily large subnetworks (such that the boundary can be made arbitrarily far from the center node) leads us to hypothesize that for positive potentials Y loopy belief revision will give the correct assignment for the network in Figure 14. Although we have not been able to prove this hypothesis, extensive simulation results are consistent with it. We performed loopy belief revision on the structure shown in Figure 14 using randomly selected transition matrices and local evidence probabilities. We repeated the experiment 10,000 times and counted the number of correct assignments. As in the previous section, a correct assignment for belief revision is one that is identical to the MAP

32

Yair Weiss

Figure 14: (Top) A Markov network with multiple loops. (Bottom) Unwrapped network corresponding to this structure. Note that all nodes and connections in the loopy structure appear equal numbers of time in the unwrapped network, except the boundary nodes A.

assignment, and a correct assignment for belief update is one that is identical to the MM assignment. Again, we considered only runs in which belief revision converged. We never observed a convergence of belief revision to an incorrect assignment for this structure, while belief update gave 247 incorrect assignments (error rate 2.47%). Since these rates are based on 10,000 trials, the difference is highly signiŽcant. While by no means a proof, these

Correctness of Local Probability Propagation

33

simulation results suggest that some of our analytic results for single-loop networks may also hold for a subclass of multiple loop networks. Trying to extend the proofs for a broader class of networks is a topic of current research. 7.1 Turbo Codes. Turbo codes are a class of error-correcting codes whose decoding algorithm has recently been shown to be equivalent to belief propagation on a network with loops. To make the connection to the update rules (equations 2.7–2.8) explicit, we briey review this equivalence. We use the formulation of turbo decoding presented in (McEliece et al., E is encoded using two codes, each of 1995). An unknown binary vector U which is easy to decode by itself, and the results are transmitted over a noisy channel. The task of turbo decoding is to infer the values of every bit E from two noisy versions Y E 1 , YE 2 that are received after transmission. of U E (which we assume here is The decoder knows the prior distribution over U E = uE ). uniform) and the two conditional probabilities pi ( yE | uE ) = P (YE i = yE | U Since both noisy versions are assumed to be conditionally independent, the marginal probability ratio is simply P E E E E E U( i)= 1 P( Y1 | U ) P ( Y 2 | U ) P(U ( i) = 1) U: = P . (7.2) E )P (YE2 | U E) P(U ( i) = 0) P( YE1 | U E U: U( i)= 0

In the typical error-correcting code application, the sum in equation 7.2 is intractable. Rather, the Turbo decoder approximates the ratio by the iterative algorithm outlined in Figure 15. The algorithm consists of two turbo decision modules, each of which E i . Each module receives a likelihood considers only one “noisy version” Y ratio vector as input and outputs a modiŽed likelihood ratio vector that E i . If TE = TDM1 ( SE), then: takes into account the observed Y P Q ( E) j6= i SE( j) uE( j) uE : uE( i)= 1 p yE1 | u . (7.3) TE (i) = P Q E) j6= i SE( j) uE( j) uE : uE( i)= 0 p( yE1 | u The output of module 2 is identical with yE1 replaced by yE2 . Note that each module still requires summing over all uE , but the codes are constructed so that pi ( yE | uE) have a factorized structure that enables calculation of TDMi ( SE) efŽciently. The turbo decoding iterations can be written as TE à TDM1 ( SE) SE à TDM2 ( TE ),

(7.4) (7.5)

and the likelihood ratio is approximated as E RE = TE ¯ S.

(7.6)

34

Yair Weiss

Figure 15: (a) The Turbo decoding algorithm consists of two Turbo decision modules such that the output of each module serves as the input to the other one. The posterior likelihood ratio is approximated by multiplying the outputs S, T at convergence. (b) The Turbo decoding algorithm as belief update in a loopy Markov network. Each unknown bit is represented by a node Ui . The belief of a node is approximated by multiplying the two incoming messages (corresponding to S and T). When the link matrices are set as explained in the text, the update rules for the messages (equation 2.7–2.8) reduce to the turbo decoding algorithm.

To see the equivalence of this scheme to belief update in Markov nets, E is of length consider Figure 15b. We assume that the unknown vector U 3 and represent the value of the ith bit with a node Ui in the graph. Each Ui is connected to exactly two nodes, X1 , X2 , that are in turn connected to the received “noisy versions” Y1 , Y2 . By the belief update equations (see equation 2.8), the belief of node Ui is equal to the pointwise multiplication of the messages it receives from X1 and X2 . Note the similarity to equation 7.6, where the likelihood ratio for U (i) is obtained by multiplying S(i) and T ( i). Indeed, we now show that when the potentials in the graph in Figure 15b are set in a particular form, S( i) is equivalent to the message that X1 sends to Ui and T ( i) is equivalent to the message that X2 sends to Ui . E and Xi are meant to represent copies of the unknown binary message U hence in the example (of message length 3), Xi can take on eight possible values. We set Y( xEi , uj ) = 1 if xEi ( j) = uj and zero otherwise. Furthermore, we set Y(xEi , yEi ) = pi ( yE| xE). When the potentials are set in this form, the belief update equations (see equation 2.7) give the turbo decoding algorithm. Denote the message vEX2 Ui sent by node X2 to Ui by the vector a(Ti , 1). The message vEUi X1 that Ui sends

Correctness of Local Probability Propagation

35

to X1 is a vector of length 8. It is obtained by taking the message Ui receives from X2 and multiplying it by the transition matrix M Ui X1 : vEU1 X1 ( x1 ) = a

X u1

Y(x1 , ui ) vEX2 Ui (ui )

(7.7)

= aT iX1 ( i) .

(7.8)

Now, the message vEX1 Ui sent from X1 to U1 is a vector of length 2 that is obtained by multiplying all incoming messages except the one coming from Ui and then passing the result through the transition matrix MX1 Ui : vEX1 Ui ( ui ) =

X x1

Y(x1 , ui ) p1 ( y1 | x1 )

Y j6 = i

x ( j)

Tj 1 .

(7.9)

If we denote by SE( i) the ratio of components of vEX1 Ui , then equation 7.9 is equivalent to equation 7.3. Thus the belief update rules in equations 2.7–2.8 give the turbo decoding algorithm. McEliece et al. (1995) showed several examples where the turbo decoding algorithm converges to an incorrect decoding. Since the turbo decoding algorithm is equivalent to belief update, we wanted to see how well a decoding algorithm equivalent to belief revision will perform. We randomly selected the probability distributions p1 ( u) and p2 ( u) and calculated the MAP assignment (often referred to as the ML decoding in coding applications) and the MM assignment (referred to as the optimal bitwise decoding, or OBD, in coding applications). We compared these assignments to the BU and BR assignments. As in previous sections, the BU assignment was judged correct if it agreed with the MM assignment, and the BR assignment was judged correct if it agreed with the MAP assignment. We repeated the experiment for 10, 000 randomly selected probability distributions. Only trials in which belief revision converged were considered. Results are shown in Figure 16. As observed by McEliece et al. (1995) turbo decoding is quite likely to converge to an incorrect decoding (16% of the time). In contrast, belief revision converges to an incorrect answer less than 0.1% of the runs. Note that this difference is based on 10,000 trials and is highly signiŽcant. However, the results considered only cases where belief revision converged, and in general belief revision converged less frequently than did belief update. A turbo decoding algorithm that used maximization rather than summation in equation 7.3 was presented in Benedetto, Montorsi, Divsalar, and Pollara (1996), where the maximization was presented as an approximation to the full sum in equation 7.3. The decoding performance of the approximate algorithm was shown to be slightly worse than standard turbo decoding. In our simulations, however, we considered only decodings where belief revision converged, whereas in Benedetto et al. (1996), the decoding

36

Yair Weiss

Figure 16: Results of belief revision and update on the turbo code problem.

obtained after a Žxed number of iterations was used, regardless of whether the algorithm converged. Our motivation in considering only decodings at convergence was based on our analysis of single-loop networks. The simulation results suggest that turbo codes exhibit a similar difference in performance between belief update and belief revision. From a practical point of view, of course, errors due to failures in convergence are as troubling as errors due to a convergence to a wrong answer. Thus understanding the behavior of the decoding algorithms when they do not converge is an important question for further research. Forney et al. (1998) have obtained some encouraging results in this regard. Unlike the networks considered in the previous sections, the turbo code Markov net has deterministic links (between the Xi and Uj ), and hence the log of the potential functions Y may be inŽnite. We believe this may be the reason for the existence of a few incorrect decodings with the belief revision. The deterministic links make it difŽcult to relate optimality in the unwrapped tree to optimality in the original loopy network. 7.2 Discussion. Turbo codes are not the only error-correctingcode whose decoding is equivalent to belief propagation. Indeed one of the Žrst derivations of an algorithm equivalent to belief propagation appears in Gallager (1963) as a means of decoding codes known as low-density parity check codes. Gallager ’s decoding algorithm is equivalent to probability propagation on a network with loops, and Gallager discussed the fact that his decoding algorithm is not guaranteed to give the optimal decoding because it considers replicas of the same evidence as if they were independent. Gal-

Correctness of Local Probability Propagation

37

lager also showed that his algorithm will give the MAP decoding for a computation tree, equivalent to our unwrapped network. He noted that if the number of iterations is small compared to the size of the loops in the graph, the computation tree will not contain replicas of the same evidence. He presented a method of constructing codes such that the size of the loops in the network will be large. Indeed, if double counting is to be avoided, we would expect performance of belief propagation in loopy nets to be best when the size of the loops is large or (equivalently) the number of iterations is small. Turbo codes, however, contradict both of these expectations. The Markov network corresponding to turbo decoding (see Figure 15b) contains many loops of size 4. Furthermore, it has been well established (McEliece et al., 1996; Benedetto et al., 1996) that turbo decoding performance improves as the number of iterations is increased. The performance of turbo codes is consistent, however, with the expectation based on the notion of equal double counting emphasized throughout this article. As we saw in the analysis of networks with a single loop, the BR assignment at convergence is guaranteed to give the MAP assignment regardless of the size of the loop. Furthermore, as we discussed in section 3, for any Žnite number of iterations, the unwrapped network will not contain an equal number of replicas of all nodes and connections. It is only as the size of the unwrapped network approaches inŽnity that the inuence of the boundary nodes can be neglected. Thus, additional iterations are expected, based on our analysis, to increase the decoding performance. Despite the importance of turbo codes, one should use caution in generalizing from their performance to general loopy belief propagation. In the typical turbo decoding setting, the probabilities P( yEi | uE) in equation 7.3 are highly peaked around the correct uE . Thus the sum in equation 7.3 is typically dominated by a single term, and therefore belief revision (in which the sum is approximated by its largest value) and belief update behave quite similarly. As we have shown throughout this article, this is not the case for general loopy belief propagation, nor is it the case for Turbo decoding when the probabilities P( yEi | uE ) are chosen randomly in simulations. We believe that additional insights into Turbo decoding will be obtained by comparing the performance of the algorithm at different probability regimes. 8 Conclusion As many authors have observed, in order for belief propagation to be exact, we must Žnd a way to avoid double counting. Since double counting is unavoidable in networks with loops, belief propagation in such networks may seem to be a bad idea. The excellent performance of turbo codes motivates a closer look at belief propagation in loopy networks. Here we have shown a class of networks for which, even though the evidence is double counted,

38

Yair Weiss

all evidence is equally double counted. For networks with a single loop, we have obtained an analytical expression relating the beliefs estimated by loopy belief propagation and the correct beliefs. This allows us to Žnd structures in which belief update may give the wrong beliefs, but the BU assignment is guaranteed to give the correct MM assignment. We have also shown that for networks with a single loop, the BR assignment is guaranteed to give the MAP assignment. For networks with multiple loops we have presented simulation results suggesting that some of the results we have obtained analytically for single-loop networks may hold for a large class of networks. These results are an encouraging Žrst step toward understanding belief propagation in networks with loops. Appendix: Converting a Bayesian Net to a Pairwise Markov Net There are various ways to convert a Bayesian network into a Markov network that represents the same probability distribution. Indeed one of the most popular inference algorithms for Bayesian networks is to convert them into a Markov network known as the join tree or the junction tree (Pearl, 1988; Jensen, 1996) and perform inference on the junction tree. Here we present a method to convert a Bayesian network into a Markov net with pairwise potentials, such that the update rules described in the article reduce to Pearl’s algorithm on the original Bayesian network. In particular, if the Bayesian network has loops in it, so will the corresponding Markov network constructed using this method. This should be contrasted with the junction tree construction in Jensen (1996) which constructs a singly connected Markov network for any Bayesian network. For concreteness, consider the Bayesian network in Figure 8a. The joint probability is given by: P ( ABCDEF) = P( A) P(B | A) P(C | A) P( D | BC) P(F | C) P(E | D)

(A.1)

The procedure is simple. For any node that has multiple parents, we create a compound node into which the common parents are clustered. This compound node is then connected to the individual parent nodes as well as the child. For all nodes that have single parents, we drop the arrows and leave the topology unchanged. Figure 17 shows an example. Since D has two parents B and C, we create a compound node Z = ( B0 , C0 ) and connect it to D, B, and C. Having created a new graph, we need to set the pairwise potentials so that the probability distributions are identical. The potentials are the conditional probabilities of children given parents, with the exception of the potentials between the parent nodes and a compound node. These are set to one if the nodes have a consistent estimate of the parent node and zero otherwise. Thus in the example, if Z = (B0 , C0 ) then Y(BZ) = 1 if B0 = B and zero otherwise.

Correctness of Local Probability Propagation

39

Figure 17: Any Bayesian netowrk can be converted into a pairwise Markov network by adding cluster nodes for all parents that share a common child. (a) Bayesian network. (b) Corresponding pairwise Markov network. A cluster node for (B, C) has been added. When the potentials are set in the way detailed in the text, the update rules presented in this article reduce to Pearl’s propagation rules in the original Bayesian network.

To complete the example, we set Y(AB) = P( A) P( B | A), Y(AC) = P(C | A), Y(FC) = P( F | C), Y(ZD) = P( D | B0 C0 ) and then the joint probability of the Markov net, P( ABCDEFZ) = Y(AB)Y(AC)Y(FC)Y(BZ)Y(DE)Y(ZB)Y(ZC), (A.2) which contains only pairwise potentials but is identical to the probability distribution of the original Bayes net (see equation A.1). We now describe the equivalence between the messages passed in the Markov net and those that are passed in the Bayes net using Pearl’s algorithm. If X is a single parent of Y, then the message X sends to Y in the Bayes net is given by p x ( y). The message that X sends to Y in the Markov net is given by vEXY . The messages are related by multiplication by the link matrix: vEXY = aM XYp x ( y). Thus, in Pearl’s formulation, the matrix multiplication is performed at the child node, while in the algorithm described here, it is performed in the parent node. The message that Y sends to X in Pearl’s algorithm is denoted byl Y ( x) and is equal to vEYX in the Markov net algorithm up to normalization. In Pearl’s algorithm the p messages are normalized,

40

Yair Weiss

but the l ones are not, and in the algorithm presented here, all messages are normalized. If X is one of many parents of Y, then in the Markov net, X is connected to Y by means of a cluster node Z. As before, vEXZ = M XZp X ( y) but since the M XZ matrix contains only ones or zeros, the message in the Markov net is simply a replication of elements in the Bayesian net message. As before l Y ( x) is given by aE vZX . Every message in Pearl’s algorithm in the Bayes net can be identiŽed with an equivalent message in the Markov net using the algorithm presented here. Furthermore, by substituting the equivalent messages into the update rules presented here, one can derive Pearl’s update rules. Acknowledgments This work was supported by NEI R01 EY11005 to E. H. Adelson. I thank the anonymous reviewers, E. Adelson, M. Jordan, P. Dayan, M. Meila, Q. Morris, and J. Tenenbaum for helpful discussions and comments. After a preliminary version of this work appeared (Weiss, 1997), I had the pleasure of several discussions with D. Forney and F. Kschischang regarding related work they have been pursuing independently. I thank them for their comments and suggestions. References Aji, S., Horn, G., & McEliece, R. (1998). On the convergence of iterative decoding on graphs with a single cycle. In Proc. 1998 ISIT. Aji, S. M., & McEliece, R. (in press). The generalized distributive law. IEEE Trans. Inform. Theory. Available at: http://www.systems.caltech.edu/ EE/Faculty/rjm. Benedetto, S., Montorsi, G., Divsalar, D., & Pollara, F. (1996). Soft-output decoding algorithms in iterative decoding of turbo codes (Jet Propulsion Laboratory, Telecommunications and Data Acquisition Progress Rep. 42-124). Pasadena, CA: California Institute of Technology. Berrou, C., Glavieux, A., & Thitimajshima, P. (1993). Near Shannon limit errorcorrecting coding and decoding: Turbo codes. In Proc. IEEE International Communications Conference ’93. Bertsekas, D. P. (1987). Dynamic programming: Deterministic and stochastic models. Englewood Cliffs, NJ: Prentice Hall. Forney, G. (1997). On iterative decoding and the two-way algorithm. In Intl. Symp. Turbo Codes and Related Topics (Brest, France). Forney, G., Kschischang, F., & Marcus, B. (1998). Iterative decoding of tail-biting trellisses. Presented at 1998 Information Theory Workshop in San Diego. Frey, B. J. (1998). Graphical models for pattern classiŽcation, data compression and channel coding. Cambridge, MA: MIT Press. Frey, B. J., Koetter, R., & Vardy, A. (1998). Skewness and pseudocodewords in iterative decoding. In Proc. 1998 ISIT. Gallager, R. (1963). Low density parity check codes. Cambridge, MA: MIT Press.

Correctness of Local Probability Propagation

41

Gelb, A.(Ed.). (1974). Applied optimal estimation. Cambridge, MA: MIT Press. Geman, S., & Geman, D. (1984). Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. PAMI, 6(6), 721–741. Jensen, F. (1996). An introduction to Bayesian networks. Berlin: Springer-Verlag. Kschischang, F. R., & Frey, B. J. (1998). Iterative decoding of compound codes by probability propagation in graphical models. IEEE Journal on Selected Areas in Communication, 16(2), 219–230. Lauritzen, S. (1996). Graphical models. New York: Oxford University Press. MacKay, D., & Neal, R. M. (1995). Good error-correcting codes based on very sparse matrices. In Cryptographyand Coding: 5th IAM Conference. Lecture Notes in Computer Science No. 1025 (pp. 100–111). Berlin: Springer-Verlag. McEliece, R., MacKay, D., and Cheng, J. (1998). Turbo decoding as an instance of Pearl’s “belief propagation” algorithm. IEEE Journal on Selected Areas in Communication, 16(2), 140–152. McEliece, R., Rodemich, E., & Cheng, J. (1995). The Turbo decision algorithm. In Proc. 33rd Allerton Conference on Communications, Control and Computing (pp. 366–379). Monticello, IL. Minc, H. (1988). Nonnegative matrices. New York: Wiley. Pearl, J. (1988). Probabilistic reasoning in intelligent systems: Networks of plausible inference. San Mateo, CA: Morgan Kaufmann. Rabiner, L. (1989).A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE, 77(2), 257–286. Smyth, P. (1997). Belief networks, hidden Markov models, and Markov random Želds: A unifying view. Pattern Recognition, 18(11), 1261–1268. Smyth, P., Heckerman, D., & Jordan, M. I. (1997). Probabilistic independence networks for hidden Markov probability models. Neural Computation, 9(2), 227–269. Strang, G. (1986). Introduction to applied mathematics. Wellesley, MA: WellesleyCambridge. Weiss, Y. (1996). Interpreting images by propagating Bayesian beliefs. In M. Mozer, M. Jordan, & T. Petsche (Eds.), Advances in neural information processing systems, 9. Cambridge, MA: MIT Press. Weiss, Y. (1997). Belief propagation and revision in networks with loops (Tech. Rep. No. 1616). Cambridge, MA: MIT ArtiŽcial Intelligence Laboratory. Wiberg, N. (1996). Codes and decoding on general graphs. Unpublished doctoral dissertation, University of Linkoping, Sweden. Received November 11, 1997; accepted November 25, 1998.

Communicated by Carl van Vreeswijk

ARTICLE

Population Dynamics of Spiking Neurons: Fast Transients, Asynchronous States, and Locking Wulfram Gerstner Center for Neuromimetic Systems, Swiss Federal Institute of Technology, EPFL-DI, CH-1015 Lausanne, Switzerland

An integral equation describing the time evolution of the population activity in a homogeneous pool of spiking neurons of the integrate-and-Žre type is discussed. It is analytically shown that transients from a state of incoherent Žring can be immediate. The stability of incoherent Žring is analyzed in terms of the noise level and transmission delay, and a bifurcation diagram is derived. The response of a population of noisy integrateand-Žre neurons to an input current of small amplitude is calculated and characterized by a linear Žlter L . The stability of perfectly synchronized “locked” solutions is analyzed. 1 Introduction In many areas of the brain, neurons are organized in populations of units with similar properties. The most prominent examples are probably columns in the somatosensory and visual cortex (Hubel & Wiesel, 1962; Mountcastle, 1957) and pools of motor neurons (Kandel & Schwartz, 1991). In such a situation, we may want to describe the mean activity of the neuronal population rather than the spiking of individual neurons. In a population of N neurons, we can formally determine the proportion of active neurons by counting the number of spikes nact ( tI t CD t) in a small time interval D t and dividing by N. Division by D t yields the activity A( t) = limD t!0

1 nact ( tI t C D t) . Dt N

(1.1)

Although equation 1.1 has units of a rate, the concept of a population average is quite distinct from the deŽnition of a Žring rate via a temporal average. Temporal averaging over many spikes of a single neuron is a concept that works well if the input is constant or changes on a time scale that is slow with respect to the size of the temporal averaging window. Sensory input in a real-world scenario, however, is never constant. Moreover, reaction times are often short, which indicates that neurons do not have the time for temporal averaging (Thorpe, Fize, & Marlot, 1996). The population activity, on the other hand, may react quickly to changes in the input Neural Computation 12, 43–89 (2000)

c 2000 Massachusetts Institute of Technology °

44

Wulfram Gerstner ext

I (t)

A(t) t

Figure 1: Population of neurons (schematic). All neurons receive the same input I ext (t) (left), which results in a time-dependent population activity A(t) (right).

(Knight, 1972a; Treves, 1992; Tsodyks & Sejnowski, 1995; van Vreeswijk & Sompolinsky, 1996). In this article we study the properties of a large and homogeneous population of spiking neurons. All neurons are identical, receive the same input I ext ( t), and are mutually coupled by synapses of uniform strength (see Figure 1). Can we write down a dynamic equation in continuous time that describes the evolution of the population activity? This problem comes up in various models and has been studied by several researchers (Knight, 1972a; Wilson & Cowan, 1972, 1973; Amari, 1974; Gerstner & van Hemmen, 1992, 1993, 1994; Bauer & Pawelzik, 1993; Gerstner, 1995; Pinto, Brumberg, Simons, & Ermentrout, 1996; Senn et al., 1996; Eggert & van Hemmen, 1997; Pham, Pakdamen, Champagnat, & Vibert, 1998). A frequently adopted solution is a differential equation (Wilson & Cowan, 1972), Z 1 dA (t) t = ¡A(t) C g (1.2) 2 (s) A( t ¡ s) C 2 Q (s) I ext ( t ¡ s) ds dt 0

¡

¢

where neurons are coupled to each other with a delay kernel 2 and receive external input (or input from other populations) Žltered by a kernel 2 Q ; g(.) is the transfer function of the population and t is a time constant. Equation 1.2 is equivalent to the equations of graded-response neurons (Cowan, 1968; Cohen & Grossberg, 1983; HopŽeld, 1984). Equation 1.2 introduces a time constant t that is basically ad hoc. Wilson and Cowan (1972, 1973) have derived (1.2) from an integral equation, " # Z abs Z A ( t) =

1¡

d

0

A( t ¡ s) ds

g

¡

1

0

2 ( s) A( t ¡ s)

¢

C 2 Q ( s) I ext (t ¡ s) ds ,

(1.3)

which is valid for stochastic neurons with an absolute refractory period of length d abs . The term in the square brackets accounts for the fact that neurons that have Žred between t ¡ d abs and t are in the refractory state and

Population Dynamics of Spiking Neurons

45

cannot Žre. g(.) is the neuronal transfer function. Wilson and Cowan subsequently used a method of time-coarse graining to transform equation 1.3 into the differential equation 1.2. The time constant t , which appears on the left-hand side of equation 1.2, is related to the window of time-coarse graining. It has been argued that t in equation 1.2 is the membrane time constant of the neurons, an assertion that may be criticized since it holds for slowly changing activities only (Wilson & Cowan, 1972, 1973; Abbott & van Vreeswijk, 1993; Gerstner, 1995). The activity is, however, not always slowly changing. It has been shown previously that the population activity can react quasi instantaneously to abrupt changes in the input (Treves, 1992; Tsodyks & Sejnowski, 1995; van Vreeswijk & Sompolinsky, 1996). Transients in networks of nonleaky integrate-and-Žre neurons can be very short (HopŽeld & Herz, 1995). A population of leaky integrate-and-Žre units can follow a periodic drive up to very high frequencies (Knight, 1972a, 1972b). Moreover, homogeneous networks may be in an oscillatory state where all neurons Žre at exactly the same time (Mirollo & Strogatz, 1990; Gerstner & van Hemmen, 1993; Gerstner, van Hemmen, & Cowan, 1996; Somers & Kopell, 1993; van Vreeswijk, Abbott, & Ermentrout, 1994; Hansel, Mato, & Meunier, 1995; Terman & Wang, 1995). In this case, the population activity changes rapidly between zero and a very high activity. Rapid transients and perfectly synchronized oscillations are inconsistent with a differential equation of the form 1.2. In this article we analyze the population dynamics and derive a dynamic equation that is exact in the limit of a large number of neurons. The relevant equation is a generalization of the integral equations of Wilson and Cowan (1972) and Knight (1972a) and has been discussed previously in Gerstner and van Hemmen (1994) and Gerstner (1995). The dynamic properties of the population activity can be analyzed directly on the level of the integral equation; there is no need to transform it into a differential equation. The population equation allows us to discuss the following four questions from a uniŽed point of view: How does a population of neurons react to a fast change in the input? We show that during the initial phase of the transient, the population activity reacts instantaneously (Treves, 1992; Tsodyks & Sejnowski, 1995; van Vreeswijk & Sompolinsky, 1996). SpeciŽcally, the initial phase of the transient reects directly the form 2 Q of the postsynaptic potential. The end of the transient, on the other hand, can be rather slow and depends on the noise level and the coupling. In the special case of noiseless uncoupled integrate-and-Žre neurons, a new periodic state is reached as soon as every neuron has Žred once. What are the conditions for exact synchrony in the Žring of all neurons? We show that neurons are “locked” together if Žring occurs while the input potential is rising. The locking theorem in Gerstner et al. (1996) follows naturally from the integral equation.

46

Wulfram Gerstner

What are the conditions to make the neurons Žre in an optimally asynchronous manner? We discuss why, without noise, asynchronous Žring is almost always unstable (Abbott & van Vreeswijk, 1993; Gerstner & van Hemmen, 1993; Gerstner, 1995). We show that with a proper choice of the delays and a sufŽcient amount of noise, the state of asynchronous Žring can be stabilized (Gerstner, 1995). A bifurcation diagram is determined in terms of the noise level and the transmission delay. What is the response of an asynchronously Žring population to an arbitrary timedependent input current? We show that at least in the low-noise regime, the population activity responds faithfully even to high-frequency components of the input current (Knight, 1972a, 1972b). The cutoff frequency of the system is therefore given by the time constant of the synaptic current rather than by the membrane time constant. For some noise models, the cut-off frequency in the high-noise regime is further reduced. Theories of population activity have a fairly long history (Wilson and Cowan, 1972, 1973; Knight, 1972a; Amari, 1974; Feldman & Cowan, 1975). Some researchers have formulated population equations as maps in discrete time for homogeneous (Gerstner and van Hemmen, 1992; Bauer & Pawelzik, 1993) or inhomogeneous populations (Senn et al., 1996; Pham et al., 1998). Others have formulated continuity equations for the probability distribution of the membrane potential with the threshold as an absorbing boundary (Abbott & van Vreeswijk, 1993; Treves, 1993; Brunel & Hakim, 1998; Nykamp, Tranchina, Shapley, & McLoughlin, 1998). The population equations discussed in this article are related to the integral equations of Wilson and Cowan (1972) and Knight (1972a). Section 3 will review the population equations in continuous time developed in detail in Gerstner and van Hemmen (1992, 1994) and Gerstner (1995). Sections 4 through 7 will apply the population equations to the questions of locking, fast transients, signal transmission, and asynchronous Žring. 2 Model 2.1 Deterministic Threshold Model. Model neurons are described by the spike response model, a variant of the integrate-and-Žre model (Gerstner, 1995; Gerstner et al., 1996). A neuron i Žres if its membrane potential ui hits the threshold #. The membrane potential is of the form ui ( t) = g(t ¡ tOi ) C hPSP ( t| tOi ),

(2.1)

where tOi is the most recent Žring time of neuron i, and XX ( f) hPSP (t| tOi ) = wij2 ( t ¡ tOi , t ¡ tj ) j2Ci t( f ) j

Z C Jext

1 0

2 Q ( t ¡ tOi , s) I ext ( t ¡ s) ds

(2.2)

Population Dynamics of Spiking Neurons

47 ( f)

is the postsynaptic potential caused by Žrings tj of presynaptic neurons j 2 C i or by external input I ext (t). The kernel g(.) in equation 2.1 is a (negative) contribution due to refractoriness. Alternatively, we could incorporate refractoriness into the threshold and deŽne a dynamic threshold #¡g ( t¡ tOi ). Since we are interested in a homogeneous population of neurons, the kernels g, 2 , and 2 Q have no indices i, j, and all neurons receive the same input I ext ( t). Moreover, the interaction strength between the neurons is uniform, wij =

J0 , N

(2.3)

where J0 is a parameter. The interaction strength scales with one over the number N of neurons so that the total input remains Žnite in the limit of N ! 1. The theoretical approach developed in this article is valid for arbitrary response kernels 2 and 2 Q and for a broad variety of refractory kernels g. Occasionally, however, we may want to specify the response kernels. For example, with an appropriate choice of the kernels 2 , 2 Q , and g, equation 2.1 gives an excellent approximation to the Hodgkin-Huxley model with timedependent input (Kistler, Gerstner, & van Hemmen, 1997). Kernels can also be adjusted to experimental data (Stevens & Zador, 1998). With a different choice of kernels, equation 2.1 can be mapped exactly to various versions of the integrate-and-Žre model (Gerstner, 1995). Although most of the theory developed in this article is general, we will use two speciŽc models for an illustration of the results. 2.1.1 Simple Spike Response Model (SRM0 ). In the Žrst model, we assume O s) = that the kernels 2 and 2 Q do not depend on their Žrst argument: 2 ( t ¡ t, O s) = 2 Q0 ( s). This is the simplest instance of a spike response 2 0 ( s) and 2 Q (t ¡ t, model (Gerstner & van Hemmen, 1992) and will be called SRM0 . In this case, the postsynaptic potential is independent of the last Žring time of the neuron. We set hPSP (t| tO) = h( t) with Z 1 XX ( f) (2.4) h( t) = wij2 0 ( t ¡ tj ) C Jext 2 Q0 ( s) I ext ( t ¡ s) ds. 0

j2C i t( f ) j

We will refer to h( t) as the input potential. In simulations we need to speciŽy the kernels. We take 2 0 (s) =

¡

s ¡ D ax s ¡ D ax exp ¡ t2 t

¢

H( s ¡ D ax ),

(2.5)

where D ax is the R axonal delay and t is a membrane time constant. The normalization is 2 0 ( s) ds = 1. As always, H (s) is the Heaviside step function

48

Wulfram Gerstner

with H (s) = 0 for s · 0 and H( s) = 1 for s > 0. For external input we use in simulations ³ s´ 1 exp ¡ H( s) (2.6) 2 Q0 ( s) = t t with the same time constant t as in equation 2.5. After each spike the membrane potential is reset by adding a refractory kernel, g(s) = ¡gQ 0 exp (¡s/ t ) H (s).

(2.7)

In simulations of SRM0 neurons we set gQ0 = 1. The spike response model with the speciŽc kernels deŽned in equations 2.5 through 2.7, can be considered as an approximation to the integrate-and-Žre model. The approximation is based on a truncation and is valid if the typical interspike interval T0 is much larger than the time constant t (Gerstner et al., 1996); the error of the approximation is of the order exp(¡T0 / t ). For the numerical analysis later on, we have adjusted the threshold # so that the mean interspike interval T0 is exactly T0 = 2t . In this case, we may expect that SRM0 gives a reasonable, albeit not perfect, approximation to the integrate-and-Žre model. A major advantage of SRM0 neurons (compared to integrate-and-Žre neurons) is that most of the results of sections 3 through 7 can be formulated in a mathematically transparent form, which allows an easy interpretation. 2.1.2 Integrate-and-Fire model (IF).

With a different choice of the kernels

2 and 2 Q in equation 2.2, it is possible to construct a perfect mapping between

equation 2.1 and the standard integrate-and-Žre model (Gerstner, 1995). Between two Žrings, the membrane potential of an integrate-and-Žre (IF) unit i changes according to tm

XX dui ( f) = ¡ui C wij a(t ¡ tj ) C J ext I ext ( t), dt (f) j

(2.8)

tj

where tm is the membrane time constant, wij is the coupling strength, and a(.) is the time course of the synaptic current. If ui hits a threshold #, the membrane potential is reset to ureset < #. The spike response equation, 2.1, can be considered an integrated version of equation 2.8. The value ureset comes in as an initial condition for the integration of equation 2.8 and leads to a kernelg as in equation 2.7 with t = tm and gQ 0 = ¡ureset . For ureset = 0, the g-kernel vanishes. Integration of the synaptic current terms in equation 2.8 yields Z 2 ( x, s) =

s

s¡x

a(s0 ) e¡(s¡s )/ tm ds 0 0

= 2 0 ( s) ¡ 2 0 ( s ¡ x) e¡x/ tm ,

(2.9)

Population Dynamics of Spiking Neurons

49

Rs where 2 0 ( s) = 0 ds0 a(s0 ) exp[¡(s ¡ s0 )/ tm ]. The response kernel deŽned in equation 2.5 corresponds to a synaptic current a(s) = ts¡1 exp(¡s/ ts ) H( s) with time constant ts = tm = t and an additional axonal transmission delay D ax . Similarly, 2 Q (x, s) = 2 Q0 ( s) ¡ 2 Q0 (s ¡ x) e¡x/ tm with 2 Q0 given by equation 2.6 and tm = t . Using the expressions for 2 Q and 2 with tm = t , the postsynaptic potential for IF units can be expressed as O

)/ hPSP ( t| tO) = h( t) ¡ h(tO) e¡(t¡t t ,

(2.10)

where the input potential h(t) is given by equation 2.4. We emphasize that the speciŽc choice of the kernel 2 0 in equation 2.5 corresponds to a synaptic current with fast rise time. The theoretical framework of this article, however, is more general and applies equally well to slowly rising synaptic currents. 2.2 Noise. The neuron model discussed so far is purely deterministic. Given a spike at tO and for a known postsynaptic potential hPSP (t| tO), we can calculate the interval until the next spike from the threshold condition, O T (tO) = minf(t ¡ tO)| u( t) = #I t > tg.

(2.11)

In words, the interval T is given by the Žrst threshold crossing after the O spike at t. In the presence of noise, the exact Žring time and, hence, the exact interval are unknown. We may, however, determine the probability density Ph ( t| tO) that Žring occurs around time t given the postsynaptic potential hPSP ( t0 | tO) O The distribution Ph ( t| tO) for tO < t0 · t and knowing the last Žring time t. will play an important role for the population equations in section 3. In the noiseless case, the interval distribution reduces to a Dirac d -function Ph ( t| tO) = d [t ¡ tO ¡ T (tO)],

(2.12)

where T ( tO) is the next interval given a spike at tO and determined implicitly by equation 2.11. To calculate Ph ( t| tO) in the noisy case, we have to decide on a noise model. There are at least three different ways to include noise in spiking neurons. A popular procedure is to add white noise on the right-hand side of the differential equation of the IF neuron. Below this is called noise model C. Calculation of Ph ( t| tO) requires the solution of a Žrst passage time problem, which is known to be hard (Tuckwell, 1988, 1989). It is therefore convenient to work with one of the following two noise models: threshold noise (noise model A) or reset noise (noise model B). 2.2.1 Noisy Threshold (Noise Model A). In this Žrst noise model, we assume that the neuron can Žre even though the formal threshold # has not

50

Wulfram Gerstner

been reached yet or may stay quiescent even though the membrane potential is above threshold. To do this consistently, we introduce an “escape rate” r , which depends on the distance between the momentary value of the membrane potential and the threshold, r ( t) = f [u(t) ¡ #].

(2.13)

In the mathematical literature, the quantity r would be called a stochastic intensity. We require f (x) ! 0 for x ! ¡1. Otherwise the choice of the function f is arbitrary. Plausible assumptions are an exponential dependence r = r 0 exp[b ( u ¡ #)] or a gaussian dependence r = r 0 exp[¡b ( u ¡ #)2 ]; or a step function r = r 0 H (u ¡ #); or a piecewise linear function r = r 0 (u ¡ #)H (u ¡ #). b and r 0 are parameters. Note that the escape rate r is implicitly time dependent, since the membrane potential u( t) = g(t ¡ tO) ChPSP ( t| tO) varies over time. In addition, we may also include an explicit time dependence, for example, to account for a reduced spiking probability immediO ately after the spike at t. Let us now calculate Ph ( t| tO), the probability density of having a spike at t O and in the presence of a postsynaptic given that the last spike occurred at t, O O ( ) potential hPSP t| t for t > t. At each moment of time, the value u(t) of the membrane potential determines the escape rate r (t) = f [u(t) ¡ #]. In order O t) without to emit the next spike at t, the neuron has to survive the interval ( t, Žring and then Žre at t. Given the escape rater (t), the probability of survival from tO to t without Žring is

¡

Sh ( t| tO) = exp ¡

Z

t tO

¢

r ( t0 ) dt0 .

(2.14)

The probability density of Žring at time t is r ( t); thus with equation 2.14 we have

¡

P h ( t| tO) = r ( t) exp ¡

Z tO

t

¢

r ( t0 ) dt0 ,

(2.15)

which is the desired result. More detailed derivations of equation 2.15 can be found in Gerstner and van Hemmen (1994) and in the appendix of Wilson and Cowan (1972) (see also Cox & Lewis, 1966) 2.2.2 Noisy Reset (Noise Model B). In this noise model, Žring is given by the exact threshold condition u(t) = #. Noise is included in the formulation of reset and refractoriness. A convenient way of doing this is by replacing gQ 0 in equation 2.7 by gQ 0 ¡! g0 ( r) = gQ 0 er/ t ,

(2.16)

Population Dynamics of Spiking Neurons

51

where r is a random variable with zero mean. In the language of the IF neuron, we can describe the effect of r as a stochastic component in the value of the reset potential. After each spike, a new value of r is drawn randomly from a gaussian distribution G s ( r) with variance s ¿ t . Let us O r) for the next interval of a neuron that has Žred at tO and was write T ( t, reset with a stochastic value r. Since r is drawn at random, the interval distribution is Z O O r)] G s (r). (2.17) Ph ( t| t) = drd[t ¡ tO ¡ T ( t, In order to keep the arguments simple, we assume in the following that the O 0) is signiŽcantly larger than the width s of the typical interval T0 ( tO) = T ( t, distribution. In the case of the simple spike response model SRM0 , a reset according to equation 2.16 with r 6 = 0 shifts the refractory kernel horizontally along the time axis. To see this, let us consider a neuron that has Žred its last spike at tO and has been reset with variable r. Its refractory kernel is g(t) = gQ 0 exp[¡(t ¡ tO ¡ r)/ t]. Thus the neuron acts like a noiseless neuron that has O r) = r C T0 ( tO C r), Žred its last spike at t0 = tO C r. We therefore have T ( t, where T0 (t0 ) is the interval of a noiseless neuron that has Žred at t0 . For a constant input potential h( t) = h0 , the interval distribution of a SRM0 neuron is therefore a gaussian distribution centered at the noise-free interval T0 , Ph0 ( t| tO) = G sO ( t¡ tO¡T0 ) with variance sO = s. Thus, the gaussian distribution G s ( r) of the noise variable r maps directly to a gaussian distribution of the intervals around the mean T 0 . In the case of the IF neuron, the argument is slightly more involved. For O r) = T0 ( tO) C r T1 (tO) C O ( r2 ) s ¿ T0 (tO) we may expand the interval T ( t, in equation 2.17 to linear order in r. From the threshold condition, equation 2.11, we Žnd T1 ( tO) = g 0 / u0 where g 0 denotes the derivative of g(s) at s = T0 ( tO) and u0 = du/ dt is to be evaluated at t = tO C T0 ( tO). The integration on the right-hand side of equation 2.17 can now be performed and yields Ph ( t| tO) = G

sO [t

¡ tO ¡ T0 (tO)]

(2.18)

with variance sO = s T1 ( tO). Hence the interval distribution is a gaussian around the noise-free interval. For a constant input potential h0 , we Žnd from equation 2.10 T1 = gQ 0 / (gQ0 Ch0 ), which may be used for the calculation of s. O 2.2.3 Noisy Integration (Noise Model C). A noise term is added on the right-hand side of equation 2.8 (Tuckwell, 1988, 1989), tm

³ ´ XX dui ( f) = ¡ui C C Jext I ext (t) Cj wij a t ¡ tj dt ( ) f j tj

(2.19)

52

Wulfram Gerstner

where j represents gaussian white noise with zero mean and correlation hj ( t) j ( t0 )i = D tm d (t¡t0 ). The noise term can be motivated either by random input from background neurons that are not modeled explicitly (Stein, 1967) or by random openings of membrane channels. Noise causes the actual trajectory to drift away from the noise-free reference trajectory. In the absence of a threshold, the actual trajectories would have a gaussian distribution around the reference trajectory with variance su2 = ( D/ 2) [1 ¡ exp(¡2t/ tm )] (Tuckwell, 1989). The threshold acts as an absorbing boundary, and Ph ( t| tO) is the distribution of Žrst passage times. In general, the Žrst passage time problem is hard to solve. Even for constant driving current, the interval distribution is not symmetric around its mean, and higher moments are important. If, however, the level of noise is low and the reference trajectory hits the threshold with Žnite slope u0 , then a gaussian approximation of the distribution is reasonable (Tuckwell, 1989). In this case, the interval distribution is the same as for noise model B with s = su / u0 . On the other hand, if the reference trajectory stays signiŽcantly below threshold, then noise model C may be approximated by a noisy threshold (noise model A) with escape rate / r 0 exp[¡b (u ¡ #)2 ] with parameters r 0 ¼ 1/ tm and b = D¡1 . (Plesser & Gerstner, 1999). To avoid the complications of noise model C, we work in the following mainly with noise models A and B, except for some statements concerning C. A disadvantage of noise model B is that it generates spikes only for suprathreshold stimuli and cannot describe noise-activated spiking with subthreshold input. On the other hand, an indirect justiŽcation of noise model B could come from the theory of excitable membranes (Gutkin & Ermentrout, 1998). 3 Population Activity Equation We consider a homogeneous and fully connected network of spiking neurons in the limit of N ! 1. We aim for a dynamic equation that describes the evolution of the population activity over time. The total number of spikes nact emitted by all neurons in the population is

nact ( tI t C D t) =

N XZ X i= 1 t( f ) i

tCD t t

( f)

d (t0 ¡ ti ) dt 0 .

(3.1)

The population activity has already been deŽned in equation 1.1. Using equation 3.1 in equation 1.1, we may write

A(t) dt =

N ³ ´ 1 X ( f) d t ¡ ti dt. N i= 1

(3.2)

Population Dynamics of Spiking Neurons

53

The deŽnition of A allows us to rewrite the postsynaptic potential hPSP in equation 2.2 in the form Z hPSP ( t| tO) = J0

1 0

O s) A( t ¡ s) ds 2 (t ¡ t,

Z

C J ext

0

1

O s) I 2 Q ( t ¡ t,

ext

( t ¡ s) ds.

(3.3)

Thus, given the activity A(t0 ) for t0 < t, we can determine the potential O What we need is another hPSP ( t) of a neuron that has Žred its last spike at t. equation that allows us to determine the present activity A(t) given the past. The equation for the activity dynamics will be derived from three observations: 1. The model neurons are supposed to show no adaptation. According to equation 2.1, the state of neuron i depends explicitly on only the most recent Žring time tOi (and, of course, on the input hPSP ), but not on the Žring times of earlier spikes of neuron i. This allows us to work in the framework of a (nonstationary) renewal theory (Cox, 1962; Gerstner, 1995). 2. In the limit of N to inŽnity, only the expectation value of a random variable matters. The exact form of a probability distribution is irrelevant, and we can work directly with the mean. Full connectivity and N ! 1 is the limit where mean-Želd theory becomes exact. 3. The total number of neurons in the population remains constant. We may exploit this fact to derive a conservation law.

3.1 Integral Equation for the Dynamics. Because of the Žrst observation above, the probability density for a spike at t depends only on tO and on O We have already used this knowledge in our notation hPSP ( t| tO) for t > t. Ph (t| tO) for the probability density of Žring. The subscript h stands for the dependence on hPSP ( t| tO). Integration of the probability density over time Rt ( O) gives the probability that a neuron that has Žred at tO Žres its tO Ph s| t ds next spike at some arbitrary time between tO and t. Consequently, we can deŽne a survival probability Z Sh ( t| tO) = 1 ¡

t tO

Ph (s| tO) ds,

(3.4)

that is, the probability that a neuron that is under the inuence of hPSP and has Žred its last spike at tO survives without Žring up to time t. We now turn to a homogeneous population of neurons in the limit of N ! 1. We consider the network state at time t. Because of the second ob-

54

Wulfram Gerstner

servation, the proportion of neurons that have Žred their last spike between t0 and t (and have not Žred since) is exactly Z

t t0

O Sh (t| tO) A( tO) dt.

(3.5)

All neurons have Žred at some point in the past;1 thus, Z

t ¡1

Sh ( t| tO) A( tO) dtO = 1.

(3.6)

Because of the third observation, the normalization equation 3.6 must hold at any arbitrary time t. Taking the derivative of equation 3.6 with respect to t yields the activity dynamics, Z A ( t) =

t ¡1

O Ph ( t| tO) A( tO) dt.

(3.7)

A different derivation of equation 3.7 is given in Gerstner and van Hemmen (1992, 1994) and Gerstner (1993, 1995). The population equation 3.7, with hPSP given by 3.3, is the starting point for the discussions in the following sections. Intuitively, equation 3.7 is easy to understand. The kernel Ph ( t| tO) is the probability density that the next spike of a neuron under the inuence of a potential hPSP (t| tO) occurs at time O The number of neurons that have Žred at t given that its last spike was at t. Ot is proportional to A( tO), and the integral runs over all the past. An important remark concerns the proper normalization of the activity. Since equation 3.7 is deŽned as the derivative of equation 3.6, the integration constant on the right-hand side of the equation is lost. The system of equations 3.7 and 3.3 is therefore invariant under a rescaling of the activity A ¡! c A and J0 ¡! c¡1 J0 with some constant c. To get the correct normalization, we have to go back to equation 3.6. In the following sections the population equations in the form of equations 3.7 or 3.6 are analyzed for various scenarios. Equation 3.7 with the kernel Ph ( t| tO) calculated for the potential hPSP given by equation 3.3 deŽnes the dynamics in a homogeneous network of spiking neurons with short-term memory. We remark that although equation 3.7 looks linear, it is in fact a highly nonlinear equation because the kernel Ph ( t| tO) depends nonlinearly on hPSP , and hPSP in turn depends again on the activity via equation 3.3. (See noise model A in section 2.2 for an explicit example of the kernel Ph ( t| tO).) 1

Neurons that have never Žred before are assigned a formal Žring time tO = ¡1

Population Dynamics of Spiking Neurons

55

3.2 Relation to the Wilson-Cowan Integral Equation. The WilsonCowan integral equation, 1.3, is a special case of the population dynamics, equation 3.7. We assume neurons of the SRM0 type, hPSP ( t| tO) = h( t), and consider noise model A. The interval distribution is equation 2.15 with instantaneous rate r (t) = f [u(t)]. We may identify f with the gain function g of the Wilson-Cowan model. As shown in appendix A, for a refractory kernel g with absolute refractoriness only, equation 3.7 reduces strictly to the Wilson-Cowan equation, 1.3. Moreover, the framework of the population equation 3.7 also works for relative refractoriness. In the appendix it is shown that equation 3.7 for SRM0 neurons with noise model A is formally equivalent to the population equations for neurons with relative refractoriness derived in the appendix of Wilson and Cowan (1972). There is, however, an important difference in the interpretation of the equations. Wilson and Cowan motivated their gain function g by a distribution of threshold values # in an inhomogeneous population. In this case, the population equation 1.3 is an approximation, since correlations are neglected. In general, it matters whether the neurons in the refractory state are those with high or low threshold (Wilson & Cowan, 1972). In our interpretation g is the instantaneous escape rate due to a noisy threshold in a homogeneous population. In this interpretation, equation 1.3 is the exact equation for neurons with absolute refractoriness. 4 Noise-Free Population Dynamics 4.1 General Results. We start with a discussion of equation 3.7 in the noise-free case. As we have seen in equation 2.12, the probability density Ph (t| tO) reduces in the limit of no noise to a Dirac d -function. We use equation 2.12 in 3.7 and Žnd Z A ( t) =

t ¡1

O d ( t ¡ tO ¡ T (tO)) A(tO) dt.

(4.1)

Here T ( tO) is the interval given implicitly by the threshold condition, equation 2.11. Note that T ( tO) is the interval starting at tO and looking forward toward the next spike. The integration over the d-function in equation 4.1 can be done. Since T O the result is in the argument of the d-function depends on t, µ A ( t) =

1C

@t h C @tOh g 0 ¡ @tOh

¶ C

A[t ¡ Tb (t)],

(4.2)

where Tb (t) is the backward interval given a spike at time t. The partial O are to derivatives in equation 4.2, @t h = dhPSP (t| tO)/ dt and @tOh = dhPSP ( t| tO)/ dt, be evaluated at tO = t ¡ Tb ( t). The derivative g 0 = dg/ ds is to be evaluated at

56

Wulfram Gerstner

A

J

A(t) u

u(t)

d t’ input potential

t

d t h(t)

Figure 2: A change in the input potential h with positive slope h0 > 0 (dashed line, bottom) shifts neuronal Žring times closer together (middle). As a result, the activity A(t) (solid line, top) is higher at t = Ot C T( Ot) than it was at time Ot (schematic diagram).

s = Tb ( t). Note that the expression in the square brackets can also be written as u0 / (g 0 C @tOh) with u0 = du/ dt| t evaluated at t. Since the threshold must be reached from below, a spike at time t is possible only if u0 > 0. For u0 < 0, the backward interval does not exist, and the activity vanishes. We have taken care of this fact by the notation [x] C = x for x ¸ 0 and zero otherwise. Equation 4.2 is the Žrst major result of our analysis. In order to facilitate the discussion, we deŽne a variable: C(t) =

@t h C @tOh . g 0 ¡ @tOh

(4.3)

Let us evaluate C for SRM0 and IF-neurons. For SRM0 neurons, all terms in equation 4.2 can be calculated explicitly. Since @tOh vanishes, equation 4.3 reduces to CSRM ( t) =

h 0 ( t) . g 0 [T b ( t)]

(4.4)

The backward interval Tb exists if h(t) > # and is found from the threshold condition, equation 2.11, viz. Tb ( t) = t lnfg0 / [h(t) ¡ #]g. With equation 4.4, the factor in square brackets in equation 4.2 becomes [1 Ch0 / g 0 ] C. Its meaning is illustrated in Figure 2. A neuron that has Žred at tO will Žre again at t = tO C T ( tO). Another neuron that has Žred slightly later at tO Cdt0 Žres its next spike at t Cd t. If the input potential is constant between t and t Cd t, then

Population Dynamics of Spiking Neurons

57

d t = d t0 . If, however, h increases between t and t C d t, as is the case in Figure 2, then the Žring time difference is reduced. The compression of Žring time differences is directly related to an increase in the activity A. To see this, we note that all neurons that Žre between tO and tO C dt0 must Žre again between t and t Cd t. This is due to the fact that the network is homogeneous and the mapping tO ! t = tO C T ( tO) is monotonous. If Žring time differences are compressed, the population activity increases. For a SRM0 neuron with a kernel g as in equation 2.7, g 0 (s) > 0 holds for all s > 0. An input with h0 > 0 implies, because of equation 4.2, an increase of the activity: h0 > 0 () A(t) > A( t ¡ Tb ). For IF neurons we have a related result. To evaluate C, we use equation 2.10 and Žnd CIF ( t) = t

h0 (t) eTb / t ¡ h0 ( t ¡ Tb ) . gQ 0 C h(t ¡ Tb ) C t h0 ( t ¡ Tb )

(4.5)

The backward interval Tb (t) is given implicitly by the threshold condition # = h( t) ¡ [gQ0 C h( t ¡ Tb )] exp(¡Tb / t ). It is shown in appendix B that the denominator on the right-hand side of equation 4.5 is always positive. Therefore, a compression of Žring times and hence an increase of the activity occurs whenever the numerator of the equation is positive: h0 (t) ¡ h0 ( t ¡ Tb ) exp(¡Tb / t) > 0 () A( t) > A(t ¡ Tb ). This is a rather general property of the noise-free IF dynamics that will be exploited in the following sections. For general spike response neurons deŽned by equation 2.1, @t h C@tOh > 0 is a necessary condition for an increase of the activity, but it is not sufŽcient: @t h C @tOh > 0 H) A( t) > A(t ¡ Tb ). Details are discussed in appendix B. 4.2 Application to Locking. We will show in this subsection that the locking theorem developed in Gerstner et al. (1996) follows directly from the noise-free population equation 4.2. We consider a population that is already close to perfect synchrony and Žres nearly regularly with period T. In order to keep the arguments transparent, let us assume that the population activity for times t < T/ 2 can be approximated by a sequence of square pulses, A ( t) =

0 X

1 H (t ¡ nT Cd n ) H (nT Cd n ¡ t), n 2d n= ¡m

(4.6)

where m is some small integer, say, m = 3, and H (.) denotes the Heaviside step function with H( s) = 1 for s > 0 and H (s) = 0 for s · 0. The d n are assumed to be small, d n ¿ T, where T, is the period. We want to check whether the “Ansatz” (see equation 4.6) is consistent with the noise-free population dynamics (see equation 4.2). More precisely, we ask, Is there some period T and some sequence d n for n = ¡m, ¡m C 1, . . . , 0, 1, 2, . . . so that the sequence of square pulses described by equation 4.6 continues for t > T/ 2? We will determine T and the sequence d n

58

Wulfram Gerstner

self-consistently. If we Žnd d n ! 0 for n ! 1, then the square pulses contract to d-functions, and we say that a synchronous T-periodic oscillation is a stable solution of the population dynamics. As a Žrst step, we determine the potential hPSP ( t| tO). Given hPSP , we can calculate the period T from the threshold condition and also the derivatives @t h and @tOh needed in equation 4.2. To get hPSP , we put equation 4.6 in 3.3. We assume d n ¿ T and integrate. To Žrst order in d n , we Žnd hPSP (t| tO) =

n max X n= 0

h i O t C n T ) C O (d n ) 2 , J0 2 (t ¡ t,

(4.7)

where ¡d 0 · tO · d 0 is the last Žring time of the neuron under consideration. O s) as a function The sum runs over all pulses back in the past. Since 2 ( t ¡ t, of s decays quickly for s À T, it is usually sufŽcient to keep only a Žnite number of terms (e.g., nmax = 1 or 2). As the next step we determine the period T. To do so, we consider a neuron in the center of the square pulse that has Žred its last spike at tO = 0. Since we consider noiseless neurons the relative order of Žring of the neurons cannot change. To make the Ansatz, equation 4.6, consistent, the next spike of this neuron must therefore occur at t = T, in the center of the next square pulse. We use tO = 0 in the threshold condition, equation 2.11, which yields ( ) n max X ( ) ( ) (4.8) T = min t|g t C J0 2 t, t C n T = # . n= 0

If a synchronized solution exists, equation 4.8 deŽnes the period. In the population equation, 4.2, we need the derivative of hPSP , @t h C @tOh = J0

n max X n= 0

d 2 (x, s)| x= T,s= n T . ds

(4.9)

According to equation 4.2, the new value of the activity at time t = T is the old value multiplied by the factor in the square brackets. A necessary condition for an increase of the activity from one cycle to the next is that the derivative deŽned by the right-hand-side of equation 4.9 is positive, which is the essence of the locking theorem (Gerstner et al., 1996). To illustrate the idea, let us study two speciŽc cases: SRM0 neurons and IF neurons. For SRM0 neurons we have 2 ( x, s) = 2 0 ( s), hence @tOh = 0 and P hPSP ( t| tO) = h( t) = J0 n 2 0 ( t Cn T ). With theg kernel deŽned by equation 2.7, 0( ) we have g T > 0 whatever T. Thus h 0 ( T ) = J0

nmax XC1 n= 1

2 00 ( n T ) > 0

()

A( T ) > A(0).

(4.10)

h {a.u.}

A {kHz}

Population Dynamics of Spiking Neurons

59

1.0 0.0 0.0

0

10

20 t {m s}

30

40

Figure 3: A sequence of activity pulses (top) contracts tod -pulses if Žring always occurs when the input potential h (dashed line, bottom) is rising. Numerical integration of the population equation, 3.7, for SRM0 neurons with inhibitory interaction J = ¡0.1 and kernel 2 0 (see equation 2.5) with delay D abs = 2 ms. There is no noise (s = 0). The activity was initialized with a square pulse A(t) = 1 kHz for ¡1 ms< t < 0 and integrated with a step size of 0.05 ms.

For IF neurons we could go through an analogous argument to show that equation 4.10 holds. Instead we may verify its validity directly from equation 4.5 with h0 ( t) = h0 (t ¡ Tb ) due to periodicity. Therefore the amplitude of the synchronous pulse grows only if h0 (T ) > 0. The growth of amplitude corresponds to a compression of the width of the pulse. It can be shown that the corner neurons, which have Žred at time §d 0 , Žre their next spike atd 1 = d 0 [1 C C]¡1 with C deŽned in equation 4.3. Thus the square pulse remains normalized, as it should be. By iteration of the argument for t = n T with n = 2, 3, 4, . . . we see that (for IF or SRM0 neurons) the sequence d n converges to zero, and the pulses approach Psquare 0( ) a Dirac d-pulse under the condition that h0 ( T ) = n 2 0 n T > 0. In words, the T-periodic synchronized solution with T given by equation 4.8 is stable if the input potential h at the moment of Žring is rising (Gerstner et al., 1996). In order for the sequence of square pulses to be an exact solution of the population equation, we must require that the factor C deŽned in equation 4.3 remains constant over the width of a pulse. The derivatives of equation 4.7, however, do depend on t. As a consequence, the form of the pulse changes over time, as is visible in Figure 3. The activity as a function of time was obtained by a numerical integration of the population equation with a square pulse as the initial condition for a network of SRM0 neurons coupled via equation 2.5 with weak inhibitory coupling J = ¡0.1 and delay D ax = 2 ms. For this set of parameters, h0 ( T ) > 0 and locking is possible. The framework of the population equation allows us also to extend the locking argument to noisy SRM0 neurons. At each cycle, the pulse of synchronous activity is compressed due to locking if h0 ( T ) > 0. At the same time it is smeared out because of noise. For SRM0 neurons with gaussian noise

A {kHz}

Wulfram Gerstner

A {kHz}

60

1 .0

1 .0

0 .0 950

975 t {m s }

1000

0 .0 950

975 t {m s }

1000

Figure 4: Synchronous activity in the presence of noise. Simulation of a population of 1000 neurons with inhibitory coupling (J = ¡1, D ax = 2 ms) and noise model B. (a) Low-noise level (s = 0.25). (b) For larger noise (s = 0.5), the periodic pulses become broader.

in the reset (noise model B), we may use equation 2.17 for the probability density in the population equation, 3.7, Z A ( t) =

t ¡1

Z dtO

1 ¡1

O r)] G drd[t ¡ tO ¡ T ( t,

s ( r)

A( tO),

(4.11)

O r) = r CT0 (tO Cr) where T0 ( t0 ) and search for periodic solutions. We insert T ( t, is the forward interval of a noiseless neuron that has Žred its last spike at t0 . The integration over tO can be done and yields µ ¶Z 1 h0 A ( t) = 1 C 0 dr G g ¡1

s ( r)

A[t ¡ Tb ( t) ¡ r],

(4.12)

where T b is the backward interval. The factor [1 C ( h0 /g 0 )] arises due to the integration over thed -function just as in the noiseless case; see equations 4.2 and 4.4. The integral over r leads to a broadening, the factor [1 C ( h0 / g 0 )] to a compression of the pulse. As shown in appendix C, a limit cycle of equation 4.11 consisting of a periodic sequence of gaussian pulses exists if the noise amplitude s is small and ( h0 /g 0 ) > 0. The width of the activity pulses in the limit cycle is d where ¡ ¢ ¡ ¢2 d = s [2 h0 /g 0 C h0 / g 0 ]¡1/ 2 . As in the noise-free situation in equation 4.10, h0 and g 0 have to be evaluated at T. A simulation of locking with noise is shown in Figure 4. The network of SRM0 neurons has inhibitory connections (J0 = ¡1) and is coupled via the response kernel 2 0 , equation 2.5, with a transmission delay of D ax = 2 ms. Doubling the noise level s leads to activity pulses with twice the width. 5 Transients In this section we study the response of the population activity to a rapid change in the input. To keep the arguments as simple as possible, we con-

Population Dynamics of Spiking Neurons

61

sider an input that has a constant value I0 for t < t0 and then changes abruptly to a new value I0 C D I . Thus, (

I

ext (

t) =

I0 I0 CD I

for for

t · t0

(5.1)

t > t0 .

For the sake of simplicity, we assume a population of independent IF or SRM0 neurons (J0 = 0). For R t · t0 , all neurons receive a constant input potential h0 = Jext I0 since 2 Q0 (s) ds = 1. For t > t0 , the input potential, equation 2.4, changes due to the additional current D I . Thus ( h( t) =

h0 h0 CJ

ext

DI

R t¡t0 0

for 2 Q0 ( s) ds

for

t · t0

t > t0 .

(5.2)

O we can calculate for Given the input potential h( t) and the last Žring time t, any given neuron its momentary membrane potential u( t)—but what is the time course of the population activity? In order to analyze the situation, let us suppose that for t < t0 , the network is in a state of incoherent Žring. In other words, all neurons Žre at the same mean Žring rate, but their spikes are not synchronized. Rather, the Žring times are maximally spread out over time. Such a state has been called the splay phase (Chow, 1998). In the limit of N ! 1, the population activity is then a constant A( t) = A0 . In section 7 we will study the conditions under which a state of incoherent Žring can be a stable state of the system. Here we just assume that we can set the parameters so that the network Žres asynchronously and with constant activity. 5.1 Transients in a Noise-Free Network. In the noiseless case, neurons that receive a constant input I0 Žre regularly with some period T0 . For t < t0 , the mean activity is simply A0 = 1/ T0 . The reason is that, for a constant activity, averaging over time and averaging over the population must be the same. We now apply the equation of the noise-free population dynamics: equation 4.2. To do so, we have to calculate the factor C( t) deŽned in equation 4.3. For IF neurons, C is given by equation 4.5 and for SRM neurons by equation 4.4. In both cases we need the derivative of equation 5.2: ( 0

h ( t) =

J

ext D I

0

for

2 Q0 ( t ¡ t0 )

for

t · t0

t > t0

.

(5.3)

For t < t0 , we have h0 = 0 and thus, from equation 4.2, A(t) = A(t ¡ T0 ), as it should be for a constant activity A0 . Let us now consider a neuron that has Žred exactly at t0 . Its next spike occurs at t0 C T, where T is given by

62

Wulfram Gerstner

the threshold condition ui (t0 C T ) = #. If the change D I at t0 is small, we expect that the interspike interval T changes only slightly. In order to keep the arguments transparent, we make a zero-order approximation and set exp(¡T/ t ) ¼ exp(¡T0 / t), hence g 0 ( T ) ¼ g 0 ( T0 ). The population equation, 4.2, then yields A(t) = [1 C a 2 Q0 ( t ¡ t0 )] A0

for

t0 < t < t0 C T,

(5.4)

with a constant a = Jext D I /g 0 (T0 ) for SRM0 neurons and a = Jext D I / [g 0 ( T0 ) C h0 exp(¡T0 / t )] for IF neurons. Thus, the time course of the initial transient reects the time course 2 Q0 of the postsynaptic potential caused by external input. So far, the results are general. We now specify the response kernel 2 Q0 . With equation 2.6, the response of the input potential to the step current is h(t) = h0 C Jext D I [1 ¡ exp[¡(t ¡ t0 )/ t ]] for t > t0 . Thus, the potential has the characteristics of a low-pass Žlter. The population activity, however, reacts instantaneously to the step current. We put equation 2.6 in 5.4 and Žnd A(t) = A0 CD A

¡

1 t¡t0 exp ¡ t t

¢

H( t¡t0 )

for t0 < t < t0 CT,

(5.5)

where D A = A0 a and H (.) is the Heaviside step function. Thus there is an immediate response at t = t0 . The simulation in Figure 5 clearly exhibits the rapid initial response of the population. It is also conŽrmed by a numerical integration of the noise-free population dynamics, equation 4.2. As an aside we note that a dynamic rate model of the form of equation 1.2 deŽned by a differential equation with time constant t would not be able to capture the initial phase of the transient. The discussion up to now has focused on the initial phase of the transient. For t > t0 CT every neuron has Žred once, and the activity on the right-hand side of equation 4.2 can no longer be considered constant. What happens? Let us treat the IF model Žrst. With the response kernel 2 Q0 deŽned in equation 2.6, we Žnd for t > t0 C T the relation h0 ( t ¡ T ) = h0 ( t) exp(T/ t ). Hence CIF deŽned in equation 4.5 vanishes for t > t0 CT. Therefore A(t) = A( t¡T ), where T is the new backward interval consistent with the constant stimulation current I0 C D I . Thus the population activity is in a new periodic state as soon as every neuron has Žred once. This is what we should expect since neurons are independent, and as soon as each has been reset once after t0 , they all feel the same constant input current. The picture is different for SRM0 neurons. Here the new periodic state is reached only asymptotically (see Figure 5). The sequence of peaks in the activity that occur at times tk ¼ t0 C k T0 is given by the iteration A( tkC1 ) = A( tk ) [1 C h0 ( tkC1 )/g 0 ( T0 )]. Since h0 ( t) / exp[¡(t ¡ t0 )/ t ], the new limit cycle is approached asymptotically with time constant t . For SRM0 neurons, the

Population Dynamics of Spiking Neurons

63 0 .4

A {kH z}

A {kH z}

0 .4 0 .2

0 .2

0 .2 0 .0

ext

0 .0

I , h {a.u.}

0 .0

ext

I , h {a.u.}

0 .0

0 .2

80

90

100 110 t {m s }

120

80

90

100 110 t {m s }

120

Figure 5: (Top) Response of the population activity to a step current for low noise (s = 0.01 ms). Solid line: Simulation of a population of 1000 neurons. Dashed line: Numerical integration of the population equation, 3.7 (a) IF neurons. (b) SRM0 neurons. (Bottom) Step current input Iext (solid line) and input potential h(t) (dashed line). Parameters: Time constant t = 4 ms with response kernel 2 Q0 deŽned in equation 2.6; input step of 0.05 units at t = 100 ms. The threshold # was chosen so that the mean interval before the step is 8 ms, corresponding to a mean activity of 0.125 kHz. There are no interactions, J = 0; integration time step, 0.05 ms. The result of the simulation has been smoothed with a running average over 0.2 ms.

sequence of activity peaks therefore follows roughly the same exponential time course as the input potential h. 5.2 Transients with Noise. So far we have considered noiseless neurons. In the presence of noise, subsequent pulses get smaller and broader, and the network approaches a new incoherent state (cf. Figure 6a). If, in

A {kHz}

0 .4

A {kHz}

0 .4 0 .2 0 .0

80

90

100 110 t {m s }

120

0 .2 0 .0

80

90

100 110 t {m s }

120

Figure 6: Transients for SRM0 neurons with noise model B. (a) Same as in Figure 5b, but at a noise level of s = 2 and J = 0. (b) Noisy network (s = 2) with inhibitory interactions deŽned by equation 2.5 with D ax = 0.5 ms and J = ¡2. In both cases the results of a simulation of 1000 SRM0 neurons (solid line) are compared with a numerical integration (dashed line) of equation 3.7. All other parameters are as in Figure 5.

64

Wulfram Gerstner

addition, there is also some interaction J0 6 = 0 between the neurons, then the value of the new stationary state is shifted. A particularly interesting state is inhibitory interaction J0 < 0. In this case the new value of the stationary activity for t > t0 is only slightly higher than that for t < t0 . For noise model B, the fast transient with a sharp Žrst peak is clearly visible and marks the moment of input switching (see Figure 6b). Fast switching has previously been seen in heterogeneous networks with balanced excitation and inhibition (Tsodyks & Sejnowski, 1995; van Vreeswijk & Sompolinsky, 1996) and also discussed in the context of associative memory: Treves, 1992; Treves, Rolls, & Simmen, 1997). Here it is demonstrated and analyzed for homogeneous networks. For a preliminary analysis of the noisy case, we take SRM0 neurons with noise model B and start from equation 4.12. To simplify the expression, we write A(t) = A0 C D A(t) and expand equation 4.12 to Žrst order in D A. The result is Z D A( t) =

1

¡1

G

s ( r)

D A(t ¡ T0 ¡ r) dr C

h 0 ( t) A0 . g 0 ( T0 )

(5.6)

The result for IF neurons with noise model B is analogous to equation 5.6 and is derived in appendix D. The Žrst term in equation 5.6 accounts for changes that are due to previous variations of the activity. During the initial phase of the transient (for t0 < t < t0 CT0 ¡s), it plays no role and can be neglected. Thus the initial phase is determined by the second term. As in the noiseless case, the initial transient is proportional to the derivative of h. After this initial phase, the convolution with the gaussian (the Žrst term on the right-hand side of equation 5.6) comes into play and leads to a rapid smoothing (see Figure 6). The initial transient, however, is sharp. We may wonder whether we can understand fast switching intuitively. Before the abrupt change, the input was stationary and the population in a state of incoherent Žring, deŽned as a state where neuronal Žring times are spread out maximally. Thus some of the neurons Žre, others are in the refractory period, and again others approach the threshold. There is always a group of neuron whose potential is just below threshold. An increase in the input causes those neurons to Žre immediately, and this accounts for the strong population response during the initial phase of the transient. The above consideration also holds for noise model C (noisy integration) in the limit of low noise and suprathreshold inputs. In order to understand why the derivative of h comes into play, consider a Žnite step in the input potential h1 ( t) = D h H (t ¡ t0 ). All neurons i that are hovering below threshold so that their potential ui ( t0 ) is between # ¡ D h and # will be put above threshold and Žre synchronously at t0 . Thus, a step in the potential causes a d-pulse in the activity D A( t) / d (t ¡ t0 ) / h0 ( t0 ). In Figure 7a we have used a current step (see equation 5.1), the same step current as in Figure 5. The

Population Dynamics of Spiking Neurons 0 .4

A {kHz}

A {kHz}

0 .4 0 .2 0 .0

80

90

80

90

80

90

100

110 120

0 .4

A {kH z}

A {kH z}

0 .2 0 .0

100 110 120

0 .4 0 .2 0 .0

65

80

90

100 110 120 t {m s}

0 .2 0 .0

100 110 120 t {m s }

Figure 7: Response of a network of 1000 neurons without interaction (J = 0) to step current input as in Figure 5, but for (a) noise model C (noisy integration) and (b) noise model A (noisy threshold). For low noise, the transition is sharp (top row), whereas for high noise rather smooth (bottom). In all cases, threshold and noise level were adjusted so that the mean interval before the current step is 8 ms so as to allow a comparison with Figure 5. (Top) Noise level chosen so that the variance of the interval distribution is 1 ms. (Bottom) Variance about 4 ms.

response at low noise (top) has roughly the form D A( t) / h0 (t) / 2 Q0 ( t ¡ t0 ), as expected. The rapid transient is slightly less pronounced than for noise model B, but nevertheless clearly visible; compare Figures 6 and 7. As the amplitude of the noise grows, the transient becomes less sharp. Thus there is a transition from a regime where the transient is proportional to h0 (see Figure 7a, top) to another regime where the transient is proportional to h (see Figure 7a, bottom). What are the reasons for the change of behavior? The above simple argument based on a potential step D h > 0 holds only p for a Žnite step size, which is at least of the order of the noise amplitude D. In noise model C, the threshold acts as an absorbing boundary. Therefore the density of neurons with potential ui vanishes for ui ! #. Thence, for D h ! 0, the proportion of neurons that are instantaneously put across threshold is p 0. In a stationary state, the boundary layer with lowpdensity is of the order D (Brunel & Hakim, 1999). A potential step D h > D puts a signiŽcant proportion of neurons above threshold and leads to a d -pulse in the activity. Thus the result that the response is proportional to the derivative of the potential is essentially valid in the low-noise regime. On the other hand, noise model C can also be used in the regime of large-noise and subthreshold input. In the high-noise regime, a step in the potential raises the instantaneous rate of the neurons but does not force them to Žre immediately. The response to a current step is therefore smooth and

66

Wulfram Gerstner

follows the potential h( t) (see Figure 7a, bottom). A comparison of Figures 7a and 7b shows that noise model A exhibits a similar transition from sharp to smooth responses with increasing noise level. It can be shown that at least in the subthreshold regime, noise model C can be well approximated by noise model A (Plesser & Gerstner, 1999; see also Collins, Carson, Capela, & Imhoff, 1996). Analytical arguments for transients with noise model A are presented below. 5.3 Theory of Transients with Noise. To formalize the above ideas on transient responses in the presence of noise, we start from the population equation, 3.7, and write it in the form 0=

d dt

Z

t ¡1

O Sh (t| tO) A( tO) dt.

(5.7)

Equation 3.7 is the derivative of the conservation law, equation 3.6, and this is made explicit by our notation in equation 5.7. We consider a small perturbation of the stationary state A( t) = A0 CD A( t) and hPSP (t| tO) = h0 ( t| tO) C h1 (t| tO) and expand equation 5.8 to linear order in D A or h1 : Z d t 0= S0 (t ¡ tO) D A( tO) dtO dt ¡1 (Z ) Z t t ( t| tO) d @S h C A0 | h1 = 0 . (5.8) dt1 dtO h1 ( t1 | tO) dt @h1 (t1 | tO) ¡1 ¡1 We have used the notation S0 ( t ¡ tO) = Sh0 (t| tO) for the survivor function of the asynchronous Žring state. For the derivative in the Žrst term in equation 5.8, d we use dt S0 ( t ¡ tO) = ¡Ph0 ( t| tO) and S0 (0) = 1. To make some progress in the treatment of the second term in equation 5.8, it is helpful to take either the SRM0 or the IF model. For SRM0 neurons, we may drop the tO dependence of the potential and set h1 ( t1 | tO) = h1 ( t1 ), which allows us to pull the h1 ( t1 ) in front of the integral over tO and write equation 5.8 in the form Z D A( t) =

t

d Ph0 (t| tO) D A( tO) dtO C A0 dt ¡1

»Z

1 0

¼

L (x) h1 (t ¡ x) dx ,

(5.9)

with a Žlter L = L SRM deŽned in the top row of Table 1. For IF neurons we set h1 ( t| tO) = h1 ( t) ¡ h1 (tO) exp[¡(t ¡ tO)/ t ]. After some rearrangements of the terms, equation 5.8 becomes identical to 5.9 with a Žlter L = L IF (see Table 1). The Žrst term on the right-hand side of equation 5.9 is of the same form as the population equation 3.7 and describes how perturbations D A( tO) in

Population Dynamics of Spiking Neurons

67

Table 1: Filter L(x) for Integrate-and-Fire or SRM0 neurons (Upper index IF or SRM, respectively). Def A

L IF (x) =

(x) = L SRM A (x) = L IF A

B

R1

|0) dj @h@S(j x 1 (j ¡x) x L SRM (x) C 0 dj e¡j / t @S(x|0) @h 1 (j ) 1 0 dj f [u(j ¡ x)] S0 (j ) x x (x) ¡ S0 (x) L SRM dj e¡j / t f 0 [u(j )] A 0

L SRM (x) = ¡

R

R

R

(x) = d£(x)/ g 0 L SRM B ¤ (x) = d (x) ¡ GsO (x ¡ T0 ) e¡T0 / t / u0 L IF B

Notes: f 0 = df/ du is the derivative of the escape function f (u); S0 (s) = Sh0 (s| 0) is the survivor function in the incoherent state. For noise model B, the width of the gaussian is sO = s T1 = s gQ 0 / (gQ 0 C h0 ) and the derivatives are g0 = dg/ ds| s= T0 and u0 = g 0 C t ¡1 h0 exp(¡T0 / t ). In B, the approximation exp( s/ O t ) ¼ 1 has been used.

the past inuence the current activity D A(t). The second term gives an additional contribution, which is proportional to the derivative of a Žltered version of the potential. As we have seen, for SRM0 neurons with noise (x) = (1/g 0 ) d ( x). Thus for noise model B, model B, we Žnd L ( x) = L SRM B the Žlter is ad -function, and the second term in equation 5.9 is proportional to h0 . On the other hand, it is shown in appendix E that for noise model A, L can often be approximated by a low-pass Žlter,

L

SRM ( ) x A

= ar e¡r x H ( x),

(5.10)

where a is a constant and r is a measure of the noise. (The result for IF neurons is given in appendix E.) The noise-free threshold process can be ( x) = ad ( x), and retrieved from equation 5.10 for r ! 1. In this limit L SRM A the initial transient is proportional to h0 , as discussed above. For small r , however, the behavior is different. We use equation 5.10 and rewrite the last term in equation 5.9 in the form d dt

Z

1 0

L

SRM A ( x) h1 ( t

¡ x) dx = ar [h1 ( t) ¡ h1 (t)],

(5.11)

R1 where h1 (t) = 0 r exp(¡r x) is a running average. Thus the activity responds to the temporal contrast h1 ( t) ¡h1 (t). At high-noise levels,r is small. During the initial phase of the transient (t ¡ t0 < r ¡1 ), we may set h1 ( t) = 0. Thus, we Žnd for noise model A in the large-noise limit D A( t) / h( t). This is

68

Wulfram Gerstner

exactly the result that would be expected for a simple rate model. For a simulation of noise model A, see Figure 7b. The generalization of equation 5.11 to IF neurons is straightforward (cf. appendix E). In all cases, the initial phase of the transient is proportional to a linearly d fL ¤ Žltered version of the potential, D A / dt hg. In the limit of low noise, the choice of noise model is irrelevant; the transient response is proportional to the derivative of the potential, D A / h0 . If the level of noise is increased, noise model B retains its sharp transients, whereas A and C turn to a different regime where the transients follow h rather than h0 . The transition from one regime to the other is most easily studied for noise model A. The Žlter L is essentially a low-pass Žlter. The time constant of the Žlter increases with the noise level. The results have an interesting relation to experimental input-output measurements on motoneurons (Fetz and Gustafsson, 1983; Poliakov, Powers, & Binder, 1997). In the low-noise regime (where the type of noise model is irrelevant) the response to a synaptic input current pulse is proportional to the derivative of the postsynaptic potential (Fetz & Gustafsson, 1983), as predicted by earlier theories (Knox, 1974). On the other hand, we have seen that for noise models A and C, the behavior changes at higher noise levels and the response follows h rather than h0 . At intermediate noise levels, the response to a synpatic input is therefore proportional to some mixture between the postsynaptic potential and its derivative (Fetz & Gustafsson, 1983; Poliakov et al., 1997). For noise model B, the initial response to a current step is D A / 2 Q0 (t ¡ t0 ), independent of the noise level. Note that if the rise time of the postsynaptic potential 2 Q0 is slow (as it is the case for slow synaptic channels), the transient will be slow as well. In other words, the response time is limited by the time constant of the synpatic current rather than by the membrane time constant. In the following section, this statement will be made more precise. 6 Signal Transmission Our considerations regarding step current input can be generalized to arbitrary input current Iext ( t). We study a population of independent IF or SRM0 neurons. For each neuron, theRmembrane potential can be calculated 1 from the input potential h(t) = J ext 0 2 Q0 ( s) Iext ( t ¡ s) ds and the last Žring time tO (cf. equations 2.1 and 2.10). We assume that the population is close to a state of asynchronous Žring: A( t) = A0 C D A(t). The linear response of the population to the change in the potential is given by equation 5.9. The Fourier transform is Jext A0 L O (v) 2 OQ (v) O I (v). AO (v) = iv 1 ¡ PO(v)

(6.1)

1.0

0.0

0

200 400 600 800 f {Hz}

I ext {a.u.}

S(f)

A {kH z}

Population Dynamics of Spiking Neurons

69

0 .2 0 .1 0 .0

0 .0 180

190 t {m s}

200

O Figure 8: (a) Signal gain S( f ) = | A(2p f )/ IO (2p f )| as a function of the frequency f for IF neurons with time constant t = 4 ms and noise model B and a mean interspike interval of 8 ms. Resonances are at the multiples of the single-neuron frequency of 125 Hz. Noise level s = 4 ms (long-dashed line); s = 2 ms (solid line); s = 0.75 ms (short-dashed); Jext = 1. (b) Response of the population activity (top) of SRM0 neurons with noise model B to a time-dependent current (bottom). The current is a superposition of 4 sine waves at 9, 47, 111, and 1000 Hz. The simulation of a population of 4000 neurons (solid line, top) is compared with the numerical integration (dashed line) of the population equation, 3.7. Note that even the 1 kHz component of the signal is well transmitted. Parameters: Response function 2 Q0 (see equation 2.6) with time constant t = 4 ms. Threshold is # = ¡0.135 so that the mean activity is A = 125 Hz; noise s = 2 ms; J0 = 0.

Hats denote transformed quantities; 2 OQ (v) = [1 Civt ]¡1 is the Fourier transform of the kernel 2 Q0 deŽned in equation 2.6 PO (v) is the Fourier transform of the interval distribution, I (v) is the Fourier transform of the current Iext , and L O (v) is the transform of the linear Žlter L . To be speciŽc, we consider IF neurons with noise model B. For gaussian reset we Žnd PO(v) = expf¡ 12 sO 2 v2 ¡ ivT0 g. The Žlter L may be read off Table 1 and gives L O (v) = f1 ¡ exp[¡ 12 sO 2 v2 ¡ ivT0 ¡ T0 / t]g/ u0 where u0 is to be evaluated at T0 . In Figure 8a we have plotted the gain S = | AO (v)/ IO (v)| as a function of the frequency f = v/ (2p ). For a medium noise level of s = 2 ms, the signal gain has a single resonance at f = 1/ T0 = 125 Hz. For lower noise, further resonances at multiples of 125 Hz appear. For a variant of noise model B, a result closely related to equation 6.1 has been derived by Knight (1972a). The result for SRM0 neurons with noise model B is obtained by neglecting a term / exp(¡T0 / t). Independent of the noise level, we obtain for IF neurons for v ! 0 the result S(0) = Jext A0 [1¡exp(¡T0 / t )]/ ( u0 T0 ). Most interesting is the behavior in the high-frequency limit. For v ! 1 we Žnd S ! Jext A0 / ( u0 t ), hence i¡1 S(1) T0 h = 1 ¡ e¡T0 / t . t S(0)

(6.2)

70

Wulfram Gerstner

We emphasize that the high-frequency components of the current are not attenuated by the population activity despite the integration on the level of the individual neurons. The reason is that the threshold process acts as a differentiator and reverses the low-pass Žltering of the integration. In fact, equation 6.2 shows that high frequencies can be transmitted more effectively than low frequencies. The good transmission characteristics of spiking neurons at high frequencies have been studied in (Knight, 1972a) and conŽrmed experimentally (Knight, 1972b). In biology there is, of course, also a time constant of synaptic channels that leads to a frequency cutoff for the input current that may enter the cell. In this sense, it is the time constant of the synaptic current that determines the cutoff frequency of the population. The membrane time constant is of minor inuence. Treves (1993) has reached a similar conclusion. So far we have discussed results of the linearized theory (see equations 5.6 and 6.1). The behavior of the full nonlinear system is shown in Figure 8b. A population of unconnected SRM0 neurons is stimulated by a time-dependent input current that was generated as a superposition of four sinusoidal components with frequencies at 9, 47, 111, and 1000 Hz, which have been chosen arbitrarily. The activity equation, 3.7, has been integrated with time steps of 0.05 ms, and the results are compared with those of a simulation of a population of 4000 neurons. The 1 kHz component of the signal I (t) is clearly reected in the population A(t). Theory and simulation are in excellent agreement. We have seen that noise model B was rather exceptional in that the transient remained sharp even in the limit of high noise. To study the relevance of the noise model, we return to equation 6.1. The signal gain S(v) = | AO (v)/ IO(v)| is proportional to L O (v). If the Žlter L ( x) is broad, the Fourier transform will fall off to zero at high frequencies, and so does the signal gain S(v). In Figure 9 we have plotted the Žlter L IF A and the signal gain S for IF neurons with noise model A at different noise levels. At low noise, the result for noise model A is similar to that of noise model B (compare Figures 9b and 8a) except for a drop of the gain at high frequencies. Increasing the noise level, however, lowers the signal gain of the system. For high noise (the long-dashed line in Figure 9) the signal gain at 1000 Hz is 10 times lower than the gain at zero frequency. Note that for noise model A, the gain at zero frequency changes with the level of noise. 7 Incoherent Firing In the preceding sections it was shown that a population of neurons may react rapidly to changes in the input if the network is in a state of incoherent Žring. For a constant input current, incoherent Žring may be deŽned as a macroscopic Žring state with constant activity A( t) = A0 . In other words, incoherent Žring corresponds to a Žxed point of the population dynamics.

Population Dynamics of Spiking Neurons

S(f)

L A IF

25

0 0

5 x {m s}

71

1.0

0.0

10

0

2 0 0 4 00 6 0 0 8 0 0 f {H z}

(x) (a) and the signal gain (b) for IF neurons with noise Figure 9: The Žlter LIF A model A and escape rate r = r 0 [u ¡ #]H(u ¡ #). For low noise (short-dashed line, r 0 = 20 which corresponds to a variance of the interval distribution of s = 0.75 ms), the Žlter LIF A has a small width. For high noise (long-dashed line; r 0 = 2.5; variance s = 4 ms), the Žlter L IF A is broad. Solid line: r 0 = 5.5, which corresponds to an interval distribution with s = 2 ms. The value of the threshold has been adjusted so that the mean interval is always 8 ms. At low noise, the signal gain shows several resonances at the single-neuron frequency of 125 Hz. For high frequencies, the signal gain declines due to the low-pass characteristics of the Žlter LIF A . In (a) the slightly negative valley around T0 = 8 ms is typical for IF neurons and would be absent for SRM0 neurons.

In this section we study the existence and stability of incoherent Žring states. 7.1 Determination of the Activity. We search for a Žxed point A( t) = A0 of the population dynamics. Given constant A0 and constant external input I0 , the input potential of IF or SRM0 neurons is also constant, h( t) = h0 = J0 A0 C Jext I0 ,

(7.1)

R R where we have used the normalization 2 0 (s) ds = 1 = 2 Q0 ( s) ds. In a noisefree situation, neurons driven by h0 Žre regularly with a period T0 , which may be determined directly from the threshold condition. The population activity is then A0 = 1/ T0 . In the noisy case, the activity is A0 = 1/ hTi, R1 where hTi = 0 s Ph0 ( tO C s| tO) ds is the mean interval length (Gerstner, 1995). The statement follows from the normalization, equation 3.6. For constant input potential h0 , the survivor function and the interval distribution cannot depend explicitly on the absolute time, but only on the time difference O Hence we may set S0 ( s) = Sh0 (tO C s| tO) and P0 ( s) = Ph0 ( tO C s| tO). The t ¡ t. normalization reduces to Z 1 = A0

1 0

Z S0 ( s) ds = A0

0

1

s P0 ( s) ds = A0 hTi.

The second equality sign follows from integration by parts using ¡P0 (s).

(7.2) d ( ) ds S0 s

=

72

Wulfram Gerstner

For noise model B, P0 ( s) is a gaussian P0 (s) = G sO ( s ¡ T0 ) where sO is the width of the distribution and T0 is the noise-free interval in the presence of the input potential h0 . Thus, for this noise model, A0 = 1/ hTi = 1/ T0 as in the noise-free case. This is a nice property of noise model B. For noise models A and C, the mean interval, and hence A0 , changes with the level of noise. 7.2 Stability of Asynchronous Firing. In this subsection, the stability of incoherent Žring is analyzed. We assume that for t > 0, the activity is subject to a small perturbation around the Žxed point at A0 , A(t) = A0 C AO 1 eivtCl t ,

(7.3)

with AO 1 ¿ A0 . The perturbation in the activity induces a perturbation in the postsynaptic potential, h( t) = h0 C h1 eivtCl t ,

(7.4)

with h0 = J02 O (0) A0 and h1 = J02 O (v ¡ il ) AO 1 , where Z

¡iy 2 O (v ¡ il ) = |2 O (v ¡ il )| e

(v¡il )

=

0

1

¡ivs¡ls

2 0 (s) e

ds

(7.5)

is a Laplacian transform of 2 0 . The Fourier transform is deŽned by equation 7.5 with l = 0. The angle y (.) is the phase shift between h and A. The change in the potential causes some of the neurons to Žre earlier (when the change in h is positive) and others to Žre later (whenever the change is negative). The perturbation may therefore build up (l > 0, the incoherent state is unstable) or decay back to zero (l < 0, the incoherent state is stable). At the transition between the region of stability and instability, the amplitude of the perturbation remains constant (l = 0, marginal stability of the incoherent state). These transition points, deŽned by l = 0, are determined now. We use equations 7.3 and 7.4 on the right-hand side of the linearized population equation, 5.9. After cancellation of a common factor AO 1 exp(ivt), the result can be written in the form 1 ¡ PO (v) = iv J0 A0 2 O (v) L O (v).

(7.6)

PO(v) is the Fourier transform of the interval distribution Ph0 ( t| tO), and L O (v) is the transform of the Žlter L in Table 1. Equation 7.6 deŽnes the bifurcation points where the incoherent Žring state loses its stability toward an oscillation with frequency v. So far the result is completely general. We now specify the model. For SRM0 neurons with gaussian noise in the reset (noise model B), the interval

Population Dynamics of Spiking Neurons

73

S f(w T 0)

1 .5 1 .0 0 .5 0 .0

0

1

2 w T0

3

4

Figure 10: Amplitude condition for the bifurcation of an oscillatory solution. The feedback gain S f is plotted as a function of the normalized frequency v T0 for two different values of the noise, s = 1 ms (solid line) and s = 0.1 ms (dashed line). Given a feedback with appropriate phase, instabilities of the incoherent Žring state are possible for frequencies where S f > 1. For low noise, S f crosses unity (dotted horizontal line) at frequencies v ¼ vn = n 2p / T0 . For s = 1 ms there is a single instability region for v T0 ¼ 1. The feedback gain is deŽned by the right-hand side of equation 7.7. For the plot we have set T0 = 2t .

distribution is a gaussian centered at T0 and the Žlter is ad-function, L ( x) = d ( x)/ g 0 . Hence equation 7.6 is of the form 1=

iv J0 A0 2 O (v) , g 0 1 ¡ G Os (v) e¡ivT0

(7.7)

where G Os (v) = expf¡ 12 s 2 v2 g is the Fourier transform of a gaussian with width s. We note that equation 7.7 could have been obtained directly from the signal gain, equation 6.1, after replacement of the input term J ext2 OQ (v)IO (v) ¡! J02 O (v) AO 1 and evaluation of PO(v) and L O (v) for noise model B. In the following we analyze solutions of equation 7.7. To do so we split the equation into two equations: one for the absolute value and the other for the phase. Let us start with the equation for the absolute values. A necessary condition for a solution to equation 7.7 is a feedback gain S f of unity, where S f is deŽned by the absolute value of the right-hand side of the equation. In Figure 10 we have plotted S f as a function of vT0 . The left-hand side of equation 7.7 corresponds to a horizontal line at unity. Solutions to equation 7.7 may exist only for frequencies v ¼ vn = n 2p / T0 with integer n, where T0 = 1/ A0 is the typical interspike interval of the neurons. A0 is determined by equation 7.2, with h0 given by equation 7.1. Thus T0 = 1/ A0 includes the effect of the feedback. T0 is not the intrinsic frequency of the free neuron, which could be quite different. A solution with v ¼ v1 implies that the period of the population activity is identical to the period of individual neurons driven by an input potential h0 . Higher harmonics correspond to cluster states: each neuron Žres with

74

Wulfram Gerstner

a mean period of T0 , but neurons tend to Žre in groups so that the activity oscillates several times faster. We see from Figure 10 that the harmonics are relevant for low noise only. At high-noise level, even the solution v ¼ v1 disappears. For s ! 0 the absolute value of the denominator of equation 7.7 is 2|sin(vT0 / 2)|, and solutions may occur for all higher harmonics. Figure 10 speciŽes the amplitude condition for the solution of equation 7.7. We now turn to the phase equation. In order to work out the phase conditions, we need the Fourier transform of the response kernel 2 0 deŽned in equation 2.5. It is given by equation 7.5 with amplitude |2 O (v)| = (1 C v2 t 2 )¡1 and phase: y (v) = v D ax C 2 arctan(v t ).

(7.8)

We note that a change in the delay D ax affects only the phase of the Fourier transform, not the amplitude. By changing the transmission delay, we can therefore systematically shift the phase of the numerator on the right-hand side of equation 7.7, a convenient property that we will exploit below. Since we know that solutions are possible only for v ¼ vn , we set v = vn Cºn . The numerical solutions of equation 7.7 for different values of the delay D ax and different levels of the noise s are shown in the bifurcation diagram in the center of Figure 11. Neurons interact with strength J0 = 1. The value of the threshold # was adjusted so that the mean interval in the presence of coupling was T0 = 2t . (For this choice of parameters, the interval of the free neuron would be about twice as long.) In the simulations shown in the four insets in Figure 11, we have taken t = 4 ms and a total number of N = 1000 neurons. 7.3 Bifurcation Diagram. Let us consider a network with interaction delay D ax = 2 ms. This corresponds to an x-value of D ax / T0 = 0.25 in Figure 11. The phase diagram predicts that at a noise level of s = 0.5 ms, the network is in a state of asynchronous Žring. The activity during a simulation run is shown in the inset in the upper right-hand corner. It conŽrms that the activity uctuates around a constant value of A0 = 1/ T0 = 0.125 kHz. If the noise level of the network is signiŽcantly reduced, the system crosses the short-dashed line. The line is the boundary at which the constant activity state becomes unstable with respect to an oscillation with v ¼ 3 (2p / T0 ). A simulation result for a network at a noise level of s = 0.1 but otherwise the same parameters as before conŽrms that the population activity exhibits an oscillation with period T osc ¼ T0 / 3 ¼ 2.6 ms. We now return to our original parameters of s = 0.5 ms and D ax = 2 ms and reduce the axonal transmission delay while keeping the noise level Žxed. This correponds to a horizontal move across the phase diagram in Figure 11. At some point, the system crosses the solid line, which marks the transition to an instability with frequency v1 = 2p / T0 . Again this is conŽrmed by the simulation results shown in the inset in the upper left

Population Dynamics of Spiking Neurons

75 3

A(t) {kHz}

A(t) {kHz}

3 2 1 0 940

960

980

2 1 0 940

1000

960

t {ms}

980

1000

t {ms}

1.0

s {ms}

0.8 0.6 0.4 0.2 0.0 0.0

0.2

0.4 D /T 0

0.6

2 1 0 940

1.0

3

A(t) {kHz}

A(t) {kHz}

3

0.8

960

980

t {m s }

1000

2 1 0 940

960

980

1000

t {ms}

Figure 11: Stability diagram (center) as a function of noise s (y-axis) and delay D ax (x-axis). The diagram shows the borders of the stability region with respect to v1 , . . . , v4 . For high values of the noise, the asynchronous Žring state is always stable. If the noise is reduced, the asynchronous state becomes unstable with respect to an oscillation with frequency v1 (solid border lines,) or v2 (longdashed border lines), or v3 (short-dashed border lines), or v4 (long-short dashed border lines). Four insets show typical patterns of the activity as a function of time taken from a simulation with 1000 neurons. Parameters s = 0.5 ms and D ax = 0.2 ms (top left); s = 0.5 ms and D ax = 2.0 ms (top right); s = 0.1 ms and D ax = 0.2 ms (bottom left); s = 0.1 ms and D ax = 2.0 ms (bottom right); Since the pattern repeats along the x-axis with period T0 , we have plotted a normalized delay D ax / T0 .

corner. If we now decrease the noise level, the oscillation becomes more pronounced (see the bottom right of Figure 11). In the limit of low noise, the incoherent network state is unstable for virtually all values of the delay. The region of the phase diagram in Figure 11

76

Wulfram Gerstner

around D ax / T0 ¼ 0.1 looks stable but hides instabilities with respect to the higher harmonics v6 and v5 , which are not shown. We emphasize that the speciŽc location of the stability borders depends on the form of the postsynaptic response function 2 . The qualitative features of the phase diagram in Figure 11 are generic and hold for all kinds of response kernels. The numerical results apply to the response kernel 2 0 (s) deŽned in equation 2.5, which corresponds to a synaptic current a(s) with zero rise time (cf. equations 2.9 and 2.8). What happens if a is a double exponential with rise time trise and decay time tsyn ? In this case, the right-hand side of equation 7.7 has an additional factor [1 C ivtrise]¡1 that leads to two changes. First, due to the reduced amplitude of the feedback, instabilities with fre¡1 quencies v > trise are suppressed. The tongues for the higher harmonics are therefore smaller. Second, the phase of the feedback changes. Thus, all tongues of frequency vn are moved horizontally along the x-axis by an amount D / T 0 = ¡atan(vn trise)/ ( n 2p ). What happens if the excitatory interaction is replaced by inhibitory coupling? A change in the sign of the interaction correponds to a phase shift of p . For each harmonic, the region along the delay axis where the incoherent state is unstable for excitatory coupling (cf. Figure 11) becomes stable for inhibition, and vice versa. In other words, we simply have to shift the instability tongues for each frequency vn horizontally by an amount D / T0 = 1/ (2n). Otherwise the pattern remains the same. 8 Discussion 8.1 Assumptions. We have discussed an integral equation for the population dynamics. The validity of the population equations relies on three assumptions: (1) a homogeneous population of (2) an inŽnite number of neurons that show (3) no adaptation. It is clear that there are no large and completely homogeneous populations in biology. The population equations may nevertheless be a useful starting point for a theory of heterogeneous populations (Tsodyks, Mitkov, & Sompolinsky, 1993; Senn et al., 1996; Chow, 1998; Pham et al., 1998; Brunel & Hakim, 1998). We expect that most of the results carry over to nonhomogeneous networks. In a sense, noise model B can be considered an annealed version of a heterogeneous model where the reset value varies from one neuron to the next. The treatment of heterogeneity as noise (reset values are randomly chosen after each reset rather than only once at the beginning) neglects, however, correlations that would be present in a truly heterogeneous model. To replace a heterogenous model with a noisy version of a homogeneous model is somewhat ad hoc but common practice in the literature. Wilson and Cowan (1972), for example, motivated the sigmoidal threshold function in their model by a heterogeneous system where the threshold value varies from one neuron to the next. In or-

Population Dynamics of Spiking Neurons

77

der to analyze the system, they neglect correlations and replace the heterogeneous system by a homogeneous population with noisy threshold (noise model A). A more recent example for the replacement of a heterogeneous system by a noisy homogeneous system can be found in Pham et al., 1988. A generalization of equations 3.3 and 3.7 to a network subdivided into several pools is possible. Within each pool, neurons are homogeneous. The activity of a pool m is Am ( t). A neuron in pool m receives input from all neurons in pool n with strength wmn = Jmn / Nn . The postsynpatic potential of neurons in pool m is then given by a straightforward generalization of R1 P O equation 3.3 (Gerstner 1995) hm ( t| tO) = n J mn 0 2 ( t ¡ t, s) An ( t ¡ s) ds. The population equation 3.7 remains unchanged but has to be applied to each pool activity Am separately. The pools are coupled via the potential hm , which determines the kernel Phm (t| tO). A transition from discrete pools to a continuous population is possible (Gerstner, 1995) and might be useful for modeling of primary visual cortex. The second condition is the limit of a large network. For N ! 1 the population activity shows no uctuations, and this fact has been used for the derivation of the population equation. For systems of Žnite size, uctuations are important since they limit the amount of information that can be transmitted by the population activity. For populations without internal coupling (J0 = 0), uctuations can be calculated directly from the interval distribution Ph ( t| tO). For networks with internal coupling, an exact treatment of Žnite size effects is difŽcult. For noise model A, Žrst attempts toward a description of the uctuations have been made (Spiridon, Chow, & Gerstner, 1998). For noise model C, Žnite size effects in the low-connectivity limit are treated in Brunel and Hakim (1999). The limit of no adaptation seems to be valid for fast spiking neurons (Connors, 1990). Most cortical neurons, however, show adaptation. A generalization of the population equations to neuron models with adaptation does not seem straightforward. From the modeling point of view, all IF neurons are in the class of nonadaptive neurons, since the membrane potential is reset (and the past forgotten) after each output spike. Both leaky and nonleaky IF neurons can therefore be mapped to a response kernel description with no dependence on earlier output spikes (Gerstner, 1995). To a very good approximation, the Hodgkin-Huxley model can also be mapped to a response kernel description (Kistler et al., 1997). The condition of short memory (that is, no adaptation) leads to the class of renewal models (Perkel, Gerstein, & Moore, 1967; Stein, 1967; Cox, 1962), and this is where the integral equation applies; (cf. Gerstner, 1995). 8.2 Signal Transmission by Incoherent Firing. We have mainly focused on the state of incoherent Žring in the low-noise regime. This state may be particularly interesting for information transmission, since the system

78

Wulfram Gerstner

can respond rapidly to changes in the input current. For noise model B, the signal gain deŽned as the amplitude of the population activity divided by that of the input current shows no cutoff at high frequencies (Knight, 1972a). The effective cutoff frequency of the system is therefore given by the input current. Changes in the input current are, of course, limited by the opening and closing times of synaptic channels. Each presynpatic spike evokes a postsynaptic current pulse of Žnite width. The time course of the current pulse determines the response time of the system (Treves, 1993). These insights may have important implications for modeling as well as for interpretations of experiments. It is often thought, that the response time of neurons is directly related to the membrane time constant tm . This idea has been criticized repeatedly since in some cases neurons do respond much faster (Treves, 1992; Koch, Rapp, & Segev, 1996). In neural network modeling, a description of the population activity by a differential equation of the form 1.2 is common practice. Our results suggest that the time constant t of the left-hand side of equation 1.2 should be of the order of the duration of the synaptic current pulse. We have seen that the Wilson-Cowan integral equation 1.3 and hence the differential equation 1.2 describes neurons with absolute refractoriness and noise model A; the gain function g in equation 1.3 may be identiŽed with the escape rate f . For neurons with absolute refractoriness and noise model A, the Žlter L is a low-pass Žlter with cutoff frequency f (h0 ) (cf. appendix E). If the mean activity of the system is low, then the cutoff freqency is low, and the activity follows during a transient the input potential h. The transient is therefore slow. On the other hand, if the rate f ( h0 ) is high, then the activity follows the derivative h0, and the transient is fast. Thus time-coarse graining would be possible with a time constant smaller than f (h0 )¡1 . We must keep in mind that even if we take a short integration time constant, the differential equation 1.2 is always a description of noise model A. We hope that the treatment presented in this article helps to clarify some important points. The limitations of the differential equation 1.2 are already mentioned in Wilson and Cowan (1972) and have been discussed in Abbott and van Vreeswijk (1993), Treves (1993), and Gerstner (1995). For noise model A, the integral equation can be replaced by a system of differential equations (Eggert & van Hemmen, 1997) or by partial differential equations (Gerstner & van Hemmen, 1992). For noise model C, partial differential equations for the distribution of the membrane potential u have been developed (Abbott & van Vreeswijk, 1993; Brunel & Hakim, 1998) and are an alternative to the approach by integral equations. Since we consider incoherent Žring as a state with useful signal transmission properties, we must be concerned about the stability of such a state. Incoherent Žring can be stabilized by a suitable choice of time constants, transmission delay, and noise (Abbott & van Vreeswijk, 1993; Gerstner &

Population Dynamics of Spiking Neurons

79

van Hemmen, 1993, 1994; Gerstner, 1995). For low noise, incoherent states are unstable for nearly all choices of parameters. Instabilities lead toward oscillations, either with a period comparable to the typical interval of the neuronal spike trains, or much faster (higher harmonics) (Golomb, Hansel, Shraiman, & Sompolinsky, 1992; Gerstner & van Hemmen, 1993; Ernst, Pawelzik, & Geisel, 1994; Golomb & Rinzel, 1994). The harmonics have also been called cluster states since neurons spontaneously split into groups of neurons that Žre approximately together. Higher harmonics can be easily suppressed by noise. The instability of asynchronous Žring, has previously been been studied in phase models (Kuramoto, 1975, 1984; Winfree, 1980; Ermentrout, 1981; Strogatz & Mirollo, 1991; Golomb et al., 1992). Phase models can describe arbitrary nonlinear oscillator systems in the limit of weak coupling (Ermentrout, 1981). In contrast to phase models, the analytical framework presented in this article does not rely on a weak coupling assumption but works just as well for strong coupling. The mean interspike interval T0 in the asynchronous or synchronous state is determined self-consistently and includes the effects of the feedback from other neurons. In the absence of coupling, the model neurons may be silent or exhibit a Poisson-like rather than an oscillatory Žring behavior. Thus neurons are not necessarily oscillators. To analyze the stability in the Poisson regime, we simply have to choose the appropriate interval distribution Ph0 ( t| tO), determine its Fourier transform PO(v), and the Žlter L O (v), and put the expressions in the bifurcation equation, 7.6. The instability of asynchronous Žring in pulse-coupled neuron models has been studied in the noise-free case (Abbott & van Vreeswijk, 1993; Gerstner & van Hemmen, 1993; Gerstner, 1995). The effect of small inhomogeneities has been studied by Tsodyks et al. (1993) and a network with several populations and types on ion currents by Treves (1993). Noise in the differential equation of delayless IF neurons was treated by Kuramoto (1991) and Abbott and van Vreeswijk (1993). An analysis similar to that of Abbott and van Vreeswijk (1993), but for heterogeneous networks with low connectivity, has been done by Brunel and Hakim (1999). The effect of threshold noise (noise model A) in the spike response model including delays has been investigated by Gerstner (1995). For the analysis in section 7, we have assumed noise in the reset (noise model B). This noise model is probably not the most realistic one, but it is particularly convenient to work with since the mean interspike interval is independent of the noise level. For other noise models, a bifurcation diagram as in Figure 11 will look more complicated since the mean interval depends on the noise level. A way out would be to readjust the threshold for each noise level so as to keep the mean interval constant. If the network is Žring incoherently, then the population activity responds immediately to an abrupt change in the input (Treves, 1992; Tsodyks & Se-

80

Wulfram Gerstner

jnowski, 1995; van Vreeswijk & Sompolinsky, 1996; Horn & Levanda, 1998). There is no integration delay. In the case of incoherent Žring, there are always some neurons close to threshold, and this is why the system as a whole can respond immediately. This property suggests that a population of neurons may transmit information fast and reliably. Fast information processing seems to be a necessary requirement for biological nervous systems if the reaction time experiments are to be accounted for (Thorpe et al., 1996). Appendix A: Wilson-Cowan Integral Equations We apply the population equation 3.7 to SRM0 neurons with noise model A. The escape rate is a sigmoidal function f [u] and # = 0. The neuron model is speciŽed by a refractory function g in three pieces. First, during an absolute refractory time 0 < s · d abs , we formally set g(s) to ¡1. Second, during the relative refractory period d abs < s < d rel , g may take some arbitrary values g(s). Third, g(s) = 0 for s ¸ d rel . Given g it seems natural to split the integral in the activity equation 3.7 into three pieces: Z A ( t) =

t¡d rel ¡1

Z

C

Z Ph ( t| tO) A(tO) dtO C

t

t¡d abs

t¡d abs t¡d rel

Ph ( t| tO) A(tO) dtO

O Ph ( t| tO) A(tO) dt.

(A.1)

The interval distribution Ph ( t| tO) for noise model A is given by equation 2.15, and is repeated here for convenience: » Z t ¼ 0 0 0 O O O (A.2) P h ( t| t) = f [h(t) Cg(t ¡ t)] exp ¡ f [h(t ) Cg(t ¡ t)] dt . tO

The last term in equation A.1 vanishes since spiking is impossible during the absolute refractory time, f [¡1] = 0. In the Žrst term we can move a factor f [h(t) Cg(t ¡ tO)] = f [h(t)] in front of the integral since g vanishes for t ¡ tO > d rel . The Žrst term in equation A.1 can therefore be rewritten as Z

f [h(t)]

t¡d rel ¡1

O Sh ( t| tO) A( tO) dt,

(A.3)

where we have used the deŽnition (see equation 2.14) of the survivor function in noise model A. The integral in equation A.3 can be treated further if we use the normalization given in equation 3.6. The result of these manipulations is µ Z t ¶ Z t¡d abs O (A.4) A(t) = f [h(t)] 1¡ Sh ( t|tO) A( tO) dtO C Ph ( t|tO) A(tO) dt. t¡d rel

t¡d rel

Population Dynamics of Spiking Neurons

81

Except for a slight change of notation,2 equation A.4 is identical to the integral equation derived by Wilson and Cowan (1972). It applies to neurons with relative refractoriness of Žnite duration. For d rel ! 1 the expression in the square brackets vanishes, and we are back to equation 3.7. If there is no relative refractoriness but absolute refractoriness only, we may set d rel = d abs . The last term in equation A.4 then vanishes. Furthermore, during the absolute refractory period, we have a survival probablity Sh (t| tO) = 1 since the neurons cannot Žre. This yields » Z ( ) A t = f [h(t)] 1 ¡

t t¡d abs

¼ 0 0 ( ) A t dt ,

(A.5)

which is after the substitution f ¡! g exactly the Wilson-Cowan integral equation 1.3. Appendix B: Compression of Firing Times We prove the following generalization of the locking theorem: 1. @t h C@tOh > 0 is a necessary condition for a compression of Žring times. 2. For IF neurons, the condition @t h C @tOh > 0 is also sufŽcient.

Proof. With view to equations 4.2 and 4.3, we need to show that @t h C@tOh < 0 H) C < 0. We will use that u0 = g 0 C @t h > 0 at the moment of Žring. The assertion follows from C=

@t h C @tOh @t h C @tOh = 0 . g 0 ¡ @tOh u ¡ [@t h C @tOh]

(B.1)

We need to show that for IF neurons the denominator in equation B.1 is always positive. With view to equation 4.5, this is equivalent to the statement gQ 0 Ch(t ¡ Tb ) Ct h0 ( t ¡ Tb ) > 0 where t ¡ Tb = tO is the last Žring time. Hence u0 ( tO) > 0 and u( tO) = #. The differential equation for the IF model, 2.8, is t u0 = ¡u C input where input comprises synaptic and external currents. The input potential h( t) is a solution to the differential equation—hence, input = h C t h0 0 < t u0 ( tO) = ¡# C input < ¡ureset C h C t h0 ,

(B.2)

where we have used ureset < #. The assertion follows since gQ 0 = ¡ureset. In Wilson and Cowan (1972), f is called S , Sh (t| tO) A( tO) is called R(t, tO), and Ph (t| tO) A( tO) is S R(t, tO). 2

82

Wulfram Gerstner

Appendix C: Locking in the Presence of Noise for SRM0 Neurons We start from equation 4.12 and search for periodic pulse-type solutions. We assumeP that the pulses are gaussians with width d and repeat with period T: A( t) = n G d ( t¡nT ). The pulse width d will be determined self-consistently from equation 4.12. The integral over r in that equation can be performed and yields a gaussian with width sQ = [d2 C s 2 ]1/ 2 . Equation 4.12 becomes X

µ

G

n

d (t

¡ nT ) =

1C

h 0 ( t) g0 ( T)

¶X

G

n

sQ [t

¡ Tb (t) ¡ nT],

(C.1)

where Tb ( t) = t lnfgQ 0 / [h(t) ¡ #]g is the interspike interval looking backward in time. Let us work out the self-consistency condition and focus on the pulse around t ¼ 0. It corresponds to the n = 0 term on the left-hand side, which must equal the n = ¡1 term on the right-hand side of equation C.1. We assume that the pulse width is small d ¿ T and expand Tb ( t) to linear order around Tb (0) = T. This yields µ ¶ h0 (0) ¡ T. t ¡ T b ( t) = t 1 C 0 g (T)

(C.2)

The expansion is valid if h0 (t) varies slowly over the width d of the pulse. We use equation C.2 in the argument of the gaussian on the right-hand side of equation C.1. Since we have assumed that h0 varies slowly, the factor h0 ( t) in equation C.1 may be replaced by h0 (0). In the following we suppress the arguments and write simply h0 and g 0 . The result is d (t)

G

=

¡

h0 1C 0 g

¢

G

µ sQ

t

¡

h0 1C 0 g

¢¶

.

(C.3)

The gaussian on the left-hand side of equation C.3 must have the same width as the gaussian on the right-hand side. The condition is d = sQ / [1 C h0 / g 0 ] with sQ = [d2 C s 2 ]1/ 2 . A simple algebraic transformation yields an explicit expression for the pulse width, h ¡ ¢ ¡ ¢2 i¡1/ 2 , d = s 2 h0 / g 0 C h0 /g 0

(C.4)

where d is the width of the pulse and s is the strength of the noise. Appendix D: The Filter L (x) for Noise Model B We calculate the linear response of IF neurons with noise model B to an input O D I . We start from equation 4.11 and formally integrate over the variable t.

Population Dynamics of Spiking Neurons

83

This yields Z A ( t) =

µ ¶¡1 d O G dr 1 C T (t, r) dtO

s ( r)

A[t ¡ Tb ( t, r)],

(D.1)

where T b ( t, r) is the backward interval of neurons that have been reset the last time with value r. The derivative has to be evaluated at tO = t ¡ Tb ( t, r). In order to calculate the derivative needed in equation D.1, we write the membrane potential in the form ! ! t ¡ tO t ¡ tO t r/ O C h( t) ¡ h(t) exp ¡ , (D.2) u(t) = ¡gQ 0 e exp ¡ t t

(

(

where we have used equations 2.1, 2.4, 2.10, and 2.16. Neurons that Žre at t have u(t) = #, hence ! ! O r) O r) T (t, T ( t, r/ t O r)]¡h(tO) exp ¡ Ch[tO CT( t, . (D.3) # = ¡gQ 0 e exp ¡ t t

(

(

We take the derivative of equation D.3 with respect to tO and Žnd o 1 n 0 d 0 ¡Tb ( t,r)/ t O r) = ¡ ( ) ( )] ¡ [t ¡ , T (t, h t h T t, r e b u0 dtO

(D.4)

where u0 is the derivative of equation D.2. We now linearize equation D.1. The variation D I in the input current causes a perturbation D h( t) of the input potential (see equation 2.4) around h0 , which in turn evokes a change D A(t) in the activity. We set h( t) = h0 C D h( t) and A(t) = A0 CD A(t) (where D A is of the order D h) and expand both sides of equation D.1 to Žrst order in D A or D h. Note that the zero-order O r) vanishes (see equation D.4). The result of the linearization term of dO T ( t, dt is Z Z d ( ) ( ) ( )] DA t = G s r D A[t ¡ T r dr ¡ A0 G s ( r) T(t,O r) dr, (D.5) dtO where T ( r) = T 0 C T1 r is the interval of a neuron in the asynchronous state (no perturbation), which has been reset with value r. T1 = gQ 0 / (gQ 0 C h0 ) has O r) is given by equation D.4 with been introduced after equation 2.18. dO T (t, dt 0 ¡1 u = t [gQ 0 exp(r/ t) Ch0 ] exp[¡T(r)/ t ]. To simplify the expression, u0 may be approximated by its value for r = 0 and moved in front of the integral. O r]/ t ] ¼ exp[¡T0 / t). The Žnal result is Similarly, we use exp[¡Tb ( t, Z D A ( t) = G sO ( r) D A(t ¡ T0 ¡ r) dr C

A0 u0

µ

Z

h0 (t) ¡ e¡T0 / t

G

¶ 0 ( ) ( ) ¡ ¡ , r h t T r dr 0 sO

(D.6)

84

Wulfram Gerstner

where sO = T1 s. The Žrst term contains the convolution of D A with the (unperturbed) interval distribution. The second term can be written as a 0 Žlter L IF B applied to h (see Table 1 and equation 5.9). Appendix E: The Filter L ( x) for Noise Model A In noise model A, we have from equation 2.14, » Z t ¼ Sh ( t| tO) = exp ¡ f [g(t0 ¡ tO) C hPSP ( t0 | tO)] dt0 tO

(E.1)

where f [u] is the instantaneous escape rate across the noisy threshold. We write hPSP ( t| tO) = h0 ( t ¡ tO) C h1 ( t| tO). Taking the derivative with respect to h1 yields ­ @Sh ( t|tO) ­ ­ ­ = ¡ H( t1¡tO) H (t¡t1 ) f 0 [g(t1¡tO)Ch0 (t1¡tO)] S0 (t¡tO), (E.2) @h1 ( t1|tO) ­ h1 = 0

where S0 ( t ¡ tO) = Sh0 (t| tO) and f 0 = df/ du. For SRM0 neurons, we have O The Žlter L is therefore h0 ( t ¡ tO) ´ h0 and h1 ( t| tO) = h1 ( t), independent of t. Z

SRM ( t A

L

¡ t 1 ) = H ( t ¡ t1 )

t1 ¡1

dtO f 0 [g(t1 ¡ tO) C h0 ] S0 ( t ¡ tO),

(E.3)

as noted in Table 1. It is now shown that equation E.3 can be reduced to SRM ( ) x A

L

= ar e¡r x H (x),

(E.4)

in two limiting cases: (1) SRM0 -neurons and noise model A with instantaneous rate f (u) = r H (u ¡ #) and arbitrary g and 2 kernels and (2) SRM0 neurons and noise model A with arbitrary f (u) for neurons with absolute refractoriness g(s) = ¡1 for 0 < s < d abs and zero otherwise. Furthermore it is shown that for IF neurons with step function threshold,

L

IF ( ) A x

= L

SRM ( ) x A

¡ e¡T0 / t L

SRM ( x A

¡ T0 ).

(E.5)

E.1 Step Function Threshold. We take f ( u) = r H( u ¡ #). For r ! 1 neurons Žre immediately as soon as u( t) > #, and we are back to the noisefree sharp threshold. For Žnite r , neurons respond stochastically with time constant r ¡1 . Let us denote by T0 the time between the last Žring time tO and the formal O ( t ¡ tO) C h0 = #g. The derivative of f is threshold crossing, T0 = minft ¡ t|g a d -function r (E.6) f 0 [g(t ¡ tO) C h0 ] = r d[g(t ¡ tO) C h0 ¡ #] = 0 d (t ¡ tO ¡ T0 ), g

Population Dynamics of Spiking Neurons

85

d where g 0 = ds g| T0 . The survivor function S0 ( s) is unity for s < T0 and S0 (s) = exp[¡r ( s ¡ T0 )] for s > T0 . Integration of equation E.3 yields A(t

L

¡ t1 ) =

1 H( t ¡ t1 ) r exp[¡r (t ¡ t1 )], g0

as claimed above. For the Žlter L Z S0 ( x )

0

x

IF A

(E.7)

we also have to evaluate the integral

dj e¡j / t f 0 [u(j 0 )] = S0 ( x) H (x ¡ T0 )

r ¡T0 / t , e g0

(E.8)

which conŽrms the assertion. E.2 Absolute Refractoriness. We take an arbitrary escape rate f (u) ¸ 0 with limu!¡1 f ( u) = 0 = limu!¡1 f 0 (u). Absolute refractoriness is deŽned by a refractory kernel g(s) = ¡1 for 0 < s < d abs and zero otherwise. This yields f [g(t ¡ tO) C h0 ] = f ( h0 ) H( t ¡ tO ¡ d abs ) and hence f 0 [g(t ¡ tO) C h0 ] = f 0 ( h0 ) H( t ¡ tO ¡ d abs ).

(E.9)

The survivor function S0 ( s) is unity for s < d abs and decays as exp[¡ f (h0 ) ( s¡ d abs )] for s > d abs . Integration of equation E.3 yields

L

A(t

¡ t1 ) = H ( t ¡ t1 )

f 0 ( h0 ) exp[¡ f (h0 ) ( t ¡ t1 )]. f ( h0 )

(E.10)

Note that for neurons with absolute refractoriness, the transition to the noiseless case is not meaningful. As shown in appendix A, absolute refractoriness leads to the Wilson-Cowan integral equation, 1.3. Thus L A = L WC deŽned in equation E.10 is the Žlter relating to equation 1.3. It is a low-pass Žlter with cutoff frequency f ( h0 ), which depends on the input potential h0 . Acknowledgments I thank Carson C. Chow for many discussions, in particular for the idea of formulating the population equation as a conservation law. Thanks are due to the referees for numerous constructive comments. Carl van Vreeswijk has helped to improve the article through his many insightful suggestions. References Abbott, L. F., & van Vreeswijk, C. (1993). Asynchronous states in a network of pulse-coupled oscillators. Phys. Rev. E, 48, 1483–1490. Amari, S. (1974). A method of statistical neurodynamics. Kybernetik, 14, 201–215.

86

Wulfram Gerstner

Bauer, H. U., & Pawelzik, K. (1993). Alternating oscillatory and stochastic dynamics in a model for a neuronal assembly. Physica D, 69, 380–393. Brunel, N., & Hakim, V. (1999). Fast global oscillations in networks of integrateand-Žre neurons with low Žring rates. Preprint submitted to Neural Computation. Chow, C. C. (1998). Phase-locking in weakly heterogeneous neuronal networks. Physica D, 118, 343–370. Cohen, M. A., & Grossberg, S. (1983). Absolute stability of global pattern formation and parallel memory storage by competitive neural networks. IEEE Trans. on Systems, Man, and Cybernetics, 13, 815–823. Collins, J. J., Carson, C. C., Capela, A. C., & Imhoff, T. T. (1996). Aperiodic stochastic resonance. Physical Review E, 54, 5575–5584. Connors, B. W., & Gutnick, M. J. (1990).Intrinsic Žring patterns of diverse cortical neurons. Trends in Neurosci., 13, 99–104. Cowan, J. D. (1968). Statistical mechanics of nervous nets. In E. R. Caianello (Ed.), Proceedings of the 1967 NATO Conference on Neural Networks. Berlin: Springer-Verlag. Cox, D. R. (1962) Renewal theory. London: Methuen. Cox, D. R., & Lewis, P. A. W. (1966). The statistical analysis of series of events. London: Methuen. Eggert, J., & van Hemmen, J. L. Derivation of pool dynamics from microscopic neuronal models. In W. Gerstner et al. (Eds.), ArtiŽcial neural networks— ICANN’97 (pp. 109–114). Heidelberg: Springer-Verlag. Ermentrout, G. B. (1981). N:M phase-locking in weakly coupled oscillators. J. Math. Biology, 12, 327–342. Ernst, U., Pawelzik, K., & Geisel, T. (1994). Multiple phase clustering of globally coupled neurons with delay. In M. Marinaro & P. G. Morasso (Eds.), Proceedings of the ICANN’94 (pp. 1063–1065). Berlin: SpringerVerlag. Feldman, J. L., & Cowan, J. D. (1975). Large-scale activity in neural nets I: Theory with application to motoneuron pool responses. Biol. Cybern., 17, 29–38. Fetz, E. E., & Gustafsson, B. (1983). Relation between shapes of post-synaptic potentials and changes in Žring probability of cat motoneurones. J. Physiol., 341, 387–410. Gerstner, W. (1993). Kodierung und Signal¨ubertragung in Neuronalen Systemen: Assoziative Netzwerke mit stochastisch feuernden Neuronen, Frankfurt/Main: Harri–Deutsch Verlag. Gerstner, W. (1995). Time structure of the activity in neural network models. Phys. Rev. E, 51(1), 738–758. Gerstner, W., & van Hemmen, J. L. (1992). Associative memory in a network of “spiking” neurons. Network, 3, 139–164. Gerstner, W., & van Hemmen, J. L. (1993). Coherence and incoherence in a globally coupled ensemble of pulse emitting units. Phys. Rev. Lett., 71(3), 312–315. Gerstner, W., & van Hemmen, J. L. (1994). Coding and information processing in neural networks. In E. Domany, J. L. van Hemmen, & K. Schulten (Eds.), Models of neural networks II (pp. 1–93). New York: Springer-Verlag. Gerstner, W., van Hemmen, J. L., & Cowan, J. D. (1996).What matters in neuronal

Population Dynamics of Spiking Neurons

87

locking. Neural Comput., 8, 1689–1712. Golomb, D., Hansel, D., Shraiman, B., and Sompolinsky, H. (1992). Clustering in globally coupled phase oscillators. Phys. Rev. A, 45, 3516–3530. Golomb, D., & Rinzel, J. (1994). Clustering in globally coupled inhibitory neurons. Physica D, 72, 259–282. Gutkin, B., & Ermentrout, G. G. (1998). Dynamics of membrane excitability determine inter-spike interval variability: A link between spike generation mechanism and cortical spike train statistics. Neural Comput., 10, 1047–1065. Hansel, D., Mato, G., & Meunier, C. (1995). Synchrony in excitatory neural networks. Neural Comput., 7, 307–337. HopŽeld, J. J. (1984). Neurons with graded response have computational properties like those of two-state neurons. Proc. Natl. Acad. Sci. USA, 81, 3088–3092. HopŽeld, J. J., & Herz, A. V. M. (1995). Rapid local synchronization of action potentials: Towards computation with coupled integrate-and-Žre networks. Proc. Natl. Acad. Sci. USA, 92, 6655. Horn, D., and Levanda, S. (1998). Fast temporal encoding and decoding with spiking neurons. Neural Comput., 10, 1705–1720. Hubel, D. H., & Wiesel, T. N. (1962). Receptive Želds, binocular interaction and functional architecture in the cat’s visual cortex. J. Physiol. (London), 160, 106– 154. Kandel, E. C., & Schwartz, J. H. (1991). Principles of neural science. New York: Elsevier. Kistler, W., Gerstner, W., & van Hemmen, J. L. (1997). Reduction of HodgkinHuxley equations to a single-variable threshold model. Neural Comput., 9, 1015–1045. Knight, B. W. (1972a). Dynamics of encoding in a population of neurons J. Gen. Physiology, 59, 734–766. Knight, B. W. (1972b).The relationship between the Žring rate of a single neuron and the level of activity in a population of neurons J. Gen. Physiology, 59, 767– 788. Knox, C. K. (1974). Cross-correlation functions for a neuronal model Biophysical J., 14, 567–582. Koch, C., Rapp, M., & Segev, I. (1996). A brief history of time constants. Cerebral Cortex, 6, 93–101. Kuramoto, Y. (1975). Self-entrainment of a population of coupled nonlinear oscillators. In H. Araki (Ed.), International Symposium on Mathematical Problems in Theoretical Physics (pp. 420-422). Berlin: Springer-Verlag. Kuramoto, Y. (1984). Chemical oscillations, waves, and turbulence. Berlin: SpringerVerlag. Kuramoto, Y. (1991).Collective synchronization of pulse-coupled oscillators and excitable units Physica D, 50, 15–30. Mirollo, R. E., & Strogatz, S. H. (1990). Synchronization of pulse coupled biological oscillators. SIAM J. Appl. Math., 50, 1645–1662. Mountcastle, V. B. (1957). Modality and topographic properties of single neurons of cat’s somatosensory cortex. J. Neurophysiol., 20, 408–434. Nykamp, D., Tranchina, D., Shapley, R., & McLoughlin, D. (1998). A population density approach facilitates large-scale modeling of neural networks. To

88

Wulfram Gerstner

appear in Journal of Computational Neuroscience. Perkel, D. H., Gerstein, G. L., & Moore, G. P. (1967). Neuronal spike trains and stochastic point processes. (I) the single spike train. Biophys. J., 7, 391– 418. Pham, J., Pakdamen, K., Champagnat, J., & Vibert, J.-R. (1998). Activity in sparsely connected excitatory neural networks: Effect of connectivity. Neural Networks, 11, 415–438. Pinto, D. J., Brumberg, J. C., Simons, D. J., & Ermentrout, G. B. (1996). A quantitative population model of whiskers barrels: Re-examining the Wilson-Cowan equations. J. Comput. Neurosci., 3, 247–264. Plesser, H. E., & Gerstner, W. (1999). Noise in integrate-and-Žre neurons: From stochastic input to escape rates. To appear in Neural Computation Poliakov, A. V., Powers, R. K., & Binder, M. D. (1997). Functional identiŽcation of the input-output transforms of motoneurons in the rat and the cat J. Physiology, 504, 2:401–424. Senn, W., Wyler, K., Streit, J., Larkum, M., Luscher, ¨ H.-R., Mey, H., Muller, ¨ L., Steinhauser, D., Vogt, K., & Wannier, T. (1996). Dynamics of random neural network with synaptic depression. Neural Networks, 9, 575–588. Somers, D., & Kopell, N. (1993). Rapid synchronization through fast threshold modulation. Biol. Cybern., 68, 393–407. Spiridon, M., Chow, C. C., & Gerstner, W. (1998). Frequency spectrum of coupled stochastic neurons with refractoriness. In L. Niklasson et al. (Eds.), ICANN’98 (pp. 337–342). Berlin: Springer-Verlag. Stein, R. B. (1967). Some models of neuronal variability. Biophys. J., 7, 37–68. Stevens, C. F., & Zador, A. (1998). Novel integrate-and-Žre-like model of repetitive Žring in cortical neurons. In Proceedings of the Fifth Joint Symposion on Neural Computation, University of California at San Diego. Strogatz, S. H., & Mirollo, R. E. (1991). Stability of incoherence in a population of coupled oscillators. J. Stat. Phys., 63, 613–635. Terman, D., & Wang, D. (1995). Global competition and local cooperation in a network of neural oscillators. Physica D, 81, 148–176. Thorpe, S., Fize, D., & Marlot, C. (1996). Speed of processing in the human visual system. Nature, 381, 520–522. Treves, A. (1992). Local neocortical processing: A time for recognition, Int. J. Neural Systems, 3(Supp.), 115–119. Treves, A. (1993). Mean-Želd analysis of neuronal spike dynamics. Network, 4, 259–284. Treves, A., Rolls, E. T., & Simmen, M. (1997). Time for retrieval in recurrent associative memories. Physica D, 107, 392–400. Tsodyks, M., Mitkov, I., & Sompolinsky, H. (1993). Patterns of synchrony in inhomogeneous networks of oscillators with pulse interaction. Phys. Rev. Lett., 71, 1281–1283. Tsodyks, M. V., & Sejnowski, T. (1995). Rapid state switching in balanced cortical networks. Network, 6, 111–124. Tuckwell, H. C. (1988). Introduction to theoretic neurobiology. Cambridge: Cambridge University Press. Tuckwell, H. C. (1989). Stochastic processes in the neurosciences. Philadelphia:

Population Dynamics of Spiking Neurons

89

SIAM. van Vreeswijk, C., Abbott, L. E., & Ermentrout, G. B. (1994). When inhibition not excitation synchronizes neural Žring. Journal of Computational Neuroscience, 1, 313–321. van Vreeswijk, C., & Sompolinsky, H. (1996). Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science, 274, 1724–1726. Wilson, H. R., & Cowan, J. D. (1972). Excitatory and inhibitory interactions in localized populations of model neurons. Biophysical J., 12, 1–24. Wilson, H. R., & Cowan, J. D. (1973). A mathematical theory of the functional dynamics of cortical and thalamic nervous tissue. Kybernetik, 13, 55–80. Winfree, A. T. (1980). The geometry of biological time. Berlin: Springer-Verlag. Received March 23, 1998; accepted January 13, 1999.

ARTICLE

Communicated by Carl van Vreeswijk

Dynamics of Strongly Coupled Spiking Neurons Paul C. Bressloff S. Coombes Nonlinear and Complex Systems Group, Department of Mathematical Sciences, Loughborough University, Loughborough, Leicestershire LE11 3TU, U.K.

We present a dynamical theory of integrate-and-Žre neurons with strong synaptic coupling. We show how phase-locked states that are stable in the weak coupling regime can destabilize as the coupling is increased, leading to states characterized by spatiotemporal variations in the interspike intervals (ISIs). The dynamics is compared with that of a corresponding network of analog neurons in which the outputs of the neurons are taken to be mean Žring rates. A fundamental result is that for slow interactions, there is good agreement between the two models (on an appropriately deŽned timescale). Various examples of desynchronization in the strong coupling regime are presented. First, a globally coupled network of identical neurons with strong inhibitory coupling is shown to exhibit oscillator death in which some of the neurons suppress the activity of others. However, the stability of the synchronous state persists for very large networks and fast synapses. Second, an asymmetric network with a mixture of excitation and inhibition is shown to exhibit periodic bursting patterns. Finally, a one-dimensional network of neurons with long-range interactions is shown to desynchronize to a state with a spatially periodic pattern of mean Žring rates across the network. This is modulated by deterministic uctuations of the instantaneous Žring rate whose size is an increasing function of the speed of synaptic response. 1 Introduction There are two basic classes of neurodynamical model that are distinguished by their representation of neuronal output activity (see Abbott, 1994, and references therein). The Žrst considers the output of a neuron to be a mean Žring rate that either speciŽes the number of spikes emitted in some Žxed time window (HopŽeld, 1984; Amit & Tsodyks, 1991; Ermentrout, 1994) or corresponds to the probability of Žring within some population (Wilson & Cowan, 1972; Gerstner, 1995). Recently, however, a number of experiments on sensory neurons have shown that the precise timing of spikes may be signiŽcant in neuronal information processing, and this has led to renewed interest in the second class of neurodynamical models based on spiking neurons. For example, spike train recordings from H1, a motion-sensitive Neural Computation 12, 91–129 (2000)

c 2000 Massachusetts Institute of Technology °

92

Paul C. Bressloff and S. Coombes

neuron in the y visual system, exhibit variability of response to constant stimuli but a high degree of reproducibility for more natural dynamic stimuli (Strong, Koberle, van Steveninck, & Bialek, 1998). This reproducibility provides an enhanced capacity for carrying information (Rieke, Warland, & van Steveninck, 1996). Similar Žndings have been obtained in retinal ganglion cells of the tiger salamander and rabbit (Berry, Warland, & Meister, 1997). It is also well known that precise spike timing is essential for sound localization in the auditory system of animals such as the barn owl (Carr & Konishi, 1990). Recent advances in computational neuroscience support the notion that synapses are capable of supporting computations based on highly structured temporal codes (Mainen & Sejnowski, 1995; Gerstner, Kreiter, Markram, & Herz, 1997). Moreover, this mode of computation suggests how only one type of neuroarchitecture can support the processing of several very different sensory modalities (HopŽeld, 1995). Such codes are undoubtedly important in sensory and cognitive processing. The simplest and most popular example of a spiking neuron is the so-called integrateand-Žre (IF) model (Keener, Hoppensteadt, & Rinzel, 1981; Tuckwell, 1988) and its generalizations (Gerstner, 1995). The state of an IF neuron changes discontinuously (resets) whenever it crosses some threshold and Žres, so a complete description in terms of smooth differential equations is no longer possible. This type of model can be derived systematically from more detailed Hodgkin-Huxley equations describing the process of action potential generation (Abbott & Kepler, 1990; Kistler, Gerstner, & van Hemmen, 1997). There are major differences in the analytical treatments of Žring-rate and spiking neural network models. A common starting point for the former is to consider conditions under which destabilization of a homogeneous low-activity state occurs, leading to the formation of a state with inhomogeneous and/ or time-dependent Žring rates (see, for example, Wilson & Cowan, 1973; Ermentrout & Cowan, 1979; Atiya & Baldi, 1989; Li & HopŽeld, 1989; Ermentrout, 1998a). On the other hand, most of the work on IF network dynamics has been concerned with the existence and stability of phase-locked solutions, in which the neurons Žre at a Žxed common frequency. Analysis of globally coupled IF oscillators in terms of return maps has shown that synchronization almost always occurs in the presence of instantaneous excitatory interactions (Mirollo & Strogatz, 1990). Subsequently this result has been extended to take into account the effects of inhomogeneities and various synaptic and axonal delays (Treves, 1993; Tsodyks, Mitkov, & Sompolinsky, 1993; Abbott & van Vreeswijk, 1993; van Vreeswijk, Abbott, & Ermentrout, 1994; Ernst, Pawelzik, & Giesel, 1995; Hansel, Mato, & Meunier, 1995; Coombes & Lord, 1997; Bressloff, Coombes, & De Souza, 1997; Bressloff & Coombes, 1998, 1999). One Žnds that for small transmission delays, inhibitory (excitatory) synapses tend to synchronize if the rise time of a synapse is longer (shorter) than the duration of an action potential. Increasing the transmission delay leads to alternating bands of stability and instability of the synchronous state. Analogous results have been ob-

Dynamics of Strongly Coupled Spiking Neurons

93

tained in the spike response model (Gerstner, Ritz, & van Hemmen, 1993; Gerstner, 1995; Gerstner, van Hemmen, & Cowan, 1996). Traveling waves of synchronized activity have been investigated in Žnite chains of IF oscillators modeling locomotion in simple vertebrates (Bressloff & Coombes, 1998), and in two-dimensional networks where spirals and target patterns are also observed (Chu, Milton, & Cowan, 1994; Kistler, Seitz, & van Hemmen, 1998). In this article we present a theory of spike train dynamics in IF networks that bridges the gap between Žring-rate and spiking models. In particular, we show that synaptic interactions that are synchronizing in the weak coupling regime can become desynchronizing for sufŽciently strong coupling. The resulting dynamics is compared with the behavior of a corresponding analog model in which the outputs of the neurons are taken to be mean Žring rates. A basic result of our work is that for slow interactions, there is good agreement between the two models (on an appropriately deŽned timescale). On the other hand, discrepancies can arise for fast synapses where IF neurons may remain phase locked. We take as our starting point the nonlinear mapping of the neuronal Žring times and show how a bifurcation analysis of this map can serve as a basis for understanding the extremely rich dynamical structure seen in networks of spiking neurons. Explicit criteria for the stability of phase-locked solutions are derived in both the weak and strong coupling regimes by considering the propagation of perturbations of the Žring times throughout a network. In the strong coupling case, the analysis predicts regions in parameter space where instabilities in the Žring times may cause transitions to nonphase-locked states. Numerical simulations are used to establish that in these regions, the full nonlinear Žring map can support several distinct types of behavior. For small networks, these include mode-locked bursting states, where packets of spikes may be generated, separated by periods of inactivity, and inhomogeneous states in which some of the oscillators become inactive. For large global inhibitory networks with vanishing mean perturbations of the Žring times, we show that such bifurcations can be suppressed, in agreement with the mode locking theorem of Gerstner et al. (1996). As a Žnal example of the importance of strong coupling instabilities, we consider a ring of IF neurons with a Mexican hat interaction function. A discrete Turing-Hopf bifurcation of the Žring times from a synchronous state to a state with periodic or quasi-periodic variations of the interspike intervals (ISIs) on closed orbits is shown to occur in the strong coupling regime. Further, it is shown how the separation of these orbits in phasespace results in a spatially periodic pattern of mean Žring rate across the network. 2 Integrate-and-Fire Model of a Spiking Neuron The standard dynamical system for describing a neuron with spatially constant membrane potential V is based on conservation of electric charge, so

94

Paul C. Bressloff and S. Coombes

that C

dV = ¡F C Is C I, dt

(2.1)

where C is the cell capacitance, F is the membrane current, Is the sum of synaptic currents entering the cell, and I describes any external currents. In the Hodgkin-Huxley model the membrane current arises mainly through the conduction of sodium and potassium ions through voltage-dependent channels in the membrane. The contribution from other ionic currents is assumed to obey Ohm’s law. In fact F is considered to be a function of V and of three time-dependent and voltage-dependent conductance variables m, n, and h, F( V, m, n, h) = gL (V ¡ VL ) C gK n4 (V ¡ VK ) C gNa hm3 ( V ¡ VNa ), (2.2) where gL , gK , and gNa are constants and VL , VK , and VNa represent the constant membrane reversal potentials associated with the leakage, potassium, and sodium channels, respectively. The conductance variables m, n, and h take values between 0 and 1 and approach the asymptotic values m1 ( V), n1 ( V ) and h1 ( V ) with time constants tm (V ), tn ( V ) and th (V ), respectively. The conductance-based Hodgkin-Huxley equations depend on four dynamical variables. A reduction of this number is often desirable in order to facilitate any mathematical analysis and to ease the computational burden for simulations of large networks. A systematic approach for doing this involves the use of equivalent potentials (Abbott & Kepler, 1990; Kepler, Abbott, & Marder, 1992). Following this approach, one can derive an IF model, which provides a caricature of the capacitative nature of cell membrane at the expense of a detailed model of the refractory process. The IF model satisŽes equation 2.1 with F = F( V), together with the condition that whenever the neuron reaches a threshold h, it Žres, and V is immediately reset to the resting potential V0 . The dynamics of the membrane conductances m, n, h is eliminated completely. Using a curve-Žtting procedure, it is possible to approximate F( V) by a cubic, F( V ) = a( V ¡ V0 )(V ¡ V1 )( V ¡ h) where the constants a, V0,1 , and h can be determined explicitly from the reduction of the underlying Hodgkin-Huxley equations. At a synapse, presynaptic Žring results in the release of neurotransmitters that causes a change in the membrane conductance of the postsynaptic neuron. This postsynaptic current may be written Is = gs s( Vs ¡ V ),

(2.3)

where V is the voltage of the postsynaptic neuron, Vs is the membrane reversal potential, and gs is a constant. The variable s corresponds to the probability that a synaptic receptor channel is in an open conducting state.

Dynamics of Strongly Coupled Spiking Neurons

95

This probability depends on the presence and concentration of neurotransmitter released by the presynaptic neuron. Under certain assumptions, it may be shown that a second-order Markov scheme for the synaptic channel kinetics describes the so-called alpha function response commonly used in synaptic modeling (Destexhe, Mainen, & Sejnowski, 1994): the synaptic response to an incoming spike at time t0 is s( t) = s0 ( t ¡ t0 )e¡a(t¡t0 ) ,

t > t0

(2.4)

for constants s0 , a. Here a determines the inverse rise time for synaptic response. If we now combine equations 2.1 and 2.3 under the approximation F = F(V ), we obtain an equation of the form (after setting C = 1) X dV = ¡F(V) C I C gs s( t ¡ tm )[Vs ¡ V]. dt m

(2.5)

We are assuming that the neuron receives a sequence of action potential spikes at times ftm g and each spike generates a synaptic response according to equation 2.4. The neuron itself Žres a spike whenever V( t) reaches the threshold h, and V is immediately reset to the resting potential, V0 . Denoting the Žring times of the neuron by fT m g, we can view the neuron as a device that maps ftm g ! fTm g. We introduce two additional simpliŽcations. First, we neglect shunting effects by setting Vs ¡ V ¼ Vs ; the possible nonlinear effects of shunting are discussed by Abbott (1991). Second, we reduce F( V) further by considering the linear approximation F( V ) = b(V ¡ V0 ) (Abbott & Kepler, 1990). This linear IF model of a spiking neuron will be used in our subsequent analysis of network dynamics (see sections 3 and 4). However, before proceeding further, we briey mention an alternative formulation of spiking neurons based on the so-called spike response model. The IF model assumes that it is the capacitative nature of the cell that in conjunction with a simple thresholding process dominates the production of spikes. The spike response (SR) model (Gerstner & van Hemmen, 1994; Gerstner, 1995; Gerstner et al., 1996) is a more general framework that can accommodate the apparent reduced excitability (or increased threshold) of a neuron after the emission of a spike. Spike reception and spike generation are combined with the use of two separate response functions. The Žrst, hs ( t), describes the postsynaptic response to an incoming spike in a similar fashion to the IF model, whereas the second, hr ( t), mimics the effect of refractoriness. The refractory function hr ( t) can in principle be related to the detailed dynamics underlying the description of ionic channels. In practice, an idealized functional form is often used, although numerical Žts to the Hodgkin-Huxley equations during the spiking process are also possible (Kistler et al., 1997). In more detail, a sequence of incoming spikes ftm g evokes P a postsynaptic potential in the neuron via V s ( t) = m hs ( t ¡ tm ), where hs ( t) incorporates details of axonal, synaptic, and dendritic processing. The total

96

Paul C. Bressloff and S. Coombes

r( ) s ( ), () membrane Ppotential of the neuron is taken to be V t = V t CV t where ( ) ¡ and fT g is the sequence of output Žring times. V r ( t) = h t T m m m r Whenever V ( t) reaches some threshold, the neuron Žres a spike, and at the same time a negative contribution hr is added to V so as to approximate the reduced excitability seen after Žring. Since the reset condition of the P IF model is equivalent to a sequence of current pulses, ¡h m d (Tm ), the linear IF model is a special case of the SR model. That is, if I = 0, F(V ) = ¡V and we neglect shunting, then we can integrate equation 2.5 to obtain the equivalent formulation

V ( t) =

X m

hr ( t ¡ Tm ) C

X m

hs ( t ¡ tm ),

(2.6)

where Z hr (t) = ¡he¡t ,

hs (t) =

t 0

e¡(t¡t ) s( t0 )dt0 , 0

t > 0,

(2.7)

and there is no reset. Most work to date on the analysis of the SR model has been based on the study of large networks using dynamical mean Želd theory (Gerstner & van Hemmen, 1994; Gerstner, 1995), which can be viewed as a generalization of the population averaging approach of Wilson and Cowan (1972). This is different from the approach we take here, which is principally concerned with the dynamics of Žnite networks of spiking neurons. However, we will link the two in section 4.3, where we discuss large, globally coupled networks and the mode-locking theorem of Gerstner et al. (1996). 3 Averaging Methods for Analyzing IF Network Dynamics We now consider a network of IF neurons that interact via synapses by transmitting spike trains to one another. Let Vi ( t) denote the state of the ith neuron at time t, i = 1, . . . , N, where N is the total number of neurons in the network. Suppose that the variables Vi (t) evolve according to the equations (cf. equation 2.5) dVi ( t) Vi (t) = ¡ C Ii C Xi (t), dt td

(3.1)

where Ii is a constant external bias, td is the membrane time constant, and Xi ( t) is the total synaptic current into the cell. We shall Žx the units of time by setting td = 1; typical values for td are in the range 5–20 msec. Equation 3.1 is supplemented by the reset condition that whenever Vi = h, neuron i Žres and is reset to Vi = 0. We shall set the threshold h = 1. The synaptic current

Dynamics of Strongly Coupled Spiking Neurons

97

Xi (t) is generated by the arrival of spikes from neuron j and takes the explicit form Xi ( t) = 2

N X

Wij

j= 1

1 X m= ¡1

J(t ¡ Tjm ),

(3.2)

where 2 Wij is the effective synaptic weight of the connection from the jth to the ith neuron, J( t) determines the time course of the postsynaptic response to a single spike with J( t) = 0 for t < 0, and Tjm denotes the sequence of Žring times of the jth neuron with m running through the integers. We have introduced a coupling constant 2 characterizing the overall strength of synaptic interactions. From the discussion in section 2, a biologically motivated choice for J(t) is J( t) = s(t ¡ ta )H( t ¡ ta ),

s(t) = a2 te¡at ,

(3.3)

where s(t) is an alpha function (with unit normalization), ta is a discrete axonal transmission delay, and H is the unit step function, H (t) = 1 if t > 0 and zero otherwise. The maximum synaptic response then occurs at a nonzero delay t = ta C a¡1 . In a companion paper, the effects of dendritic interactions (both passive and active) will be analyzed by taking J (t) to be the Green’s function of some cable equation (Bressloff, 1999). Although equations 3.1 and 3.2 are considerably simpler than those describing a network of Hodgkin-Huxley neurons, the analysis of the resulting dynamics is still a nontrivial problem since one has to handle both the presence of delays and discontinuities arising from reset. Most work to date has therefore been based on some form of approximation scheme. We shall describe two such schemes: phase reduction in the weak coupling regime and time averaging with slow synapses. In section 4 we shall then carry out a direct analysis of the IF network dynamics without recourse to such approximations. 3.1 Weak Coupling: Phase-Oscillator Models. We Žrst show how in the weak coupling limit, equation 3.1 with reset can be reduced to a phaseoscillator equation. For concreteness, assume that Ii = I > 1 for all i = 1, . . . , N, so that in the absence of any coupling (2 = 0), each neuron acts as a regular oscillator by Žring spikes with a constant period T = ln[I/ ( I ¡ 1)]. Following van Vreeswijk et al. (1994), we introduce the phase variable y i (t) according to (mod 1) yi ( t) C

1 t = Y(Vi ( t)) ´ T T

Z

Vi ( t ) 0

dx , F( x)

(3.4)

where F( x) = I¡x for the linear IF model. (One can also apply this transform to the nonlinear IF model discussed in section 2 with F( x) now given by a

98

Paul C. Bressloff and S. Coombes

cubic.) Under such a transformation, equation 3.1 becomes yP i ( t) = RT (y i ( t) C t/ T ) Xi (t),

(3.5)

with 1 1 = RT (h ) ´ ¡1 T F[Y (h )]

£

¤ 1 ¡ e¡T eTh

(3.6)

T

for 0 · h < 1 and RT (h C k) = RT (h ) for all integers k. In the absence of any coupling, the phase variable y i (t) is constant in time, and all oscillators Žre with their natural periods T. For weak coupling, each oscillator still approximately Žres at the unperturbed rate, but now the phases slowly drift according to equation 3.5 since Xi ( t) = O (2 ). To Žrst order in 2 , we can take the Žring times to be Tjn = (n ¡ yj ( t))T. Under this approximation, equations 3.2 and 3.5 lead to the following equation for the shifted phases hi ( t) = yi ( t) C t/ T: N X dh i 1 = C2 W ij RT (h i ) PT (hj ) C O (2 2 ), dt T j= 1

(3.7)

where PT (h C 1) = PT (h ) for all h and P T (h ) =

1 X m= ¡1

J((h C m)T ),

0 · h < 1.

(3.8)

The summation over m in equation 3.8 is easily performed for J(t ), satisfying equation 3.3, and gives PT (h ) = b JT (h ¡ ta / T ), where b JT (h ) is a periodic function of h with µ ¶ a2 e¡ahT Te¡aT b hT C , JT (h ) = 1 ¡ e¡aT 1 ¡ e¡aT

0 · h < 1.

(3.9)

The function RT may be interpreted as the phase-response curve (PRC) of an individual IF oscillator, and PT is the corresponding pulselike function that contains all details concerning the synaptic interactions. Note that phase equations of the form 3.7 can also be derived for a system of weakly coupled limit cycle oscillators based on more detailed biophysical models such as Hodgkin-Huxley neurons (Ermentrout & Kopell, 1984; Kuramoto, 1984; Hansel et al., 1995). As discussed in some detail by Hansel et al. (1995), linear IF oscillators have a type I PRC, which means that an instantaneous excitatory stimulus always advances its phase (RT (h ) is positive for all h ). On the other hand, Hodgkin-Huxley neurons are of type II since a stimulus can either advance or retard the phase depending on the point on the cycle

Dynamics of Strongly Coupled Spiking Neurons

99

at which the stimulus is applied (RT (h ) takes on both positive and negative values over the domain h 2 [0, 1]). Equation 3.7 can be simpliŽed further by averaging over the natural period T. This leads to an equation of the form N X dhi = v C2 Wij HT (hj ¡ h i ) C O (2 2 ) dt j= 1

with v = 1/ T and Z 1 HT (w) = RT (h )P T (h C w ) dh . 0

(3.10)

(3.11)

For positive kernels J(t ), the interaction function HT (w ) is positive for all w since IF oscillators are of type I. However, it is possible to mimic an effective type II interaction function by introducing a combination of excitatory and inhibitory interactions between the IF neurons in the deŽnition of J(t ) (Bressloff & Coombes, 1998a). To illustrate this idea, suppose that we decompose J(t ) as J(t ) = J C(t ) ¡ J¡ (t ), 2 t e¡a§ t , s§ (t ) = a§

J§ (t ) = s§ (t ¡ t§ )H (t ¡ t§ ),

(3.12)

where J§ (t ) represent a functions for excitatory (C) and inhibitory (¡) synapses. Assuming that the inhibitory pathways are delayed with respect to the excitatory ones (t¡ > t C) then the effective interaction function of an IF oscillator can be approximately sinusoidal. This is illustrated in Figure 1. It is not clear that such a combination of synapses is directly realized in cortical microcircuits. However, it is known that recurrent excitatory connections also stimulate inhibitory interneurons, and this might lead to an effective delay kernel of the form 3.14 at the population level. We deŽne a phase-locked solution of equation 3.10 to be of the form hi (t) = w i C Vt, where w i is a constant phase and V = 1/ T C O (2 ) is the collective frequency of the coupled oscillators. Substitution of this solution into equation 3.10 and working to O (2 ) leads to the Žxed-point equations, V=

N X 1 C2 Wij HT (w j ¡ w i ). T j= 1

(3.13)

After choosing some reference oscillator, the N equations (3.13) determine the collective period V and N¡1 relative phases, with the latter independent of 2 . In order to analyze the local stability of a phase-locked solution W = (w 1 , . . . , w N ), we linearize equation 3.10 by setting h i (t) = w i C Vt C hei (t) and expanding to Žrst order in hei : N X dhei bij (W)hej , = 2 H dt j= 1

(3.14)

100

Paul C. Bressloff and S. Coombes

Figure 1: (Top) Delay kernel J(t) and (bottom) associated interaction function HT (w ) for a combination of excitatory and delayed inhibitory synaptic interactions given by equation 3.12. Here a§ = 10, t C = 0, t¡ = 0.6, and T = ln 2.

where bij (W) = Hij (W) ¡ d i, j H

N X k= 1

Hik (W),

Hij (W) = Wij HT0 (w j ¡ w i )

(3.15)

b is always and HT0 (w ) = dHT (w )/ dw . One of the eigenvalues of the Jacobian H zero, and the corresponding eigenvector points in the direction of the ow, that is (1, 1, . . . , 1). The phase-locked solution will be stable provided that all other eigenvalues have a negative real part (Ermentrout, 1985). The existence and stability of phase-locked solutions in the case of a symmetric pair of excitatory or inhibitory IF neurons with synaptic interactions have been studied in some detail by van Vreeswijk et al. (1994). In this case, N = 2 and W 11 = W22 = 0, W12 = W21 = 1. Equation 3.13 then shows that the allowed solutions for the relative phase w = w 2 ¡ w 1 are given by the zeroes of the odd interaction function HT¡ (w ) = HT (w) ¡ HT (¡w). The underlying symmetry of the pair of neurons guarantees the existence of the in-phase or synchronous solution w = 0 and the antiphase or antisynchronous solution w = 1/ 2. Suppose that ta = 0. For small a, one Žnds that the in-phase and antiphase solutions are the only phase-locked solutions. However, as a is increased, a critical value ac is reached, where

Dynamics of Strongly Coupled Spiking Neurons

101

1.0 (a)

0.8 f

0.6 0.4 0.2 0.0 2

6

10

14 a

18

1.0 0.8 f

0.6 0.4 0.2 0.0 0

0.2

t

0.4

0.6

Figure 2: (a) Relative phase w = w 2 ¡w 1 for a pair of IF oscillators with symmetric inhibitory coupling as a function of a with I = 2. In each case the antiphase state undergoes a bifurcation at a critical value of a = ac , where it becomes stable, and two additional unstable solutions w , 1 ¡ w are created. The synchronous state remains stable for all a. (b) Relative phase w versus discrete delay ta for a pair of pulse-coupled IF oscillators with I = 2 and a = 2.

there is a bifurcation of the antiphase solution, leading to the creation of two partially synchronized states h and 1 ¡ h , with 0 < h < 1/ 2 and h ! 0 as a ! 1 (see Figure 2a and van Vreeswijk et al., 1994). As shown by Coombes and Lord (1997), variation of ta for Žxed a produces a checkerboard pattern of alternating stable and unstable solution branches (see Figure 2b) that can overlap to produce multistable solutions. Using equation 3.15, one obtains the following necessary and sufŽcient condition for local asymptotic stability of a phase-locked state in the weak coupling regime: 2

dHT¡ (w ) > 0. dw

(3.16)

One Žnds that for ta = 0 and inhibitory coupling (2 < 0), the synchronous state is stable for all 0 < a < 1. Moreover, the antiphase solution w = 1/ 2 is unstable for a < ac , but it gains stability when a > ac with the creation of two unstable partially synchronized states. The stability properties of

102

Paul C. Bressloff and S. Coombes 2 1.6 1.2 a

- 1

0.8 0.4 0

0

0.2

0.4 t

0.6

0.8

1

Figure 3: Stability of the synchronous solution w = 0 as a function of a¡1 and ta for weak excitatory coupling with I = 2. Stable and unstable regions are denoted by s and u, respectively. The stability diagrams are periodic in ta with period ln[I/ (I ¡ 1)]. (This periodicity would be distorted in the strong coupling regime since the collective period of oscillation depends on the strength of coupling).

all solutions are reversed in the case of excitatory coupling (2 > 0) so that the synchronous state is now unstable for all a. If the discrete delay ta is increased from zero, then alternating bands of stability and instability are created. An example of such regions is displayed in Figure 3. The corresponding stability diagram for the antisynchronous state may be obtained by shifting the delay according to ta ! ta C T/ 2. 3.2 Slow Synapses: Analog Models. A second approximation scheme for analyzing IF network dynamics is to assume that the synaptic interactions are sufŽciently slow so that the output of a neuron can be characterized reasonably well by a mean (time-averaged) Žring rate (see, for example, Amit & Tsodyks, 1991; Ermentrout, 1994). Therefore, let us consider the case in which J(t ) is given by the alpha function (see equation 3.3) with a synaptic rise time a¡1 signiŽcantly longer than all other timescales in the system. Suppose that the total synaptic current Xi ( t) to neuron i is described by a slowly varying function of time t. If the neuronal dynamics is fast relative to a¡1 , then the actual Žring rate Ei (t) of a neuron will quickly relax to approximately its steady-state value, that is, E i (t) = f (Xi ( t) C Ii ),

(3.17)

where, from equation 3.1, the Žring-rate function f is of the form f ( X) =

» µ ln

X X¡1

¶¼

¡1

H(X ¡ 1).

(3.18)

Dynamics of Strongly Coupled Spiking Neurons

103

(Note that we have ignored the effects of absolute refractory period, which is reasonable when the system is operating well below its optimal Žring rate). Equation 3.17 relates the dynamics of the Žring rate directly to the stimulus dynamics Xi (t) through the steady-state response function. In effect, the use of a slowly varying kernel J(t ) allows a consistent deŽnition of the Žring rate so that a dynamical network model can be based on the steady-state properties of an isolated neuron. Within the above approximation, we can derive a closed set of equations for the synaptic currents Xi (t). This is achieved by rewriting equation 3.2 as a pair of differential equations that generate the alpha function J( t) of equation 3.3, and replacing the output spike train of a neuron by the Žringrate function (see equation 3.18): a¡1

dXi (t) C Xi ( t) = Yi ( t), dt

(3.19)

a¡1

N X dYi (t) C Y i ( t) = 2 W ij Ej ( t ¡ ta ). dt j= 1

(3.20)

Here Yi ( t) is an auxiliary current. A common starting point for the analysis of analog models is to consider conditions under which destabilization of a homogeneous low-activity state occurs, leading to the formation of a state with inhomogeneous and/ or timedependent Žring rates (see Ermentrout, 1998a, for a review). To simplify our analysis, we shall impose the following condition on the external bias Ii , I = Ii C 2 f ( I )

N X

(3.21)

Wij

j= 1

for some Žxed I > 1. Then for sufŽciently weak coupling, |2 | ¿ 1, the P analog model has a single stable Žxed point given by Yi = Xi = 2 f ( I) j Wij , such that the Žring rates are kept at the value f (I). Suppose that we linearize equations 3.19 and 3.20 about this Žxed point and substitute into the linearized equations a solution of the form ( Xk (t), Yk ( t)) = el t (Xk , Yk ). This leads to the eigenvalue equation l p§ a

q § = ¡1 § 2 f 0 ( I)ºp e¡l p ta / 2 ,

p = 1, . . . , N,

(3.22)

whereºp , p = 1, . . . , N, are the eigenvalues of the weight matrix W. The Žxed point will be asymptotically stable if and only if Rel pt < 0 for all p. As |2 | is increased from zero, an instability may occur in at least two distinct ways. If a single real eigenvalue l crosses the origin in the complex l -plane, then a static bifurcation can occur, leading to the emergence of additional Žxedpoint solutions that correspond to inhomogeneous Žring rates. For example,

104

Paul C. Bressloff and S. Coombes

if 2 > 0 and W has real eigenvalues º1 > º2 > ¢ ¢ ¢ > ºN with º1 > 0, then a static bifurcation will occur at the critical coupling 2 c for which l 1C = 0, p that is, 1 = 2 c f 0 ( I)º1 . On the other hand, if a pair of complex conjugate eigenvalues l = l R § il I crosses the imaginary axis (l R = 0) from left to right in the complex plane, then a Hopf bifurcation can occur, leading to the formation of periodic solutions, that is, time-dependent Žring rates. For example, suppose that ta = 0 and W has a pair of complex eigenvalues (º, º¤ ) with º = reih and 0 < h < p . Denote the corresponding solutions of equation 3.22 by (l § , l ¤§ ). Assuming that all other eigenvalues l have negative real part, a Hopf bifurcation p will occur at the critical coupling 2 c for which Rel C = 0, that is, 1 = 2 c f 0 ( I) r cos(h/ 2). An alternative way of generating oscillatory solutions is to have nonzero delays ta (Marcus & Westervelt, 1989). Note that the basic stability results are independent of the inverse rise time a. To illustrate, consider a symmetric pair of analog neurons with inhibitory coupling, 2 < 0, and W11 = W22 = 0, W 12 = W 21 = 1. The weight matrix W has eigenvalues §1 and eigenmodes (1, §1). Let xi = Xi ¡ 2 f ( I) so that the Žxed-point equations become x1 = 2 f ( x2 C I) ¡ 2 f (I) and x2 = 2 f (x1 C I) ¡ 2 f ( I). One solution is the homogeneous Žxed point xi = 0. A full bifurcation diagram showing the location of the Žxed points x1 as a function of |2 | is shown in Figure 4 (top). The homogeneous Žxed point xi = 0 is stable for sufŽciently small coupling |2 | but destabilizes at the critical point |2 | = 2 c with 2 c = 1/ f 0 ( I), where it coalesces with two unstable Žxed points. For |2 | > 2 c , the unstable Žxed point at the origin coexists with two stable Žxed points (arising from saddle-node bifurcations). Just beyond the bifurcation point, the system jumps from a homogeneous state to a state in which one neuron is active with a constant Žring rate f ( I ¡ 2 f (I)) and the other is passive with zero Žring rate. Note that a pair of excitatory analog neurons would bifurcate into another homogeneous state. For example, since Ii has to decrease to keep the Žring rate constant when 2 > 0 (see equation 3.21), for strong enough coupling Ii < 1 so that the quiescent state is also a valid solution. A well-known result from the analysis of analog neurons is that an excitatory-inhibitory pair can undergo a Hopf bifurcation to an oscillatory solution (Atiya & Baldi, 1989). As an illustration, consider the case 2 > 0, ta = 0 and W11 = W22 = 0, W 12 = ¡2, W21 = 1. Here neuron 2 inhibits neuron 1, whereas neuron 1 excites neuron 2. The eigenvalues of the weight matrix are §i so that a Hopf bifurcation arises as 2 is increased. (See the discussion below equation 3.22.) Let x1 = X1 C22 f ( I) and x2 = X2 ¡ 2 f ( I) so that there exists a Žxed point at xi = 0. The bifurcation diagram for the amplitude x1 as a function of 2 is shown in Figure 4 (bottom). It can be seen that the system undergoes a so-called subcritical Hopf bifurcation in which the homogeneous Žxed point xi = 0 becomes unstable and the system jumps to a coexisting stable limit cycle, signaling a solution with periodic Žring rates.

Dynamics of Strongly Coupled Spiking Neurons

1

105

active

passive e 1

e

Figure 4: (Top) Bifurcation diagram for a pair of analog neurons with symmetric inhibitory coupling and external input I = 2. (Bottom) Subcritical Hopf bifurcation for a pair of analog oscillators with self-interactions. a = 0.5, I = 2, W11 = W22 = 0, W21 = 1, and W12 = ¡2. Open circles denote the amplitude of the resulting limit cycle from the Hopf bifurcation point 2 ¼ 2.0, and s (u) stands for stable (unsta ble) dynamics.

This form of jump is often referred to as a hard excitation. The system also exhibits hysteresis. It is interesting to note that if the Žring-rate function f ( X) of equation 3.18 were taken to be the usual smooth sigmoid function, then, for the given weights, the Hopf bifurcation would be supercritical in the sense that the limit cycle would grow smoothly from the unstable Žxed point so that there is no jump phenomenon or hysteresis. This is called a soft excitation. (See Atiya & Baldi, 1989.) Another important point is that if the analog model were described by a Žrst-order equation rather than a second-order equation (as given by equations 3.19 and 3.20), then it would be necessary to introduce additional self-interactions (W12 , W21 6 = 0) in order for a Hopf bifurcation to occur. This is difŽcult to justify on neurobiological grounds. (A Žrst-order equation would be obtained if the delay kernel were taken to be an exponential function rather than the alpha function 3.3.) We shall return to this issue in section 4.6. 4 Spike Train Dynamics in the Strong-Coupling Regime Recall from section 3 that networks of weakly coupled IF neurons can have stable phase-locked solutions in which all the neurons have the same con-

106

Paul C. Bressloff and S. Coombes

stant ISI. On the other hand, a strongly coupled network of analog neurons can bifurcate from a stable homogeneous state with identical timeindependent Žring rates to a state with inhomogeneous and/ or time-varying Žring rates. (Note that a homogeneous state of the analog model does not distinguish between different phase-locked solutions since all phase information is lost during time averaging.) This suggests that there should exist some mechanism for destabilizing phase-locked solutions of the IF model in the strong-coupling regime. Here we identify such a mechanism based on a discrete Hopf bifurcation of the Žring times. This induces inhomogeneous and periodic variations in the ISIs, which supports a variety of complex dynamics, including oscillator death (section 4.3), bursting (section 4.4), and pattern formation (section 4.5). 4.1 Phase Locking for Arbitrary Coupling. An important simplifying aspect of the dynamics of pulse-coupled (linear) IF neurons is that it is possible to derive phase-locking equations without the assumption of weak coupling (van Vreeswijk et al., 1994; Bressloff et al., 1997; Bressloff & Coombes, 1999). This can be achieved by solving equation 3.1 directly under the ansatz that the Žring times are of the form Tjn = ( n ¡ w j ) T for some self-consistent period T and constant phases wj . Integrating equation 3.1 over the interval t 2 (¡Tw i , T ¡ Tw i ) and incorporating the reset condition by setting Ui (¡w i T ) = 0 and Ui ( T ¡ w i T ) = 1 leads to the result 1 = (1 ¡ e¡T )Ii C 2

N X j= 1

Wij KT (w j ¡ w i ),

(4.1)

where Z KT (w ) = e¡T

T 0

et

1 X m= ¡1

J(t C (m C w ) T ) dt = ¡

T 2 e¡T ¢ HT (w). (4.2) 1 ¡ e¡T

Equation 4.1 has an identical structure to that of equation 3.13 and involves the same phase interaction function (up to a multiplicative factor). Indeed, in the weak coupling regime with Ii = I for all i, equation 4.1 reduces to equation 3.13 with 1/ T = V. This means that techniques previously developed for studying phase locking in weakly coupled oscillator networks can be extended to strongly coupled IF networks. For example, in the case of a ring of identical IF oscillators with symmetric coupling, group-theoretic methods can be used to classify all phase-locked solutions and construct bifurcation diagrams showing how new solution branches emerge via spontaneous symmetry breaking (Bressloff et al., 1997; Bressloff & Coombes, 1999). Furthermore, in the case of a Žnite chain of IF oscillators with a gradient of external inputs and sinusoidal-like phase interaction functions of the form shown in Figure 1, one can establish the existence of “traveling

Dynamics of Strongly Coupled Spiking Neurons

107

wave” solutions in which the phase varies monotonically along the chain (except in some narrow boundary layer); such systems can be used to model locomotion in simple vertebrates (Bressloff & Coombes, 1998a). There are, however, a number of signiŽcant differences between phaselocking equations 3.13 and 4.1. First, equation 4.1 is exact, whereas equation 3.13 is valid only to O (2 ) since it is derived under the assumption of weak coupling. Second, the collective period of oscillations T must be determined self-consistently in equation 4.1. Assume for the moment that T is given. Suppose that we choose h 1 as a reference oscillator and subtract from equation 4.1 for i = 2, . . . , N the corresponding equation for i = 1. This leads to N ¡1 Žxed-point equations for the N ¡1 relative phases wOj = wj ¡w 1 , j = 2, . . . , N, which for Ii = I take the form 0=

N X j= 1

W ij KT ( wOj ¡ wOi ) ¡

N X j= 1

W1j KT (wOj ),

where wO1 ´ 0. The resulting solutions for wOj , j = 2, . . . , N, are functions of T, which can then be substituted back into equation 4.1 for i = 1 to give an implicit self-consistency condition for T. The analysis is considerably simpler in the weak-coupling regime since the relative phases are then functions of the natural period T. The third difference between weak and strong coupling is that although the equations for phase locking in the two models are formally the same, the underlying dynamical systems are distinct, thus leading to differences in their stability properties. For example, in the special case of a pair of IF neurons, a return map argument can be used to show that equation 3.16 with T replaced by the collective period T is a necessary condition for the stability of a phase-locked state for any 2 (van Vreeswijk et al., 1994). However, as we shall establish below, it is no longer a sufŽcient condition for stability in the strong coupling regime. (See also Chow, 1998.) 4.2 Desynchronization in the Strong Coupling Regime. In order to investigate the linear stability of phase-locked solutions of equation 4.1, we consider perturbations d in of the Žring times (van Vreeswijk, 1996; Gerstner et al., 1996; Bressloff & Coombes, 1999). That is, we set Tin = (n ¡ w i ) T Cd in in equation 3.2 and then integrate equation 3.1 from Tin to TinC1 using the reset condition. This leads to a mapping of the Žring times that can be expanded to Žrst order in the perturbations (Bressloff & Coombes, 1999):

8 9 < N =h i X Ii ¡ 1 C 2 Wij P T (wj ¡ w i ) d inC1 ¡ d in : ; 1 j=

= 2

N X j= 1

Wij

1 X m= ¡1

h i Gm (wj ¡ w i , T ) d jn¡m ¡d in ,

(4.3)

108

Paul C. Bressloff and S. Coombes

where PT satisŽes equation 3.8 and Gm (w , T ) =

e¡T T

Z

T 0

et J0 (t C ( m C w)T) dt.

(4.4)

The linear delay-difference equation 4.3 has solutions of the form djn = enld j with 0 · Im(l) < 2p . The eigenvalues l and eigenvectors (d 1 , . . . , d N ) satisfy the equation 2

N X

4 Ii ¡ 1 C 2 = 2

j= 1 N X j= 1

3 Wij PT (w j ¡ w i ) 5 (el ¡ 1)d i

Wij GT (wj ¡ w i , l )d j ¡ 2

N X j= 1

Wij GT (w j ¡ w i , 0)d i ,

(4.5)

with GT (w , l) =

1 X

e¡ml Gm (w , T ).

(4.6)

m= ¡1

One solution to equation 4.5 is l = 0 with d i = d for all i = 1, . . . , N. This reects the invariance of the dynamics with respect to uniform phase shifts in the Žring times, Tin ! Tin Cd . Thus the condition for linear stability of a phase-locked state is that all remaining solutions l of equation 4.5 have negative real part. This ensures that d jn ! 0 as n ! 1 and, hence, that the phase-locked solution is asymptotically stable. By performing an expansion in powers of the coupling 2 , it can be established that for sufŽciently small coupling, equation 4.5 yields a stability condition that is equivalent to the one based on the Jacobian of the phase-averaged model, equations 3.14 and 3.15. (See Bressloff & Coombes, 1999.) We wish to determine whether this stability condition breaks down as |2 | is increased (with the sign of 2 Žxed). For concreteness, we shall focus on the stability of synchronous solutions. In order to ensure that such solutions exist, we impose the condition 2 Ii = I 41 ¡ 2 KT (0)

N X

3 Wij 5 , i = 1, . . . , N

(4.7)

j= 1

for some Žxed I > 1 and T = ln[I/ ( I ¡ 1)]. The condition on Ii for the IF model plays an analogous role to equation 3.21 for the analog model in section 3.2. The synchronous state w i = w for all i and arbitrary w is then a solution of equation 4.1 with collective period T. By Žxing T we can make

Dynamics of Strongly Coupled Spiking Neurons

109

a more direct comparison with the analog model whose Žxed point at the origin will have a Žring rate f ( I) = 1/ T in equation 3.18. DeŽne the matrix b according to W b ij = Wij ¡ d i, j W

N X

Wik .

(4.8)

k= 1

For sufŽciently weak coupling, equations 3.14, 3.15, and 4.2 imply that the synchronous state is linearly stable if and only if £ ¤ (4.9) 2 KT0 (0)Re b ºp < 0, p = 1, . . . , N ¡ 1, b with b where b ºp , p = 1, . . . , N are the eigenvalues of the matrix W ºN = 0, and KT0 (h ) = T ¡1 dK T (h )/ dh . In the particular case of a symmetric pair of neurons, equation 4.9 reduces to the condition 2 K0T (0) > 0, which is equivalent to equation 3.16 for w = 0. (The case w 6 = 0 is obtained in exactly the same fashion). It follows that the stability diagram of Figure 3 displays the sign of KT0 (0) as a function of a and ta . For example, in the case of zero axonal delays (ta = 0), Figure 3 implies that K0T (0) < 0 for all Žnite a and T, and equation 4.9 reduces to the stability condition 2 Reb ºp > 0 for all p = 1, . . . , N ¡ 1. The stability of the synchronous state for weak coupling implies that the zero eigenvalue is nondegenerate and all other eigenvalues l satisfying equation 4.5 have negative real parts. As the coupling strength |2 | is increased from zero, destabilization of the synchronous state will be signaled by one or more complex conjugate pairs of eigenvalues crossing the imaginary axis in the complexl -plane from left to right. (It is simple to establish that the synchronous state cannot destabilize due to a real eigenvalue’s crossing the origin by setting l = 0 in equation 4.5.) We proceed, therefore, by substituting w i = w , i = 1, . . . , N and imposing the condition 4.7. Equation 4.5 reduces to the form £¡

N X ¢¡ ¢ ¤ el ¡ 1 I ¡ 1 C 2 C i IK0T (0) C 2 C i KT0 (0) d i = 2 GT (0, l) Wijdj ,

(4.10)

j= 1

PN 0 where C i = j= 1 Wij . We have used the identities PT (0) ¡ IKT (0) = IK T (0) and GT (0, 0) = K0T (0). We then substitute l = ib into equation 4.10 and look for the smallest value of the coupling strength, |2 | = 2 c , for which a real solution b exists. This determines the Hopf bifurcation point at which the synchronous state becomes unstable. (In the special case b = p this reduces to a period doubling bifurcation.) Note that when C i is independent of i, equation 4.10 can be simpliŽed by choosing d i to lie along one of the eigenvectors of the weight matrix W. We shall explore the nature of the Hopf instability through a number of speciŽc examples. We shall also establish that for slow synapses, the strong

110

Paul C. Bressloff and S. Coombes

coupling behavior of the IF model is consistent with that of the mean Žringrate model of section 3.2 (on an appropriately deŽned timescale). In order to compare the two types of model, it will be useful to introduce a few deŽnitions. For a network of N IF neurons labeled i = 1, . . . , N, let us deŽne ¡1 the long-term Žring rate of a neuron to be ai = D i , where D i is the mean ISI, M X 1 Dm i , M!1 2M C 1 m= ¡M

D i = lim

(4.11)

mC1 ¡T m . A necessary requirement for good agreement between with D m i = Ti i the analog and IF models is that the mean Žring rates ai of the IF model match the corresponding Žring rates of the analog model. In general, one would also expect to observe uctuations in the ISIs of the IF network that are not resolved by the analog model. A Hopf bifurcation for maps (also known as a Neimark-Sacker bifurcation) is usually associated with the formation of periodic (or quasiperiodic) orbits, which in the current context corresponds to periodic variations in the ISIs. In cases where the analog model bifurcates to a state with time-independent Žring rates, we expect the periodic orbits of the ISIs to be small relative to the Žring period, at least for slow synapses; that is, |D m i ¡ D i |/ D i ¿ 1 for all i, m. (See the example of pattern formation in section 4.5, in particular, Figure 12.) On the other hand, in cases where an analog network destabilizes to a state with time-varying Žring rates, we expect the periodic orbits of the ISIs in the IF model to be relatively large. Under such circumstances, we can introduce a sliding window of width 2P C 1 for the IF model in order to deŽne a short-term average Žring rate of the form ai ( m) = D i (m) ¡1 where

D i ( m) =

P X 1 mCp Di . 2P C 1 p= ¡P

One can then see if there is a good match between the time-dependent Žring rates of the two models for an appropriately chosen value of P. Actually, in practice it is simpler to compare variations in the Žring rate of an analog network directly with variations in the ISIs of the corresponding IF network (see Figure 8). 4.3 Oscillator Death in a Globally Coupled Inhibitory Network. As our Žrst example illustrating the Hopf instability identiŽed in section 4.2, we consider a network of N identical IF oscillators with all-to-all inhibitory coupling and no self-interactions. This type of architecture has been used, for example, to model the reticular thalamic nucleus (RTN), which is thought to act as a pacemaker for synchronous spindle oscillations observed during sleep or anesthesia (Wang & Rinzel, 1992; Golomb & Rinzel, 1994). In the

Dynamics of Strongly Coupled Spiking Neurons

111

biophysical model of RTN developed by Wang and Rinzel (1992), neural oscillations are sustained by postinhibitory rebound, a phenomenon whereby a neuron can Žre after being hyperpolarized over an extended period and then released. This should be contrasted with our simple IF model in which oscillations are sustained by an external bias. We shall show that for slow synapses, desynchronization via a Hopf bifurcation in the Žring times occurs, leading to oscillator death in the strong coupling regime, that is, certain cells suppress the activity of others. (See also the recent study of mutually inhibitory Hodgkin-Huxley neurons by White, Chow, Ritt, Soto-Trevino, & Kopell, 1998.) The weight matrix W of a globally coupled inhibitory network with 2 < 0 is given by Wii = 0 and Wij = 1/ ( N ¡ 1) so that C i = 1 for all i = 1, . . . , N. It follows that W has a nondegenerate eigenvalue ºC = 1 with corresponding eigenvector (1, 1, . . . , 1) and an ( N ¡ 1)-fold degenerate b equation 4.8, eigenvalue º¡ = ¡1/ ( N ¡ 1). The eigenvalues of the matrix W, are b ºC = 0 and b º¡ = ¡N/ (N ¡ 1), so that the synchronous state is stable in the weak coupling regime provided that 2 K0T (0) > 0 (see equation 4.9). This is certainly true for zero axonal delays (see Figure 3). Note that the underlying permutation symmetry of the system means that there are additional phase-locked states in which the neurons are divided into two or more clusters of synchronized cells (Golomb & Rinzel, 1994). We shall investigate the stability of the synchronous state as |2 | is increased. Take (d 1 , . . . , d N ) in equation 4.10 to be an eigenvector corresponding to either º = ºC or º = º¡ and set l = ib. Equating real and imaginary parts then leads to the pair of equations, 0 0 2 KT (0) C [cos(b) ¡ 1][I ¡ 1 C 2 IKT (0)] = 2 C(b)º

sin(b)[I ¡ 1 C 2 IK0T (0)] = ¡2 S(b)º,

(4.12)

where C(b) = ReGT (0, ib, ), S(b) = ¡ImGT (0, ib ). In Figure 5 we plot the solutions 2 of equation 4.12 as a function of the inverse rise time a for a pair of inhibitory neurons (N = 2) with T = ln 2 and ta = 0. The solid (dashed) solution branch corresponds to the eigenvalue º = ¡1 (º = 1). The lower branch determines the critical coupling |2 | = 2 c (a) for a Hopf instability. An important result that emerges from this Žgure is that there exists a critical value a0 of the inverse rise time beyond which a Hopf bifurcation cannot occur. That is, if a > a0 , then the synchronous state remains stable for arbitrarily large inhibitory coupling. On the other hand, for a < a0 , destabilization of the synchronous state occurs as the coupling strength 2 crosses the solid curve of Figure 5 from below. This signals the activation of the eigenmode (d 1 , d 2 ) = (1, ¡1), which suggests that an inhomogeneous state will be generated. Indeed, direct numerical simulation of the IF model shows that just after destabilization of the synchronous state (|2 | > 2 c ), the system jumps to an inhomogeneous state consisting of one active neuron and one passive neuron. We conclude

112

Paul C. Bressloff and S. Coombes

Figure 5: Region of stability for a synchronized pair of identical IF neurons with inhibitory coupling and collective period T = ln 2. The solid curve |2 | = 2 c (a) denotes the boundary of the stability region, which is obtained by solving equation 4.12 for º = ¡1 as a function of a with ta = 0. Crossing the boundary from below signals excitation of the linear mode (1, ¡1), leading to a stable state in which one neuron becomes quiescent (oscillator death). For a > a0 , the synchronous state is stable for all 2 . The dashed curve denotes the solution of equation 4.12 for º = 1.

that for sufŽciently slow synapses, a pair of IF neurons displays similar behavior to a corresponding pair of analog neurons in the strong coupling regime (see section 3.2 and Figure 4 (top)). A rough order estimate for the critical inverse rise time a0 is a0¡1 ¼ T, where T is the collective period of oscillations before destabilization. Such a result is not surprising since one would expect that for a reasonable match between the IF and (timeaveraged) analog models to occur, the IF neurons should sample incoming spike trains over a sliding window that is not too small. The width of such a sliding window is determined by the rise time a¡1 . The above result appears to be quite general. For example, suppose that we have a nonzero discrete delay ta such that the synchronous state is unstable and the antiphase state is stable for a given a. The latter state also destabilizes via a Hopf bifurcation in the Žring times for small a, leading to oscillator death. The result extends to larger values of N, for which º¡ = ¡1/ ( N¡1) in equation 4.12. Here desynchronization of a phase-locked state leads to clusters of active and passive neurons. Interestingly, the critical value a0 decreases with N, indicating the greater tendency for phase locking to occur in large, globally coupled networks. We plot 2 c as a function of a for a range of values of network size N in Figure 6, where it can be seen that a0 ( N ) ! 0 as N ! 1. This implies that for large networks, the neurons remain synchronized for arbitrarily large coupling, even for slow synapses. Indeed, since real neurons typically have an inverse rise time a of the order 2 or larger, it follows that for realistic synaptic time constants, the network can never experience oscillator death for N larger than 5 or so. (There is one subtlety to be noted here. The dashed solution curve shown

Dynamics of Strongly Coupled Spiking Neurons

113

e c

a

Figure 6: Plot of critical coupling 2 c as a function of a for various network sizes N. The critical inverse rise time a0 (N) is seen to be a decreasing function of N, with a0 (N) ! 0 as N ! 1.

in Figure 5 corresponds to excitation of the uniform mode (1, 1, . . . , 1) and is independent of N. As N increases, it is crossed by the curve 2 c (a) so that for a certain range of values of a, it is possible for the synchronous state to destabilize due to excitation of the uniform mode. The neurons in the new state will still be synchronized, but the spike trains will no longer have a simple periodic structure.) The persistence of synchrony in large networks is consistent with the mode-locking theorem obtained by Gerstner et al. (1996) in their analysis of the spike response model. We shall briey discuss their result within the context of the IF model. For the given globally coupled network, the linearized map of the Žring times, equation 4.3, becomes i © ªh I ¡ 1 C 2 IKT0 (0) d inC1 ¡ d in 2 3 1 X X 1 = 2 d jn¡m ¡ d in 5 . Gm (0, T ) 4 ¡ 1 N m= ¡1 j6 = i

(4.13)

The major assumption underlying the analysisP of Gerstner et al. (1996) is m that for large N, the mean perturbation hd m i = j6 = i d j / ( N ¡ 1) ¼ 0 for all m ¸ 0 say. Equation 4.13 then simpliŽes to the one-dimensional, Žrst-order mapping, d inC1 =

I ¡ 1 C 2 ( I ¡ 1)K0T (0) n d i ´ kTd in . I ¡ 1 C 2 IKT0 (0)

(4.14)

The synchronous (coherent) state will be stable if and only if |kT | < 1. It is instructive to establish explicitly that equation 4.14 is equivalent to the

114

Paul C. Bressloff and S. Coombes

corresponding result derived for the spike-response model (see equation 4.8 of Gerstner et al., 1996), which in our notation can be written as P 0 nC1¡l l¸1 hr ( lT )d i P d inC1 = P . (4.15) 0( ) 0( ) l¸1 hr lT C 2 l¸1 hs lT HereP h0r ( t) = dhr ( t)/ dt. First, using equations 2.7, P3.3, and 4.2, it can be shown that l¸1 h0r ( lT ) = e¡T / (1 ¡ e¡T ) = I ¡ 1 and l¸1 h0s ( lT ) = IKT0 (0). Setting A( T ) = I ¡ 1 C 2 IKT (0), we can then rewrite equation 4.15 as A( T )d inC1 = P ¡lT nC1¡l di . It follows that A(T )d inC1 ¡ e¡T A(T )d in = e¡Td in , which is l¸1 e identical to equation 4.14 since e¡T = ( I¡1)/ I. Equation 4.15 or equation 4.14 implies synchronous state is stable in the large N limit provided Pthat the 0( ) that 2 > 0, that is, 2 KT0 (0) > 0 (cf. equation 3.16). This is the h lT l¸1 s essence of the mode-locking theorem of Gerstner et al. (1996). Our analysis also highlights one of the simplifying features of the IF model with alpha function response kernels: that the summations over l in equation 4.15 can be performed explicitly. It would be of interest, however, to investigate to what extent the results of this article carry over to more general choices of hr and hs in the construction of the spike-response model. We expect that the inclusion of details concerning refractoriness will not alter the basic picture, for oscillator death is also found in a pair of mutually inhibitory HodgkinHuxley neurons, as shown by White et al. (1998) in the case of synapses with Žrst-order channel kinetics, and also conŽrmed numerically by ourselves in the case of second-order kinetics. Finally, in other related work, van Vreeswijk (1996) has shown how a network of IF neurons with global excitatory coupling can destabilize from an asynchronous state via a Hopf bifurcation in the Žring times. However, this leads to small-amplitude quasiperiodic variations in the ISIs of the oscillators (the ISIs lie on relatively small, closed orbits). This can be understood by looking at a corresponding network of excitatory analog neurons, which can bifurcate only to another homogeneous time-independent state. 4.4 Bursting in an Excitatory-Inhibitory Pair of IF Neurons. Now let us consider an excitatory-inhibitory pair of IF neurons with 2 > 0, W11 = W22 = 0 and W12 = ¡2, W21 = 1 and zero axonal delays ta = 0. The analog version of this network, studied in section 3.2, was shown to exhibit periodic variations in the mean Žring rate in the strong coupling regime. It can be established from equation 4.9 that the synchronous state is stable for sufŽciently weak coupling. In order to investigate Hopf instabilities in the strong coupling regime, we set l = ib in equation 4.10 and solve for (2 , b) as a function of a. For each a, the smallest solution 2 = 2 c determines the Hopf bifurcation point, which leads to a stability diagram of the form shown in Figure 7. In contrast to the previous example, a critical coupling for a Hopf bifurcation in the Žring times exists for all a. Direct numerical solution of the system shows that beyond the Hopf bifurcation point, the

Dynamics of Strongly Coupled Spiking Neurons

115

e

a

Figure 7: Region of stability for an excitatory-inhibitory pair of IF neurons with inhibitory self-interactions, 2 > 0 and W11 = W22 = 0, W21 = 1, W12 = ¡2. The collective period of synchronized oscillations is taken to be T = ln 2. Solid and dashed curves denote solutions of equation 4.10 for 2 with l = ib, real b. Crossing the solid boundary of the stability region from below signals destabilization of the synchronous state, leading to the formation of a periodic bursting state.

system jumps to a state in which the two neurons exhibit periodic bursting patterns (see Figure 8). This can be understood in terms of mode-locking associated with periodic variations of the ISIs on closed attracting orbits (see Figure 9). Suppose that the kth oscillator has a periodic solution of length Mk nCpMk so that D k = D nk for all integers p. If D 1k À D nk for all n = 2, . . . , M k , say, then the resulting spike train exhibits bursting with the interburst interval equal to D 1k and the number of spikes per burst equal to Mk . Although both oscillators have different interburst intervals (D 11 6 = D 12 ) and numbers of spikes per burst (M1 6 = M2 ), their spike trains have the same total period, P 1 n PM 2 n that is, M n= 1 D 1 = n= 1 D 2 . Due to the periodicity of the activity, the ISIs fall on only a number of discrete points on the orbit, which is radically different from the case found in van Vreeswijk (1996), where the whole curve is visited due to the quasiperiodical nature of the Žring. (Quasiperiodicity is also observed in the pattern formation example presented in section 4.5.) The variation of the ISIs D ni with n is compared directly with the corresponding variation of the Žring rates of the analog model in Figure 8. Good agreement can be seen between the two models in the case of small a (see Figure 8a), but discrepancies between the two arise as a increases (see Figure 8b). As in the case of oscillator death, the occurrence of bursting through a Hopf bifurcation in the Žring times appears to be quite general. For example, suppose that we have a nonzero discrete delay ta such that the synchronous state is unstable and the antiphase state is stable for a given a. We Žnd that the latter state also destabilizes via a Hopf bifurcation in the Žring times for small a, leading to a periodic bursting state.

116

Paul C. Bressloff and S. Coombes

t Figure 8: Spike train dynamics for a pair of IF neurons with both excitatory and inhibitory coupling corresponding to points A and B in Figure 7, respectively. The Žring times of the two oscillators are represented with lines of different heights (marked with a +). Smooth curves represent variation of Žring rate in the analog version of the model.

D

D

D

D

Figure 9: A plot of the ISIs (D n¡1 , D nk ) of the spike trains shown in Figure 8a, ilk lustrating how they lie on closed periodic orbits. Points on an orbit are connected by lines. Each triangular region is associated with only one of the neurons, highlighting the difference in interburst intervals (see also Figure 8a). The inset is a blowup of orbit points for one of the neurons within a burst.

Dynamics of Strongly Coupled Spiking Neurons

117

Figure 10: (a) Example of a Mexican hat function weight distribution W(k) and (b) its eigenvalue distribution º(p).

Our analysis of a pair of excitatory-inhibitory IF neurons suggests that bursting can arise at a network level due to strong synaptic coupling without the need for additional slow ionic currents (see Wang & Rinzel, 1995). An alternative mechanism for generating bursts in networks of interacting neurons is based on electrical (diffusive) coupling between cells. This has been studied in small networks of leaky-integrator neurons with postinhibitory rebound (Mulloney, Perkel, & Budelli, 1981) and in large networks of Hodgkin-Huxley neurons (Han, Kurrer, & Kuramoto, 1995). Our analysis also shows that bursting can occur in the absence of the self-interaction terms considered in our previous work (Bressloff & Coombes, 1998b); it is hard to justify the presence of such terms on neurophysiological grounds. (See section 4.6.) 4.5 Pattern Formation in a One-Dimensional Network. As our Žnal example of strong-coupling Hopf instabilities in IF networks, we consider a ring of N = 2M C 1 neurons evolving according to equations 3.1 and 3.2 with 2 > 0, ta = 0 and a weight matrix Wij = W (i ¡ j) where W ( k) = A1 exp(¡k2 / (2s12 )) ¡ A2 exp(¡k2 / (2s22 )),

0 < k · M. (4.16)

W (k) = 0 for |k| > M and W (0) = 0 (no self-interactions). We choose A1 > A2 and s1 < s2 . W ( k) then represents a lattice version of a Mexican hat interaction function in which there is competition between short-range excitation and long-range inhibition. A continuum version of W ( k) is plotted in Figure 10a for illustration. This type of architecture is well known to support spatial pattern formation in analog neural systems (Ermentrout & Cowan, 1979; Ermentrout, 1998a). We shall show that similar behavior occurs in the IF model. For the given weight matrix W and homogeneous external inputs Ii = I, i = 1, . . . , N, the network is translationally invariant. This means that one class of solution to the phase-locking equations 4.1 consists of “traveling waves” of the form w k = qk/ N for integers q = 0, . . . , N ¡ 1. This type of phase-locked solution also arises in studies of the spike-response model

118

Paul C. Bressloff and S. Coombes

(Kistler et al., 1998) and in studies of weakly coupled phase oscillators (Crook, Ermentrout, Vanier, & Bower, 1997; Bressloff & Coombes, 1997). We shall concentrate on the synchronous solution q = 0. For convenience, we shall assume that there is an equal balance between excitation and inhibition PN PM ( ) by Žxing A1 , A2 in equation 4.16 so that C i ´ j= 1 Wij = k= ¡M W k = 0 for all i. (This condition is not necessary for pattern formation to occur.) The collective period of synchronous oscillations is then T = ln[I/ (I ¡ 1)] (see equation 4.7.) In order to investigate the linear stability of the synchronous state, we need to determine the eigenvalues of the weight matrix W. These are given by º( p) = 2

M X

¡ ¢ W (k) cos pk ,

p = 0,

k= 1

2p 2p ( N ¡ 1) , ..., , N N

(4.17)

with º( p) > º(0) = 0 for all p 6 = 0. The corresponding eigenvectors are d k = eikp . Equation 4.9 then shows that the synchronous state is stable in the weak coupling regime, since KT0 (0) < 0 for all a (when ta = 0) and b º( p) = º( p). Let §pmax be the wave numbers at whichº(p) attains its maxima. (This is illustrated for the continuum Mexican hat function in Figure 10b.) Take d k = eikp and C i = 0, and set l = ib in equation 4.10. Equating real and imaginary parts leads to the pair of equations [cos(b) ¡ 1][I ¡ 1] = 2bC(b) sin(b)[I ¡ 1] = ¡b 2 S(b),

(4.18)

where b 2 = 2 ºmax and ºmax = º( pmax ). We Žnd that there exist non-trivial solutions of equation 4.18 for all values of a (see Figure 11). It follows that a Hopf bifurcation occurs due to activation of the eigenmodes d k = e§ipmax k . Since pmax 6 = 0, the instability will involve spatially periodic perturbations of the Žring times, and this leads to the formation of regular activity patterns across the network, as illustrated in Figure 12. When the network undergoes a Hopf bifurcation, it leads to the creation of periodic orbits for the ISIs. DeŽne the long-term average Žring rate ac¡1

cording to ak = D k , where D k is deŽned in equation 4.11. It follows that spatial (k-dependent) variations in the Žring rates ak can occur if there is a corresponding spatial distribution of the orbits in phase-space. Figure 12 shows that the orbits are indeed separated from each other in phase-space (although they remain small relative to the natural timescales of the system). The resulting long-term average behavior of the system is characterized by spatially regular patterns of output activity as shown in the inset of Figure 12. These patterns are found to be consistent with those observed in the corresponding analog version of the model presented in section 3.2, and such agreement holds over a wide range of values of a that includes both fast and slow synapses. However, the IF model has additional Žne structure

Dynamics of Strongly Coupled Spiking Neurons

119

20 15

^e

10

patterns

patterns

5 region of stability 0

0

1

2

3 a

4

5

6

Figure 11: Region of stability for a ring of N = 51 IF neurons having the weight distribution W(k) of equation 4.16 with s1 = 2.1, s2 = 3.5, A1 = 1.77, and P k W(k) = 0. The collective period of synchronized oscillations is taken to be T = ln 1.5. Solid and dashed curves denote solutions of equation 4.18 for b 2 . Crossing the solid boundary of the stability region from below signals destabilization of the synchronous state, leading to the formation of spatially periodic patterns of activity as shown in Figure 12.

associated with the dynamics on the periodic orbits of Figure 12, which is not resolved by the analog model. This is illustrated in the insets of Figure 13, where we plot temporal variations in the inverse ISI (or instantaneous Žring rate) of a single oscillator for two different values of a. It can be seen that there are deterministic uctuations in the mean Žring rate. In order to characterize the size of these uctuations, we deŽne a deterministic coefŽcient of variation CV ( k) for a neuron k according to r ¡ CV ( k) =

Dk ¡Dk Dk

¢2 ,

(4.19)

with averages deŽned by equation 4.12 for some sliding window of width P. The CV for a single neuron is plotted as a function of a in Figure 13. This shows that the relative size of the deterministic uctuations in the mean Žring rate is an increasing function of a. For slow synapses (a ! 0), the CV is very small, indicating an excellent match between the IF and analog models. However, the uctuations become more signiŽcant when the synapses are fast. For IF networks with a type of disordered Mexican hat interaction, instantaneous synapses, and stochastic external input, it is also known that a further component to the CV arises from the ampliŽcation of correlated uctuations (Usher, Stemmler, Koch, & Olami, 1994). Interestingly Softy and Koch (1992) have shown that stochastic input alone cannot account for the high ISI variability exhibited by cortical neurons in awake monkeys.

120

Paul C. Bressloff and S. Coombes

0.8

n D k

0.7

a k2 1 0

0

15

k 30

45

0.6 k

0.5 0.4 0.4

0.5

0.6

n-1 D k

0.7

0.8

Figure 12: Separation of the ISI orbits in phase-space for a ring of N = 51 IF corresponding to point A in Figure 11 (a = 2, 2 = 0.4). The attractor of the embedded ISI with coordinates, (D n¡1 , D nk ), is shown for all N neurons. k (Inset) Regular spatial variations in the long-term average Žring rate ak (dashed curve) are in good agreement with the corresponding activity pattern (solid curve) found in the analog version of the network constructed in section 3.2, with ak now determined by the mean Žring-rate function (see equation 3.18).

One potential application of the above analysis is to the study of an IF model of orientation selectivity in simple cells of cat visual cortex. Such a model has been investigated numerically by Somers, Nelson, and Sur (1996), who consider a ring of IF neurons with k labeling the orientation preference h = p k/ N of a given neuron. The neurons interacted via synapses with spike-evoked conductance changes described by alpha functions and with a pattern of connectivity consisting of short-range excitation and long-range inhibition analogous to the Mexican hat function of Figure 10. Numerical investigation showed how the recurrent interactions led to sharpening of orientation tuning curves. Note that the model of Somers et al. (1996) also included shunting terms and time-dependent thresholds; it should be possible to extend the analysis of our article to account for such features. 4.6 Bursting and Oscillator Death Revisited. In section 3.2 we pointed out that in the case of a Žrst-order analog model, an excitatory-inhibitory pair of neurons without self-interactions cannot undergo a Hopf bifurcation. This suggests that the bursting behavior studied in section 4.4 might

Dynamics of Strongly Coupled Spiking Neurons

121

Figure 13: Plot of the coefŽcient of variation CV for a single IF oscillator as a function of the inverse rise time a. All other parameter values are as in Figure 12. For each a, the oscillator is chosen to be one with the maximum mean Žring rate. (Insets) Time series showing variation of the inverse ISI (D n ) ¡1 with n for two particular values of a.

be sensitive to the rise time of synaptic response even in the case of slow synaptic decay. In order to explore the distinct roles played by the rate of rise and fall of synaptic interactions, we generalize the alpha function of equation 3.3 by considering the difference of exponentials J ( t) =

¢ a1 a2 ¡ ¡a1 t e ¡ e¡a2 t H (t). a2 ¡ a1

(4.20)

In the limit a2 ! a1 ! a, this reduces to the alpha function (see equation 3.3). If a1 > a2 , then the asymptotic behavior of a synapse is approximately given by e¡a2 t , and the time to peak is tp = [a1 ¡ a2 ]¡1 ln(a1 / a2 ). The roles of a1 and a2 are reversed if a2 > a1 . In Figure 14a, we plot the critical coupling for a discrete Hopf bifurcation in the Žring times for the excitatory-inhibitory pair previously analyzed in section 4.4. Now, however, the delay distribution J( t) is taken to be given by equation 4.20, with a2 = 0.2 and a1 a free parameter. Since the synaptic decay time is relatively slow, we expect there to be a good match between the IF and analog models. This is indeed found to be the case, as illustrated in Figure 14a, which shows that the critical coupling for bursting in the analog and IF models is in very close agreement. More surprising, it is seen that bursting persists even for large values of a1 , that is, for fast synaptic rise times. (Bursting still occurs even when a1 is of the order 100 or more.) One might still expect the frequency of the oscillations to approach zero

122

Paul C. Bressloff and S. Coombes

e bursting synchrony (a) a |e |

1

oscillator death

synchrony (b) a 1

Figure 14: Critical coupling for a Hopf bifurcation in the Žring times as a function of the rise time a1 for Žxed decay time a2 = 0.2. (a) Excitatory-inhibitory pair with 2 > 0, W11 = W22 = 0, W21 = 1, and W12 = ¡2. Bursting can occur even for fast rise times. The corresponding critical coupling for the analog model is shown as a dashed curve. (b) Inhibitory pair of IF neurons with 2 < 0, W11 = W22 = 0, W21 = W12 = 1. Critical coupling is only weakly dependent on a1 .

in the limit a1 ! 1. However, this is not found to be the case, which can be understood from the fact that the Hopf bifurcations are subcritical (cf. Figure 4b). Finally, in Figure 14b we plot the critical coupling for oscillator death in an inhibitory pair of IF neurons. There is only a weak dependence on a1 , which is consistent with the analog model. 5 Discussion We have presented a dynamical theory of spiking neurons that bridges the gap between weakly coupled phase oscillator models and strongly coupled Žring-rate (analog) models. The basic results of our article can be summarized as follows: 1. A fundamental mechanism for the desynchronization of IF neurons with strong synaptic coupling is a discrete Hopf bifurcation in the Žring times. This typically leads to nonphase-locked behavior characterized by periodic or quasiperiodic variations of the ISIs on closed orbits. In the case of slow synapses, the coarse-grained dynamics can

Dynamics of Strongly Coupled Spiking Neurons

123

be represented very well by a corresponding analog model in which the output of a neuron is taken to be a mean Žring rate. 2. A homogeneous network of N synchronized IF neurons with global inhibitory coupling can destabilize in the strong coupling regime to a state in which some of the neurons became inactive (oscillator death). Thus the stability criterion obtained by van Vreeswijk et al. (1994) for N = 2 is valid only in the weak coupling regime. There exists a critical value of the inverse rise time, a0 ( N ), such that synchrony persists for arbitrarily large coupling when a > a0 ( N ) (fast synapses). Moreover, a0 ( N ) ! 0 as N ! 1. This establishes that the stability condition speciŽed in the mode-locking theorem of Gerstner et al. (1996) is valid in the thermodynamic limit but no longer holds for Žnite networks with slow synapses. 3. Rhythmic bursting patterns can occur in asymmetric IF networks with a mixture of inhibitory and excitatory synaptic coupling. The ISIs evolve on periodic orbits that are large relative to the mean ISI within a burst. There is good agreement between the temporal variation of the ISIs in the IF model and the variation of the Žring rate in the corresponding analog model. 4. A network of IF neurons with long-range interactions can desynchronize to a state with a regular spatial variation in the mean Žring rate that is in good agreement with the corresponding analog model. The ISIs evolve on closed orbits that are well separated in phase-space. The Žne temporal structure of the IF model leads to deterministic uctuations of the activity patterns whose coefŽcient of variation is an increasing function of the inverse rise time a. In other words, fast synapses lead to higher levels of deterministic noise. An important question that we hope to address elsewhere concerns the extent to which the complex behavior exhibited by the IF networks studied in this article is mirrored by more biophysically detailed models of spiking neurons. Preliminary numerical studies of Hodgkin-Huxley neurons suggest a strong connection between the two classes of model. (See also White et al., 1998.) The advantage of working with the IF model is that it allows precise analytical statements to be made. Moreover, there are a number of ways of extending our analysis to take into account additional aspects of neuronal processing. The effects of diffusion along the dendritic tree of a neuron can be incorporated into the model by taking J(t ) in equation 3.2 to be the Green’s function of the corresponding cable equation. It is also possible to take into account active membrane properties under the assumption that variations of the dendritic membrane potential are small so that the channel kinetics can be linearized along the lines developed by Koch (1984). The resulting membrane impedance of the dendrites displays resonant-like behavior due to the

124

Paul C. Bressloff and S. Coombes

additional presence of inductances. This leads to a corresponding form of resonant-like synchronization. In other words, there is a strongly enhanced (reduced) tendency to synchronize for excitatory (inhibitory) coupling when the resonant frequency of the dendritic membrane approximately matches the frequency of the oscillators. Active dendrites can also desynchronize homogeneous networks of IF oscillators with strong excitatory or inhibitory coupling, leading to periodic bursting patterns (Bressloff, 1999). The response of a network to a sinusoidal external stimulus may be studied by taking Ii = A CB sin(vt) in equation 3.1, where A, B are constants. Since the resulting dynamical system is now nonautonomous, it is no longer possible to specify phase locking in terms of N ¡ 1 relative phases and a self-consistent collective period (see equation 4.1). However, one can derive conditions under which the network is entrained to the external stimulus. For example, in the case of 1:1 mode locking, the Žring times of the IF neurons are of the form Tjm = ( m ¡ w j ) T, with T = 2p / v. The absolute phases wj are determined from the set of equations 1 = (1 ¡ e¡T ) A C (1 ¡ e¡T )F(w i ) C 2

N X j= 1

Wij KT (w j ¡ w i ),

(5.1)

with KT given by equation 4.2 and F(w ) = B[sin(2p w ) Cv cos(2p w )]/ (1 Cv2 ). It is also possible to investigate the stability of such solutions by a relatively straightforward generalization of the analysis presented in section 4.2. Another interesting extension is to p: q mode locking. For example, suppose that the neurons have T-periodic Žring patterns in which the jth neuron Žres qj times over an interval T. In order to generate such solutions, it is necessary to take the Žring times to have the general form (Coombes & Bressloff, 1999): µ ¶ n ¡ w j,n mod qj , (5.2) Tjn = T qj PN where [.] denotes the integer part. This generates a set of j= 1 qj closed equations for the absolute phases w j,n mod q . One can use these equations j to determine the borders in parameter space where the ansatz 5.2 for mode locking fails. The set of such borders will deŽne an Arnold tongue diagram, with overlapping tongues indicating possible regions of multistability and chaos. Similar techniques can be used to study the bursting solutions found in section 4.4. The existence of chaotic solutions can be studied using an extension of the notion of a Lyapunov exponent that takes into account the presence of discontinuities in the dynamics due to reset (Coombes & Bressloff, 1999). Incidentally, the construction of a Lyapunov exponent also allows one to understand the spike-time reliability of an IF neuron in response to small, aperiodic signals as recently observed by Hunter, Milton, Thomas, & Cowan (1998).

¡

¢

Dynamics of Strongly Coupled Spiking Neurons

125

Another important issue concerns the effects of noise. We expect that sufŽciently small levels of noise will not alter the basic results of this article. On the other hand, the presence of noise can lead to new and interesting phenomena as illustrated by the numerical study of pattern formation by Usher et al. (1994) and Usher, Stemmler, & Olami (1995). They considered a two-dimensional network of IF neurons with short-range excitation and long-range inhibition, fast synapses, and a stochastic external input. They observed patterns of activity in the mean Žring rate that are two-dimensional versions of the patterns shown in the inset of Figure 12. It was found that in a certain parameter regime, these patterns were metastable in the presence of noise and tended to diffuse through the network. This macroscopic dynamics resulted in 1/ f power spectra and power law distributions of the ISIs on the microscopic scale of a single neuron, and led to a large coefŽcient of variation, CV > 1. We are currently carrying out a detailed study of pattern formation in two-dimensional IF networks using the methods presented in our article and hope to gain further insight into the observations of Usher et al. (1994, 1995), in particular, the interplay between noise and the underlying deterministic dynamics. One way of including noise in the IF model or the spike-response model is by introducing a probability P of Žring during an inŽnitesimal time dt: P( UId t) = t ¡1 ( U)d t. To mimic more realistic models, the response time t(U) is chosen to be large if U < h and to vanish if U À h. This is achieved by writing t (U ) = t0 e¡b ( U¡h) , where t0 is the response time at threshold and b determines the level of noise. The deterministic case is recovered in the limit b ! 1. In the case of large, homogeneous networks, one can use a meanŽeld approach to reduce the dynamics to an analytically tractable form (see Gerstner, 1995). Note that another consequence of noise is to smooth out the Žring-rate function (see equation 3.18) in the analog version of the model. A Žnal aspect we would like to highlight here is the distinction between excitable and oscillatory networks. In this article we have focused on the latter case by taking Ii > 1 in equation 3.1 so that the network may be viewed as a coupled system of oscillators with natural periods Ti = ln[Ii / (Ii ¡ 1)]. However, it is also possible for spontaneous network oscillations to occur in the excitable regime, whereby Ii < 1 for all i = 1, . . . , N. All of the basic analytical results presented in section 4 can be carried over to the excitable regime, in particular, the phase-locking equation 4.1 and the eigenvalue equation 4.5. On the other hand, the weak coupling theory of section 3.1 is no longer applicable since one requires a Žnite level of coupling to maintain oscillatory behaviour. There have also been a number of studies of waves in excitable spiking networks in both one dimension (Ermentrout & Rinzel, 1981; Ermentrout, 1998b) and two dimensions (Chu et al., 1994; Kistler et al., 1998). We are currently carrying out a detailed comparison of wave phenomena in excitable and oscillatory IF networks using some of the ideas presented in this article.

126

Paul C. Bressloff and S. Coombes

Acknowledgments We thank Jack Cowan at the University of Chicago for helpful discussions during the completion of this work. We are also grateful for the constructive comments of the referees and the suggestion to include the example in section 4.6. The research was supported by grant number GR/K86220 from the EPSRC (UK). References Abbott, L. F. (1991). Realistic synaptic inputs for model neural networks. Network, 2, 245–258. Abbott, L. F. (1994). Decoding neuronal Žring and modeling neural networks. Quart. Rev. Biophys., 27, 291–331. Abbott, L. F., & Kepler, T. B. (1990). Model neurons: From Hodgkin-Huxley to HopŽeld. In Luis Garrido (Ed.), Statistical mechanics of neural networks. Berlin: Springer-Verlag. Abbott, L. F., & van Vreeswijk, C. (1993).Asynchronous states in neural networks of pulse-coupled oscillators. Phys. Rev. E., 48, 1483–1490. Amit, D. J., & Tsodyks, M. V. (1991). Quantitative study of attractor neural networks retrieving at low spike rates I: Substrate—spikes, rates and neuronal gain. Network, 2, 259–274. Atiya, A., & Baldi, P. (1989).Oscillations and synchronization in neural networks: An exploration of the labeling hypothesis. Int. J. Neural Syst., 1, 103–124. Berry, M. J., Warland, D. K., & Meister, M. (1997). The structure and precision of retinal spike trains. Proc. Natl. Acad. Sci. USA, 94, 5411–5416. Bressloff, P. C. (1999). Resonant-like synchronization and bursting in a model of pulse-coupled neurons with active dendrites. J. Comp. Neurosci., 6, 237–249. Bressloff, P. C., & Coombes, S. (1997). Physics of the extended neuron. Int. J. Mod. Phys., 11, 2343–2392. Bressloff, P. C., & Coombes, S. (1998a). Travelling waves in chains of pulsecoupled oscillators. Phys. Rev. Lett., 80, 4185–4188. Bressloff, P. C., & Coombes, S. (1998b). Desynchronization, mode-locking and bursting in strongly coupled IF oscillators. Phys. Rev. Lett., 81, 2168–2171. Bressloff, P. C., & Coombes, S. (1999). Symmetry and phase-locking in a ring of pulse-coupled oscillators with distributed delays. Physica D, 126, 99–122. Bressloff, P. C., Coombes, S., & De Souza, B. (1997). Dynamics of a ring of pulsecoupled oscillators: A group theoretic approach. Phys. Rev. Lett., 79, 2791– 2794. Carr, C. E., & Konishi, M. (1990). A circuit for detection of interaural differences in the brain stem of the barn owl. J. Neurosci., 10, 3227–3246. Chow, C. C. (1998). Phase-locking in weakly heterogeneous neuronal networks. Physica D, 118, 343–370. Chu, P. H., Milton, J. G., & Cowan, J. D. (1994). Connectivity and the dynamics of integrate-and-Žre neural networks. Int. J. Bif. Chaos, 4, 237–243. Coombes, S., & Bressloff, P. C. (1999). Mode-locking and Arnold tongues in

Dynamics of Strongly Coupled Spiking Neurons

127

periodically forced integrate-and-Žre oscillators. Phys. Rev. E, 60, 2086–2096. Coombes, S., & Lord, G. J. (1997). Desynchronisation of pulse-coupled integrateand-Žre neurons. Phys. Rev. E, 55, 2104–2107. Crook, S. M., Ermentrout, G. B., Vanier, M. C., & Bower, J. M. (1997). The role of axonal delay in the synchronization of networks of coupled cortical oscillators. J. Comp. Neurosci., 4, 161–172. Destexhe, A., Mainen, Z. F., & Sejnowski, T. J. (1994). Synthesis of models for excitable membranes synaptic transmission and neuromodulation using a common kinetic formalism. J. Comp. Neurosci., 1, 195–231. Ermentrout, G. B. (1985). The behavior of rings of coupled oscillators. J. Mathemat. Biol., 23, 55–74. Ermentrout, G. B. (1994). Reduction of conductance-based models with slow synapses to neural nets. Neural Computation, 8, 679–695. Ermentrout, G. B. (1998a). Neural networks as spatial pattern forming systems. Rep. Prog. Phys., 61, 353–430. Ermentrout, G. B. (1998b). The analysis of synaptically generated traveling waves. J. Comp. Neurosci., 5, 191–208. Ermentrout, G. B., & Cowan, J. D. (1979). A mathematical theory of visual hallucination pattens. Biol. Cybern., 34, 137–150. Ermentrout, G. B., & Kopell, N. (1984). Frequency plateaus in a chain of weakly coupled oscillators. SIAM J. Math. Anal., 15, 215–237. Ermentrout, G. B., & Rinzel, J. (1981). Waves in a simple excitable or oscillatory, reaction-diffusion model. J. Math. Biol., 11, 1269–294. Ernst, U., Pawelzik, K., & Giesel, T. (1995). Synchronization induced by temporal delays in pulse-coupled oscillators. Phys. Rev. Lett., 74(9), 1570–1573. Gerstner, W. (1995). Time structure of the activity in neural-network models. Phys. Rev. E, 51(1), 738–758. Gerstner, W., Kreiter, A. K., Markram, H., & Herz, A .K. M. (1997). Neural codes: Firing rates and beyond. Proc. Natl. Acad. Sci. USA, 94, 12740–12741. Gerstner, W., Ritz, R., & van Hemmen, J. L. (1993). A biologically motivated and analytically soluble model of collective oscillations in the cortex I: Theory of weak locking. Biol. Cybern., 68, 363–374. Gerstner, W., & van Hemmen, J. L. (1994). Coding and information processing in neural networks. In E. Domany, J. L. van Hemmen, and K. Schulten (Eds.), Models of neural networks II (pp. 1–93). Berlin: Springer-Verlag. Gerstner, W., van Hemmen, J. L., & Cowan, J. D. (1996).What matters in neuronal locking. Neural Comput., 94, 1653–1676. Golomb, D., & Rinzel, J. (1994). Clustering in globally coupled inhibitory neurons. Physica D, 72, 259–282. Han, S. K., Kurrer, C., & Kuramoto, Y. (1995). Dephasing and bursting in coupled neural oscillators. Phys. Rev. Lett., 75, 3190–3193. Hansel, D., Mato, G., & Meunier, C. (1995). Synchrony in excitatory neural networks. Neural Comput., 7, 307–337. HopŽeld, J. J. (1984).Neurons with graded response have computational properties like those of two-state neurons. Proc. Natl. Acad. Sci. U.S.A., 81, 3088–3092. HopŽeld, J. J. (1995). Pattern recognition computation using action potential timing for stimulus representation. Nature, 376, 33–36.

128

Paul C. Bressloff and S. Coombes

Hunter, J. D., Milton, J. G., Thomas, P. J., & Cowan, J. D. (1998). A resonance effect for neural spike time reliability J. Neurophysiol., 80, 1427–1438. Keener, J. P., Hoppensteadt, F. C., and Rinzel, J. (1981). Integrate-and-Žre models of nerve membrane response to oscillatory input. SIAM J. Appl. Math., 41(3), 503–517. Kepler, T. B., Abbott, L. F., and Marder, E. (1992). Reduction of conductancebased neuron models. Biological Cybernetics, 66, 381–387. Kistler, W. M., Gerstner, W., & van Hemmen, J. L. (1997). Reduction of the Hodgkin-Huxley equations to a single-variable threshold model. Neural Comput., 9, 1015–1045. Kistler, W. M., Seitz, R., & van Hemmen, J. L. (1998). Modeling collective excitations in cortical tissue. Physica D, 114, 273–295. Koch, C. (1984). Cable theory in neurons with active, linearized membranes. Biol. Cybern., 50, 15–33. Kuramoto, Y. (1984). Chemical oscillations, waves and turbulence. Berlin: SpringerVerlag. Li, Z., & HopŽeld, J. J. (1989). Modeling the olfactory bulb and its neural oscillatory processings. Biol. Cybern., 61, 379–392. Mainen, Z., & Sejnowski, T. J. (1995). Reliability of spike timing in neocortical neurons. Science, 268, 1503–1506. Marcus, C. M., & Westervelt, R. M. (1989). Stability of analog neural networks with delay. Phys. Rev. A, 39, 347–365. Mirollo, R. E., & Strogatz, S. H. (1990). Synchronisation of pulse-coupled biological oscillators. SIAM J. Appl. Maths, 50, 1645–1662. Mulloney, B., Perkel, D. H., & Budelli, R. W. (1981). Motor-pattern production: Interaction of chemical and electrical synapses. Brain Research, 229, 25–33. Rieke, F., Warland, D., van Steveninck, R. R. D., & Bialek, W. (1996). Spikes: Exploring the neural code. Cambridge, MA: MIT Press. Softky, W. R., & Koch, C. (1992). Cortical cells should Žre regularly, but do not. Neural Computation, 4, 643–645. Somers, D. C., Nelson, S. B., & Sur, M. (1995). An emergent model of orientation selectivity in cat visual cortical simple cells. J. Neurosci., 15, 5448–5465. Strong, S. P., Koberle, R., van Steveninck, R. R. D., & Bialek, W. (1998). Entropy and information in neural spike trains. Phys. Rev. Lett., 80, 197–200. Treves, A. (1993). Mean-Želd analysis of neuronal spike dynamics. Network, 4, 259–284. Tsodyks, M., Mitkov, I., & Sompolinsky, H. (1993). Pattern of synchrony in inhomogeneous networks of oscillators with pulse interactions. Phys. Rev. Lett., 71, 1280–1283. Tuckwell, H. C. (1988). Introduction to theoreticalneurobiology (Vol. 1). Cambridge: Cambridge University Press. Usher, M., Stemmler, M., Koch, C., & Olami, Z. (1994). Network ampliŽcation of local uctuations causes high spike rate variability, fractal Žring patterns and oscillatory local Želd potentials. Neural Comput., 5, 795–835. Usher, M., Stemmler, M., & Olami, Z. (1995). Dynamic pattern formation leads to 1/ f noise in neural populations. Phys. Rev. Lett., 74, 326–329.

Dynamics of Strongly Coupled Spiking Neurons

129

van Vreeswijk, C. (1996). Partial synchronization in populations of pulsecoupled oscillators. Phys. Rev. E., 54(5), 5522–5537. van Vreeswijk, C., Abbott, L. F., & Ermentrout, G. B. (1994). When inhibition not excitation synchronizes neural Žring. J. Comp. Neurosci., 1, 313–321. Wang, X.-J., & Rinzel, J. (1992). Alternating and synchronous rhythms in reciprocally inhibitory model neurons. Neural Comput., 4, 89–97. Wang, X.-J., & Rinzel, J. (1995). Oscillatory and bursting properties of neurons. In M. Arbib (Ed.), Handbook of brain theory and neural networks (pp. 686–691). Cambridge, MA: MIT Press. White, J. A., Chow, C. C., Ritt, J., Soto-Trevino, C., & Kopell, N. (1998). Synchronization and oscillatory dynamics in heterogeneous, mutually inhibited neurons. J. Comput. Neurosci., 5, 5–16. Wilson, H. R., & Cowan, J. D. (1972). Excitatory and inhibitory interactions in localized populations of model neurons. J. Biophys., 12, 1–24. Wilson, H. R., & Cowan, J. D. (1973). A mathematical theory of the functional dynamics of cortical and thalamic nervous tissue. Kybernetik, 13, 55–80. Received August 27, 1998; accepted January 27, 1999.

NOTE

Communicated by Bard Ermentrout

On Connectedness: A Solution Based on Oscillatory Correlation DeLiang L. Wang Department of Computer and Information Science and Center for Cognitive Science, The Ohio State University, Columbus, OH 43210-1277, U.S.A.

A long-standing problem in neural computation has been the problem of connectedness, Žrst identiŽed by Minsky and Papert (1969). This problem served as the cornerstone for them to establish analytically that perceptrons are fundamentally limited in computing geometrical (topological) properties. A solution to this problem is offered by a different class of neural networks: oscillator networks. To solve the problem, the representation of oscillatory correlation is employed, whereby one pattern is represented as a synchronized block of oscillators and different patterns are represented by distinct blocks that desynchronize from each other. Oscillatory correlation emerges from LEGION (locally excitatory globally inhibitory oscillator network), whose architecture consists of local excitation and global inhibition among neural oscillators. It is further shown that these oscillator networks exhibit sensitivity to topological structure, which may lay a neurocomputational foundation for explaining the psychophysical phenomenon of topological perception. Thirty years ago Minsky and Papert (1969), in their milestone book, pointed out fundamental limitations of perceptrons. They proved that all except one topologically invariant predicates are not of Žnite order. In particular, the topological predicate of connectedness was used as the cornerstone in their mathematical analysis. To understand the implications of this famous (negative) result, let us Žrst explain what it means. A perceptron (Rosenblatt, 1962) may be viewed as a binary classiŽcation device that computes a predicate. As shown in Figure 1, the class of perceptrons that Minsky and Papert studied consists of a binary input layer R (symbolizing retina), a layer of binary feature-detecting units, and a response unit that signiŽes the classiŽcation outcome. The feature detectors sense a speciŽc area of R, and the response unit is a threshold unit operating on a weighted sum of all the units in the detector layer. The order of a predicate is the smallest number k for which one can compute the predicate with feature detectors that sense no more than k units of R. The MinskyPapert connectedness theorem states that to tell whether an input pattern is connected requires arbitrarily large orders as R grows in size. In fact, the order increases at least as fast as |R| 1/ 2 . Note that for any Žxed R, the theoNeural Computation 12, 131–139 (2000)

c 2000 Massachusetts Institute of Technology °

132

DeLiang L. Wang

Figure 1: Diagram of a perceptron. A feature detector senses a speciŽc area of the input layer, R, and the response unit computes whether the weighted sum of all the detectors is above a certain threshold.

rem does not prevent a theoretical solution to the problem by a perceptron. With a discrete R, there are a Žnite (but exponentially high except for onedimensional R)1 number of connected patterns, and one can trivially Žnd a perceptron whose feature detectors detect individual connected patterns. However, problems that are not of Žnite order require nonlocal (global) feature detectors, and too many of them to be feasible (Minsky & Papert, 1969). In a sense, their result is a negative statement on the scalability of machinery. Perceptrons are best known for their amazing ability to learn. What is powerful about the Minsky-Papert theory is that it identiŽes limitations on the kinds of patterns that perceptrons are capable of recognizing, regardless of whatever learning rule is employed. Much of the Minsky-Papert analysis is about single-layer (simple) perceptrons, and the following critical question arises: Is the limitation applicable to multilayer perceptrons with backpropagation learning, which represents the most inuential development of modern neural networks? General analytical statements are not available. As far as connectedness is concerned, Minsky and Papert (1988, p. 52) claimed that multilayer perceptrons are no more powerful. For multilayer perceptrons, order is of less concern because hidden units in typical architecture receive input from all of the input units. The central questions are, How many hidden units are required? and How long is the training process? If the number or the duration increase very quickly as R grows in size, one must consider the problem not solvable in practice. For twodimensional (2D) R, the number of connected patterns grows exponentially with respect to |R| . Thus, for R not too small, a practical training process

1

I am grateful to N. Robertson for discussions that led to this conclusion.

On Connectedness

133

can employ only a tiny percentage of all possible training samples, and it is hard to expect that a multilayer perceptron can successfully generalize from relatively so few training samples. Given this consideration and that no success has been reported on recognizing connectedness, it is reasonable to project, based on the result on simple perceptrons, that the limitation exists for multilayer perceptrons (see also Minsky & Papert, 1988). Here we point out that a new class of neural networks—neural oscillator networks—provides a solution to the problem of connectedness. The basic unit of an oscillator network is a neural oscillator, and with an additional degree of freedom—the phase of an oscillator—one can speak of synchronization and desynchronization in an oscillator network. In particular, Terman and Wang (1995; Wang & Terman, 1997) proposed and analyzed LEGION (locally excitatory globally inhibitory oscillator network). Representation in these networks is based on oscillatory correlation, a special form of von der Malsburg’s temporal correlation theory (von der Malsburg, 1981; von der Malsburg & Schneider, 1986). More speciŽcally, a pattern is represented by a synchronized group of oscillators, and different patterns are represented by distinct oscillator groups that desynchronize from each other. A LEGION network consists of three parts: (1) its basic unit, a relaxation oscillator with two time scales; (2) oscillators coupled with local excitation, which leads to rapid synchronization within a group corresponding to one pattern; and (3) a global inhibitor, which desynchronizes different groups of oscillators. Formally, a single oscillator i in LEGION is deŽned by a pair of excitatory variable xi and inhibitory variable yi . Oscillator i receives external stimulation Ii , and overall coupling Si from the network. The deŽnition of oscillator i includes a relaxation parameter e, which induces two time scales that characterize typical relaxation oscillations. The limit cycle for a relaxation oscillation quickly alternates (or jumps) between an active phase of relatively high x values and a silent phase of relatively low x values. To study connectedness, we employ a 2D LEGION with four nearest-neighbor coupling and the global inhibitor z. (See Terman & Wang, 1995, and Wang & Terman, 1997, for precise deŽnitions of LEGION.) Given Terman-Wang analysis on LEGION, it is not difŽcult to use LEGION to detect connectedness. Before describing how to do it, we show the response of a 30£30 LEGION network to two Žgures: one connected and one disconnected. The connected Žgure is a cup shown in Figure 2A, while the disconnected one is the image of the word cup shown in Figure 2B. The LEGION network is solved using a Runge-Kutta method. The oscillators of the network start with random phases. Figure 2C displays the temporal activity of all the stimulated oscillators (Ii > 0) for the connected cup image. Unstimulated oscillators (Ii < 0) are omitted from the display because they do not oscillate. The oscillators corresponding to each pattern are combined in the display and thus appear as a single oscillator when they are in synchrony. The upper panel shows the oscillator block corresponding to the cup, and the middle panel shows the activity of the global inhibitor. Syn-

134

DeLiang L. Wang

chrony occurs in the Žrst cycle of oscillations. The case for the disconnected cup is shown in Figure 2D, where the upper three traces show the three blocks corresponding to the three patterns, respectively. From Figure 2D, it is clear that synchronization within each block and desynchronization between different blocks are achieved in the Žrst two cycles. These simulations illustrate clearly that after a few cycles, all the oscillators corresponding to one (connected) pattern are synchronized, whereas those corresponding to different patterns are desynchronized. When a synchronized block jumps to the active phase, the global inhibitor is triggered and approaches from 0 to 1 on the fast timescale. Within an oscillation period, the global inhibitor is triggered as many times as is the number of patterns in the input Žgure. Thus, one can Žnd how many patterns are in the input image by comparing the response of any stimulated oscillator and that of the global inhibitor. If their frequencies of oscillations are the same, then the input Žgure contains one pattern, and thus the Žgure is connected. Otherwise the input Žgure contains more than one pattern and the Žgure is disconnected. More speciŽcally, we compute the accumulated activity of RT the global inhibitor over an oscillation period t by T¡t z dt, where T denotes the current time. The corresponding average accumulated activity of a stimulated oscillator is given by PRT ( ¡ hx ) dt T¡t H xi i P . H ( Ii ) i

Here H stands for the Heaviside step function, and h x is chosen between x values that correspond to the silent phase and those to the active phase. Note that the denominator indicates how many oscillators are stimulated. Regarding the period t of a stimulated oscillator, in the singular limit e ! 0, it can be precisely expressed in terms of the parameters of a single Figure 2: Facing page. (A) A connected cup image as mapped to a 30£30 LEGION. (B) A disconnected image with three patterns forming the word cup as mapped to a 30£30 network. (C) Results for the connected cup image. (D) Results for the disconnected cup image. In C and D, the upper traces give the combined x activities of the oscillator blocks indicated by their respective labels. The nextto-bottom trace gives the activity of the global inhibitor, and the bottom one shows the temporal activity of the RHS of equation 1, together with h . The oscillator activity is normalized in the Žgure. The simulation in each case takes 7500 integration steps. The parameter values are (see Wang & Terman, 1997, for the meanings of these parameters): e = 0.02, b = 0.1, c = 6.0, r = 0.02, hx = ¡0.5, h z = 0.1, w = 3.0, Wz = 1.0, and WT = 8.0 (weights are identical before dynamic normalization). I = 0.2 for a stimulated oscillator and I = ¡0.02 otherwise.

On Connectedness

135

136

DeLiang L. Wang

oscillator (see equation 10 in Linsay & Wang, 1998). When e > 0 as in the previous simulations, the calculated period is not precise. But as will be clear later, we do not need precise values of t in order to detect connectedness. To summarize, the connectedness predicate is calculated by 0 1 , P R T H (x ¡ h ) dt Z T i x T¡t B i C P (1) z dt @ A < h. ( ) H Ii T¡t i

The threshold h should be chosen between 1 and 2 for the following reason. Given a positive e and system noise, synchrony within each pattern is not perfect (see Figure 2). Since z is triggered as long as a single oscillator jumps to the active phase, the width of the active phase of the inhibitor is slightly wider than a stimulated oscillator. Thus, for a connected Žgure, the righthand side (RHS) of equation 1 is slightly greater than 1 but certainly less than 2. The two bottom traces in Figures 2C and 2D show the RHS value of equation 1 for these two cases. For the parameter values used in the simulations, t = 5.27. A threshold h = 1.6 is used in Figure 2, and this is sufŽcient to detect connectedness. Beyond a short beginning duration corresponding to the process of synchronization and desynchronization, equation 1 correctly reveals the connectedness predicate. More generally, the RHS of equation 1 reveals the number of connected components in the input Žgure. With this formula, the LEGION network can be used to count how many patterns are in a picture. Note that equation 1 does not consider the degenerate case of no stimulus. What if there are numerous patterns in a picture? As analyzed in Wang and Terman (1997), for a Žxed set of parameters, the LEGION network exhibits a Žxed capacity in segmentation. If the number of patterns in an input picture exceeds the capacity, the network separates the picture into as many segments as the capacity. This observation, together with the result on the speed of synchronization and desynchronization (Terman & Wang, 1995; Wang & Terman, 1997), suggests that no matter how numerous patterns are in the input, one needs to wait for at most as many cycles as the capacity before connectedness can be correctly detected. With any capacity greater than 1, the connectedness predicate of equation 1 is not affected when numerous patterns appear in the input, because it is an assertion of whether the Žgure contains just one pattern. Thus, we obtain an upper bound on how long the system takes to compute the connectedness predicate. We emphasize that this detection of connectedness is analytically established, regardless of speciŽc shapes, sizes, or arrangements of patterns in a picture. Connectedness is not only computationally interesting, but also of fundamental signiŽcance to perception. It is one of the basic perceptual grouping principles (Rock & Palmer, 1990). According to Palmer and Rock (1994),

On Connectedness

137

Figure 3: Three pairs of stimuli used in Chen’s experiments (adapted from Chen, 1982).

connectedness accounts for the Žrst-level organization. Furthermore, based on a series of psychophysical experiments Chen (1982, 1989) observed that the human visual system is sensitive to topological properties of stimuli, and topological perception constitutes a basic and early part of perceptual organization. Figure 3 illustrates a typical experiment with three pairs of stimuli, which manifest topological perception. Chen found that humans are more accurate in discriminating the annulus-disk pair under rapid display conditions. Note that the pairs in Figures 3A and 3B are topologically equivalent, whereas the annulus-disk pair is topologically distinct. Based on the difŽculty of neural networks on connectedness, he challenged that topological perception remains a mystery for current neural network models (Chen, 1989). Given that all six patterns in Figure 3 are connected, connectedness alone cannot explain the experimental result. However, we observe that although every pattern has an open “outside,” only the annulus has an inside hole. With this observation, we can assume that a hole is a perceptually distinct pattern. Thus, the annulus consists of two connected patterns (or three, if one considers the open outside as a separate region), whereas the other Žve stimuli contain one pattern. When the stimuli in Figure 3 are presented to a LEGION network, all of them result in one segment, except for the annu-

138

DeLiang L. Wang

lus that results in two segments. 2 The qualitative difference in the number of segmented patterns emerging from LEGION provides a foundation for explaining topological perception. Our explanation applies equally well to cases involving more than one hole (Chen, 1990). The fact that LEGION exhibits a Žxed segmentation capacity results in the following novel and testable prediction: topology-based discrimination occurs only up to a certain number of holes. This prediction differs from that of Chen’s (1990) topological hypothesis, rooted in mathematical topology. To conclude, we have found a solution to the problem of detecting connectedness. The solution is given by a class of neural oscillator networks, which differ fundamentally from layered perceptrons. These networks are a kind of recurrent neural network and a dynamical system due to their continuous-time deŽnitions. As such, oscillators in a network are effectively of unbounded order, which provides a necessary means to address the problem of connectedness. Furthermore, with their ability to exhibit sensitivity to topological structure, these oscillator networks provide a foundation for explaining psychophysical data of topological perception, and therefore have answered another challenge to neural networks. With their success in solving these difŽcult problems as well as their unique capability in scene analysis (Wang & Terman, 1997) and biological plausibility (Singer & Gray, 1995), oscillator networks may suggest a promising direction for future research in neural computation. Acknowledgments Thanks to B. Chandrasekaran, J. Josephson, and R. Lewis for useful discussions and reading an earlier draft of this article. This work was supported in part by an NSF grant (IRI-9423312) and an ONR Young Investigator Award (N00014-96-1-0676). References Chen, L. (1982). Topological structure in visual perception. Science, 218, 699–700. Chen, L. (1989). Topological perception: A challenge to computational approaches to vision. In R. Pfeifer, Z. Schreter, F. Fogelman-Soulie, & L. Steels (Eds.), Connectionism in perspective (pp. 317–329). Amsterdam: Elsevier. Chen, L. (1990). Holes and wholes: A reply to Rubin and Kanwisher. Percept. Psychophys., 47, 47–53. Linsay, P. S., & Wang, D. L. (1998). Fast numerical integration of relaxation oscillator networks based on singular limit solutions. IEEE Trans. Neural Net., 9, 523–532. 2 For simulations involving holes, as in Figure 3C, oscillators corresponding to holes are also stimulated and excitatory connections are formed only between neighbors with the same pixel levels (black or white).

On Connectedness

139

Minsky, M. L., & Papert, S. A. (1969). Perceptrons. Cambridge, MA: MIT Press. Minsky, M. L., & Papert, S. A. (1988). Perceptrons (Exp. ed.). Cambridge, MA: MIT Press. Palmer, S., & Rock, I. (1994). Rethinking perceptual organization: The role of uniform connectedness. Psychol. Bull. Rev., 1, 29–55. Rock, I., & Palmer, S. (1990). The legacy of Gestalt psychology. Sci. Am., 263, 84–90. Rosenblatt, F. (1962). Principles of neural dynamics. New York: Spartan. Singer, W., & Gray, C. M. (1995). Visual feature integration and the temporal correlation hypothesis. Ann. Rev. Neurosci., 18, 555–586. Terman, D., & Wang, D. L. (1995). Global competition and local cooperation in a network of neural oscillators. Physica D, 81, 148–176. von der Malsburg, C. (1981). The correlation theory of brain function (Internal Rep. 81-2). Gottingen: ¨ Max-Planck-Institute for Biophysical Chemistry. von der Malsburg, C., & Schneider, W. (1986). A neural cocktail-party processor. Biol. Cybern., 54, 29–40. Wang, D. L., & Terman, D. (1997). Image segmentation based on oscillatory correlation. Neural Comp., 9, 805–836. (For errata see Neural Comp., 9, 1623– 1626, 1997.)

Received June 24, 1998; accepted January 22, 1999.

Communicated by Peter Dayan

NOTE

Practical IdentiŽability of Finite Mixtures of Multivariate Bernoulli Distributions ´ Carreira-Perpi n´ Miguel A. ˜ an Steve Renals Department of Computer Science, University of ShefŽeld, ShefŽeld S1 4DP, U.K.

The class of Žnite mixtures of multivariate Bernoulli distributions is known to be nonidentiŽable; that is, different values of the mixture parameters can correspond to exactly the same probability distribution. In principle, this would mean that sample estimates using this model would give rise to different interpretations. We give empirical support to the fact that estimation of this class of mixtures can still produce meaningful results in practice, thus lessening the importance of the identiŽability problem. We also show that the expectation-maximization algorithm is guaranteed to converge to a proper maximum likelihood estimate, owing to a property of the log-likelihood surface. Experiments with synthetic data sets show that an original generating distribution can be estimated from a sample. Experiments with an electropalatography data set show important structure in the data. 1 Introduction Finite mixtures of multivariate Bernoulli distributions have been extensively used in diverse Želds (such as bacterial taxonomy) to model a population of binary, multivariate measurements in terms of a few latent classes (see Everitt & Hand, 1981; Gyllenberg, Koski, Reilink, & Verlaan, 1994 and references therein). However, it has been recently proved that this class of mixture models is nonidentiŽable (Gyllenberg et al., 1994), which potentially undermines the interpretation of sample estimates, since the same sample could be equally attributed to a number of different estimates. We consider a Žnite mixture distribution (Everitt & Hand, 1981) deŽned on the D-dimensional binary space f0, 1gD : p(t) =

M X

p m p(t| m),

m= 1

where M is the number of components, t = ( t1 , . . . , tD )T is a binary Ddimensional vector, the p m = p( m) are the mixture proportions, and p(t| m) is a multivariate Bernoulli distribution with parameters (called prototypes) Neural Computation 12, 141–152 (2000)

c 2000 Massachusetts Institute of Technology °

´ Carreira-Perpin´ Miguel A. ˜ an and Steve Renals

142

pm = ( pm1 , . . . , pmD ) T , that is, p(t| m) =

D Y d= 1

³ ´1¡td td 1 ¡ pmd . pmd

These parameters must obey the following constraints: M X

p m = 1I

m= 1

pmd 2 [0, 1]

p m 2 (0, 1)

8m = 1, . . . , MI

8m = 1, . . . , M, d = 1, . . . , D.

It has been proved (Gyllenberg et al., 1994) that this class of mixtures is nontrivially nonidentiŽable1 for all dimensions D. This means that there are M g many combinations of values of the parameter tuple Q = fM, fp m , pm gm= 1 that produce an identical distribution p(t): there exist at least two tuples Q , 0 0 Q for which p(t| Q ) = p(t| Q ) 8t 2 f0, 1gD . For example, it can be readily veriŽed that the four mixtures given by the following parameter tuples represent the same distribution (here D = 3): n n ¡ ¢T oo Q : p 1 = 1, p 1 = 12 12 12 M = 1, n n n ¡ ¢T o ¡ ¢T oo Q 0: p 1 = 12 , p1 = 12 0 12 , p 2 = 12 , p2 = 12 1 12 M = 2, n n n ¡ ¢T o ¡ ¢T oo Q 00 : p 1 = 14 , p1 = 12 0 12 , p 2 = 34 , p2 = 12 23 12 M = 2, n n n ¡ ¢T o ¡ ¢T oo Q 000 : p 1 = 14 , p1 = 1 12 12 , p 2 = 34 , p2 = 13 12 12 . M = 2, However, it does not mean that for every parameter tuple Q there must exist at least one different Q 0 representing the same distribution. IdentiŽability is a property of the class of mixtures rather than of a particular parameter tuple. Hence, in principle there are many tuples in parameter space that are completely equivalent but would give rise to different interpretations. This may seem an insurmountable difŽculty for parameter estimation, but our practical studies have produced promising results. We will show that, given a sample from a mixture of multivariate Bernoulli distributions, maximum likelihood estimates of the parameters, obtained by an expectation-maximization (EM) algorithm, can still be interpretable. Before giving some experimental results to support this claim, we give some properties of the log-likelihood surface of mixtures of multivariate Bernoulli distributions and introduce an EM algorithm for them. 1

As opposed to trivial nonidentiŽability, which is given by permutations of the mixture components or by coincident component distributions p(t| m) for several components.

Practical IdentiŽability of Finite Bernoulli Mixtures

143

2 Maximum Likelihood Parameter Estimation We assume a Žxed number of components M. Maximum likelihood estimation can be achieved by an EM algorithm. DeŽne ¼ = (p 1 , . . . , p M ) T and P = (p1 , . . . , pM ). The log-likelihood of the parameters f¼ , Pg given a sample ftn gN n= 1 is l( ¼ , P) =

N X n= 1

log p(tn I ¼ , P) =

N X

log

n= 1

(

M X

pm

m= 1

D Y d= 1

!

nd (1 ¡ ptmd

pmd ) 1¡tnd

(2.1)

and its gradient is easily seen to be N 1 X @l = p( m|tn I ¼ , P) ¡ N p m n= 1 @p m

m = 1, . . . , M

(2.2)

N X 1 @l = p( m| tn I ¼, P)(tnd ¡ pmd ) @pmd pmd (1 ¡ pmd ) n= 1

m = 1, . . . , M

d = 1, . . . , D,

(2.3)

where p(tn |mI ¼, P)p(m) p(m|tn I ¼ , P) = P M 0 0 m0 = 1 p(tn |m I ¼ , P)p(m ) Q tnd (1 ¡ pmd ) 1¡tnd pm D d= 1 pmd = PM QD tnd 1¡tnd 0 m0 = 1 p m d= 1 pm0 d (1 ¡ pm0 d )

(2.4)

is the posterior probability (or responsibility) that component m generated data point tn . The term ¡N in equation 2.2 results from the constraint PM m= 1 p m = 1 introduced in the log-likelihood via a Lagrange multiplier. Derivation of the EM algorithm for Žnite mixtures of multivariate Bernoulli distributions is straightforward and can be found elsewhere (Everitt & Hand, 1981; Wolfe, 1970). Here we give the basic equations: • E-step: Computation of the responsibilities using equation 2.4 from the current parameter estimates f¼(t ) , P(t ) g at iteration t, p( m|tn I ¼ (t ) , P(t ) ). • M-step: Reestimation of f¼ (t C1) , P(t C1) g: p m(t C1) = C1) p (t = m

N 1 X p( m| tn I ¼(t ) , P(t ) ) N n= 1

1

N X

( m|tn I ¼ (t ) , P (t ) )tn .

p (t C1) Np m n= 1

(2.5)

´ Carreira-Perpin´ Miguel A. ˜ an and Steve Renals

144

The sequence of parameters obtained for t = 0, 1, 2, . . . by iterating between the E- and M-steps from any starting point f¼ (0) , P(0) g produces a monotonically increasing sequence of values for the log-likelihood. A common problem of estimation in mixture distributions is that of singularities, that is, points in parameter space whose log-likelihood tends to positive inŽnity (e.g., a mixture of gaussians in which one of the components is located on a data point and its variance tends to zero, thereby becoming a Dirac delta). Such singularities are undesirable because they give rise to degenerate distributions. Fortunately, the log-likelihood surface of a Žnite mixture of multivariate Bernoulli distributions has no singularities of value C1 (although it does have singularities of value ¡1). The reason is that both the log-likelihood (see equation 2.1) and its gradient (see equations 2.2 and 2.3) are bounded above in the whole parameter space, including its boundaries. 2 This means that estimation by the above EM algorithm from any nonpathological starting point, which is always possible by choosing pmd in (0, 1), will always lead to a proper stationary point of the log-likelihood. 3 Stationary Points of the Log-Likelihood We note that the second derivatives of the log-likelihood with respect to the mixing proportions are always negative (for clarity, we omit the dependence on the parameters): N 1 X @2 l = ¡ p( m| tn ) p(m0 |tn ) · 0. p mp m0 n= 1 @p m @p m0

Therefore the Hessian of the log-likelihood has negative numbers in its diagonal, and it cannot be positive deŽnite. Hence, no stationary point of the log-likelihood is a minimum. Note that this is a general property of Žnite mixtures. At any stationary point of the log-likelihood, equations 2.5 hold, so that we have M X p m Ep(t| m) ftg Ep(t) ftg = m= 1

=

M X m= 1

2

p m pm =

M N N X 1 X 1 X tn = Nt, p( m|tn )tn = N N m= 1 n= 1 n= 1

When pmd ! 0 for some m, d, the log-likelihood gradient in equation 2.3 remains bounded above, because for each n, either tnd ¡ pmd ! k1 pmd (if tnd = 0) or p(m|tn ) / p(tn |m) ! k2 pmd (if tnd = 1), where k1 and k2 are constants. The same is true in the case pmd ! 1. Therefore the log-likelihood is differentiable for pmd 2 [0, 1], except in pathological situations where |pmd ¡t nd | = 1 for all m and Žxed n. In these cases p(tn | m) = 0 ) ! ¡1. Since the EM algorithm always climbs the logfor all m and l(fp m , pm gM m= 1 likelihood surface, it will not be attracted by such singularities.

Practical IdentiŽability of Finite Bernoulli Mixtures

145

and so the mean of the mixture coincides with the sample mean at any stationary point. The converse does not hold generally. The point in parameter space where pm = Nt for all m = 1, . . . , M and any distribution of the mixing proportions p m is a stationary point of the QD tnd N N (1¡tnd ) is independent of log-likelihood, because p(tn | m) = d= 1 td (1 ¡ td ) m and therefore p(m|tn ) = p m and the gradient is zero there. This point is equivalent to a single multivariate Bernoulli distribution and should be avoided because experience shows that its log-likelihood, D ³ ´ ³ ´1¡tNd Y tNd M N N = log 1 ¡ , l fp m , pm = Ntgm= N t t 1 d d d= 1

is much smaller than that of other local maxima (an intuitive fact since the mixture is trivial). Observe that a starting point of the EM algorithm in which pm is the same for all components (e.g., the apparently innocuous starting point pmd = 1/ 2 for all m and d) will lead to the mentioned trivial mixture after one EM iteration for any original distribution of the p m . Our experiments showed that random starting points in (0, 1) were less prone to leading to trivial mixtures. 4 Experimental Results 4.1 Synthetic Data. We generated N = 10,000 vectors in a binary space of D = 16 dimensions from a Žxed mixture of M = 8 16-variate Bernoulli distributions, whose parameters are shown in Figure 1. We call these the original parameters and denote them with an “o” superscript (e.g., p om ). From the sample alone, 10 maximum likelihood estimates were found for mixtures of M = 4, M = 8, and M = 10 components. We used the above EM algorithm with random starting values of the parameters pmd in the range [ 14 , 34 ], stopping it when the relative change in log-likelihood was smaller than 10¡6 . The starting values for the p m parameters were Žxed to 1/ M, thus giving each component the same weight at the beginning. The results were as follows:3 • Using the original number of components (M = 8), EM found the original parameters (both pm and p m ) 9 out of 10 times; the normalized distance between the original and the estimated parameters was smaller than 0.0013 in those cases, and the log-likelihood was ¡94,990 (to four signiŽcant digits). However, the remaining estimate was a suboptimal maximum of the log-likelihood in which two of the compo3

We quantify the distance between two vectors p, q in the D-dimensional rectangle [0, 1]D with the normalized undirected distance D1 kp ¡ qk22 , where k ¢ k2 is the Euclidean norm. This distance is a real number in [0, 1] that averages to 1/ 6 for two uniformly random vectors. Observe that if pd = qd C2 for d = 1, . . . , D, then D1 kp¡qk22 = |2 | independent of D.

146

´ Carreira-Perpin´ Miguel A. ˜ an and Steve Renals

Figure 1: Parameters of the mixture of M = 8 16-variate Bernoulli distributions used to generate the sample in section 4.1. For m = 1, . . . , 8, each 16-dimensional pm vector is represented as a 4 £ 4 image in which the area of each pixel is proportional to the value of its associated pmd parameter; for example, for the leftmost image, p1d is 0.8 for d · 4 (row 1) and 0.2 for d ¸ 5 (rows 2–4).

nents had the same pm parameter (p5 ¼ p8 ¼ p o5) and a log-likelihood of ¡95,201. The difference between the log-likelihood values of both kinds of estimates was 0.2%. • Using fewer components than necessary (M = 4) produced an estimate in which each prototype pm was approximately either one of the original prototypes or a linear combination of several of the original prototypes (normalized distance smaller than 0.0023). Figure 2 shows the situation, for a particular estimate in which pm ¼ pom for m = 1, 2, 3 and p 4 ¼ 23 po4 C 13 po5 . As in the previous case, estimates having different prototypes also had very close log-likelihood values (differing by 0.5%).

Figure 2: Parameters of a mixture of M = 4 16-variate Bernoulli distributions estimated by maximum likelihood from the sample. Observe that p1 , p2 , p3 , and p4 coincide very closely with po1 , po2 , po3 , and 23 po4 C 13 po5 , respectively.

Practical IdentiŽability of Finite Bernoulli Mixtures

147

• Using more components than necessary (M = 10) always produced the eight original pom vectors (normalized distance smaller than 0.0022) plus two extra ones, typically either repeated instances of some of the original ones or linear combinations of them. The log-likelihood value of each estimate did not differ from any of the others in more than 0.02%, indicating that once the eight original prototypes are found, the remaining ones are largely irrelevant and reect peculiarities of the sample used. Experiments performed with other synthetic data sets produced the same results. We propose the following interpretation of the experimental facts. Given a large sample generated from a known mixture of multivariate Bernoulli distributions, let us construct another mixture in this way. First, pick up freely the number of components M; then choose its p1 , . . . , p M prototypes either as some of the original ones or as linear combinations of them. We claim that for certain values of the mixing proportions, such a point in parameter space is very close to a maximum of the log-likelihood surface. However, we do not have theoretical support for this and do not have a valid interpretation for the values of the mixing proportions. These results also suggest a procedure to follow when estimating an unknown mixture of multivariate Bernoulli distributions from a sample. Choose freely a number of components M, and, using EM from random starting points, Žnd several (say, 10) maximum likelihood estimates for it. Inspect the prototypes obtained. If they look the same for every estimate, then M is probably the right number of components, and the estimate is very close to the true generating distribution. If a Žxed group of prototypes appears in each estimate, and the rest of the prototypes are repetitions of those in the group, then M is probably too big; reduce it and start again. However, if there are different prototypes in different estimates, then M is probably too small; increase it and start again. 4.2 EPG Data. The technique of electropalatography (Hardcastle, Jones, Knight, Trudgeon, & Calder, 1989) records the presence or absence of contact between the tongue and the hard palate in a number of Žxed locations of the latter and at Žxed intervals during continuous speech. The result is a stream of two-dimensional binary patterns, or electropalatograms (EPGs), which can be used in speech therapy and assessment. We used a subset of electropalatography data from the EUR-ACCOR database (Marchal & Hardcastle, 1993) containing 11,852 different 62-dimensional vectors (EPGs), obtained from 12 different utterances by a native English speaker. We estimated the density of its distribution in 62-dimensional space using a Žnite mixture of multivariate Bernoulli distributions with M = 6 components. A number of estimates were found with the EM algorithm. As with the synthetic data set, the starting parameter £ ¤ values were 1/ M for the p m parameters and a random number in 14 , 34 for the parameters pmd , and EM

148

´ Carreira-Perpin´ Miguel A. ˜ an and Steve Renals

Figure 3: The nine prototypes p1 , . . . , p9 for a mixture of M = 9 multivariate Bernoulli distributions trained with the EPG data set. Each pm vector, consisting of D = 62 values in the range [0, 1], is customarily displayed as an image resembling the palate. The top row (alveolar part) contains parameters pm1 to pm6 from left to right, row 2 contains pm7 to pm14 , and so on until the bottom row (velar part), always from left to right. The area of each pixel is proportional to the value of its associated pmd parameter.

was stopped when the relative change in log-likelihood was smaller than 10¡6 . Examination of the parameter values at this point showed that: • Several different kinds of estimates were found, each characterized by a subset of the prototypes shown in Figure 3 and by speciŽc values for the mixing proportions. Prototypes p1 , p2 , p5 , and p9 appeared in almost every estimate (the normalized distance between corresponding prototypes did not exceed 0.02), and p3 and p8 were very common too. • The log-likelihood of these mixtures was quite close, varying from ¡138,341 for the combination fp 1 , p2 , p5 , p 7 , p8 , p9 g to ¡141,697 for the combination fp1 , p2 , p3 , p5 , p6 , p9 g. • Occasionally a prototype was found that could be expressed as a linear combination of some of the prototypes of Figure 3, for example, 12 p8 C 1 2 p9 . We also found estimates for mixtures of M components, where M varied from 1 to 15. This made apparent the fact that for M 0 9, some prototypes appeared several times (with slight differences) in the same mixture, which therefore becomes trivial. This suggested selecting M = 9 as the optimum

Practical IdentiŽability of Finite Bernoulli Mixtures

149

Figure 4: Log-likelihood of the mixture of multivariate Bernoulli distributions model for the EPG data as a function of the number of components M. Each value plotted corresponds to an average over 10 independent estimates.

number of components for this data set, 4 with typical values for the optimal parameters (both pm and p m ) given in Figure 3. Considering a sequence of mixture estimates starting from M = 1 to M = 15, prototypes with high mixture proportions tended to appear early in the sequence, although at times somewhat distorted due to the interference with other prototypes. For example, prototype p1 in Figure 3 was present in all mixtures, while p6 starts to appear only (very unfrequently) for M ¸ 6. However, prototype p9 appears very frequently for M ¸ 4. Also, the log-likelihood for the data set considered increases with M and reaches a plateau for M ¼ 9 (see Figure 4). An interesting fact is that these prototypes are highly interpretable, corresponding to physically feasible EPGs and in fact assimilable to well-known quasi-static patterns in EPG studies (e.g., velar, alveolar; Hardcastle et al., 1989), thus revealing important structure patterns in the data. These prototypes are similar to those produced by other methods, in particular latent variable models (Carreira-Perpin´ ˜ an & Renals, 1998). 4

An alternative way to select the critical M is to examine the log-likelihood curves for a validation set.

´ Carreira-Perpin´ Miguel A. ˜ an and Steve Renals

150

5 Conclusion We have shown that for the class of Žnite mixtures of multivariate Bernoulli distributions, the EM algorithm always converges to a proper stationary point of the log-likelihood, provided it is not started from a pathological point in parameter space (which is always possible). The reasons are the absence of singularities of value C1 in the log-likelihood surface and the fact that the EM algorithm always climbs that surface. We have given empirical evidence that for this particular class of Žnite mixture models, sensible and interpretable maximum likelihood estimates can be found even though that class of mixtures is not identiŽable. This would suggest that the lack of identiŽability might not be important from the practical point of view in some cases. Let us analyze the possible reasons with some detail. First, we note that the nonidentiŽability result is not surprising if one considers that a distribution deŽned over a discrete domain can be speciŽed by a Žnite number of equations, one for each point of the domain. In our case, this means 2D ¡ 1 equations (plus another one linearly dependent with them due to all the equations adding to one). Since the mixture has MD C M ¡ 1 free parameters (one of the mixing proportions being linearly 2D dependent on the others), making M larger than DC1 would yield an underdetermined system of equations with multiple solutions. However, even for small dimensions D, this number is extremely large, and the number of components employed in practice will be much smaller. We restrict the rest of this discussion to a situation where the maximum number of components 2D is much smaller than DC1 , which yields an overdetermined system (where multiple solutions can still exist). One reason for the apparent sparseness of the nonidentiŽability effect may be a low population of equivalent parameter tuples. Let us call equivalent to two different parameter tuples Q , Q 0 that produce the same distribution. Although we know that for any dimension D, there exist equivalent parameter tuples, this does not mean that for each parameter tuple Q there 2D will exist an equivalent, different one Q 0 (always for the case M ¿ DC1 ). For example, it is easily seen that for the mixture in D dimensions with parameters given by,

Q :

n

M = 2,

n o n oo p 1 2 (0, 1), p1 = (1 1 . . . 1)T , 1 ¡ p 1 , p2 = (0 0 . . . 0) T ,

which produces a distribution

8

t = (1 1 . . . 1) t = (0 0 . . . 0) otherwise,

Practical IdentiŽability of Finite Bernoulli Mixtures

151 D

2 there does not exist any equivalent parameter tuple Q 0 for M < DC1 (disregarding, as usual, permutations of mixture components and coincident component distributions). Thus, the actual practical problem of identiŽability is which parameter tuples Q have equivalent parameter tuples, or, considering the partition of the space of parameter tuples into classes of equivalence (each class of equivalence consisting of all equivalent parameter tuples), what is the cardinality of each class. This is a difŽcult question to answer analytically with generality, but the experimental results suggest that nontrivial equivalence classes (consisting of more than one element) may be rare, perhaps pathological. Another reason for the possibility of estimating a sensible parameter tuple seems to be that every estimate contains some of the original prototypes and that the original number of components may be selected by inspection of a collection of maximum likelihood estimates obtained with the EM algorithm (as we did in section 4.1). Note that the likelihood function associated with a small sample (compared to the total number of possible different vectors, usually 2D ) need not be maximized by the original mixture. Finally, a reciprocal situation to the one described here may also be possible, as the following example in D = 1 dimension shows. Consider the class of mixtures of normal distributions, which is known to be identiŽable for all dimensions (Everitt & Hand, 1981). It can be readily veriŽed that the two-component equiprobable mixture with component means at §1 and standard deviations of 1.5 is virtually equal to a norm of mean 0 and standard deviation 1.85 (e.g., the distance between both mixture distributions in the L2 -norm sense is 50 times smaller than the L2 -norm of any of the mixtures). A sample of enormous size would be necessary to tell one mixture from the other in terms of likelihood of the parameters. Thus, theoretical identiŽability does not guarantee practical identiŽability, and interpretation problems may still arise.

Acknowledgments This work was supported by ESPRIT Long Term Research Project SPRACH (20077) and by a scholarship from the Spanish Ministry of Education and Science. M. A. C.-P. acknowledges helpful comments from Zoubin Ghahramani. References ´ & Renals, S. (1998). Dimensionality reduction of elecCarreira-Perpin´ ˜ an, M. A., tropalatographic data using latent variable models. Speech Communication, 26, 259–282. Everitt, B. S., & Hand, D. J. (1981). Finite mixture distributions. London: Chapman & Hall.

152

´ Carreira-Perpin´ Miguel A. ˜ an and Steve Renals

Gyllenberg, M., Koski, T., Reilink, E., & Verlaan, M. (1994). Non-uniqueness in probabilistic numerical identiŽcation of bacteria. J. Appl. Prob., 31, 542–548. Hardcastle, W. J., Jones, W., Knight, C., Trudgeon, A., & Calder, G. (1989). New developments in electropalatography: A state-of-the-art report. J. Clinical Linguistics and Phonetics, 3, 1–38. Marchal, A., & Hardcastle, W. J. (1993). ACCOR: Instrumentation and database for the cross-language study of coarticulation. Language and Speech, 36, 137– 153. Wolfe, J. H. (1970). Pattern clustering by multivariate mixture analysis. Multivariate Behavioral Research, 5, 329–350. Received October 21, 1997; accepted July 2, 1998.

LETTER

Communicated by Michael Shadlen

The Effects of Pair-wise and Higher-order Correlations on the Firing Rate of a Postsynaptic Neuron S. M. Bohte Graduate School Neurosciences Amsterdam, AMC, Department of Visual System Analysis, University of Amsterdam, P.O. Box 12011, 1100 AA Amsterdam, and Netherlands Center for Mathematics and Computer Science, (W1) Amsterdam, The Netherlands

H. Spekreijse P. R. Roelfsema Graduate School Neurosciences Amsterdam, AMC, Department of Visual System Analysis, University of Amsterdam, P.O. Box 12011, 1100 AA Amsterdam, The Netherlands

Coincident Žring of neurons projecting to a common target cell is likely to raise the probability of Žring of this postsynaptic cell. Therefore, synchronized Žring constitutes a signiŽcant event for postsynaptic neurons and is likely to play a role in neuronal information processing. Physiological data on synchronized Žring in cortical networks are based primarily on paired recordings and cross-correlation analysis. However, pair-wise correlations among all inputs onto a postsynaptic neuron do not uniquely determine the distribution of simultaneous postsynaptic events. We develop a framework in order to calculate the amount of synchronous Žring that, based on maximum entropy, should exist in a homogeneous neural network in which the neurons have known pair-wise correlations and higher-order structure is absent. According to the distribution of maximal entropy, synchronous events in which a large proportion of the neurons participates should exist even in the case of weak pair-wise correlations. Network simulations also exhibit these highly synchronous events in the case of weak pair-wise correlations. If such a group of neurons provides input to a common postsynaptic target, these network bursts may enhance the impact of this input, especially in the case of a high postsynaptic threshold. The proportion of neurons participating in synchronous bursts can be approximated by our method under restricted conditions. When these conditions are not fulŽlled, the spike trains have less than maximal entropy, which is indicative of the presence of higher-order structure. In this situation, the degree of synchronicity cannot be derived from the pair-wise correlations. 1 Introduction The occurrence of correlations in the spike trains of neurons responding to the same object has raised considerable excitement during the past decade Neural Computation 12, 153–179 (2000)

c 2000 Massachusetts Institute of Technology °

154

S. M. Bohte, H. Spekreijse, and P. R. Roelfsema

(review by Singer & Gray, 1995). Correlations between pairs of neurons are thought to reect a high degree of synchronous Žring within a larger assembly of neurons (Singer, 1995; Engel, Konig, ¨ Kreiter, Schillen, & Singer, 1992) and can have a high temporal precision, in the range of a few milliseconds (Eckhorn et al., 1988; Gray, Konig, ¨ Engel, & Singer, 1989; Roelfsema, Engel, Konig, ¨ & Singer, 1997; Alonso, Usrey, & Reid, 1996; Abeles, Bergman, Margalit, & Vaadia, 1993; Gray & Singer, 1989). von der Malsburg (1981) suggested that assemblies of neurons might convey additional information by Žring in synchrony, since synchrony could be instrumental in forming relationships between the members of such an assembly. However, the possible relevance of Žne temporal structure in spike trains opposes another widespread belief. The irregular timing of cortical action potentials is quite often attributed to stochastic forces acting on the neuron (Bair, Koch, Newsome, & Britten, 1994; Shadlen & Newsome, 1994). In such a stochastic model, the information is thought to be conveyed to the next processing stage (cortical layer) by pools of neurons using a noisy rate code. Each individual neuron is considered to be a slow, unreliable information processor, reecting changes in its receptive Želd by modulating its average Žring rate. Only by pooling the information from a larger number of neurons can a reliable rate code be obtained. Obviously, this scheme does not need precise timing of the individual spikes to convey much information. These two opposing views on the role of temporal structure of neuronal information processing are the subject of considerable debate (K¨onig, Engel, & Singer, 1996; Shadlen & Newsome, 1995; Softky & Koch, 1993). This debate has focused on two important questions: Is the cortical neuron a coincidence detector (on the millisecond timescale), and how much coincident input is there? The Žrst question refers to the relevance of synchronous presynaptic spikes. It has been suggested that synchronous input induces a higher Žring rate in the postsynaptic target cell. Does this assumption hold, especially on a millisecond timescale? This question has been amply recognized, and several studies have attempted to answer it. Shadlen and Newsome (1995) argue that based on physiological considerations, a cortical neuron is not capable of detecting very tightly synchronized input. However, others have argued that cortical neurons might have a high sensitivity for the synchronicity in their input (Softky, 1995; Konig ¨ et al., 1996). Softky (1995) pointed out that the biological data leave too many parameters undetermined to draw any deŽnite conclusions on biological properties that distinguish the various models. Two further studies on the impact of synchronized input on a postsynaptic target reinforce this observation. Using detailed models of groups of neurons, Bernander, Koch, and Usher (1994) and Murthy and Fetz (1994) studied the impact of coincident input on the Žring rate of a postsynaptic neuron. Their conclusions are similar to Softky’s: within the biologically plausible parameter ranges, synchrony may either increase or decrease the Žring rate of postsynaptic neurons.

Effects of Pair-wise and Higher-order Correlations

155

Figure 1: (A) Three neurons with correlated activity. The pair-wise correlation coefŽcients are r AB , r AC , and r BC . (B) Examples of different spike conŽgurations in windows of the same size. The horizontal lines represent the spike trains, and each tick denotes a spike. Spikes occurring with a time window (dotted box) are considered to be coincident. (C) Three examples of correlated spike trains. Pairs of neurons in the three panels have the same pair-wise correlation. Spike doublets are shown as unŽlled arrows and triplets as Žlled arrows. The number of triplets differs from panel to panel.

In this study we attempt to shed more light on the second question: How much synchrony is there? In general, it is implicitly assumed that pair-wise correlations provide a good estimate of the amount of synchrony in a pool of neurons from which recordings are obtained. However, to date there are no direct electrophysiological measurements of large, synchronous pools of cortical neurons. Most of the physiological data on neuronal synchronization so far have been obtained using cross-correlation techniques (with the notable exception of the work of Abeles et al., 1993). These techniques merely provide information on the probability of events in which a pair of neurons Žres at the same time (that is, within some time window). Unfortunately, pair-wise correlations provide only an indirect estimate of the probability of higher-order events, like the coincident Žring of, say, 5 or 50 neurons. Even when the pair-wise correlations between all neurons of a network are Žxed, the probability of these higher-order events remains undetermined, as is illustrated in Figure 1.

156

S. M. Bohte, H. Spekreijse, and P. R. Roelfsema

Pair-wise correlation is deŽned as the difference between the probability that two neurons Žre simultaneously and the product of their Žring rates (the coincidence rate dictated by chance). This value is the same for any pair of neurons in the three panels of Figure 1C. However, the number of spike triplets differs considerably from panel to panel. Given this example, it is quite clear that the number of neurons that Žre within some time window is not exclusively determined by the pair-wise correlation coefŽcient. And although this is an artiŽcial example, it is already quite difŽcult to determine how many of these triplets can be attributed to higher-order correlation and how many result from two doublets that happen to occur at the same time. In a Žrst attempt to quantify the incidence of higher-order correlations, Martignon, von Hasseln, Grun, ¨ Aertsen, and Palm (1995) analyzed data from six cortical neurons. Unfortunately, we found that their methods cannot be used for the analysis of large numbers of neurons (as will be discussed). The main goal of this article is to study the relationship between pair-wise correlations and the amount of synchronicity in a pool of neurons and to determine the impact of the synchronous events on a postsynaptic target cell. 2 Solution of the Three-Neuron Problem To illustrate the general methodology used for estimation of the probability of higher-order events, we examine the three neuron network of Figure 1. We wish to calculate the probability that a triplet (or an N cluster, where N equals 3) occurs within a given time window. The null hypothesis is that no structure is present in the spike trains other than the pair-wise correlations. That is, all triplets should be due to the occurrence of two doublets at the same time, by chance forming a triplet. Csiszar (1975) has proved the unique existence of a distribution with just this property. The basic approach for calculating the probability distribution involves maximizing the informational entropy of the data while preserving the pair-wise correlation and the Žring rate (see also Martignon et al., 1995). This informational entropy is a measure of the “order” in the data: the more structured the data, the lower the entropy. By measuring the pair-wise correlations and the Žring rates, a certain degree of order is Žxed. Taking these constraints into account, maximizing the entropy will minimize all higher-order correlations since higher-order correlations will add structure to the distribution, further lowering the entropy. Therefore, maximal entropy implies minimal higher-order correlations. Our aim is to obtain the distribution of N clusters that has maximal entropy. We will illustrate the procedure by computing this distribution for three connected neurons. The neurons are labeled A, B, and C; their Žring probabilities are f1A , f1B , and f1C ; and the pair-wise correlation coefŽcients are denoted as r AB , r BC , and r AC ( f1i denotes the probability that neuron i Žres in a particular time bin and depends on the Žring rate and the size of the time bins; the

Effects of Pair-wise and Higher-order Correlations

157

rationale of the sufŽx 1 will become clear below). Given these coefŽcients as constraints, probabilities for all 2n possible events can be calculated. For example G010 designates the probability of the event in which B is Žring and A and C are silent (see Figure 1B). There are eight probabilities to solve for. Since Žring probabilities and pair-wise correlation coefŽcients are Žxed, seven equations can easily be obtained. The sum of the probabilities of all possible events equals one: G000 C G001 C G010 C G011 C G100 C G101 C G110 C G111 = 1.

(2.1)

The Žring probability of a neuron A is: G100 C G110 C G101 C G111 = f1A .

(2.2–2.4)

(The equations for f1B and f1C are derived analogously.) Let f2AB denote the probability that A and B Žre simultaneously (the sufŽx now indicates a cluster of size 2). f2AB is determined by the correlation coefŽcient r AB , since r AB = q

f2AB ¡ f1A f1B

2 )( 2 ) ( f1A ¡ f1A f1B ¡ f1B

.

(2.5)

This implies that r AB determines f2AB , and since f1A and f1B are Žxed, this yields: G110 C G111 = f2AB ,

(2.6–2.9)

and corresponding equations are derived for r AC and r BC . These seven equations (2.1–2.4 and 2.6–2.9) yield one free parameter (G111, for example), which determines the entropy of the probability distribution. By calculating the value at which the distribution has maximal entropy, we Žx this last free parameter. The entropy function is deŽned as: S´¡

X

Gi ln(Gi )

i over all possible spike conŽgurations. (2.10)

i

The entropy is maximized by solving the following equation, which has a unique solution (see the appendix): @S = 0. @G111

(2.11)

Using this equation, the probability distribution of conŽgurations with the desired zero higher-order correlation can easily be calculated (see also Martignon et al., 1995).

158

S. M. Bohte, H. Spekreijse, and P. R. Roelfsema

3 Calculating the Distribution with N Identical Neurons For more than three neurons, the equations can no longer be solved easily by analytical means. For instance, for four neurons, the same calculations yield the entropy as a function of Žve free parameters. Calculating the maximum of this function is already quite complicated. Extending this to N neurons, only 1 CN(N C1)/ 2 equations are determined by the pair-wise correlations and Žring probabilities, with 2N parameters to solve for. To overcome this problem, an iterative algorithm for calculating the maximal entropy distribution has been provided by Gokhale and Kullback (1978). Unfortunately, this algorithm has a serious drawback: the number of calculations increases exponentially with the number of neurons. If one is interested in the behavior of many correlated neurons, computational restrictions prohibit this method of calculating the distribution for more than 20 neurons on a workstation, thus putting the more serious number of neurons out of reach of even the fastest supercomputers. To overcome the problem of increasing computation time, we made the additional assumption that all neurons have identical properties, which implies a major degeneration of the probability space. Since any two neurons are equal, the same will hold for the probability of all permutations of spike conŽgurations. In addition, the N Žring probabilities and the N (N ¡ 1)/ 2 pair-wise correlation coefŽcients will also be equal. The distribution of spike conŽgurations is described by N C 1 variables, D0 through DN , where Di denotes the probability of a particular spike conŽguration in which exactly i neurons Žre. In the case of N = 7: D1 = G1000000 = G0100000 = ¢ ¢ ¢ = G0000001; D2 = G1100000 = G1010000 = G0010010, etc. Since all permutations of i spiking and N ¡ i silent neurons have the same probability Di , the requirement that all probabilities add up to 1 now reads 1=

¡ ¢

N X N ¢ Di . i

(3.1)

i= 0

The Žring probability of a cell equals (see the appendix) f1 =

¡

¢

N X N¡1 ¢ Di . i¡1

(3.2)

i= 1

Under the assumption of identical neurons, in equation 2.5, f1B equals f1A . Rewriting equation 2.5, replacing f1A and f1B by f1 , and f2AB by f2 yields r =

f2 ¡ f12 . f1 (1 ¡ f1 )

(3.3)

Effects of Pair-wise and Higher-order Correlations

159

Thus, f2 can once again be calculated from the Žring probability of a neuron and the correlation. f2 , the probability that any two particular neurons Žre at the same time, equals f2 =

¡

¢

N X N¡2 ¢ Di . i¡2

(3.4)

i= 2

By maximizing the entropy S, we derive the remaining N ¡ 2 equations. The entropy is deŽned as S= ¡

¡ ¢

N X N Di ln(Di ). i

(3.5)

i= 0

Maximization of entropy yields @S = 0 @Di

for i = 3, . . . , N.

(3.6)

A proof of a unique solution of this set of equations is included in the appendix. After some calculations the N ¡ 2 equations turn out to have the form

¡

¢

1 ln(Di ) = ¡ ¡ i C 1 (i ¡ 1) ln(D0 ) ¡ i( i ¡ 2) ln(D1 ) 2 1 C i( i ¡ 1) ln(D2 ) 2

for i = 3, . . . , N

(3.7)

Inserting these N ¡ 2 equations into equations 3.1, 3.2, and 3.4, we can numerically solve for D0 , D1 , and D2 using the Newton-Raphson method for nonlinear systems of equations (Press, Flanney, Teukolsky, & Vetterling, 1986). The maximal entropy distribution, thus deŽned, depends on only two parameters: the Žring probability f1 and the pair-wise correlationr . Figure 2 illustrates the shape of the N-cluster distribution with maximal entropy for a network of 150 neurons and its dependence on f1 and r . Figure 2 shows the probability Pi that a particular number of neurons Žre simultaneously, where P i equals the sum of the probabilities of all conŽgurations in which exactly i neurons Žre: Pi =

¡ N¢ i

Di .

(3.8)

For small values ofr , the distribution of N clusters approaches a binomial distribution. This determines the Žrst peak of the N-cluster distribution,

160

S. M. Bohte, H. Spekreijse, and P. R. Roelfsema

corresponding to small cluster sizes. For larger values of r , a second peak appears in the distribution, the amplitude of which grows with increasing r . A second inuence of increasing r is a divergence of the two peaks. This divergence can be seen most clearly in the contour plots of Figures 2D–F. Indeed, in the limiting case ofr = 1, the Žrst peak approaches a cluster size of 0 and the second peak a cluster size of 150, since all neurons Žre at exactly the same time. An increase in f1 shifts the Žrst peak of the distribution to larger values, as is predicted by the binomial distribution (see Figures 2A–C). Remarkably, an increase in f1 is also associated with a shift of the second peak, to smaller values. Thus, the maximal entropy distribution predicts that low Žring probabilities are associated with sparse but highly synchronized bursts. With a higher Žring probability, synchronous bursts occur more frequently but comprise fewer spikes. 4 An ArtiŽcial Neuron Network We now compare N-cluster distributions obtained from simulations of an artiŽcial neural network to the maximal entropy distribution. Any difference between the two distributions can then be attributed to higher-order correlations. The network used is based on a network described by Deppisch et al. (1993), which, in turn, is based on the work of MacGregor and Oliver (1974). This network was chosen since it could easily be adapted to consist of identical neurons. 4.1 The Neuron Model. The neuron model involves four variables: the membrane potential E( t), a potassium current g( t), the spiking threshold h ( t), and the neuronal output o( t). The dynamics of neuron i are described by four coupled equations: tE

dEi ( t) = ¡(Ei ( t) ¡ E0 ) ¡ ( gi (t) ¡ go )( Ei ( t) ¡ Ek ) Cg i ( t) C dt 2 3 X ¡4 wij ¢ oj (t ¡ tij ) 5 ( Ei ( t) ¡ Eex )

(4.1)

j

th

dhi ( t) = ¡(hi (t) ¡ h 0 ) C c( Ei ( t) ¡ E0 ) dt

tg

dgi ( t) = ¡(gi ( t) ¡ g0 ) C tg boi ( t) dt »

o i ( t) =

1 0

if Ei (t) > hi ( t) otherwise.

0·c·1

(4.2)

(4.3)

(4.4)

Effects of Pair-wise and Higher-order Correlations

161

Figure 2: (A–C) Probability of N clusters (p) as a function of the pair-wise correlation r , for three Žring probabilities, f1 = 0.05 (A), 0.146 (B), and 0.225 (C). Probabilities are clipped at a value of 10¡4 . (D–F) Contour plots of the same data show that the separation between the two peaks becomes larger with increasing r . Contour levels indicate probabilities of 0.001 0.01, and 0.1.

Without input, the membrane potential Ei (t) is driven towards its resting value E0 with a time constant tE . An inux of potassium drives the potential toward the potassium equilibrium potential Ek . Excitatory input moves the potential toward the ionic equilibrium value Eex . In the simulations, the

162

S. M. Bohte, H. Spekreijse, and P. R. Roelfsema

values used by Deppisch et al. (1993) were adopted: E 0 = 0, tE = 2.5 msec, Ek = ¡1, E ex = 7. The membrane time constant tE may appear to be rather small, although some have argued that it may be within the biologically plausible range (Konig ¨ et al., 1996; Bernander, Douglas, Martin, & Koch, 1991). In the discussion we will address the dependence of our results on this particular choice. Input to the cells was strictly excitatory, with synaptic delay tij = 0.5 msec. External input is provided by external stochastically spiking units and is treated equivalent to internal input. Internal noise is added by the additive noise term gi ( t) in equation 4.1, with a standard deviation of 0.06 ¢ Ei ( t). Equation 4.2 describes the slow adaptation of the threshold hi to the membrane potential, modeling an adaptation of the neuron to excitation (h0 = 1, th = 10 msec, c = 0.3). Equation 4.3 governs the potassium current decay dynamics, which is driven toward g0 = 0 with time constant tg (5 msec). In the case of a spike, the potassium current rises by an amount b (4.0) corresponding to the outward potassium current. A spike is generated each time the membrane potential exceeds the threshold (see equation 4.4), after which the neuron is in a refractory state for another 1.5 msec. In the actual numerical implementation of the neuron model, a discrete-time analog of equations 4.1–4.4 was used. These equations were iterated with time steps corresponding to 0.5 msec of real time. Each neuron was connected to every other neuron with a Žxed homogeneous synaptic weight wij = wint / N > 0. Here, wint denotes the sum of the strengths of all synapses from within the network impinging on a single cell. External input consisted of N independent stochastic elements generating spikes with p = 0.05 per msec (50 Hz), connected in a one-to-one fashion with the network neurons with a constant weight wext = 0.8. The parameter wint was varied during the simulation. Increasing the value of the weights drives the network from a stochastic mode into an oscillatory mode (Deppisch et al., 1993). Somewhere within this range of synaptic weights, the network alternates between the stochastic and oscillatory modes. All results are based on simulations with a network with N = 150 neurons, with the exception of the results in section 4.5. This was a compromise between a realistic number of neurons and a network consideration: the absence of inhibitory neurons makes a network very quickly prone to saturation, a state where the constant excitation induces a very sharp oscillation. A further increase in the number of neurons also narrows the weight range in which the network is in the alternating state. The network output was obtained by recording the output of all 150 neurons. A sample of the output of 50 neurons is shown in Figure 3. 4.2 Network Simulations. In order to vary the average pair-wise correlation, simulations were performed with different values of the synaptic weight wint . As was noted by Deppisch et al. (1993), the network exhibits three distinct modes, which depend on the value of the synaptic weight.

Effects of Pair-wise and Higher-order Correlations

163

Figure 3: (Upper panel) Activity of 50 of the 150 neurons. On the horizontal axis, time is displayed (sampling time, 0.5 msec). The vertical ticks indicate action potentials. Oscillatory episodes are alternated by periods of more random activity. (Lower panel) Summed activity of all 150 neurons.

The Žrst, at low values of the synaptic weights, is the stochastic mode in which the average correlation between pairs of neurons is near zero. At the other end of the scale is the mode with high values of internal weight. In this mode, the network activity is highly oscillatory. In between these extremes are synaptic weight values for which the network exhibits episodes in which many neurons Žre synchronously, alternated by episodes in which neurons are synchronized to a lesser degree. Typical activity of the neurons in these three network modes is shown in Figures 4A–C. Also plotted are the cross-correlations between a pair of neurons in the network (see Figures 4D–F). As observed by Deppisch et al. (1993), these cross-correlograms show qualitative resemblance to data obtained from electrophysiological recordings (e.g., Konig ¨ et al., 1995). Remarkably, the network with intermediate synaptic strength (w = 3.875, r = 0.03, for 2 msec bin size) exhibits occasional population bursts, which barely show up in the correlation function (see Figures 4B and 4E). Thus, a network state in which a number of highly synchronous population bursts occur can be associated with correlation functions indicative of weak pair-wise coupling. When investigating the relationship between occurrences of N clusters and the pair-wise correlation, an additional assumption has to be made with regard to the maximal time difference between two spikes that are considered to be synchronous. As a Žrst approach, we used a time window of 2 msec. The maximal Žring rate of the neurons is determined by the refrac-

164

S. M. Bohte, H. Spekreijse, and P. R. Roelfsema

Figure 4: Network activity for three values of the synaptic weight, wint . (A–C) Summed activity of the 150 neurons. The synaptic weights were 3.5 (A, r = 0.003), 3.875 (B, r = 0.03), and 4.375 (C, r = 0.20). Pair-wise correlations were determined for coincidences within a window of 2 msec. (D–F) Cross-correlation functions of two randomly selected neurons for the same synaptic weights as are plotted on the left. Even for a low pair-wise correlation (r = 0.03), highly synchronous network bursts occur, but these barely show up in the cross-correlogram.

tory period (1.5 msec) and the duration of an action potential (0.5 msec). During network bursts, neurons reach this maximal Žring rate, and a single spike occurs in each 2-msec bin. The effect of changing the bin width on the distribution of cluster sizes will be investigated in section 4.4.

Effects of Pair-wise and Higher-order Correlations

165

Figure 5 shows the relationship between the occurrence of N clusters and the pair-wise correlation. Plotted is the probability Pi (see also equation 3.8) of a particular number of coincident spikes, for simulations with different pair-wise correlations (different values of wint ). As can be seen in Figure 5A, most 2-msec bins are occupied by N clusters containing a fairly low number of spikes. This corresponds to the stochastic activity between bursts, and the probability of these events is approximated by a binomial distribution. Synchronous bursts are represented by the second peak in the probability distribution, as can be seen more clearly in the logarithmic plot of Figure 5B. Between these two peaks is a plateau of time bins in which an intermediate number of neurons Žre. These events are caused by time bins that are aligned on the onset or end of a burst. Increases in the pair-wise correlation are associated with a slightly smaller Žrst peak and an enhanced second peak. For very large, and biologically implausible, pair-wise correlations, a third peak containing intermediate cluster sizes is visible, which can be attributed to the onset and offset of bursts. Let us now consider the impact of this distribution on a postsynaptic cell, receiving input from all network neurons. For such a cell, the exact number of synchronous spikes (the quantity plotted in Figures 5A and 5B) is not important. Rather, for a postsynaptic neuron with a Žring threshold of m excitatory postsynaptic potentials (EPSPs), the incidence of m or more coincident spikes is a much more signiŽcant quantity, since this determines its Žring rate to a large extent. Therefore, we computed a cumulative probability distribution: the probability that m or more neurons Žre synchronously. The cumulative probability distributions (the cumulatives of Figure 5B) are plotted in Figure 5C. Suppose that the membrane potential of a postsynaptic neuron, which receives input from all cells in our network, equals the resting potential at time t. The probability that this neuron Žres at time t C 1 is equal to the probability ofh 0 or more coincident spikes, where h 0 is the ratio between the postsynaptic threshold (h ) and the EPSP amplitude. In other words, the curves in Figure 5C illustrate the relation between the threshold and Žring probability of a neuron, which receives input from all neurons in the network, in the case of a very short membrane time constant. The curve for a very low pair-wise correlation coefŽcient (r = 0.003) shows the rapidly declining probability of higher-order events. The curves for network activity with a larger pair-wise correlation exhibit a plateau before their decline at very high numbers of synchronized spikes. This plateau results from the second peak in the N cluster probability distribution. It is remarkable that even small changes in the strength of the pair-wise correlations, which are not physiologically implausible (e.g., Gray et al., 1989; Livingstone, 1996), exhibit a strong effect on the Žring probability in case of an intermediate threshold h 0 . Indeed, the Žring probability of a neuron with a threshold of, say, 50 EPSPs is raised by an order of magnitude due to an increase in the pair-wise correlation as small as 0.08 (compare the distributions forr = 0.03 and r = 0.12).

166

S. M. Bohte, H. Spekreijse, and P. R. Roelfsema

The question remains how closely the Žring rate of a postsynaptic neuron with nonzero membrane time constant is approximated by the cumulative probability distribution. There would obviously be a one-to-one relationship for a postsynaptic neuron that has “zero” memory—that is, a neuron that is inuenced only by the input in the previous time step (the bin size for which spikes are considered synchronous, as discussed above). However, typical neurons have parameters with larger time constants, including the refractory period, and the time constant of the membrane. In order to assess the impact of this “nonzero memory” an additional 151st neuron was included in the network. This cell was identical to the other 150 neurons from which it received input, but it did not project back to them. The pair-wise correlation was Žxed at 0.12, and simulations were performed with different values of the postsynaptic thresholdh 0 by varying wi,151 (1 < i < 150), the strength of the synapses projecting onto neuron 151. The dependence of the Žring probability on h 0 (h/ wi,151 ) is shown in Figure 5D. Superimposed on this graph is the distribution of N clusters. As expected, some minor differences between the Žring probability and the cumulative probability distribution are observed. Most of these differences can be explained, given the fact that the nonzero average activity in a pool of neurons keeps the membrane potential of individual neurons at a somewhat higher level than the resting potential. This lowers the average number of coincident spikes the neuron would require to cross threshold. On the whole, however, the Žring probability of the postsynaptic neuron with a nonzero membrane constant is predicted with good accuracy by the cumulative probability distribution. A remarkable feature of Figure 5D is that the Žring probability of a neuron with a threshold of, for example, 40 does not differ much from that of a neuron with a much higher threshold (e.g., 130). Most of the spikes of a postsynaptic cell with a threshold larger than 40 are triggered by synchronous bursts in which the majority of neurons participate.

Figure 5: Facing page. (A,B) Probability distribution of N clusters, plotted on a normal scale (A) and a logarithmic scale (B). (C) Cumulative probability of N clusters. Abscissa, cluster size (h 0 ). Ordinate, probability of a cluster with a size that is equal to or larger than h 0 . (D) Comparison of the cumulative N-cluster distribution (solid line) to the Žring probability of a postsynaptic neuron, which receives input from all 150 neurons in the network. The threshold h 0 is deŽned as h / wi,151 . (E) Dependence of the Žring probability of a neuron receiving input from all 150 neurons on its relative threshold h 0 and the pair-wise correlation in the network. Different curves correspond to different values ofh 0 . Triangles indicate dependence of the average Žring probability f1 on the pair-wise correlation. (F) Same as E, but the Žring probability of the postsynaptic cell and f1 are plotted as a function of wint .

Effects of Pair-wise and Higher-order Correlations

167

Using this interpretation of the cumulative probability distribution, the effect of an increase in the pair-wise correlation in the network on postsynaptic neurons was investigated by varying the synaptic weight (wint ). Figure 5E shows the relationship between the average pair-wise correlation and the Žring probability of a hypothetical postsynaptic neuron with threshold h 0 . Calculated were the Žring probabilities for thresholds h 0 = 30, 50, and 100. It can be seen that in the biologically relevant range of pair-

168

S. M. Bohte, H. Spekreijse, and P. R. Roelfsema

wise correlations (r = 0–0.2), there is a monotonic relationship between the postsynaptic Žring probability and the pair-wise correlation. For values of r below 0.2, changes in the pair-wise correlation are associated with a relatively large increase in the Žring probability of the postsynaptic neuron. Importantly, higher values of r are also associated with an enhanced Žring rate of the presynaptic network neurons (see Figure 5E). This increase in activity undoubtedly contributes to the enhanced probability of higher-order events. However, it should be noted that the probability of higher-order events exhibits a steeper dependence on r than the Žring probability of the network neurons (see Figure 5E). Figure 5F illustrates the dependency of the Žring probability of the postsynaptic neuron on wint , the parameter that was actually varied during the simulations. 4.3 Estimation of the N-Cluster Distribution by Entropy Maximization. In most electrophysiological experiments, only data on Žring probabilities and pair-wise correlations are available. Using the mathematical framework developed in section 3, we investigated whether the observed relationship between the probability of N clusters and the correlation coefŽcient could have been predicted from these two types of measurements. For the network behavior at different values of wint (3.5, 3.875, and 4.125), distributions of N clusters were calculated that maximized entropy under the constraints of the observed Žring probability and pair-wise correlations. A comparison between distributions based on the maximal entropy calculation and the experimental distribution is shown in Figures 6A–C. The distribution that maximized the entropy deviates somewhat from the observed distribution for larger values of r . The maximal entropy calculation underestimates the incidence of clusters between 60 and 100 spikes, which occur during the start and end of population bursts, as was discussed above. In addition, the second peak in the maximal entropy distribution is located at a cluster size of 130, whereas the actual location of this peak is 150. Typically, all neurons Žre within a 2-msec window during a network burst. In order to estimate the effect of these discrepancies on the Žring probability of a neuron receiving input from the network, the cumulative probability distributions are plotted in Figures 6D–F. It can be seen that the underestimation of the incidence of intermediate cluster sizes by the maximal entropy calculation is compensated by the overestimation of the incidence of clusters with a size between 100 and 140. For values of h 0 smaller than 130, the largest deviation is about a factor of 2. This approximation is reasonable, since the maximal entropy calculation depends on only two parameters, which are estimated from the network activity: the Žring probability and the pair-wise correlation. Nevertheless, large deviations occur for values of h 0 larger than 130. However, these high-threshold values are physiologically implausible, because they would imply that a postsynaptic neuron would Žre only when almost every input is active within a narrow time window.

Effects of Pair-wise and Higher-order Correlations

169

Figure 6: Comparison of the probability of N clusters in the network simulation and their probability based on maximal entropy. (A–C) Probabilities of N clusters in the simulation are shown as solid curves for three values of the synaptic weights. Dashed line: The corresponding maximal entropy probability distributions with the same Žring probability and pair-wise correlation. (D–F) Relationship between Žring probability (ordinate) and threshold (abscissa) of a neuron with an integration window of 2 msec that receives input from all 150 neurons of the network (cumulatives of A–C).

170

S. M. Bohte, H. Spekreijse, and P. R. Roelfsema

4.4 Effects of Varying Bin Width on the Maximal Entropy Distribution. The results of the maximal entropy calculation described so far were obtained with a bin width of 2 msec, which equals the minimal interspike interval. Figure 7 shows the dependence of the N cluster distribution and the maximal entropy estimation on the bin width. Spike trains obtained with a synaptic weight (wij ) of 4.125 were rebinned, using bin widths of 0.5, 1.0, 2.0, and 4.0 msec. The rebinning process inuenced the pair-wise correlation and f1 , the probability of Žring in a time bin (see equation 3.2). Smaller bin widths reduce f1 . This results in a leftward shift in the location of the Žrst peak in the distribution of cluster sizes, which represents stochastic activity between bursts (compare Figures 7A and 7B to Figure 7C). A second effect of reducing the bin width is a disappearance of clusters with sizes larger than 110. This disappearance is related to the refractory period, which prohibits neurons from Žring in consecutive bins during population bursts. Spikes Žred by different neurons during these bursts are therefore divided between successive time bins, and no bins remain in which all neurons Žre simultaneously. This modiŽcation of the distribution of cluster sizes is not captured by the maximal entropy estimation, which underestimates the incidence of clusters with sizes between 20 and 110 and overestimates the incidence of clusters with a size larger than 130 (see Figures 7A–B). In other words, the refractory period adds structure to the spike trains, which therefore have less than maximal entropy. When a bin width is used that is longer than the refractory period, this additional structure is lost, and the maximal entropy calculation may provide a reasonable estimate of the distribution of cluster sizes. For bin sizes that are longer than the minimal interspike interval, there are time windows during which individual cells Žre more than a single spike. It is possible, in principle, to adapt the maximal entropy estimation to this situation. One approach, in the case of a bin size that may include two spikes, is to compute the probability that N neurons Žre once and M neurons Žre twice in a bin, for each combination of N and M. Unfortunately, this increases the number of variables that should be calculated from 151 (D0 to D150) to more than 10,000. The computational requirements increase further if three or more spikes can occur in a single bin. Therefore, we took an alternative approach in which the state of a neuron was labeled “off” in the case of no spike and labeled “on” in the case of one or more spikes within a time bin. This keeps the computational requirements within bounds, but at the cost of losing spikes in the rebinning process. Figure 7D compares the distribution of cluster sizes to the maximal entropy distribution for a bin width of 4 msec. The experimental distribution is largely described by a broad Žrst peak, which is shifted to the right, and a very narrow second peak at a cluster size of 150. An increase in f1 also shifts the Žrst peak of maximal entropy distribution to the right. However, larger values of f1 are also associated with a leftward shift of the second peak in the distribution of maximal entropy, as was discussed in section 3. This causes large deviations

Effects of Pair-wise and Higher-order Correlations

171

Figure 7: (A–D) Comparison of the maximal entropy distribution to the distribution of N clusters in the simulation for bin sizes ranging from 0.5 msec to 4 msec. The quality of the maximal entropy approximation depends on bin size, that is, on what is considered coincident. In our simulations, a bin size of 2 msec yields the best approximation (C). For smaller bin sizes (A: 0,5 msec, B: 1 msec), the actual and predicted distributions deviate considerably. (E–H) Impact of these deviations on a postsynaptic neuron with threshold h 0 .

172

S. M. Bohte, H. Spekreijse, and P. R. Roelfsema

for cluster sizes between 60 and 120, the incidence of which is overestimated by the maximal entropy estimation. Moreover, the narrow peak at a cluster size of 150 is absent in the distribution of maximal entropy. In summary, reasonable estimates of the N-cluster distribution are obtained only for a bin width that is equal to the minimal interspike interval. In the discussion, we will address the question of whether these results can be generalized to other network architectures. 4.5 Effects of Network Scaling. In order to investigate how the network behavior and the goodness of Žt of the maximal entropy distribution depend on network size, simulations were run with networks composed of 75, 150, and 200 neurons. Figure 8A shows the cumulative distribution of N clusters for a network of 75 neurons with a wint of 3.55, which exhibited a pair-wise correlation of 0.07. It should be noted that wint represents the sum of the synaptic weights wij of all inputs converging onto a single neuron (section 4.1). When the network size is doubled, each cell receives input from twice as many neurons, and the strength of the individual synapses wij was reduced accordingly, in order to maintain a constant value of wint . Nevertheless, a doubling of the network size resulted in a clear leftward shift of the distribution of N clusters and a reduction of the pair-wise correlation to 0.003. This indicates that a constant value of wint is not sufŽcient to guarantee a qualitatively similar network behavior when the network size is increased. Indeed, a larger number of synapses with reduced weight impinging on a network unit results in a reduction of the uctuations in the input, as long as the network is in the stochastic mode. This reduces the probability of bursts in the network (Tsodyks & Sejnowski, 1995). Tsodyks and Sejnowski (1995) have suggested that the variance in the input to network units may be kept approximately constant by reducing the release probability of the synapses rather than reducing their strength wij , when network size is increased. Figure 8C shows the cumulative distribution of N clusters for a network of 150 neurons, in which the effective input strength was kept constant by reducing the release probability to 50%. The pair-wise correlation was 0.12, and the cumulative distribution of N clusters was qualitatively similar to that of the smaller network, in accordance with Tsodyks and Sejnowski (1995). Figure 8D shows a similar result for a network composed of 200 neurons. In all cases the estimate based on maximal entropy was reasonable (dashed lines in Figures 8A–D), which indicates that the quality of the maximal entropy calculation does not depend critically on the size of the network. 5 Discussion In most physiological studies on the synchronization behavior of cortical neurons, recordings are obtained from pairs of neurons or pairs of cell clusters. Our results illustrate that these data can supply only limited informa-

Effects of Pair-wise and Higher-order Correlations

173

Figure 8: Effect of scaling the network size. (A) Continuous line, probability of N clusters with a minimal size of h 0 , as a function of h 0 , for a network with 75 neurons. Dashed line: Distribution of maximal entropy with the same Žring probability and pair-wise correlation. Parameters of the simulation: wij = 4.73 ¢ 10¡2 , r = 0.07. (B) N cluster distribution for a larger network with 150 neurons. The total input converging on a neuron, wint , was kept constant by reducing wij to 2.37¢ 10¡2 . Nevertheless, the distribution of N clusters was shifted to the left, and the pair-wise correlation was reduced to 0.003. (C) Same as B, but the effective wint was kept constant by reducing the release probability to 50% rather than by reducing wij . The synaptic strength wij was 4.73¢10¡2 , andr = 0.12. (D) N-cluster distribution for a network of 200 neurons, with a r of 0.15. The synaptic weight wij was identical to that in A, but release probability was reduced to 37.5%. Note the similarity of the distributions in C and D to the distribution in A.

tion about the probability of higher-order events—for example, the probability that 30 or more neurons that project to a target neuron Žre simultaneously. As an approximation for the probability of higher-order events, we used the distribution of maximal entropy. This method provides the most unstructured distribution, given the constraints supplied by the Žring probabilities and the pair-wise correlations. The N-cluster distribution with maximal entropy for a homogeneous network without higher-order

174

S. M. Bohte, H. Spekreijse, and P. R. Roelfsema

correlations exhibits two peaks. The Žrst peak represents stochastic activity and resembles a binomial distribution. This peak also occurs in the absence of correlations. If the Žring probability is moderate (< 0.25), a second peak occurs at a relatively large cluster size. The magnitude of this second peak depends on the strength of the pair-wise correlation. Thus, an absence of higher-order correlations dictates that the network should generate coincidences in a number of tightly synchronized network bursts. The N-cluster distribution observed in the simulations also exhibited two peaks, although the position of the second peak was different from the second peak in the maximal entropy distribution. Questions about the generality of these results, in particular, about their dependence on the details of the network implementation, will have to await further experimentation. Previous studies on the impact of correlations among neurons that project to a common postsynaptic target have used N-cluster distributions with a drastically different shape (Bernander et al., 1994; Murthy & Fetz, 1994). In these studies correlations were introduced in the input by forcing a subset of the presynaptic neurons to Žre in perfect synchrony, but independent of the other presynaptic neurons. Since the resulting N-cluster distribution is relatively devoid of highly synchronous events and has less than maximal entropy, the generality of the results obtained in these earlier studies may also be limited. It is obvious that network bursts in which a large number of neurons participate are rather effective in driving a postsynaptic neuron, especially in the case of a high postsynaptic threshold. Indeed, a recent study (Alonso et al., 1996) uncovered tight correlations, with a peak width in the crosscorrelogram of less than 1 msec, among pairs of neurons in the lateral geniculate nucleus (LGN). In cases in which the LGN neurons projected to a common cortical neuron, the impact of synchronous events was stronger than that of asynchronous spikes. We observed an orderly relationship between the incidence of highly synchronous network bursts and the pair-wise correlation. Relatively small increases in the pair-wise correlation, which are not physiologically implausible, may raise the incidence of highly synchronous events, and thereby the Žring rate of a postsynaptic cell, by an order of magnitude (see Figures 5C and 5E). We will now discuss the limitations of the maximal entropy estimation and the way in which these limitations may be overcome by future studies. First, the quality of the approximation by the distribution of maximal entropy exhibited a strong dependence on the choice of the bin size. Reasonable results were obtained only for bin sizes larger than the refractory period of the neurons. If the bin size was smaller than the refractory period, structure was added to the N-cluster distribution, which therefore had less than maximal entropy. This limitation is a direct consequence of the way in which the entropy was deŽned. The entropy was determined by the distribution of N clusters within individual time bins and was independent of the order of N clusters in successive bins. The equivalent situation for

Effects of Pair-wise and Higher-order Correlations

175

a cross-correlation study would be to calculate only the center bin of the cross-correlation functions, that is, only the probability that two neurons Žre at exactly the same time. A possible extension of the method would be to reformulate entropy in order to include second-order correlations with a time delay (i.e., the noncentral bins in the auto- and cross-correlation functions) in the calculation. However, such an extension is likely to result in a substantial increase in the number of variables and equations. Second, the quality of approximation by the maximal entropy distribution was also degraded if a bin width was chosen that was larger than the minimal interspike interval (larger than 2 msec in our simulations). In this situation, multiple spikes of a single neuron occurred within a single time bin. The membrane time constant of the postsynaptic neuron provides a natural temporal window over which input is integrated, that is, the natural coincidence window. Currently the value of the effective membrane time constant in cortical neurons is a topic of considerable debate (Bernander et al., 1991; Konig ¨ et al., 1996; Shadlen & Newsome, 1995). A coincidence window of 2 msec, which we used, is presumably at the lower end of the physiologically plausible range. However, it seems likely that an increase in the width of the bin in which spikes are considered to be coincident to 5 or even 10 msec may be unproblematic in the case of cortical neurons. Typical peak Žring rates of cortical neurons are around 100 Hz, which implies that the probability of multiple spikes within a 5- or 10-msec bin will still be rather low. Alternatively, the calculation of the maximal entropy may be adapted to allow for multiple spikes in a single bin, as was discussed in section 4.4. If a very large bin size is chosen, the distribution of the number of spikes Žred by a single neuron in a bin may approach a normal distribution. In this case it is relatively easy to calculate the distribution of N clusters if the only correlations are of second order. Third, the method according to which we derived the distribution of maximal entropy is valid only for a homogeneous network. For a nonhomogeneous network, the number of variables grows exponentially with the number of neurons (see section 3). We have incorporated a single inhibitory neuron in our network, which received input from all excitatory cells and provided strong inhibitory feedback (data not shown). The inhibitory neuron added structure to the spike trains by curtailing population bursts. We did not explicitly incorporate the Žring pattern of the inhibitory neuron in our calculations, which caused a considerable discrepancy between the maximal entropy distribution and the actual distribution of N clusters. These limitations, taken together, imply that it will be rather problematic to predict the probability of highly synchronous events from Žring rates and pair-wise correlations in physiological experiments. In a physiological study on higher-order events among neurons of the frontal cortex, Martignon et al. (1995) obtained discrepancies between the probability of actually occurring events and their probability predicted by entropy maximization. Unfortunately, even in this study, in which simultaneous recordings from

176

S. M. Bohte, H. Spekreijse, and P. R. Roelfsema

six neurons were studied, sampling problems occurred that were caused by the exponentially growing number of spike conŽgurations and the limited recording time available during such experiments. Another way in which the probability of highly synchronous events might be estimated is by direct, in vivo measurements of the distribution of the postsynaptic potential. A comparison of the postsynaptic potential to the size of individual EPSPs could provide valuable insight in the degree of synchronicity among neurons that project to a common target cell. Appendix: Maximizing the Entropy in a Homogenous Network For N neurons three equations are derived from the pair-wise correlation, the Žring probability, and the fact that all probabilities should add up to 1. First, the probabilities should add up to 1: 1=

¡¢

n X n ¢ Di i

(3.1)

i= 0

Second, the Žring probability f1 is Žxed. For example, in the case that N = 7, it is easy to see that f1 equals: f1 =

1 X 1 X 1 X 1 X 1 X 1 X

G1ijklmn =

i= 0 j= 0 k= 0 l= 0 m= 0 n= 0

7 X i= 1

¡

¢

6 Di . i¡1

(A.1)

In the general case of N = n, this reads: f1 =

¡

¢

n X n¡1 ¢ Di . i¡1

(3.2)

i= 1

Third, the pair-wise correlations are Žxed, and this implies that the probability that two neurons Žre simultaneously is also Žxed. Analogously to equation 3.2, we Žnd for f2 : f2 =

¡

¢

n X n¡2 ¢ Di . i¡2

(3.4)

i= 2

These equations can be rewritten as: D0 = 1 ¡

¡¢

¡¢

¡¢

n n X X n n n Di = 1 ¡ nD1 ¡ D2 ¡ Di 2 i i

D 1 = f1 ¡ D 2 = f2 ¡

i= 1 n X i= 2 n X i= 3

¡ n ¡ 1¢ i¡1

¡ n ¡ 2¢ i¡2

Di = f1 ¡ [n ¡ 1]D2 ¡ Di .

i= 3 n X i= 3

¡ n ¡ 1¢ i¡1

Di (A.2)

Effects of Pair-wise and Higher-order Correlations

Solving for D0 , D1 , and D2 : D0 = 1 ¡ nf1 C n( n ¡ 1) f2 ¡

¡ n¢

µ C

2

Cn

¡ n( n ¡ 1)

¡

¡ n¢ 2

¶X n

¢

f2

¡ n ¡ 2¢ i¡2

i= 3

Di

¡¢

n n X X n¡1 n Di ¡ Di i¡1 i i= 3

i= 3

D1 = f1 ¡ ( n ¡ 1) f2 C ( n ¡ 1)

¡

¢

177

¡

¢

¡

¢

n n X X n¡2 n¡1 Di ¡ Di i¡2 i¡1 i= 3

i= 3

n X n¡2 D2 = f2 ¡ Di , i¡2

(A.3a–c)

i= 3

with the entropy deŽned as S= ¡

¡¢

n X n Di ln(Di ). i

(3.5)

i= 0

This is a concave function (which is easily veriŽed), and therefore it has a single, unique maximum. To obtain the maximum, we calculate the derivative to Di : @S = 0 @Di

@S = ¡ @Di

for i = 3, . . . , n

»µ

¡ n¢ 2

¶ ¡ n( n ¡ 1)

¡

(3.6)

¡ n ¡ 2¢ i¡2

¢ ¡

Cn

¢

¡ n ¡ 1 ¢ ¡ n¢ ¼ i¡1

¡

i

ln(D0 ) C

» ¼ n¡2 n¡1 ¡ n (n ¡ 1) ¡ ln(D1 ) C i¡2 i¡1 C ¡

¡ n¢ ¡ n ¡ 2 ¢ 2

¡ n¢ i

i¡2

ln(D2 ) C

ln(Di ) = 0.

(A.4)

We note that equations A.3a–c are linear in Di and that a concave function remains concave on a linear subspace. In other words, equations A.3a–c and 3.6, taken together, also have a unique solution. Equation A.4 can be

178

S. M. Bohte, H. Spekreijse, and P. R. Roelfsema

rewritten as:

¡

¢

1 ln(Di ) = ¡ ¡ i C 1 ( i ¡ 1) ln(D0 ) ¡ i(i ¡ 2) ln(D1 ) 2 1 C i(i ¡ 1) ln(D2 ) 2

for i = 3, . . . , n.

(3.7)

Inserting these values for Di into equations A.3a–c yields three equations with three unknowns and a unique solution, which is found using the Newton-Raphson method as described by Press et al. (1986). Use of this algorithm in Maple V v4.0 allowed us to solve the equations for up to 150 neurons in about 2 hours on a Pentium 150 Mhz. Acknowledgments We thank Holger Bosch for helpful comments and a critical reading of the manuscript for this article. The research of P.R.R. was funded by a fellowship of the Royal Netherlands Academy of Arts and Sciences. References Abeles, M., Bergman, H., Margalit, E., & Vaadia, E. (1993). Spatiotemporal Žring patterns in the frontal cortex of behaving monkeys. J. Neurophysiol., 70, 1629– 1658. Alonso, J-M., Usrey, W. M., & Reid, R. C. (1996). Precisely correlated Žring in cells of the lateral geniculate nucleus. Nature, 383, 815–819. Bair, W., Koch, C., Newsome, W., & Britten, K. (1994).Power spectrum analysis of bursting cells in area MT in the behaving monkey.J. Neurosci., 14, 2870–2892. ¨ Douglas, R. J., Martin, K. A. C., & Koch, C. (1991). Synaptic backBernander, O., ground activity inuences spatiotemporal integration in single pyramidal cells. Proc. Natl. Acad. Sci. USA, 88, 11569–11573. ¨ Koch, C., & Usher, M. (1994). The effect of synchronized inputs Bernander, O., at the single neuron level. Neural Comp., 6, 622–641. Csiszar, I. (1975). I-divergence geometry of probability distributions and minimization problems. Ann. Probab., 3, 146–158. Deppisch, J., Bauer, H-U., Schillen, T., Konig, ¨ P., Pawelzik, K., & Geisel, T. (1993). Alternating oscillatory and stochastic states in a network of spiking neurons. Network, 4, 243–257. Eckhorn, R., Bauer, R., Jordan, W., Broch, M., Kruse, W., Munk, M., & Reitboeck, H. J. (1988). Coherent oscillations: A mechanism of feature linking in the visual cortex? Biol. Cybern., 60, 121–130. Engel, A. K., Konig, P., Kreiter, A. K., Schillen, T. B., & Singer, W. (1992).Temporal coding in the visual cortex: New vistas on integration in the nervous system. Trends Neurosci., 15, 218–226. Gokhale, D. V., & Kullback, S. (1978). The information in contingency tables. New York: Dekker.

Effects of Pair-wise and Higher-order Correlations

179

Gray, C. M., Konig, ¨ P., Engel, A. K., & Singer, W. (1989). Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reects global stimulus properties. Nature, 338, 334–337. Gray, G. M., & Singer, W. (1989). Stimulus-speciŽc neuronal oscillations in orientation columns of cat visual cortex. Proc. Natl. Acad. Sci. USA, 86, 1698–1702. Konig, ¨ P., Engel, A. K., & Singer, W. (1995). Relation between oscillatory activity and long-range synchronization in cat visual cortex. Proc. Natl. Acad. Sci. USA, 92, 290–294. Konig, ¨ P., Engel, A. K., & Singer, W. (1996). Integrator or coincidence detector? The role of the cortical neuron revisited. Trends NeuroSci., 19, 130–137. Livingstone, M. S. (1996). Oscillatory Žring and interneuronal correlations in squirrel monkey striate cortex. J. Neurophysiol., 75, 2467–2485. MacGregor, R. J., & Oliver, R. M. (1974). A model for repetitive Žring in neurons. Kybernetik, 16, 53–64. Martignon, L., von Hasseln, H., Grun, ¨ S., Aertsen, A., & Palm, G. (1995). Detecting higher-order interactions among the spiking events in a group of neurons. Biol. Cybern., 73, 69–81. Murthy, V. N., & Fetz, E. E. (1994). Effects of input synchrony on the Žring rate of a three-conductance cortical neuron model. Neural Comp., 6, 1111–1126. Press, W. H., Flanney, B. P., Teukolsky, S. A., & Vetterling, W. T. (1986). Numerical recipes:The art of scientiŽc computing. Cambridge: Cambridge University Press. Roelfsema, P. R., Engel, A. K., Konig, ¨ P., & Singer, W. (1997). Visuomotor integration is associated with zero time-lag synchronization among cortical areas. Nature, 385, 157–161. Shadlen, M. N., & Newsome, W. T. (1994). Noise, neural codes and cortical organization. Curr. Opin. Neurobiol., 4, 569–579. Shadlen, M. N., & Newsome, W. T. (1995). Is there a signal in the noise? Curr. Opin. Neurobiol., 5, 248–250. Singer, W. (1995). Development and plasticity of cortical processing architectures. Science, 270, 758–764. Singer, W., & Gray, C. M. (1995). Visual feature integration and the temporal correlation hypothesis. Annu. Rev. Neurosci., 18, 555–586. Softky, W. R. (1995). Simple codes versus efŽcient codes. Curr. Opin. Neurobiol., 5, 239–247. Softky, W. R., & Koch, C. (1993). The highly irregular Žring of cortical cells is inconsistent with temporal integration of random EPSP’s. J. Neurosci., 13, 334–350. Tsodyks, M. V., & Sejnowski, T. (1995). Rapid state switching in balanced cortical network models. Neural Comp., 6, 111–124. von der Malsburg, C. (1981). The correlation theory of brain function (Internal Rep. No. 81-2). Gottingen: ¨ Max-Planck-Institute for Biophysical Chemistry. Received May 29, 1998; accepted January 19, 1999.

LETTER

Communicated by Peter Konig ¨

Effects of Spike Timing on Winner-Take-All Competition in Model Cortical Circuits Erik D. Lumer Wellcome Department of Cognitive Neurology, London, WC1N 3BG, U.K.

Synaptic interactions in cortical circuits involve strong recurrent excitation between nearby neurons and lateral inhibition that is more widely spread. This architecture is commonly thought to promote a winner-takeall competition, in which a small fraction of neuronal responses is selected for further processing. Here I report that such a competition is remarkably sensitive to the timing of neuronal action potentials. This is shown using simulations of model neurons and synaptic connections representing a patch of cortical tissue. In the simulations, uncorrelated discharge among neuronal units results in patterns of response dominance and suppression, that is, in a winner-take-all competition. Synchronization of Žring, however, prevents such competition. These results demonstrate a novel property of recurrent cortical-like circuits, suggesting that the temporal patterning of cortical activity may play an important part in selection among stimuli competing for the control of attention and motor action. 1 Introduction The brain constantly receives a welter of sensory stimuli, only a small fraction of which seems to reach awareness or inuence behavior. Consequently, in the central nervous system, mechanisms must exist to Žlter out unwanted activity. This selectivity typically operates at the level of perceptual groups rather than simple features, biasing processing in favor of objects that are perceptually salient or currently relevant to behavior (Humphreys, Romani, Olson, Riddoch, & Duncan, 1994). Such a selectivity probably reects contentions for limited processing capacity and the control of motor action. Thus, it is central to the organization of adaptive responses. Neurophysiological experiments in primates suggest that this selection process involves a suppression of neuronal responses to irrelevant stimuli (Moran & Desimone, 1985; Motter, 1993; Treue & Maunsell, 1996; Chellazzi, Miller, Duncan, & Desimone, 1993; Schall & Sanes, 1993). For example, attention-dependent modulation of discharge rates has been observed in the visual cortex of monkeys trained to attend to stimuli at one location and ignore stimuli placed at another location (Moran & Desimone, 1985; Motter, 1993). Similarly, responses of direction-selective neurons in middle temporal visual areas are transiently suppressed when attention is diverted away from their activatNeural Computation 12, 181–194 (2000)

c 2000 Massachusetts Institute of Technology °

182

Erik D. Lumer

ing stimuli (Treue & Maunsell, 1996). This modulation of neuronal Žring is commonly thought to reect the effect of attentional biases on intrinsic mechanisms of competition within circumscribed cortical areas, mediated by mutual inhibition (Kastner, De Weerd, Desimone, & Ungerleider, 1998; Chellazzi et al., 1993; Schall & Sanes, 1993). How such mechanisms of competition interact with those underlying basic perceptual processes, including segmentation and grouping, and top-down attentional biases, however, remains uncertain. Here, I describe a novel property of cortical circuits that may in part mediate such an interaction. Using simulations of spiking neurons grouped in recurrent networks that conform to the basic principles of organization of local cortical circuitry, I show that the competition that arises naturally in such networks exhibits a remarkable sensitivity to the relative timing of action potentials. SpeciŽcally, synchronization of neuronal activity can switch the network dynamics from a winner-take-all regime, in which a small subset of neurons can sustain elevated Žring rates, to one in which most cells are active concurrently. Hence, this effect links local mechanisms of response selection to temporal aspects of cortical activity that have been previously implicated in perceptual organization (Milner, 1974; von der Malsburg, 1981; Gray, Konig, ¨ Engel, & Singer, 1989; Singer, 1993). Previous modeling studies have suggested that the combined inuences of recurrent excitation between nearby cells and more widely spread inhibition within cortical areas provide an intrinsic mechanism by which unwanted signals could be suppressed (Stemmler, Usher, & Niebur, 1995; Salinas & Abbott, 1996). When stimuli activating different groups of neurons are presented simultaneously, the recurrent connections generate a nonlinear behavior, yielding neuronal responses that are restricted to the cell group with strongest input. This winner-take-all competition reects a synaptic organization that is common to cortical circuits (Kisv´arday & Eysel, 1993; Douglas, Koch, Mahowald, Martin, & Suarez, 1995). Such a mechanism has been proposed as a basic computational strategy for learning and action selection (Hertz, Krogh, & Palmer, 1991), and underlies several models of brain function, ranging from the development of cortical maps (von der Malsburg, 1973) to the selection of targets during saccadic eye movements (Salinas & Abbott, 1996) and visual search (Koch & Ullman, 1985), and to the suppression of redundancies in visual scenes (Stemmler et al., 1995). However, most models of winner-take-all mechanisms in neural circuits focus on the modulation of mean discharge rates due to competitive interactions. Accordingly, these simulation studies often fail to model individual neurons or their precise spike timing. Yet recent evidence in animals suggests that the temporal coordination of neuronal activity may play an important role in neural network operations. In particular, synchronization of action potentials on a millisecond timescale can occur selectively between cortical neurons that signal related features (Gray et al., 1989). It

Effects of Spike Timing on Winner-Take-All Competition

183

has been suggested that such synchronization may serve to integrate distributed neural events into coherent percepts (Milner, 1974; von der Malsburg, 1981; Singer, 1993). Moreover, it has been shown, in both the hippocampus and neocortex, that networks of recurrently connected inhibitory neurons can both suppress neuronal responses and affect their temporal patterning (Cobb, Bohl, Halasy, Paulsen, & Somogyi, 1995; Whittington, Traub, & Jefferys, 1995). It remains unclear, however, whether the temporal coordination of neuronal activity can inuence the ability of recurrent circuits to induce response suppression. To investigate this possibility, I model neural network interactions in a small patch of cortex in which the relative spike timing between competing units is varied systematically. 2 Methods The model consists of populations of excitatory and inhibitory cells that are grouped in local recurrent circuits similar to those in previous simulation studies of neocortex (Douglas et al., 1995; Somers, Nelson, & Sur, 1995; Stemmler et al., 1995; Salinas & Abbott, 1996). Synaptic interactions within a group are both excitatory and inhibitory, but are restricted to inhibition between cell groups (see Figure 1). This synaptic organization mimics that found in a local sector of cortex, for example, a hypercolumn of V1, in which massively recurrent excitatory connections dominate nearby cells, whereas inhibitory interactions extend over larger distances. Given the topographic nature of cortical maps, this lateral inhibition interconnects adjacent groups of cells with dissimilar response speciŽcity, for example, with an orthogonal orientation preference in V1 (Kisv´arday & Eysel, 1993). In most simulations presented in section 3, two cell groups coupled through mutual inhibition are considered (target groups). Simulations involving three competing target groups are also reported. Target cells are driven by a population of neurons with similar intrinsic connectivity (input groups). In the model, each input group either projects to a single target group or provides common (divergent) activation to several target groups. Each group of cells comprises 800 excitatory units and 200 inhibitory units. Units of each type receive 80 excitatory connections and either 20 (input groups) or 12 (target groups) inhibitory connections from cells within the same group. Intergroup connections consist of 12 inhibitory contacts per cell between target groups. Input groups contribute 40 excitatory projections onto each unit in the target layer, with the proportion of common input varied systematically in the simulations. These speciŽcations yield a ratio of excitatory to inhibitory units, as well as proportions of excitatory and inhibitory synapses, that are consistent with anatomical data (Beaulieu, Kisv´arday, Somogyi, Cynader, & Cowey, 1992). Individual connections are established at random within and between interconnected populations of cells. These connections incorporate transmission delays with a mean of 2 ms.

184

Erik D. Lumer

Figure 1: Model architecture. Spiking units are grouped in cortical-like recurrent circuits (boxes), each comprising 800 excitatory (Ex) and 200 inhibitory (Inh) cells. Input groups can provide either independent or common drive to two target groups (1 and 2) that compete through mutual inhibition. Excitatory and inhibitory connections are represented by lines with arrowheads and round terminals, respectively.

Neurons are modeled as single-voltage compartments with realistic membrane time constants and Žring characteristics. SpeciŽcally, each cell’s membrane potential obeys the equation tm dV/ dt = ¡V C E0 ¡ gEx ( t)( V ¡ EEx ) ¡ gInh ( t)(V¡Einh ), where tm is the membrane time constant, E0 is the cell’s resting potential, gEx and gInh are, respectively, time-dependent excitatory and inhibitory synaptic conductances expressed in units of a leak conductance, and EEx and Einh are the reversal potentials of the corresponding channels. Membrane time constants are set to 16 ms for excitatory cortical units and 8 ms for inhibitory units. These passive membrane time constants are consistent with empirical data (Connors, Gutnick, & Prince, 1982; BloomŽeld, Hamos, & Sherman, 1987). E0 is ¡60 mV for both cell types, in agreement with in vivo measurements (Ferster & Jagadeesh, 1992). E Ex and Einh are 0 mV and ¡70 mV, respectively. Spiking is modeled by a standard integrateand-Žre mechanism, in which units that reach a threshold are reset to the potassium reversal potential (¡90 mV). Thresholds are ¡51 mV for excitatory units and ¡52 mV for inhibitory units. These threshold values fall in the in vivo range (Baranyi, Szente, & Woody, 1993). The choice of different Žring thresholds for these two cell types makes it easier to reproduce experimental levels of spontaneous activity, as described below. Absolute refractory periods are 1.5 ms. Relative refractory periods are generated in excitatory units by adding 10 mV to their Žring threshold after each spike; relaxation to baseline occurs exponentially with a 10-ms time constant. Similar cellular parameters have been justiŽed in previous simulations of striate cortex

Effects of Spike Timing on Winner-Take-All Competition

185

Table 1: Synaptic Channel Parameters. Receptor

gpeak (units of gleak )

t1 (ms)

t2 (ms)

E (mv)

AMPA GABAa

0.05 0.1

0.5 1

2.4 7

0 ¡70

(e.g., Somers et al., 1995; Lumer, Edelman, & Tononi, 1997). For simplicity, spike rate adaptation is not incorporated in excitatory units. Preliminary simulations showed that this choice does not affect the processes studied in this article. When a unit Žres, a spiking event is relayed to its postsynaptic targets. Synaptic activation is modeled as a transient conductance change, using dual exponential functions deŽned as (Wilson & Bower, 1989): g( t) = gpeak N ( e¡t/ t 1 ¡ e¡t/ t 2 ),

where t1 and t2 are the rising and decay time constants, N is a normalizing factor, and gpeak is the peak conductance elicited by a single spike. In the model, synapses provide fast inhibition and excitation, with time constants of activation consistent with empirical data in vitro (Otis & Mody, 1992; Stern, Edwards, & Sakmann, 1992) (see Table 1). At rest, peak postsynaptic potentials (PSPs) elicited by a single spike are 0.5 and 0.25 mV for excitatory and inhibitory synaptic channels, respectively. These PSPs are consistent with the higher end of experimental values (Mason, Nicholl, & Stratford, 1991; Komatsu, Nakajima, Toyama, & Fetz, 1988); large, singleevent PSPs must be used in the model because it includes fewer synapses per cell compared with in vivo connectivity. Background synaptic activity is simulated by a random (Poisson distributed) activation of both types of synaptic channels in each unit. This results in a balance of mean excitation and inhibition and a spontaneous discharge of individual units at a rate of about 3 Hz for excitatory units and 15 Hz for inhibitory units. In addition to this background activity, extrinsic activation of the network is achieved by applying a stochastic (Poisson distributed, l = 0.4 kHz) synaptic excitation independently to each cell in the input groups. This activation occurs with an amplitude equivalent to Žve synchronous excitatory postsynaptic potentials (EPSPs) per synaptic event. To clarify the inuence of relative spike timing on competition, the temporal coordination between neuronal responses in the two target groups is manipulated selectively. This is implemented by varying systematically the proportion of common versus independent activation that each target group receives. It must be emphasized that this manipulation of feedforward connectivity is not meant to represent any physiological mechanism, but rather to provide a simple control over the degree of coherence between competing populations of neurons. The question of what mechanisms can

186

Erik D. Lumer

produce dynamic changes in neuronal synchronization is beyond the scope of this study. SufŽce it to note that changes in relative spike timing do take place in a task and stimulus-dependent manner in vivo (Gray et al., 1989; Roelfsema, Engel, Konig, ¨ & Singer, 1997). A number of aggregate measures are used to characterize the model’s behavior. The average membrane potential is computed within each cell group. This variable, which is reminiscent of a local Želd potential, serves as a useful indicator of within-group coherence. Instantaneous populationaveraged Žring rates are computed by cumulating all of the spikes emitted by excitatory units within a given cell group and a sliding 100-ms time window and by normalizing for cell number and window size. Finally, the competition index of two groups of neurons is deŽned as the mean absolute value of the difference between their instantaneous Žring rates divided by the highest instantaneous rate among the two cell groups. This index varies between 0 for noncompeting groups of cells and 1 for groups of neurons with mutually exclusive patterns of activity (i.e., winner-take-all regime). Thus, it provides a useful measure of the degree of competition in the target layer. 3 Results Mutual inhibition in cortical circuits can both suppress neuronal responses and affect their temporal patterning by inducing synchronous oscillations. This is illustrated in Figure 2A, in which the population-averaged membrane potential in input and target groups is plotted, along with single-unit activity from representative target cells. As shown by the autocorrelation function, input populations exhibit weak rhythmic activity in the 60-Hz range (see Figure 2B). By contrast, target populations exhibit strong coherent oscillations with a frequency at around 30 Hz (see Figure 2C). The population oscillations can be shown to arise as an emergent property of recurrent synaptic coupling. This is because individual inhibitory neurons target several excitatory cells, forcing them to pause in concert; the silencing of excitatory neurons in turn causes a drop in inhibition, resulting in a depolarizing rebound and a synchronous volley of action potentials. The sequence repeats itself periodically at a pace that depends in part on the time constants of synaptic inhibition. Because this aspect of the neural dynamics has been investigated in detail in previous studies (Bush & Douglas, 1991; Whittington et al., 1995; Lumer et al., 1997), it is not further characterized here. The simulations also reveal that changes in spike coordination modify the effects of lateral inhibition that is not matched by excitation. We illustrate this by comparing the responses in a model with two target groups: when they sample their inputs from either a common neuronal pool or independent groups of cells. Although the mean activation of each target group remains identical in both cases, they yield different correlation patterns. In the Žrst instance, the common input provided to the target populations causes them to Žre in synchrony. With independent inputs, however, the

Effects of Spike Timing on Winner-Take-All Competition

187

Figure 2: Coherent activity in target groups. (A) The three upper plots show the population- averaged membrane potential of excitatory units in an input group (In) and in each target group (Pop1 and Pop2) over a period of 250 ms before and 750 ms during application of extrinsic activation to the model circuit. The remaining plots show the corresponding single-unit activity for representative cells in each target group. The arrow at the bottom marks the stimulation onset. Note the oscillations in the population-averaged responses, despite somewhat variable interspike intervals evidenced by the single unit traces. (B) Autocorrelogram of the mean membrane potential in the input group reveals a weak oscillation at around 60 Hz. (C) Autocorrelogram of the mean membrane potential in one target group reveals synchronous oscillations at around 30 Hz.

target populations exhibit little correlation in the timing of their action potentials. As shown in Figure 3A, these two input conditions yield markedly different behavior in the target layer. Independent activation of the target groups results in a winner-take-all competition, in which one group of cells exhibits elevated Žring rates, whereas responses in the other neuronal group are suppressed. This contrasts with the absence of response suppression in the target layer when target groups are driven by a common (synchronizing) input. Thus, synchronization of Žring prevents competition in recurrent neural circuits, despite the presence of strong mutual inhibition. The inuence exerted by spike coordination on winner-take-all mechanisms operates very rapidly. This is demonstrated in Figure 3B, in which a histogram of the mean absolute difference in Žring rate between the two target groups is shown immediately before and after a change from independent to common afferent input. The histogram reveals that a signiŽcant modulation of differential Žring levels is detectable within the Žrst 50 ms following the change in spike coordination. To quantify the sensitivity of neural competition to the degree of zerolag synchronization, I compute the competition index (CI), as deŽned in section 2, between two target groups for different proportions of common

188

Erik D. Lumer

Figure 3: Temporal control of winner-take-all mechanism. (A) Instantaneous Žring rates, as deŽned in section 2, are plotted for excitatory units in two competing target groups, using black and gray traces, respectively. The inputs to the two target groups are changed from common (sync) to separate (async) once every 500 ms. Uncorrelated inputs yield a pattern of response dominance and suppression, which is abolished when common inputs are used instead. Input conditions are shown below the activity traces. (B) Differential discharge rate histogram for the target groups during successive suppressed and coordinated time intervals (bin = 50 ms).

input. As shown in Figure 4A, the degree of competition between target groups rises rapidly as the proportion of common input, and hence of their spike correlation, decreases. Because competition in the model depends on the presence of strong mutual inhibition between target groups, the inuence of spike coordination can be interpreted as a modulation of the effective connectivity between these groups (Gerstein & Perkel, 1969; Friston, Frith, Little, & Frackowiak, 1993). This view is consistent with the results of simulations in which, for each setting of input sources, we determine the strength of intergroup synaptic inhibition that yields an equivalent degree of competition in the target layer—in this instance, CI = 0.4. This functionally equivalent inhibitory strength is plotted as a function of the proportion of common input provided to the target populations (see Figure 4B). To a Žrst approximation, it establishes a boundary between regions of the parameter space yielding, on the one hand, a winner-take-all-regime (above the boundary) and, on the other, a regime in which both target groups are activated simultaneously (below the boundary). Accordingly, physiological factors that transiently alter the efŽcacy of inhibitory postsynaptic potentials (e.g., neuromodulation) or input coherence (e.g., stimulus conŽguration) could switch the network dynamics between these two regimes. Finally, it must be noted that similar effects are seen when more than two

Effects of Spike Timing on Winner-Take-All Competition

189

Figure 4: Sensitivity of competition to spike timing. (A) The competition index (CI) is plotted for increasing proportions of common input to the two target groups. The CI for uncorrelated inputs to the two groups corresponds to the leftmost point in the Žgure. (B) The plot shows the peak conductance of intergroup inhibitory synapses yielding a constant CI (0.4), as a function of the proportion of common input to the two target groups. This plot separates the parameter space in a region of increased competition (above) and one of coordination (below).

competing groups are considered. This is illustrated in simulations of three competing populations of neurons that are driven by independent input groups (see Figure 5A) or by inputs that are common to two (see Figure 5C) or all of the target groups (see Figure 5B). As shown in the Žgures, target groups that receive a common input, and hence Žre synchronously, do not suppress each other ’s activity. This contrasts with the winner-take-all competition between populations with uncorrelated synaptic inputs. These simulations illustrate how dynamic changes in input correlation can reconŽgure the competition among target cells; neurons can switch their membership from one coactivated assembly to another on the basis of changes in relative spike timing. Thus, these results are consistent with the notion that the same group of neurons can be involved in the representation of multiple objects. At any one moment, the speciŽc binding of neuronal activity to single objects can be determined by a buildup of synchronization patterns and selection of neuronal responses that conform to such patterns.

190

Erik D. Lumer

Figure 5: Instantaneous Žring rates for three competing groups. (A) Independent activation of each group yields a winner-take-all regime, in which a single group maintains elevated Žring rates at any one moment. (B) Competition is prevented by the introduction of common inputs to all target populations. (C) Common input to two groups prevents them from competing with each other but not with a third uncorrelated group. Activity in the third group is suppressed in this example, but it could also dominate competition in some runs.

4 Discussion A prominent feature of the local circuitry in neocortex is its topographic organization in clusters, with massively recurrent excitation between neighbor cells and lateral inhibition that is more widely spread. Recent simulation studies suggest that this architecture yields stereotyped responses, in which weak afferent inputs undergo nonlinear ampliŽcation due to intracortical recurrent excitation and the strongest inputs are differentially activated (i.e., selected) as a result of lateral inhibitory interactions. The models show that such response properties can account for the emergence of direction and orientation selectivity in striate cortex and for the multiplicative gain modulation of parietal neurons by eye and head position signals (Douglas et al., 1995; Somers et al., 1995; Salinas & Abbott, 1996). The simulations here demonstrate that the basic design principles of cortical architectures can also produce stereotyped temporal patterns of neuronal activity, in particular synchronous and oscillatory population responses in the high-frequency range (20–70 Hz). More important, it has been shown that the winner-take-all competition that arises naturally in such circuits exhibits a remarkable sensitivity to the temporal coordination of neuronal action potentials. SpeciŽcally, the onset of a winner-takeall behavior can be prevented by a zero-lag synchronization of discharges among pools of neurons competing through mutual inhibition. Transitions

Effects of Spike Timing on Winner-Take-All Competition

191

from competitive to cooperative Žring regimes occur very rapidly—within roughly 50 msec of a change from uncorrelated to synchronous population activity. This novel network property reects the sensitivity of reciprocal inhibitory interactions to the relative timing of action potentials. Indeed, the simulations reported in this article show that the effective connectivity, of an inhibitory nature, mediating lateral interactions responsible for competition is decreased in the context of synchronization. This is an interesting and counterintuitive observation; increased functional connectivity (deŽned as the correlation between neurophysiological events) is, in this instance, accompanied by a decrease in (inhibitory) effective connectivity (deŽned as the inuence one neuronal system exerts over another). The effects of relative spike timing on competition described here are robust to changes in model parameters. In particular, these effects are maintained when the total number or relative proportion of excitatory and inhibitory units in the model are varied by a twofold factor with respect to the values used in the reported simulations. The network organization underlying these effects—a center of recurrent excitation with a inhibitory surround— does not take into account the long-range horizontal connections between pyramidal cells in upper and lower cortical laminae. However, these distal horizontal connections are organized in discrete patches placed outside the afferent region of inhibitory cells and restricted to cells of similar response speciŽcity (Lund, Yoshioka, & Levitt, 1993). Moreover, physiological evidence suggests that these long- range connections have a modulatory rather than a driving inuence on their neuronal targets (Hirsh & Gilbert, 1991). Thus, in Žrst approximation these nonlocal connections are not relevant for the local competition mechanisms considered in this article. These results may help in understanding how the brain selects among the welter of stimuli constantly competing for the control of attention and motor action. Physiologically, selective sensory processing is often accompanied by a suppression of neuronal responses to contextually irrelevant stimuli (Moran & Desimone, 1985; Motter, 1993; Treue & Maunsell, 1996). A candidate mechanism for the selective suppression of neuronal responses is mutual inhibition between cortical cells responsive for relevant and irrelevant stimuli (Chellazzi et al., 1993; Schall & Sanes, 1993). Because selection typically operates at the level of perceptual groups rather than simple features, however, such competition must be conŽned between cells belonging to different object representations. The results here suggest a mechanism whereby this speciŽcity of competitive interactions can be achieved dynamically. It exploits the fact that the relative timing of action potentials in sensory cortices can vary in a context-dependent manner, with synchronization of activity arising selectively among cells representing related features of an object (Gray et al., 1989; Singer, 1993). Thus, according to the effects reported in this article, the neuronal responses associated with various attributes of an object emerge with mutual immunity from competition by virtue of their synchronization. By contrast, the patterns of activity associated with differ-

192

Erik D. Lumer

ent objects or alternative stimulus interpretations may be uncorrelated and hence subject to competition. It has recently been proposed that such effects may underlie the competition between alternative stimulus representations during binocular rivalry (Lumer, 1998). More generally, they link, within a common computational framework, early processes of perceptual organization based on spike timing and subsequent selection among perceptual groups competing for the control of behavior. These newly described effects of spike coordination are functionally distinct from the ability of neuronal synchronization to augment the responses of target cells due to maximal temporal summation of synaptic events (Abeles, 1982). 5 Conclusion What emerges from these simulations is the notion of a parameter space for cortical operation that spans two domains. One domain is capable of supporting activity in multiple populations of neurons for a short period of time, after which one population is selected by competitive lateral interactions. In the other domain, neuronal responses in multiple populations can coexist, provided that they are expressed synchronously. The parameters deŽning this space can be divided into those that dynamically modulate synaptic efŽcacy (e.g., classical neuromodulation and voltage-dependent interactions) and the temporal patterning of activity among the interacting populations. I have focused on the latter here and have demonstrated how temporal coordination can move the network dynamics from one domain to the other. The role of temporal coordination in facilitating or attenuating competitive interactions in cortical circuits may represent a fundamental component of perceptual synthesis. Acknowledgments I thank Karl Friston and Richard Frackowiak for their encouragement and for helpful discussions and comments on the manuscript for this article. This research was supported by the Wellcome Trust. References Abeles, M. (1982). Role of cortical neuron: Integrator or coincidence detector? Isr J Med Sci, 18, 83–92. Baranyi, A., Szente, M., & Woody, C. (1993). Electrophysiological characterization of different types of neurons recorded in vivo in motor cortex of the cat. II. Membrane parameters, action potentials, current-induced voltage responses and electrotonic structures. J Neurophysiol, 69, 1865–1879. Beaulieu, C., Kisv a´ rday, Z., Somogyi, P., Cynader, M., & Cowey, A. (1992). Quantitative distribution of GABA-immunonegative neurons and synapses in the monkey striate cortex (area 17). Cerebral Cortex, 2, 295–309.

Effects of Spike Timing on Winner-Take-All Competition

193

BloomŽeld, S. A., Hamos, J. E., & Sherman, S. M. (1987). Passive cable properties and morphological correlates of neurones in the lateral geniculate nucleus of the cat. J Physiol, 383, 653–692. Bush, P., & Douglas, R. (1991). Synchronization of bursting action potentials in a model network of neocortical neurons. Neural Comp, 3, 19–30. Chellazzi, L., Miller, E. K., Duncan, J., & Desimone, R. (1993). A neural basis for visual search in inferior temporal cortex. Nature 363, 345–347. Cobb, S. R., Buhl, E., Halasy, K., Paulsen, O., & Somogyi, P. (1995). Synchronization of neuronal activity in hippocampus by individual GABAergic interneurons. Nature, 378, 75–78. Connors, B., Gutnick, N. I., & Prince, D. (1982). Electrophysiological properties of neocortical neurons in vitro. J Neurophysiol, 48, 1302-1320. Douglas, R. J., Koch, C., Mahowald, M., Martin, K., & Suarez, H. (1995).Recurrent excitation in neocortical circuits. Science, 269, 981–985. Ferster, D., & Jagadeesh, B. (1992). EPSP-IPSP interactions in cat visual cortex studied with in vivo whole-cell patch recording. J Neurosci, 12, 1262–1274. Friston, K. J., Frith, C. D., Little, P. F., & Frackowiak, R. S. J. (1993). Functional connectivity: The principal component analysis of large (PET) data sets. J Cereb Blood Flow Metab, 13, 5–14. Gerstein, G. L., & Perkel, D. H. (1969). Simultaneously recorded trains of action potentials: Analysis and functional interpretation. Science, 164, 828–830. Gray, C. M., Konig, ¨ P., Engel, A. K., & Singer, W. (1989). Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reects global stimulus properties. Nature, 338, 334–337. Hertz, J., Krogh, A., & Palmer, R. G. (1991). Introduction to the theory of neural computation. Reading, MA: Addison-Wesley. Hirsh, J., & Gilbert, C. (1991). Synaptic physiology of horizontal connections in the cat visual cortex. J. Neurosci, 11, 1800–1809. Humphreys, G. W., Romani, C., Olson, A., Riddoch, M. J., & Duncan, J. (1994). Non-spatial extinction following lesions of the parietal lobe in humans. Nature, 372, 357–359. Kastner, S., De Weerd, P., Desimone, R., & Ungerleider, L. G. (1998). Mechanisms of directed attention in the human extrastriate cortex as revealed by functional MRI. Science, 282, 108–111. Kisva´ rday, Z. F., & Eysel, U. T. (1993). Functional and structural topography of horizontal inhibitory connections in cat visual cortex. Europ J Neurosci, 5, 1558–1572. Koch, C., & Ullman, S. (1985).Shifts in selective attention: Towards an underlying neural circuitry. Human Neurobiology, 4, 219–227. Komatsu, Y., Nakajima, S., Toyama, K., & Fetz, E. (1988). Intracortical connectivity revealed by spike-triggered averaging in slice preparation of cat visual cortex. Brain Res, 442, 359–362. Lumer, E. D. (1998).A neural model of binocular integration and rivalry based on the coordination of action-potential timing in primary visual cortex. Cerebral Cortex, 8, 553–561. Lumer, E. D., Edelman, G. M., & Tononi, G. (1997). Neural dynamics in a model of the thalamocortical system. I. Layers, loops, and the emergence of fast

194

Erik D. Lumer

synchronous rhythms. Cerebral Cortex, 7, 207–227. Lund, J. S., Yoshioka, T., & Levitt, J. B. (1993). Comparison of intrinsic connectivity in different areas of macaque monkey cerebral cortex. Cerebral Cortex, 3, 148–162. Mason, A., Nicholl, A., & Stratford, K. (1991). Synaptic transmission between individual pyramidal neurons of the rat visual cortex in vitro. J Neurosci, 11, 72–84. Milner, P. M. (1974). A model of visual shape recognition. Psychol Rev, 81, 521– 535. Moran, J., & Desimone, R. (1985). Selective attention gates visual processing in the extrastriate cortex. Science, 229, 782–784. Motter, B. C. (1993). Focal attention produces spatially selective processing in visual cortical areas V1, V2, and V4 in the presence of competing stimuli. J Neurophysiol, 70, 909–919. Otis, T. S., & Mody, I. (1992). Differential activation of GABAa and GABAb receptors by spontaneously released transmitter. J Neurophysiol, 67, 227–235. Roelfsema, P. R., Engel, A. K., Konig, ¨ P., & Singer W. (1997). Visuomotor integration is associated with zero-lag synchronization among cortical areas. Nature, 385, 157–161. Salinas, E., & Abbott, L. F. (1996). A model of multiplicative neural responses in parietal cortex. Proc Natl Acad Sci USA, 93, 11956–11961. Schall, J. D., & Sanes, D. P. (1993). Neural basis of saccade target selection in frontal eye Želd during visual search. Nature, 366, 467–469. Singer, W. (1993). Synchronization of cortical activity and its putative role in information processing. Annu Rev Physiol, 55, 349–374. Somers, D., Nelson, S., & Sur, M. (1995). An emergent model of orientation selectivity in cat visual cortical simple cells. J Neurosci, 15, 5448–5465. Stemmler, M., Usher, M., & Niebur, E. (1995). Lateral interactions in primary visual cortex: A model bridging physiology and psychophysics. Science, 269, 1877–1880. Stern, P., Edwards, F. A., & Sakmann, B. (1992). Fast and slow components of unitary EPSCS on stellate cells elicited by focal stimulation in slices of rat visual cortex. J Physiol (London), 449, 247–278. Treue, S., & Maunsell, J. H. R. (1996). Attentional modulation of visual motion processing in cortical areas MT and MST. Nature, 382, 539–541. von der Malsburg, C. (1973). Self-organization of orientation sensitive cells in primary visual cortex. Kybernetik, 14, 85–100. von der Malsburg, C. (1981). The correlation theory of the brain (Internal Rep. No. 81-2). Gottingen: ¨ Max Planck Institute for Biophysical Chemistry. Whittington, M. A., Traub, R. D., & Jefferys, J. G. R. (1995). Synchronized oscillations in interneuron networks driven by metabotropic glutamate receptor activation. Nature, 373, 612–615. Wilson, M. A., & Bower, J. M. (1989). In C. Koch & I. Segev (Eds.), Methods in neuronal modeling (pp. 291–333). Cambridge, MA: MIT Press. Received April 21, 1998; accepted February 17, 1999.

LETTER

Communicated by George Gerstein

Model Dependence in QuantiŽcation of Spike Interdependence by Joint Peri-Stimulus Time Histogram Hiroyuki Ito Satoshi Tsuji Department of Information and Communication Sciences, Faculty of Engineering, Kyoto Sangyo University, Kita-ku, Kyoto 603-8555, Japan and CREST, Japan Science and Technology

Multineuronal recordings have enabled us to examine context-dependent changes in the relationship between the activities of multiple cells. The joint peri-stimulus time histogram (JPSTH) is a much-used method for investigating the dynamics of the interdependence of spike events between pairs of cells. Its results are often taken as an estimate of interaction strength between cells, independent of modulations in the cells’ Žring rates. We evaluate the adequacy of this estimate by examining the mathematical structure of how the JPSTH quantiŽes an interaction strength after excluding the contribution of Žring rates. We introduce a simple probabilistic model of interacting point processes to generate simulated spike data and show that the normalized JPSTH incorrectly infers the temporal structure of variations in the interaction parameter strength. This occurs because, in our model, the correct normalization of Žringrate contributions is different than that used in Aertsen, Gerstein, Habib, and Palm’s (1989) effective connectivity model. This demonstrates that Žring-rate modulations cannot be corrected for in a model-independent manner, and therefore the effective connectivity does not represent a universal characteristic that is independent of modulation of the Žring rates. Aertsen et al.’s (1989) effective connectivity may still be used in the analysis of experimental data, provided we are aware that this is simply one of many ways of describing the structure of interdependence. We also discuss some measure-independent characteristics of the structure of interdependence. 1 Introduction Recently, a great amount of attention has been focused on the understanding of network mechanisms of information processing in the brain. The advancement of multineuronal recording techniques has enabled us to record simultaneously the spike activities of multiple single units from spatially distant sites as well as from local cortical sites. The availability of multiple neuron spike data has led to a novel coding paradigm called the relational Neural Computation 12, 195–217 (2000)

c 2000 Massachusetts Institute of Technology °

196

Hiroyuki Ito and Satoshi Tsuji

code. In this paradigm, the information or function is not necessarily localized at the single-cell level. This information or function is dynamically encoded by the context of the relationship among spike activities of multiple cells. Such a relationship can be either a coherence in the mean Žring rates in a psychological timescale (Hebb, 1949) or a Žne temporal interdependence among spike events (Abeles, 1982a, 1991; von der Malsburg, 1981, 1986; Gerstein, Bedenbaugh, & Aertsen, 1989; Engel, Konig, ¨ Kreiter, Schillen, & Singer, 1992; Fujii, Ito, Aihara, Ichinose, & Tsukada, 1996). In order to understand the network mechanism involved in information processing, we investigate the relationship among multiple cells and discuss its relation within the context of information processing (stimuli presentation or task performance). Thus, we need a powerful analytical tool that visualizes the spatiotemporal statistical structure in the activities of the neuronal network based on the spike data of multiple single units. Traditionally, we have computed peri-stimulus time histograms (PSTH) for all of the recorded cells to examine coherence in the Žring rates. A crosscorrelogram has then been applied to a pair of cells to detect the temporal interdependence among spike activities. However, the cross-correlogram has a limitation in analyzing a rapid change of the structure of interdependence due to averaging over the sample interval. Recent experimental reports of a time domain analysis suggest that the spike interdependence between two cells evolves in a nonstationary way even within the trial duration of a few seconds (Gray, Engel, Konig, ¨ & Singer, 1992; Vaadia et al., 1995). Aertsen, Gerstein, Habib, & Palm, (1989) introduced a novel statistical analytical method for the spike trains of a pair of cells, the joint peri-stimulus time histogram (JPSTH). The JPSTH was designed to quantify the nonstationary modulation of the spike interdependence within the trial duration. Currently, the JPSTH is considered to be the most advanced tool for the statistical analysis of spike interdependence. The number of experimental reports applying the JPSTH has been increasing, and this increased interest reects the interest in the dynamical structure of interdependence within a cell assembly. Here we examine the mathematical structure of how the JPSTH quantiŽes a degree of interdependence after excluding the contribution of Žring rates. In section 2, we provide a basic description of the JPSTH method. Then, following the introduction of two different ways of using the JPSTH, we discuss how Aertsen et al. (1989) deŽned the effective connectivity to quantify the degree of interdependence. We present a statistical evaluation of the JPSTH in section 3. We introduce a simple probabilistic model of the point processes where the interaction strength is characterized by a different parameter from the effective connectivity. Through the normalization procedure of the JPSTH of the simulated spike data, we examine how the temporal structure of interdependence described by our interaction parameter is related to that by the effective connectivity. We discuss the results of

Model Dependence

197

the simulations in section 4. We summarize detailed mathematical derivations of the formula used in the text in the appendix. In this article, we consistently use the term interdependence instead of the well-used correlation because the latter is often confused with the correlation coefŽcient, which is a particular measure of interdependence. The essence of this study has been presented elsewhere (Ito & Tsuji, 1997; Ito, 1998). 2 J-PSTH 2.1 Method. By simultaneously recording spike activities of two cells, cell 1 and cell 2, for a trial duration T over K trials under the same experimental condition, we get K pairs of spike trains. Here we assume that the spike trains of each trial are time locked to some trigger event such as the stimulus onset. We divide the trial duration into multiple bins with a small interval D so that there is at most a single spike within any bin for each trial. ( ) Let ni k ( u) be the spike count of cell i (i = 1, 2) in the uth bin of the kth trial. ( ) Then, the variable ni k ( u) can take either 0 or 1. The average spike count, hni ( u)i =

K 1X n( k) ( u), K k= 1 i

(2.1)

represents the probability of the occurrence of the spike event at cell i at bin u. The histogram of hni ( u)i over index u represents the PSTH. In the computation of the raw-JPSTH, a square panel of a size of the trial duration T is prepared, and time axes are set along the bottom and the left sides. The two time axes are locked to the stimulus onset at the left-bottom corner. The square panel is subdivided into tiny compartments of size D , each of which is speciŽed by a pair of two integer indices (u, v). At each compartment, we assign the matrix element that quantiŽes the probability of the occurrence of a joint event, where cell 1 has a spike event at bin u and cell 2 has one at bin v, hn12( u, v)i =

K 1X ( k) n (u, v), K k= 1 12

(2.2)

where ( )

( )

( )

k k k n12 ( u, v) = n1 ( u) n2 (v).

(2.3)

An example of the raw-JPSTH is shown in the left panel in Figure 2a. The magnitude of each matrix element is represented by a color scale. At the bottom of the square, the PSTH of cell 1 is plotted upside down. The PSTH of cell 2 is plotted on the left axis.

198

Hiroyuki Ito and Satoshi Tsuji

The matrix elements on the main diagonal line represent the coincident spike events of the two cells. These elements are used for the PST coincidence histogram, which is plotted along the diagonal line in the right panel in Figure 2a. On the other hand, the summation within paradiagonal bins is used to estimate the ordinary cross-correlogram, which is plotted in the upper right corner in Figure 2a. The central peak of the cross-correlogram represents the cumulative frequency of coincident spike events in entire trial duration. The PST coincidence histogram visualizes Žne temporal modulation of the frequency of the coincident events within the trial duration, that is, the time domain of more frequent coincident events. In the raw-JPSTH, the statistical structure in the spike data is represented by three quantities: hn1 ( u)i, hn2 ( v)i, hn12 (u, v)i. The goal of the JPSTH is to provide a statistical measure extracting the net degree of interdependence from the joint probability hn12 (u, v)i after excluding the contribution of Žring rates. The element in the predicted-JPSTH is given by hpredicted-JPSTHi = hn1 ( u)ihn2 ( v)i,

(2.4)

which represents the probability of the accidentally correlated events that appear even when the two spike trains are statistically independent (null hypothesis). The departure of the test data from the null hypothesis is estimated by the covariance, which is a subtraction of the predicted-JPSTH from the raw-JPSTH at each matrix element, h(raw ¡ predicted )-JPSTHi = hn12 (u, v)i¡hn1 (u)ihn2 (v)i.

(2.5)

In general, there exist modulations of Žring rates within the trial duration. For a comparison of the net degree of spike interdependence over the different compartments, we must further normalize the covariance at each compartment by a factor depending on the Žring rates of the two cells. This normalization procedure is the critical issue in the formulation of the JPSTH. The main concern of this study is how to introduce an adequate normalization. 2.2 Standpoint of Analysis. There are two different ways of using the JPSTH to extract the degree of interdependence. In the Žrst way, we consider the JPSTH as an abstract statistical measure (statistic) of the spike interdependence between the two cells. The element in the raw-JPSTH is transformed into some statistical measure, such as the correlation coefŽcient. The measure is deŽned purely for statistical purposes and has nothing to do with the mechanism of the point processes generating the spike data. In this sense, this approach is essentially model free. The choice of a particular measure of interdependence is not unique, and various statistical measures of different characteristics (e.g., efŽcacy, contribution, correlation density, correlation coefŽcient, and so on) have been introduced (Brillinger, 1975; Palm, Aertsen, & Gerstein, 1988; Aertsen et al., 1989).

Model Dependence

199

The second way starts from an explicit deŽnition of a simple probabilistic model of the point processes, which is characterized by the Žring rates of the two cells and a parameter controlling the interaction strength between the spike events at the two cells. Let this parameter be called interaction mechanism strength. We Žrst solve a forward problem to obtain the relationship between these model parameters and the covariance of the spike data generated by this model. With this relation, the normalization of the covariance is deŽned to reproduce the model parameter of interaction mechanism strength. Therefore, the JPSTH works as a tool to facilitate the inference of the interaction mechanism strength from the given spike data. In this model-based approach, the model is introduced as a conventional tool (equivalent circuit) used for our description of the structure of spike interdependence. We attempt to interpret the structure of interdependence in the data through its correspondence to the phenomena in the particular model. Here again the choices of the model and the parameter of interaction mechanism strength are not unique. Aertsen et al. (1989) took a somewhat intermediate approach between the two approaches. They Žrst deŽned a speciŽc probabilistic spike model (Aertsen & Gerstein, 1985). The model was then used to generate spike data with the temporal modulations of both the Žring rates and the interaction mechanism strength intermingled. Subsequently, they searched for a way to normalize the JPSTH so that the modulation proŽle of the interaction mechanism strength of their model was qualitatively reproduced. After various normalization trials, they Žnally adopted the one that divided the (raw ¡ predicted)-JPSTH (covariance) by the product of the standard deviations of the Žring rates of the two cells. This normalization procedure coincided with the transformation of the raw-JPSTH into the correlation coefŽcient. Their choice of normalization was through empirical trial and error, not through the forward problem of their original spike model. Aertsen et al. (1989) calibrated their normalization by positively interdependent spike data from two cells with the same rate of low Žring. As we will show in section 3, the interaction mechanism strength of their original model is well approximated by the correlation coefŽcient only in such a limited parameter region. Aertsen et al. (1989) called the elements of their normalized-JPSTH the effective connectivity between the two cells. Logically there are two possible interpretations of the effective connectivity. In the model-free approach, the effective connectivity is nothing but a correlation coefŽcient. It is an abstract statistical measure of interdependence and has nothing to do with any model of point processes. In the model-based approach, the effective connectivity in the full parameter regime, represents a parameter of interaction mechanism strength of some other model than the original model by Aertsen and Gerstein (1985). The model, which is deŽned implicitly by their normalization, and its interaction mechanism strength (effective connectivity) coincide with those in the original model setup only in the limited parameter

200

Hiroyuki Ito and Satoshi Tsuji

Figure 1: Algorithm for imposing interaction mechanism strength a to independent spike trains of Žring rates r 1 and r 2 .

region. It is important to keep the two standpoints clearly separate to avoid a confusion in our discussion. Aertsen et al. (1989) stressed a model-based approach by regarding the effective connectivity as “a minimum neuronal circuit model that would reproduce such correlation measurement.” In the following study, we also apply the model-based approach to examine the characteristics of the effective connectivity. 3 Statistical Evaluation of the JPSTH Our main concern with the JPSTH is the characteristics of the effective connectivity’s method of estimating the temporal structure in the interaction parameter in the spike data. Its results are often taken as a universal estimate of interaction strength, independent of modulations in the Žring rates. In order to evaluate the adequacy of this estimate, we introduce another spike model where the interaction mechanism strength is characterized by a different parameter from the effective connectivity. Then we examine how the temporal structure of interdependence described by our parameter is related to that by the effective connectivity. 3.1 Model. We introduce a simple model of two cells. The generation of interdependent spike trains by the model is schematically depicted in Figure 1. First, spike trains of the Poisson randomness are generated for cell 1 and cell 2 with a mean Žring rate ofr 1 and r 2 , respectively (0 · r 1 , r 2 · 1).1 The two spike trains do not have any signiŽcant interdependence more than an accidental component. Then a positive interdependence is imposed 1 In actual numerical simulations, the trial duration is divided into multiple bins with a width D . Initially we assign a value 0 at each bin, representing the absence of a spike event. The value is replaced by 1 when the algorithm imposes a spike event at the corresponding bin. Thus, the spike timing is quantized with the unit D . The mean Žring rate r is the probability of Žnding a spike event at each bin.

Model Dependence

201

on the spike trains by the following algorithm. Let t1 ( i) be the occurrence time of the ith spike event in the spike train of cell 1. Consider the event at time t1 ( i) C t in the spike train of cell 2, where t is a delay interval of the interdependent spike events. When there is already a spike event at that time, leave it unaffected. However, when there is no spike, add a spike event with a probability of a, (0 · a · 1). In Figure 1, the inserted spike events are represented by thick lines. Repeat this procedure for all the spike events in the spike train of cell 1. As a result of those procedures, we can impose the spike interdependence of the speciŽed strength with the time delay t. Pairs of spike trains with the same statistical property are generated for multiple trials for statistical averaging. A negative interdependence between the two spike trains can be imposed by the reverse of the above procedure. When cell 2 has a spike event at time t1 (i) C t , remove it with a probability |a|, where ¡1 · a · 0. When there is no spike event at that instant, do nothing. The degree of interdependence is quantiŽed by the parameter of interaction mechanism strength a in our model. Our model is identical to the one by Aertsen and Gerstein (1985), except that they did not consider the case when cell 2 already has a spike event in the insertion of the interdependent spike. Thus, the two models become truly identical when the Žring rate of cell 2 is low. 3.2 Simulation. We show a typical example of the JPSTH analysis of the simulated spike trains. Figure 2a shows the raw-JPSTH of the spike trains with a trial duration of 300 msec. As seen in the two PSTHs, both cells had a lower Žring rate (r 1 = r 2 = 0.14) in the Žrst and last 100-msec intervals in the trial duration. The Žring rates of the two cells increased to a higher value (r 1 = r 2 = 0.32) in the intermediate 100-msec interval. However, the interaction mechanism strength was kept to be a negative constant value a = ¡0.8, and the time delay t was set to be 0 msec throughout the trial duration. Thus, cell 2 had a coincident spike Žring with cell 1 much less frequently than the purely accidental component. In the computation of the PST coincidence histogram in Figures 2a–c, we performed a smoothing of the diagonal matrix elements along the diagonal line by a moving average using a gaussian Žlter with s = 2 msec. Figures 2b and 2c, respectively, show the predicted-JPSTH and the (raw¡predicted)-JPSTH. The degree of departure from the null hypothesis (covariance) in the PST coincidence histogram in Figure 5c remains apparently inuenced by the nonstationary modulation of the Žring rates, even though the interaction mechanism strength was kept constant throughout the trial duration. 3.3 Normalization. We attempt to separate the modulation in the (raw¡ predicted)-JPSTH into two components: the modulation derived purely from the Žring rates and that of the interaction mechanism strength in the individual pair of spikes of the two cells. This procedure requires normalization of the matrix elements of the (raw ¡ predicted)-JPSTH by a factor depending

202

Hiroyuki Ito and Satoshi Tsuji

Figure 2: JPSTH of simulated spike data. (a) raw-JPSTH. (b) predicted-JPSTH. (c) (raw ¡ predicted)-JPSTH. Scales of color code of JPSTH, PST coincidence histogram, and cross-correlogram are different in the three plots. Minimum and maximum values of matrix elements in the three JPSTHs are: (a) 0.00, 0.220, (b) 0.00, 0.176, (c) ¡0.104, 0.124. Firing rates of two cells r 1 and r 2 had nonstationary modulation (0.14 ! 0.32 ! 0.14). Interaction mechanism strength was constant (a = ¡0.8, t = 0 msec). PST coincidence histograms were smoothed using a gaussian Žlter with s = 2 msec. (T = 300 msec, D = 1 msec, K = 50).

Model Dependence

203

on the Žring rates of the two cells. At Žrst, we apply the normalization procedure by Aertsen et al. (1989) that transforms the matrix element into the effective connectivity: hnormalized ¡ JPSTHi =

h(raw ¡ predicted )-JPSTHi p , r 1 (1 ¡ r 1 )r 20 (1 ¡ r 20 )

(3.1)

where r 1 = hn1 ( u)i and r 20 = hn2 ( v)i are the Žring rates of cell 1 at the uth bin and that of cell 2 at the vth bin, respectively. The prime is added to r 20 , which is not to be confused with r 2 , the Žring rate of the initial spike train. Figure 3a shows the normalized-JPSTH computed by equation 3.1. We omit the display of the matrix elements. It is evident that this normalization does not work with our data. The inferred interaction mechanism strength in the PST coincidence histogram still has an apparent modulation by the Žring rates. This discrepancy derives from the fact that our interaction mechanism strength, parameter a, is different from that in the model setup by Aertsen et al., (1989), the effective connectivity. Recall that Aertsen et al., (1989) introduced the effective connectivity as a parameter of interaction mechanism strength in their implicit model setup. The temporal structure of modulations in the model-based parameter is different when we apply a different assumption about the model and the interaction mechanism strength that led to the data. In order to Žnd the proper normalization of the JPSTH that reproduces our model parameter, we need to know the relationship between the (raw ¡ predicted)-JPSTH (covariance) and the model parameters. This is how sensitively the raw-JPSTH departs from the predicted-JPSTH when the interaction mechanism strength a is imposed on the two independent spike trains having Žring rates of r 1 and r 2 . This relationship is obtained only in the forward problem from the spike model. Thus, the computation of the normalized-JPSTH inevitably becomes model dependent. As summarized in the appendix, straightforward calculations give the following expressions for our spike model: h(raw ¡ predicted )-JPSTHi = r 1 (1 ¡ r 1 )(1 ¡ r 2 )a,

(for positive interdependence).

(3.2)

By adding the interdependent spikes, the Žring rate of cell 2 was increased to r 20 = r 2 Cr 1 (1 ¡ r 2 )a.

(3.3)

Note that only the quantities r 1 , r 20 , and h(raw ¡ predicted)-JPSTHi are available in the analysis of the experimental data. We need to compute the two unknown parameters, r 2 and a, by solving the simultaneous equations (3.2) and (3.3). Substitution of equation 3.3 into equation 3.2 leads to h(raw ¡ predicted )-JPSTHi =

¡

¢

a r 1 (1 ¡ r 1 )(1 ¡ r 20 ). 1 ¡ ar 1

(3.4)

204

Hiroyuki Ito and Satoshi Tsuji

Figure 3: Normalized-JPSTH of simulated spike data applying (a) the form by Aertsen et al. (1989) and (b) our form. Scales of PST coincidence histogram and cross-correlogram are different in the two plots. The display of matrix elements is omitted, and PST coincidence histograms are smoothed using a gaussian Žlter with s = 2 msec.

The normalization of the (raw ¡ predicted)-JPSTH is attained by solving the above algebraic equation for a. For negative interdependence (¡1 · a · 0), we get h(raw ¡ predicted )-JPSTHi = r 1 (1 ¡ r 1 )r 2 a,

(for negative interdependence).

(3.5)

Model Dependence

205

Substitution of the relationship r 20 = r 2 Cr 1r 2 a results in h(raw ¡ predicted )-JPSTHi =

(3.6)

¡

¢

a r 1 (1 ¡ r 1 )r 20 . 1 C ar 1

(3.7)

The normalized-JPSTH applying equation 3.7 is shown in Figure 3b. Now, the PST coincidence histogram shows almost constant interaction mechanism strength throughout the trial duration. The mean value averaged over the whole trial duration, ¡0.78, has a good quantitative agreement with the value of the original spike data, ¡0.8. 3.4 Comparison to Aertsen et al.’s (1989) Normalization. In our modelbased formulation, the Žring rates before adding the interdependent spikes r 1 , r 2 , and the interaction mechanism strength a are the independent variables, which we can set arbitrarily. From equations 3.2 and 3.5, the (raw ¡ predicted)-JPSTH was expressed in a simpler form by using these variables. In Aertsen et al.’s (1989) formulation with an implicit model setup, on the other hand, the Žring rates of the interdependent spike data, r 1 , r 20 , and the effective connectivity are the independent variables. Their normalization factor was given by (Aertsen et al.’s (1989) normalization) =

q r 1 (1 ¡ r 1 )r 20 (1 ¡ r 20 ) ,

(3.8)

for both positive and negative interdependence. The different independent variables between the two formulations prevent us from a straightforward comparison. From equations 3.4 and 3.7, we can obtain the corresponding factors in our formulation: (Our normalization for positive interdependence) r 1 (1 ¡ r 1 )(1 ¡ r 20 ) = , 1 ¡ ar 1

(3.9)

(Our normalization for negative interdependence) r 1 (1 ¡ r 1 )r 20 = . 1 C ar 1

(3.10)

In this format, the normalization factor itself depends on the interaction mechanism strength a. This is not a problem when we consider parameter a as the solution of equation 3.4 or 3.7 rather than an unknown variable.

206

Hiroyuki Ito and Satoshi Tsuji

Figure 4: Dependence of normalization factor on Žring rate r with (a) positive interdependence and (b) negative interdependence. Firing rates of two cells r 1 andr 20 are constrained to ber . Graphs are plotted for different values of a: (a) 0.0, 0.2, 0.4, 0.6, 0.8, 1.0; (b) 0.0, ¡0.2, ¡0.4, ¡0.6, ¡0.8, ¡1.0 (Thick lines represent normalization factor by Aertsen et al. (1989); the two panels have different scales in their ordinates).

Showing the difference between the normalization factors of the two formulations, we plot their graphs as a function of Žring rate r in Figure 4 for the positive interdependence and the negative interdependence, respectively. Here, we consider only a special case with r 1 = r 20 = r for simplicity. In each panel, the graphs are plotted for different values of a, and the thick line represents the normalization factor by Aertsen et al., (1989) that is, r (1 ¡r ). Note that the ordinate has a different scale in the two panels. Since the (raw ¡ predicted)-JPSTH is an estimate of the covariance between the probabilities of a spike event at the two cells, it becomes zero when cells either

Model Dependence

207

always Žre (r = 1) or never Žre (r = 0), and so does the normalization.2 For positive interdependence, the graphs are rather similar at lower Žring rates (r · 0.1). Our normalization coincides with that by Aertsen et al. (1989) when a = 1. For negative interdependence, on the other hand, they are very different even at lower Žring rates. Therefore the difference in the normalized-JPSTH between our result and that of Aertsen et al. (1989) becomes more prominent with negative interdependence. In the general parameter region of lower Žring rates (r 1 , r 20 ¿ 1), the normalization factors are approximated as (Aertsen et al.’s (1989) normalization) ’

q r 1r 20 ,

(3.11)

and (Our normalization for positive interdependence) ’ r 1 ,

(Our normalization for negative interdependence) ’ r 1r 20 .

(3.12) (3.13)

Our spike model is identical to the one by Aertsen and Gerstein (1985) in this parameter region. Thus, we see that Aertsen et al.’s (1989) normalization (effective connectivity) can reproduce the interaction mechanism strength of their original spike model only when r 1 = r 20 ¿ 1 for positive interdependence. This was exactly the parameter region that they used for simulating spike data to calibrate their normalization procedure (Aertsen et al., 1989). For negative interdependence, their normalization always underestimates the interaction mechanism strength of the original model. This asymmetry between a positive and negative interdependence is an intrinsic property of the point processes of the spike events (Palm et al., 1988) and was the main conclusion of the study by Aertsen and Gerstein (1985) on the cross-correlogram. 3.5 Simulation of Nonstationary Interdependent Structure. We examined another example where the interaction mechanism strength showed a nonstationary modulation but the Žring rates of the two cells were kept constant. The initial Poisson random spike trains were generated with a Žring rate of r 1 = r 2 = 0.32 throughout the trial duration of 300 msec. Then, in the intermediate 100-msec interval, positively interdependent spike events were added to the spike train of cell 2 by imposing a strength of a = 0.3. The time delay was set at t = 0 msec. The interdependent spikes were not added in the interval of the Žrst and last 100 msec where a = 0. Only the 2 When a = ¡1, our normalization factor has a singularity at r = 1, which derives from our enforcement of its expression by r 20 . Since r 1 = 1 and a = ¡1, every initial spike event in cell 2 should be removed by the algorithm. This is impossible under the current constraint of r 20 =1.

208

Hiroyuki Ito and Satoshi Tsuji

Figure 5: Normalized-JPSTH of simulated spike data applying our normalization. Data show stationary Žring rates of two cells (r 1 = r 2 = 0.32) and nonstationary modulation of interaction mechanism strength within trial duration (a: 0 ! 0.3 ! 0, t = 0 msec). Display of matrix elements omitted and PST coincidence histogram smoothed using a gaussian Žlter with s = 2 msec. (T = 300 msec, D = 1 msec, K = 50.)

PST coincidence histogram of the normalized-JPSTH applying our normalization is shown in Figure 5 to demonstrate that the nonstationary structure of interdependence is faithfully reproduced. The mean value of the interaction mechanism strength averaged over the intermediate 100-msec interval is 0.32 and close to the original value. 4 Discussion 4.1 Model Dependence. We could correctly normalize the JPSTH in Figure 3b so as to reproduce our model parameter. This was because we knew that the given spike data were initially generated by the same model. However, if we start our analysis only from the spike data without knowing the spike model, as in the analysis of experimental data, the model selected would no longer be unique. There are other possible models that generate spike data with the same statistical structure as the given data. The parameter of interaction mechanism strength quantifying the degree of interdependence could be deŽned differently in each model, so we need a different normalization procedure to reproduce its parameter. Thus, when we apply a different assumption about the model or mechanism that led to the data, the temporal structure of modulations in that model-based parameter (interaction mechanism strength) is different. The quantiŽcation of the degree

Model Dependence

209

of interdependence through the inference of the model-based parameter is an ill-posed inverse problem without any unique solution. Aertsen et al. (1989) explicitly cautioned against a misunderstanding that the temporal modulation of the effective connectivity represents a universal characteristic, which is independent of the modulation of the Žring rates. However, this common misunderstanding still seems to exist. Like our parameter a, the effective connectivity is not a universal measure of the spike interdependence. It is just one of many possible ways of describing the degree of interdependence. 4.2 Logic in the Determination of Measures of Interdependence. Up to this point, we retained a model-based approach in order to examine the characteristic of the effective connectivity. On the other hand, as explained in section 2, a variety of model-free measures of interdependence have been suggested. Although each abstract statistical measure is mathematically well deŽned, there exists a nonuniqueness in our choices for a particular measure adopted for characterizing the structure of interdependence in the given spike data. Of course, when the model-free measures of different characteristics (e.g., correlation density and correlation coefŽcient) are applied to the same data, we get qualitatively different modulation proŽles of the degree of interdependence (Aertsen et al., 1989). Those descriptions can be used to study different aspects of the structure of spike interdependence. The logical structure in the nonuniqueness of the measure of interdependence can be clearly understood by applying an elemental concept borrowed from the information geometry (Amari, 1985, 1998). At each compartment ( u, v) in the JPSTH, a set of joint probabilities fpij g = ( p00 , p01 , p10, p11 ) can be computed from the spike data. The quantity pij represents the joint probability of the event: the spike count at the uth bin of cell 1 is i (0 or 1) and that at the vth bin of cell 2 is j. A particular set of probabilities fpij g can be regarded as a point in the space of the probability distribution under a constraint p00 Cp01 Cp10 Cp11 = 1. Characterization of the statistical structure in the set fpij g is performed by a transformation into a new coordinate set composed of the statistical measures. Initially, the Žring rates of the two cells, r 1 = p10 C p11 and r 20 = p01 C p11, are selected as universal measures. Then the set fpij g is represented by the new coordinate set (r 1 , r 20 , p11 ), which is exactly the raw-JPSTH. In this article, we formulated our measure of interdependence through the normalization of the covariance, p11 ¡r 1r 20 . However, the measure j can be introduced in a more general form by imposing a particular relation, j = j (r 1 , r 20 , p11 ), satisfying the mathematical condition of one-to-one mapping.3 Then the set fpij g can be represented by the coordinate 3 Actually, the measure proposed by Amari (1998) in the framework of information p p geometry has a form j = log p 00 p 11 . 01 10

210

Hiroyuki Ito and Satoshi Tsuji

system (r 1 , r 20 , j ), which corresponds to the normalized-JPSTH. Just as there is nonuniqueness in the selection of the coordinate system in the Euclidean space, there are inŽnite possibilities in the selection of such a measure of interdependence. Any measure is equally adequate as long as the coordinate system (r 1 , r 20 , j ) uniquely indicates the universal point fpij g. The value of the coordinate j (degree of interdependence) has only a relative meaning to the selected coordinate system. We can discuss the variations of the degree of interdependence within the trial duration by applying a particular measure. However, the description does not have a universal meaning independent of the choice of measure. The interrelationship between the descriptions by the different measures can be examined, as we did with the numerical simulation in the last section. Although the measure of interdependence is a model parameter of interaction mechanism strength in the model-based approach, it is an abstract statistical measure (statistic) in the model-free approach. However, the two approaches share the same logical structure that for determination of a measure j , we need to impose some constraint or criterion on the value leading to a particular relation j = j (r 1 , r 20 , p11 ). In the model-based approach, the introduction of a particular model of the point processes imposes a speciŽc parameter of interaction mechanism strength j . Aertsen et al. (1989) introduced an implicit model and deŽned their model-based measure (effective connectivity) under the condition that in the limited parameter regime, it approximated the interaction mechanism strength of another explicit model proposed by them previously. We also introduced a similar explicit model and formulated a JPSTH normalization procedure that reproduces our parameter of interaction mechanism strength in the entire parameter regime. In the model-free approach, on the other hand, the selection of a particular measure should be guided by a purely statistical value (Brillinger, 1975; Palm et al., 1988). Among various statistical measures having a symmetrical dependence on the Žring rates of the two cells, the correlation coefŽcient is most often used because of its established popularity. Amari (1998) suggested that a measure can be derived in a less subjective manner in the mathematical framework of information geometry by imposing a particular measure of distance between the points in the space of the probability distribution fpij g. 4.3 Universal Characteristics of Structure of Interdependence. Although the modulation proŽle of the degree of interdependence itself is measure dependent, we can discuss universal characteristics in special cases. The Žrst case is when there is a signiŽcant modulation in the degree of interdependence within the trial duration, despite the Žring rates of the two cells showing little modulation. We have simulated such a case in Figure 5. Another case is when the modulation proŽle of the degree of interdependence shows a signiŽcant change between the different contexts of the experiment (different stimuli or tasks), but the PSTH of each cell remains unchanged.

Model Dependence

211

In these cases, at least the fact that there exists a change in the degree of interdependence is a measure-independent characteristic. An example was introduced by Vaadia et al. (1995) in the recordings of two cells in the prefrontal cortex of a monkey. In both of the two different paradigms in the delayed localization task, there was very little modulation of the Žring rate in both cells throughout the trial duration. Nevertheless, the JPSTH of Aertsen et al.’s (1989) normalization revealed a nonstationary modulation of the effective connectivity on the order of a few tens of milliseconds. Interestingly, the modulation showed signiŽcantly different proŽles between the two paradigms. The modulation of the Žring rates and its inuence on the degree of interdependence have the same timescale as seen in Figure 3. Thus, if there is a statistically signiŽcant modulation of the degree of interdependence at a different (possibly, faster) timescale than that of the Žring rates, it would be an indication of a universal statistical characteristic in the structure of interdependence. In any measure of interdependence, the statistical independence between the two spike trains is assigned a null value. Then the sign of the interdependence (positive or negative) has a universal meaning. By adopting any measure of interdependence, we can statistically test whether the test data depart signiŽcantly from the null hypothesis of statistical independence. Thus, the standard cross-correlogram and unitary event analysis (Riehle, Grun, ¨ Diesmann, & Aertsen, 1997), by which we look for signiŽcantly interdependent events, are basically a model-free analysis. However, the same difŽculty as with the JPSTH arises whenever we attempt to normalize the correlogram for a comparison of the degree of interdependence between the data of different Žring rates (Abeles, 1982b). Nevertheless, even if we apply a different normalization procedure, it simply changes the overall scale of the frequency in the histogram, so that the structure of the cross-correlogram remains qualitatively the same. However, different normalization procedures in the JPSTH can lead to results that are qualitatively different. 4.4 Future Problems. We have avoided introducing a large-scale model composed of physiologically plausible units like a detailed compartment model. Our simple probabilistic model was nothing more than an abstract algorithm for imposing spike interdependence. In addition, the model parameters used in the simulations were not necessarily within the physiologically relevant range of the actual cortical system, which has low Žring rates and weak neuronal connectivity. This is because we introduced our model as a conventional tool to describe structure of interdependence rather than as a simulator of realistic cortical dynamics. The following two tasks can be independently carried out: the characterization of the statistical structure of spike data and solving the inverse problem to determine what actual neuronal network mechanism realizes the observed statistical characteristics.

212

Hiroyuki Ito and Satoshi Tsuji

For future research, it would be valuable to simulate an in virtu experiment (Aertsen, 1988; Aertsen, Erb, & Palm, 1994) using a physiologically plausible network model on a larger scale. We could numerically simulate the dynamics of the network and perform a virtual recording of the spike activities of only two cells. Then we would infer the effective connectivity by the JPSTH as we do for the actual experimental data. Contrary to the actual experiment, all numerical data would be available on the activities of other “nonrecorded” cells. We could then discuss how the global activities of the network inuence the temporal modulation of the degree of interdependence between the two cells. Such knowledge is important for the inference of network activities based on the real experimental data without knowing the activities of other nonrecorded cells. Also, it is valuable to execute systematic evaluations of what measures are relevant for detecting particular network events. Melssen and Epping (1987) investigated the adequacy of cross-correlation procedures for detecting and estimating neuronal connectivity by computer simulations of small networks composed of fairly realistic model neurons. They demonstrated that besides the strength of connectivity and the Žring rate, the neurons’ internal state—the nonlinear spike-generating characteristics—also inuenced the visibility and detectability of neuronal connectivity. In their in virtu experiment, they examined how nonlinear superposition of the stimulus-driven activities degraded the reliability of estimations of neuronal connectivity. Their results suggested that there is no universal procedure to estimate adequately the neuronal connectivity equally for all the modes of neuronal activities. This conclusion is supported also by our current study. Throughout this study, we assume that the bin width of the JPSTH is small enough that no bin ever holds more than one spike event per trial. However, in many analyses of experimental data, this assumption is not made. In the actual recordings, the Žring rates are low, and neurons are recorded from only during Žnite numbers of trials. Neurons nevertheless sometimes Žre in bursts. Therefore, if bins small enough to guarantee the above condition were used, most bins would be empty, and the structure in the JPSTH would be very scattered and difŽcult to visualize. Broader bins are used for smoothing the scattered structure in the JPSTH. We stress that our conclusion of the JPSTH has a broader nature that is not dependent on this assumption of the bin width. 5 Conclusion We examined the mathematical structure of how the JPSTH quantiŽes a degree of interdependence between the spike events of the two cells after excluding the contribution of Žring rates. We introduced a simple probabilistic model of interacting point processes to generate simulated spike data and showed that the normalized JPSTH incorrectly inferred the temporal structure of variations in the interaction parameter strength. This occurred

Model Dependence

213

because, in our model, the correct normalization of Žring rate contributions was different from that used in Aertsen et al.’s (1989) “effective connectivity” model. This demonstrated that Žring-rate modulations cannot be corrected for in a model-independent manner. Finally, our purpose is not to encourage using our normalization form instead of Aertsen et al.’s (1989) standard normalization. It is to provide the awareness that no model parameter of spike interdependence—neither Aertsen et al.’s (1989) effective connectivity nor ours—represents a universal characteristic that is independent of modulations of Žring rates. Aertsen et al.’s (1989) effective connectivity may still be used in the analysis of experimental data, provided that there is an awareness that this is simply one of many ways of describing the structure of interdependence. We recommend Žrst characterizing the statistical structure of interdependence by applying an a priori model-free measure (correlation coefŽcient). Then its results are used to infer the interaction mechanism strength of a particular model after assumptions about the interaction mechanism are made. Appendix:

Expression of (raw ¡ predicted)-JPSTH

The proper normalization of the (raw ¡ predicted)-JPSTH can be obtained by the relationship between the JPSTH and the model parameters. A.1 Positive Interdependence. We consider the case when the algorithm imposes spike interdependence with a strength a and a delay t on the two independent random spike trains of mean Žring rates r 1 and r 2 . Owing to the constraint that the width of the temporal bin should contain no more than one spike event, the mean Žring rate represents a probability of Žnding a spike event at each bin. Let ni ( u) be a variable representing an occurrence of a spike event at cell i (i = 1, 2) at the uth bin. It takes either 1 or 0 depending on whether there is a spike. The matrix element of the rawJPSTH at (u, u C t ) is given by a probability: P[n1 (u) = 1 ^ n2 (u C t) = 1], where P[A ^ B] represents the joint probability of two events A and B. This is computed by P[n1 ( u) = 1^n2 ( u Ct ) = 1] = P[n1 ( u) = 1]P[n2 ( u Ct ) = 1 | n1 ( u) = 1],

(A.1)

where P(A | B) represents the conditional probability of event A under the condition of event B. In the spike trains before adding interdependent spikes, two events n1 (u) = 1 and n2 ( u C t ) = 1 are independent so that P[n1 (u) = 1] = r 1 and P[n2 (u C t ) = 1 | n1 ( u) = 1] = P[n2 ( u C t) = 1] = r 2 . Then, P[n1 ( u) = 1 ^ n2 (u C t ) = 1] = r 1r 2

(before imposing interdependence).

214

Hiroyuki Ito and Satoshi Tsuji

On the other hand, after the algorithm adds interdependent spikes, there are two possibilities for the occurrence of event n2 ( u C t ) = 1 under the condition of n1 ( u) = 1: 1. n2 ( u C t ) = 1 already in the initial spike train.

2. n2 ( u C t ) = 0 in the initial spike train; then a spike is added with the probability a. The conditional probability of the Žrst event is given by r 2 and that of the second event by (1 ¡ r 2 )a. Since the two events are exclusive, we get P[n2 ( u C t ) = 1 | n1 ( u) = 1] = r 2 C (1 ¡ r 2 )a.

(A.2)

Substitution of this result into equation A.1 leads to the matrix element of the raw-JPSTH, P[n1 ( u) = 1 ^ n2 ( u C t ) = 1] = r 1 fr 2 C (1 ¡ r 2 )ag,

(A.3)

hraw-JPSTHi.

By adding interdependent spikes, the algorithm increased the Žring rate of cell 2, which can be computed by a relation, P[n2 ( u C t ) = 1] = P[n1 ( u) = 0]P[n2 ( u C t ) = 1 | n1 ( u) = 0]

C P[n1 ( u) = 1]P[n2 (u C t) = 1 | n1 ( u) = 1].

Substitutions of P[n2 ( u Ct ) = 1 | n1 ( u) = 0] = r 2 and equation A.2 into the above relation result in P[n2 ( u C t ) = 1] = (1 ¡ r 1 )r 2 Cr 1 fr 2 C (1 ¡ r 2 )ag = r 2 Cr 1 (1 ¡ r 2 )a.

The matrix element of the predicted-JPSTH is given by the probability of an accidental joint event between the two independent spike trains with rates r 1 and r 20 = r 2 Cr 1 (1 ¡ r 2 )a, that is, P[n1 ( u) = 1 ^ n2 ( u C t ) = 1] = r 1r 20 = r 1 fr 2 Cr 1 (1 ¡ r 2 )ag,

(A.4)

hpredicted-JPSTHi.

Finally, the (raw ¡ predicted)-JPSTH is given by subtracting equation A.4 from equation A.3, d P[n1 ( u) = 1 ^ n2 ( u C t ) = 1] = r 1 (1 ¡ r 1 )(1 ¡ r 2 )a, h(raw ¡ predicted)-JPSTHi.

Model Dependence

215

A.2 Negative Interdependence. After the algorithm removes interdependent spikes from the spike train of cell 2, there is a unique possibility for the occurrence of event n2 (u C t ) = 1 with the condition n1 ( u) = 1. That is, if n2 ( u C t) = 1 in the initial spike train, then the spike is not removed with a probability 1 ¡ |a|, where ¡1 · a · 0. The probability of this event is given by P[n2 ( u C t ) = 1 | n1 (u) = 1] = r 2 (1 ¡ |a| ).

(A.5)

Substitution of this result into equation A.1 leads to the matrix element of the raw-JPSTH, P[n1 ( u) = 1 ^ n2 ( u Ct ) = 1] = r 1r 2 (1 ¡ |a|),

hraw-JPSTHi.

(A.6)

The Žring rate of cell 2 was decreased by the removal of interdependent spikes so that P[n2 ( u C t ) = 1] = (1 ¡ r 1 )r 2 Cr 1r 2 (1 ¡ |a|) = r 2 ¡ r 1r 2 | a|.

Then the matrix element of the predicted-JPSTH is given by P[n1 ( u) = 1 ^ n2 (u C t ) = 1] = r 1 (r 2 ¡ r 1r 2 |a|),

(A.7)

hpredicted-JPSTHi.

Subtraction of equation A.7 from equation A.6 gives the Žnal result dP[n1 (u) = 1 ^ n2 ( u C t ) = 1] = ¡r 1 (1 ¡ r 1 )r 2 |a|,

h(raw ¡ predicted )-JPSTHi.

Acknowledgments We thank S. Amari for his important comments. We also express our thanks to M. Abeles for his comments and encouragement. We thank L. Kay for critical readings of the manuscript. Finally, we are grateful to anonymous referees for numerous instructive comments helping us to improve the focus of the article. References Abeles, M. (1982a). Local cortical circuits. Berlin: Springer-Verlag. Abeles, M. (1982b). QuantiŽcation, smoothing, and conŽdence limits for singleunits’ histograms. Journal of Neuroscience Methods, 5, 317–325.

216

Hiroyuki Ito and Satoshi Tsuji

Abeles, M. (1991). Corticonics. Cambridge: Cambridge University Press. Aertsen, A. (1988).Discourse on current issues in multi-neuron studies. In W. von Seelen, G. Shaw, & U. M. Leinhos (Eds.), Organization of neural networks (pp. 87–107). Weinheim: VHC Verlag. Aertsen, A., Erb, M., & Palm, G. (1994). Dynamics of functional coupling in the cerebral cortex: An attempt at a model-base interpretation. Physica D, 75, 103–128. Aertsen, A. M. H J., & Gerstein, G. L. (1985).Evaluation of neuronal connectivity: Sensitivity of cross-correlation. Brain Research, 340, 341–354. Aertsen, A. M. H. J., Gerstein, M. K., Habib, M. K., & Palm, G. (1989). Dynamics of neuronal Žring correlation: Modulation of “effective connectivity.” Journal of Neurophysiology, 61, 900–918. Amari, S. (1985). Differential-geometrical methods in statistics. Berlin: SpringerVerlag. Amari, S. (1998). Information geometry of neuro-manifolds. In S. Usui & T. Omori (Eds.), Proceedings of ICONIP’98 (pp. 591–593). Amsterdam: IOS Press. Brillinger, D. R. (1975). The identiŽcation of point process systems. Ann. Prob., 3, 909–929. Engel, A. K., Konig, ¨ P., Kreiter, A. K., Schillen, T. B., & Singer, W. (1992).Temporal coding in the visual cortex: New vistas on integration in the nervous system. Trends in Neuroscience, 15, 218–226. Fujii, H., Ito, H., Aihara, K., Ichinose, N., & Tsukada, M. (1996). Dynamical cell assembly hypothesis—Theoretical possibility of spatio-temporal coding in the cortex. Neural Networks, 9 (8), 1303–1350. Gerstein, G. L., Bedenbaugh, P., & Aertsen, A. M. H. J. (1989). Neuronal assemblies. IEEE Transactions in Biomedical Engineering, 36, 4–14. Gray, C. M., Engel, A. K., Konig, ¨ P., & Singer, W. (1992). Synchronization of oscillatory neuronal responses in cat striate cortex: Temporal properties. Visual Neurosciences, 8, 337–347. Hebb, D. O. (1949). The organization of behavior. New York: Wiley. Ito, H. (1998). Need for development of statistical analysis tools of dynamical neuronal code—Statistical evaluation of joint-PSTH. In S. Usui & T. Omori (Eds.), Proceedings of ICONIP’98 (pp. 175–178). Amsterdam: IOS Press. Ito, H. & Tsuji, S. (1997). Statistical evaluation of joint peri-stimulus time histogram for analysis of dynamical neuronal code. Soc. Neurosci. Abstr., 23, 1400. Melssen, W. J., & Epping, W. J. M. (1987). Detection and estimation of neural connectivity based on crosscorrelation analysis. Biol. Cybern., 57, 403–414. Palm, G., Aertsen, A. M. H. J., & Gerstein, G. L. (1988). On the signiŽcance of correlations among neuronal spike trains. Biol. Cybern., 59, 1–11. Riehle, A., Grun, ¨ S., Diesmann, M., & Aertsen, A. (1997). Spike synchronization and rate modulation differently involved in motor cortical function. Science, 278, 1950–1953. Vaadia, E., Haalman, I., Abeles, M., Bergman, H., Prut, Y., Slovin, H., & Aertsen, A. M. H. J. (1995). Dynamics of neuronal interactions in monkey cortex in relation to behavioural events. Nature, 373, 5151–5158.

Model Dependence

217

von der Malsburg, C. (1981). The correlation theory of brain function (Internal Rep. No. 81-2), Gottingen: ¨ Max-Planck-Institute for Biophysical Chemistry. von der Malsburg, C. (1986). Am I thinking assemblies? In G. Palm & A. Aertsen (Eds.), Brain theory (pp. 161–176). Berlin: Springer-Verlag. Received March 30, 1998; accepted February 4, 1999.

LETTER

Communicated by Peter Dayan

Reinforcement Learning in Continuous Time and Space Kenji Doya ATR Human Information Processing Research Laboratories, Soraku, Kyoto 619-0288, Japan

This article presents a reinforcement learning framework for continuoustime dynamical systems without a priori discretization of time, state, and action. Based on the Hamilton-Jacobi-Bellman (HJB) equation for inŽnitehorizon, discounted reward problems, we derive algorithms for estimating value functions and improving policies with the use of function approximators. The process of value function estimation is formulated as the minimization of a continuous-time form of the temporal difference (TD) error. Update methods based on backward Euler approximation and exponential eligibility traces are derived, and their correspondences with the conventional residual gradient, TD(0), and TD(l ) algorithms are shown. For policy improvement, two methods—a continuous actor-critic method and a value-gradient-based greedy policy—are formulated. As a special case of the latter, a nonlinear feedback control law using the value gradient and the model of the input gain is derived. The advantage updating, a model-free algorithm derived previously, is also formulated in the HJBbased framework. The performance of the proposed algorithms is Žrst tested in a nonlinear control task of swinging a pendulum up with limited torque. It is shown in the simulations that (1) the task is accomplished by the continuous actor-critic method in a number of trials several times fewer than by the conventional discrete actor-critic method; (2) among the continuous policy update methods, the value-gradient-based policy with a known or learned dynamic model performs several times better than the actor-critic method; and (3) a value function update using exponential eligibility traces is more efŽcient and stable than that based on Euler approximation. The algorithms are then tested in a higher-dimensional task: cartpole swing-up. This task is accomplished in several hundred trials using the value-gradient-based policy with a learned dynamic model. 1 Introduction The temporal difference (TD) family of reinforcement learning (RL) algorithms (Barto, Sutton, & Anderson, 1983; Sutton, 1988; Sutton & Barto, 1998) provides an effective approach to control and decision problems for which optimal solutions are analytically unavailable or difŽcult to obtain. A numNeural Computation 12, 219–245 (2000)

c 2000 Massachusetts Institute of Technology °

220

Kenji Doya

ber of successful applications to large-scale problems, such as board games (Tesauro, 1994), dispatch problems (Crites & Barto, 1996; Zhang & Dietterich, 1996; Singh & Bertsekas, 1997), and robot navigation (Mataric, 1994) have been reported (see, e.g., Kaelbling, Littman, & Moore, 1996, and Sutton & Barto, 1998, for reviews). The progress of RL research so far, however, has been mostly constrained to the formulation of the problem in which discrete actions are taken in discrete time-steps based on the observation of the discrete state of the system. Many interesting real-world control tasks, such as driving a car or riding a snowboard, require smooth, continuous actions taken in response to high-dimensional, real-valued sensory input. In applications of RL to continuous problems, the most common approach has been Žrst to discretize time, state, and action and then to apply an RL algorithm for a discrete stochastic system. However, this discretization approach has the following drawbacks: • When a coarse discretization is used, the control output is not smooth, resulting in poor performance. • When a Žne discretization is used, the number of states and the number of iteration steps become huge, which necessitates not only large memory storage but also many learning trials. • In order to keep the number of states manageable, an elaborate partitioning of the variables has to be found using prior knowledge. Efforts have been made to eliminate some of these difŽculties by using appropriate function approximators (Gordon, 1996; Sutton, 1996; Tsitsiklis & Van Roy, 1997), adaptive state partitioning and aggregation methods (Moore, 1994; Singh, Jaakkola, & Jordan, 1995; Asada, Noda, & Hosoda, 1996; Pareigis, 1998), and multiple timescale methods (Sutton, 1995). In this article, we consider an alternative approach in which learning algorithms are formulated for continuous-time dynamical systems without resorting to the explicit discretization of time, state, and action. The continuous framework has the following possible advantages: • A smooth control performance can be achieved. • An efŽcient control policy can be derived using the gradient of the value function (Werbos, 1990). • There is no need to guess how to partition the state, action, and time. It is the task of the function approximation and numerical integration algorithms to Žnd the right granularity. There have been several attempts at extending RL algorithms to continuous cases. Bradtke (1993) showed convergence results for Q-learning algorithms for discrete-time, continuous-state systems with linear dynam-

Reinforcement Learning in Continuous Time and Space

221

ics and quadratic costs. Bradtke and Duff (1995) derived a TD algorithm for continuous-time, discrete-state systems (semi-Markov decision problems). Baird (1993) proposed the “advantage updating” method by extending Q-learning to be used for continuous-time, continuous-state problems. When we consider optimization problems in continuous-time systems, the Hamilton-Jacobi-Bellman (HJB) equation, which is a continuous-time counterpart of the Bellman equation for discrete-time systems, provides a sound theoretical basis (see, e.g., Bertsekas, 1995, and Fleming & Soner, 1993). Methods for learning the optimal value function that satisŽes the HJB equation have been studied using a grid-based discretization of space and time (Peterson, 1993) and convergence proofs have been shown for grid sizes taken to zero (Munos, 1997; Munos & Bourgine, 1998). However, the direct implementation of such methods is impractical in a highdimensional state-space. An HJB-based method that uses function approximators was presented by Dayan and Singh (1996). They proposed the learning of the gradients of the value function without learning the value function itself, but the method is applicable only to nondiscounted reward problems. This article presents a set of RL algorithms for nonlinear dynamical systems based on the HJB equation for inŽnite-horizon, discounted-reward problems. A series of simulations are devised to evaluate their effectiveness when used with continuous function approximators. We Žrst consider methods for learning the value function on the basis of minimizing a continuous-time form of the TD error. The update algorithms are derived either by using a single step or exponentially weighed eligibility traces. The relationships of these algorithms with the residual gradient (Baird, 1995), TD(0), and TD(l ) algorithms (Sutton, 1988) for discrete cases are also shown. Next, we formulate methods for improving the policy using the value function: the continuous actor-critic method and a value-gradientbased policy. SpeciŽcally, when a model is available for the input gain of the system dynamics, we derive a closed-form feedback policy that is suitable for real-time implementation. Its relationship with “advantage updating” (Baird, 1993) is also discussed. The performance of the proposed methods is Žrst evaluated in nonlinear control tasks of swinging up a pendulum with limited torque (Atkeson, 1994; Doya, 1996) using normalized gaussian basis function networks for representing the value function, the policy, and the model. We test (1) the performance of the discrete actor-critic, continuous actor-critic, and valuegradient-based methods; (2) the performance of the value function update methods; and (3) the effects of the learning parameters, including the action cost, exploration noise, and landscape of the reward function. Then we test the algorithms in a more challenging task, cart-pole swing-up (Doya, 1997), in which the state-space is higher-dimensional and the system input gain is state dependent.

222

Kenji Doya

2 The Optimal Value Function for a Discounted Reward Task In this article, we consider the continuous-time deterministic system, x(t) P = f (x( t), u(t)),

(2.1)

where x 2 X ½ Rn is the state and u 2 U ½ Rm is the action (control input). We denote the immediate reward for the state and the action as r(t) = r(x(t), u(t)).

(2.2)

Our goal is to Žnd a policy (control law), u(t) = m (x( t)),

(2.3)

that maximizes the cumulative future rewards, Z 1 s¡t Vm (x(t)) = e¡ t r(x(s), u(s)) ds t

(2.4)

for any initial state x(t). Note that x(s) and u(s) ( t · s < 1) follow the system dynamics (2.1) and the policy (2.3). Vm (x) is called the value function of the state x, and t is the time constant for discounting future rewards. An important feature of this inŽnite-horizon formulation is that the value function and the optimal policy do not depend explicitly on time, which is convenient in estimating them using function approximators. The discounted reward makes it unnecessary to assume that the state is attracted to a zero-reward state. The value function V¤ for the optimal policy m ¤ is deŽned as µZ V ¤ (x( t)) = max

u[t,1)

1 t

e¡

s¡t t

¶ r(x( s), u(s))ds ,

(2.5)

where u[t, 1) denotes the time course u(s) 2 U for t · s < 1. According to the principle of optimality, the condition for the optimal value function at time t is given by µ ¶ 1 ¤ @V ¤ (x) (x( )) (x( ), (x( ), V t = max r t u(t)) C f t u(t)) , u(t)2U t @x

(2.6)

which is a discounted version of the HJB equation (see appendix A). The optimal policy is given by the action that maximizes the right-hand side of the HJB equation: µ ¶ @V¤ (x) u(t) = m ¤ (x( t)) = arg max r(x( t), u) C f (x( t), u) . u2U @x

(2.7)

Reinforcement Learning in Continuous Time and Space

223

Reinforcement learning can be formulated as the process of bringing the current policym and its value function estimate V closer to the optimal policy m ¤ and the optimal value function V¤ . It generally involves two components: 1. Estimate the value function V based on the current policym . 2. Improve the policy m by making it greedy with respect to the current estimate of the value function V. We will consider the algorithms for these two processes in the following two sections. 3 Learning the Value Function For the learning of the value function in a continuous state-space, it is mandatory to use some form of function approximator. We denote the current estimate of the value function as Vm (x( t)) ’ V (x( t)I w),

(3.1)

where w is a parameter of the function approximator, or simply, V( t). In the framework of TD learning, the estimate of the value function is updated using a self-consistency condition that is local in time and space. This is given by differentiating deŽnition 2.4 by t as 1 VP m (x( t)) = Vm (x(t)) ¡ r(t). t

(3.2)

This should hold for any policy, including the optimal policy given by equation 2.7. If the current estimate V of the value function is perfect, it should satisfy the consistency condition VP (t) = t1 V (t)¡ r( t). If this condition is not satisŽed, the prediction should be adjusted to decrease the inconsistency, d (t) ´ r( t) ¡

1 V (t) C VP ( t). t

(3.3)

This is the continuous-time counterpart of the TD error (Barto, Sutton, & Anderson, 1983; Sutton, 1988). 3.1 Updating the Level and the Slope. In order to bring the TD error (3.3) to zero, we can tune the level of the value function V (t), its time derivative VP (t), or both, as illustrated in Figures 1A–C. Now we consider the objective function (Baird, 1993), E( t ) =

1 |d ( t)| 2 . 2

(3.4)

224

Kenji Doya

A d (t)

^ B V(t)

t0

t

t0

t

^ C V(t)

^ D V(t)

O Figure 1: Possible updates for the value function estimate V(t) for an instantaP neous TD error d (t) = r(t) ¡ 1t V(t) C V(t). (A) A positive TD error at t = t0 can be corrected by (B) an increase in V(t), or (C) a decrease in the time derivative P V(t), or (D) an exponentially weighted increase in V(t) (t < t0 ).

From deŽnition 3.3 and the chain rule VP ( t) = @V xP ( t), the gradient of the @x objective function with respect to a parameter wi is given by µ ¶ 1 @E(t) @ P = d ( t) r ( t ) ¡ V ( t) C V ( t ) t @wi @wi µ ¶ 1 @V (xI w) @ @V (xI w) ( ) ( ) P = d t ¡ C xt . t @wi @wi @x

¡

¢

Therefore, the gradient descent algorithm is given by µ 1 @V (xI w) @E @ @V (xI w) = gd (t) ¡ wP i = ¡g t @wi @wi @wi @x

¡

¢

¶ xP ( t) ,

(3.5)

where g is the learning rate. A potential problem with this update algorithm is its symmetry in time. Since the boundary condition for the value function is given at t ! 1, it would be more appropriate to update the past estimates without affecting the future estimates. Below, we consider methods for implementing the backup of TD errors. 3.2 Backward Euler Differentiation: Residual Gradient and TD(0). One way of implementing the backup of TD errors is to use the backward Euler

Reinforcement Learning in Continuous Time and Space

225

approximation of time derivative VP (t). By substituting VP (t) = ( V( t) ¡ V (t ¡ D t))/ D t into equation 3.3, we have d (t) = r( t) C

1 Dt

¡

µ

1¡

Dt t

¢

¶ V ( t) ¡ V ( t ¡ D t) .

(3.6)

Then the gradient of the squared TD error (see equation 3.4) with respect to the parameter wi is given by µ ¶ 1 D t @V (x(t)I w) @E( t) @V (x( t ¡ D t)I w) = d ( t) 1¡ ¡ . Dt t @wi @wi @wi

¡

¢

A straightforward gradient descent algorithm is given by

¡

µ Dt ( ) P wi = gd t ¡ 1 ¡ t

¢ @V(x(t)I w) @wi

¶ @V(x(t ¡ D t)I w) C . @wi

(3.7)

An alternative way is to update only V ( t ¡ D t) without explicitly changing V ( t) by wP i = gd ( t)

@V (x( t ¡ D t)I w) . @wi

(3.8)

The Euler discretized TD error, equation 3.6, coincides with the conventional TD error, d t = rt Cc Vt ¡ Vt¡1 ,

Dt

by taking the discount factor c = 1 ¡ Dt t ’ e¡ t and rescaling the values as Vt = D1t V( t). The update schemes, equations 3.7 and 3.8, correspond to the residual-gradient (Baird, 1995; Harmon, Baird, & Klopf, 1996) and TD(0) algorithms, respectively. Note that time step D t of the Euler differentiation does not have to be equal to the control cycle of the physical system. 3.3 Exponential Eligibility Trace: TD(l). Now let us consider how an instantaneous TD error should be corrected by a change in the value V as a function of time. Suppose an impulse of reward is given at time t = t0 . Then, from deŽnition 2.4, the corresponding temporal proŽle of the value function is ( t ¡t ¡ 0t t · t0 , m V ( t) = e 0 t > t0 . Because the value function is linear with respect to the reward, the desired correction of the value function for an instantaneous TD errord ( t0 ) is ( t0 ¡t ( t0 ) e¡ t d t · t0 , O ( ) V t = 0 t > t0 ,

226

Kenji Doya

as illustrated in Figure 1D. Therefore, the update of wi given d ( t0 ) should be made as Z wP i = g

t0

¡1

@V VO ( t)

(x( t)I w) @wi

Z dt = gd (t0 )

t0 ¡1

e¡

t0 ¡t t

@V (x( t)I w) dt. (3.9) @wi

We can consider the exponentially weighted integral of the derivatives as the eligibility trace ei for the parameter wi . Then a class of learning algorithms is derived as wP i = gd ( t) ei ( t),

1 @V (x( t)I w) , ePi ( t) = ¡ ei ( t) C k @wi

(3.10)

where 0 < k · t is the time constant of the eligibility trace. If we discretize equation 3.10 with time step D t, it coincides with the eligibility trace update in TD(l ) (see, e.g., Sutton & Barto, 1998), ei ( t C D t) = lc ei ( t) C with l =

1¡D t/ k 1¡D t/ t

@Vt , @wi

.

4 Improving the Policy Now we consider ways for improving the policy u(t) = m (x( t)) using its associated value function V (x). One way is to improve the policy stochastically using the actor-critic method, in which the TD error is used as the effective reinforcement signal. Another way is to take a greedy policy with respect to the current value function, µ ¶ @V(x) u(t) = m (x( t)) = arg max r(x( t), u) C f (x(t), u) , u2U @x

(4.1)

using knowledge about the reward and the system dynamics. 4.1 Continuous Actor-Critic. First, we derive a continuous version of the actor-critic method (Barto et al., 1983). By comparing equations 3.3 and 4.1, we can see that the TD error is maximized by the greedy action u(t). Accordingly, in the actor-critic method, the TD error is used as the reinforcement signal for policy improvement. We consider the policy implemented by the actor as ³ ´ u(t) = s A(x(t)I wA ) C sn(t) ,

(4.2)

Reinforcement Learning in Continuous Time and Space

227

where A(x( t)I wA ) 2 Rm is a function approximator with a parameter vector wA , n(t) 2 Rm is noise, and s() is a monotonically increasing output function. The parameters are updated by the stochastic real-valued (SRV) unit algorithm (Gullapalli, 1990) as wP iA = g Ad ( t)n( t)

@A(x( t)I wA ) . @wiA

(4.3)

4.2 Value-Gradient Based Policy. In discrete problems, a greedy policy can be found by one-ply search for an action that maximizes the sum of the immediate reward and the value of the next state. In the continuous case, the right-hand side of equation 4.1 has to be minimized over a continuous set of actions at every instant, which in general can be computationally expensive. However, when the reinforcement r(x, u) is convex with respect to the action u and the system dynamics f (x, u) is linear with respect to the action u, the optimization problem in equation 4.1 has a unique solution, and we can derive a closed-form expression of the greedy policy. Here, we assume that the reward r(x, u) can be separated into two parts: the reward for state R(x), which is given by the environment and unknown, and the cost for action S(u), which can be chosen as a part of the learning strategy. We speciŽcally consider the case r(x, u) = R(x) ¡

m X

Sj (uj ),

(4.4)

j= 1

where Sj () is a cost function for action variable uj . In this case, the condition for the greedy action, equation 4.1, is given by ¡Sj0 ( uj ) C where

@f (x,u) @uj

@V (x) @f (x, u) = 0 @x @uj

( j = 1, . . . , m),

is the jth column vector of the n £ m input gain matrix

@f (x,u) @u

(x,u)

of the system dynamics. We now assume that the input gain @f @u is not dependent on u; that is, the system is linear with respect to the input and the action cost function Sj () is convex. Then the the above equation has a unique solution, uj = Sj0

¡1

¡ @V(x) @f (x, u) ¢ @x

@uj

,

where Sj0 () is a monotonic function. Accordingly, the greedy policy is represented in vector notation as ! @f (x, u) T @V (x) T 0 ¡1 u= S , (4.5) @u @x

(

228

Kenji Doya

where

@V (x) T @x

represents the steepest ascent direction of the value function, (x,u) T

which is then transformed by the “transpose” model @f @u into a direction in the action space, and the actual amplitude of the action is determined by gain function S0 ¡1 (). (x) Note that the gradient @V@x can be calculated by backpropagation when the value function is represented by a multilayer network. The assumption of linearity with respect to the input is valid in most Newtonian mechanical systems (e.g., the acceleration is proportional to the force), and the gain @f (x,u) matrix @u can be calculated from the inertia matrix. When the dynamics is linear and the reward is quadratic, the value function is also quadratic, and equation 4.5 coincides with the optimal feedback law for a linear quadratic regulator (LQR; see, e.g., Bertsekas, 1995). 4.2.1 Feedback Control with a Sigmoid Output Function. A common constraint in control tasks is that the amplitude of the action, such as the force or torque, is bounded. Such a constraint can be incorporated into the above policy with an appropriate choice of the action cost. Suppose that the amplitude of the action is limited as |uj | · ujmax ( j = 1, . . . , m). We then deŽne the action cost as ! Z uj u ¡1 (4.6) Sj ( uj ) = cj s du, ujmax 0

(

where s() is a sigmoid function that saturates as s(§1) = §1. In this case, the greedy feedback policy, equation 4.5, results in feedback control with a sigmoid output function, uj =

ujmax s

(

! 1 @f (x, u) T @V (x) T . cj @uj @x

(4.7)

In the limit of cj ! 0, the policy will be a “bang-bang” control law, "

uj =

ujmax

# @f (x, u) T @V(x) T sign . @uj @x

(4.8)

4.3 Advantage Updating. When the model of the dynamics is not available, as in Q-learning (Watkins, 1989), we can select a greedy action by directly learning the term to be maximized in the HJB equation, r(x( t), u(t)) C

@V ¤ (x) f (x( t), u(t)). @x

This idea has been implemented in the advantage updating method (Baird, 1993; Harmon et al., 1996), in which both the value function V(x) and the

Reinforcement Learning in Continuous Time and Space

229

advantage function A(x, u) are updated. The optimal advantage function A¤ (x, u) is represented in the current HJB formulation as A¤ (x, u) = r(x, u) ¡

1 ¤ @V ¤ (x) V (x) C f (x, u), t @x

(4.9)

which takes the maximum value of zero for the optimal action u. The advantage function A(x, u) is updated by A(x, u) Ã max[A(x, u)] C r(x, u) ¡ u

= max[A(x, u)] Cd (t) u

1 ¤ V (x) C VP (x) t (4.10)

under the constraint maxu [A(x, u)] = 0. The main difference between the advantage updating and the valuegradient-based policy described above is while the value V and the advantage A are updated in the former, the value V and the model f are updated @f (x,u) and their derivatives are used in the latter. When the input gain model @u is known or easy to learn, the use of the closed-form policy, equation 4.5, in the latter approach is advantageous because it simpliŽes the process of maximizing the right-hand side of the HJB equation. 5 Simulations We tested the performance of the continuous RL algorithms in two nonlinear control tasks: a pendulum swing-up task (n=2, m=1) and a cart-pole swingup task (n=4, m=1). In each of these tasks, we compared the performance of three control schemes: 1. Actor-critic: control by equation 4.2 and learning by equation 4.3 2. Value-gradient-based policy, equation 4.7, with an exact gain matrix 3. Value-gradient-based policy, equation 4.7, with concurrent learning of the input gain matrix The value functions were updated using the exponential eligibility trace (see equation 3.10) except in the experiments of Figure 6. Both the value and policy functions were implemented by normalized gaussian networks, as described in appendix B. A sigmoid output function s( x) = p2 arctan( p2 x) (HopŽeld, 1984) was used in equations 4.2 and 4.7. In order to promote exploration, we incorporated a noise term sn(t) in both policies, equations 4.2 and 4.7 (see equations B.2 and B.3 in appendix B). P We used low-pass Žltered noise tn n(t) = ¡n(t) C N(t), where N(t) denotes normal gaussian noise. The size of the perturbation s was tapered off as the performance improved (Gullapalli, 1990). We took the modulation scheme

230

Kenji Doya

q l T mg Figure 2: Control of a pendulum with limited torque. The dynamics were given by hP = v and ml2 vP = ¡m v C mgl sinh C u. The physical parameters were m = l = 1, g = 9.8, m = 0.01, and umax = 5.0. The learning parameters were t = 1.0, k = 0.1, c = 0.1, tn = 1.0, s0 = 0.5, V0 = 0, V1 = 1, g = 1.0, g A = 5.0, and g M = 10.0, in the following simulations unless otherwise speciŽed.

s = s0 min[1, max[0, VV11¡V(t) ¡V0 ]], where V0 and V1 are the minimal and maximal levels of the expected reward. The physical systems were simulated by a fourth-order Runge-Kutta method, and the learning dynamics was simulated by a Euler method, both with the time step of 0.02 sec. 5.1 Pendulum Swing-Up with Limited Torque. First, we tested the continuous-time RL algorithms in the task of a pendulum swinging upward with limited torque (see Figure 2) (Atkeson, 1994; Doya, 1996). The control of this one degree of freedom system is nontrivial if the maximal output torque umax is smaller than the maximal load torque mgl. The controller has to swing the pendulum several times to build up momentum and also has to decelerate the pendulum early enough to prevent the pendulum from falling over. The reward was given by the height of the tip of the pendulum, R(x) = cosh . The policy and value functions were implemented by normalized gaussian networks with 15£15 basis functions to cover the two-dimensional state-space x = (h , v). In modeling the system dynamics, 15 £ 15 £ 2 bases were used for the state-action space (h , v, u). Each trial was started from an initial state x(0) = (h (0), 0), where h (0) was selected randomly in [¡p , p ]. A trial lasted for 20 seconds unless the pendulum was over-rotated (|h | > 5p ). Upon such a failure, the trial was terminated with a reward r( t) = ¡1 for 1 second. As a measure of the swing-up performance, we deŽned the time in which the pendulum stayed up (|h | < p / 4) as tup . A trial was regarded as “successful” when tup > 10 seconds. We used the number of trials made before achieving 10 successful trials as the measure of the learning speed.

Reinforcement Learning in Continuous Time and Space

231

2p

V

1 0.5 0 ­ 0.5 ­1

0 0

- 2p p

2p

w

q Figure 3: Landscape of the value function V(h , v) for the pendulum swing-up task. The white line shows an example of a swing-up trajectory. The state-space was a cylinder with h = § p connected. The 15 £ 15 centers of normalized gaussian basis functions are located on a uniform grid that covers the area [¡p , p ] £ [¡5/ 4p , 5/ 4p ].

Figure 3 illustrates the landscape of the value function and a typical trajectory of swing-up using the value-gradient-based policy. The trajectory starts from the bottom of the basin, which corresponds to the pendulum hanging downward, and spirals up the hill along the ridge of the value function until it reaches the peak, which corresponds to the pendulum standing upward. 5.1.1 Actor-Critic, Value Gradient, and Physical Model. We Žrst compared the performance of the three continuous RL algorithms with the discrete actor-critic algorithm (Barto et al., 1983). Figure 4 shows the time course of learning in Žve simulation runs, and Figure 5 shows the average number of trials needed until 10 successful swing-ups. The discrete actor-critic algorithm took about Žve times more trials than the continuous actor-critic. Note that the continuous algorithms were simulated with the same time step as the discrete algorithm. Consequently, the performance difference was due to a better spatial generalization with the normalized gaussian networks. Whereas the continuous algorithms performed well with 15 £ 15 basis functions, the discrete algorithm did not achieve the task using 15 £15, 20 £ 20, or 25 £ 25 grids. The result shown here was obtained by 30 £ 30 grid discretization of the state.

232

Kenji Doya B 20

20

15

15 t_up

t_up

A

10 5 0

10 5

0

500

1000 trials

1500

0

2000

50

100 trials

150

200

D 20

20

15

15 t_up

t_up

C

0

10 5 0

10 5

0

20

40

trials

60

80

0

100

0

20

40

trials

60

80

100

Figure 4: Comparison of the time course of learning with different control schemes: (A) discrete actor-critic, (B) continuous actor-critic, (C) value-gradientbased policy with an exact model, (D) value-gradient policy with a learned model (note the different scales). tup : time in which the pendulum stayed up. In the discrete actor-critic, the state-space was evenly discretized into 30 £30 boxes and the action was binary (u = § umax ). The learning parameters were c = 0.98, l = 0.8, g = 1.0, and g A = 0.1. 400 350 300 Trials

250 200 150 100 50 0

DiscAC

ActCrit

ValGrad PhysModel

Figure 5: Comparison of learning speeds with discrete and continuous actorcritic and value-gradient-based policies with an exact and learned physical models. The ordinate is the number of trials made until 10 successful swing-ups.

Reinforcement Learning in Continuous Time and Space

233

Among the continuous algorithms, the learning was fastest with the value-gradient-based policy using an exact input gain. Concurrent learning of the input gain model resulted in slower learning. The actor-critic was the slowest. This was due to more effective exploitation of the value function in the gradient-based policy (see equation 4.7) compared to the stochastically improved policy (see equation 4.2) in the actor-critic. 5.1.2 Methods of Value Function Update. Next, we compared the methods of value function update algorithms (see equations 3.7, 3.8, and 3.10) using the greedy policy with an exact gain model (see Figure 6). Although the three algorithms attained comparable performances with the optimal settings for D t and k , the method with the exponential eligibility trace performed well in the widest range of the time constant and the learning rate. We also tested the purely symmetric update method (see equation 3.5), but its performance was very unstable even when the learning rates for the value and its gradient were tuned carefully. 5.1.3 Action Cost, Graded Reward, and Exploration. We then tested how the performance depended on the action cost c, the shape of the reward function R(x), and the size of the exploration noise s0 . Figure 7A compares the performance with different action costs c = 0, 0.01, 0.1, and 1.0. The learning was slower with a large cost for the torque (c = 1) because of the weak output in the early stage. The results with bang-bang control (c = 0) tended to be less consistent than with sigmoid control with the small costs (c = 0.01, 0.1). Figure 7B summarizes the effects of the reward function and exploration noise. When binary reward function, » 1 R(x) = 0

|h | < p / 4 otherwise,

was used instead of cos(h ), the task was more difŽcult to learn. However, a better performance was observed with the use of negative binary reward function, » 0 |h | < p / 4 R(x) = ¡1 otherwise. The difference was more drastic with a Žxed initial state x(0) = (p , 0) and no noise s = 0, for which no success was achieved with the positive binary reward. The better performance with the negative reward was due to the initialization of the value function as V(x) = 0. As the value function near h = p is learned as V (x) ’ ¡1, the value-gradient-based policy drives the state to unexplored areas, which are assigned higher values V(x) ’ 0 by default.

234

Kenji Doya

Figure 6: Comparison of different value function update methods with different settings for the time constants. (A) Residual gradient (see equation 3.7). (B) Single-step eligibility trace (see equation 3.8). (C) Exponential eligibility (see equation 3.10). The learning rate g was roughly optimized for each method and each setting of time step D t of the Euler approximation or time constant k of the eligibility trace. The performance was very unstable, with D t = 1.0 in the discretization-based methods.

Reinforcement Learning in Continuous Time and Space

235

A 100

Trials

80 60 40 20 0

0.

0.01 0.1 control cost coef.

1.0

B 100

Trials

80 60 40 20 0

cosq s

[0,1] [-1,0] 0 = 0.5

cosq s

[0,1] [-1,0] 0 = 0.0

Figure 7: Effects of parameters of the policy. (A) Control-cost coefŽcient c. (B) Reward function R(x) and perturbation size s0 .

5.2 Cart-Pole Swing-Up Task. Next, we tested the learning schemes in a more challenging task of cart-pole swing-up (see Figure 8), which is a strongly nonlinear extension to the common cart-pole balancing task (Barto et al., 1983). The physical parameters of the cart pole were the same as in Barto et al. (1983), but the pole had to be swung up from an arbitrary angle and balanced. Marked differences from the previous task were that the dimension of the state-space was higher and the input gain was state dependent. The state vector was x = ( x, v, h , v), where x and v are the position and the velocity of the cart. The value and policy functions were implemented

236

Kenji Doya

A

B

C

Figure 8: Examples of cart-pole swing-up trajectories. The arrows indicate the initial position of the pole. (A) A typical swing-up from the bottom position. (B) When a small perturbation v > 0 is given, the cart moves to the right and keeps the pole upright. (C) When a larger perturbation is given, the cart initially tries to keep the pole upright, but then brakes to avoid collision with the end of the track and swings the pole up on the left side. The learning parameters were t = 1.0, k = 0.5, c = 0.01, tn = 0.5, s0 = 0.5, V0 = 0, V1 = 1, g = 5.0, g A = 10.0, and g M = 10.0.

Reinforcement Learning in Continuous Time and Space

237

by normalized gaussian networks with 7£7£15£15 bases. A 2£2£4£2£2 basis network was used for modeling the system dynamics. The reward was given by R(x) = cosh2 ¡1 . When the cart bumped into the end of the track or when the pole overrotated (|h | > 5p ), a terminal reward r(t) = 1 was given for 0.5 second. Otherwise a trial lasted for 20 seconds. Figure 8 illustrates the control performance after 1000 learning trials with the greedy policy using the value gradient and a learned input gain model. Figure 9A shows the value function in the 4D state-space x = ( x, v, h, v). Each of the 3 £ 3 squares represents a subspace (h , v) with different values of ( x, v). We can see the 1-shaped ridge of the value function, which is similar to the one seen in the pendulum swing-up task (see Figure 3). Also note the lower values with x and v both positive or negative, which signal the danger of bumping into the end of the track. Figure 9B shows the most critical components of the input gain vector v @P . The gain represents how the force applied to the cart is transformed @u as the angular acceleration of the pole. The gain model could successfully capture the change of the sign with the upward (|h | < p / 2) orientation and the downward (|h | > p / 2) orientation of the pole. Figure 10 is a comparison of the number of trials necessary for 100 successful swing-ups. The value-gradient-based greedy policies performed about three times faster than the actor-critic. The performances with the exact and learned input gains were comparable in this case. This was because the learning of the physical model was relatively easy compared to the learning of the value function. 6 Discussion The results of the simulations can be summarized as follows: 1. The swing-up task was accomplished by the continuous actor-critic in a number of trials several times fewer than by the conventional discrete actor-critic (see Figures 4 and 5). 2. Among the continuous methods, the value-gradient-based policywith a known or learned dynamic model performed signiŽcantly better than the actor-critic (see Figures 4, 5, and 10). 3. The value function update methods using exponential eligibility traces were more efŽcient and stable than the methods based on Euler approximation (see Figure 6). 4. Reward-related parameters, such as the landscape and the baseline level of the reward function, greatly affect the speed of learning (see Figure 7). 5. The value-gradient-based method worked well even when the input gain was state dependent (see Figures 9 and 10).

238

Kenji Doya A

v

q

- p

0

p

4p 0

2.4

w

- 4p 0.0 V +0.021 -2.4

-0.347

-2.4

0.0 x

B

v

-0.715

2.4

- p

q

0

p

4p 0

2.4

w

- 4p 0.0 df/du +1.423 -2.4

-0.064

-2.4

0.0 x

2.4

-1.551

Figure 9: (A) Landscape of the value function for the cart-pole swing-up task. (B) Learned gain model @@uvP .

Reinforcement Learning in Continuous Time and Space

239

Figure 10: Comparison of the number of trials until 100 successful swing-ups with the actor-critic, value-gradient-based policy with an exact and learned physical models.

Among the three major RL methods—the actor critic, Q-learning, and model-based look-ahead—only the Q-learning has been extended to continuous-time cases as advantage updating (Baird, 1993). This article presents continuous-time counterparts for all three of the methods based on the HJB equation, 2.6, and therefore provides a more complete repertoire of continuous RL methods. A major contribution of this article is the derivation of the closed-form policy (see equation 4.5) using the value gradient and the dynamic model. One critical issue in advantage updating is the need for Žnding the maximum of the advantage function on every control cycle, which can be computationally expensive except in special cases like linear quadratic problems (Harmon et al., 1996). As illustrated by simulation, the value-gradient-based policy (see equation 4.5) can be applied to a broad class of physical control problems using a priori or learned models of the system dynamics. The usefulness of value gradients in RL was considered by Werbos (1990) for discrete-time cases. The use of value gradients was also proposed by Dayan and Singh (1996), with the motivation being to eliminate the need for updating the value function in advantage updating. Their method, however, in which the value gradients @V are updated without updating the value @x function V (x) itself, is applicable only to nondiscounted problems. When the system or the policy is stochastic, HJB equation 2.6 will include second-order partial derivatives of the value function, µ 1 ¤ @V ¤ (x) V (x( t)) = max r(x( t), u(t)) C f (x( t), u(t)) u(t)2U t @x

240

Kenji Doya

C tr

¡ @ V (x) ¢ ¶ 2

¤

@x2

C

,

(6.1)

where C is the covariance matrix of the system noise (see, e.g., Fleming & Soner, 1993). In our simulations, the methods based on deterministic HJB equation 2.6 worked well, although we incorporated noise terms in the policies to promote exploration. One reason is that the noise was small enough so that the contribution of the second-order term was minor. Another reason could be that the second-order term has a smoothing effect on the value function, and this was implicitly achieved by the use of the smooth function approximator. This point needs further investigation. The convergent properties of HJB-based RL algorithms were recently shown for deterministic (Munos, 1997) and stochastic (Munos & Bourgine, 1998) cases using a grid-based discretization of space and time. However, the convergent properties of continuous RL algorithms combined with function approximators remain to be studied. When a continuous RL algorithm is numerically implemented with a Žnite time step, as shown in sections 3.2 and 3.3, it becomes equivalent to a discrete-time TD algorithm, for which some convergent properties have been shown with the use of function approximators (Gordon, 1995; Tsitsiklis & Van Roy, 1997). For example, the convergence of TD algorithms has been shown with the use of a linear function approximator and on-line sampling (Tsitsiklis & Van Roy, 1997), which was the case with our simulations. However, the above result considers value function approximation for a given policyand does not guarantee the convergence of the entire RL process to a satisfactory solution. For example, in our swing-up tasks, the learning sometimes got stuck in a locally optimal solution of endless rotation of the pendulum when a penalty for over-rotation was not given. The use of Žxed smooth basis functions has a limitation in that steep cliffs in the value or policy functions cannot be achieved. Despite some negative didactic examples (Tsitsiklis & Van Roy, 1997), methods that dynamically allocate or reshape basis functions have been successfully used with continuous RL algorithms, for example, in a swing-up task (Schaal, 1997) and in a stand-up task for a three-link robot (Morimoto & Doya, 1998). Elucidation of the conditions under which the proposed continuous RL algorithms work successfully—for example, the properties of the function approximatorsand the methods for exploration—remains the subject of future empirical and theoretical studies. Appendix A: HJB Equation for Discounted Reward According to the optimality principle, we divide the integral in equation 2.5 into two parts [t, t C D t] and [t C D t, 1) and then solve a short-term opti-

Reinforcement Learning in Continuous Time and Space

241

mization problem, µZ V¤ (x( t)) = max

u[t,tCD t]

tCD t t

e¡

s¡t t

¶ Dt r(x( s), u(s))ds Ce¡ t V ¤ (x( t CD t)) . (A.1)

For a small D t, the Žrst term is approximated as r(x( t), u(t))D t C o(D t), and the second term is Taylor expanded as V¤ (x( t C D t)) = V ¤ (x( t)) C

@V ¤ f (x(t), u(t))D t C o(D t). @x( t)

By substituting them into equation A.1 and collecting V ¤ (x( t)) on the lefthand side, we have an optimality condition for [t, t C D t] as Dt

(1 ¡ e¡ t ) V ¤ (x( t)) µ ¶ ¤ ¡ Dt t @V = max r(x(t), u(t))D t C e f (x( t), u(t))D t C o(D t) . (A.2) u[t,tCD t] @x( t) By dividing both sides by D t and taking D t to zero, we have the condition for the optimal value function, µ ¶ 1 ¤ @V ¤ V (x( t)) = max r(x( t), u(t)) C f (x( t), u(t)) . u(t)2U t @x

(A.3)

Appendix B: Normalized Gaussian Network A value function is represented by V(xI w) =

K X

wk bk (x),

(B.1)

k= 1

where ak (x) , bk (x) = PK (x) l= 1 al

(x¡ck )k . ak (x) = e¡ksk T

2

The vectors ck and sk deŽne the center and the size of the kth basis function. Note that the basis functions located on the ends of the grids are extended like sigmoid functions by the effect of normalization. In the current simulations, the centers are Žxed in a grid, which is analogous to the “boxes” approach (Barto et al., 1983) often used in discrete RL. Grid allocation of the basis functions enables efŽcient calculation of their

242

Kenji Doya

activation as the outer product of the activation vectors for individual input variables. In the actor-critic method, the policy is implemented as u(t) = u

max

s

(

X k

! wkA bk (x(t))

C sn(t) ,

(B.2)

where s is a component-wise sigmoid function and n(t) is the noise. In the value-gradient-based methods, the policy is given by u(t) = u

max

! 1 @f (x, u) T X @bk (x) T C sn(t) . s wk c @u @x k

(

(B.3)

To implement the input gain model, a network is trained to predict the time derivative of the state from x and u, x(t) P ’ fO(x, u) =

X k

wkM bk (x( t), u(t)).

(B.4)

The weights are updated by M( () P ¡ fO(x( t), u(t)))bk (x( t), u(t)), x(t) wP M k t = g

(B.5)

and the input gain of the system dynamics is given by ­ X (x, u) ­ @f (x, u) M @bk ­ . ’ wk ­ @u @u u= 0 k

(B.6)

Acknowledgments I thank Mitsuo Kawato, Stefan Schaal, Chris Atkeson, and Jim Morimoto for their helpful discussions. References Asada, M., Noda, S., & Hosoda, K. (1996). Action-based sensor space categorization for robot learning. In Proceedings of IEEE/ RSJ International Conference on Intelligent Robots and Systems (pp. 1502–1509). Atkeson, C. G. (1994). Using local trajectory optimizers to speed up global optimization in dynamic programming. In J. D. Cowan, G. Tesauro, & J. Alspector (Eds.), Advances in neural information processing system, 6 (pp. 663–670). San Mateo, CA: Morgan Kaufmann. Baird, L. C. (1993). Advantage updating (Tech. Rep. No. WL-TR-93-1146). Wright Laboratory, Wright-Patterson Air Force Base, OH.

Reinforcement Learning in Continuous Time and Space

243

Baird, L. C. (1995). Residual algorithms: Reinforcement learning with function approximation. In A. Prieditis & S. Russel (Eds.), Machine learning: Proceedings of the Twelfth International Conference. San Mateo, CA: Morgan Kaufmann. Barto, A. G., Sutton, R. S., & Anderson, C. W. (1983). Neuronlike adaptive elements that can solve difŽcult learning control problems. IEEE Transactions on Systems, Man, and Cybernetics, 13, 834–846. Bertsekas, D. P. (1995). Dynamic programming and optimal control. Cambridge, MA: MIT Press. Bradtke, S. J. (1993). Reinforcement learning applied to linear quadratic regulation. In C. L. Giles, S. J. Hanson, & J. D. Cowan (Eds.), Advances in neural information processing systems, 5 (pp. 295–302). San Mateo, CA: Morgan Kaufmann. Bradtke, S. J., & Duff, M. O. (1995). Reinforcement learning methods for continuous-time Markov decision problems. In G. Tesauro, D. S. Touretzky, & T. K. Leen (Eds.), Advances in neural information processing systems, 7 (pp. 393– 400). Cambridge, MA: MIT Press. Crites, R. H., & Barto, A. G. (1996). Improving elevator performance using reinforcement learning. In D. S. Touretzky, M. C. Mozer, & M. E. Hasselmo (Eds.), Advances in neural information processing systems, 8 (pp. 1017–1023). Cambridge, MA: MIT Press. Dayan, P., & Singh, S. P. (1996). Improving policies without measuring merits. In D. S. Touretzky, M. C. Mozer, & M. E. Hasselmo (Eds.), Advances in neural information processing systems, 8 (pp. 1059–1065). Cambridge, MA: MIT Press. Doya, K. (1996). Temporal difference learning in continuous time and space. In D. S. Touretzky, M. C. Mozer, & M. E. Hasselmo (Eds.), Advances in neural information processing systems, 8 (pp. 1073–1079). Cambridge, MA: MIT Press. Doya, K. (1997). EfŽcient nonlinear control with actor-tutor architecture. In M. C. Mozer, M. I. Jordan, & T. P. Petsche (Eds.), Advances in neural information processing systems, 9 (pp. 1012–1018). Cambridge, MA: MIT Press. Fleming, W. H., & Soner, H. M. (1993). Controlled Markov processes and viscosity solutions. New York: Springer-Verlag. Gordon, G. J. (1995). Stable function approximation in dynamic programming. In A. Prieditis & S. Russel (Eds.), Machine learning: Proceedings of the Twelfth International Conference. San Mateo, CA: Morgan Kaufmann. Gordon, G. J. (1996). Stable Žtted reinforcement learning. In D. S. Touretzky, M. C. Mozer, & M. E. Hasselmo (Eds.), Advances in neural informationprocessing systems, 8 (pp. 1052–1058). Cambridge, MA: MIT Press. Gullapalli, V. (1990). A stochastic reinforcement learning algorithm for learning real-valued functions. Neural Networks, 3, 671–192. Harmon, M. E., Baird, III, L. C., & Klopf, A. H. (1996). Reinforcement learning applied to a differential game. Adaptive Behavior, 4, 3–28. HopŽeld, J. J. (1984). Neurons with graded response have collective computational properties like those of two-state neurons. Proceedings of National Academy of Science, 81, 3088–3092. Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996). Reinforcement learning: A survey. Journal of ArtiŽcial Intelligence Research, 4, 237–285.

244

Kenji Doya

Mataric, M. J. (1994). Reward functions for accelerated learning. In W. W. Cohen & H. Hirsh (Eds.), Proceedings of the 11th International Conference on Machine Learning. San Mateo, CA: Morgan Kaufmann. Moore, A. W. (1994). The parti-game algorithm for variable resolution reinforcement learning in multidimensional state-spaces. In J. D. Cowan, G. Tesauro, & J. Alspector (Eds.), Advances in neural informationprocessingsystems,6 (pp. 711– 718). San Mateo, CA: Morgan Kaufmann. Morimoto, J., & Doya, K. (1998). Reinforcement learning of dynamic motor sequence: Learning to stand up. In Proceedings of IEEE/ RSJ International Conference on Intelligent Robots and Systems, 3, 1721–1726. Munos, R. (1997). A convergent reinforcement learning algorithm in the continuous case based on a Žnite difference method. In Proceedings of International Joint Conference on ArtiŽcial Intelligence (pp. 826–831). Munos, R., & Bourgine, P. (1998).Reinforcement learning for continuous stochastic control problems. In M. I. Jordan, M. J. Kearns, & S. A. Solla (Eds.), Advances in neural information processing systems, 10 (pp. 1029–1035). Cambridge, MA: MIT Press. Pareigis, S. (1998). Adaptive choice of grid and time in reinforcement learning. In M. I. Jordan, M. J. Kearns, & S. A. Solla (Eds.), Advances in neural information processing systems, 10 (pp. 1036–1042). Cambridge, MA: MIT Press. Peterson, J. K. (1993). On-line estimation of the optimal value function: HJBestimators. In C. L. Giles, S. J. Hanson, & J. D. Cowan (Eds.), Advances in neural information processing systems, 5 (pp. 319–326). San Mateo, CA: Morgan Kaufmann. Schaal, S. (1997). Learning from demonstration. In M. C. Mozer, M. I. Jordan, & T. Petsche (Eds.), Advances in neural information processing systems, 9 (pp. 1040– 1046). Cambridge, MA: MIT Press. Singh, S., & Bertsekas, D. (1997). Reinforcement learning for dynamic channel allocation in cellular telephone systems. In M. C. Mozer, M. I. Jordan, & T. Petsche (Eds.), Advances in neural information processing systems, 9 (pp. 974– 980). Cambridge, MA: MIT Press. Singh, S. P., Jaakkola, T., & Jordan, M. I. (1995). Reinforcement learning with soft state aggregation. In G. Tesauro, D. S. Touretzky, & T. K. Leen (Eds.), Advances in neural information processing systems, 7 (pp. 361–368). Cambridge, MA: MIT Press. Sutton, R. S. (1988). Learning to predict by the methods of temporal difference. Machine Learning, 3, 9–44. Sutton, R. S. (1995). TD models: Modeling the world at a mixture of time scales. In Proceedings of the 12th International Conference on Machine Learning (pp. 531– 539). San Mateo, CA: Morgan Kaufmann. Sutton, R. S. (1996). Generalization in reinforcement learning: Successful examples using sparse coarse coding. In D. S. Touretzky, M. C. Mozer, & M. E. Hasselmo (Eds.), Advances in neural information processing systems, 8 (pp. 1038– 1044). Cambridge, MA: MIT Press. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning. Cambridge, MA: MIT Press.

Reinforcement Learning in Continuous Time and Space

245

Tesauro, G. (1994). TD-Gammon, a self teaching backgammon program, achieves master-level play. Neural Computation, 6, 215–219. Tsitsiklis, J. N., & Van Roy, B. (1997). An analysis of temporal-difference learning with function approximation. IEEE Transactions on Automatic Control, 42, 674– 690. Watkins, C. J. C. H. (1989). Learning from delayed rewards. Unpublished doctoral dissertation, Cambridge University. Werbos, P. J. (1990). A menu of designs for reinforcement learning over time. In W. T. Miller, R. S. Sutton, & P. J. Werbos (Eds.), Neural networks for control (pp. 67–95). Cambridge, MA: MIT Press. Zhang, W., & Dietterich, T. G. (1996). High-performance job-shop scheduling with a time-delay TD(l ) network. In D. S. Touretzky, M. C. Mozer, & M. E. Hasselmo (Eds.), Advances in Neural Information Processing Systems, 8. Cambridge, MA: MIT Press. Received January 18, 1998; accepted January 13, 1999.

Neural Computation, Volume 12, Number 2

CONTENTS Article

Minimizing Binding Errors Using Learned Conjunctive Features Bartlett W. Mel and J´ozsef Fiser

247

Notes

Relationship Between Phase and Energy Methods for Disparity Computation Ning Qian and Samuel Mikaelian

279

N-tuple Network, CART, and Bagging Aleksander Ko lcz

293

Improving the Practice of ClassiŽer Performance Assessment N. M. Adams and D. J. Hand

305

Letters

Do Simple Cells in Primary Visual Cortex Form a Tight Frame? Emilio Salinas and L. F. Abbott

313

Learning Overcomplete Representations Michael S. Lewicki and Terrence J. Sejnowski

337

Noise in Integrate-and-Fire Neurons: From Stochastic Input to Escape Rates Hans E. Plesser and Wulfram Gerstner

367

Modeling Synaptic Plasticity in Conjunction with the Timing of Pre- and Postsynaptic Action Potentials Werner M. Kistler and J. Leo van Hemmen

385

On-line EM Algorithm for the Normalized Gaussian Network Masa-aki Sato and Shin Ishii

407

A General Probability Estimation Approach for Neural Computation Maxim Khaikine and Klaus Holthausen

433

On the Synthesis of Brain-State-in-a-Box Neural Models with Application to Associative Memory Fation Sevrani and Kennichi Abe

451

ARTICLE

Communicated by Shimon Edelman

Minimizing Binding Errors Using Learned Conjunctive Features Bartlett W. Mel Department of Biomedical Engineering, University of Southern California, Los Angeles, CA 90089, U.S.A. Jozsef ´ Fiser Department of Brain and Cognitive Sciences, University of Rochester, Rochester, NY 14627-0268, U.S.A. We have studied some of the design trade-offs governing visual representations based on spatially invariant conjunctive feature detectors, with an emphasis on the susceptibility of such systems to false-positive recognition errors—Malsburg’s classical binding problem. We begin by deriving an analytical model that makes explicit how recognition performance is affected by the number of objects that must be distinguished, the number of features included in the representation, the complexity of individual objects, and the clutter load, that is, the amount of visual material in the Želd of view in which multiple objects must be simultaneously recognized, independent of pose, and without explicit segmentation. Using the domain of text to model object recognition in cluttered scenes, we show that with corrections for the nonuniform probability and nonindependence of text features, the analytical model achieves good Žts to measured recognition rates in simulations involving a wide range of clutter loads, word sizes, and feature counts. We then introduce a greedy algorithm for feature learning, derived from the analytical model, which grows a representation by choosing those conjunctive features that are most likely to distinguish objects from the cluttered backgrounds in which they are embedded. We show that the representations produced by this algorithm are compact, decorrelated, and heavily weighted toward features of low conjunctive order. Our results provide a more quantitative basis for understanding when spatially invariant conjunctive features can support unambiguous perception in multiobject scenes, and lead to several insights regarding the properties of visual representations optimized for speciŽc recognition tasks. 1 Introduction

The problem of classifying objects in visual scenes has remained a scientiŽc and a technical holy grail for several decades. In spite of intensive research in Neural Computation 12, 247–278 (2000)

c 2000 Massachusetts Institute of Technology °

248

Bartlett W. Mel and J´ozsef Fiser

the Želd of computer vision and a large body of empirical data from the Želds of visual psychophysics and neurophysiology, the recognition competence of a two-year-old human child remains unexplained as a theoretical matter and well beyond the technical state of the art. This fact is surprising given that (1) the remarkable speed of recognition in the primate brain allows for only very brief processing times at each stage in a pipeline containing only a few stages (Potter, 1976; Oram & Perrett, 1992; Heller, Hertz, Kjær, & Richmond, 1995; Thorpe, Fize, & Marlot, 1996; Fize, Boulanouar, Ranjeva, Fabre-Thorpe, & Thorpe, 1998), (2) the computations that can be performed within each neural processing stage are strongly constrained by the structure of the underlying neural tissue, about which a great deal is known (Hubel & Wiesel, 1968; Szentagothai, 1977; Jones, 1981; Gilbert, 1983; Van Essen, 1985; Douglas & Martin, 1998), (3) the response properties of neurons in each of the relevant brain areas have been well studied, and evolve from stage to stage in systematic ways (Oram & Perrett, 1994; Logothetis & Sheinberg, 1996; Tanaka, 1996), and (4) computer systems may already be powerful enough to emulate the functions of these neural processing stages, were it only known what exactly to do. A number of neuromorphic approaches to visual recognition have been proposed over the years (Pitts & McCullough, 1947; Fukushima, Miyake, & Ito, 1983; Sandon & Urh, 1988; Zemel, Mozer, & Hinton, 1990; Le Cun et al., 1990; Mozer, 1991; Swain & Ballard, 1991; Hummel & Biederman, 1992; Lades et al., 1993; Califano & Mohan, 1994; Schiele & Crowley, 1996; Mel, 1997; Weng, Ahuja, & Huang, 1997; Lang & Seitz, 1997; Wallis & Rolls, 1997; Edelman & Duvdevani-Bar, 1997). Many of these neurally inspired systems involve constructing banks of feature detectors, often called receptive Želds (RFs), each of which is sensitive to some localized spatial conŽguration of image cues but invariant to one or more spatial transformations of its preferred stimulus—typically including invariance to translation, rotation, scale, or spatial distortion, as speciŽed by the task. 1 Since the goal to extract useful invariant features is common to most conventional approaches to computer vision as well, the “neuralness” of a recognition system lies primarily in its emphasis on feedforward, hierarchical construction of the invariant features, where the computations are usually restricted to simple spatial conjunction and disjunction operations (e.g., Fukushima et al., 1983), and/or specify the inclusion of a relatively large number of invariant features in the visual representation (see Mozer, 1991; Califano & Mohan, 1994; Mel, 1997)—anywhere from hundreds to millions. Neural approaches typically also involve some form of learning, whether supervised by category labels at the output layer (Le Cun et al., 1990), supervised directly within 1 A number of the above-mentioned systems have emphasized other mechanisms instead of or in addition to straightforward image Žltering operations (Zemel et al., 1990; Mozer, 1991; Hummel & Biederman, 1992; Lades et al., 1993; Califano & Mohan, 1994; Edelman & Duvdevani-Bar, 1997).

Minimizing Binding Errors

249

intermediate network layers (Fukushima et al., 1983), or involving purely unsupervised learning principles that home in on features that occur frequently in target objects (Fukushima et al., 1983; Zemel et al., 1990; Wallis & Rolls, 1997; Weng et al., 1997). While architectures of this general type have performed well in a variety of difŽcult, though limited, recognition problems, it has yet to be proved that a few stages of simple feedforward Žltering operations can explain the remarkable recognition and classiŽcation capacities of the human visual system, which is commonly confronted with scenes acquired from varying viewpoints, containing multiple objects drawn from thousands of categories, and appearing in an inŽnite panoply of spatial conŽgurations. In fact, there are reasons for pessimism. 1.1 The Binding Problem. One of the most persistent worries concerning this class of architecture is due to Malsburg (1981), who noted the potential for ambiguity in visual representations constructed from spatially invariant Žlters. A spatial binding problem arises, for example, when each detector in the visual representation reports only the presence of some elemental object attribute but not its spatial location (or, more generally, its pose). Under these unfavorable conditions, an object is hallucinated (i.e., detected when not actually present) whenever all of its elemental features are present in the visual Želd, even when embedded piecemeal in improper conŽgurations within a scattered coalition of distractor objects (see Figure 1A). This type of failure mode for a visual representation emanates from the dual pressures to cope with uncontrolled variation in object pose, which forces the use of detectors with excessive spatial invariances, and the necessity to process multiple objects simultaneously, which overstimulate the visual representation and increase the probability of ambiguous perception. One approach to the binding problem is to build a separate, full-order conjunctive detector for every possible view (e.g., spatial pose, state of occlusion, distortion, degradation) of every possible object, and then for each object provide a huge disjunction that pools over all of the views of that object. Malsburg (1981) pointed out that while the binding problem could thus be solved, the remedy is a false one since it leads to a combinatorial explosion in the number of needed visual processing operations (see Figure 1B). Other courses of action include (1) strategies for image preprocessing designed to segment scenes into regions containing individual objects, thus reducing the clutter load confronting the visual representation, and (2) explicit normalization procedures to reduce pose uncertainty (e.g., centering, scaling, warping), thereby reducing the invariance load that must be sustained by the individual receptive Želds. Both strategies reduce the probability that any given object’s feature set will be activated by a spurious collection of features drawn from other objects. 1.2 Wickelsystems. Preprocessing strategies aside, various workers have explored the middle ground between “binding breakdown” and “com-

250

Bartlett W. Mel and J´ozsef Fiser

Figure 1: (A) A binding problem occurs when an object’s representation can be falsely activated by multiple (or inappropriate) other objects. (B) A combinatorial explosion arises when all possible views of all possible objects are represented by explicit conjunctions. (C) Compromise representation containing an intermediate number of receptive Želds that bind image features to intermediate order.

Minimizing Binding Errors

251

binatorial catastrophe,” where a visual representation contains an intermediate number of detectors, each of which binds together an intermediatesized subset of feature elements in their proper relations before integrating out the irrelevant spatial transformations (see Mozer, 1991, for a thoughtful advocacy of this position). Under this compromise, each detector becomes a spatially invariant template for a speciŽc minipattern, which is more complex than an elemental feature, such as an oriented edge, but simpler than a full-blown view of an object (see Figure 1C). A set of spatially invariant features of intermediate complexity can be likened to a box of jumbled jigsaw puzzle pieces: although the Žnal shape of the composite object—the puzzle—is not explicitly contained in this peculiar “representation,” it is nonetheless unambiguously speciŽed: the collection of parts (features) can interlock in only one way. An important early version of this idea is due to Wickelgren (1969), who proposed a scheme for representing the phonological structure of spoken language involving units sensitive to contiguous triples of phonological features but not to the absolute position of the tuples in the speech stream. Though not phrased in this language, “Wickelfeatures” were essentially translation-invariant detectors of phonological minipatterns of intermediate complexity, analyzing the one-dimensional speech signal. Representational schemes inspired by Wickelgren’s proposal have since cropped up in other inuential models of language processing (McClelland & Rumelhart, 1981; Mozer, 1991). The use of invariant conjunctive features schemes to address representational problems in Želds other than vision highlights the fact that the binding problem is not an inherently visuospatial one. Rather, it can arise whenever the set of features representing an object (visual, auditory, olfactory, etc.) can be fully, and thus inappropriately, activated by input “scenes” that do not contain the object. In general, a binding problem has the potential to exist whenever a representation lacks features that conjoin an object’s elemental attributes (e.g., parts, spatial relations, surface properties) to sufŽciently high conjunctive order. On the solution side, binding ambiguity can be mitigated as in the scenario of Figure 1 by building conjunctive features that are increasingly object speciŽc, until the probability of false-positive activation for any object is driven below an acceptable threshold. Given that this conjunctive order-boosting prescription to reduce ambiguity is algorithmically trivial, it shifts emphasis away from the question as to how to eliminate binding ambiguity, toward the question as to whether, for a given recognition problem, ambiguity can be eliminated at an acceptable cost. 1.3 A Missing Theory. In spite of the importance of spatially invariant conjunctive visual representations as models of neural function, and the demonstrated practical successes of computer vision systems constructed along these lines, many of the design trade-offs that govern the performance of visual “Wickelsystems” in response to cluttered input images have re-

252

Bartlett W. Mel and J´ozsef Fiser

mained poorly understood. For example, it is unknown in general how recognition performance depends on (1) the number of object categories that must be distinguished, (2) the similarity structure of the object category distribution (i.e., whether object categories are intrinsically very similar or very different from each other), (3) the featural complexity of individual objects, (4) the number and conjunctive order of features included in the representation, (5) the clutter load (i.e., the amount of visual material in the Želd of view from which multiple objects must be recognized without explicit segmentation), and (6) the invariance load (i.e., the set of spatial transformations that do not affect the identities of objects, and that must be ignored by each individual detector). In a previous analysis that touched on some of these issues, Califano and Mohan (1994) showed that large, parameterized families of complex invariant features (e.g., containing 106 elements or more) could indeed support recognition of multiple objects in complex scenes without prior segmentation or discrimination among a large number of highly similar objects in a spatially invariant fashion. However, these authors were primarily concerned with the mathematical advantages of large versus small feature sets in a generic sense, and thus did not test their analytical predictions using a large object database and varying object complexity, category structure, clutter load, and other factors. Beyond our lack of understanding regarding the trade-offs inuencing system performance, the issue as to which invariant conjunctive features and of what complexity should be incorporated into a visual representation, through learning, has also not been well worked out as a conceptual matter. Previous approaches to feature learning in neuromorphic visual systems have invoked (1) generic gradient-based supervised learning principles (e.g., Le Cun et al., 1990) that offer no direct insight into the qualities of a good representation, (2) or straightforward unsupervised learning principles (e.g., Wallis & Rolls, 1997), which tend to discover frequently occurring features or principal components rather than those needed to distinguish objects from cluttered backgrounds on a task-speciŽc basis. Network approaches to learning have also typically operated within a Žxed network architecture, which largely predeŽnes and homogenizes the complexity of top-level features to be used for recognition, though optimal representations may actually require features drawn from a range of complexities, and could vary in their composition on a per-object basis. All this suggests that further work on the principles of feature learning is needed. In this article, we describe our work to test a simple analytical model that captures several trade-offs governing the performance of visual recognition systems based on spatially invariant conjunctive features. In addition, we introduce a supervised greedy algorithm for feature learning that grows a visual representation in such a way as to minimize false-positive recognition errors. Finally, we consider some of the surprising properties of “good” representations and the implications of our results for more realistic visual recognition problems.

Minimizing Binding Errors

253

2 Methods 2.1 Text World as a Model for Object Recognition. The studies described here were carried out in text world, a domain with many of the complexities of vision in general (see Figure 2): target objects are numerous (more than 40,000 words in the database), are highly nonuniform in their prior probabilities (relative probabilities range from 1 to 400,000), are constructed from a set of relatively few underlying parts (26 letters), and individually can contain from 1 to 20 parts. Furthermore, the parts from which words are built are highly nonuniform in their relative frequencies and contain strong statistical dependencies. Finally, word objects occur in highly cluttered visual environments—embedded in input arrays in which many other words are simultaneously present—but must nonetheless be recognized regardless of position in the visual Želd. Although text world lacks several additional complexities characteristic of real object vision (see section 6), it nonetheless provides a rich but tractable domain in which to quantify binding trade-offs and to facilitate the testing of analytical predictions and the development of feature learning algorithms. An important caveat, however, is that the use of text as a substrate for developing and testing our analytical model does not imply that our conclusions bear directly on any aspect of human reading behavior. Two databases were used in the studies. The word database contained 44,414 entries, representing all lowercase punctuation-free words and their relative frequencies found in 5 million words in the Wall Street Journal (WSJ) (available online from the Association for Computational Linguistics at http://morph.ldc.upenn.edu/). The English text database consisted of approximately 1 million words drawn from a variety of sources (KuLcera & Francis, 1967). 2.2 Recognition Using n-Grams. A natural class of visual features in text world is the position-invariant n-gram, deŽned here as a binary detector that responds when a speciŽc spatial conŽguration of n letters is found anywhere (one or more times) in the input array.2 The value of n is termed the conjunctive or binding order of the n-gram, and the diameter of the n-gram is the maximum span between characters speciŽed within it. For example, th** is a 3-gram of diameter 5 since it speciŽes the relative locations of three characters (including the space character but not wild card characters *), and spans a Želd Žve characters in width. The concept of an n-gram is used also in computational linguistics, though usually referring to n-tuples of consecutive words (Charniak, 1993).

2 An n-gram could also be multivalued, that is, respond in a graded fashion depending on the number of occurrences of the target feature in the input; the binary version was used here to facilitate analysis.

254

Bartlett W. Mel and J´ozsef Fiser

10

6

10

5

10

4

10

3

10

2

B

Non-Uniform Word Frequencies

Non-Uniform Word Sizes 8

Word Counts (x 1000)

Log Relative Frequency

A

10

7 6 5 4 3 2 1 0

1

1 2 3 4 5 6 7 8 9 1011121314 151617181920

Words Rank-Ordered by Frequency

C

Word Size (Letters)

Non-Uniform Letter Frequencies

D

450

Relative Frequency (x 1000)

1200

Relative Frequency

400 350 300 250 200 150 100 50 _ e t a o n h i s r d l u c wm f g y p b v k x j q z

Single Letters Ordered by Frequency Info Gain About Adjacent Letter (Bits)

1000 800 600 400

unifor m appr oximatio n

200 0

0

E

Non-Uniform 2-gram Frequencies

0

100

200

300

400

500

600

Adjacent 2-grams Ordered by Frequency

Statistical Dependencies Among Adjacent Letters 5 letter to left

4

letter to right 3 2 1 0 a

b

c d

e

f

g h

i

j

k

l m n o

p

q

r

s

t

u

v w x y

z

Figure 2: Text world was used as a surrogate for visual recognition in multiobject scenes. (A) Word frequencies in the 5-million-word Wall Street Journal (WSJ) corpus are highly nonuniform, following an approximately exponential decay according to Zipf’s law. (B) Words range from 1 to 20 characters in length; 7letter words are most numerous. (C,D) Relative frequencies of individual letters or adjacent letter pairs are highly nonuniform. Ordinate values represent the number of times the feature was found in the WSJ corpus. The dashed rectangle in D represents simplifying assumption regarding the 2-gram frequency distribution used for quantitative predictions below. (E) The parts of words show strong statistical dependencies. Columns show information gain about identity of the adjacent letter (or space character) to left or right, given the presence of letter indicated on the x-axis. The dashed horizontal line indicates perfect knowledge—log 2 (27) = 4.75 bits.

Minimizing Binding Errors

255

In relation to conventional visual domains, n-grams may be viewed as complex visual feature detectors, each of which responds to a speciŽc conjunction of subparts in a speciŽc spatial conŽguration, but with built-in invariance to a set of spatial transformations predeŽned by the recognition task. In relation to a shape-based visual domain, a 2-gram is analogous to a corner detector that signals the co-occurrence of a vertical and a horizontal edge, with appropriate spatial offsets, anywhere within an extended region of the image. Let R be a representation containing a set of position-invariant n-grams analyzing the input array and W be a set of word detectors analyzing R, with one detector for every entry in the word database. Each word detector receives input from some or all of the n-gram units in R that are contained in the respective word. However, not all of a word’s n-grams are necessarily contained in R, and not all of the word’s n-grams that are contained in R necessarily provide input to the word’s detector. For example, the representation might contain four spatially invariant features R = fa, c, o, sg, while the detector for the word cows receives input from features c and s, but not from the o feature, which is present in R, or from the w feature, which is not contained in R. Word detectors are binary-valued conjunctions, responding 1 when all of their inputs are active and 0 otherwise. The collection of ngrams feeding a word detector is hereafter referred to as the word’s minimal feature set. A correct recognition is scored when an input array of characters is presented, and each word detector is activated if and only if its associated word is present in the input. Any word detector activated when its corresponding word is not actually present in the input array is counted as a hallucination, and the recognition trial is scored as an error. Recognition errors for a particular representation R are reported as the percentage of input arrays drawn randomly from the English text database that generated one or more word hallucinations.

3 Results 3.1 Word-Word Collisions in a Simple 1-Gram Representation. We begin by considering the case without visual clutter and ask how often pairs of isolated words collide with each other within a binary 1-gram representation, where each word is represented as an unordered set of individual letters (with repeats ignored). This is the situation in which the binding problem is most evident, as schematized in Figure 1A. Results for the approximately 2 billion distinct pairs of words in the database are summarized in Table 1. About half of the 44,414 words contained in the database collide with at least one other word (e.g., cerebrum ´ cucumber , calculation ´ unconstitutional ). The largest cohort of 1-gram-identical words contained 28 members.

256

Bartlett W. Mel and J´ozsef Fiser

Table 1: Performance of a Binary 1-Gram Representation in Word-Word Comparisons. Quantity

Value

Comment

Number of 1-grams in R

27

a, b, . . . , z,

Number of distinct words

44,414

From 5 million words in the WSJ

Word-word comparisons

»2 billion

Number of ambiguous words

24,488

analogies suction scientists

´ ´ ´

gasoline continuous nicest

stare, arrest, tears, restates, reassert, rarest, easter . . .

Largest self-similar cohort

28 words

Baseline entropy in wordfrequency distribution

9.76 bits

< log 2 (44, 414) = 15.4 bits

1.4 bits

Narrows Želd to » 3 possible words

Residual uncertainty about identity of a randomly drawn word, knowing its 1-grams

As an aside, we noted that if the two words compared were required also to contain the same number of letters, the number of unambiguously represented words fell to 11,377, and if the words were compared within a multivalued 1-gram representation (where words matched only if they contained the identical multiset of letters, i.e., in which repeats were taken into account), the ambiguous word count fell further to 5,483, or about 12% of English words according to this sample. The largest self-similar cohort in this case contained only 5 words (traces , reacts , crates , caters , caster ). Although a 1-gram representation contains no explicit part-part relations and fails to pin down uniquely the identity of more than half of the English words in the database, it nonetheless provides most of the needed information to distinguish individually presented words. The baseline entropy in the WSJ word-frequency distribution is 9.76 bits, signiŽcantly less than the 15.4 = log2 (44,414) bits for an equiprobable word set. The average residual uncertainty about the identity of a word drawn from the WSJ word frequency distribution, given its 1-gram representation, is only 1.4 bits, meaning that for individually presented words, the 1-gram representation narrows the set of possibilities to about three words on average. 3.2 Word-Word Collisions in an Adjancent 2-Gram Representation.

Since many word objects have identical 1-gram representations even when full multisets of letters are considered, we examined the collision rate when an adjacent binary-valued 2-gram representation was used (i.e., coding the

Minimizing Binding Errors

257

Table 2: Performance of a Binary Adjacent 2-Gram Representation in WordWord Comparisons. Quantity Number of adjacent 2-grams Number of words Word-word comparisons Number of ambiguous words Largest self-similar cohort

Ambiguous word pairs that are linguistically distinct Words with identical adjacent 2-gram multisets

Value

Comment

729

[aa], [ab], . . . , [a ], . . . , [ z], [ ]

44,414 »2 billion 57 5 words

4 pairs

2 words

ohhh, ahhh, shhh, hmmm, whoosh, whirr , . . . ohhh ´ ohhhh ´ ohhhhh ´ ohhhhhh ´ ohhhhhhh asses possess seamstress intended

´ ´ ´ ´

assess possesses seamstresses indented

intended ´ indented

set of all adjacent letter pairs, including the space character). This representation is signiŽcant in that adjacent 2-grams are the simplest features that explicitly encode spatial relations between parts. The results of this test are summarized in Table 2. There were only 46 word-pair collisions in this case, comprising 57 distinct words. Forty-two of the 46 problematic word pairs differed only in the length of a string of repeated letters (e.g., ahhh ´ ahhhh ´ ahhhhh . . . , similarly for ohhh, ummm, zzz, wheee, whirrr ) or into sets representing varying spellings or misspellings of the same word also involving repeated letters (e.g., tepees ´ teepees, llama ´ lllama ). Only four pairs of distinct words collided that had different morphologies in the linguistic sense (see Table 2). When words were constrained to be the same length or to contain the identical multiset of adjacent 2-grams, only a single collision persisted: intended ´ indented . At the level of word-word comparisons, therefore, the binding of adjacent letter pairs is sufŽcient practically to eliminate the ambiguity in a large database of English words. We noted empirically that the inclusion of fewer than 20 additional nonadjacent 2-grams to the representation entirely eliminated the word-word ambiguity in this database, without recourse to multiset comparisons or word-length constraints. 4 An Analytical Model

The results of these word-word comparisons suggest that under some circumstances, a modest number of low-order n-grams can disambiguate a

258

Bartlett W. Mel and J´ozsef Fiser

large, complex object set. However, one of the most dire challenges to a spatially invariant feature representation, object clutter, remains to be dealt with. To this end, we developed a simple probabilistic model to predict recognition performance when a cluttered input array containing multiple objects is presented to a bank of feature detectors. To permit analysis of this problem, we make two simplifying assumptions. First, we assume that the features contained in R are statistically independent. This assumption is false for English n-grams: for example, [th] predicts that [he] will follow about one in three times, whereas the baseline rate for [he] is only 0.5%. Second, we assume that all the features in R are activated with equal probability. This is also false in English: for example ed occurs approximately 90,000 times more often than [mj] (as in ramjet ). Proceeding nonetheless under these two assumptions, it is straightforward to calculate the probability that no object in the database is hallucinated in response to a randomly drawn input image. We Žrst calculate the probability o that all of the features feeding a given object detector are activated by an arbitrary input image:

o=

¡ d ¡ w¢ c¡w ¡ d¢ = c

( d ¡ w)! c! ³ c ´w ¼ , ( c ¡ w)! d! d

(4.1)

where d is the total number of feature detectors in R , w is the size of the object’s minimal feature set, i.e., the number of features required by the target object (word) detector, and c is the number of features activated by a “cluttered” multiobject input image. The left-most combinatorial ratio term in equation 4.1 counts the number of ways of choosing c from the set of d detectors, including all w belonging to the target object, as a fraction of the total number of ways of choosing c of the d detectors in any fashion. The approximation in equation 4.1 holds when c and d are large compared to w, as is generally the case. Equation 4.1 is, incidentally, the value of a “hypergeometric” random variable in the special case where all of the target elements are chosen; this distribution is used, for example, to calculate the probability of picking a winning lottery ticket. The probability that an object is hallucinated (i.e., perceived but not actually present) is given by hi =

oi ¡ qi , 1 ¡ qi

(4.2)

where qi is the probability that the object does in fact appear in the input image, which by deŽnition means it cannot be hallucinated. According to Zipf’s law, however, which tells us that most objects in the world are quite uncommon (see Figure 2A), we may in some cases make the further simpli-

Minimizing Binding Errors

259

Žcation that qi ¿ oi , giving hi ¼ oi . We adopt this assumption in the following. (This simpliŽcation would result also from the assumption that inputs consist of object-like noise structured to trigger false-positive perceptions.) We may now write the probability of veridical perception—the probability that no object in the database is hallucinated, µ ³ c ´w ¶N , pv = (1 ¡ h) N ¼ 1 ¡ d

(4.3)

where N is the number of objects in the database. This expression assumes a homogeneous object database, where all N objects require exactly w features in R. Given the independence assumption, the expression for a heterogeneousQobject database consists of a product of object-speciŽc probabilities c wi pv = i (1 ¡ ( d ) ), where wi is the number of features required by object i. Note that objects (words) containing the same number of parts (letters) do not necessarily activate the same number of features (n-grams) in R, depending on both the object and the composition of R. From an expansion to Žrst order of equation 4.3 around ( c/ d) w = 0, which gives pv = 1¡N(c/ d) w , it may be seen that perception is veridical (i.e., pv ¼ 1) when ( c/ d) w ¿ 1/ N. Thus, recognition errors are expected to be small when (1) a cluttered input scene activates only a small fraction of the detectors in R, (2) individual objects depend on as many detectors in R as possible, and (3) the number of objects to be discriminated is not too large. The Žrst two effects generate pressure to grow large representations, since if d is scaled up and c and w scale with it (as would be expected for statistically independent features), recognition performance improves rapidly. Conveniently, the size of the representation can always be increased by including new features of higher binding order or larger diameters, or both. It is also interesting to note that item 1 in the previous paragraph appears to militate for a sparse visual representation, while equation 4.2 militates for a dense representation. This apparent contradiction reects the fundamental trade-off between the need for individual objects to have large minimal feature sets in order to ward off hallucinations and the need for input scenes containing many objects to activate as few features as possible, again in order to minimize hallucinations. This trade-off is not captured within the standard conception of a feature space, since the notion of “similarity as distance” in a feature space does not naturally extend to the situation in which inputs represent multiobject scenes and require multiple output labels. 4.1 Fitting the Model to Recognition Performance in Text World. We ran a series of experiments to test the analytical model. Input arrays varying in width from 10 to 50 characters were drawn at random from the text database and presented to R. The representation contained either all adjacent 2-grams, called Rf2g to signify a diameter of 2, or the set of all 2-grams of diameter 2 or 3, called Rf2,3g , where Rf2,3g was twice the size of Rf2g .

260

Bartlett W. Mel and J´ozsef Fiser

In individual runs, the word database was restricted to words of a single length—two-letter words (N = 253), Žve-letter words (N = 4024), or sevenletter words (N = 7159)—in order for the value of w to be kept relatively constant within a trial.3 For each word size and input size and for each of the two representations, average values for w and c were measured from samples drawn from the word or text database. Not surprisingly, given the assumption violations noted above (feature independence and equiprobability), predicting recognition performance using these measured average values led to gross underestimates of the hallucination errors actually generated by the representation. Two kinds of corrections to the model were made. The Žrst correction involved estimating a better value for d, the number of n-grams contained in the representation, since some of the detectors nominally present in Rf2g and Rf2,3g were infrequently, if ever, actually activated. The value for d was thus down-corrected to reect the number of detectors that would account for more than 99.9% of the probability density over the detector set: for Rf2g , the representation size dropped from d = 729 to d0 = 409, while for Rf2,3g , d was cut from 1,458 to 948. The second correction was based on the fact that the n-grams activated by English words and phrases are highly statistically redundant, meaning that a set of d0 n-grams has many fewer than d0 underlying degrees of freedom. We therefore introduced a redundancy factor r, which allowed us to “pretend” that the representation contained only d0 / r virtual n-grams, and a word or an input image activated only w/ r or c/ r virtual n-grams in the representation, respectively. (Note that since c and d appear only as a ratio in the approximation of equation 4.3, their r-corrections cancel out.) By systematically adjusting the redundancy factor depending on test condition (ranging from 1.60 to 3.45), qualitatively good Žts to the empirical data could be generated, as shown in Figure 3. The redundancy factor was generally larger for long words relative to short words, reecting the fact that long English words have more internal predictability (i.e., Žll a smaller proportion of the space of long strings than short words Žll in the space of short strings). The r-factor was also larger for Rf2,3g than for Rf2g , reecting the fact that the increased feature overlap due to the inclusion of larger feature diameters in Rf2,3g leads to stronger interfeature correlations, and hence fewer underlying degrees of freedom. Thus, with the inclusion of correction factors for nonuniform activation levels and nonindependence of the features in R , the remarkably simple relation in equation 4.3 captures the main trends that describe recognition performance in scenarios with database sizes ranging from 253 to 7159 words, words containing between

3 Since R f2g and R f2, 3g contained all n-grams of their respective sizes, the value of w could vary only from word to word to the extent that words contained one or more repeated n-grams.

Minimizing Binding Errors

261

Figure 3: Fits of equation 4.3 to recognition performance in text world. (A) The representation, designated R f2g , contained the complete set of 729 adjacent 2grams (including the space character). The x-axis shows the width of a text window presented to the n-gram representation, and the y-axis shows the probability that the input generates one or more hallucinated words. Analytical predictions are shown in dashed lines using redundancy factors speciŽed in the legend. (B) Same as A, with results for R f2,3g .

262

Bartlett W. Mel and J´ozsef Fiser

2 and 7 letters, input arrays containing from 10 to 50 characters, and using representations varying in size by a factor of 2. We did not attempt to Žt larger ranges of these experimental variables. Regarding the sensitivity of the model Žts to the choice of d0 and r, we noted Žrst that Žts were not strongly affected by the cumulative probability cutoff used to choose d0 —qualitatively reasonable Žts could be also generated using a cutoff value of 95%, for example, for which d0 values for R f2,3g and Rf2g shrank to 220 and 504, respectively. In contrast, we noted that a change of one part per thousand in the value of r could result in a signiŽcant change in the shape a curve generated by equation 4.3, and with it significant degradation of the quality of a Žt. However, the systematic increase in the optimal value of r with increasing word size and representation size indicates that this redundancy correction factor is not an entirely free parameter. It is also worth emphasizing that a redundancy “factor,” which uniformly scales w, c, and d, is a very crude model of the complex statistical dependencies that exist among English n-grams; the fact that any set of small r values can be found that tracks empirical recognition curves over wide variations in the several other task variables is therefore signiŽcant. 5 Greedy Feature Learning

Several lessons from the analysis and experiments are that (1) larger representations can lead to better recognition performance, (2) only those features that are actually used should be included in a representation, (3) considerable statistical redundancy exists in a fully enumerated (or randomly drawn) set of conjunctive features, which should be suppressed if possible, and (4) features of low binding order, while potentially adequate to represent isolated objects, are insufŽcient for veridical perception of scenes containing multiple objects. In addition, we infer that different objects are likely to have different representational requirements depending on their complexity and that the composition of an ideal representation is likely to depend heavily on the particular recognition task. Taken together, these points suggest that a representation should be learned that includes higher-order features only as needed. In contrast, blind enumeration of an ever-increasing number of higher-order features is an expensive and relatively inefŽcient way to proceed. Standard network approaches to supervised learning could in principle address all of the points raised above, including Žnding appropriate weights for features of various orders depending on their utility and helping to decorrelate the input representation. However, the potential need to acquire features of high order—so high that enumeration and testing of the full cohort of high-order features is impractical—compels the use of some form of explicit search as part of the learning process. The use of greedy, incremental, or heuristic techniques to grow a neural representation through increasing nonlinear orders is not new (e.g., Barron, Mucciardi, Cook, Craig,

Minimizing Binding Errors

263

& Barron, 1984; Fahlman & Lebiere, 1990), though such techniques have been relatively little discussed in the context of vision. To cope with the many constraints that must be satisŽed by a good visual representation, we developed a greedy supervised algorithm for feature learning that (1) selects the order in which features should be added to the representation, and (2) selects which features added to the representation should contribute to the minimal feature sets of each object Q in the database. By differentiating the log probability uv = log(pv ) = log i (1 ¡ hi ) with respect to the hallucination probability of the ith object, we Žnd that ¡1 duv = , 1 ¡ hi dhi

(5.1)

indicating that the largest multiplicative increments to pv are realized by squashing the largest values of hi = ( c/ d) wi . Practically, this is achieved by incrementing the w-values (i.e., growing the minimal feature sets) of the most frequently hallucinated objects. The following learning algorithm aims to do this in an efŽcient way: 1. Start with a small bootstrap representation. (We typically used 50 2grams found to be useful in pilot experiments.) 2. Present a large number of text “images” to the representation. 3. Collect hallucinated words—any word that was ever detected but not actually present—in an input image. 4. For each hallucinated word and the input array that generated it, increment a global frequency table for every n-gram (up to a maximum order of 5 and diameter of 6) that is (i) contained in the hallucinated word, (ii) not contained in the offending input, and (iii) not currently included in the representation. 5. Choose the most frequent n-gram in this table of any order and diameter, and add it to the current representation. 6. Build a connection from the newly added n-gram to any word detector involved in a successful vote for this feature in step 4. Inclusion of this n-gram in these words’ minimal feature sets will eliminate the largest number of word hallucination events encountered in the set of training images. 7. Go to 2. We ran the algorithm using input arrays 50 characters in width (often containing 10 or more words) and plotted the number of hallucinated words (see Figure 4A) and proportion of input arrays causing hallucinations (see Figure 4B) as a function of increasing representation size during learning. The periodic ratcheting in the performance curves (most visible in the lower curve of Figure 4B) was due to staging of the training process for efŽciency

264

Bartlett W. Mel and J´ozsef Fiser

reasons: batches of 1000 input arrays were processed until a Žxed hallucination error criterion of 1% was reached, when a new batch of 1000 random input arrays was drawn, and so on. The envelope of the oscillating performance curves provides an estimate of the true performance curve that would result from an inŽnitely large training corpus. Learning was terminated when the average initial error rate for three consecutive new training batches fell below 1%. Given that this termination condition was typically reached in fewer than 50 training epochs (consisting of 50,000 50-character input images), no more than half the text database was visited by random drawings of training-testing sets. We found that for 50-character input arrays, the hallucination error rate fell below 1% after the inclusion of 2287 n-grams in R. These results were typical. The inferior performance of a simple unsupervised algorithm, which adds n-grams to the representation in order of their raw frequencies in English, is shown in Figure 4AB for comparison (upper dashed curves). The counts of total words hallucinated (see Figure 4A), with initial values indicating many 10s of hallucinated words per input, can be seen to fall far faster than the hallucination error rate (see Figure 4B), since a sentence was scored as an error until its very last hallucinated word was quashed.

Figure 4: Facing page. Results of greedy n-gram learning algorithm using input arrays 50 characters in width and a 50-element bootstrap representation. Training was staged in batches of 1000 input arrays until the steady-state hallucination error rate fell below 1%. (A) Growth of the representation is plotted along the x-axis, and the number of words hallucinated at least once in the current 1000-input training batch is plotted on the y-axis. The drop is far faster for the greedy algorithm (lower curve) than for a frequency-based unsupervised algorithm (upper curve). Unsupervised bootstrap representation also included the 27 1-grams (i.e., individual letters plus space character) that were excluded from the bootstrap representation used in the greedy learning run. Without these 1-grams, performance of the unsupervised algorithm was even more severely hampered; with these 1-grams incorporated in the greedy learning runs, the wvalues of every object were unduly inated. (B) Same as (A), but the y-axis shows the proportion of input arrays producing at least one hallucination error. Jaggies occur at transitions to new training batches each time the 1% error criterion was reached. Asymptote is reached in this run after the inclusion of 2287 n-grams. (C) A breakdown of learned representation showing relative numbers of n-grams of each order and diameter. (D) The column height shows the minimal feature set size after learning, that is, average number of n-grams feeding word conjunctions for words of various lengths. The dark lower portion of the columns shows initial counts based on full connectivity to bootstrap representation; the light upper portions show an increment in w due to learning. Nearly constant w values after learning reect the tendency of the algorithm to grow the smallest minimal feature sets. The dashed line corresponds to one n-gram per letter in word.

Minimizing Binding Errors

265

One of the most striking outcomes of the greedy learning process is that the learned representation is heavily weighted toward low-order features, as shown in Figure 4C. Thus, while the learned representation includes ngrams up to order 5, the number of higher-order features required to remedy perceptual errors falls off precipitously with increasing order—far faster than the explosive growth in the number of available features at each order. Quantitatively, the ratios of n-grams included in R to n-grams contained in the word database for orders 2, 3, 4, and 5 were 0.42, 0.0095, 0.00017, and 0.0000055, respectively. The rapidly diminishing ratios of useful features to available features at higher orders conŽrm the impracticality of learning on a fully enumerated high-order feature set and illustrate the natural bias in the learning procedure to choose the lowest-order features that “get the job done.” The dominance of low-order features is of considerable practical signiŽcance, since higher-order features are more expensive to compute, less robust to image degradation, operate with lower duty cycles, and provide a more limited basis for generalization. The relatively broad distribution of

A

Fewer Words are Hallucinated as Representation Grows

Inputs Causing Hallucinations (%)

10000

Words Hallucinated

9000 8000 7000 6000

Unsupervised

5000

Greedy

4000 3000 2000 1000 0 0

1000

2000

3000

B

4000

Fewer Inputs Cause Hallucinations as Representation Grows 100 90 80 70 60

30 20 10 0

5000

Sizes of N-grams in Final Learned Representation

0

D

6 5

1200 1000

4

800

3

600

2

400 200 0 2

3 4 N-gram Order

5

Mean Number of N-grams

Number of N-grams

diameter

1000

2000

3000

4000

5000

N-grams Included in Representation

1600 1400

Greedy

40

N-grams Included in Representation

C

Unsupervised

50

Number of N-grams Feeding Words of Varying Lengths 20 18 16 14 12 10 8 6 4

initial learned

2 0 1 2 3 4 5 6 7 8 9 1011121314151617 181920

Word Size (Letters)

266

Bartlett W. Mel and J´ozsef Fiser

diameters of the learned feature set are also shown in Figure 4C for n-grams of each order. In spite of its relatively small size, it is likely that the n-gram representation produced by the greedy algorithm could be signiŽcantly compacted without loss of recognition performance, by alternately pruning the least useful n-grams from R (which lead to minimal increments in error rate) and retraining back to criterion. However, such a scheme was not tried. After learning to the 1% error criterion, a 50-character input array activated an average of 168 (7.3%) of the 2287 detectors in R, while the average minimal feature set size for an individual word was 11.4. By comparison, the initial c/ d ratio for 50-character inputs projected onto the 50-element bootstrap representation was a much less favorable 44%. Thus, a primary outcome of the learning algorithm in this case involving heavy clutter is a larger, more sparsely activated representation, which minimizes collisions between input images and the feature sets associated with individual target words. Further, the estimated r value from this run was 1.95, indicating signiŽcantly less redundancy in the learned representation than was estimated using seven-letter words (r = 3.45) in the fully enumerated 2-gram representations reported in Figure 3; this is in spite of the fact that the learned representation contained more than twice the number of active features as the fully enumerated 2-gram representation. The total height of each column in Figure 4D shows the w value—the size of the minimal feature sets after learning for words of various lengths. The dark lower segment of each column counts initial connections to the word conjunctions from the 50-element bootstrap representation, while the average number of new connections gained during learning for words of each length is given by the height of each gray upper segment. Given the algorithm’s tendency to expand the smallest minimal feature sets, the w values for short words grew proportionally much more than those for longer words, producing a Žnal distribution of w values that was remarkably independent of word length for all but the shortest words. The w values for one-, two-, and three-letter words were lower either because they reached their maximum possible values (4 for one-letter words) or because these words contained rarer features, on average, derived from a relatively high concentration in the database of unpronounceable abbreviations, acronyms, and so forth. The dashed line indicates a minimal feature set size equal to the number of letters contained in the word; the longest words can be seen to depend on substantially fewer n-grams than they contain letters. The pressures inuencing both the choice of n-grams for inclusion in R and the assignment of n-grams to the minimal feature sets of individual words were very different from those associated with a frequency-based learning scheme. Thus, the features included in R were not nececssarily the most common ones, and the most commonly occurring English words were not necessarily the most heavily represented.

Minimizing Binding Errors

267

Table 3: First 10 n-Grams to Be Added to R During a Learning Run with 50-Character Input Arrays, Initialized with a 200-Element Bootstrap Representation. Order of Inclusion 1 2 3 4 5 6 7 8 9 10

n-Gram

Relative Frequency

Cumulative %

[z ] [ j] [k* ] [x ] [ki] [ q] [ *v] [ **z] [o*i] [p*** ]

417 14,167 39,590 6090 20,558 8771 24,278 3184 37,562 55,567

8 55 71 42 62 47 64 31 70 76

Note: The “ ” character represents the space character, and “*” matches any character. A value of k in the cumulative distribution column indicates that the total probability summed over all less common features is k%.)

First, in lieu of choosing the most commonly occurring n-grams, the greedy algorithm prefers features that are relatively rare in English text as a whole but relatively common in those words most likely to be hallucinated: short words and words containing relatively common English structure. The paradoxical need to Žnd features that distinguish objects from common backgrounds, but do so as often as possible, leads to a representation containing features that occur with moderate frequency. Table 3 shows the Žrst 10 n-grams added to a 200-element bootstrap representation during learning in one experimental run. The features’ relative frequencies in English are shown, which jump about erratically, but are mostly from the middle ranges of the cumulative probability distribution for English n-grams. Most of these Žrst 10 features include a space character, indicating that failure to represent elemental features properly in relation to object boundaries is a major contributor to hallucination errors in this domain. SpeciŽcally, we found that hallucination errors are frequently caused by the presence in the input of multiple “distractor ” objects that together contain most or all of the features of a target object, for example, in the sense that national and vacation together contain nearly all the features of the word nation. To distinguish nation from the superposition of these two distractors requires the inclusion in the representation of a 2-gram such as [n***** ] whose diameter of 7 exceeds the maximum value of 6 arbitrarily established in the present experiments. Having established that the features included in R are not keyed in a simple way to their relative frequencies in English, we also observed that no simple relation exists between word frequency and the per-word representational cost, that is, the minimal feature set size established during

268

Bartlett W. Mel and J´ozsef Fiser Table 4: Correlation Between Word Frequency in English and Minimal Feature Set Size After Learning Was Only 0.1. Word

Relative Frequency

Minimal Feature Set Sizea

resting leading

100 315

25 24

buffalo unhappy

188 144

8 7

thereat fording

2 1

25 23

bedknob jackdaw

1 1

7 6

a Number of

n-grams in R that feed the word’s conjunction, that is, which must be present in order for a word to be detected. Table shows examples of common and uncommon words with large and small minimal features sets.

learning. In fact, if attention is restricted to the set of all words of a given length, with nominally equivalent representational requirements, the correlation between relative frequency of the word in English and its postlearning minimal feature set size was only 0.1. Thus some common words require a conjunction of many n-grams in R, while others require few, and some rare words require many n-grams while others require few (see Table 4). 5.1 Sensitivity of the Learned Representation to Object Category Structure. Words are unlike common objects in that every word forms its own

category, which must be distinguished from all other words. In text world, therefore, the critical issue of generalization to novel class exemplars cannot be directly studied. To address this shortcoming partially and assess the impact of broader category structures on the composition of the n-gram representation, we modiŽed the learning algorithm so that a word was considered to be hallucinated only if no similar—rather than identical—word was present in the input when the target word was declared recognized. In this case, similar was deŽned as different up to any single letter replacement—for example, dog = ffog, dig, . . .g, but dog 6 = ffig, og, dogs, . . .g. The resulting category structure imposed on words was thus a heavily overlapping one in which most words were similar to several other words.4 4 This learning procedure, combined with the assumption that every word is represented by only a simple conjunction of features in R (disjunctions of multiple prototypes per word were not supported), cannot guarantee that every word detector is activated

Minimizing Binding Errors

269

Decrease in Representational Costs for Broader, Overlapping Object Categories 1400 tolerance = 0

Number of N-grams

1200

tolerance = 1

1000 800 600 400 200 0 2

3 4 N-gram Order

5

Figure 5: Breakdown of representations produced by two learning runs differing in breadth of category structure. Fifty-character input arrays were used as before to train to an asymptotic 1% error criterion, this time beginning with a 200-element bootstrap representation. Run with tolerance = 0 was as in Figure 4, where a hallucination was scored whenever a word’s minimal feature set was activated, but the word was not identically present in the input. In a run with tolerance = 1, a hallucination was not scored if any similar word was present in the input, up to a single character replacement. The latter run, with a more lenient category structure, led to a representation that was signiŽcantly smaller, since it was not required to discriminate as frequently among highly similar objects.

The main result of using this more tolerant similarity metric during learning is that fewer word hallucinations are encountered per input image, leading to a Žnal representation that is smaller by more than half (see Figure 5). In short, the demands on a conjunctive n-gram representation are lessened when the object category structure is broadened, since fewer distinctions among objects must be made. 5.2 Scaling of Learned Representation with Input Clutter. As is conŽrmed by the results of Figure 3, one of the most pernicious threats to a spatially invariant representation is that of clutter, deŽned here as the quantity of input material that must be simultaneously processed by the representain response to any of the legal variants of the word. Rather, the procedure allows a word detector to be activated by any of the legal variants without penalty.

270

Bartlett W. Mel and J´ozsef Fiser Table 5: Measured Average Values of c, d, and w and Calculated Values of r in a Series of Three Learning Runs with Input Clutter Load Increased from 25 to 75 Characters, Trained to a Fixed Error Criterion of 3%. Input Width

c

d

w

r

25 50 75

55 152 253

816 1768 2634

7.6 10.7 12.8

1.45 1.85 2.11

Note: The 50-element bootstrap representation was used to initialize all three runs. Equation 4.3 was used to Žnd values of r for each run, such that with w ! w/ r, the measured values of c and d, and N = 44,414, a 3% error rate was predicted.

tion. The difŽculty may be easily seen in equation 4.3, where an unopposed increase in the value of c leads to a precipitous drop in recognition performance. On the other hand, equation 4.3 also indicates that increases in c can be counteracted by appropriate increases in d and w, which conserve the value of ( c/ d) w . To examine this issue, we systematically increased the clutter load in a series of three learning runs from 25 to 75 characters, training always to a Žxed 3% error criterion, and recorded for each run the postlearning values of c, d, and w. Although the WSJ database contained a wide range of word sizes (from 1 to 20 characters in length), the measurement of a single value for w averaged over all words was justiŽed since, as shown in Figure 4D, the learning algorithm tended to equalize the minimal feature set sizes for words independent of their lengths. Under the assumption of uniformly probable, statistically independent features in the learned representations, the value of h = (c/ d)w for all three runs should be equal to » 7 £ 10¡7 , the value that predicts a 3% error rate for a 44,414-object database using equation 4.3. However, corrections (w ! w/ r) were needed to factor out unequal amounts of statistical redundancy in representations of various sizes, where larger representations contained more redundancy. The measured values of c, d, and w and the empirically determined r values are shown for each run in Table 5. As in the runs of Figure 3, redundancy values were again systematically greater for the larger representations generated, and the redundancy values calculated here for the learned representation were again signiŽcantly smaller than those seen for the fully enumerated 2-gram representation. 6 Discussion

Using a simple analytical model as a taking-off point and a variety of simulations in text world, we have shown that low-ambiguity perception in un-

Minimizing Binding Errors

271

segmented multiobject scenes can occur whenever the probability of fully activating any single object’s minimal feature set, by a randomly drawn scene, is kept sufŽciently low. One prescription for achieving this condition is straightforward: conjunctive features are mined from hallucinated objects until false-positive recognition errors are suppressed to criterion. Even in a complex domain containing a large number of highly similar object categories and severe background clutter, this prescription can produce a remarkably compact representation. This conŽrms that brute-force enumeration of large families or, worse, all high-order conjunctive features—the combinatorial explosion recognized by Malsburg—is not inevitable. The analysis and experiments reported here have provided several insights into the properties of feature sets that support hallucination-proof recognition and into learning procedures capable of building these feature sets: 1. EfŽcient feature sets are likely to (1) contain features that span the range from very simple to very complex features, (2) contain relatively many simple features and relatively few complex features, (3) emphasize features that are only moderately common (giving a representation that is neither sparse nor dense) in response to the conicting constraints that features should appear frequently in objects but not in backgrounds also composed of objects, and (4) in spatial domains, emphasize features that encode the relations of parts to object boundaries. 2. An efŽcient learning algorithm works to drive toward zero, and therefore to equalize, the false-positive recognition rates for all objects considered individually. Thus, frequently hallucinated objects—objects with few parts or common internal structure, or both—demand the most attention during learning. Two consequences of this focus of effort on frequently hallucinated objects are that (1) the average value of w, the size of the minimal feature set required to activate an object, becomes nearly independent of the number of parts contained in the object, so that simpler objects are relatively more intensively represented than complex objects, and (2) among objects of the same part complexity, the minimal conjunctive feature sets grow largest for objects containing the most common substructures, though these are not necessarily the most common objects. A curious implication of the pressure to represent objects heavily that are frequently hallucinated is that the backgrounds in which objects are typically embedded can strongly inuence the composition of the optimal feature set for recognition. 3. The demands on a visual representation are heavily dependent on the object category structure imposed by the task. Where object classes are large and diffuse, the required representation is smaller and weighted

272

Bartlett W. Mel and J´ozsef Fiser

to features of lower conjunctive order, whereas for a category structure like words, in which every object forms its own category that must often be distinguished from a large number of highly similar objects (e.g., cat from cats, cut, rat), the representation must be larger and depend more heavily on features of higher conjunctive order. 6.1 Further Implications of the Analytical Model. Equation 4.3 provides an explicit mathematical relation among several quantities relating to multiple-object recognition. Since the equation assumes statistical independence and uniform activation probability of the features contained in the representation, neither of which is a valid assumption in most practical situations, we were unable to predict recognition errors using measured values for d, c, and w. However, we found that error rates in each case examined could be Žtted using a small correction factor for statistical redundancy, which uniformly scaled d, c, and w and whose magnitude grew systematically for larger collections of features activated by larger—more redundant—units of text. From this we conclude that equation 4.3 at least crudely captures the quantitative trade-offs governing recognition performance in receptive-Želd-based visual systems. One of the pithier features of equation 4.3 is that it provides a criterion for good recognition performance: ( c/ d) w ¿ 1/ N. The exponential dependence on the minimal feature set size, w, means that modest increases in w can counter a large increase in the number of object categories, N, or unfavorable increases in the activation density, c/ d, which could be due to either an increase in the clutter load or a collapse in the size of the representation. For example, using roughly the postlearning values of c/ d » 0.07 and r » 2 from the run of Figure 4, we Žnd that increasing w from 10 to 12 can compensate for a more than 10-fold increase N, or a nearly 60% increase in the c/ d ratio, while maintaining the same recognition error rate. The Žrst-order expansion of equation 4.3 also states that recognition errors grow as the wth power of c/ d, where w is generally much larger than 1. A visual system constructed to minimize hallucinations therefore abhors uncompensated increases in clutter or reduction in the size of the representation. These effects are exactly those that have motivated the various compensatory strategies discussed in section 1. The abhorrence of clutter generates strong pressure to invoke any readily available segmentation strategies that limit processing to one or a small number of objects at a time. For example, cutting out just half the visual material in the input array when w/ r = 5, as in the above example, knocks down the expected error rate by a factor of 32. The pressure to reduce the number of objects simultaneously presented to the visual system could account in part for the presence in biological visual systems of (1) a fovea, which leads to a huge overrepresentation of the center of Žxation and marginalization of the surround, (2) covert attentional mechanisms, which selectively upor down-modulate sensitivity to different portions of the visual Želd, and

Minimizing Binding Errors

273

(3) dynamical processes that segment “good” Žgures (in the Gestalt sense) from background. The second pressure involved in maintaining a low c/ d ratio involves maintaining a visual representation of adequate size, as measured by d. One situation in which this is particularly difŽcult arises when the task mandates that objects be distinguished under excessively wide ranges of spatial variation, an invariance load that must be borne in turn by each of the individual feature detectors in the representation. For example, consider a relatively simple task in which the orientation of image features is a valid cue to object identity, such as a task in which objects usually appear in a standard orientation. In this case a bank of several orientation-speciŽc variants of each feature can be separately maintained within the representation. In contrast, in a more difŽcult task that requires that objects be recognized in any orientation, each bank of orientation-speciŽc detectors must be pooled together to create a single, orientation-invariant detector for that conjunctive feature. The inclusion of this invariance in the task deŽnition can thus lead to an order-of-magnitude reduction in the size of the representation, which, unopposed could produce a catastrophic breakdown of recognition according to equation 4.3. As a rule of thumb, therefore, the inclusion of an additional order-of-magnitude invariance must be countered by an order-of-magnitude increase in the number of distinct feature conjunctions included in the representation, drawn from the reservoir of higher-order features contained in the objects in question. The main cost in this approach lies in the additional hardware, requiring components that are more numerous, expensive, prone to failure, and used at a far lower rate. 6.2 Space, Shape, Invariance, and the Binding Problem. While one emphasis in this work has been the issue of spatial invariance in a visual representation, we reiterate that the mathematical relation expressed by equation 4.3 knows nothing of space or shape or invariance, but rather views objects and scenes as collections of statistically independent features. What, then, is the role of space? From the point of view of predicting recognition performance and with regard to the quantitative trade-offs discussed here, we can Žnd no special status for worlds in which spatial relations among parts help to distinguish objects, since spatial relations may be viewed simply as additional sources of information regarding object identity. (Not all worlds have this property; an example is olfactory identiŽcation.) Further, the issue of spatial invariance plays into the mathematical relation of equation 4.3 only indirectly, in that it speciŽes the degree of spatial pooling needed to construct the invariant features included in the representation; once again, the details relating to the internal construction of these features are not visible to the analysis. In this sense, the spatial binding problem, which appears at Žrst to derive from an underspeciŽcation of the spatial relations needed to glue an object together, in fact reduces to the more mundane problem of ambiguity arising

274

Bartlett W. Mel and J´ozsef Fiser

from overstimulation of the visual representation by input images—that is, a c/ d ratio that is simply too large. The hallucination of objects based on parts contained in background clutter can thus occur in any world, even one in which there is no notion of space (e.g., matching bowls of alphabet soup). However, the fact that object recognition in real-world situations typically depends on spatial relations between parts and the fact that recognition invariances are also very often spatial in nature together strongly inuence the way in which a visual representation based on spatially invariant, receptive Želds can be most efŽciently constructed. In particular, many economies of design can be realized through the use of hierarchy, as exempliŽed by the seminal architecture of Fukushima et al. (1983). The key observation is that a population of spatially invariant, higher-order conjunctive features generally shares many underlying processing operations. The present analysis is, however, mute on this issue. 6.3 Relations to “Real” Vision. The quantitative relations given by equation 4.3 cast the problem of ambiguous perception in cluttered scenes in the simple terms of set intersection probabilities: a cluttered scene activates c of the d feature detectors at random and is correctly perceived with high probability as long as the set of c activated features only rarely includes all w features associated with any one of the N known objects. The analysis is thus heavily abstracted from the details of the visual recognition problem, most obviously ignoring the particular selectivites and invariances that parameterize the detectors contained in the representation. On the other hand, the analysis makes it possible to understand in a straightforward way how the simultaneous challenges of clutter and invariance conspire to create binding ambiguity and the degree to which compensatory increases in the size of the representation, through conjunctive order boosting, can be used to suppress this ambiguity. These basic trade-offs have operated close to their predicted levels in the text world experiments carried out in this work, in spite of violations of the simplifying assumptions of feature independence and equiprobability. Since the application of our analysis to any particular recognition problem involves estimating the domain-speciŽc quantities c, d, w, and N, we do not expect predictions relating to levels of binding ambiguity or recognition performance to carry over directly from the problem of word recognition in blocks of symbolic text to other visual recognition domains, such as viewpoint-independent recognition of real three-dimensional objects embedded in cluttered scenes. On the other hand, we expect the basic representational trade-offs to persist. The most important way that recognition in more realistic visual domains differs from recognition in text world lies in the invariance load, which in most interesting cases extends well beyond simple translation invariance. Basic-level object naming in humans, for example, is largely invariant to

Minimizing Binding Errors

275

translation, rotation, scale, and various forms of distortion, occlusion, and degradation—invariances that persist even for certain classes of nonsense objects (Biederman, 1995). Following the logic of our probabilistic analysis, the daunting array of invariances that must be maintained in this and many other natural visual tasks would seem to present a huge challenge to biological visual systems. In keeping with this, performance limitations in human perception suggest that our visual systems have responded in predictable ways to the pressures of the various tasks with which they are confronted; in particular, recognition in a variety of visual domains appears to include only those invariances that are absolutely necessary and those for which simple compensatory strategies are not available. For example (1) recognition is restricted to a highly localized foveal acceptance window for text or other detailed Žgures, where by “giving up” on translation invariance, this substream of the visual system frees hardware resources for the several other essential invariances that cannot be so easily compensated for (e.g. size, font, kerning, orientation, distortion), (2) face recognition operates under strong restrictions on the image-plane oriention and direction of lighting of faces (Yin, 1969; Johnston, Hill, & Carman, 1992), a reasonable compromise given that faces almost always present themselves to our visual systems upright and with positive contrast and lighting from above, and (3) unlike the case for common objects, reliable discrimination of complex three-dimensional nonsense objects (e.g., bent paper clips, crumpled pieces of paper) is restricted to a very limited range of three-dimensional orientations in the vicinity of familiar object views (Biederman & Gerhardstein, 1995), though this deŽciency can be overcome in some cases by extensive training (Logothetis & Pauls, 1995). When viewed within our framework, the exceptional difŽculty of this latter task arises from the need for, or more precisely a lack of, the very large number of very-high-order conjunctive features— essentially full object views locked to speciŽc orientations (see Figure 1B)— that are necessary to support reliable viewpoint-invariant discrimination among a large set of objects with highly overlapping part structures. In summary, the primate visual system manifests a variety of domain-speciŽc design compromises and performance limitations, which can be interpreted as attempts to achieve simultaneously the needed perceptual invariances, acceptably low levels of binding ambiguity, and reasonable hardware costs. In considering the astounding overall performance of the human visual system in comparison to the technical state of the art, it is also worth noting that human recognition performance is based on neural representations that could contain tens of thousands of task-appropriate visual features (extrapolated from the monkey; see Tanaka, 1996) spanning a wide range of conjunctive orders and Žnely tuned invariances. In continuing work, we are exploring further implications of the tradeoffs discussed here in relation to the neurobiological underpinnings of spatially invariant conjunctive receptive Želds in visual cortex (Mel, Ruderman & Archie, 1998) and on the development of more efŽcient supervised

276

Bartlett W. Mel and J´ozsef Fiser

and unsupervised hierarchical learning procedures needed to build highperformance, task-speciŽc visual recognition machines. Acknowledgments

Thanks to Gary Holt for many helpful comments on this work. This work was funded by the OfŽce of Naval Research. References Barron, R., Mucciardi, A., Cook, F., Craig, J., & Barron, A. (1984). Adaptive learning networks: Development and Applications in the United States of algorithms related to GMDH. In S. Farrow (Ed.), Self-organizing methods in modeling. New York: Marcel Dekker. Biederman, I. (1995). Visual object recognition. In S. Kosslyn & D. Osherson (Eds.), An invitation to cognitive science (2nd ed.) (pp. 121–165). Cambridge, MA: MIT Press. Biederman, I., & Gerhardstein, P. (1995). Viewpoint-dependent mechanisms in visual object recognition: Reply to Tarr and Bulthoff ¨ (1995). J. Exp. Psychol. (Human Perception and Performance), 21, 1506–1514. Califano, A., & Mohan, R. (1994). Multidimensional indexing for recognizing visual shapes. IEEE Trans. on PAMI, 16, 373–392. Charniak, E. (1993). Statistical language learning. Cambridge, MA: MIT Press. Douglas, R., & Martin, K. (1998). Neocortex. In G. Shepherd (Ed.), The synaptic organization of the brain (pp. 567–509). Oxford: Oxford University Press. Edelman, S., & Duvdevani-Bar, S. (1997). A model of visual recognition and categorization. Phil. Trans. R Soc. Lond. [Biol], 352, 1191–1202. Fahlman, S., & Lebiere, C. (1990). The cascade-correlation learning architecture. In D. Touretzky (Ed.), Advances in neural information processing systems, 2 (pp. 524–532). San Mateo, CA: Morgan Kaufmann. Fize, D., Boulanouar, K., Ranjeva, J., Fabre-Thorpe, M., & Thorpe, S. (1998). Brain activity during rapid scene categorization—a study using event-related FMRI. J. Cog. Neurosci., suppl S, 72–72. Fukushima, K., Miyake, S., & Ito, T. (1983). Neocognition: A neural network model for a mechanism of visual pattern recognition. IEEE Trans. Sys. Man & Cybernetics, SMC-13, 826–834. Gilbert, C. (1983). Microcircuitry of the visual cortex. Ann. Rev. Neurosci, 89, 8366–8370. Heller, J., Hertz, J., Kjær, T., & Richmond, B. (1995). Information ow and temporal coding in primate pattern vision. J. Comput. Neurosci., 2, 175–193. Hubel, D., & Wiesel, T. (1968). Receptive Želds and functional architecture of monkey striate cortex. J. Physiol., 195, 215–243. Hummel, J., & Biederman, I. (1992). Dynamic binding in a neural network for shape recognition. Psych. Rev., 99, 480–517. Johnston, A., Hill, H., & Carman, N. (1992).Recognizing faces: Effects of lighting direction, inversion, and brightness reversal. Perception, 21, 365–375.

Minimizing Binding Errors

277

Jones, E. (1981). Anatomy of cerebral cortex: Columnar input-output relations. In F. Schmitt, F. Worden, G. Adelman, & S. Dennis (Eds.), The organization of cerebral cortex. Cambridge, MA: MIT Press. Kuù cera, H., & Francis, W. (1967). Computational analysis of present-day American English. Providence, RI: Brown University Press. Lades, M., Vorbruggen, J., Buhmann, J., Lange, J., Malsburg, C., Wurtz, R., & Komen, W. (1993). Distortion invariant object recognition in the dynamic link architecture. IEEE Trans. Computers, 42, 300–311. Lang, G. K., & Seitz, P. (1997). Robust classiŽcation of arbitrary object classes based on hierarchical spatial feature-matching. Machine Vision and Applications, 10, 123–135. Le Cun, Y., Matan, O., Boser, B., Denker, J., Henderson, D., Howard, R., Hubbard, W., Jackel, L., & Baird, H. (1990). Handwritten zip code recognition with multilayer networks. In Proc. of the 10th Int. Conf. on Patt. Rec. Los Alamitos, CA: IEEE Computer Science Press. Logothetis, N., & Pauls, J. (1995). Psychophysical and physiological evidence for viewer-centered object representations in the primate. Cerebral Cortex, 3, 270–288. Logothetis, N., & Sheinberg, D. (1996). Visual object recognition. Ann. Rev. Neurosci., 19, 577–621. Malsburg, C. (1994).The correlation theory of brain function (reprint from 1981). In E. Domany, J. van Hemmen, & K. Schulten (Eds.), Models of neural networks II (pp. 95–119). Berlin: Springer. McClelland, J., & Rumelhart, D. (1981). An interactive activation model of context effects in letter perception. Psych. Rev., 88, 375–407. Mel, B. W. (1997). SEEMORE: Combining color, shape, and texture histogramming in a neurally-inspired approach to visual object recognition. Neural Computation, 9, 777–804. Mel, B. W., Ruderman, D. L., & Archie, K. A. (1998). Translation-invariant orientation tuning in visual “complex” cells could derive from intradendritic computations. J. Neurosci., 17, 4325–4334. Mozer, M. (1991). The perception of multiple objects. Cambridge, MA: MIT Press. Oram, M., & Perrett, D. (1992). Time course of neural responses discriminating different views of the face and head. J. Neurophysiol., 68(1), 70–84. Oram, M. W., & Perrett, D. I. (1994). Modeling visual recognition from neurobiological constraints. Neural Networks, 7(6/7), 945–972. Pitts, W., & McCullough, W. (1947). How we know universals: The perception of auditory and visual forms. Bull. Math. Biophys., 9, 127–147. Potter, M. (1976). Short-term conceptual memory for pictures. J. Exp. Psychol.: Human Learning and Memory, 2, 509–522. Sandon, P., & Urh, L. (1988). An adaptive model for viewpoint-invariant object recognition. In Proc. of the 10th Ann. Conf. of the Cog. Sci. Soc. (pp. 209–215). Hillsdale, NJ: Erlbaum. Schiele, B., & Crowley, J. (1996). Probabilistic object recognition using multidimensional receptive Želd histograms. In Proc. of the 13th Int. Conf. on Patt. Rec. (Vol. 2 pp. 50–54). Los Alamitos, CA: IEEE Computer Society Press. Swain, M., & Ballard, D. (1991). Color indexing. Int. J. Computer Vision, 7, 11–32.

278

Bartlett W. Mel and J´ozsef Fiser

Szentagothai, J. (1977). The neuron network of the cerebral cortex: A functional interpretation. Proc. R. Soc. Lond. B, 201, 219–248. Tanaka, K. (1996). Inferotemporal cortex and object vision. Ann. Rev. Neurosci, 19, 109–139. Thorpe, S., Fize, D., & Marlot, C. (1996).Speed of processing in the human visual system. Nature, 381, 520–522. Van Essen, D. (1985). Functional organization of primate visual cortex. In A. Peters & E. Jones (Eds.), Cerebral cortex (pp. 259–329). New York: Plenum Publishing. Wallis, G., & Rolls, E. T. (1997).Invariant face and object recognition in the visual system. Prog. Neurobiol., 51, 167–194. Weng, J., Ahuja, N., & Huang, T. S. (1997). Learning recognition and segmentation using the cresceptron. Int. J. Comp. Vis., 25(2), 109–143. Wickelgren, W. (1969). Context-sensitive coding, associative memory, and serial order in (speech) behavior. Psych. Rev., 76, 1–15. Yin, R. (1969). Looking at upside down faces. J. Exp. Psychol., 81, 141–145. Zemel, R., Mozer, M., & Hinton, G. (1990). TRAFFIC: Recognizing objects using hierarchical reference frame transformations. In D. Touretzky (Ed.), Advances in neural information processing systems, 2 (pp. 266–273). San Mateo, CA: Morgan Kaufmann. Received September 29, 1998; accepted March 5, 1999.

NOTE

Communicated by Terence Sanger

Relationship Between Phase and Energy Methods for Disparity Computation Ning Qian Center for Neurobiology and Behavior, Columbia University, New York, NY 10032, U.S.A. Samuel Mikaelian Center for Neural Science, New York University, New York, NY 10003, U.S.A. The phase and energy methods for computing binocular disparity maps from stereograms are motivated differently, have different physiological relevances, and involve different computational steps. Nevertheless, we demonstrate that at the Žnal stages where disparity values are made explicit, the simplest versions of the two methods are exactly equivalent. The equivalence also holds when the quadrature-pair construction in the energy method is replaced with a more physiologically plausible phase-averaging step. The equivalence fails, however, when the phase-difference receptive Želd model is replaced by the position-shift model. Additionally, intermediate results from the two methods are always quite distinct. In particular, the energy method generates a distributed disparity representation similar to that found in the visual cortex, while the phase method does not. Finally, more elaborate versions of the two methods are in general not equivalent. We also briey compare these two methods with some other stereo models in the literature. 1 Introduction

Binocular disparity is deŽned as the positional difference between the two retinal projections of a given point in space. It is well known that the horizontal component of disparity provides the sensory cue for stereoscopic depth perception. Many computational models for disparity estimation have been proposed. Here we compare two computational models that appear to have the strongest biological relevance: the phase method and the energy method. The phase method (Sanger, 1988; Fleet, Jepson and Jenkin, 1991) for disparity computation is based on the mathematical result that the displacement of a function generates a proportional phase shift in its complex Fourier transform. The binocular disparity at each location is therefore proportional to the difference of the Fourier phases of the corresponding left and right Neural Computation 12, 279–292 (2000)

c 2000 Massachusetts Institute of Technology °

280

Ning Qian and Samuel Mikaelian

image patches. Sanger (1988) used sine and cosine Gabor Žlters to estimate the local Fourier phases of both left and right images, and then calculated the Fourier phase difference at each location to Žnd disparity. The algorithm has been tested on both random dot stereograms and natural images (Sanger, 1988). The energy method for disparity computation (Qian, 1994, 1997; Zhu & Qian, 1996; Qian & Zhu, 1997; Qian & Andersen, 1997; Fleet, Wagner, & Heeger, 1996) was derived from the energy model of binocular cell responses in the cat’s primary visual cortex (Ohzawa, DeAngelis, & Freeman, 1990, 1996, 1997; Freeman & Ohzawa, 1990; DeAngelis, Ohzawa, & Freeman, 1991). Through quantitative physiological experiments, Freeman and coworkers found that a typical binocular simple cell can be described by two Gabor functions—one for its left and the other for its right receptive Želds. There is a relative phase difference (or positional shift, or both) between the two receptive Želds that generates disparity sensitivity. The activities of such simple cells were determined by Žrst convolving the left and right images with the left and right receptive Želds, respectively, and then summing the two contributions. Freeman et al. further found that responses of binocular complex cells in cat’s primary visual cortex can be well modeled by summing the squared outputs of a quadrature pair of simple cells. We analyzed this so-called energy model for binocular complex cells and found that a population of such complex cells can be used to extract stimulus disparity (see Qian, 1997, for a review). The resulting energy method for disparity computation has been tested on random dot stereograms (Qian, 1994; Qian & Zhu, 1997). These two methods are different in several ways. First, although both methods use Gabor Žlters at the front end, the phase method applies two monocular Gabor Žlters separately and delays binocular comparison until the last step, when the Fourier phases of the left and right image patches are subtracted, while the energy method uses a set of binocular Žlters that combines contributions from the two eyes at the Žrst step. In addition, the phase method explicitly computes and represents the complex Fourier phases of the left and right image patches, while the energy method uses the quadrature-pair construction (or the equivalent phase averaging; see section 2) at the complex cell stage to remove the simple cells’ dependence on image Fourier phases. Furthermore, complex cell responses in the energy method form a distributed representation of disparity; such a representation is absent in the phase method. Despite these differences, we show that the simplest versions of the two methods are exactly equivalent at the Žnal steps where disparity values are made explicit. The intermediate steps, however, differ in the two methods and have different physiological implications. In addition, the two methods can be elaborated in different ways, resulting in nonequivalent algorithms.

Phase and Energy Methods for Disparity Computation

281

2 Results

Before we start, a few potential confusions in terminology should be clariŽed. First, the phase method should not be confused with the phasedifference (also called phase-parameter or phase-shift) receptive Želd model. The former speciŽes a method for disparity estimation from stereograms, while the latter is a model for describing receptive Želd proŽles of binocular simple cells often used in the energy method. Second, the phase parameters in the phase-difference receptive Želd model (w l and w r in equations 2.1 and 2.2) should not be confused with the image Fourier phases. The former determine the positions of the excitatory-inhibitory bands relative to the receptive Želd center, while the latter are the phases of the complex Fourier transforms of retinal images. Finally, mathematically complex quantities, with real and imaginary components, should not be confused with complex cell properties. We now demonstrate the exact equivalence of the simplest versions of the phase and the energy methods. Elaborations of the methods such as frequency pooling (for both methods), conŽdence measure (for the phase method), and spatial pooling (for the energy method) are ignored in this section (but see section 3). We start by reformulating the energy method (Qian, 1994) and convert it into a form identical to that for the phase method (Sanger, 1988). Consider a binocular simple cell centered at x = 0, with the left and right receptive Želds given by the following Gabor functions (Marcelja, 1980; Daugman, 1985; McLean & Palmer, 1989; Ohzawa et al., 1990): fl ( x) = g( x) cos(vx C w l ) fr ( x) = g( x) cos(vx C w r ) with g( x) = p

1 2p s

¡

exp ¡

x2 2s 2

(2.1) (2.2)

¢

,

(2.3)

where v is the preferred (angular) horizontal spatial frequency of the cell, s is the gaussian width determining the receptive Želd size, and w l and w r are the left and right phase parameters. Its response to a stereo image pair Il ( x) and Ir ( x) is given by: Z rs =

1 ¡1

£

¤ fl ( x) Il ( x) C fr ( x) Ir ( x) dx.

(2.4)

(For a layer of topographically arranged cells of the same type, the convolution operation between images and receptive Želds should be used instead.) The simple cell with a quadrature phase relationship to the above cell has

282

Ning Qian and Samuel Mikaelian

receptive Želds of the same form but with the cosine functions replaced by sines (Pollen, 1981; Adelson & Bergen, 1985; Watson & Ahumada, 1985; Ohzawa et al., 1990; Qian, 1994). It is easy to see that the responses of these two simple cells are the real and imaginary parts of Z 1 £ ¤ (2.5) RC ´ g( x)eivx eiw l Il ( x) C eiw r Ir (x) dx. ¡1

According to the quadrature-pair construction, the complex cell response is given by the sum of the squared responses of these two simple cells, (or equivalently, four half-wave rectiŽed simple cells (Ohzawa et al., 1990)), and can thus be written as ­ ­ Z 1 2 ­ £ ¤ ­ 2 ivx iw ¡ ­ (2.6) rq = |R C| = ­ g(x) e Il ( x) C e Ir ( x) dx­ ­, ¡1

where (2.7)

w ¡ ´ wr ¡ w l

is the phase parameter difference. Therefore, complex cell responses depend on only w ¡ instead of on w r and w l individually. For a given binocular stimulus, complex cells with different w ¡ will give different responses. These cells at a given spatial location form a distributed representation of the stimulus disparity at that location. According to the energy method (Qian, 1994), the stimulus disparity can be explicitly estimated from the distribution as: D=

wO ¡ , v

(2.8)

where wO ¡ is the phase difference w ¡ that maximizes the complex cell response rq . Let Z

1

¡1 Z 1 ¡1

g(x) Il ( x) eivx dx ´ Âl (v),

(2.9)

g( x) Ir ( x) eivx dx ´ Âr (v),

(2.10)

so that Âl (v) and Âr (v) are the Fourier transforms of the left and right image patches under the gaussian envelope g( x), respectively. Equation 2.6 then becomes:

­

­

2 rq = ­ Âl (v) C eiw ¡ Âr (v) ­.

(2.11)

To Žnd wO ¡ , note that the maximum of rq is attained when the two terms inside the norm of the above expression are along the same direction in the

Phase and Energy Methods for Disparity Computation

283

complex plane, that is, wO¡ = arg(Âl ) ¡ arg(Âr ) "R1 # ( ) ( ) ¡1 dx Il x g x sin vx R = arctan 1 ( ) ( ) ¡1 dx Il x g x cos vx "R1 # ( ) ( ) ¡1 dx I r x g x sin vx ¡ arctan R 1 . ( ) ( ) ¡1 dx Ir x g x cos vx

(2.12)

(2.13)

Equation 2.13 combined with equation 2.8 is precisely the expression for the phase method proposed by Sanger (1988). The two terms in equation 2.13 are the local complex Fourier phases of the left and right image patches, respectively, estimated through the sine and cosine Gabor Žlters. This establishes the equivalence between the two methods. It is interesting to note that w ¡ simply compensates for the different Fourier phases of the right and left image patches in equation 2.11. Scaling the relative contrast of the image pair while preserving the contrast polarity will not affect the disparity estimates because only the relative amplitudes (but not phases) of Âl and Âr will be changed by this manipulation (Qian, 1994). When the contrast polarity of one image in the pair is reversed, however, w ¡ has to be shifted by p to compensate, resulting in an inverted complex cell turning curve. This prediction is also contained in equation 2.13 of Qian (1994) and has been veriŽed experimentally (Ohzawa et al., 1990; Cumming & Parker, 1997; Masson, Busettini, & Miles, 1997). We have previously discussed the similarities and differences between the energy method and the cross-correlator (Qian & Zhu, 1997). With the current formulation, it is easy to see that Žnding wO¡ , which maximizes the quadrature-pair response, is equivalent to determining the Fourier phase of the cross-correlation between the left and right image patches under the gaussian envelope g( x). This is because the Fourier transform of the crosscorrelation, Z Clr ( x) =

1 ¡1

g( x0 )Il (x0 ) g( x C x0 )Ir ( x C x0 ) dx0 ,

(2.14)

is simply Âl (v) Âr¤ (v) where ¤ denotes complex conjugation. Therefore, the phase of this cross-correlator, or its normalized version, is just the expression for wO¡ in equation 2.12. Correlation methods usually use the peak location of Clr ( x) to estimate disparity. If Clr ( x) is sharply peaked at xo , its Fourier phase is approximately vxo . It is in deviations from this approximation that the cross-correlator and the energy method can yield different estimates. We used the quadrature-pair construction for obtaining complex cell responses above. As we pointed out previously (Qian & Zhu, 1997), a less demanding and physiologically more plausible alternative is to integrate

284

Ning Qian and Samuel Mikaelian

squared responses of many simple cells all with the same w ¡ but with (2.15)

w C ´ wr C w l

uniformly distributed in the entire 2p range (see equation 2.19). The energy method with this uniform phase averaging approach is still equivalent to the phase method because the phase averaging gives exactly the same complex cell expression as the quadrature-pair construction. To demonstrate, deŽne: Z 1 (2.16) R¡ ´ R¤C = g( x) e¡ivx [e¡iw l Il ( x) C e¡iw r Ir ( x)]dx, ¡1

and rewrite the simple cell response as: rs =

1 ( R C C R¡ ). 2

(2.17)

Since rs is real, we have: r2s = |rs | 2 =

´ 1³ ´ 1³ |R C| 2 C |R¡ | 2 C 2ReR CR¤¡ = | R C| 2 C ReR2C . 4 2

(2.18)

According to the phase averaging approach, the complex cell response is given by the integration: 1 ra = 2p

Z

2p 0

r2s dw C

1 = 2p

Z

2p 0

´ 1³ | R C| 2 C ReR2C dw C, 2

(2.19)

while w ¡ is kept constant. Since |R C| 2 is not a function of w C (see equation 2.6), the averaging leaves the Žrst term unchanged. The second term integrates to zero because: " Z # h i 2 1 2 ivx ¡iw ¡ / 2 ( ) iw ¡ / 2 ( ) iw C ( ) ReR C = Re (2.20) dxg x e e Il x C e Ir x e

¡

¢

¡1

and Z

2p 0

eiw C dw C = 0.

(2.21)

Therefore, the phase averaging approach generates a complex cell response proportional to that of the quadrature pair in equation 2.6, ra =

1 1 |R C| 2 = rq , 2 2

(2.22)

where the proportionality constant is immaterial for disparity computation. Intuitively, one can imagine dividing the 2p range for w C into many small

Phase and Energy Methods for Disparity Computation

285

intervals in equation 2.19. Then each pair of intervals differing by p / 2 can be considered a quadrature pair. Therefore, the above phase averaging can be viewed as averaging together a continuum of quadrature-pair responses, all with the same w ¡ . Two types of receptive Želd models for binocular simple cells have been proposed: the position-shift model (Bishop, Henry, & Smith, 1971) and the phase-differences model (Ohzawa et al., 1990; DeAngelis et al., 1991). The latter was used in the above derivations. Real cortical cells may use a combination of both mechanisms for coding disparity (Jacobson, Gaska, & Pollen, 1993; Zhu & Qian, 1996; Fleet et al., 1996; Anzai, Ohzawa, & Freeman, 1997). We showed previously that although there are important differences between them, under reasonable assumptions, both models (or their hybrid) can be used with the energy method for computing disparity maps from stereograms (Zhu & Qian, 1996; Qian & Zhu, 1997). However, the exact equivalence between the phase and the energy methods does not hold when the position-shift receptive Želd model is used in the energy method. To see this, note that according to the position-shift receptive Želd model, equation 2.2 should be replaced by fr ( x) = fl (x C d) = g(x C d) cos[v(x C d) C w l ],

(2.23)

where d represents the amount of positional shift between the left and right receptive Želd centers. The complex cell responses in equation 2.6 should then be written as: ­ Z 1 2 h i ­ ­ ­ ivx ivd ­ (2.24) rq = ­ e g(x) Il ( x) C e g( x C d) Ir ( x) dx­ ­, ¡1

and the stimulus disparity is given by (Zhu & Qian, 1996): O D = d,

(2.25)

where dO is the positional shift that maximizes complex cell response. Equation 2.24 can be rewritten as

­ ­

­ 2 ­

rq = ­ Âl C eivd Âr ( d) ­, with the deŽnitions: Z 1 g( x) Il ( x)eivx dx ´ Âl , ¡1 Z 1 g(x C d) Ir ( x)eivx dx ´ Âr (d) . ¡1

(2.26)

(2.27) (2.28)

It is obvious that a closed-form expression similar to equation 2.13 cannot be obtained for dO because Âr is also a function of d, and the maximum of

286

Ning Qian and Samuel Mikaelian

rq as a function of d may not occur when the two terms inside the norm of equation 2.26 have the same direction on the complex plane. Therefore, the energy method with the position-shift receptive Želd model is not equivalent to the phase method. However, under the oft-invoked assumption that the disparity is small compared to the receptive Želd size, the d-dependence of Âr in equation 2.28 is negligible and vd in equation 2.26 plays the same role as w ¡ in equation 2.11. Clearly, in this approximation the energy method with the positional-shift model corresponds to the phase method in the same way as the energy method with the phase-difference model does. 3 Discussion

We have demonstrated that the simplest versions of the energy and the phase methods for disparity computation are exactly equivalent at the Žnal stages where the disparity values are made explicit. The equivalence still holds when the quadrature-pair construction in the energy method is replaced by a less demanding phase-averaging procedure. However, when the position-shift type of simple cell receptive Želd model is used to replace the phase-difference type in the energy method, the two methods are no longer exactly equivalent. This result is consistent with the fact that both the phase method and the energy method combined with the phase-difference receptive Želd model predict a relationship between the spatial scale v and the computed disparity range (the so-called size-disparity correlation), while the methods with the position-shift receptive Želd model do not make such a prediction (Sanger, 1988; Smallman & MacLeod, 1994; Qian, 1994; Zhu & Qian, 1996; Fleet et al., 1996). Regardless of the receptive Želd models used, the intermediate results in the two methods are always very different. Indeed, the energy method contains simple and complex cell stages that simulate binocular interactions observed in simple and complex cortical cells, while none of the steps in the phase method resembles actual binocular cells. More importantly, the complex cells in the energy method form a distributed representation of binocular disparity at each location; such a representation is absent in the phase method. The distributed representation is useful because it could directly guide motor behavior such as vergence eye movement (Masson et al., 1997) without Žrst generating an explicit disparity map. In fact, the Žnal explicit disparity extraction in both methods does not seem to happen in the brain since “grandmother cells” for disparity coding have never been recorded from the visual cortex. While an explicit, grandmother-cell type of representation is more convenient for us to comprehend, the brain appears to rely more on implicit, distributed representations for perception and control. The simplest versions of the phase and the energy methods can also be elaborated in many ways. For example, both methods can be extended to combine results from different spatial scales (i.e., Žlters with different vs).

Phase and Energy Methods for Disparity Computation

287

Obviously, if the same extension is made after the Žnal stages in both methods, they remain equivalent. Any elaborations at intermediate steps or those that can only be applied to one method, however, will in general render the two methods nonequivalent. One example is the spatial pooling procedure in the energy method, which greatly improves the algorithm by making complex cell responses more independent of image Fourier phases (Zhu & Qian, 1996; Qian & Zhu, 1997). This procedure cannot be easily incorporated into the phase method because image Fourier phases are precisely the useful information in the phase method. Similarly, the conŽdence measure in the phase method for appropriately weighting different spatial scales (Sanger, 1988) cannot be readily extended to the energy method since the measure is derived from the Fourier amplitudes of the left and right image patches; these monocular amplitudes are not computed in the energy method that starts with binocular Žlters. Of course, a different spatial pooling step could be incorporated into the phase method. For example, one could average the Fourier phases estimated from several nearby spatial locations. Similarly, a different conŽdence measure could be added to the energy method. One possibility would be using the normalized range of the complex cell responses, O 2 Re(e¡iw ¡ Âl Âr¤ ) rq (wO ¡ ) ¡ rq (wO ¡ C p ) 2|Âi ||ÂrO | = = , 2 C | |2 O O | | | | 2 C | Âr | 2  ( ) ( ) rq w ¡ C rq w ¡ C p r l l

(3.1)

as the conŽdence measure because a larger modulation would allow a more reliable estimation of wO ¡ . In conclusion, the equivalence between the simplest versions of the phase and energy methods demonstrated in this article indicates that the two methods can be viewed as different implementations of the same underlying mathematical principle. The differences in implementation, however, allow the methods to be elaborated, interpreted, and used in different ways by subsequent processes. Our results are reminiscent of the relationship between various motion models that have been documented in the literature (Adelson & Bergen, 1985; van Santen & Sperling, 1985; Borst & Egelhaaf, 1989; Simoncelli & Adelson, 1991; Heeger & Simoncelli, 1992). 3.1 Other Methods. One of the reviewers pointed out that disparity can also be computed by calculating the magnitude of the bandpass-Žltered sum of the left and right image patches. This magnitude is equivalent to setting w ¡ to zero in equation 2.6 (or setting d to zero in equation 2.24) and dropping the immaterial squaring outside the norm. For a Žlter well-tuned to horizontal frequency v and for left and right image patches given by Il = I ( x) and Ir = I ( x CD), this magnitude is approximately proportional to | cos(vD/ 2)|| IQ(v)|. If the image spectrum | IQ(v)| does not vary much with v, the disparity D can be estimated by using two or more Žlters with different

288

Ning Qian and Samuel Mikaelian

v. We will refer to this algorithm as the frequency method because it applies identical Žlters to the left and right images and consequently has to rely on more than one frequency band for disparity computation. In essence, the frequency method Žxes w ¡ (or d) to zero and varies v, while the energy method Žxes v and varies w ¡ (or d). Obviously the frequency method is not equivalent to the energy (or the phase) method because the former cannot compute disparity within a single frequency band, while the latter can. Existing psychophysical evidence is against the frequency method in its pure form: human subjects do perceive stereoscopic depth in band-limited stereograms (Julesz, 1971). Psychophysical observations also indicate that different frequency channels interact with each other in human stereo vision (Wilson, Blake, & Halpern, 1991; Rohaly & Wilson, 1993, 1994; Smallman, 1995; Mallot, Gillner, & Arndt, 1996). It is thus possible that the brain may combine the frequency and the energy methods by using cells with different v and different w ¡ (or d) simultanously in disparity computation. Alternatively, the brain could Žrst apply the energy method to obtain one disparity estimate from each frequency channel and then pool the estimates across channels at a later stage (Sanger, 1988; Qian & Zhu, 1997). Further studies are needed for differentiating these two possibilities. In addition to the phase and the energy methods, many other stereo models have been proposed over the years. Similar to the phase method that Žrst estimates Fourier phases of the left and right image patches separately, most of the other models also start with a monocular detection of some matching primitives. For example, several studies (Marr & Poggio, 1976; Pollard, Mayhew, & Frisby, 1985; Prazdny, 1985; Qian & Sejnowski, 1989; Marshall, Kalarickal, & Graves, 1996) let individual dots in random-dot stereograms be the matching primitives. Marr and Poggio (1979) later used zero crossings at different spatial scales as the primitives. For convenience, we classify these methods as initially monocular. In contrast, the energy method is initially binocular because it starts with binocular Žltering without performing any monocular feature detection. In this aspect it is similar to the cepstral method (Yeshurun & Schwartz, 1989) and the frequency method discussed above. The cepstral and the energy methods are very different in other aspects, however. For instance, the former contains a complex logrithmic operation and complex Fourier transforms, while the latter does not. (Note that the Fourier transform used in this article and elsewhere in connection with the energy method is only for mathematically analyzing the method; it is not a step in the method.) In addition, the cepstral method does not use binocular receptive Želds from the physiological experiments. The initially monocular algorithms generally contain a subsequent step that matches the monocular primitives to solve the correspondence problem. Most methods (Marr & Poggio, 1976, 1979; Pollard et al., 1985; Prazdny, 1985; Qian & Sejnowski, 1989; Marshall et al., 1996) do so by introducing explicit constraints or rules that determine correct matches between the left and right primitives among all possible matches within a certain disparity

Phase and Energy Methods for Disparity Computation

289

range. These methods will be referred to as the explicit methods. In contrast, the phase method is implicit (Sanger, 1988) because it Žnds disparity by simply subtracting the two monocular phases without performing any explicit matching. (If the monocular Fourier amplitudes are used to calculate a conŽdence weighting factor [Sanger, 1988], then the matching is slightly more explicit). The initially binocular algorithms deŽned above are necessarily implicit since there are no monocular primitives for explicit matching in the Žrst place. An advantage of the implicit methods is that they can have subpixel disparity resolution without the burden of sorting through a huge number of potential matches (Sanger, 1988; Qian & Zhu, 1997). As we discussed elsewhere (Qian, 1997), the implicit methods are also more physiologically plausible. To summarize, most stereo models are initially monocular for feature extraction and are then explicit in binocular matching. The cepstral, the energy, and the frequency methods, on the other hand, are initially binocular and implicit. The phase method is somewhere in between: it is initially monocular and is implicit. Obviously, there are many other ways of classifying stereo models. For example, some models measure and rely on disparity gradients, while others do not; some implement the uniqueness constraint through inhibitory interactions, while others emphasize facilitation and allow multiple matches of monocular primitives. The current formulation of the energy method assumes that the disparity is constant over the small image patches covered by the receptive Želds (Qian, 1994; Qian & Zhu, 1997). Because of this assumption, the performance tends to degrade when there are large disparity gradients in the images such as across disparity boundaries. However, the energy method can be extended to encompass disparity gradients. SpeciŽcally, it can be shown that a disparity gradient of D0 (= dD( x)/ dx) can be best measured by binocular cells whose left and right preferred spatial frequencies (vl and vr ) are related by vr = (1 C D0 )vl . Therefore, the maximum measurable disparity gradient depends on the differences between vl and vr among real binocular cells. If vl and vr are equal, then the cells’ frequency tuning widths have to be at least (1 C D0 )vl in order to detect D0 (Sanger, 1988). This extension of the energy method can be viewed as an implicit counterpart of the gradient limit rule employed explicitly by Pollard et al. (1985). It is also worth pointing out that the energy method in its current form (Qian, 1994; Qian & Zhu, 1997) already contains an implicit implementation of the uniqueness constraint (used by many explicit models in various forms). Because of the approximately cosine tuning behavior, there is only one maximum of the complex cell response as a function of w ¡ within the [¡p , p ) range even when there are two overlapping stimulus disparities. The stereo transparency problem could still be solved by an interdigitating representation of the two disparities by cells tuned to different spatial locations, similar to a previous motion-transparency model (Qian, Andersen, & Adelson, 1994). This approach is also related to those explicit methods

290

Ning Qian and Samuel Mikaelian

that use the uniqueness constraint to model stereo transparency (Qian & Sejnowski, 1989; Pollard & Frisby, 1990). It thus appears that despite the major differences between the explicit and implicit methods, they could share certain conceptual similarities. Acknowledgments

This work was supported by a Sloan Research Fellowship and NIH grant MH54125, both to N. Q. We are grateful to the anonymous reviewers for their helpful comments. References Adelson, E. H., & Bergen, J. R. (1985). Spatiotemporal energy models for the perception of motion. J. Opt. Soc. Am. A, 2(2), 284–299. Anzai, A., Ohzawa, I., & Freeman, R. D. (1997). Neural mechanisms underlying binocular fusion and stereopsis: Position vs. phase. Proc. Nat. Acad. Sci. USA, 94, 5438–5443. Bishop, P. O., Henry, G. H., & Smith, C. J. (1971). Binocular interaction Želds of single units in the cat striate cortex. J. Physiol., 216, 39–68. Borst, A., & Egelhaaf, M. (1989). Principles of visual motion detection. Trends Neurosci., 12, 297–306. Cumming, B. G., & Parker, A. J. (1997). Responses of primary visual cortical neurons to binocular disparity without depth perception. Nature, 389, 280– 283. Daugman, J. G. (1985). Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical Žlters. J. Opt. Soc. Am. A, 2, 1160–1169. DeAngelis, G. C., Ohzawa, I., & Freeman, R. D. (1991). Depth is encoded in the visual cortex by a specialized receptive Želd structure. Nature, 352, 156–159. Fleet, D. J., Jepson, A. D., & Jenkin, M. (1991). Phase-based disparity measurement. Comp. Vis. Graphics Image Proc., 53, 198–210. Fleet, D. J., Wagner, H., & Heeger, D. J. (1996). Encoding of binocular disparity: Energy models, position shifts and phase shifts. Vision Res., 36, 1839–1858. Freeman, R. D., & Ohzawa, I. (1990). On the neurophysiological organization of binocular vision. Vision Res., 30, 1661–1676. Heeger, D. J., & Simoncelli, E. P. (1992). Model of visual motion sensing. In L. Harris & M. Jenkins (Eds.), Spatial vision in humans and robots. Cambridge: Cambridge University Press. Jacobson, L., Gaska, J. P., & Pollen, D. A. (1993). Phase, displacement and hybrid models for disparity coding. Invest. Ophthalmol. and Vis. Sci. Suppl. (ARVO), 34, 908. Julesz, B. (1971). Foundations of cyclopean perception. Chicago: University of Chicago Press. Mallot, H. A., Gillner, S., & Arndt, P. A. (1996). Is correspondence search in human stereo vision a coarse-to-Žne process? Biol. Cybern., 74, 95–106.

Phase and Energy Methods for Disparity Computation

291

Marcelja, S. (1980). Mathematical description of the responses of simple cortical cells. J. Opt. Soc. Am. A, 70, 1297–1300. Marr, D., & Poggio, T. (1976). Cooperative computation of stereo disparity. Science, 194, 283–287. Marr, D., & Poggio, T. (1979). A computational theory of human stereo vision. Proc. R. Soc. Lond. B, 204, 301–328. Marshall, J. A., Kalarickal, G. J., & Graves, E. B. (1996). Neural model of visual stereomatching: Slant, transparency, and clouds. Network: Comput. Neural Sys., 7, 635–639. Masson, G. S., Busettini, C., & Miles, F. A. (1997). Vergence eye movements in response to binocular disparity without depth perception. Nature, 389, 283– 286. McLean, J., & Palmer, L. A. (1989). Contribution of linear spatiotemporal receptive Želd structure to velocity selectivity of simple cells in area 17 of cat. Vision Res., 29, 675–679. Ohzawa, I., DeAngelis, G. C., & Freeman, R. D. (1990). Stereoscopic depth discrimination in the visual cortex: Neurons ideally suited as disparity detectors. Science, 249, 1037–1041. Ohzawa, I., DeAngelis, G. C., & Freeman, R. D. (1996). Encoding of binocular disparity by simple cells in the cat’s visual cortex. J. Neurophysiol., 75, 1779– 1805. Ohzawa, I., DeAngelis, G. C., & Freeman, R. D. (1997). Encoding of binocular disparity by complex cells in the cat’s visual cortex. J. Neurophysiol., 77, 2879– 2909. Pollard, S. B., & Frisby, J. P. (1990). Transparency and the uniqueness constraint in human and computer stereo vision. Nature, 347, 553–556. Pollard, S. B., Mayhew, J. E., & Frisby, J. P. (1985). PMF: A stereo correspondence algorithm using a disparity gradient limit. Perception, 14, 449–470. Pollen, D. A. (1981). Phase relationship between adjacent simple cells in the visual cortex. Nature, 212, 1409–1411. Prazdny, K. (1985). Detection of binocular disparities. Biol. Cybern., 52, 93–99. Qian, N. (1994). Computing stereo disparity and motion with known binocular cell properties. Neural Comput., 6, 390–404. Qian, N. (1997). Binocular disparity and the perception of depth. Neuron, 18, 359–368. Qian, N., & Andersen, R. A. (1997). A physiological model for motion-stereo integration and a uniŽed explanation of Pulfrich-like phenomena. Vision Res., 37, 1683–1698. Qian, N., Andersen, R. A., & Adelson, E. H. (1994). Transparent motion perception as detection of unbalanced motion signals III: Modeling. J. Neurosci., 14, 7381–7392. Qian, N., & Sejnowski, T. J. (1989). Learning to solve random-dot stereograms of dense and transparent surfaces with recurrent backpropagation. In D. S. Touretzky, G. E. Hinton, & T. J. Sejnowski (Eds.), Proceedings of the 1988 Connectionist Models Summer School (pp. 435–443). San Mateo, CA: Morgan Kaufmann.

292

Ning Qian and Samuel Mikaelian

Qian, N., & Zhu, Y. (1997). Physiological computation of binocular disparity. Vision Res., 37, 1811–1827. Rohaly, A. M., & Wilson, H. R. (1993). Nature of coarse-to-Žne constraints on binocular fusion. J. Opt. Soc. Am. A, 10, 2433–2441. Rohaly, A. M., & Wilson, H. R. (1994). Disparity averaging across spatial scales. Vision Res., 34, 1315–1325. Sanger, T. D. (1988). Stereo disparity computation using Gabor Žlters. Biol. Cybern., 59, 405–418. Simoncelli, E. P., & Adelson, E. H. (1991).Relationship between gradient, spatiotemporal energy, and regression models for motion perception. Invest. Ophthalmol. and Vis. Sci. Suppl. (ARVO), 32, 893. Smallman, H. S. (1995). Fine-to-coarse scale disambiguation in stereopsis. Vision Res., 35, 1047–1060. Smallman, H. S., & MacLeod, D. I. (1994).Size-disparity correlation in stereopsis at contrast threshold. J. Opt. Soc. Am. A, 11, 2169–2183. van Santen, J. P. H., & Sperling, G. (1985). Elaborated Reichardt detectors. J. Opt. Soc. Am. A, 2, 300–321. Watson, A. B., & Ahumada, A. J. (1985).Model of human visual-motion sensing. J. Opt. Soc. Am. A, 2, 322–342. Wilson, H. R., Blake, R., & Halpern, D. L. (1991). Coarse spatial scales constrain the range of binocular fusion on Žne scales. J. Opt. Soc. Am. A, 8, 229–236. Yeshurun, Y., & Schwartz, E. L. (1989). Cepstral Žltering on a columnar image architecture—A fast algorithm for binocular stereo segmentation. IEEE Pat. Anal. Mach. Intell., 11, 759–767. Zhu, Y., & Qian, N. (1996). Binocular receptive Želds, disparity tuning, and characteristic disparity. Neural Comput., 8, 1647–1677. Received September 30, 1998; accepted February 12, 1999.

NOTE

Communicated by Leo Breiman

N-tuple Network, CART, and Bagging Aleksander Ko  lcz Electrical and Computer Engineering Department, University of Colorado at Colorado Springs, Colorado Springs, CO 80918, U.S.A. Similarities between bootstrap aggregation (bagging) and N-tuple sampling are explored to propose a retina-free data-driven version of the Ntuple network, whose close analogies to aggregated regression trees, such as classiŽcation and regression trees (CART), lead to further architectural enhancements. Performance of the proposed algorithms is compared with the traditional versions of the N-tuple and CART networks on a number of regression problems. The architecture signiŽcantly outperforms conventional N-tuple networks while leading to more compact solutions and avoiding certain implementational pitfalls of the latter.

1 Introduction

Recent years have seen a reemergence of interest in the N-tuple network (NTN)architecture (Bledsoe & Browning, 1959). Despite numerous analyses (Bledsoe & Bisson, 1962; Steck, 1962; Roy & Sherman, 1967) and signiŽcant progress (Luttrell, 1992; Rohwer, 1995; Ko lcz & Allinson, 1996; Jung, Krishnamoorthy, Nagy, & Shapira, 1996; Bradshaw, 1997; Rohwer & Morciniec, 1998), the computational capabilities of this architecture are still not very well understood. An NTN combines multiple models randomly partitioning the input space. In this article we show that a single NTN node can be represented by a binary tree, which can beneŽt from learning algorithms similar to those used by classiŽcation and regression trees (CART) (Breiman, Friedman, Olshen, & Stone, 1984). This abstraction, together with analogies between N-tuple sampling and bootstrap aggregation (bagging) (Breiman, 1996a), are used to propose a data-driven method of creating a set of NTN tree-based nodes, resulting in a network related to bagged CART. The article is organized as follows. In section 2 we demonstrate a basic similarity between N-tuple sampling and bagging, and propose a retina-free version of the NTN. Section 3 introduces two tree architectures combining the attributes of CART and NTN. In section 4 numerical experiments are used to demonstrate the advantages of the proposed tree versions of the NTN when compared to the original architecture and to CART on a number of regression problems. Section 5 contains the conclusion. Neural Computation 12, 293–304 (2000)

c 2000 Massachusetts Institute of Technology °

294

Aleksander Ko lcz

2 Retina-Free NTN and Bagging Estimators

Assuming a binary input pattern space, usually represented in the form of a two-dimensional “retina,” NTN performs N-tuple sampling, where a set of K memory nodes, each having an N-bit-long address word, randomly distribute their address lines among the retina bits, with each address line being associated with a particular random feature of the binary input space. The NTN mapping is thus an aggregate of K random low-complexity models. In continuous variable environments, real-valued inputs are converted into a binary format suitable for N-tuple sampling. In particular, when thermometer encoding (Aleksander & Stonham, 1979) is used, an instance of N-tuple sampling becomes equivalent to placing N quantization thresholds (constrained by a uniform quantization grid in each dimension) within the range of the D input variables (Tattersall, Foster, & Johnston, 1991), leading to hyper-rectangular quantization of the input space. The network is biased by the assumption of uniformity imposed on the input space. The alternative we consider is to bypass the input-conversion stage altogether and perform N-tuple sampling by directly distributing the quantization thresholds within the ranges of the input variables, a process that can be guided by the empirical input-space distribution provided by a training set. In the light of this idea, certain similarities between the nature of NTN design and some recent advances in learning algorithms become apparent. Breiman (1996a) showed that signiŽcant improvements in network prediction performance can be gained by using the concept of bagging, where the variance component of the generalization error is reduced by averaging several variants of the network. Each variant is designed with a different version of the training set, with each version created by sampling with repetitions from the original collection. To see the similarity between bagging and NTN design, let us Žrst note that each node of an NTN represents a crude model of the desired mapping, which is highly dependent on the positioning of the N-tuple thresholds. Averaging a large number of such randomly designed nodes reduces the variance of the estimate irrespective of the data set considered. However, when N-tuple sampling is constrained to choose the quantization thresholds from within the set training-point coordinates (according to the sampling with repetition regimen), the process of selecting a set of K NTN nodes becomes data driven and effectively corresponds to bagging. Although such an approach should lead to performance improvements, we propose to structure the process of training individual NTN nodes further by taking advantage of their inherent similarity to binary trees. 3 NTN Analogies to CART

CART is a well-known binary tree network based on hyper-rectangular quantization of the input space. In the regression variant, its output to an

N-tuple Network, CART, and Bagging

295

input x is given by an arithmetic average of the training points falling into the same tree leaf as x. During the tree-growing process, which starts with a root node containing all training points, CART chooses the next node to be partitioned such that the split of the corresponding cell maximizes the decrease of the overall training set error. A split is performed parallel to one of the axes by choosing the threshold point out of the set of coordinate values of training points falling into that node. Node splitting continues until a sufŽciently large tree is created, at which point a pruning process is initiated to optimize the tree and avoid the problem of overŽtting. Although a node of the NTN is created differently from CART, certain commonalities can be seen. Starting with the unpartitioned input space, the placement of the Žrst quantization threshold of an NTN node is equivalent to splitting the space at the root node of CART. The placement of any additional threshold results in splitting all of the current leaf nodes of the tree at that threshold, instead of splitting exactly one node, as is the case in CART. Despite this important difference, we can create an equivalent tree corresponding to a single node of the NTN, considering that the order of threshold-point placement is Žxed (i.e., each N-tuple is ordered). The value of N corresponds to the depth of the equivalent tree. 3.1 Bagged-Tree Implementation of an NTN. In view of the similarities between NTN and CART, a question arises whether the structured approach of tree design could be applied to the NTN in order to enhance the network performance. One obvious problem is that CART constructs essentially one tree, whereas NTN uses K nodes to achieve satisfactory performance levels. However, correspondence between bagging and N-tuple sampling allows substitution of the collection of randomly designed N-tuple nodes for a collection of bagged trees. K bootstrapped CART trees can also be interpreted as nonidentical quantizations of the input space, where each point resides in exactly K quantization cells, as in the NTN. Taking these observations into account, if a structured (i.e., data driven) node design were to be applied to the NTN, bagging could be used to derive the K network nodes. 3.2 Adapting the CART Node-Splitting Criterion. At each step in the CART tree-growing algorithm, all current tree leaves are evaluated for the possibility of a split. The leaf leading to the highest reduction in error is then chosen. The best-split selection process can be expressed as

¡

n, d, t

¢ best

£ ¡ ¡ ¢ ¡ ¢ ¢¤ = arg max E ( n) ¡ E n, d, t l C E n, d, t r , ( n,d,t)

(3.1)

where E (n) represents the error estimate at node n, d represents ¡ an ¢input variable (d = 1, . . . , D), t is a particular threshold value, and E n, d, t l and ¡ ¢ E n, d, t r correspond to the error estimates for points at node n residing on both (i.e., “left” and “right”) sides of the partitioning threshold xd = t.

296

Aleksander Ko lcz

Generally, splitting any node at any point will reduce the error (provided that it results in a division of points at that node), and the procedure chooses the best overall split. In the case of the NTN, the splitting occurs for all current terminal nodes. Consequently, it makes sense to choose the best split on the basis of the aggregated decrease of the training-set error, taking all nodes into account, that is, X£ ¡ ¢ ¡ ¡ ¢ ¡ ¢ ¢¤ (3.2) d, t best = arg max E ( n) ¡ E n, d, t l C E n, d, t r . ( d,t) n Note that at some nodes, the gain might be zero if all points in that node lie on the same side of the threshold. In such cases when the split is carried out, the tree will have some empty terminal nodes, which might be split again as the process continues. To avoid this undesirable effect, splits are allowed only if the node contains more than one element and the split does not result in an empty cell. In this way no empty cells will be added during the training process. 3.3 Pruning.

3.3.1 Cost-Complexity Pruning. Since every split results in an error reduction, the greedy algorithm outlined above can, in principle, continue until each leaf contains just one element. Clearly, such a network provides a good Žt to the training set, but can be expected to have poor generalization properties. To overcome this effect, the CART algorithm utilizes cost-complexity pruning, where the magnitude of the training-set error is weighted against the complexity of the tree measured in terms of the number of its terminal nodes, E = ER C a ¢ e T, where ER is the mean squared error over the training set and e T is the tree size. After a cross-validated estimate of a is obtained, the tree is pruned by merging pairs of terminal nodes, one pair at a time, until the desired tree complexity is reached. The cost-complexity pruning can be applied directly to the tree created by the NTN algorithm. The resulting tree will not correspond to the NTN anymore, since while for some nodes a particular threshold may be implemented, it may be omitted for others, even though carrying out of the split would not result in the creation of an empty cell. A tree grown according to the NTN strategy and pruned according to the cost-complexity approach will be called CART-NTN. An example input-space quantization characteristic of CART-NTN is illustrated in Figure 1 (left). 3.3.2 Optimal-Depth Pruning. The cost-complexity pruning makes the resulting tree analogous to CART, since the tree can have branches of different lengths not necessarily dictated by the lack of data in the terminal nodes. To keep within the NTN framework, however, each pruning step should merge all cells at the current level. A simple pruning method conforming with the NTN methodology is to choose a depth of the tree that

N-tuple Network, CART, and Bagging

297

Figure 1: Input space quantization due to (left) CART-NTN and (right) NTNCART. In CART-NTN, some of the initial quantization thresholds (indicated by heavier lines) will affect the entire input domain, while others will be more local, due to the CART-like pruning procedure. In NTN-CART most of the partitioning thresholds have a global scope, just as in the NTN, but in certain local cases, the splits are not implemented (the shaded regions) due to the lack of training data.

results in the best generalization error, obtained through cross validation. Note that this type of pruning is signiŽcantly simpler than cost-complexity pruning. The order of node removal is exactly the same as the order of node generation since we remove all nodes residing at a current level in each stage. Therefore, while the original tree is grown, a cross-validated estimate of the generalization error is computed and associated with that level each time a split is made. After the original tree is grown, the depth corresponding to the minimum cross-validation error is then chosen as the optimal depth of the Žnal tree. This value corresponds essentially to the value of N in the case of NTN. A tree grown according to the NTN strategy and pruned according to the optimal-depth approach will be called NTN-CART. An example input-space quantization typical of NTN-CART is shown in Figure 1 (right). 4 Numerical Experiments

In view of the extensions proposed, CART-NTN and NTN-CART can be considered intermediate architectures, bridging CART with NTN. Performance of CART, NTN, and the intermediate CART-NTN and NTN-CART networks was evaluated in a regression setting using one real and three simulated data sets, which have been used in Breiman (1996a). One hundred random variants of each data set were considered. For the simulated data, the different variants were generated using the formulas given below; for

298

Aleksander Ko lcz

the real data set, the different samples were obtained by a random split of the whole set into its training and test components. In all experiments with bagged networks 25 bagging repetitions were used. The real-world data set Boston corresponds to the Boston Housing experiment (Belsley, Kuh, & Welsh, 1980), where a mix of 13 social and environmental variables is used to estimate real estate prices. The total data set consists of 506 elements, where in each random case 481 points were used to train a network and the remaining 25 were used for testing. The three artiŽcial data sets are due to Friedman (1991). In each random case, a training set consisted of 200 points, and a test set contained 1000 points. The data set Friedman-1 is generated according to the formula y = 10 ¢ sin (p x1 x2 ) C 20 ¢ ( x3 ¡ 0.5) 2 C 10 ¢ x4 C 5 ¢ x5 C e, where e à N (0, 1) and [x1 , . . . , x10 ] is a 10-dimensional input vector, whose elements represent independent variables distributed uniformly within [0, 1]; e represents a normally distributed noise component. Note that only 5 of the 10 variables have any inuence on the output. In the case of data sets Friedman-2 and Friedman-3, the variables of the four-dimensional input space are independent and distributed uniformly in the ranges: x1 2 [0, 100] , x2 2 [40p , 560p ] , x3 2 [0, 1] , x4 2 [1, 11]. For the q data set Friedman-2, the output variable is generated according to y =

x21 C (x2 x3 ¡ 1/ x2 x4 )2 C e (where e à N (0, 127)), while for the case of the ³ ´ ¡1/ Friedman-3 data set, the output variable is given by y = arctan x2 x3 x1 x2 x4 C e (where e à N (0, 0.106)). 4.1 Network Implementation. CART was implemented according to the description provided in Breiman et al. (1984). The initial tree was grown by successive terminal node splitting (see equation 3.1), while at least one tree leaf contained more than Žve points and a split reducing the training set error was possible. Cost complexity reduced the number of terminal nodes once the original tree had been grown. Tenfold cross validation was used to estimate the optimum pruning parameter a. Both CART-NTN and NTN-CART followed a similar initial tree-growing process, except that all current terminal nodes were split at each step, if possible, according to equation 3.2. In the case of CART-NTN, the costcomplexity pruning identical to the one used in CART was then applied, while NTN-CART found the optimal tree depth, minimizing the estimated generalization error obtained via 10-fold cross validation. In the case of the NTN, in all experiments all inputs were mapped to the 0, . . . , 255 range and thermometer encoded to form a binary retina of size R = 256 ¢ D, where D is the number of input variables. In one set of experiments, the number K of N-tuple memories was chosen to allow complete sampling of the retina bits for a given value of N, that is, K » = R/ N.

N-tuple Network, CART, and Bagging

299

Another set of experiments used a generally larger but Žxed number of Ntuple nodes (K = 500). For each variant of the data set, 25 random N-tuple retina samplings were considered, and the test-set errors obtained in each case were then averaged. The N-tuple memories were implemented using hashing (Knuth, 1973), with each memory node containing 5000 locations. In this way, large N-tuple sizes could be readily considered, and since the training set sizes were in all cases fewer than 500, the probability of hash collisions was negligibly small. To reduce the generalization error due to empty memory locations, the NTN mapping was modeled as y = b C NTN (x ), with the bias b estimated as an average of the training-set values. The network response formation was considered in two variants. One with the output response calculated according to Ko lcz and Allinson (1996), NTN (x ) =

PK

k= 1 wk ( x ) PK k= 1 ck ( x )

=

PT

¡

t t t= 1 y ¢ k x, x ¡ ¢ PT t t= 1 k x , x

¢ ,

(4.1)

where T is the size ¡ t of ¢ the training set, ck ( x ) is the counter associated with t represents the tth input-output training pair, and ( ), weight x x , w y ¡ ¢ k k x , x t is the equivalent kernel function of the NTN. In a one-pass training procedure, wk ( x ) is set to the sum of training values of points addressing that memory location, and ck ( x ) is set to their number. The other form of NTN response was produced according to NTN (x ) =

K 1X wk (x ) , K k= 1 ck ( x )

(4.2)

with the weights and counters set in the same way. The latter variant directly corresponds to the CART network, since the response of each memory location represents an average of the training-point values that address that location and hence fall into the corresponding quantization cell. Since equation 4.2 makes NTN directly related to the remaining networks, it has been chosen as a basis for performance comparison. 4.2 Discussion of Results. The combined results for all networks and data sets considered are presented in Table 1. The complete results for both variants of the NTN and all values of N considered are shown in Figure 2. The accuracy of CART and CART-NTN was closely matched in all cases (with similar depths of the trees produced), which is not surprising since the same pruning algorithm was applied in both cases. The advantage of one network over the other proved to be data dependent: for the Friedman1 and Friedman-3 experiments, CART-NTN was slightly better than CART, while for the Friedman-2 and Boston experiments, the situation was reversed. It appears that the depths of the trees produced by CART-NTN were unaf-

300

Aleksander Ko lcz

Table 1: Combined Mean Squared Error (MSE) Results for All Data Sets. Network

MSE

MSE-bag

Decrease

ATD

ATD-bag

9.0 8.7 6.9 12.3

31% 29% 40% 3%

5 4 4 25

7 4 9 25

24,840 26,297 24,911 30,797

32% 31% 32% 5%

8 4 5 20

9 4 14 20

0.0370 0.0346 0.0281 0.044

38% 36% 37% 2%

4 4 5 20

6 4 14 20

16.2 16.2 13.1 15.9

42% 48% 42% 5%

6 5 8 40

7 5 25 45

Results for the Friedman-1 data set

CART CART-NTN NTN-CART NTN

13.1 12.2 11.4 12.7

Results for the Friedman-2 data set

CART CART-NTN NTN-CART NTN

36,438 37,930 36,510 32,437

Results for the Friedman-3 data set

CART CART-NTN NTN-CART NTN

0.0600 0.0544 0.0449 0.045

Results for the Boston data set

CART CART-NTN NTN-CART NTN

27.8 31.4 22.6 16.7

Notes: MSE-bag refers to the error obtained for the bagged variant of CART, CARTNTN, and NTN-CART and for the variant of NTN with the number of N-tuple nodes Žxed to K = 500 (in the MSE column, the number of nodes of an NTN is just large enough to cover the retina with N-tuples). A decrease in the test-set error due to bagging is indicated in column 4, and the average tree depths (ATD) (and the value of N for NTN)for the standard and bagged variants of the networks are given in columns 5 and 6, respectively. The NTN results correspond to the optimal values of N.

fected by bagging, whereas for both CART and NTN-CART, bagging led to deeper trees. NTN-CART proved to be superior to both CART and CART-NTN for all data sets considered, except for the Friedman-2 experiment, where its performance was essentially equal to that of CART. This can probably be attributed to the optimal-depth pruning used by NTN-CART, which resulted in trees of higher complexity from those used by the other networks. This may indicate that cost-complexity pruning applied to CART and CART-NTN can prune the trees too much in certain cases. Interestingly, the distribution-based trees proposed in Breiman (1996b) also combined higher tree complexity with better prediction accuracy. All three tree networks demonstrated similar levels of sensitivity to the training data, ranging between 29% and 48% for the data sets considered, where the sensitivity is measured by the decrease of the test-set error due

N-tuple Network, CART, and Bagging

301

NTN : BOSTON (Kmin=67, Kmax=666)

NTN : FRIEDMAN­ 1 (Kmin=52, Kmax=513)

80

35

tree

70

tree

kernel

kernel 30

prediction error

tree­ 500

prediction error

tree­ 500

60

kernel­ 500

kernel­ 500

25

50 40

20

30 15

20 10

4

30

40

50

10

FRIEDMAN­ 2 (Kmin=21, Kmax=205) 0.1

kernel

N

30

40

50

tree kernel

0.09

tree­ 500

10

20

NTN : FRIEDMAN­ 3 (Kmin=21, Kmax=205)

tree

12 prediction error

N

tree­ 500

prediction error

x 10

20

kernel­ 500

kernel­ 500

0.08

8

0.07

6

0.06

4

0.05 10

20

N

30

40

50

10

20

N

30

40

50

Figure 2: Combined NTN results for the data sets considered and different values of N. The continuous graph lines correspond to NTN with a retina-size dependent K, while the marker graphs refer to the Žxed K cases. The kernel label refers to the response of an NTN according to equation 4.1 and tree according to equation 4.2.

to bagging. This suggests that although CART-NTN and NTN-CART have more data at their disposal than CART when making node-splitting decisions, the splitting process is inherently less stable than in CART since each split affects the whole of the input space. As far as the N-tuple networks are concerned, their performance proved to be comparable, and in some cases better than the performance of singlenode CART, CART-NTN, and NTN-CART, but generally worse than the bagged versions of these networks. This is largely expected and can be attributed to the random nature of N-tuple sampling, which is far from optimal unless the joint distribution of the input-output space is actually uniform. Increasing the resolution of the input retina by using more than 256 quantization levels per input variable did not result in a signiŽcant

302

Aleksander Ko lcz

improvement of NTN performance. Conforming with the results published in Rohwer and Lamb (1993) and Ko lcz and Allinson (1996), the optimal values of N tend to be rather large compared to the ones usually used in implementations of the NTN. Figure 2 conŽrms that by increasing the number of N-tuple nodes, the network performance is improved. The improvement is often small, which may be attributed to the random nature of N-tuple sampling, which does not take full advantage of the distribution-related information provided by the training data. The performance increase due to large K is more pronounced for the synthetic data sets, for which the input space distribution is truly uniform. The results obtained for NTN indicate a clear advantage of combining the individual node responses according to equation 4.2 when compared to the regression-NTN method of equation 4.1. The explanation lies in the higher bias induced by equation 4.1. Let fw1 , . . . , wK g denote the sums of training values in the cells selected for a particular input. Let fa1 , . . . , aK g and fc1 , . . . , cK g correspond to the averages and numbers of training points in these cells, respectively. The node aggregation method analogous to bagged CART (see equation 4.2) produces the response as K K 1X 1X wk = ak , K k= 1 ck K k= 1

while the response of the regression NTN (see equation 4.1) is given by PK

k= 1 wk PK k= 1 ck

=

PK

k= 1 ck

PK

¢ ak

k= 1 ck

=

K X k= 1

b k ¢ ak ,

where b k is proportional to the number of training points in the kth cell, and K X

bk = 1.

k= 1

Thus cell averages generated by larger number of points have more inuence on the output in this case. But a larger number of training points falling into a cell is likely to be associated with a larger “footprint” of the cell in the input space, in which many points inuencing the cell average ak are likely to be farther away from the actual input point for which the network response is computed. Consequently, a higher bias will result. For larger values of N, the size of partitioning cells is small enough for the diluting effects of equation 4.1 to become insigniŽcant (see Figure 2). Note that if all cells selected by x contain an identical number of points, the responses produced according to equations 4.2 and 4.1 will be identical.

N-tuple Network, CART, and Bagging

303

5 Conclusion

The analogy between a single N-tuple memory node and a tree network, as well as between N-tuple sampling and bagging, was used to propose a tree network (in two variants: CART-NTN and NTN-CART) using the datadriven, tree-growing strategy of CART and the structure of input-space quantization of an NTN node. The concept of bagging was applied to create a number of such network nodes in a data-driven instead of a purely random way. A simple tree-pruning method maintaining a close similarity to the original NTN architecture has also been suggested. NTN-CART, combining all these features, provides a data-driven, tree-based version of an NTN using the concept of bagging for the generation of multiple network nodes. Experimental results show that bagged NTN-CART provides an attractive alternative to the NTN, since its data-driven approach to network design is able to match the placement of quantization thresholds to the empirical data distribution provided by the training set. The performance increase is accompanied by a smaller number of nodes, a smaller value of N, and a way of eliminating the problem of empty memory locations characteristic of N-tuple networks. At the same time, the simpler nature of pruning proposed for NTN-CART can also have advantages for this network when compared with the traditional cost-complexity pruning of CART. References Aleksander, I., & Stonham, T. J. (1979). A guide to pattern recognition using random-access memories. IEE Proceedings—E Computers and Digital Techniques, 2(1), 29–40. Belsley, D., Kuh, E., & Welsh, R. (1980). Regression diagnostics. New York: Wiley. Bledsoe, W. W., & Bisson, C. L. (1962). Improved memory matrices for the Ntuple pattern recognition method. IRE Transactions on Electronic Computers, EC-11, 414–415. Bledsoe, W., & Browning, I. (1959). Pattern recognition and reading by machine. IRE Joint Computer Conference, pp. 225–232. Bradshaw, N. P. (1997). The effective VC dimension of the N-tuple classiŽer. In W. Gerstner, A. Germond, M. Hasler, & J. D. Nicoud (Eds.), ArtiŽcial neural networks—ICANN ’97. 7th International Conference Proceedings (pp. 511–516). Berlin: Springer-Verlag. Breiman, L. (1996a). Bagging predictors. Machine Learning, 24(2), 123–140. Breiman, L. (1996b). Distribution based trees are more accurate. In S. I. Amari, L. Xu, L. W. Chan, I. King, & K. S. Leung (Eds.), Progress in neural information processing: Proceedings of the International Conference on Neural Information Processing (pp. 133–138). Berlin: Springer-Verlag. Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984). ClassiŽcation and regression trees. Monterey, CA: Wadsworth and Brooks/Cole.

304

Aleksander Ko lcz

Friedman, J. H. (1991). Multivariate adaptive regression splines. Annals of Statistics, 19(1), 1–141. Jung, D. M., Krishnamoorthy, M. S., Nagy, G., & Shapira, A. (1996). N-tuple features for OCR revisited. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(7), 734–745. Knuth, D. E. (1973). The art of computer programming (Vol. 3). Reading, MA: Addison-Wesley. Ko lcz, A., & Allinson, N. M. (1996).N-tuple regression network. Neural Networks, 9, 855–869. Luttrell, S. P. (1992). Gibbs distribution theory of adaptive N-tuple networks. In I. Aleksander & J. Taylor (Eds.), ArtiŽcial neural networks 2: Proceedings of the 1992 International Conference on ArtiŽcial Neural Networks (pp. 313–316). New York: Elsevier. Rohwer, R. J. (1995).Two Bayesian treatments of the N-tuple recognition method. Proceedings of the 4th International Conference on ArtiŽcial Neural Networks (pp. 171–176). London: IEE. Rohwer, R. J., & Lamb, A. (1993). An exploration of the effect of super large N-tuples on single layer RAMnets. In N. M. Allinson (Ed.), Proceedings of Weightless Neural Network Workshop ’93 (pp. 33–37). York, UK: University of York. Rohwer, R., & Morciniec, M. (1998). The theoretical and experimental status of the N-tuple classiŽer. Neural Networks, 11(1), 1–14. Roy, R. J., & Sherman, J. (1967). Two viewpoints of K-tuple pattern recognition. IEEE Transactions on System Science and Cybernetics, 3(2), 117–120. Steck, G. P. (1962). Stochastic model for the Bledsoe-Browning pattern recognition scheme. IRE Transactions on Electronic Computers, 11, 274–282. Tattersall, G. D., Foster, S., & Johnston, R. D. (1991). Single-layer lookup perceptrons. IEE Proc. F, 138, 46–54. Received May 1, 1998; accepted February 14, 1999.

NOTE

Communicated by Robert Tibshirani

Improving the Practice of ClassiŽer Performance Assessment N. M. Adams D. J. Hand Department of Mathematics, Imperial College, Huxley Building, 180 Queen’s Gate, London, SW7 2BZ, U. K. In this note we use examples from the literature to illustrate some poor practices in assessing the performance of supervised classiŽcation rules, and we suggest guidelines for better methodology . We also describe a new assessment criterion that is suitable for the needs of many practical problems. 1 Introduction

The construction of classiŽcation tools (Hand, 1997) in real applications can involve a sophisticated design stage, but this is often followed by a much less sophisticated, and sometimes inadequate, performance assessment stage. The consequence of poor assessment methodology is that an inferior rule may be chosen. In order to assess performance, one must Žrst decide on the criteria for measuring it. However, in many applications, assessment criteria are chosen that do not match the problem very well. Restricting ourselves to the two-class case, two criteria are in common use. One is error rate, or misclassiŽcation rate, which assumes that misallocation costs (the respective costs, or utilities, of classifying a class 0 object as class 1, and the reverse) are equal. The other is the area under a receiver operating characteristic (ROC) curve, or the equivalent Gini index, which assumes nothing whatever is known about misallocation costs. Both of these assumptions are unrealistic: costs are unlikely to be equal, and though they are seldom known precisely, it is very rare that nothing is known about them. Using real examples from the literature, our purpose here is to describe shortcomings of routine practice in assessing classiŽer performance and present a set of guidelines that may be useful. In addition, we describe a new loss-based method for comparative analysis of pairs of classiŽers. We begin with the confusion matrix, which shows the cross-classiŽcation of predicted class by true class:

Predicted

Neural Computation 12, 305–311 (2000)

0 1

True 0 1 a b c d

c 2000 Massachusetts Institute of Technology °

306

N. M. Adams and D. J. Hand

where a C b C c C d = n and n is the total number of objects to which the classiŽer has been applied. In order to obtain unbiased performance estimates, one would normally apply the classiŽer to an independent test set or use more sophisticated methods, but our purpose here is to discuss performance assessment measures rather than issues of estimation. Suppose that class 0 represents diseased people and class 1 represents healthy people. Then a/ ( a Cc) is called sensitivity, denoted Se, and d/ ( b Cd) is called speciŽcity, denoted Sp. If we knew that the exact cost of misclassifying a diseased individual as healthy was c0 , and the reverse misallocation cost was c1 , then the overall expected loss from using the rule would be (1 ¡ Se)p 0 c0 C (1 ¡ Sp)p 1 c1 , where p i refers to the prior probability of class i. Loss is the appropriate criterion for many classiŽcation problems. MisclassiŽcation rate corresponds to equal costs. However, eliciting misallocation costs in real situations can be extremely difŽcult, and instead people often adopt the area under an ROC curve. An ROC curve is (normally) a plot of sensitivity on the vertical axis and (1-speciŽcity) on the horizontal axis. The area under this curve (denoted AUC) is a popular summary measure of performance that incorporates all possible costs, so that one does not have to choose a particular cost pair (Bradley, 1997). However, this goes to the other extreme from the explicit loss value above and assumes that nothing can be elicited about costs. This is just as unrealistic as assuming the costs are known exactly. 2 Examples from the Literature

In this section, we discuss Žve examples from the applications literature that could have used better assessment methodology. Our purpose is not to criticize these authors, since many of these practices seem endemic, but rather to demonstrate scope for improvement in the practice of classiŽer assessment. 2.1 Assuming Equal Costs. Koziol, Adams, and Frutos (1996) were interested in seeing if certain indicators were useful for classifying interstitial cystitis cases into ulcer and nonulcer categories. They compared LDA (linear discriminant analysis), logistic regression, and recursive partitioning using error rate. However, it is very rare in a medical context that the seriousness of the two kinds of misclassiŽcations are equal. Typically (though not always) allocating a case as a noncase is more serious (i.e., incurs a greater cost) than the reverse.

ClassiŽer Performance Assessment

307

2.2 Integrating over Costs. Page et al. (1996) discuss the use of neural networks to discriminate between the brain scans of healthy subjects and Alzheimer’s patients. They conclude that for this purpose, neural nets are better than the conventional statistical method (LDA) on the basis of the AUC. The AUC summarizes classiŽer performance across the entire cost range. It is difŽcult to believe that in this problem, the entire range of possible relative costs is equally legitimate. This would mean, for example, that one was prepared to entertain the possibility that misclassifying an Alzheimer’s patient as healthy was vastly more serious than the reverse— and also to entertain the possibility that misclassifying a healthy subject as suffering from Alzheimer’s was vastly more serious than the reverse. It is unlikely that both cost allocations would be regarded as legitimate. It is much more likely that a narrower range would be appropriate—that one type of misclassiŽcation is more serious than the other, even if one cannot say by how much. 2.3 Crossing ROC Curves. Comparing the AUCs of two classiŽers can be appropriate only if one curve entirely dominates the other (one classiŽer is better than the other for all costs). If ROC curves cross, as they do in Page et al. (1996, p. 199), it means that different classiŽers are better for different values of the costs. Summarizing over all possible costs could lead to completely incorrect conclusions. 2.4 Fixing Costs. There are very few studies in the applications literature that use explicit misallocation costs. This is because there are normally intrinsic uncertainties surrounding the outcome (just how serious is one type of misdiagnosis compared with the other?). Even in domains where one might have expected precise costs to be available, such as commercial and banking domains, they are rare. Michie, Spiegelhalter, and Taylor (1994) give precise costs for a problem of classifying applicants for loans to good and bad classes, but in our experience these misallocation costs are seldom available (Hand & Henley, 1997). The costs depend on proŽtability, which includes issues such as marketing and advertising costs. Moreover, the relative misclassiŽcation costs are likely to change over time as the economy evolves. Although it is probable that a range of likely costs can be given, it is improbable that exact ones can be given. 2.5 Variability. Dybowski, Weller, Chang, and Gant (1996) discuss the decision-making process involved in admitting patients to intensive care. They conducted a study to assess the performance of various neural network models and logistic discrimination in predicting death during admission, speciŽcally of predicting in-hospital mortality among patients with systemic inammatory response syndrome and hemodynamic shock. Comparison

308

N. M. Adams and D. J. Hand

was made using both AUC and Brier score (Hand, 1997). However, no standard error is quoted for any of the summary statistics for the competing classiŽers, and no signiŽcance test results are given. The chosen architecture may have had the highest AUC purely by chance. 3 Guidelines for Comparing ClassiŽers

Our examples illustrate that equal costs (error rate) are rarely an appropriate assumption, that assuming nothing is known about the costs is unrealistic, that assuming they are known exactly is often unrealistic, that AUC may be entirely inappropriate if ROC curves cross, and that in comparing rules, the variation in the performance statistics must be taken into account. We could have easily picked other articles to demonstrate these points. In general, the assessment criteria must be chosen to match the objectives. Methodological and simulation studies out of any explicit context may have little bearing on real problems. In particular, it is important to take into account what is known about the relative misclassiŽcation costs because this can radically inuence the choice of method. In section 4 we describe a performance measure that does this, making use of what can be said about the costs (since something can always be said) even if precise values cannot be given. 4 LC Assessment Method

In this section we outline the LC assessment method (described in detail in Adams and Hand, 1999), using the German credit data from the STATLOG project (Michie, Spiegelhalter, & Taylor, 1994). These data consist of 24 variables measured on 1000 people: 700 regarded as good credit risks (class 0) and 300 regarded as bad credit risks (class 1). These data were divided into training and test sets of 800 and 200 observations, respectively. The ROC curves for linear and quadratic discriminant analysis (QDA), estimated on the training set and applied to the test set, are given in Figure 1. In terms of the shortcomings described in Section 2, this comparison involves crossing ROC curves. The AUCs (and associated standard errors) for LDA and QDA are 0.8371 (0.0268) and 0.8135 (0.0288), respectively. These summaries provide no evidence to favor either classiŽer, although published studies may ignore the estimates of standard error. Such a comparison also involves integrating over all possible costs and ignoring estimates of variability, both shortcomings described above. In fact, the German credit data have a misallocation matrix, which states that the severity of misclassifying a bad risk is Žve times that of misclassifying a good risk. With this extra information, standard ROC curve analysis is certainly inappropriate. However, our experience (Hand & Henley, 1997;

309

0.8

1.0

ClassiŽer Performance Assessment

0.0

0.2

0.4

F^(t|0)

0.6

LDA QDA

0.0

0.2

0.4

0.6

0.8

1.0

F^(t|1)

Figure 1: ROC curves for LDA and QDA applied to the German credit data.

Hand & Jacka, 1998) suggests that such precision in the deŽnition of misallocation costs for credit problems is misplaced, and we therefore specify a range of 3 to 7 for this misclassiŽcation cost, with most probable value remaining at the quoted one of 5. It is not meaningful for a given classiŽer to compare the losses arising from two different costs pairs (c0 , c1 ) and ( c00 , c01 ). All one can do is compare the losses from different classiŽers at a Žxed cost pair. Since the losses due to different cost pairs cannot be compared, the only useful information in comparing different classiŽers is whether one classiŽer has a smaller loss than the other for particular cost pairs. It follows that the values of a pair ( c0 , c1 ) are not important, but only the relative size of c0 and c1 . For practical reasons outlined in Adams and Hand (1999), it is convenient to impose the constraint c0 C c1 = 1 and work just in terms of c1 . Domain experts can normally specify upper and lower bounds on the possible values c1 can take (although in our experience they do this most comfortably in terms of the cost ratio cc10 ) and they can also specify a most likely value for c1 . We use these three values to deŽne a belief distribution over the possible values of c1 . For reasons explained in Adams and Hand (1999), we take a triangular function with support deŽned by the upper and lower bounds and maximum at the most likely value. This belief function, L( c1 ) tells us how probable the expert thinks it is that the different values of c1 will occur.

310

N. M. Adams and D. J. Hand

For each possible value of c1 , we can see which classiŽer incurs the smallest loss. This allows us to deŽne an indicator function I ( c1 ) for each classiŽer that takes the value 0 if the classiŽer does not have the smallest loss for a given value of c1 and the value 1 if it does. For each classiŽer we combine I and L as Z LC = I ( c1 ) L( c1 ) dc1 to give an overall measure of expected performance. The best classiŽer is that which gives the largest value of the LC index (Adams & Hand (1999) are concerned with only two classiŽers and work with the difference of the two LC values.) Returning to the example, we have the predictions of LDA and QDA on the test set and the belief distribution deŽned earlier. The estimates in this case are LCLDA = 1 and LCQDA = 0. SuperŽcially, then, this is strong evidence in favor of LDA. Estimates of standard error for the LC index are not as readily obtained as estimates of standard error for more conventional summaries like error rate or AUC. The approach we adopt in Adams and Hand (2000) is to obtain a bootstrap estimate (Efron & Tibshirani, 1993; Davidson & Hinkley, 1997) of the standard error of the LC index. This involves drawing random resamples (of the same size as the test set) with replacement from the test set. The LC index for the two classiŽers is computed on each resample. The standard deviation of the resampled indexes is the bootstrap estimate of standard error. Since this example involves two classes, the LC index is a comparative measure (complementary results are obtained for the two classes). This implies that it is not necessary to use a paired test: the ratio of LC index to bootstrap standard error can serve as a test statistic. In this case, the standard error (common to both classiŽers) is 0.0224. Using LC analysis in this case has come to a Žrm conclusion using an appropriate cost interval and incorporating variability estimates. 5 Conclusion

Efforts to optimize classiŽer performance can be wasted by the use of an inappropriate assessment criterion. A classiŽer that is suboptimal with respect to the characteristics of the problem can be installed. We have presented some guidelines and a tool that should help reduce these problems. Acknowledgments

The work of N. A. on this project was supported by grant GR/K55219 from the EPSRC, under its “Neural Networks—The Key Questions” initiative.

ClassiŽer Performance Assessment

311

References Adams, N. M., & Hand, D. J. (2000). An improved method for comparing diagnostic tests. To appear in Computers in Biology and Medicine. Adams, N. M., & Hand, D. J. (1999). Comparing classiŽers when the misallocation costs are uncertain. Pattern Recognition, 32, 1139–1147. Bradley, A. P. (1997). The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recognition, 30(7), 1145–1159. Davison, A. C., & Hinkley, D. V. (1997). Bootstrap methods and their application. Cambridge: Cambridge University Press. Dybowski, R., Weller, P., Chang, R., & Gant, V. (1996). Prediction of outcome in critically ill patients using artiŽcial neural-network synthesized by genetic algorithm. Lancet, 347, 1146–1150. Efron, B., & Tibshirani, R. J. (1993). An introduction to the bootstrap. London: Chapman and Hall. Hand, D. J. (1997). Construction and assessment of classiŽcation rules. Chichester: Wiley. Hand, D. J., & Henley, W. E. (1997).Statistical classiŽcation methods in consumer credit scoring: A review. Journal of the Royal Statistical Society, Series A, 160(3), 523–541. Hand, D. J., & Jacka, S. D. (1998). Statistics in Žnance. London: Arnold. Koziol, J. A., Adams, H. P., & Frutos, A. (1996). Discrimination between the ulcerous and the nonulcerous forms of interstitial cystitis by noninvasive Žndings. Journal of Urology, 155(1), 89–90. Michie, D., Spiegelhalter, D. J., & Taylor, C. C. (1994). Machine learning, neural networks and statistical classiŽcation. New York: Ellis Horwood. Page, M. P. A., Howard, R. J., O’Brien, J. T., Buxton-Thomas, M. S., & Pickering, A. D. (1996).Use of neural networks in brain SPECT to diagnose Alzheimer ’s disease. Journal of Nuclear Medicine 37(2), 195–200. Received August 24, 1998; accepted March 1, 1999.

LETTER

Communicated by Alexandre Pouget

Do Simple Cells in Primary Visual Cortex Form a Tight Frame? Emilio Salinas Instituto de Fisiolog´õa Celular, UNAM, Ciudad Universitaria S/N, 04510 M´exico D.F., M´exico L. F. Abbott Volen Center for Complex Systems and Department of Biology, Brandeis University, Waltham, MA 02254-9110, U.S.A Sets of neuronal tuning curves, which describe the responses of neurons as functions of a stimulus, can serve as a basis for approximating other functions of stimulus parameters. In a function-approximating network, synaptic weights determined by a correlation-based Hebbian rule are closely related to the coefŽcients that result when a function is expanded in an orthogonal basis. Although neuronal tuning curves typically are not orthogonal functions, the relationship between function approximation and correlation-based synaptic weights can be retained if the tuning curves satisfy the conditions of a tight frame. We examine whether the spatial receptive Želds of simple cells in cat and monkey primary visual cortex (V1) form a tight frame, allowing them to serve as a basis for constructing more complicated extrastriate receptive Želds using correlationbased synaptic weights. Our calculations show that the set of V1 simple cell receptive Želds is not tight enough to account for the acuity observed psychophysically. 1 Introduction

There are a number of parallels between representation and learning in neural networks and methods of functional expansion in mathematical analysis. Populations of neurons with responses tuned to properties of a stimulus may be well suited for approximating functions of stimulus parameters (Poggio, 1990; Poggio & Girosi, 1990; Girosi, Jones, & Poggio, 1995). If certain conditions are satisŽed, such a population can act as a highly efŽcient substrate for function representation. Accurate representation of a given function depends critically on the synaptic weights connecting input and output neurons. These weights are often adjusted with a delta-type learning rule that is efŽcient but not biologically plausible. However, if the tuning curves of the input neurons form what is called a tight frame (Daubechies, Grossmann, & Meyer, 1986; Daubechies, 1990, 1992), function approximation can be achieved using a more biologically reasonable Hebbian or correlationNeural Computation 12, 313–335 (2000)

c 2000 Massachusetts Institute of Technology °

314

Emilio Salinas and L. F. Abbott

Figure 1: A feedforward network for function approximation. The input neurons at the bottom of the Žgure respond with Žring rates that are functions of the stimulus variable x. The Žring rate of input neuron j is fj (x). The upper output neuron is driven by the input neurons through synaptic connections; wj represents the strength, or weight, of the synapse from input neuron j to the output neuron. The task is to make the Žring rate R of the output neuron approximate a given function F(x).

based rule for the synaptic weights. We investigate here whether the simple cells of primary visual cortex provide such a basis. To introduce the hypothesis to be tested, we begin with a simple example. Consider a population of N neurons that respond to a stimulus by Žring at rates that depend on a single stimulus parameter x. The Žring rate of neuron j as a function of x, also known as the response tuning curve, is fj ( x). We wish to examine whether such a population can generate an approximation of another function F( x) that is not equal to any of the input tuning curves. Figure 1 shows a simple network for function approximation. In this network, a population of neurons acting as an input layer drives a single output neuron through a set of synapses. The target function F is represented by the Žring rate of the output neuron. The weight of the synapse connecting input neuron j to the output neuron is denoted by wj . Following standard neural network modeling practice, we write the Žring rate R of the output neuron in response to the input x as a synaptically weighted sum of the input Žring rates, R=

N X

wj fj (x).

(1.1)

j= 1

If R = F( x), or at least R ¼ F(x) to a sufŽciently high degree of accuracy, the tuning curve of the output neuron will provide a representation of the target function. The problem is to Žnd a set of input tuning curves, and the synaptic weights corresponding to these tuning curves, that produces an output response close to the target function. The expansion of a function as a weighted sum of other functions is a standard problem in mathematical analysis. In the limit N ! 1, any set

Simple Cells and Tight Frames

315

of functions that provides a complete basis over the range of x values considered can be used. Completeness is not very restrictive, but an additional condition is usually applied. The weights wj that make F( x ) =

1 X

wj fj (x)

(1.2)

j= 1

can be computed easily if the basis functions fj are orthogonal and normalized so that Z (1.3) dy fi ( y) fj ( y) = d ij , where d ii = 1 for all i and d ij = 0 for all i 6 = j. In this case the weights are given by Z (1.4) wj = dy fj ( y) F( y). Equation 1.4 for the weights is closely related to forms of Hebbian and correlation-based learning commonly used in neural networks. A correlationbased rule sets the synaptic weights to values given by wj = h fj Ri,

(1.5)

where the brackets indicate an average over stimuli. Equation 1.5 is equivalent to the statement that the synaptic weight is proportional to the correlation of the Žring rates of the pre- and postsynaptic neurons. The idea that synaptic weights are related to correlations in pre- and postsynaptic Žring is a standard hypothesis going back to the work of Hebb (1949). Often this principle is expressed in the form of a learning rule that adjusts the weights incrementally toward their desired values during a training period. We can also think of equation 1.5 as a consistency condition required for the maintenance of weights at their correct values. Suppose that a network has been set up to approximate a given function, so that R = F( x) to a high degree of accuracy. In order for the network to function properly over an extended period of time, some mechanism must maintain the weights at values that produce this desired output. If synaptic weights are maintained by a process that uses a correlation-based rule, condition 1.5 can be interpreted as a consistency condition on maintainable weights. In this interpretation, the Hebbian rule is applied continuously; there is no distinct training period. The average over stimuli in equation 1.5 is then over all stimuli typically encountered, and the output Žring rate R can be approximated by the function F(x). Thus, the consistency condition is wj = h fj (x) F(x)i.

(1.6)

316

Emilio Salinas and L. F. Abbott

If all stimulus values x occur with equal probability, the average over stimuli is proportional to an integral over x, and the weights given by equations 1.4 and 1.6 are proportional to each other. Thus, the Hebbian consistency condition is equivalent to the formula for the coefŽcients of an orthogonal expansion. The relationship between equation 1.4 and correlation-based synaptic weights suggests a close relationship between Hebbian ideas about synaptic plasticity and standard mathematical approaches to functional expansion. Unfortunately, a direct link of this type is inappropriate because most sets of neuronal tuning curves are highly overlapping and far from orthogonal. However, all is not lost. What happens if the values of the weights given by equation 1.4 are used in equation 1.1 even when the input tuning functions are not orthogonal? This question can be answered simply by substituting these weights into equation 1.1: R=

N Z X

dy fj ( y) F( y) fj ( x).

(1.7)

j= 1

If we interchange the order of summation and integration and deŽne D( x, y) ´

N X

fj ( x) fj ( y),

(1.8)

j= 1

we Žnd that the output Žring rate is a convolution of the target function with D, Z (1.9) R = dy D( x, y) F( y). By computing the D function for a given set of input tuning curves, we can determine how accurately a set of neurons can represent a function using synaptic weights given by equation 1.4. The output Žring rate in equation 1.9 will be exactly equal to the target function if D( x, y) = d ( x ¡ y), since Z

R=

dyd (x ¡ y) F( y) = F( x),

(1.10)

by the deŽnition of the Dirac d function. The condition D( x, y) = d ( x ¡ y) is less restrictive than the orthogonality condition. A set of functions fj that satisfy D(x, y) = d ( x ¡ y) is called a tight frame (Daubechies et al., 1986; Daubechies, 1990, 1992). A Žnite number of well-behaved tuning curves can never generate a true d function in equation 1.8, but they can create a good approximation to one. If the integral of D( x, y) over y is one for all values of x and if

Simple Cells and Tight Frames

317

D(x, y) is signiŽcantly different from zero only for values of y near x, the integral in equation 1.9 is approximately equal to F(x), provided that F varies more slowly than D. Thus, Žnite sets of tuning curves can act as approximate tight frames. The importance of D( x, y) is that, as a function of y, its width determines the resolution with which an output function can be approximated using a correlation rule like equation 1.6 for the synaptic strengths. For this reason, it is interesting to examine D functions for real neuronal tuning curves. The D function provides a useful characterization of the computational capabilities of a population of neurons for function expansion. To apply ideas about tight frames to real neural circuits, we must Žnd areas where function approximation may be taking place. Primary visual cortex seems to be a good candidate. Neurons in primary visual cortex provide the Žrst cortical representation of the visual world, and their receptive Želds provide a basis set from which the more complicated receptive Želds of extrastriate neurons are constructed. If the tuning curves of simple cells form a tight frame, the construction of more complicated representations using an architecture like that in Figure 1 can be based on Hebbian synapses whose efŽcacy is maintained by correlation-based rules of the form given by equation 1.6. To investigate these ideas, we will compute the D function (and a related function K) for the receptive Želds of simple cells in the primary visual cortices of cats and monkeys. This will allow us to determine how well the receptive Želd functions of simple cells collectively approximate a tight frame. Lee (1996) has explored the conditions under which sets of Žlters with properties like those of simple cells satisfy the mathematical deŽnition of a tight frame. We will use the experimentally measured Žlter distributions to determine explicitly the resolution with which simple cells can generate arbitrary visual Žlters for downstream neurons using correlation-based synaptic weights. 2 Function Approximation by Simple Cells

Representation of the visual world is more complicated than the simpliŽed scenario described in section 1, so we start by recasting the basic problem in terms that are appropriate for this application. Simple cell responses can be expressed approximately through spatiotemporal linear Žlters acting on visual image intensity, but we will focus exclusively on the spatial aspects of these responses. If I (x ) is the luminance (relative to background) of an image that appears at the point x on the retina, where x is a two-dimensional vector, the Žring rate of simple cell j is approximated as Z rj = g

dx fj ( x) I ( x ),

(2.1)

318

Emilio Salinas and L. F. Abbott

where g is an overall scale factor determining the magnitude of the Žring rate. Because this expression can be positive or negative, we must interpret it as the difference between the Žring rates of two neurons having receptive Želd Žlters Cfj and ¡ fj . The Žlter functions fj for simple cells are often Žt using plane waves multiplied by gaussian envelopes, known as Gabor functions (Daugman, 1985; Jones & Palmer, 1987; Webster & De Valois, 1985), ! | x ¡ aj | 2 1 exp ¡ cos( kj ¢ x ¡ w j ). (2.2) fj (x ) = 2p sj2 2sj2

(

These Žlters are parameterized by the receptive Želd center a, receptive Želd envelope width s, preferred spatial frequency k ´ | k |, preferred orientation h (the direction perpendicular to k ), and spatial phase w . For simplicity, we restrict ourselves to circular rather than elliptical receptive Želd envelopes. We have veriŽed that for values in the physiologically measured range, elliptical receptive Želds give essentially the same results reported below for circular receptive Želds. We have included a factor of 1/ s 2 in the normalization of the Gabor function so that neurons with different Žlters give similar maximum Žring rates when the optimum images are presented with a given overall amplitude. In the case of visual responses, the Žlter functions Žll the role that the tuning curves played in the example given in section 1. Similarly, the target function we wish to represent in this case is a Žlter function. In other words, we attempt to construct a downstream neuron that is selective for some target function or target image F( x ). The target response is given by Z = (2.3) R G dz F(z ) I ( z), with G the scale factor for the output rate. For Žxed image power, the maximum value of R is evoked when I is equal to the target function F. This output neuron is driven, through synaptic weights, by the responses of a population of simple cells, so that R=

N X j= 1

wj rj = g

N X

Z wj

dx fj ( x) I ( x ),

(2.4)

j= 1

where we have used equation 2.1. Again we impose the condition that synaptic weights are equal to the average product of pre- and postsynaptic Žring rates, assuming that the postsynaptic neuron Žres as prescribed by equation 2.3. The weights are then given by Z = hr = (2.5) wj Gg dy dz fj ( y )F( z )hI ( y )I ( z )i. j Ri

Simple Cells and Tight Frames

319

In this case the average over stimuli, denoted by angle brackets, is an average over the ensemble of visual images. Substituting these weights into equation 2.4, we Žnd Z R = Gg2

dx dz K( x, z )F( z ) I (x ),

(2.6)

with Z K ( x , z) =

dy D( x, y )hI ( y ) I (z )i

(2.7)

and N X

D( x , y ) =

fj ( x ) fj ( y).

(2.8)

j= 1

Equation 2.6 tells us that the Žring rate of the output neuron is proportional to a Žltered integral of the image, Z R = Gg

2

dx Fef f (x ) I ( x).

(2.9)

The effective Žlter Fef f is a smeared version of the desired Žlter F and is given by Z Fef f ( x ) =

dy K(x , y )F( y ).

(2.10)

The condition for perfect function representation is then K( x , y) = d ( x ¡ y ), and the resolution with which a given Žlter function can be represented is related to the degree to which K approximates a d function. To determine how well simple cells approximate a tight frame, we Žrst compute both the D and K functions. An interesting aspect of the computation of D from equation 2.8 is that it involves not only the form of the receptive Želd Žlters of simple cells, but also the distribution of their parameters across the population of cortical neurons. Once D is determined, we compute K from equation 2.7 by including the correlation function hI( y ) I (z )i for natural images (Field, 1987; Ruderman & Bialek, 1994; Dong & Atick, 1995). Then we discuss whether D and K behave like a d function. 3 Calculation of D( x , y )

Equation 2.8, which deŽnes D, is a sum over simple cell Žlters. To compute it, we reparameterize the Žlters 2.2 in terms of quantities whose distributions

320

Emilio Salinas and L. F. Abbott

have been measured. We then approximate the sum over cells by integrals over those parameters using mathematical Žts to reported distributions. To simplify the calculation, we group the simple cell Žlters into pairs with spatial phases 90 degrees apart (quadrature pairs), corresponding to sine and cosine terms. Each pair of Žlters can then be described by a single complex exponential Žlter gj , ! ¡ ¡ ¢¢ | x ¡ aj | 2 1 exp ¡ exp i kj ¢ x ¡ wj , (3.1) gj (x ) = 2 2 2p sj 2sj

(

p where i ´ ¡1. Neurophysiological experiments have found pairs of adjacent simple cells roughly 90 degrees out of phase with each other (Pollen & Ronner, 1981), and we assume that combinations as in equation 3.1 can be formed for all cells. Using complex exponential Žlters, the sum that deŽnes D becomes D( x , y) =

N X j= 1

=

N X

gj ( x ) gj¤ ( y) 1

4p 2 sj4 j= 1

(

exp ¡

| x ¡ aj | 2 C | y ¡ aj | 2 2sj2

!

¡ ¢ exp ikj ¢ ( x ¡ y ) ,

(3.2)

where gj¤ denotes the complex conjugate of gj . Note that the spatial phase cancels out in the expression for D. Simple cell receptive Želd Žlters are characterized by location, envelope width, and preferred spatial wavelength, phase, and orientation. The envelope width and preferred spatial frequency are correlated; larger receptive Želds tend to have lower preferred spatial frequencies. To eliminate this dependence, we will characterize the receptive Želds by bandwidth rather than by envelope width. The bandwidth is the width of the spatial frequency tuning curve measured in octaves. It is deŽned as b ´ log2 (v2 / v1 ), where v2 and v1 are the higher and lower spatial frequencies, respectively, at which the neuron gives half of its maximum response. The bandwidth is related to the envelope width s and the preferred frequency k by p 2 ln(2) 2b C 1 s= . (3.3) 2b ¡ 1 k We use b and k as independent parameters, with the tuning curve width s given by equation 3.3. To compute D, we use an integral over receptive Želd parameters as an approximation to the sum over neurons, making the replacement N X j= 1

Z !

da dkdbdh dw r a ( a )r k ( k)r b (b)r h (h )r w (w ),

(3.4)

Simple Cells and Tight Frames

321

where the different r functions are the neuronal coverage densities (the number of neurons per unit parameter range) for the different receptive Želd parameters. The validity of this approximation is addressed at the end of this section. Preferred orientations and spatial phases appear to be uniformly distributed over neurons in primary visual cortex (Hubel & Wiesel, 1974; Field & Tolhurst, 1986; Bartfeld & Grinvald, 1992; Bonhoeffer & Grinvald, 1993) so we assume that rh (h ) and r w (w ) are constants. Over a small region of the cortex, r a ( a) can also be approximated as a constant. This means that we are considering a small enough region of the visual Želd so that the dependence of the cortical magniŽcation factor on eccentricity can be ignored. Since the D function we compute will be quite narrow, this is a reasonable approximation. With these assumptions, we Žnd Z D( x , y ) =

dadkdbdh dw r k (k)r b (b)

¡

| x ¡ a| 2 C | y ¡ a | 2 c exp ¡ s4 2s 2

¢

¡ ¢ exp ik| x ¡ y |) cos(h ) .

(3.5)

In this and all the following expressions, c denotes a constant that absorbs all of the factors that affect the amplitude but not the functional form of D. To simplify the above expression, we have made a shift in the deŽnition of the angle h , which is allowed because the h integral is over all values. The integrals over a and w in equation 3.5 can be done immediately to obtain Z (| ) D x ¡ y | = dk dbdh r k ( k)r b (b)

¡

| x ¡ y| 2 c exp ¡ 2 s 4s 2

¢

¡ ¢ exp ik| x ¡ y | cos(h ) .

(3.6)

Note that D depends on only the magnitude of x ¡ y . To proceed further, we need to determine the density functions r k ( k) and r b (b). We have done this by Žtting three different data sets. We Žrst performed Žts on the data reported by Movshon, Thompson, and Tolhurst (1978) for neurons with 5 degree eccentricity or less in cat area 17. The data and Žts are shown in Figure 2 (top). The mathematical Žts are given by

(

(b ¡ 1.39) 2 r b (b) = p exp ¡ 2(0.44) 2 2p 0.44 16.6

! (3.7)

and r k ( k) =

1.7 k2 ¡ ¢5 , 1 C k/ 5.2

(3.8)

322

Emilio Salinas and L. F. Abbott

Figure 2: Distributions of optimal spatial frequencies (left) and bandwidths (right) for simple cells. The black circles correspond to experimental data, and the solid curves are mathematical Žts. (Top) Cat area 17 simple cells with eccentricities smaller than 5 degrees. Data from Movshon et al. (1978). (Middle) Monkey V1 simple cells within 1.5 degrees of the fovea. Data from De Valois et al. (1982). (Bottom) Monkey V1 simple cells with eccentricities from 3 to 5 degrees (parafoveal sample). Data from De Valois et al. (1982). Notice the different scales in the frequency axes of cat and monkey data. Parameters in the Žtting equations were found by minimizing the  2 statistic (Press et al., 1992). All Žts had P > 0.3, except for the preferred frequency distribution in the monkey parafoveal sample (lower left), which had P > 0.01.

and we assume that r k ( k) = 0 for k > 20 radians per degree. We also Žt two data sets of De Valois, Albrecht, and Thorell (1982) from area V1 in macaque monkeys. These correspond to a region from 0 to 1.5 degrees eccentricity

Simple Cells and Tight Frames

323

(foveal) and another from 3 to 5 degrees eccentricity (parafoveal). The Žts for the region closest to the fovea are shown in Figure 2 (middle), and are given by

(

(b ¡ 1.49) 2 r b (b) = p exp ¡ 2(0.62) 2 2p 0.62 31.7

! (3.9)

and r k ( k) =

0.17 k2 ¡ ¢5 . 1 C k/ 22.2

(3.10)

We assume that r (k) = 0 for k > 90 radians per degree. The Žts for the parafoveal region are shown in Figure 2 (bottom) and are given by

(

(b ¡ 1.3) 2 r b (b) = p exp ¡ 2(0.49) 2 2p 0.49 16.2

! (3.11)

and r k ( k) =

0.22 k2 ¡ ¢5 . 1 C k/ 14.6

(3.12)

Here we assume that r ( k) = 0 for k > 50 radians per degree. The distributions we have extracted exhibit an interesting property: the bandwidth distributions from all three data sets are very similar, and the preferred spatial frequency distributions are scaled versions of each other to a high degree of accuracy. Because the receptive Želd width s is inversely proportional to the preferred spatial frequency k (see equation 3.3), the integrand in equation 3.6 depends on | x ¡ y | only through the combination k| x ¡ y |. This means that scaling the distribution r k ( k) by a factor s will have the effect of scaling the argument of the resulting D function to give D(s| x ¡ y |). As a result, we can compute the D function for any one of the distribution pairs and determine D for the others simply by scaling the spatial argument of the function. We have veriŽed through numerical integration that the three bandwidth distributions are equivalent and that this approximation is practically exact (compare Figures 3A and 4A). We will use the experimental results for cells nearest to the fovea in the monkey. The D function we compute and show in the Žgures corresponds to this case, but the results for the parafoveal region of the monkey can be obtained by broadening these curves by a factor of 1.52, and for the cat data with a broadening factor of 4.27. Results for any eccentricity in either animal can similarly be obtained by scaling the curves by appropriate factors.

324

Emilio Salinas and L. F. Abbott

Figure 3: D function for monkey simple cells. All D curves shown are normalized to a maximum height of one. (A) Result when the experimentally measured neuronal distributions, equations 3.9 and 3.10, are used in the calculation of D. The Žlled circles correspond to numerical integration of equation 3.6 for different values of | x ¡ y |; the continuous line is a Žt to the numerical results using a combination of three gaussian functions (see equation 3.13). The width of the curve at half maximum height, which determines the resolution for function approximation, is 0.06 degree. (B) Results obtained when the distributions of bandwidths and preferred frequencies are manipulated. The dotted line is the D function when only neurons near the peaks of both distributions are used; the width at half-height is 0.13 degree. The continuous line is the result when only neurons with optimal frequencies lower than 50 radians per degree are included in the calculation; the width at half-height is 0.086 degree. The dashed line is the result when only neurons with optimal frequencies higher than 50 radians per degree are included; the width at half-height is 0.042 degree. In these last two cases the full bandwidth distribution was used.

Substituting equations 3.3, 3.9, and 3.10 into 3.6, the triple integral can be computed numerically for different values of | x ¡ y | to yield the function D. The result is shown in Figure 3A, where D has been normalized to a maximum value of one. The continuous line is a Žt to the numerical result, indicated by dots. The Žt was obtained by combining three gaussians,

¡

D f it (| x ¡ y | ) = 0.8590 exp ¡

| x ¡ y| 2 2(0.024) 2

¡

¡ 0.2579 exp ¡

¢

| x ¡ y| 2 2(0.085) 2

¡

C 0.3590 exp ¡

¢

.

| x ¡ y| 2 2(0.059)2

¢

(3.13)

The Žgure shows that the full width at half-maximum height is 0.06 degrees. It also shows that the sidebands around the central peak are very small; the function does not oscillate. This suggests that D can approximate ad function to a resolution of roughly 4 minutes of arc.

Simple Cells and Tight Frames

325

Figure 4: D function for simple cells and retinal ganglion cells of the cat. Both curves are normalized to a maximum height of one. (A) The D function for simple cells of the cat primary visual cortex obtained by numerical integration of equation 3.6. This curve is practically indistinguishable from the result in Figure 3A scaled by a factor of 4.27. For comparison, the x-axes in this Žgure have been scaled precisely by this amount. The full width at half maximum height is 0.26 degree. (B) The D function for a uniformly distributed population of difference of gaussian Žlters resembling those of retinal ganglion cells (see the appendix). The full width at half maximum height is 0.5 degree.

In Figure 3B we examine how the D function depends on the distributions of preferred spatial frequencies and bandwidths used in the calculation. First, we investigate what happens if only neurons with parameters near the peaks of the distributions in Figure 2 are considered. We include bandwidths ranging between 1.4 and 1.6 octaves and optimal frequencies ranging between 21 and 23 radians per degree, with no neurons outside these ranges. The D function that results is shown in Figure 3B (dotted line). It has large inhibitory sidebands, and its width at half-height is more than twice the width of the original function. When the full bandwidth distribution is used but the same restricted range of preferred frequencies is included, the result is similar. The neurons at the tail of the preferred frequency distribution are important in setting the width of D, even though proportionally they represent a small fraction of the total. When the distribution is truncated, allowing only optimal frequencies below 50 radians per degree, the resulting D function has a width of 0.086 degrees. Conversely, when only optimal frequencies above 50 radians per degree are included, the resulting D function has a width of 0.042 degrees. Thus, the resolution of the full set of simple cells seems to be limited by the highest frequencies included. The 1/ s 2 factor that appears in equation 2.2 is partly responsible for this effect. It is interesting to compare the D function of simple cells with that of neurons at earlier stages in the visual pathway. In the case of retinal gan-

326

Emilio Salinas and L. F. Abbott

glion cells, the computation is straightforward because their Žlters can be described by differences of gaussians (see the appendix). The D functions for simple cells and retinal ganglion cells corresponding to data collected in cats are shown in Figure 4. The full widths at half-maximum height are 0.26 and 0.5 degree, respectively. The resolution of simple cells for function approximation in the same animal is better, roughly by a factor of 2. This is related to the fact that some simple cells have high preferred spatial frequencies or, equivalently, Žlters that vary more rapidly than those of retinal ganglion cells. Because the integral in equation 3.6 was computed numerically using the trapezoidal rule (Press, Flannery, Teukolsky, & Vetterling, 1992), the D function was actually computed as a sum. This is appropriate since the biological system involves individual neurons, not a continuum. The number of terms in the sum is proportional to the number of neurons with receptive Želds overlapping the same spatial location. Examining how the number of terms affects the sum reveals how many neurons are needed to obtain a smooth D function. The results shown in Figure 3A were obtained with 30 steps in orientation h (in the range 0–2p ), 50 steps in preferred frequency k (in the range 0–90 radians per degree), and 50 steps in bandwidth b (in the range 0.1–3.0 radians per degree). With a much coarser grid of 8 orientations, 10 frequencies, and a single bandwidth (same ranges), the resulting curve was almost identical to the one in Figure 3A. In general, the distribution of bandwidths had little impact on the results. Hence, for a given point in visual space, sampling at 8 orientations and 10 frequencies per orientation is enough to construct a D function with a 0.06 degree resolution. Are the neurophysiological data consistent with these requirements? Within the dimensions of a hypercolumn, preferred orientation changes smoothly and uniformly throughout 180 degrees (Hubel & Wiesel, 1974; Bartfeld & Grinvald, 1992; Bonhoeffer & Grinvald, 1993). In fact, in orientation maps obtained with optical recordings, 8 orientation bins in the 0–180 degrees range can easily be distinguished by eye (Bartfeld & Grinvald, 1992; Bonhoeffer & Grinvald, 1993). Thus, sampling in orientation space is not a limitation. On the other hand, the 10 frequency steps do not correspond to 10 neurons, for two reasons. First, each complex Žlter represents a sine-cosine pair where each element is the difference between Cfj and ¡ fi , which means 4 neurons per Žlter. Second, r k ( k) is not uniform. To get at least 4 complex Žlters with optimal frequencies inside the 90 radians per degree bin, the 10 frequency bins must involve a total of about 300 Žlters, or 1200 neurons. This means that for each point in space, at least 1200 frequencies are needed in each of the orientation bins to achieve the resolution implied by the D function. De Valois et al. (1982) estimated that in monkeys, on the order of 100,000 neurons respond to stimuli at a given location in visual space. Assuming that half of those are simple cells and that frequencies are uniformly distributed in the 8 orientation bins (De Valois et al., 1982; Field & Tolhurst, 1986; Hubener, ¨ Shoham, Grinvald, & Bonhoeffer, 1997) leaves about 6000 frequencies per

Simple Cells and Tight Frames

327

orientation bin. This number, although only a rough Žgure, suggests that the sampling density in spatial frequency is sufŽcient to achieve the resolution implied by the D functions that we have computed. 4 Calculation of K( x , y)

To compute the function K from D, we need to include corre« the two-point ¬ lation function of ensembles of natural images, Q ´ I (x ) I ( y ) . This function has been characterized in the frequency domain (Field, 1987; Ruderman & Bialek, 1994; Dong & Atick, 1995), so we will use its Fourier transform QQ to compute the convolution 2.7 through which the K function is deŽned (Bracewell, 1965). Using a minimal set of assumptions, Dong & Atick (1995) derived an analytic expression for the full spatiotemporal power spectrum of natural images and Žt it to data from movies and video recordings. Taking k as the spatial frequency, in radians per degree, and v as the temporal frequency, in radians per second, they proposed that QQ ( k, v) =

A

Z

km¡1 w2

r2 v/ ck r1 v/ ck

dv vP( v),

(4.1)

where A is a constant that absorbs the amplitude factors, c = 360/ 2p , m has been measured from snapshot images to be about 2.3, r1 and r2 are, respectively, the minimum and maximum viewing distances at which objects are resolved, and P( v) is the probability distribution of velocities of objects that are seen. Dong & Atick (1995) suggested that P( v ) »

1 , ( v C v0 ) n

(4.2)

which Žt their two data sets extremely well using n = 3.7, v0 = 1.02 meters per second, r1 equal to 2 or 4 meters, and r2 either 23 or 40 meters (depending on the data set). The parameter v0 is related to the mean speed by v = v0 / ( n ¡ 2). We will use the above expression in our calculation of K, but to investigate whether the particular choice for P(v) is critical to our results, we also consider another distribution with a different shape: P(v) » v exp(¡av).

(4.3)

Because we are interested only in the spatial component of QQ (k, v), we need to average over v, or consider it as an additional parameter. Given typical ocular movements, a reasonable choice is to set v equal to the mean saccade frequency—about 4 per second or v = 8p radians per second. We will use this value and consider QQ as a function of k only.

328

Emilio Salinas and L. F. Abbott

Figure 5: K functions for monkey and cat simple cells. The curves shown correspond to numerical integration of equation 4.5 for different values of | x ¡ y |. (A) Results for monkey simple cells based on the D f it function of equation 3.13 and shown in Figure 3A. The thick line is the result when the following parameters were used in equations 4.1 and 4.2: n = 3.7, m = 2.3, r1 = 0.2 m, r2 = 100 m, v0 = 1.02 m/s, v = 8p rad/s. The full width at half height is 0.112 degree. The thin lines were obtained with the same parameters, except v = 32p (narrower curve; width at half-height is 0.105 degree) and v = 2p (wider curve; width at half-height is 0.135± ). (B) Results for cat simple cells also based on D f it in equation 3.13, but scaled by a factor of 4.27. The x-axis in this Žgure was scaled by the same amount. The width at half-height is 0.45 degree.

We Žrst compute the Fourier transform of K. This is equal to the product of the Fourier transforms of D and Q, since K is deŽned by a convolution (Bracewell, 1965; Press et al., 1992). We actually use the Fourier transform of D f it , which is simply the sum of the Fourier transforms of three gaussians. Q Transforming back to the spatial The Fourier transform of K is KQ = DQ f it Q. representation we have K( x , y ) =

1 4p 2

Z

¡ ¢ dk exp ik ¢ (x ¡ y ) DQ f it ( k) QQ ( k).

(4.4)

Writing k in polar coordinates and shifting the angle variable, we obtain K(| x ¡ y | ) =

1 4p 2

Z

¡ ¢ dkdh k exp ik| x ¡ y | cos(h ) DQ f it ( k) QQ ( k).

(4.5)

Q the above integral can be computed Given values for the parameters of Q, numerically for each value of | x ¡ y |. Figure 5 shows the results for the data collected in monkeys (see Figure 5A) and cats (see Figure 5B) integrating equation 4.1 using equation 4.2. For the monkey data, equation 3.13 was used for D f it , and for the cat data this same curve was scaled by a factor of

Simple Cells and Tight Frames

329

4.27. Comparison between Figures 5A and 5B shows that the K functions are approximately scaled versions of each other; the K function for cats is just slightly narrower than predicted by scaling. The thick curve in Figure 5A is the result for a standard set of parameters, whose values are listed in the caption of Figure 5. The full width at half-height of the curve is 0.112 degree, which is about twice the width of the original D function. Thus, averaging over images worsens the resolution. The thin curves in the same Žgure were obtained by multiplying the standard value of v by 1/4 (wider curve) and by 4 (narrower curve). This has the same effect as multiplying r1 and r2 or v0 by these same factors, because these parameters multiply or divide v. The thin curves are quite close to the thick one, which means that the parameters in equations 4.1 and 4.2 have relatively little importance in deŽning the shape and width of K. The precise functional form of P( v) did not seem to be crucial either. When equation 4.3 was used (parameter a was set so that the mean speed was the same as before), the results were Q which goes as indistinguishable from those in Figure 5. It is the tail of Q, m¡1 1/ f , that makes K broader than D and deŽnes its shape; the detailed form of QQ near k = 0, which is where the parameters have an inuence, does not seem to matter much. 5 Assessment of Tightness

We have compared the computed D and K kernels to a d function based only on the width of their central peaks, but the tightness of a set of basis functions can be more easily grasped by plotting the Fourier transform of its associated D function. In this way, frequency ranges in which DQ is approximately at can be promptly identiŽed. This is important, because the class of functions that are band-limited to such frequency ranges will be accurately represented by the fj basis, that is, the fj functions will behave as a tight frame for the class of functions whose Fourier transform is zero outside the identiŽed ranges. Flatness is quantiŽed by the ratio max( DQ )/ min( DQ ), which is commonly used as a rigorous measure of tightness (Lee, 1996). Figure 6 shows the Fourier transforms of several D and K functions. Because K is related to D through a convolution with the correlation of natural images (see equation 2.7), in all cases KQ is the product of DQ times the Q Here DQ reveals the Fourier transform of the natural image correlation, Q. tightness of the original set of functions, when no averaging over images takes place; in contrast, KQ reveals the tightness of the set when averaging over multiple images does take place (see section 6). Figures 6E and 6F show an ideal case in which the basis set provides perfect approximation of functions with frequency components below 30 cycles per degree; the associated DQ function is at in this range. This corresponds, for example, to a set of Q so Figure 6F wavelets (Daubechies, 1990, 1992). For this set KQ is equal to Q, simply shows the Fourier transform of the natural image correlation func-

330

Emilio Salinas and L. F. Abbott

Figure 6: Fourier transforms of different D (left) and K (right) functions. In all cases the plots on the right were obtained by multiplying the plots on the left Q (shown by the Fourier transform of the natural image correlation function, Q in F). The top row shows the results for monkey V1 foveal neurons. (A) Fourier transform of the D function shown in Figure 3A. (B) Fourier transform of the K function shown in Figure 5A. The middle row shows the results for retinal ganglion cells of the cat (see the appendix). (C) Fourier transform of the D function shown in Figure 4B. (D) Fourier transform of the K function associated with retinal ganglion cells of the cat. The bottom row shows an ideal case of a set of basis functions whose associated D function is exactly ad function. Thus, Q The Q Q and D Q curves were scaled to a maximum of the KQ function is equal to Q. one. Note that in these plots the units of spatial frequency are cycles per degree.

tion. This function is not at all at; although it looks fairly narrow in x space (not shown, but very similar to Figure 5A), its slowly decaying tail makes it a poor approximation to ad function. Figures 6A and 6B show the results for Q suggestmonkey V1 simple cells (foveal). The KQ function is very similar to Q, ing that the ensemble of V1 neurons is not compensating for the average over natural images. On the other hand, DQ is at up to about 1 cycle per degree, in-

Simple Cells and Tight Frames

331

dicating that simple cell receptive Želds provide an excellent approximation of a tight frame for functions with frequency cutoffs no higher than 1 cycle per degree. Even up to 10 cycles per degree, where simple cell responses start to attentuate signiŽcantly, DQ varies only by a factor between 4 and 5, not too far from the factor of 2 or less that corresponds to a standard deŽnition of a tight frame (Lee, 1996). As a comparison, Figures 6C and 6D show the results for retinal ganglion neurons from the cat (see the appendix). In this case KQ is Q albeit with a cutoff between 1 and 2 cycles per degree. much atter than D, 6 Discussion

We have evaluated the capacity of populations of simple cells in primary visual cortex to provide a basis for constructing more complicated receptive Želds through Hebbian synaptic weights. There are some similarities between the conditions we have studied, in particular, the suggestion that K is proportional to a d function, and the work of Atick and Redlich (1990, 1992) concerning ideas proposed by Barlow (1989) regarding redundancy in neuronal representations. However, there are substantial differences between our approach and theirs. Atick and Redlich (1990, 1992) imposed translational invariance so that all of the neurons they considered had the same receptive Želd structure. Their formulations resulted in consideration of functions such as K, including averages over natural images. Because all the receptive Želds were taken to be the same, this approach generated an optimality condition on a single receptive Želd. Atick and Redlich (1990, 1992) computed several optimal receptive Želd Žlters, among them those that solved the equation for K close to a d function. The case of function approximation that we consider involves cells with different receptive Želd structures and depends critically on the distribution of their parameters across the cortical population. Our D and K functions characterize the capacity of the ensemble of all simple cells for function approximation; they are not a property of single cells, as in the work of Atick and Redlich. It is interesting that both cases involve averages over natural images and some similar mathematical expressions, but the quantities being considered are very different: the capacity to transmit information in one case and the accuracy for function approximation in the other. From the DQ and KQ curves computed (see Figure 6), we conclude that the set of monkey V1 simple cells approximates a tight frame with a resolution, Q on the order of 1/10 of a degree. The resolution given by the cutoff in D, for cats, obtained by appropriate scaling of D, is about 0.43 degrees. In both cases the D function is more similar to the ideal d function than the K function, which includes corrections for natural image statistics. Thus, we Žnd no indication that this basis set is counteracting the effects of averaging over many images. In contrast, sets of Žlters like those of retinal ganglion neurons seem to be better suited for function approximation when the ef-

332

Emilio Salinas and L. F. Abbott

fect of image correlations is included. However, their resolution is rather poor—on the order of 1 degree. We should stress that these resolutions are meaningful only for synaptic modiŽcation processes based on correlational mechanisms. In particular, our results do not imply that simple cells have a higher overall resolution than retinal ganglion cells, an impossibility given that retinal ganglion cells provide the input to simple cells. Our results imply only that simple cells serve as a better basis from which neurons selective for arbitrary images can be constructed on the basis of linear superposition and Hebbian synaptic modiŽcation. To understand the difference between using the D and K functions to determine whether simple cell receptive Želds form a tight frame, we might imagine two forms of Hebbian learning. For the type of Hebbian learning we discussed in section 1, synaptic adaptation occurs slowly while an ensemble of images is displayed. In this case, the function K sets the resolution for function approximation with Hebbian synapses. Suppose instead that Hebbian modiŽcation of synapses occurs rapidly and only when an image equal to the target function appears. In this case the process is fast enough that images different from the target function do not affect the synaptic weights. For this learning procedure, the resolution for function approximation is set by D. Visual acuity in humans and monkeys, in image discrimination or recognition tasks that involve the activities of large numbers of neurons, is about 1 minute of arc, or 0.017 degree (Westheimer, 1980). This is about six times better than the resolution we have estimated for approximation of an arbitrary function based on D. It might seem odd that our estimation is considerably below psychophysical discrimination thresholds, but our analysis applies speciŽcally to single neurons downstream from V1 that combine their inputs in a perfectly linear fashion. The discrepancy indicates that the visual system does not rely exclusively on the tightness of the V1 neuronal ensemble to achieve the acuity exhibited at the behavioral level; additional mechanisms beyond linear summation are required. There are two prominent possibilities for this. First, the neural circuitry may be able to exploit some of its nonlinearities to increase accuracy. On the other hand, higher accuracy may also be achieved through a population of neurons. Considering the ubiquity of population codes throughout the cortex, the capacity of V1 simple cells to act as a basis for Hebbian-based function approximation by single postsynaptic neurons may appear substantial—only a factor of 6 below behavioral performance. However, to try to answer the question posed in the title, the frame formed by simple cells in primary visual cortex is tighter than would be expected from a random distribution of Gabor Žlters, but not tight enough to account for behavioral acuity. Appendix

Here we compute the D function for retinal ganglion cells (X cells). The Žring of these cells is well characterized through a linear Žltering operation (af-

Simple Cells and Tight Frames

333

terward rectifying the result to eliminate negative rates), as in equation 2.1. In this case each Žlter is equal to the difference of two gaussians, ¡ ¢2 ! ¡ ¢2 ! x ¡ aj x ¡ aj A1 A2 exp ¡ ¡ exp ¡ . (A.1) fj ( x) = 2p s12 2s12 2p s22 2s22

(

(

We assume that the only parameter that varies across neurons is a, the receptive Želd location. Values for the rest of the parameters are as follows: A1 / A2 = 17/ 16 (Linsenmeier, Frishman, Jakiela, & Enroth-Cugell, 1982), s1 = 0.17666 ± and s2 = 0.53± (Peichl & W¨asle, 1979). The D function is computed as in equation 2.8, using the above Žlters. The sum over neurons is approximated by an integral over a assuming that the receptive Želds are uniformly distributed, so r ( a ) is a constant. The integrals can then be computed analytically, and the result is ¡ ¢2 ! ¡ ¢2 ! x¡y x¡y A21 A22 (| ) exp ¡ C exp ¡ D x ¡ y| = 4p s12 4s12 4p s22 4s22 ¡ ¢2 ! x¡y A1 A2 ¡ exp ¡ . (A.2) 4p (s12 C s22 ) 2(s12 C s22 )

(

(

(

The Fourier transform of this expression is DQ ( k) =

(

(

s 2 k2 A1 exp ¡ 1 2

!

(

s 2 k2 ¡ A2 exp ¡ 2 2

!!2 ,

(A.3)

where k is in radians per degree. Acknowledgments

We thank Sacha Nelson for his helpful advice. This research was supported by the Sloan Center for Theoretical Neurobiology at Brandeis University, the National Science Foundation (DMS-9503261), and the W. M. Keck Foundation. References Atick, J. J., & Redlich, A. N. (1990). Towards a theory of early visual processing. Neural Comput., 2, 308–320. Atick, J. J., & Redlich, A. N. (1992). What does the retina know about natural scenes? Neural Comput., 4, 196–210. Barlow, H. B. (1989). Unsupervised learning. Neural Comput., 1, 295–311. Bartfeld, E., & Grinvald, A. (1992). Relationships between orientation preference pinwheels, cytochrome oxidase blobs and ocular dominance columns in primate striate cortex. Proc. Natl. Acad. Sci. USA, 89, 11905–11909.

334

Emilio Salinas and L. F. Abbott

Bonhoeffer, T., & Grinvald, A. (1993). The layout of iso-orientation domains in area 18 of cat visual cortex: Optical imaging reveals a pinwheel-like organization. J. Neurosci., 13, 4157–4180. Bracewell, R. (1965). The Fourier transform and its applications. New York: McGraw-Hill. Daubechies, I. (1990). The wavelet transform, time-frequency localization and signal analysis. IEEE Trans. Information Theory, 36, 961–1004. Daubechies, I. (1992). Ten lectures on wavelets. Philadelphia: SIAM. Daubechies, I., Grossmann, A., & Meyer, Y. (1986). Painless nonorthogonal expansions. J. Math. Phys., 27, 1271–1283. Daugman, J. G. (1985). Uncertainty relation for resolution in space, spatial frequency, and orientation optimization by two-dimensional visual cortical Žlters. J. Opt. Soc. Am., 2, 1160–1169. De Valois, R. L., Albrecht, D. G., & Thorell, L. G. (1982). Spatial frequency selectivity of cells in macaque visual cortex. Vision Res., 22, 545–559. Dong, D. W., & Atick, J. J. (1995). Statistics of natural time-varying images. Network, 6, 345–358. Field, D. (1987). Relations between the statistics of natural images and the response properties of cortical cells. J. Opt. Soc. Am. A, 4, 2379–2394. Field, D., & Tolhurst, D. J. (1986). The structure and symmetry of simple-cell receptive-Želd proŽles in the cat’s visual cortex. Proc. R. Soc. Lond. B, 228, 379–400. Girosi, F., Jones, M., & Poggio, T. (1995). Regularization theory and neural networks. Neural Comput., 7, 219–269. Hebb, D. O. (1949). The organization of behavior: A neuropsychological theory. New York: Wiley. Hubel, D. H., & Wiesel, T. N. (1974). Sequence regularity and geometry of orientation columns in the monkey striate cortex. J. Comp. Neurol., 158, 267–294. Hubener, ¨ M., Shoham, D., Grinvald, A., & Bonhoeffer, T. (1997). Spatial relationships among three columnar systems in cat area 17. J. Neurosci., 17, 9270–9284. Jones, J., & Palmer, L. (1987). An evaluation of the two-dimensional Gabor Žlter model of simple receptive Želds in cat striate cortex. J. Neurophysiol.,58, 1233– 1259. Lee, T. S. (1996). Image representation using 2D Gabor wavelets. IEEE Trans. Pattern Analysis and Machine Intelligence, 18, 959–971. Linsenmeier, R. A., Frishman, L. J., Jakiela, H. G., & Enroth-Cugell, C. (1982). Receptive Želd properties of X and Y cells in the cat retina derived from contrast sensitivity measurements. Vision Res., 22, 1173–1183. Movshon, J. A., Thompson, I. D., & Tolhurst, D. J. (1978). Spatial and temporal contrast sensitivity of neurons in area 17 and 18 of the cat’s visual cortex. J. Physiol., 283, 101–120. Peichl, L., & Wa¨ sle, H. (1979).Size, scatter and coverage of ganglion cell receptive Želd centers in the cat retina. J. Physiol. (Lond.), 291, 117–141. Poggio, T. (1990). A theory of how the brain might work. Cold Spring Harbor Symposium on Quantitative Biology, 55, 899–910. Poggio, T., & Girosi, F. (1990). Regularization algorithms for learning that are equivalent to multilayer networks. Science, 247, 978–982.

Simple Cells and Tight Frames

335

Pollen, D. A., & Ronner, S. F. (1981).Phase relationships between adjacent simple cells in the visual cortex. Science, 212, 1409–1411. Press, W. H., Flannery, B. P., Teukolsky, S. A., & Vetterling, W. T. (1992).Numerical recipes in C. New York: Cambridge University Press. Ruderman, D. L., & Bialek, W. (1994). Statistics of natural images: Scaling in the woods. Phys. Rev. Let., 73, 814–817. Webster, M. A., & De Valois, R. L. (1985).Relationship between spatial-frequency and orientation tuning of striate-cortex cells. J. Opt. Soc. Am. A, 2, 1124–1132. Westheimer, G. (1980). The eye, including central nervous system control of eye movements. In V. B. Mountcastle (Ed.), Medical physiology (Vol. 1, pp. 481– 503). St. Louis: Mosby. Received June 11, 1998; accepted February 4, 1999.

LETTER

Communicated by Joachim Buhmann

Learning Overcomplete Representations Michael S. Lewicki Computer Science Dept. and Center for the Neural Basis of Cognition, Carnegie Mellon Univ., 115 Mellon Inst., 4400 Fifth Ave., Pittsburgh, PA 15213 Terrence J. Sejnowski Howard Hughes Medical Institute, Computational Neurobiology Laboratory, The Salk Institute, La Jolla, CA 92037, U.S.A. In an overcomplete basis, the number of basis vectors is greater than the dimensionality of the input, and the representation of an input is not a unique combination of basis vectors. Overcomplete representations have been advocated because they have greater robustness in the presence of noise, can be sparser, and can have greater exibility in matching structure in the data. Overcomplete codes have also been proposed as a model of some of the response properties of neurons in primary visual cortex. Previous work has focused on Žnding the best representation of a signal using a Žxed overcomplete basis (or dictionary). We present an algorithm for learning an overcomplete basis by viewing it as probabilistic model of the observed data. We show that overcomplete bases can yield a better approximation of the underlying statistical distribution of the data and can thus lead to greater coding efŽciency. This can be viewed as a generalization of the technique of independent component analysis and provides a method for Bayesian reconstruction of signals in the presence of noise and for blind source separation when there are more sources than mixtures. 1 Introduction

A common way to represent real-valued signals is with a linear superposition of basis functions. This can be an efŽcient way to encode a highdimensional data space because the representation is distributed. Bases such as the Fourier or wavelet can provide a useful representation of some signals, but they are limited because they are not specialized for the signals under consideration. An alternative and potentially more general method of signal representation uses so-called overcomplete bases (also called overcomplete dictionaries), which allow a greater number of basis functions (also called dictionary elements) than samples in the input signal (Simoncelli, Freeman, Adelson, & Heeger, 1992; Mallat & Zhang, 1993; Chen, Donoho, & Saunders, 1996). Overcomplete bases are typically constructed by merging a set of complete Neural Computation 12, 337–365 (2000)

c 2000 Massachusetts Institute of Technology °

338

Michael S. Lewicki and Terrence J. Sejnowski

bases (e.g., Fourier, wavelet, and Gabor), or by adding basis functions to a complete basis (e.g., adding frequencies to a Fourier basis). Under an overcomplete basis, the decomposition of a signal is not unique, but this can offer some advantages. One is that there is greater exibility in capturing structure in the data. Instead of a small set of general basis functions, there is a larger set of more specialized basis functions such that relatively few are required to represent any particular signal. These can form more compact representations, because each basis function can describe a signiŽcant amount of structure in the data. For example, if a signal is largely sinusoidal, it will have a compact representation in a Fourier basis. Similarly, a signal composed of chirps is naturally represented in a chirp basis. Combining both of these bases into a single overcomplete basis would allow compact representations for both types of signals (Coifman & Wickerhauser, 1992; Mallat & Zhang, 1993; Chen et al., 1996). It is also possible to obtain compact representations when the overcomplete basis contains a single class of basis functions. An overcomplete Fourier basis, with more than the minimum number of sinusoids, can compactly represent signals composed of small numbers of frequencies, achieving superresolution (Chen et al., 1996). An additional advantage of some overcomplete representations is increased stability of the representation in response to small perturbations of the signal (Simoncelli et al., 1992). Unlike the case in a complete basis, where signal decomposition is well deŽned and unique, Žnding the “best” representation in terms of an overcomplete basis is a challenging problem. It requires both an objective for decomposition and an algorithm that can achieve that objective. Decomposition can be expressed as Žnding a solution to x = As ,

(1.1)

where x is the signal, A is a (nonsquare) matrix of basis functions (vectors), and s is the vector of coefŽcients, that is, the representation of the signal. Developing efŽcient algorithms to solve this equation is an active area of research. One approach to removing the degeneracy in equation 1.1 is to place a constraint on s (Daubechies, 1990; Chen et al., 1996), for example, by Žnding s satisfying equation 1.1 with minimum L1 norm. A different approach is to construct iteratively a sparse representation of the signal (Coifman & Wickerhauser, 1992; Mallat & Zhang, 1993). In some of these approaches, the decomposition can be a nonlinear function of the data. Although overcomplete bases can be more exible in terms of how the signal is represented, there is no guarantee that hand-selected basis vectors will be well matched to the structure in the data. Ideally, we would like the basis itself to be adapted to the data, so that for signal class of interest, each basis function captures a maximal amount of structure. One recent success along these lines was developed by Olshausen and Field (1996, 1997) from the viewpoint of learning sparse codes. When adapted

Learning Overcomplete Representations

339

to natural images, the basis functions shared many properties with neurons in primary visual cortex, suggesting that overcomplete representations might a useful model for neural population codes. In this article, we present an algorithm for learning an overcomplete basis by viewing it as a probabilistic model of the observed data. This approach provides a natural solution to decomposition (see equation 1.1) by Žnding the maximum a posteriori representation of the data. The prior distribution on the basis function coefŽcients removes the redundancy in the representation and leads to representations that are sparse and are a nonlinear function of the data. The probabilistic approach to decomposition also leads to a natural method of denoising. From this model, we derive a simple and robust learning algorithm by maximizing the data likelihood over the basis functions. This work generalizes the algorithm of Olshausen and Field (1996) by deriving a algorithm for learning overcomplete bases from a direct approximation to the data likelihood. This allows learning for arbitrary input noise levels, allows for the objective comparison of different models, and provides a way to estimate a model’s coding efŽciency. We also show that overcomplete representations can provide a better and more efŽcient representation, because they can better approximate the underlying statistical density of the input data. This also generalizes the technique of independent component analysis (Jutten & H´erault, 1991; Comon, 1994; Bell & Sejnowski, 1995) and provides a method for the identiŽcation of more sources than mixtures. 2 Model

We assume that each data vector, x = x1 , . . . , xL , can be described with an overcomplete linear basis plus additive noise, x = As C ²,

(2.1)

where A is an L£ M matrix with M > L. We assume gaussian additive noise, ². The data likelihood is log P( x | A , s ) / ¡

1 (x ¡ As )2 , 2s 2

(2.2)

where s 2 is the noise variance. One criticism of overcomplete representations is that they are redundant— a given data point can have many possible representations—but this redundancy can be removed by a proper choice for the prior probability of the basis coefŽcients, P( s), which speciŽes the probability of the alternative representations. This density determines how the underlying statistical structure is modeled and the nature of the representation. Standard approaches to signal representation do not specify a prior for the coefŽcients, because

340

Michael S. Lewicki and Terrence J. Sejnowski

for most complete bases and assuming zero noise, the representation of the signal is unique. If A is invertible, the decomposition of the signal x is given by s = A ¡1 x . Because A ¡1 is expensive to compute, there is strong incentive to Žnd basis matrices that are easy to invert, such as restricting the basis functions to be orthogonal or further restricting the basis functions to those for which there are fast algorithms, such as Fourier or wavelet analysis. A more general approach is afforded in the probabilistic formulation. The parameter values of the internal states are found by maximizing the posterior distribution of s , sˆ = argmax P( s| x, A ) = argmax P(x | A , s) P( s), s

s

(2.3)

where sˆ is the most probable decomposition of the signal. This formulation of the problem offers the advantage that the model can Žt more general types of distributions. Q For simplicity, we assume independence of the coefŽcients: P( s) = m P( sm ). Another advantage of this formulation is that the process of Žnding the most probable representation determines the best representation for the noise level deŽned by s. This automatically performs “denoising” in that s encodes the underlying signal without noise. The noise level can be set to arbitrary levels, including zero. It is also possible to use Bayesian methods to infer the most probable noise level, but this will not be pursued here. 2.1 The Relation Between the Prior and the Representation. Figure 1 shows how different priors induce different representations of the data. A standard choice might be a gaussian prior (equivalent to factor analysis), but this yields no advantage to having an overcomplete representation because the underlying assumption is still that the data are gaussian. An alternative choice, advocated by some authors (Field, 1994; Olshausen & Field, 1996; Chen et al., 1996), is to use priors that assume sparse representations. This is accomplished by priors that have high kurtosis, such as the Laplacian, P( sm ) / exp(¡h |sm | ). Compared to a gaussian, this distribution puts greater weight on values close to zero, and as result the representations are sparser— they have a greater number of small-valued coefŽcients. For overcomplete bases, we expect a priori that the representation will be sparse, because only L out of M nonzero coefŽcients are needed to represent arbitrary Ldimensional input patterns. In the case of zero noise and P( s) gaussian, maximizing equation 2.3 is equivalent to mins || s || 2 subject to x = As . The solution can be obtained with the pseudoinverse, s = ACx , and is a linear function of A and s. In the case of zero noise and P(s ) Laplacian, maximizing equation 2.3 is equivalent to mins || s || 1 subject to x = As. Unlike the gaussian prior, this solution cannot be obtained by a simple linear operation.

Learning Overcomplete Representations

341

Figure 1: Different priors induce different representations. (a) The data distribution is two-dimensional and has three main arms. The overlayed axes form an overcomplete representation. (b, c) Optimal scaled basis vectors for the data point x under a gaussian and Laplacian prior, respectively. Assuming low noise, a gaussian for P( s) is equivalent to Žnding s with minimum L2 norm such that x = As . The solution is given by the pseudoinverse s = ACx and is a linear function of the data. A Laplacian prior (P(sm ) / exp[¡h |sm |]) Žnds s with minimum L1 norm. This is a nonlinear operation that essentially selects a subset of basis vectors to represent the data (Chen et al., 1996), so that the resulting representation is sparse. (d) A 64-sample segment of natural speech was Žt to a 2£ overcomplete Fourier representation (128 basis functions) (see section 7). The plot shows rank-order distribution of the coefŽcients of s under a gaussian prior (dashed) and a Laplacian prior (solid). Under a gaussian prior, nearly all of the coefŽcients in the solution are nonzero, whereas far fewer are nonzero under a Laplacian prior.

3 Inferring the Internal State

A general approach for optimizing s in the case of Žnite noise (² > 0) and nongaussian P(s ) is to use the gradient of the log posterior in an optimization algorithm (Daugman, 1988; Olshausen & Field, 1996). A suitable initial condition is s = A T x or s = ACx .

342

Michael S. Lewicki and Terrence J. Sejnowski

An alternative method, which can be used when the prior is Laplacian and ² = 0, is to view the problem as a linear program (Chen et al., 1996): min c T | s|

subject to

As = x .

(3.1)

Letting c = (1, . . . , 1), the objective function in the linear program, c T | s| = P m | sm |, corresponds to maximizing the log posterior likelihood under a Laplacian prior. This can be converted to a standard linear program (with only positive coefŽcients) by separating positive and negative coefŽcients. Making the substitutions, s à [u I v ], c à [1 I 1 ], and A à [A I ¡A ], equation 3.1 becomes min 1 T [u I v ]

subject to

[A I ¡A ][u I v ] = x ,

u , v ¸ 0,

(3.2)

which replaces the basis vector matrix A with one that contains both positive and negative copies of the vectors. This separates the positive and negative coefŽcients of the solution s into the positive variables u and v , respectively. This can be solved efŽciently and exactly with interior point linear programming methods (Chen et al., 1996). Quadratic programming approaches to this type of problem have also recently been suggested (Osuna, Freund, & Girosi, 1997) for similar problems. We have used both the linear programming and gradient-based methods. The linear programming methods were superior for Žnding exact solutions in the case of zero noise. The standard implementation handles only the noiseless case but can be generalized (Chen et al., 1996). We found gradientbased methods to be faster in obtaining good approximate solutions. They also have the advantage that they can easily be adapted for more general models, such as positive noise levels or different priors. 4 Learning

To derive a learning algorithm we must Žrst specify an appropriate objective function. A natural objective is to maximize the probability of the data given the model. For a set of K independent data vectors X = x 1 , . . . , x K , P(X | A) =

K Y

P ( xk | A ).

(4.1)

k= 1

The probability of a single data point is computed by marginalizing over the states of the network, Z (4.2) P (x k | A ) = ds P( s) P( x k | A , s ). This formulates the problem as one of density estimation and is equivalent to minimizing the Kullback-Leibler divergence between the model density

Learning Overcomplete Representations

343

and the distribution of the data. If implementation-related issues such as synaptic noise are ignored, this is equivalent to the methods of redundancy reduction and maximizing the mutual information between the input and the representation (Nadal & Parga, 1994a, 1994b; Cardoso, 1997), which have been advocated by several researchers (Barlow, 1961, 1989; Hinton and Sejnowski, 1986; Daugman, 1989; Linsker, 1988; Atick, 1992). 4.1 Fitting Bases to the Data Distribution. To understand how overcomplete representations can yield better approximations to the underlying density, it is helpful to contrast different techniques for adapting a basis to a particular data set. Principal component analysis (PCA) models the data with a multivariate gaussian. This representation is commonly used to Žnd the directions in the data with largest variation—the principal components—even if the data do not have gaussian structure. The basis functions are the eigenvectors of the covariance matrix and are restricted to be orthogonal. An extension of PCA, called independent component analysis (ICA) (Jutten & H´erault, 1991; Comon, 1994; Bell & Sejnowski, 1995), allows the learning of nonorthogonal bases for data with nongaussian distributions. ICA is highly effective in several applications such as blind source separation of mixed audio signals (Jutten & H´erault, 1991; Bell & Sejnowski, 1995), decomposition of electroencephalographic (EEG) signals (Makeig, Jung, Bell, Ghahremani, & Sejnowski, 1996), and the analysis of functional magnetic resonance imaging (fMRI) data (McKeown et al., 1998). In all of these techniques, the number of basis vectors is equal to the number of inputs. Because these bases span the input space, they are complete and are sufŽcient to represent the data, but we will see that this representation can be limited. Figure 2 illustrates how different data densities are modeled by various approaches in a simple two-dimensional data space. PCA assumes gaussian structure, but if the data have nongaussian structure, the vectors can point in directions that contain very few data, and the probability density deŽned by the model will predict data where none occur. This means that the model will underestimate the likelihood of data in the dense regions and overestimate it in the sparse regions. By Shannon’s theorem, this will limit the efŽciency of the representation. ICA assumes the coefŽcients have nongaussian structure and allows the vectors to be nonorthogonal. In this example, the basis vectors point along high-density regions of the data space. If the density is more complicated, however, as in the case of the three-armed density, neither PCA nor ICA captures the underlying structure. Although it is possible to represent every point in the two-dimensional space with a linear combination of two vectors, the underlying density cannot be described without specifying a complicated prior on the basis function coefŽcients. An overcomplete basis, however, allows an efŽcient representation of the underlying density with simple priors on the basis function coefŽcients.

344

Michael S. Lewicki and Terrence J. Sejnowski

a

b

c

d

Figure 2: Fitting two-dimensional data with different bases. (a) PCA makes the assumption that the data have a gaussian distribution. The optimal basis vectors are orthogonal and are not efŽcient at representing nonorthogonal density distributions. (b, c) ICA does not require that the vectors be orthogonal and can Žt more general types of densities. (c) Some data distributions, however, cannot be modeled adequately by either PCA or ICA. (d) Allowing the basis to be overcomplete allows the three-armed distribution to be Žt with simple and independent distributions for the basis vector coefŽcients.

4.2 Approximating the Data Probability. In deriving the learning algorithm, the Žrst problem is that, in general, the integral in equation 4.2 is intractable. In the special case of zero noise and a complete representation (i.e., A is invertible) this integral can be solved and leads to the well-known ICA algorithm (MacKay, 1996; Pearlmutter & Parra, 1997; Olshausen & Field, 1997; Cardoso, 1997). But in the case of an overcomplete basis, such a solution is not possible. Some recent approaches have tried to approximate this integral by evaluating P( s )P( x | A , s ) at its maximum (Olshausen & Field, 1996, 1997), but this ignores the volume information of the posterior. This means representations that are well determined, i.e., have a sharp posterior distribution) are treated in the same way as those that are ill determined, which can introduce biases into the learning. Another problem is that this leads to a trivial solution where the magnitude of the basis functions diverge. This problem can be somewhat circumvented by adaptive normalization (Olshausen & Field, 1996), but setting the adaptation rates can be tricky in practice, and, more important,

Learning Overcomplete Representations

345

there is no guarantee the desired objective function is the one being optimized. The approach we take here is to approximate equation 4.2 with a gaussian around the posterior mode, sˆ . This yields log P( x | A ) ¼

l l L M log C log(2p ) C log P(sˆ ) ¡ ( x ¡ A sˆ ) 2 2 2p 2 2 1 ¡ log det H , 2

(4.3)

where l = 1/ s 2 , and H is the Hessian of the log posterior at sˆ , H = l ATA ¡ rs rs log P( sˆ ). This is a saddle-point approximation. (See appendix for derivation details.) Care must be taken that the log prior has some curvature, because det( ATA ) = 0 arising from the fact that ATA has the same rank as A , which is assumed to be rectangular. The accuracy of the saddlepoint approximation is determined by how closely the posterior mode can be approximated by a gaussian. The approximation will be poor if there are multiple modes or there is excessive skew or kurtosis. We present methods for addressing some of these issues in section 6.1. A learning rule is obtained by differentiating log P( x| A ) with respect to A (see the appendix), which leads to the following expression: D A = AAT rA log P( x| A ) ¼ ¡A ( zˆsT C I ),

(4.4)

where zk = @log P( sk )/ @sk . This rule contains no matrix inverses, and the vector z involves only the derivative of the log prior. In the case where A is square, this form of the rule is exactly the natural gradient ICA learning rule for the basis matrix (Amari, Cichocki, & Yang, 1996). The difference in the more general case where A is rectangular is in how the coefŽcients s are calculated. In the standard ICA learning algorithm (A square, zero noise), the coefŽcients are given by s = Wx , where W = A ¡1 is the Žlter matrix. For the learning rule to work the optimization of s must maximize the posterior distribution P( s| x, A ) (see equation 2.3). 5 Examples

We now demonstrate the algorithm on two simple data sets. Figure 3 shows the results of Žtting two different two-dimensional data sets. The learning procedure was as follows. The initial bases were random normalized vectors with the constraint that no two vectors be closer in angle than 30 degrees. Each basis was adapted to the data using the learning rule given in equation 4.4. The bases were adapted with 50 iterations of simple gradient descent with a batch size of 500 and a step size of 0.1 for the Žrst 30 iterations, which was reduced to 0.001 over the last 20. The most probable coefŽcients were obtained using a publicly available interior point

346

Michael S. Lewicki and Terrence J. Sejnowski a

b

c

Figure 3: Three examples illustrating the Žtting of 2D distributions with overcomplete bases. The gray vectors show the true basis vectors used to generate the distribution. The black vectors show the learned basis vectors. For clarity, the vectors have been rescaled. The learned basis vectors can be antiparallel to the true vectors, because both positive and negative coefŽcients are allowed.

linear programming package (Meszaros, 1997). Convergence of the learning algorithm was rapid, usually reaching the solution in fewer than 30 iterations. A solution was discarded if the magnitude of one of the basis vector dropped to zero. This happened occasionally when one vector was “trapped” between two others that were already representing the data in that region. The Žrst example is shown in Figure 3a. The data were generated from the true basis vectors (shown in gray) using x = As. To illustrate better the arms of the distribution, the elements of s were drawn from an exponential distribution (i.e., only positive coefŽcients) with unit mean. The direction of the learned vectors always matched that of the true generating vectors. The magnitude was smaller than the true basis vectors, possibly due to the approximation used for P(x | A ) (see equation 4.3). Identical results were obtained when the coefŽcients were generated from a Laplacian prior (i.e., both positive and negative coefŽcients). The second example (see Figure 3b) has four arms, again generated from the true basis vectors using a Laplacian (both positive and negative coefŽcients) with unit variance. The learned vectors were close to the true vectors but showed a consistent bias. This might occur because the data are too dense for there to be any distinct direction or from the approximation to P( x| A ). The bias can be greatly reduced if the coefŽcients are drawn from a distribution with greater sparsity. The example in Figure 3c uses a data set from the same underlying vectors, but generated from a generalized Laplacian distribution, P( sm ) / exp(¡|sm | p ). When this distribution is Žtted to wavelet subband coefŽcients of images, Buccigrossi and Simoncelli (1997) found that p was in the range [0.5, 1.0]. Varying this exponent varies the kurtosis of the distribution, which is one measure of a sparseness. In

Learning Overcomplete Representations

347

Figure 3c, the data were generated with p = 0.6. The arms of this data set are more distinct, and the directions match those of the generating vectors. 6 Quantifying the EfŽciency of the Representation

One advantage of the probabilistic framework is that it provides a natural means for comparing objectively alternative representations and models. In this section we compare two methods for comparing the coding efŽciency of the learned basis functions. 6.1 Estimating Coding Cost Using P( x| A ) . The objective function P( x | A ) for the probability of the data under the model is a natural measure for comparing different models. It is helpful to convert this value into a more intuitive form. According to Shannon’s coding theorem, the probability of a code word gives the lower bound on the length of that code word, under the assumption that the model is correct. The number of bits required to encode the pattern is given by

#bits ¸ ¡ log2 P( x | A ) ¡ L log2 (sx ),

(6.1)

where L is the dimensionality of the input pattern x, and sx speciŽes the quantization level of the encoding. 6.1.1 A More Accurate Approximation to P( x| A ) . The learning algorithm has been derived by making a gaussian approximation around the posterior maximum sˆ . In practice this is sufŽciently accurate to generate a useful gradient for learning overcomplete bases. One caution, however, is that there is not necessarily a relation between the local curvature and the volume for general priors, as for a gaussian prior. When comparing alternative models or basis functions, we may want a more accurate approximation. One approach that works well is a method based on second differences. Rather than rely on the analytic Hessian to estimate the volume around sˆ , the volume can be estimated directly by calculating the change in the posterior, P(s ) P(x | s , A ) around the maximum sˆ using second differences. Computing the entire Hessian in this manner would require O( M2 ) evaluations of the posterior, which itself is an O( M2 ) operation. We can reduce the number of posterior evaluations to O( M) if we estimate the curvature only in the direction of the eigenvectors of the Hessian. To obtain an accurate volume estimate, the volume of gaussian approximation should match as closely as possible to the volume around the posterior maximum. This requires a choice of the step size along the eigenvector direction. In principle, this Žt could be optimized, for example, by minimizing the cross entropy, but at signiŽcant computational expense. A relatively quick and reliable estimate can be obtained by choosing the step length, gi , which produces a

348

Michael S. Lewicki and Terrence J. Sejnowski

Žxed drop, D , in the value of the posterior log-likelihood, £ ¤ g i = argmingi log P( sˆ | x, A ) ¡ log P( sˆ Cgi ei | x, A ) ¡ D ,

(6.2)

where gi is the step length along the direction of eigenvector e i . This method of choosing g i requires a line search, but this can typically be done with three or four function evaluations. In the examples below, we used D = 3.8, the optimal value for approximating a Laplacian with a gaussian. This O , which is a more accurate estimate procedure computes a new Hessian, H of the posterior curvature. The estimated Hessian is then substituted into (equation 4.3) to obtain a more accurate estimate of log P( x| A ). Figure 4 shows cross-sections of the true posterior and the gaussian approximation using the direct Hessian and the second-differences method. The direct Hessian method produced poor approximations in directions where the curvature was sharp around the sˆ . Estimating the volume with second differences produced a much better estimate along all of the eigenvector directions. There is no guarantee, however, that this is the best estimate, because there could still exist some directions that are poorly approximated. For example, the Žrst plots in Figures 4a and 4b (labeled s ¡ smp) show the direction between sˆ using a convergence tolerance of 10¡4 and sˆ using a convergence tolerance of 10¡6 . This was the direction of the gradient when optimization was terminated and is likely to be in the direction of coefŽcients whose values are poorly determined. The maximum has not been reached, and the volume in this direction is underestimated. Overall, however, the second-differences method produces a much more accurate estimate of log P (x | A ) than that using the analytic Hessian and obtains more accurate estimates of the relative coding efŽciencies of different models. 6.1.2 Application to Test Data. We now quantify the efŽciency of different representations for various test data sets using the methods described above. The estimated coding cost was calculated from 10 sets of 1000 randomly sampled data points using an encoding precision sx = 0.01. The relative encoding efŽciency of the different representations depends on the relative difference of their predictions—how closely P( x | A ) matches that of the underlying distribution. If there is little difference between the predictive distribution, then there will be little difference in the expected coding cost. Table 1 shows the estimated coding costs of various models for the data sets used above. The uniform distribution gives an upper bound on the coding cost by assuming equal probability for all points within the range spanned by the data points. A simple bivariate gaussian model of the data points, equivalent to a code based on the principal components of the data, does signiŽcantly better. The remaining table entries assume the model deŽned in equation 2.1 with a Laplacian prior on the basis coefŽcients. The 2 £ 2 basis matrix is

a

349

P(s|x,A)

Learning Overcomplete Representations

­ 5 ­ 4 ­ 3 ­ 0.5 s ­ smp

0 e128

0.5 ­ 0.4 ­ 0.2 0 e112

0.2

0

0.5 1 e96

1.5 ­ 3

­ 2.5 ­ 2 e80

­ 1.5

P(s|x,A)

­6

b

e64

­ 1 ­ 1.2

­1 e48

­ 0.8 ­ 0.1

0 e32

0.1 ­ 1.15

e16

­ 1 ­ 0.02 0

0.02 0.04 e1

P(s|x,A)

­ 1.05

­ 5 ­ 4 ­ 3 ­ 0.5 s ­ smp

0 e128

0.5 ­ 0.4 ­ 0.2 0 e112

0.2

0

0.5 1 e96

1.5 ­ 3

­ 2.5 ­ 2 e80

­ 1.5

P(s|x,A)

­6

­ 1.05

e64

­ 1 ­ 1.2

­1 e48

­ 0.8 ­ 0.1

0 e32

0.1 ­ 1.15

e16

­ 1 ­ 0.02 0

0.02 0.04 e1

Figure 4: The solid lines in the Žgure show the cross-sections of a 128dimensional posterior distribution (P( s| x , A ) along the directions of the eigenvectors of the Hessian matrix evaluated at sˆ (see equation 2.3). This posterior resulted from Žtting a 2£-overcomplete representation to a 64-sample segment of natural speech (see section 7). The Žrst cross-section is in the direction between sˆ using a tolerance of 10¡4 and a more probable sˆ found using a tolerance of 10¡6 (labeled s ¡ smp). The remaining cross-sections are in order of the eigenvalues, showing a sample from smallest (e128) to largest (e1). The dots show the gaussian approximation for the same cross-section. The C indicates the position of sˆ . Note that the x-axes have different scales. (a) The gaussian approximation obtained using the analytic Hessian to estimate the curvature. (b) The same cross-sections but with the gaussian approximation calculated using the seconddifference method described in the text.

equivalent to the ICA solution (under the assumption of a Laplacian prior). The 2£3 and 2£4 matrices are overcomplete representations. All bases were learned with the methods discussed above. The most probable coefŽcients were calculated using the linear programming method as was done during learning. Table 1 shows that the coding efŽciency estimates obtained using the approximation to P( x | A ) are reasonably accurate compared to the estimated

350

Michael S. Lewicki and Terrence J. Sejnowski

Table 1: Estimated Coding Costs using P( x | A ). Bits per Pattern Model uniform Gaussian A 2£2 (ICA) A 2£3 A 2£4

Figure 1a 22.31 § 18.96 § 18.84 § 18.59 § —

0.17 0.06 0.04 0.03

Figure 3a 21.65 § 17.98 § 17.79 § 16.87 § —

0.19 0.07 0.07 0.06

Figure 3b 21.57 § 18.55 § 18.38 § 18.04 § 18.15 §

0.16 0.06 0.05 0.04 0.08

Figure 3c 26.67 § 22.12 § 21.46 § 21.12 § 21.24 §

0.31 0.15 0.08 0.08 0.08

bits per pattern for the uniform and gaussian models for which the expected coding costs are exact. Apart from the uniform model, the differences in the predicted densities are rather subtle and do not show any large differences in coding costs. This would be expected because the densities predicted by the various models are similar. For all the data sets, using a three-vector basis (A 2£3 ) gives greater expected coding efŽciencies, which indicates that higher degrees of overcompleteness can yield more efŽcient coding. For the four-vector basis (A 2£4 ), there is no further improvement in coding efŽciency. This reects the fact that when the prior distribution is limited to a Laplacian, there is an inherent limitation on the model’s degrees of freedom; increasing the number of basis functions does not always increase the space of distributions that can be modeled. 6.2 Estimating Coding Cost Using CoefŽcient Entropy. The probability of the data gives a lower bound on code length but does not specify a code. An alternative method for estimating the coding efŽciency is to estimate the entropy of a proposed code. For these models, a natural choice for the code is the vector coefŽcients, s. In this approach, the distribution of the coefŽcients s (Žt to a training data set) is estimated with a function f ( s). The coding cost for a test data set is computed estimating the entropy of the Žtted coefŽcients to a quantization level needed to maintain an encoding noise level of sx . This has the advantage that f ( s) can give a better approximation to the observed distribution of s (better than the Laplacian assumed under the model). In the examples below, we assume that all the coefŽcients have the same distribution, although it is straightforward to use a separate function for the distribution estimate of each coefŽcient. One note of caution with this technique, however, is that it does not include any cost of misŽtting. A basis that does not fully span the data space will result in poor Žts to the data but can still yield low entropy. If the bases under consideration fully span the data space, then the entropy will yield a reasonable estimate of coding cost. To compute the entropy, the precision to which each coefŽcient is encoded needs to be speciŽed. Ideally, f (s) should be quantized to maintain

Learning Overcomplete Representations

351

an encoding noise level of sx . In the examples below, we use the approximation dsi ¼ sx / | A i |. This relates a change in si to a change in the data xi and is exact if A is orthogonal. We use the mean value of d si to obtain a single quantization level d s for all coefŽcients. The function f ( s) by applying kernel density estimation (Silverman, 1986) to the distribution of coefŽcients Žt to a training data set. We use a Laplacian kernel with a window width of 2d s. The most straightforward method of estimating the coding cost (in bits per pattern) is to sum over the individual entropies, #bits ¸ ¡

X ni log2 f [i], i N

(6.3)

where f ( s) is quantized as f [i], the index i ranges over all the bins in the quantization, and ni is the number of counts observed in each bin for each of the coefŽcients Žt to a test data set consisting of N patterns. The distinction between the test and training data set is necessary because the probabilities of the code words need to be speciŽed a priori. Using the same data set for test and training underestimates the cost, although for large data sets, the difference is minimal. An alternative method of computing the coding cost using the entropy is to make use of the fact that for sparse-overcomplete representations, a subset of the coefŽcients will be zero. The coding cost can be computed by summing the entropy of the nonzero coefŽcients plus the cost of identifying them, #bits ¸ ¡

X ni log2 fnz [i] C min(M ¡ L, L) log2 ( M) , i N

(6.4)

where fnz [i] is the quantized density estimate of the nonzero coefŽcients in the ith bin and the index i ranges over all the bins. This method is appropriate when the cost of identifying the nonzero coefŽcients is small compared to the cost of sending M ¡ L (zero-valued) coefŽcients. 6.2.1 Application to Test Data. We now quantify the efŽciency of various representations for the data sets shown in Figure 3 by computing the entropy of the coefŽcients. As before, the estimated coding cost was calculated on 1000 randomly sampled data points, using an encoding precision of 0.01. A separate training data set consisting of 10,000 data points was used to estimate the distribution of the coefŽcients. The entropy was estimated by computing the entropy of the nonzero coefŽcients (see equation 6.4), which yielded lower estimated coding costs for the overcomplete bases compared to summing the individual entropies (see equation 6.3). Table 2 shows the estimated coding costs for the same learned bases and data sets used in Table 1. The second column lists the cost of labeling the

352

Michael S. Lewicki and Terrence J. Sejnowski

Table 2: Estimated Entropy of Nonzero CoefŽcients.

Model A 2£2 A 2£3 A 2£4

Bits per Pattern

Labeling Cost

Figure 1a

Figure 3a

Figure 3b

Figure 3c

0.0 1.6 2.0

18.88 § 0.04 18.02 § 0.03 —

17.78 § 0.07 17.28 § 0.05 —

18.18 § 0.05 17.75 § 0.05 17.88 § 0.07

21.45 § 0.09 20.95 § 0.08 20.81 § 0.09

nonzero coefŽcients. The other columns list the coding cost for the values of the nonzero coefŽcients. The total coding cost is the sum of these two numbers. For the 2 £ 2 matrices (the ICA solution), the entropy estimate yields approximately the same coding cost as the estimate based on P( x| A ), which indicates that in the complete case, the computation of P(x | A ) is accurate. This entropy estimate, however, yields a higher coding for the overcomplete bases. The coding cost computed from P(x | A ) suggests that lower average coding costs are achievable. One strategy for lowering the average coding cost would be to avoid identifying the nonzero coefŽcients for every pattern, for example, by grouping patterns that share common nonzero coefŽcients. This illustrates the advantages of contrasting the minimal coding cost estimated from the probability with that of a particular code. 7 Learning Sparse Representations of Speech

Speech data were obtained from the TIMIT database, using speech a single speaker, speaking 10 different example sentences. Speech segments were 64 samples in duration (8 msecs at the sampling frequency of 8000 Hz). No preprocessing was done. Both a complete (1£-overcomplete or 64 basis vectors) and a 2£-overcomplete basis (128 basis vectors) were learned. The bases were initialized to the standard Fourier basis and a 2£-overcomplete Fourier basis, which was composed of twice the number of sine and cosine basis functions, linearly spaced in frequency over the Nyquist range. For larger problems, it is desirable to make the learning faster. Some simple modiŽcations to the basic gradient descent procedure were used that produced more rapid and reliable convergence. For each gradient (see equation A.33), a step size was computed by d i = 2 i / amax , where amax is the element of the basis matrix, A , with largest absolute value. The parameter 2 was reduced from 0.02r to 0.001r over the Žrst 1000 iterations and Žxed at 0.001r for the remaining iterations, where r is a measure of the data range and was deŽned to be the standard deviation of the data. For the learning rule, we replaced the term A in equation A.39 with an approximation to l AATAH ¡1 (see equation A.36) as suggested in Lewicki

Learning Overcomplete Representations a

353 b

Figure 5: An example of Žtting a 2£-overcomplete representation to segments of natural speech. Speech segments consisted of 64 samples in duration (8 msecs at the sampling frequency of 8000 Hz). (a) A random sample of 30 of the 128 basis vectors (each scaled to full range). (b) The power spectral densities (0 to 4000 Hz) of the corresponding basis in a.

and Olshausen (1998, 1999): ¡l ATAH ¡1 ¼ I ¡ BQ diag¡1 [V C Q T BQ ]Q T ,

(7.1)

where B = rs rs log P( sˆ ) and Q and V are obtained from the singular value decomposition l ATA = QVQ T . This yielded solutions similar to equation 4.4, but resulted in fewer zero-length basis vectors. One hundred patterns were used to estimate each learning step. Learning was terminated after 5000 steps. The most probable basis function coefŽcients, sˆ , were obtained using a modiŽed conjugate gradient routine (Press, Teukolsky, Vetterling, & Flannery, 1992). The basic routine was modiŽed to replace the line search with an approximate Newton step. This approach resulted in a substantial speed improvement and produced much better solutions in a Žxed amount of time than the standard routine. To improve speed, a convergence criterion in the conjugate gradient routine was started at value of 10¡2 and reduced to 10¡3 over the course of learning. Figure 5 shows a sample of the learned basis vectors from the 2£-overcomplete basis. The waveforms in Figure 5a show a random sample of the learned basis vectors; the corresponding power spectral densities (0– 4000 Hz) are shown in Figure 5b. An immediate observation is that, unlike the Fourier basis vectors, many of the learned basis vectors have a broad bandwidth, and some have multiple spectral peaks. This is an indication that the basis functions contain some of the broadband and harmonic structure inherent in the speech signal. Another way of contrasting the learned representation with the Fourier representation is shown in Figure 6. This shows the log-coefŽcient magnitudes over the duration of a speech sentence taken from the TIMIT database for the 2£-overcomplete Fourier (middle plot) and 2£-overcomplete learned

354

Michael S. Lewicki and Terrence J. Sejnowski

she

had your

dark

suit

in

greasy

wash

water

all

year

3

20 40

2

60

1

80 100

log |s|

coefficient number

Fourier basis

0

120

20 40 60 80 100 120

3 2

log |s|

coefficient number

learned basis

1 0

Figure 6: Comparison of coefŽcient values over the duration of a speech sample from the TIMIT database. (Top) The amplitude envelope of a sentence of speech. The vertical lines are a rough indication of the word boundaries, as listed in the database. (Middle) Plots the log-coefŽcient magnitudes for a 2£-overcomplete Fourier basis over the duration of the speech example, using nonoverlapping blocks of 64 samples without windowing. (Bottom) Shows the log-coefŽcient magnitudes for a 2£-overcomplete learned basis. Only the largest 10% of the coefŽcients (in terms of magnitude) are plotted. The basis vectors are ordered in terms of their power spectra, with higher frequencies toward the top.

representations (bottom plot). The middle plot is similar to a standard spectrogram except that no windowing was performed and the windows were not overlapping. Figure 6 shows that the coefŽcient values for the learned basis are used much more uniformly than in the Fourier basis. This is consistent with the learning objective, which optimizes the basis vectors to make the coefŽcient values as independent as possible. One consequence is that the correlations across frequency that exist in the Fourier representation (reecting the harmonic structure in the speech) are less apparent in the learned representation. 7.1 Comparing EfŽciencies of Representations. Table 3 shows the estimated number of bits per sample to encode a speech segment to a precision of 1 bit out of 8 (sx = 1/ 256 of the amplitude range). The table shows the

Learning Overcomplete Representations

355

Table 3: Estimated Bits per Sample for Speech Data. Estimation Method ¡ log 2 P( x |bA) ¡ L log 2 (sx )

Basis 1£ learned 1£ Fourier 2£ learned 2£ Fourier

4.17 § 5.39 § 5.85 § 6.87 §

Total Entropy

0.07 0.03 0.06 0.06

3.40 § 4.29 § 5.22 § 6.57 §

Nonzero Entropy

0.05 0.07 0.04 0.14

3.36 § 4.24 § 3.07 § 4.12 §

0.08 0.09 0.07 0.11

estimated coding costs using the probability of the data (see equation 6.1) and using the entropy, computed by summing the individual entropy estimates of all coefŽcients (see equation 6.3) (total). Also shown is the entropy of only the nonzero coefŽcients. By both estimates, the complete and the 2£-overcomplete learned basis perform signiŽcantly better than the corresponding Fourier basis. This is not surprising given the observed redundancy in the Fourier representation (see Figure 6). For all of the bases, the coding cost estimates derived from the entropy of the coefŽcients are lower than those derived from P (x | A ). A possible reason is that the coefŽcients are sparser than the Laplacian prior assumed by the model. This is supported by looking at the histogram of the coefŽcients (see Figure 7), which shows a density with kurtosis much higher than the as-

10

Distribution of basis coefficients

0

2x learned 2x Fourier

P(S)

10

10

10

­1

­2

­3

­4

10 ­ 1.5

­1

­ 0.5

0 S

0.5

1

1.5

Figure 7: Distribution of basis function coefŽcients for the 2£ learned and Fourier bases. Note the log scale on the y-axis.

356

Michael S. Lewicki and Terrence J. Sejnowski

sumed Laplacian. The entropy method can obtain a lower estimate, because it Žts the observed density of s. The last column in the table shows that although the entropy per coefŽcient of the 2£-overcomplete bases is less than for the complete bases, neither basis yields an overall improvement in coding efŽciency. There could be several reasons for this. One likely Q reason is due to the independence ( )). This assumes that there is no assumption of the model (P( s ) = m P sm structure among the learned basis vectors. From Figure 6, however, it is apparent that there is considerably more structure left in the values of basis vector coefŽcients. Possible generalizations for capturing this kind of structure are discussed below. 8 Discussion

We have presented an algorithm for learning overcomplete bases. In some cases, overcomplete representations allow a basis to approximate better the underlying statistical density of the data, which can lead to representations that better capture the underlying structure in the data and have greater coding efŽciency. The probabilistic formulation of the basis inference problem offers the advantages that assumptions about the prior distribution on the basis coefŽcients are made explicit. Moreover, by estimating the probability of the data given the model or the entropy of the basis function coefŽcients, different models can be compared objectively. This algorithm generalizes ICA so that the model accounts for additive noise and allows the basis to be overcomplete. Unlike standard ICA, where the internal states are computed by inverting the basis function matrix, in this model the transformation from the data to the internal representation is nonlinear. This occurs because when the model is generalized to account for additive noise or when the basis is allowed to be overcomplete, the internal representation is ambiguous. The internal representation is then obtained by maximizing the posterior probability of s given the assumptions of the model, which is, in general, a nonlinear operation. A further advantage of this formulation is that it is possible to “denoise” the data. The inference procedure of Žnding the most probable coefŽcients automatically separates the data into the underlying signal and the additive noise. By setting the noise to appropriate levels, the model speciŽes what structure in the data should be ignored and what structure should be modeled by the basis functions. The derivation given here also allows the noise level to be set to zero. In this case, the model attempts to account for all of the variability in the data. This approach to denoising has been applied successfully to natural images (Lewicki & Olshausen, 1998, 1999). Another potential application is the blind separation of more sources than mixtures. For example, the two-dimensional examples in Figure 3 can be viewed as a source separation problem in which a number of sources are mixed onto a smaller number of channels (three to two in Figure 3a and

Learning Overcomplete Representations

357

four to two in Figures 3b and 3c). The overcomplete bases allow the model to capture the underlying statistical structure in the data space. True source separation will be limited, however, because the sources are being mapped down to a smaller subspace and there is necessarily a loss of information. Nonetheless, is it possible to separate three speakers on two channels with good Ždelity (Lee, Lewicki, Girolami, & Sejnowski, 1999). We have also shown, in the case of natural speech, that learned bases have better coding properties than commonly used representations such as the Fourier basis. In these examples, the learned basis resulted in an estimated coding efŽciency for (near-lossless compression) that was about 1.4 bits per sample better than the Fourier basis. This reects the fact that spectral representations of speech are redundant, because of spectral structures such as speaker harmonics. In the complete case, the model is equivalent to ICA, but with additive noise. We emphasize that these numbers reect only a very simple coding scheme based on 64 sample speech segments. Other coding schemes can achieve much greater compression. The analysis presented here serves only to compare the relative efŽciency of a code based on Fourier representation with that of a learned representation. The learned overcomplete representations also showed greater coding efŽciency than the overcomplete Fourier representation but did not show greater coding efŽciency than the complete representation. One possible reason is that the approximations are inaccurate. The approximation used to estimate the probability of the data (see equation 4.2) works well for learning the basis functions and also gives reasonable estimates when exact quantities are available. However, it is possible that the approximation becomes increasingly inaccurate for higher degrees of overcompleteness. An open challenge for this and other inference problems is how to obtain accurate and computationally tractable approximations to the data probability. Another, perhaps more plausible, reason for the lack of coding efŽciency for the overcomplete representations is that the assumptions of the model are inaccurate. An obvious one is the prior distribution for the coefŽcients. For example, the observed coefŽcient distribution for the speech data (see Figure 7) is much sparser than that of the Laplacian assumed by the model. Generalizing these models so that P( s) better captures the kind of structure would improve the accuracy of the model and allow it to Žt a broader range of distributions. Generalizing these techniques to handle more general prior distributions, however, is not straightforward because of the difŽculties in optimization techniques for Žnding the most probable coefŽcients and the approximation of the data probability, log P (x | A ). These are directions we are currently investigating. We need to question the fundamental assumption that the coefŽcients are statistically independent. This is not true for structures such as speech, where there is a high degree of nonstationary statistical structure. This type of structure is clearly visible in the spectrogram-like plots in Figure 6. Be-

358

Michael S. Lewicki and Terrence J. Sejnowski

cause the coefŽcients are assumed to be independent, it is not possible to capture mutually exclusive relationships that might exist among different speech sounds. For example, different bases could be specialized for different types of phonemes. Capturing this type of structure requires generalizing the model so that it can represent and learn higher-order structure. In applications of overcomplete representations, it is common to assume the coefŽcients are independent and have Laplacian distributions (Mallat & Zhang, 1993; Chen et al., 1996). These results show that although overcomplete representations can potentially yield many beneŽts, in practice these can be limited by inaccurate prior assumptions. An improved model of the data distribution will lead to greater coding efŽciency and also allow for more accurate inference. The Bayesian framework presented in this article provides a direct method for evaluating the validity of these assumptions and also suggests ways in which these models might be generalized. Finally, the approach taken here to overcomplete representation may also be useful for understanding the nature of neural codes in the cerebral cortex. One million ganglion cells in the optic nerve are represented by 100 million cells in the primary visual cortex of the monkey. Thus, it is possible that at the Žrst stage of cortical processing, the visual world is overrepresented by a large factor. Appendix A.1 Approximating P (x | A ) . The integral

Z P(x | A ) =

d s P( s ) P( x | s , A )

(A.1)

can be approximated with the gaussian integral, Z ds f (s ) ¼ f ( sˆ )(2p ) k/ 2 | ¡ rs rs log f ( s )| ¡1/ 2 ,

(A.2)

where | ¢ | indicates the determinant and rs rs indicates the Hessian, which is evaluated at the solution sˆ (see equation 2.3). Using f ( s) = P(s ) P(x | s , A ) we have £ ¤ ¡rs rs log P( s) P( x| s, A ) = H ( s)

µ ¶ l = ¡rs rs ¡ (x ¡ As) 2 C log P(s ) 2 h i = rs ¡l AT (x ¡ As) ¡ rs P (s ) = l ATA ¡ rs rs P( s).

(A.3) (A.4) (A.5)

Learning Overcomplete Representations

359

A.2 The Hessian of the Laplacian. In the case of a Laplacian prior on

sm , log P( s ) ´

X m

log P( sm ) = M logh ¡ h

M X

|sm |.

(A.6)

m= 1

Because log P( s ) is piece-wise linear in s , the curvature is zero, and there is a discontinuity in the derivative at zero. To approximate the volume contribution from P (s ), we use @ log P( s) ¼ ¡h tanh(bsm ), @sm

(A.7)

which is equivalent to using the approximation P( sm ) / cosh¡h/ b (b sm ). For large b this approximates the true Laplacian prior while staying smooth around zero. This leads to the following diagonal expression for the Hessian: @2 log P( sm ) = ¡hb sech2 (b sm ). @s2m

(A.8)

A.3 Derivation of the Learning Rule. For a set of independent data vectors X = x 1 , . . . , x K , expand the posterior probability density by a saddlepoint approximation:

log P( X | A ) = log

Y

P( x k | A )

k

¼

l KL KM log C log(2p ) (A.9) 2 2p 2 ¶ K µ X l 1 C log P( sˆ k ) ¡ ( x k ¡ A sˆ k ) 2 ¡ log det H ( sˆ k ) . 2 2 k= 1

A learning rule can be obtained by differentiating log P( x | A ) with respect to A . Letting rA = @/ @A and for clarity letting H = H ( s), we see that there are three main derivatives that need to be considered: rA log P( x | A ) = rA log P( sˆ ) ¡

l 1 rA ( x ¡ A sˆ ) 2 ¡ r A log det H . (A.10) 2 2

We consider each in turn. A.3.1 Deriving rA log P( sˆ ) . The Žrst term in the learning rule speciŽes how to change A to make the representation sˆ more probable. If we assume a prior distribution with high kurtosis, this component will change the weights to make the representation sparser.

360

Michael S. Lewicki and Terrence J. Sejnowski

From the chain rule, X @log P( sOm ) @Osm @log P(sˆ ) = , (A.11) @A @Osm @A m Q assuming P(sˆ ) = sm / @A , we need to describe sˆ as a m P(Osm ). To obtain @O function of A . If the basis is complete (and we assume low noise), then we can simply invert A to obtain sˆ = A ¡1 x. When A is overcomplete, however, there is no simple expression, but we can still make an approximation. For certain priors, the most probable solution, sˆ , will yield at most L nonzero elements. In effect, the procedure for computing sˆ selects a complete basis from A that best accounts for the data x . We can use this hypothetical reduced basis to derive a learning algorithm for the full basis matrix A . L represent a reduced basis for a particular x , that is, A L is composed Let A of the basis vectors in A that have nonzero coefŽcients. More precisely, let c1 , . . . , cL be the indices of the nonzero coefŽcients in sˆ . If there are fewer than L nonzero coefŽcients, zero-valued coefŽcients can be included without loss L 1:L,i ´ A 1:L,c . We then have of generality. Then, A i L ¡1 (x ¡ ²), sL = A

(A.12)

@Lsk @ X L ¡1 = A kl ( xl ¡ 2 l ), @Laij @Laij l

(A.13)

L ¡1 obtained where sL is equal to sˆ with M¡L zero-valued elements removed. A by removing the columns of A corresponding to the M ¡ L zero-valued elements of sˆ . This allows the use of results obtained for the case when A is invertible. Note that the construction of the reduced basis is only a mathematical device used for this derivation. The Žnal gradient equation does not depend on this construction. Following MacKay (1996) we have

Using the identity @A ¡1 / @aij = ¡A ¡1 Ajl¡1 , kl ki X @Lsk L ¡1 A L ¡1 (xl ¡ 2 l ). = ¡ A ki jl @Laij l L ¡1 sLj . = ¡A ki

(A.14) (A.15)

Letting zL k = @log P( sLk )/ @Lsk , X @log P(sL ) L ¡1 sLj . = ¡ zL k A ki @Laij k

(A.16)

Changing back to matrix notation, @log P(sL ) L ¡T zL sL T . = ¡A L A @

(A.17)

Learning Overcomplete Representations

361

This derivative can be expressed in terms of the original variables (that is, nonreduced). We invert the mapping used to obtain the reduced coefŽcients, sˆ ! sL . We have zcm = zL m , with the remaining (M ¡ L) values of z deŽned L ¡1 , with the remaining rows to be zero. We deŽne the matrix W 1:L,ci ´ A 1:L,i equal to zero. Then @log P( s) = ¡W T zsT . @A

(A.18)

This gives a gradient of zero for the columns of A that have zero-valued coefŽcients. A.3.2 Deriving rA ( x ¡ A sˆ ) 2 . The second term speciŽes how to change A to minimize the data misŽt. Letting ek = [x ¡ As ]k and using the results and notation from above: X X @sl @ lX 2 ek = lei sj Cl ek akl @aij 2 k @aij k l X X = lei sj Cl ¡akl wli sj ek k

(A.19) (A.20)

l

= lei sj ¡ l ei sj = 0.

(A.21)

Thus, there is no gradient component arising from the error term that might be expected if the residual error has no structure. A.3.3 Deriving rA log det H . The third term in the learning rule speciŽes how to change the weights to minimize the width of the posterior distribution P(x | A ) and thus increase the overall probability of the data. Using the chain rule, 1 X @det H @H mn @log det H = . det H mn @H mn @aij @aij

(A.22)

An element of H is deŽned by H mn =

X k

l akm akn C bmn

= cmn C bmn ,

(A.23) (A.24)

where bmn = (¡rs rs log P( sˆ )) mn . Using the identity @det H / @H mn = (det H ) H ¡1 nm we obtain µ ¶ X @log det H @bmn ¡1 @cmn = H nm C . @aij @aij @aij mn

(A.25)

362

Michael S. Lewicki and Terrence J. Sejnowski

Considering the Žrst term in the square brackets,

@cmn @aij

8 > > <0 l ain = > l aim > : 2l aij

m, n 6 = j, m = j, n = j, m = n = j.

(A.26)

We then have X mn

H ¡1 mn

X X @cmn ¡1 = H ¡1 H jm l aim C H jj¡1 2l aij mj laim C @aij 6 6 = = m j m j X ¡1 = 2l H mj aim ,

(A.27) (A.28)

m

¡1 using the fact that H ¡1 = H jm due to the symmetry of the Hessian. In matrix mj notation this becomes

X mn

H ¡1 mn

@cmn = 2l AH ¡1 . @A

Next we derive @bmm / @aij . Assuming P( sˆ ) = rs rs log P( sˆ ) is diagonal and thus X mn

H ¡1 nm

(A.29) Q

X @bmn @bmm @Osm = H ¡1 . mm @aij @Osm @aij m

mP

( sOm ) we have that

(A.30)

Letting 2ym = H ¡1 sm and using the result under the reduced repremm @bmm / @O sentation (see equation A.15), X mm

H ¡1 mm

@bmm = ¡2W T yˆs T . @A

(A.31)

Finally, putting the results for @cmm / @A and @bmm / @A back into equation A.25, @log det H = 2l AH ¡1 ¡ 2W T yˆsT . @A

(A.32)

Gathering the three components of rA log P(x | A ) together yields the following expression for the learning rule: r A log P( x| A ) = ¡W T zˆs T ¡ l AH ¡1 C W T yˆs T .

(A.33)

Learning Overcomplete Representations

363

A.3.4 Stabilizing and Simplifying the Learning Rule. Using the gradient given in equation A.33 directly is problematic due to the matrix inverses, which make it both impractical and unstable. This can be alleviated, however, by multiplying the gradient by an appropriate positive deŽnite matrix (Amari et al., 1996). This rescales the components of the gradient but still preserves a direction valid for optimization. The fact that AT W T = I for all W allows us to eliminate W from the equation for the learning rule: AAT rA log P( x| A ) = ¡Az sˆ T ¡ l AATAH ¡1 C AyˆsT .

(A.34)

Additional simpliŽcations can be made. Ifl is large (low noise), then the Hessian is dominated by l ATA and ¡l AATAH ¡1 = ¡Al ATA (l ATA C B ) ¡1 ¼ ¡A .

(A.35) (A.36)

It is also possible to obtain more accurate approximations of this term (Lewicki & Olshausen, 1998, 1999). The vector y hides a computation involving the inverse Hessian. If the basis vectors in A are randomly distributed, then as the dimensionality of A increases, the basis vectors become approximately orthogonal, and consequently the Hessian becomes approximately diagonal. In this case, ym ¼ =

1

@bmm

(A.37)

H mm @Osm

@bmm / @Osm . P 2 l k akm C bmm

(A.38)

Thus if log P( s ) and its derivatives are smooth, ym vanishes for large l. In practice, excluding y from the learning rule yields more stable learning. We conjecture that this term can be ignored, possibly because it represents curvature components that are unrelated to the volume. We thus obtain the following expression for a learning rule: D A = AAT rA log P( x| A ) ¼ ¡Az sˆ T ¡ A

= ¡A ( zˆsT C I ).

(A.39) (A.40)

Acknowledgments

We thank Bruno Olshausen and Tony Bell for many helpful discussions.

364

Michael S. Lewicki and Terrence J. Sejnowski

References Amari, S., Cichocki, A., & Yang, H. H. (1996).A new learning algorithm for blind signal separation. In D. Touretzy, M. Mozer, & M. Hasselmo (Eds.), Advances in neural and information processing systems, 8 (pp. 757–763). San Mateo, CA: Morgan Kaufmann. Atick, J. J. (1992). Could information-theory provide an ecological theory of sensory processing? Network-Computation in Neural Systems, 3(2), 213–251. Barlow, H. B. (1961). Possible principles underlying the transformation of sensory messages. In W. A. Rosenbluth (Ed.), Senosory communication (pp. 217– 234). Cambridge, MA: MIT Press. Barlow, H. B. (1989). Unsupervised learning. Neural Computation, 1, 295–311. Bell, A. J., & Sejnowski, T. J. (1995). An information maximization approach to blind separation and blind deconvolution. Neural Computation, 7(6), 1129– 1159. Buccigrossi, R. W., & Simoncelli, E. P. (1997). Image compression via joint statistical characterization in the wavelet domain (Tech. Rep. No. 414). Philadelphia: University of Pennsylvania. Cardoso, J.-F. (1997). Infomax and maximum likelihood for blind source separation. IEEE Signal Processing Letters, 4, 109–111. Chen, S., Donoho, D. L., & Saunders, M. A. (1996). Atomic decomposition by basis pursuit (Technical Rep.). Stanford, CA: Department of Statistics, Stanford University. Coifman, R. R., & Wickerhauser, M. V. (1992). Entropy-based algorithms for best basis selection. IEEE Transactions on Information Theory, 38(2), 713–718. Comon, P. (1994). Independent component analysis, a new concept. Signal Processing, 36(3), 287–314. Daubechies, I. (1990). The wavelet transform, time-frequency localization, and signal analysis. IEEE Transactions on Information Theory, 36(5), 961–1004. Daugman, J. G. (1988). Complete discrete 2-D Gabor transforms by neural networks for image-analysis and compression. IEEE Transactions on Acoustics Speech and Signal Processing, 36(7), 1169–1179. Daugman, J. G. (1989). Entropy reduction and decorrelation in visual coding by oriented neural receptive-Želds. IEEE Trans. Bio. Eng., 36(1), 107–114. Field, D. J. (1994). What is the goal of sensory coding? Neural Computation, 6(4), 559–601. Hinton, G. E., & Sejnowski, T. J. (1986). Learning and relearning in Boltzmann machines. In D. E. Rumelhart & J. L. McClelland (Eds.), Parallel distributed processing (Vol. 1, pp. 282–317). Cambridge, MA: MIT Press. Jutten, C., & H´erault, J. (1991). Blind separation of sources. 1. An adaptive algorithm based on neuromimetic architecture. Signal Processing, 24(1), 1–10. Lee, T.-W., Lewicki, M., Girolami, M., & Sejnowski, T. (1999). Blind source separation of more sources than mixtures using overcomplete representations. IEEE Sig. Proc. Lett., 6, 87–90. Lewicki, M. S., & Olshausen, B. A. (1998). Inferring sparse, overcomplete image codes using an efŽcient coding framework. In M. Kearns, M. Jordan, & S. Solla

Learning Overcomplete Representations

365

(Eds.),Advances in neural and informationprocessing systems, 10. San Mateo, CA: Morgan Kaufmann. Lewicki, M. S., & Olshausen, B. A. (1999). A probabilistic framework for the adaptation and comparison of image codes. J. Opt. Soc. Am. A: Optics, Image Science, and Vision, 16(7), 1587–1601. Linsker, R. (1988). Self-organization in a perceptual network. Computer, 21(3), 105–117. MacKay, D. J. C. (1996). Maximum likelihood and covariant algorithms for independent component analysis. Unpublished manuscript. Cambridge: University of Cambridge, Cavendish Laboratory. Available online at: ftp://wol.ra.phy.cam.ac.uk/pub/mackay/ica.ps.gz. Makeig, S., Jung, T. P., Bell, A. J., Ghahremani, D., & Sejnowski, T. J. (1996). Blind separation of event-related brain response components. Psychophysiology, 33, S58–S58. Mallat, S. G., & Zhang, Z. F. (1993). Matching pursuits with time-frequency dictionaries. IEEE Transactions on Signal Processing, 41(12), 3397–3415. McKeown, M. J., Jung, T.-P., Makeig, S., Brown, G. G., Kindermann, S. S., Lee, T., & Sejnowski, T. (1998). Spatially independent activity patterns in functional magnetic resonance imaging data during the stroop color-naming task. Proc. Natl. Acad. Sci. USA, 95, 803–810. Meszaros, C. (1997).BPMPD: An interior point linear programming solver. Code available online at: ftp://ftp.netlib.org/opt/bpmpd.tar.gz. Nadal, J.-P. & Parga, N. (1994a). Nonlinear neurons in the low-noise limit: A factorial code maximizes information transfer. Network, 5, 565–581. Nadal, J.-P., & Parga, N. (1994b). Redundancy reduction and independent component analysis: Conditions on cumulants and adaptive approaches. Network, 5, 565–581. Olshausen, B. A., & Field, D. J. (1996). Emergence of simple-cell receptive-Želd properties by learning a sparse code for natural images. Nature, 381, 607–609. Olshausen, B. A., & Field, D. J. (1997). Sparse coding with an overcomplete basis set: A strategy employed by V1? Vision Res., 37(23). Osuna, E., Freund, R., & Girosi, F. (1997). An improved training algorithm for support vector machines. Proc. of IEEE NNSP’97 (pp. 24–26). Pearlmutter, B. A., & Parra, L. C. (1997). Maximum likelihood blind source separation: A context-sensitive generalization of ICA. In M. Mozer, M. Jordan, & T. Petsche (Eds.), Advances in neural and information processing systems, 9. San Mateo, CA: Morgan Kaufmann. Press, W. H., Teukolsky, S. A., Vetterling, W. T., & Flannery, B. P. (1992).Numerical recipes in C: The art of scientiŽc programming (2nd ed.). Cambridge: Cambridge University Press. Silverman, B. W. (1986). Density estimation for statistics and data analysis. New York: Chapman and Hall. Simoncelli, E. P., Freeman, W. T., Adelson, E. H., & Heeger, D. J. (1992). Shiftable multiscale transforms. IEEE Trans. Info. Theory, 38, 587–607. Received February 8, 1998; accepted February 1, 1999.

LETTER

Communicated by Adi Bulsara

Noise in Integrate-and-Fire Neurons: From Stochastic Input to Escape Rates Hans E. Plesser ¨ Str¨omungsforschung, D-37073 G¨ottingen, Germany MPI fur Wulfram Gerstner MANTRA Center for Neuromimetic Systems, Swiss Federal Institute of Technology, EPFL/DI, CH-1015 Lausanne, Switzerland We analyze the effect of noise in integrate-and-Žre neurons driven by time-dependent input and compare the diffusion approximation for the membrane potential to escape noise. It is shown that for time-dependent subthreshold input, diffusive noise can be replaced by escape noise with a hazard function that has a gaussian dependence on the distance between the (noise-free) membrane voltage and threshold. The approximation is improved if we add to the hazard function a probability current propor tional to the derivative of the voltage. Stochastic resonance in response to periodic input occurs in both noise models and exhibits similar characteristics.

1 Introduction

Given the enormous number of degrees of freedom in single neurons due to millions of ion channels and thousands of presynaptic signals impinging on the neuron, it seems natural to describe neuronal activity as a stochastic process (Tuckwell, 1989). Since in most neurons the output signal consists of stereotypical electrical pulses (spikes), the theory of point processes has long been employed to analyze neuronal activity (Perkel, Gerstein, & Moore, 1967; Johnson, 1996). The simplest point process model of neural responses to stimuli is that of an inhomogeneous Poisson process with an instantaneous Žring rate that depends on the stimulus, and thus on time. Models of this type have successfully been employed to characterize neural activity in the auditory system (Siebert, 1970; Johnson, 1980; Gummer, 1991) and to investigate the facilitation of signal transduction by noise, a phenomenon known as stochastic resonance (Collins, Chow, Capela, Imhoff, 1996). Stevens and Zador (1996) have recently suggested that in the limit of very low Žring rates, Poisson models of neuronal activity indeed capture most of the dynamics of more complex models. Neural Computation 12, 367–384 (2000)

c 2000 Massachusetts Institute of Technology °

368

Hans E. Plesser and Wulfram Gerstner

On the other hand, much recent research indicates that the precise timing of neuronal spikes plays a prominent role in information processing (Rieke, Warland, de Ruyter van Steveninck, & Bialek, 1997). Thus, spiking neuron models are required that provide more insight into the relation between the input current a neuron receives and the spike train it generates as output. In short, these model neurons generate an output spike whenever the membrane potential v, driven by an input current I, reaches a threshold H; afterward the potential v is reset to some value vres . For a recent review, see Gerstner (1998). The archetypical spiking neuron model is the integrate-and-Žre neuron introduced by Lapicque (Tuckwell, 1988). Stein (1965) Žrst replaced a continuous input current with more realistic random sequences of excitatory and inhibitory input pulses. If each pulse is small and a large number of pulses impinges on the neuron per membrane time constant, the effective input to the neuron can be described by a time-dependent deterministic current I ( t) and additive noise j (t), which is taken to be gaussian. In this limit, the evolution of the membrane potential from reset to threshold is a time-dependent Ornstein-Uhlenbeck process with the threshold as an absorbing boundary (Johannesma, 1968; L´ansky, ´ 1997). We will refer to this as the diffusion approximation of neuronal activity. The separation of the total input current into a deterministic part I ( t) and noise is much less clear-cut in neuronal modeling than in physical processes such as Brownian motion. Recent research suggests that the variability of input spike arrival times plays a much greater role than intrinsic noise (Mainen & Sejnowski, 1995; Bryant & Segundo, 1976). Stochastic spike arrival may arise in noise-free networks with heterogeneous connections as shown in recent theoretical studies (van Vreeswijk & Sompolinsky, 1996; Brunel & Hakim, 1999). In these studies balanced excitation and inhibition prepared the neuron in a state that is just subthreshold. Spikes are then triggered by the uctuations in the input. Neurons in this regime have a large coefŽcient of variation just as cortical neurons. For a recent review, see Konig, ¨ Engel, & Singer, 1996. The subthreshold regime is also particularly interesting from the point of view of information transmission. Neurons optimally detect temporal information if the mean membrane potential is roughly one standard deviation of the noise below threshold (Kempter, Gerstner, van Hemmen, & Wagner, 1998). The improvement of signal transduction by noise, known as stochastic resonance, also appears to be limited to this regime (Bulsara & Zador, 1996). Therefore we will focus here on neurons driven by weakly subthreshold input—those neurons that would be silent in the absence of noise. In this work, we take the diffusion approximation sketched above as the reference model for neuronal noise. A disadvantage of this model is that it is difŽcult to solve analytically, particularly for time-dependent input I (t). The interval distribution for a given input I ( t) is mathematically equivalent

Noise in Integrate-and-Fire Neurons

369

to a Žrst-passage-time problem, which is known to be hard. Our aim in this study is to replace the stochasticity introduced by the diffusion term by a mathematically more tractable inhomogeneous point process. To this end, we investigate various escape processes across the Žring threshold. An escape process is completely described by a hazard function h( t), which depends only on the momentary values of the membrane potential v(t) and the input current I ( t). We want to identify that function h which reproduces as closely as possible the behavior of the diffusion model, for both periodic and aperiodic inputs. We demonstrate that the optimal hazard function not only reproduces the generation of individual spikes well, but that complex response properties such as stochastic resonance are preserved. 2 Integrate-and-Fire Neuron Models 2.1 Diffusion Model. Between two output spikes, the membrane potential of an integrate-and-Žre neuron receiving input I ( t) evolves according to the following Langevin equation (Tuckwell, 1989):

vP ( t) = ¡v(t) C I ( t) C sj ( t).

(2.1)

Upon crossing the threshold, v( t) = 1, a spike is recorded and the « potential ¬ reset to v = 0. j ( t) is gaussian white noise with autocorrelation j ( t)j (t0 ) = d ( t ¡ t0 ). The diffusion constant s is the root mean square (rms) amplitude of this background noise. Note that we have used dimensionless units above; time is measured in units of the membrane time constant, while voltage is given in terms of the threshold H. As a consequence of the diffusion approximation, the input I ( t) will not contain any d-pulses, and v(t) follows continuous trajectories from reset potential to threshold. Let us denote the Žring time of the last output spike by t¤ . In the noisefree case (s = 0), the potential v( t) for t > t¤ depends on the input I ( t0 ) for t¤ < t0 < t and can be found by integration of equation 2.1: Z v0 (t | t¤ , I (¢)) = e¡t

t 0

I ( t¤ C s) es ds.

(2.2)

Our notation makes explicit that the noise-free trajectory can be calculated given the last Žring time t¤ and the input I. The subscript 0 is intended to remind the reader that equation 2.2 describes the noise-free trajectory. The next output spike of the noiseless model would occur at a time t¤ Ct deŽned by the Žrst threshold crossing, © ª t = min t 0 > 0 | v0 (t 0 | t¤ , I (¢)) ¸ 1 .

(2.3)

In the presence of noise, the actual membrane trajectory will uctuate around the reference trajectory (see equation 2.2) with a variance of the

370

Hans E. Plesser and Wulfram Gerstner

order of s. We can no longer predict the exact timing of the next spike, but we may ask for the probability distribution r (t | t¤ , I (¢)) that a spike occurs at t¤ C t , given that the last spike was at t¤ and the current is I ( t0 ) for t¤ < t0 < t¤ C t : 8 9 < Spike in [t¤ C t, t¤ C t C D t) = r (t | t¤ , I (¢))D t = Pr given the last reset at t¤ and input . (2.4) : ¤ ; I ( t C t 0 ), t 0 > 0. Our notation makes explicit that no information about the past of the neuron going further back than the last Žring or reset at t¤ is needed. We may think of equation 2.4 as a conditional interspike interval (ISI) distribution. For the diffusion noise model, analytical expressions for the interval distributions are not known, except for some rare special cases (Tuckwell, 1989). It is, however, possible to calculate these ISI distributions numerically based on Schrodinger ¨ ’s renewal ansatz (Plesser & Tanaka, 1997; Linz, 1985). (Source code is available from us on request.) In the following we use the diffusion model as the reference noise model. If there were no absorbing threshold, equation 2.1 would describe free diffusion with drift. Then the pertaining Fokker-Planck equation, ¡ ¢ ¤ ¡ ¢ @ @ £ P f v, t¤ C t | t¤ = ¡ ¡v C I (t¤ C t ) P f v, t¤ C t | t¤ @t @v C

¡ ¢ s 2 @2 P f v, t¤ C t | t¤ , 2 2 @v

(2.5)

with boundary conditions limv!§ 1 P f ( v, t¤ C t | t¤ ) = 0 has the solution ¡ ¢ P f v, t¤ C t | t¤ = p

» ¼ [v ¡ v0 (t | t¤ , I (¢))]2 exp ¡ , 2g 2 (t ) 2pg 2 (t ) 1

(2.6)

¢ 2 ¡ where g 2 (t ) = s2 1 ¡ e¡2t . As the exponential term ing(t ) decays at twice the rate of those in v0 (t | t¤ , I (¢)), we may approximate for t À 1 » ¼ ¡ ¢ 1 [v ¡ v0 (t | t¤ , I (¢))]2 P f v, t¤ C t | t¤ ¼ p exp ¡ . s2 ps

(2.7)

2.2 Escape Noise and Hazard Functions. In a noise-free model, the membrane trajectory v0 (t | t¤ , I (¢)) is given by equation 2.2, and the time of the next spike determined by equation 2.3. In the presence of noise, Žring is no longer precise but may occur even though the noise-free trajectory has not yet reached threshold. A simple intuitive noise model can be based on the idea of an escape probability: at each moment of time, the neuron may Žre with an instantaneous rate h, which depends on the momentary distance

Noise in Integrate-and-Fire Neurons

371

between the noise-free trajectory v0 and the threshold, and possibly the momentary input current as well. More generally, we may introduce a hazard function h(t | t¤ , I (¢)) that describes the risk of escape across the threshold and depends on the last Žring time t¤ and the input I ( t0 ) for t¤ < t0 < t¤ C t. Once we know the hazard function, the ISI distribution is given by Cox & Lewis, 1966) µ Z r (t | t , I (¢)) = h(t | t , I (¢)) exp ¡ ¤

¤

t 0

¶ h(s | t , I (¢)) ds . ¤

(2.8)

The exponential term accounts for the probability that a neuron survives from t¤ to t¤ Ct without Žring; the factor h(t | t¤ , I (¢)) gives the rate of Žring at t¤ C t , provided that the neuron has survived thus far. We discuss in this work four models of simpliŽed neuronal dynamics, all of which aim to approximate the diffusion model by an escape noise ansatz. The various models differ in the choice of the function h(t | t¤ , I (¢)) (see Figure 1). These hazard functions depend on the noise-free membrane potential v0 and the input current I only through two scaled quantities. One is the momentary distance x between the membrane potential v0 and the threshold H = 1, scaled by the noise amplitude: x(t | t¤ ) =

1 ¡ v0 (t | t¤ ) . s

(2.9)

The other is the net current charging the membrane in the noise-free case, also scaled by the noise amplitude: Y(t | t¤ ) =

¡v0 (t | t¤ ) C I ( t¤ C t ) 1 d = v0 (t | t¤ ). s s dt

(2.10)

The last equality follows immediately from equation (2.1) with s = 0 and indicates that Y can be interpreted as a relative velocity of the noise-free membrane potential v0 . 2.2.1 Arrhenius Model. As long as the membrane potential v stays sufŽciently far below threshold, we may describe neuronal Žring as a noiseactivated process. The latter is most easily characterized by an Arrhenius ansatz for the hazard function (van Kampen, 1992): » ¼ (1 ¡ v0 (t | t¤ )) 2 ¤ 2 = w e¡x(t | t ) , hArr (t | t¤ ) = w exp ¡ 2 s wopt = 0.95.

(2.11)

Here, 1 ¡ v0 (t | t¤ ) is the voltage gap that needs to be bridged to initiate a spike. Its square may be interpreted as the required activation energy

372

Hans E. Plesser and Wulfram Gerstner

(a)

h(v)

1 0.5 0 0.7

0.8

0.9

0.8

0.9

v

1

1.1

1.2

1

1.1

1.2

(b)

h(v)

1 0.5 0 0.7

v

Figure 1: Hazard functions h(v) versus membrane potential v. (a) Arrhenius model (solid), error function model (dash-dotted), and Tuckwell model (dashed). (b) Arrhenius&Current model for different values of the input current: I = 0.95 (solid), 1 (dash-dotted), and 1.1 (dashed); the case I = 0 is drawn dotted for comparison. In all cases, the threshold is at v = 1 and the noise amplitude is s = 0.1. The vertical dotted lines mark one noise amplitude below threshold.

and s 2 as the energy supplied by the noise. The free parameter w was determined by an optimization procedure described below. We found w = wopt = 0.95 to be the optimal value. Note that for strongly superthreshold stimuli (v0 À 1), the hazard function vanishes exponentially. This might seem paradoxical at Žrst, but is of little concern as long as the input I ( t) contains no d-pulses and v0 reaches the threshold only along continuous trajectories. Then, superthreshold levels of the potential are accessible only via periods of maximum hazard, so that the neuron will usually have Žred before v0 becomes signiŽcantly superthreshold. 2.2.2 Arrhenius&Current Model. Strong, positive transients in the input current will push the membrane potential toward and across the threshold in a time that is short on the timescale of diffusion. Figuratively, the membrane potential distribution P f is shifted as a whole. The simple Arrhenius ansatz

Noise in Integrate-and-Fire Neurons

373

will not be able to reproduce these transients well. We thus consider, in addition to the diffusive current, the probability current induced by shifting the probability density at the threshold, P f (H = 1, t | t¤ ), across threshold d with the speed of the center of this distribution, dt v0 (t | t¤ ) = ¡v0 (t | t¤ ) C I ( t¤ C t). Since there can be no drift from above threshold downward, we set all negative currents to zero and obtain the drift probability current: ¡ ¢ Jdrift (t | t¤ ) = [¡v0 (t | t¤ ) C I ( t¤ C t )] C P f H = 1, t | t¤ =

[Y(t | t¤ )] C ¡x(t | t¤ ) 2 e p . p

P f (v, t | t¤ ) is again the free gaussian distribution given by equation 2.7 and [x] C ´ ( x C|x|)/ 2. Together with the original, diffusive Arrhenius term, we obtain the hazard function: hAC (t | t¤ ) = hArr (t | t¤ ) C Jdrift ( v, t | t¤ ) =

¡

wC

¢

[Y(t | t¤ )] C ¡x(t |t¤ ) 2 e , p p

wopt = 0.72.

(2.12)

As before, w is a free parameter with optimal value wopt . Below we will refer to the Žrst term of equation 2.12 as the diffusion term and to the second one as the drift term. 2.2.3 Sigmoidal Model. Abeles (1982) has suggested that the hazard should be related to the proportion of the free membrane R 1potential distribution P f that is beyond the threshold, that is, h(t | t¤ ) » H P f (v, t | t¤ ) dv = erfc x(t | t¤ ). This ansatz is questionable, as the correct potential distribution vanishes beyond the threshold. In view of the widespread use of sigmoidal activation functions in the theory of neural networks (Wilson & Cowan, 1972), we nevertheless test a sigmoidal model but provide it with two free parameters w1 and w2 : herf (t | t¤ ) = w1 erfc[x(t | t¤ ) ¡ w2 ],

opt

w1

opt

= 0.66 , w2

= 0.53.

(2.13)

erfc(x) = 1 ¡ erf(x) is the complementary error function. Given the great similarity of the error function and the hyperbolic tangent, we have not extensively investigated the latter. Preliminary results indicate negligible differences (data not shown). 2.2.4 Tuckwell Model. Suppose that the input I ( t) varies sufŽciently slowly so that we may, at any point, consider the membrane potential of the neuron to be stationary. A stationary potential v0 (t | t¤ ) corresponds to

374

Hans E. Plesser and Wulfram Gerstner

a constant input current Is = v0 (t | t¤ ). In this case, the average Žring rate, and thus the hazard, can be approximated in closed form (Tuckwell, 1989): hTuck (t | t¤ ) =

x(t | t¤ ) ¡x(t | t¤ ) 2 e p h (x(t | t¤ )). p

(2.14)

The Tuckwell model has no free parameter. The Heaviside step function h ( x) is introduced to avoid negative values of the hazard. The approximation made in computing the rate is valid only for stimuli far below threshold, that is, x(t | t¤ ) À 1. Note that this rate is identical to the diffusive current at the location of the threshold for the threshold-free case, ¡

Jdiff v = 1, t | t

¤

¢

­

¡ ¢­ s 2 @2 = P f v, t | t¤ ­ ­ , 2 @v2 v= 1

where P f ( v, t | t¤ ) is the free gaussian distribution (see equation 2.7). 2.3 Input. To compare the responses of integrate-and-Žre neurons with various hazard functions to those with diffusion noise, we have to choose an input scenario. We will try to be as general as possible and consider time-dependent inputs with both aperiodic and periodic time course. The general form of the input is an oscillatory component of rms amplitude q uctuating around a DC value m , p q 2 X aj cos(vj t C w j ), I ( t) = m C q P 2 a j k k

(2.15)

R where vj = j D v with D v the base frequency. Note that q2 = limT!1 T1 0T (I (t) ¡m ) 2 dt is the expected variance of the current. The membrane potential in response to the input in the absence of noise is found from equation 2.2: p £ ¡ ¢ ¤ v0 (t | t¤ ) = m 1 ¡ e¡t C q 2 F( t¤ C t ) ¡ e¡t F( t¤ ) , 1 F ( t) = q P

2 k ak

X j

q

aj

1 C vj2

¡ ¢ sin vj t C wj C acot vj .

(2.16)

Aperiodic stimuli are characterized by a cutoff frequency Vc and are generated by drawing the phases w j at random from a uniform distribution over [0, 2p ), while the amplitudes are given by

8 > <0 aj = 1 > : exp(¡ 12 ( j ¡ jc ) 2 )

vj · 0,

0 < vj · Vc , jc < j.

(2.17)

Noise in Integrate-and-Fire Neurons

375

Periodic input is taken to be cosinusoidal with frequency V = vk . We set aj = d jk in equation 2.15 and obtain p I (t) = m C q 2 cos(Vt C w ). To compare the performance of the various models across stimuli, we describe each stimulus by its relative distance from threshold,

e=

q « ¬ 1 ¡ (m C 2 D v20 ) s

,

(2.18)

where Z D E D v20 = lim

T!1 0

T

2 q2 X aj (v0 (t | t¤ ) ¡ m ) 2 dt = P 2 2 k ak j 1 C vj

is the rms amplitude of the membranep potential oscillations in the noise-free case. For periodic input, the factor of 2 in the deŽnition of e yields ! p q 2 se = 1 ¡ m C p = 1 ¡ sup v0 (t | t¤ ). 1 C V2 t ¸0

(

Thus subthreshold stimuli have e > 0, superthreshold stimuli e < 0. Note that for periodic stimuli e > 0 indeed guarantees that the stimulus is subthreshold; in the absence of noise, the neuron never Žres. For aperiodic input, however, the deŽnition holds only in an rms sense, so that v0 may occasionally cross the threshold even for e > 0. 2.4 Choice of Stimuli. The results presented here were obtained from a set of 14,400 periodic and 14,400 aperiodic stimuli. DC inputs were in p the range 0.55 · m · 1.2, AC amplitudes approximately 0.1(1 ¡ m ) < q 2 < 1.5(1 ¡ m ), stimulus/cutoff frequencies 0.02p · Vc · 2p . For each set of these parameters, Žve phases were chosen randomly from [0, 2p ) for periodic stimuli and Žve different random stimuli generated for the aperiodic case. Each stimulus was tested at eight noise amplitudes, randomly chosen so that the stimuli would have a roughly uniform distribution with respect to their relative distance from threshold with 0.1 < |e| < 3. Some two-thirds of all stimuli were subthreshold. Some stimuli were excluded from analysis. The rejection was based solely on the ISI distributions computed for the diffusion approximation. Specifically, stimuli were excluded if the Žring probability was so low that their R norm was insufŽcient ( 0T r (t ) dt < 0.8), where T was 20 stimulus periods for periodic and 409.4 for aperiodic stimuli. Furthermore, stimuli were rejected if the numerical algorithm was unstable at the time resolution chosen. We veriŽed for some of these cases that with appropriate discretization,

376

Hans E. Plesser and Wulfram Gerstner

the algorithm did converge. The instabilities were caused by very steep threshold crossings of the membrane potential in combination with small noise. After defective stimuli had been excluded, we were left with 8714 periodic stimuli (out of these 4706 subthreshold stimuli) and 12,181 aperiodic stimuli (8027 subthreshold). 2.5 Parameter Optimization. The weights in the models with parameters were chosen as follows. We split the entire set of stimuli into an optimization and a validation set. For each single stimulus, we determined that weight w or set of weights w1 , w2 that minimized the error E deŽned in equation 3.1 below for this one stimulus, using MATLAB minimization routines. We repeated the procedure for all stimuli in the optimization set and constructed the distribution of weights found in this way. The weights opt opt wopt [respectively, w1 , w2 ] are the medians of the weight distributions. Evaluating the error E over both the optimization and the validation set with the Žxed weight wopt yielded identical results. OverŽtting can therefore be excluded. 3 Results 3.1 ISI Distributions. Figure 2 shows the ISI distribution evoked from the diffusion model by a subthreshold aperiodic stimulus. The predicted ISI distribution for the Arrhenius&Current model (dashed) has been based on equation 2.4. To observe this ISI in an experiment, one would present the stimulus shown as the dotted line in Figure 2c repeatedly, starting over at t = 0 each time a spike has been Žred. We may also think of Figure 2a as a PSTH (peri-stimulus time histogram), where each spike triggers a reset of the stimulus. To measure the agreement between this reference solution and the distributions rendered by the escape noise models, we use the relative integrated mean square error (Scott, 1992):

E=

R1 0

dt [r (t | t¤ = 0) ¡ r esc (t | t¤ = 0)]2 R1 . (t | t¤ = 0) 2 0 dt r

(3.1)

Here, r (t | t¤ = 0) is the ISI distribution for the diffusion model, while r esc(t | t¤ = 0) is the distribution obtained from any one of the escape noise models. The example in Figure 2 corresponds to an error of E = 0.026. Figure 3 shows the cumulative distribution of the error E for all four models. The Arrhenius&Current model obviously performs signiŽcantly better than all the other models for sub- and superthreshold stimuli, both periodic and aperiodic, while the Tuckwell model does worst. The Arrhenius and the error function model are tied for second, with the Arrhenius

Noise in Integrate-and-Fire Neurons

377

r (t |t*)

(a) 0.3 0.2 0.1 0 0

5

10

15

20

25

30

5

10

15

20

25

30

5

10

15 t

20

25

30

h(t |t*)

(b) 1 0.75 0.5 0.25 0 0

0.9

0

I(t ), v (t |t*)

(c)

0.7 0.5 0

Figure 2: (a) Interval distribution from the diffusion model (solid) and the Arrhenius&Current model (dashed) for aperiodic input with DC offset m = 0.85, AC amplitude q = 0.1, cutoff frequency Vc = p , and noise amplitude s = 0.1. The relative distance from threshold is e = 0.88 and the error E = 0.026. (b) Hazard as a function of time for the Arrhenius&Current model (solid). The dashed line is the diffusive component of the hazard deŽned in equation 2.12. (c) Input current (dotted) and noise-free membrane potential for the given stimulus (solid). The membrane potential stays below the threshold (dashed) but approaches it closely around t ¼ 27. Note that peaks in the input current in (c) lead to sharp transients of the hazard (b) that would not be reproduced correctly by the diffusion term alone. Thus the Žring probability of the neuron is correlated to maxima of the current rather than the membrane potential, in excellent agreement with the exact diffusion model; see the nice Žt in a.

378

Hans E. Plesser and Wulfram Gerstner

P(E)

(a)

(b)

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

0

10

­3

­2

10 10 E

­1

10

(c)

P(E)

0

0

­3

10 10 E

­2

­1

10

0

­3

10 10 E

­2

­1

10

0

10

(d)

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

0

10

­3

­2

10 10 E

­1

10

0

0

10

Figure 3: Cumulative distribution of the relative mean square error E over all stimuli for Arrhenius (solid), Arrhenius&Current (dash-dotted), error function (dashed), and Tuckwell (dotted) model. The stimuli were (a) subthreshold periodic, (b) superthreshold periodic, (c) subthreshold aperiodic, and (d) superthreshold aperiodic. To interpret the cumulative distributions, let us study the likelihood that the error E of our approximation is smaller than 0.1. We may read off from the graphs in, for example, a, that for subthreshold periodic stimuli, the Tuckwell model would give a probability of E · 0.1 of about 50%, whereas the Arrhenius model and the error function give about 80% and the Arrhenius&Current model a probability of more than 95%.

model yielding slightly smaller errors for superthreshold stimuli. To quantify the performance of the models, we give in Table 1 the median and 90th percentile of the error for all models. Note that for the Arrhenius&Current model for all stimulus conditions, the error is E < 0.077 in 90% of all cases. We therefore conclude that this model provides an excellent approximation to the dynamics of the diffusion model of neuronal activity. We now describe in more detail how the model error depends on the stimuli and briey discuss why the models perform so differently. Figure 4

Noise in Integrate-and-Fire Neurons

379

Table 1: Median (= 50th ) and 90th Percentiles of Errors E. Periodic Aperiodic Periodic Aperiodic Subthreshold Subthreshold Superthreshold Superthreshold Overall 50% 90% 50% 90% 50% 90% 50% 90% 50% 90% Arrhenius Arrhenius &Current erf Tuckwell

0.055 0.148 0.075 0.336 0.157

0.409

0.130

0.375

0.091 0.334

0.018 0.044 0.024 0.070 0.042 0.061 0.147 0.083 0.346 0.189 0.137 0.333 0.242 0.837 0.747

0.090 0.429 0.896

0.034 0.160 0.713

0.090 0.397 0.901

0.026 0.077 0.103 0.358 0.398 0.864

shows the error of the models versus relative distance from threshold of the stimuli, for both periodic and aperiodic stimuli. All models perform better for sub- than for superthreshold stimuli, but the Arrhenius&Current is relatively good across the entire range. Note that the Tuckwell model becomes quite good for periodic stimuli that are far below threshold (e > 1), as is to be expected, since the Tuckwell approximation is valid in this regime. It does not fare as well for aperiodic stimuli, for they may contain intervals during which the membrane potential of a subthreshold stimulus comes close to threshold. 3.2 Stochastic Resonance. Recent years have seen mounting evidence that signal transmission in neurons can be improved by noise, an effect known as stochastic resonance (Douglass, Wilkens, Pantazelou, & Moss, 1993; Wiesenfeld & Moss, 1995; Cordo et al., 1996; Levin & Miller, 1996; Moss & Russell, 1998). The typical experimental paradigm is to stimulate a neuron sinusoidally, adding noise of varying intensity, and to measure the signal-to-noise ratio (SNR) of the spike train elicited from the neuron. If this SNR peaks for nonvanishing input noise, the system is said to exhibit stochastic resonance. (For a recent review of the Želd, see Gammaitoni, H¨anggi, Jung, & Marchesoni, 1998.) Plesser and Geisel (1999) have demonstrated that noise can induce a further kind of resonance in periodically stimulated integrate-and-Žre neurons: the optimal SNR is attained for a particular stimulation frequency. This latter resonance arises by matching the period of stimulation to the timescale set by the membrane time constant of the neuron. We shall show here that the simpliŽed Arrhenius&Current model reproduces this behavior well and that it is thus well suited to investigate integrate-and-Žre dynamics. (For details of the methods, see Plesser & Geisel, 1999.) Briey, one proceeds as follows to obtain the SNR in response to sinusoidal stimulation. For periodic input I ( t) = m C q cos(Vt C w), the ISI distributions r (t | t¤ , I (¢)) depend on input and external time only through the input phase y ¤ = [Vt¤ C w 0 ] mod 2p at the time t¤ of the most recent spike. With respect to this spike phase, the spike train reduces to a Markov

380

Hans E. Plesser and Wulfram Gerstner

(a) 0.4

(b) 0.08

E

0.3 0.2

0.04

0.1 0

­2

0 e

0

2

0.4

(d) 0.8

0.3

0.6

0.2

0.4

0.1

0.2

E

(c)

0

­2

0 e

0

2

­2

0 e

2

­2

0 e

2

Figure 4: Error E versus relative distance from threshold e for (a) Arrhenius, (b) Arrhenius&Current, (c) error function, and (d) Tuckwell model. Symbols mark the medians, and vertical bars span the 20th to 80th percentiles of 250 stimuli with neighboring e-values. Circles indicate periodic stimuli, crosses aperiodic ones. e > 0 corresponds to the regime of subthreshold stimuli. Note the different scales of the y-axis.

process with transition probability (or kernel), Z

T (y k | y k¡1 ) =

1 0

r (t | y k¡1 )d (y k ¡ [Vt C y k¡1 ] mod 2p )

dt , V

(3.2)

between the phases of the kth and the k ¡ 1st spike. The power spectral density at the stimulus frequency V of a d-spike train of total length To is then given by 1 STo (V) = p To

*t ,t < T jX k o j,k

+ e

iV (tj ¡tk )

* + No X 1 i(yj ¡yk ) e ¼ . p No ht i j,k= 1

(3.3)

Noise in Integrate-and-Fire Neurons

381

16 14 12

SNR

10 8 6 4 2 0 0

0.05 s

0.1

0.15

p Figure 5: SNR versus noise amplitude s form = 0.95, q 2 = 0.05 and V = 0.33p (highest SNR), V = p (lowest SNR) and V = 0.1p . Solid lines are for the diffusion model, dashed lines for the Arrhenius&Current model.

Here, tj are the spike times and yj the spike phases, and we have exploited that yj ¡ yk = [V (tj ¡ tk )] mod 2p . The mean ISI length is given by ht i, and No = bTo / ht ic is the average number of spikes in To . The approximation in equation 3.3 consists in averaging the number of spikes in To and their phases separately. Since the correlations between the spike phases yj are given by equation 3.2, the right-hand side of equation 3.3 can be evaluated in closed form. Results are in excellent agreement with spectra computed via fast Fourier transform from simulated spike trains. As an estimate of the noise background on which this signal will be transmitted, we use the power spectral density of a Poisson process of the same mean number of spikes per time as the spike train studied in equation 3.3 (see Stemmler, 1996). A Poisson process with intensity ht i¡1 has a white spectrum with power SP = (p ht i) ¡1 (see Cox & Lewis, 1966). The SNR of a spike train transmitted within a Žxed observation period To is then * + No 1 X STo (V) (yj ¡yk ) i e SNR = ¼ . SP No j,k= 1 Figure 5 shows results for the integrate-and-Žre model with diffusion noise and Arrhenius&Current escape noise. The agreement is good, indicating that the Arrhenius&Current model is well suited to replace diffusive noise in the integrate-and-Žre neuron when investigating more intricate

382

Hans E. Plesser and Wulfram Gerstner

problems of neuronal dynamics. Note in particular that both models show a twofold stochastic resonance. The SNR exhibits, as always in stochastic resonance, a peak as a function of the noise amplitude. The height of this peak is maximal around a stimulus frequency V = 0.33p and decreases for both larger (V = p ) and smaller frequencies (V = 0.1p ). The optimal frequency V = 0.33p corresponds to a period of T = 6, which should be compared to the mean ISI length ht i ¼ 9.7 at the resonance. (For a more detailed discussion of the timescale matching underlying this effect, see Plesser & Geisel, 1999.)

4 Discussion

We have demonstrated that the dynamics of the integrate-and-Žre neuron with diffusive noise may be well approximated by simple escape noise models and have identiŽed the Arrhenius&Current hazard function as the optimal choice of model. Upon periodic stimulation, the SNR of the neuron’s output exhibits stochastic resonance for the Arrhenius&Current noise in the same way as it does for diffusive noise. This indicates that this escape noise model renders the correlations within the spike train—induced by the timedependent stimulus—correctly. The relative error of the model depends only weakly on the stimulus, permitting a widespread use. Of the remaining models, the pure Arrhenius model Žnishes as a strong second, at least for subthreshold stimuli. Compared to the error function model, it has the additional charm of mathematical simplicity. The model based on Tuckwell’s approximation of the mean Žring rate, in contrast, is useful only if stimuli are sufŽciently subthreshold. With this study, we have provided a tool for the efŽcient investigation of large networks of model neurons. In fact, escape noise models have been used in the past in the context of the spike response model (Gerstner & van Hemmen, 1992; Gerstner, 1995), and this study provides additional support to such simpliŽed treatment of noise. For escape noise models, signal transmission properties of spiking neurons can be studied analytically (Gerstner, forthcoming). Several issues remain to be tackled, particularly the extension of the investigation presented here to the case of colored noise. Finally, although the Arrhenius&Current model is the best model we have found so far, this is as yet no proof that it is indeed the optimal model.

Acknowledgments

We acknowledge helpful comments by two anonymous referees on an earlier version of the manuscript. H. E. P. was supported by Deutsche Forschungsgemeinschaft through SFB 185 “Nichtlineare Dynamik.”

Noise in Integrate-and-Fire Neurons

383

References Abeles, M. (1982).Role of the cortical neuron: Integrator or coincidence detector? Israel J Med Sci, 18, 83–92. Brunel, N., & Hakim, V. (1999). Fast global oscillations in networks of integrateand-Žre neurons with low Žring rates. Neural Comput, 11, 1621–1671. Bryant, H. L., & Segundo, J. P. (1976).Spike initiation by transmembrane current: A White-noise analysis. J Physiol, 260, 279–314. Bulsara, A. R., & Zador, A. (1996). Threshold detection of wideband signals: A noise-induced maximum in the mutual information. Phys Rev E, 54, R2185– R2188. Collins, J. J., Chow, C. C., Capela, A. C., & Imhoff, T. T. (1996). Aperiodic stochastic resonance. Phys Rev E, 54, 5575–5584. Cordo, P. Inglis, J. T., Verschueren, S., Collins, J. J., Merfeld, D. M., Rosenblum, S., Buckley, S., & Moss, F. (1996). Noise in human muscle spindels. Nature, 383, 769–770. Cox, D. R., & Lewis, P. A. W. (1966). The statistical analysis of series of events. London: Methuen. Douglass, J. K., Wilkens, L., Pantazelou, E., & Moss, F. (1993).Noise enhancement of information transfer in crayŽsh mechanoreceptors by stochastic resonance. Nature, 365, 337–340. Gammaitoni, L., H¨anggi, P., Jung, P., & Marchesoni, F. (1998). Stochastic resonance. Rev Mod Phys, 70, 223–287. Gerstner, W. (1995). Time structure of the activity in neural network models. Phys Rev E, 51, 738–758. Gerstner, W. (1998). Spiking neurons. In W. Maass & C. M. Bishop (Eds.), Pulsed neural networks (pp. 3–54). Cambridge, MA: MIT Press. Gerstner, W. (forthcoming). Population dynamics of spiking neurons: Fast transients, asynchronous states, and locking. Neural Comput. Gerstner, W., & van Hemmen, J. L. (1992). Associative memory in a network of “spiking” neurons. Network, 3, 139–164. Gummer, A. W. (1991). Probability density function of successive intervals of a nonhomogeneous Poisson process under low-frequency conditions. Biol Cybern, 65, 23–30. Johannesma, P. I. M. (1968). Diffusion models of the stochastic activity of neurons. In E. R. Caianiello (Ed.), Neural networks (pp. 116–144). Berlin: Springer. Johnson, D. H. (1980). The relationship between spike rate and synchrony in responses of auditory-nerve Žbers to single tones. J Acoust Soc Am, 68, 1115– 1122. Johnson, D. H. (1996). Point process models of single-neuron discharges. J Comp Neurosci, 3, 275–299. Kempter, R., Gerstner, W., van Hemmen, J. L., & Wagner, H. (1998).Extracting out oscillations: Neural coincidence detection with periodic spike input. Neural Comput, 10, 1987–2017. Konig, ¨ P., Engel, A. K., & Singer, W. (1996). Integrator or coincidence detector? The role of the cortical neuron revisited. Trends Neurosci, 19(4), 130–137.

384

Hans E. Plesser and Wulfram Gerstner

L´ansky, ´ P. (1997). Sources of periodical force in noisy integrate-and-Žre models of neuronal dynamics. Phys Rev E, 55, 2040–2043. Levin, J. E., & Miller, J. P. (1996). Broadband neural encoding in the cricket cercal sensory system enhanced by stochastic resonance. Nature, 380, 165–168. Linz, P. (1985). Analytical and numerical methods for Volterra equations. Philadelphia: SIAM. Mainen, Z. F., & Sejnowski, T. J. (1995). Reliability of spike timing of neocortical neurons. Science, 268, 1503–1506. Moss, F., & Russell, D. F. (1998). Animal behavior enhanced by noise. Paper presented at the Computational Neuroscience Meeting ’98, Santa Barbara, CA. Perkel, D. H., Gerstein, G. L., & Moore, G. P. (1967). Neuronal spike trains and stochastic point processes. Biophys J, 7, 391–418. Plesser, H. E., & Geisel, T. (1999). Markov analysis of stochastic resonance in a periodically driven integrate-Žre neuron. Phys Rev E, 59, 7008–7017. Plesser, H. E., & Tanaka, S. (1997). Stochastic resonance in a model neuron with reset. Phys Lett A, 225, 228–234. Rieke, F., Warland, D., de Ruyter van Steveninck, R., & Bialek, W. (1997). Spikes: Exploring the neural code. Cambridge, MA: MIT Press. Scott, D. W. (1992). Multivariate density estimation. New York: Wiley. Siebert, W. M. (1970). Frequency discrimination in the auditory system: Place or periodicity mechanisms. Proc IEEE, 58, 723–730. Stein, R. B. (1965). A theoretical analysis of neuronal variability. Biophys J, 5, 173–194. Stemmler, M. (1996). A single spike sufŽces: The simplest form of stochastic resonance in neuron models. Network, 7, 687–716. Stevens, C. F., & Zador, A. (1996). When is an integrate-and-Žre neuron like a Poisson neuron? In D. S. Touretzky, M. C. Mozer, & M. E. Hasselmo (Eds.), Advances in neural information processing systems, 8 (pp. 103–109). Cambridge, MA: MIT Press. Tuckwell, H. C. (1988). Introduction to theoretical neurobiology (Vol. 1). Cambridge, UK: Cambridge University Press. Tuckwell, H. C. (1989). Stochastic processes in the neurosciences. Philadelphia: SIAM. van Kampen, N. G. (1992). Stochastic processes in physics and chemistry (2nd ed.). Amsterdam: North-Holland. van Vreeswijk, C., & Sompolinsky, H. (1996). Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science, 274, 1724–1726. Wiesenfeld, K., & Moss, F. (1995). Stochastic resonance and the beneŽts of noise: From ice ages to crayŽsh and SQUIDs. Nature, 373, 33–36. Wilson, H. R., & Cowan, J. D. (1972). Excitatory and inhibitory interactions in localized populations of model neurons. Biophys J, 12, 1–24. Received April 27, 1998; accepted March 31, 1999.

LETTER

Communicated by Misha Tsodyks

Modeling Synaptic Plasticity in Conjunction with the Timing of Pre- and Postsynaptic Action Potentials Werner M. Kistler J. Leo van Hemmen ¨ ¨ Physik Department der TU Munchen, D-85747 Garching bei Munchen, Germany We present a spiking neuron model that allows for an analytic calculation of the correlations between pre- and postsynaptic spikes. The neuron model is a generalization of the integrate-and-Žre model and equipped with a probabilistic spike-triggering mechanism. We show that under certain biologically plausible conditions, pre- and postsynaptic spike trains can be described simultaneously as an inhomogeneous Poisson process. Inspired by experimental Žndings, we develop a model for synaptic long-term plasticity that relies on the relative timing of pre- and postsynaptic action potentials. Being given an input statistics, we compute the stationary synaptic weights that result from the temporal correlations between the pre- and postsynaptic spikes. By means of both analytic calculations and computer simulations, we show that such a mechanism of synaptic plasticity is able to strengthen those input synapses that convey precisely timed spikes at the expense of synapses that deliver spikes with a broad temporal distribution. This may be of vital importance for any kind of information processing based on spiking neurons and temporal coding.

1 Introduction

There is growing experimental evidence that the strength of a synapse is persistently changed according to the relative timing of the arrival of presynaptic spikes and the triggering of postsynaptic action potentials (Markram, Lubke, ¨ Frotscher, & Sakmann, (1997); Bell, Han, Sugawara, & Grant, 1997). A theoretical approach to this phenomenon is far from being trivial because it involves the calculation of correlations between pre- and postsynaptic action potentials (Gerstner, Kempter, van Hemmen, & Wagner 1996). In the context of simple neuron models such as the standard integrate-and-Žre model, the determination of the Žring-time distribution involves the solution of Žrst-passage time problems (Tuckwell, 1988). These problems are known to be hard, the more so if the neuron model is extended so as to include biologically realistic postsynaptic potentials. Neural Computation 12, 385–405 (2000)

c 2000 Massachusetts Institute of Technology °

386

Werner M. Kistler and J. Leo van Hemmen

We are going to use a neuron model, the spike-response model, which is a generalization of the integrate-and-Žre model (Gerstner & van Hemmen, 1992; Gerstner, 1995; Kistler, Gerstner, & van Hemmen, 1997). This model combines analytic simplicity with the ability to give a faithful description of “biological” neurons in terms of postsynaptic potentials, afterpotentials, and other aspects (Kistler, et al., 1997). In order to be able to solve the Žrst-passage time problem, the sharp Žring threshold is replaced by a probabilistic spike-triggering mechanism (section 2). In section 3 we investigate conditions that allow for a description of the postsynaptic spike train as an inhomogeneous Poisson process, if the presynaptic spike trains are described by inhomogeneous Poisson processes too. Using this stochastic neuron model, we can analytically compute the distribution of the Žrst postsynaptic Žring time for a given ensemble of input spike trains in two limiting cases: for low- and high-Žring threshold (section 4). Finally, in section 5 a model of synaptic plasticity is introduced and analyzed in the context of a neuron that receives input from presynaptic neurons with different temporal precision. We show analytically and by computer simulations that the mechanism of synaptic plasticity is able to distinguish between synapses that convey spikes with a narrow or a broad temporal jitter. The resulting stationary synaptic weight vector favors synapses that deliver precisely timed spikes at the expense of the other synapses so as to produce again highly precise postsynaptic spikes. 2 The Model

We will investigate a neuron model—the spike-response model—that is in many aspects similar to but much more general than the standard integrateand-Žre model (Gerstner & van Hemmen, 1992; Gerstner, 1995). The membrane potential is given by a combination of pre- and postsynaptic contributions described by a response kernel 2 that gives the form of an elementary postsynaptic potential and a kernel g that accounts for refractoriness and has the function of an afterpotential, so that h i ( t) =

X j, f

f

Jij2 ( t ¡ tj ¡ D ij ) ¡

X f

f

g(t ¡ ti ).

(2.1)

Here, hi is the membrane potential of neuron i, Jij is the strength of the synapse connecting neuron j to neuron i, D ij is the corresponding axonal f

delay from j to i, and ftj , f = 1, 2, . . .g are the Žring times of neuron j. Causality is respected if both 2 ( s) and g(s) vanish identically for s < 0. Depending on the choice for 2 and g, the spike-response model can reproduce the standard integrate-and-Žre model or even mimic complex HodgkinHuxley-type neuron models (Kistler et al., 1997). A typical choice for 2 is 2 ( t) = t/ tM exp(1 ¡ t/ tM )H (t), where tM is a membrane time constant and

Modeling Synaptic Plasticity

387

H the Heaviside step function with H ( t) = 1 for t > 0 and H(t) = 0 elsewhere. As for the refractory function g, a simple exponential can be used, for example, g(t) = g0 exp(¡t/ tref )H (t). Equation 2.1 could also be generalized so as to account for short-term depression and facilitation if the constant coupling Jij is replaced by a function of the previous spike arrival times at synapse i 7! j (Kistler & van Hemmen, 1999). In this context, however, we will concentrate on situations where only a single spike is transmitted across each synapse, so that effects of short-term plasticity do not come into action. The crucial point for what follows is the way in which new Žring events are deŽned. Instead of using a deterministic threshold criterion for spike release, we describe the spike train of the postsynaptic neuron as a stochastic process with the probability density of having a spike at time t being a nonlinear function º of the membrane potential h( t). If we neglect the effect of the afterpotential, g = 0, the spike train of neuron i is an inhomogeneous Poisson process with rate º = º( hi ) for every Žxed set of presynaptic Žring f

times ftj , j 6 = ig. Spike triggering in neurons is more or less a threshold process. In order to mimic this observation, we have chosen the rate function º( h) to be º(h) = ºmax H ( h ¡ #),

(2.2)

where ºmax is a positive constant and # the Žring threshold. We note that ºmax is not the maximum Žring rate of the neuron, which is given by the form of g(t), but a kind of a reliability parameter. The larger ºmax , the faster the neuron will Žre after the threshold # has been reached from below. The role of the parameter ºmax is further discussed in section 4. 3 Noisy Input f

For a Žxed set of presynaptic spikes at times ftj , j 6 = ig and a negligible afterpotential (g = 0), the membrane potential of neuron i at time t is uniquely determined and given by hi (t) =

X j, f

f

Jij2 ( t ¡ tj ¡ D ij ).

(3.1)

In this case, the spike train of neuron i is a Poisson process with rate ºmax for those time intervals with hi (t) > # and no spikes in between. Things become more complicated if we include an afterpotential after each spike of neuron i and if the arrival times of the presynaptic spikes are no longer Žxed but become random variables too. We will not discuss the Žrst point because we are mainly interested in the Žrst spike of neuron i after a period of inactivity. As a consequence of the second point, the membrane

388

Werner M. Kistler and J. Leo van Hemmen

potential hi ( t) is no longer a Žxed function of time, but a stochastic process f because every realization of presynaptic Žring times ftj , j 6 = ig results in a different realization of the time course of the membrane potential hi ( t) as given through equation 3.1. The composite process is thus doubly stochastic in the sense that in a Žrst step, a set of input spike trains is drawn from an ensemble characterized by Poisson rates ºi . This realization of input spike trains then determines the output rate function, which produces in a second step a speciŽc realization of the postsynaptic spike train. The natural question is how to characterize the nature of the resulting postsynaptic spike train. For an inhomogeneous Poisson process with rate function º, the expectation of the number k of events in an interval [T1 , T2 ] is Z E fk | ºg =

T2 T1

dtº(t).

(3.2)

If the rate function º is replaced by a stochastic process, we have to integrate over all possible realizations of º (with respect to a measure m ) in order to obtain the (unconditional) expectation of the number of events, Z E fkg =

« ¬ dm (º)E fk | ºg = E fk | ºg º =

Z

T2 T1

dt ºN (t),

(3.3)

R with h¢iº = ¢dm (º) and ºN ( t) = hº(t)iº. Hence, the composite process has the same expectation of the number of events as an inhomogeneous Poisson process with rate function ºN (t), which is just the expectation of º( t). Nevertheless, the composite process is not equivalent to a Poisson process. This can be seen by calculating the probabilities of observing k events within [T1 , T2 ]. For a Poisson process with Žxed rate function º, this probability equals 1 prob fk events in [T1 , T2 ] | ºg = k!

"Z

" Z dtº(t) exp ¡

T2 T1

#k

T2 T1

#

dtº(t) , (3.4)

whereas for the composite process, we obtain prob fk events in [T1 , T2 ]g « ¬ = prob fk events in [T1 , T2 ] | ºg º *"Z #nCk + 1 T2 X (¡1)n = dtº(t) k!n! T1 n= 0

(3.5)

º

Z Z T2 1 X « ¬ (¡1)n T2 = dt1 . . . dtnCk º( t1 ) . . .º(tnCk ) º . k!n! T1 T1 n= 0

(3.6)

Modeling Synaptic Plasticity

389

The last expression contains expectations of products of the rate function º evaluated at times t1 , . . . , tnCk . Since the membrane potential h is a continuous function of time, the rate function is (at least) piecewise continuous, and º( ti ) and º( tj ) with ti 6 = tj are in general not statistically independent. The calculation of correlations such as C(ti , tj ) := hº(ti )º( tj )i ¡ hº(ti )ihº( tj )i is thus far from being trivial (Bartsch & van Hemmen, 1998). In the following we approximate the composite process by an inhomogeneous Poisson process with rate function º. N This approximation can be justiŽed for large numbers of overlapping excitatory postsynaptic potentials (EPSPs), when, due to the law of large numbers (Lamperti, 1966), the membrane potential as a sum of independent EPSPs and thus the rate function º( t) is almost always close to its expectation value ºN ( t). The approximation is equivalent to neglecting correlations in the rate function º. That is, to good approximation (as we will see shortly), we are allowed to assume º( t1 ), . . . , º(tkCn ) to be independent for ti 6 = tj , i 6 = j. In doing so, we can replace the expectation of the product by the product of expectations under the time integrals of equation 3.5 and obtain prob fk events in [T1 , T2 ]g Z Z T2 1 X (¡1) n T2 = dt1 . . . dtnCkºN ( t1 ) ¢ . . . ¢ ºN ( tnCk ) k!n! T1 T1 n= 0 1 = k!

"Z

T2 T1

#k

dt ºN ( t)

" Z exp ¡

T2 T1

#

dt ºN (t) ,

(3.7)

which is the probability of observing k events for an inhomogeneous Poisson process with rate function ºN that is simply the expectation of the process º. The efŽcacy of this approximation remains to be tested. This is done throughout the rest of this article by comparing theoretical results based on this approximation with numerical simulations. We now calculate ºN for neuron i. We assume that the spike train of the presynaptic neurons, labeled j, can be described by an inhomogeneous Poisson process with rate function ºj . A description of spike trains of cortical neurons in terms of inhomogeneous Poisson processes is appropriate for time windows below approximately 1 second (Rieke, Warland, de Ruyter van Steveninck, & Bialek, 1997), which is sufŽcient for our purpose. With these preliminaries we can calculate the Žrst and the second moment of the distribution of the resulting membrane potential. The average postsynaptic potential generated by a single presynaptic neuron with rate function ºj is *

X f

+ 2 (t

f ¡ tj )

= (ºj ¤ 2 )( t).

(3.8)

390

Werner M. Kistler and J. Leo van Hemmen

R Here ¤ denotes convolution, that is, ( f ¤ g)(t) = dt0 f (t0 ) g( t ¡t0 ). The second moment is (Kempter, Gerstner, van Hemmen, & Wagner, 1998, appendix A) *2 4

X f

32 + 2 (t ¡

f tj ) 5

£ ¤2 ³ = (ºj ¤ 2 )( t) C ºj ¤ 2

2

´

(t).

(3.9)

Returning to equation 3.1, we see that the expectation value hN i of the membrane potential and the corresponding variance si2 are given by hN i ( t) = E fhi (t)g =

X j

Jij (ºj ¤ 2 )( t)

(3.10)

and si2 ( t) = Var fhi (t)g =

X j

Jij2 (ºj ¤ 2

2 )(

t).

(3.11)

Because of the central limit theorem, the membrane potential at a certain Žxed time t has a gaussian distribution, if there are sufŽciently many EPSPs that superimpose at time t; (for an estimate by the Berry-Esseen inequality we refer to Kempter et al., 1998). Equipped with the gaussian assumption, we can calculate ºN ( t), Z

dh G [h ¡ hN i (t), si ( t)]º(h) ( " #) ºmax hN i ( t) ¡ # = 1 C erf p , 2 2si2 (t)

ºN ( t) =

(3.12)

£ ¤ with G being a normalized gaussian, G (h, s) = (2p s 2 ) ¡1/ 2 exp ¡h2 / (2s 2 ) . We note that this is the Žrst time that we have used explicitly the functional dependence (see equation 2.2) of the Žring rate º on the membrane potential h. The result so far is that if the membrane potential is a sum of many EPSPs so that we can assume h(t) to be gaussian distributed and the membrane potentials for different times to be independent, and if the effect of the afterpotential is negligible, then the postsynaptic spike train is given by an inhomogeneous Poisson process with rate function ºN as given by equation 3.12. 4 First Passage Time

In the previous section we discussed an approximation that allows us to replace the doubly stochastic process that describes the postsynaptic spike

Modeling Synaptic Plasticity

391

train by an inhomogeneous Poisson process with a rate function ºN that is independent of the speciŽc realization of the presynaptic input. This approximation allows us to calculate the distribution of the Žrst postsynaptic spike analytically. The probability of observing no spikes in the interval [t0 , t] is µ Z t ¶ 0 0 prob fno spike in [t0 , t]g = exp ¡ dt ºN ( t ) , t0

(4.1)

and the density for the Žrst Žring time after time t0 is d (1 ¡ prob fno spike in [t0 , t]g) dt µ Z t ¶ = ºN ( t) exp ¡ dt0 ºN ( t0 ) .

pŽrst (t) =

(4.2)

t0

We are going to study the Žrst-spike distribution (Žrst passage time) in a speciŽc context. We consider a single neuron having N independent synapses. Each synapse i delivers a spike train determined by an£ inhomogeneous ¤ Poisson process with rate function ºi ( t) = (2p si2 ) ¡1/ 2 exp ¡t2 / 2si2 , so that on average, one spike arrives at each synapse with a temporal jitter si around t = 0. We can calculate the averaged rate function ºN for this setup,

8 2 39 PN = ( ) ¡ ºmax < J 2 N t # i i 5 , 1 C erf 4 q i= 1 ºN (t) = P ; 2 : 2 N2 2 N i= 1 Ji 2 i ( t)

(4.3)

with expectation and variance being given as convolutions 2 Ni ( t) = (ºi ¤ 2 )( t),

and 2 Ni2 (t) = (ºi ¤ 2

2 )(

t).

(4.4)

The resulting density of the Žrst postsynaptic spike is shown in Figure 1. 4.1 High Threshold. We calculate the density of the Žrst postsynaptic spike explicitly for two limiting cases. First, we assume that the probability of observing at least one postsynaptic spike is very small, Z dt ºN ( t) ¿ 1.

(4.5)

In this case, the neuron will Žre, if it will Žre at all, in the neighborhood of the maximum of the postsynaptic potential. We expand the terms involving the

392

Werner M. Kistler and J. Leo van Hemmen

Figure 1: Plots to compare the density function for the Žrst postsynaptic spike as reconstructed from simulations (solid line) with the analytical approximation using the averaged rate function of equation 4.3 (dotted line). The dashed line represents the mean rate º. N (A, B) N = 100 synapses with strength Ji = 1/ N receive spike trains determined£ by an inhomogeneous Poisson process with rate ¤ function ºi = (2p s 2 ) ¡1/ 2 exp ¡t2 / 2s 2 and s = 1. The EPSPs are described by alpha functions t/ t exp(1 ¡ t/ t ) with time constant t = 1 so that the maximum of the membrane potential amounts to h = 1, if all spikes were to arrive simultaneously. The postsynaptic response is characterized by ºmax = 1 and # = 0.5 in A and # = 0.75 in B. Increasing the threshold obviously improves the temporal precision of the postsynaptic response, but the overall probability of a postsynaptic spike is decreased. (C, D) The same plots as A and B but for N = 10 instead of 100 synapses. The synaptic weights have been scaled so that Ji = 1/ 10. The central approximation that asserts statistical independence of the membrane potential at different times produces surprisingly good results for as little as N = 10 overlapping EPSPs.

membrane potential in the neighborhood of the maximum at t0 to leading order in t, PN Ji2 Ni (t) ¡ # q i= 1 = h0 ¡ h2 ( t ¡ t0 )2 C O (t ¡ t0 ) 3 . P N 2 N2 2 i= 1 Ji 2 i (t)

(4.6)

The averaged rate function ºN can thus be written ºN ( t) =

h io ºmax n 1 C erf h0 ¡ h2 (t ¡ t0 )2 C O ( t ¡ t0 ) 3 2

= ºN0 ¡ ºN 2 ( t ¡ t0 ) 2 C O (t ¡ t0 ) 3 ,

(4.7)

Modeling Synaptic Plasticity

393

with ºN0 =

ºmax [1 C erf ( h0 ) ] , 2

and

ºN2 =

ºmax h2 ¡h2 p e 0. p

(4.8)

It turns out that for ¡0.5 < h0 < 0.5, the averaged rate function ºN can be approximated very well by the clipped parabola ( p ºN 0 ¡ ºN2 ( t ¡ t0 )2 , for | t ¡ t0 | < ºN 0 / ºN2 , (4.9) ºN (t) ¼ 0, elsewhere. Using this approximation, we can calculate the integral over ºN and obtain the distribution of the Žrst postsynaptic spike, s # " h i 2ºN0 ºN0 ºN 2 2 3 ( t ¡ t0 ) ¡ ºN 0 ( t ¡ t0 ) ¡ pŽrst (t) = ºN 0 ¡ ºN2 ( t ¡ t0 ) exp 3 3 ºN2 s ! ºN 0 £H ¡ |t ¡ t0 | . (4.10) ºN 2

(

For h0 < ¡0.5, however, a gaussian function is the better choice, ºN (t) ¼ ºN post G ( t ¡ t0 , spost ), with ºN post

(4.11)

q p = p ºN03 / ºN 2 and spost = ºN 0 / (2ºN 2 ). Since ºN post ¿ 1, we have

pŽrst (t) ¼ ºN ( t).

(4.12)

4.2 Low Threshold. We now discuss the case where the postsynaptic neuron Žres with probability near to one, that is, Z 1 ¡ dt ºN ( t) ¿ 1. (4.13)

In this case, the neuron will Žre approximately at time t0 when the membrane potential crosses the threshold, h( t0 ) = #. We again expand the terms containing the membrane potential to leading order in ( t ¡ t0 ), PN Ji2 Ni ( t) ¡ # q i= 1 = h1 ( t ¡ t0 ) C O (t ¡ t0 ) 2 , PN 2 N2 ( ) 2 i= 1 Ji 2 i t

(4.14)

and assume that the averaged rate function ºN is already saturated outside the region where this linearization is valid. We may therefore put ºN (t) =

ª ºmax © 1 C erf [h1 (t ¡ t0 )] , 2

(4.15)

394

Werner M. Kistler and J. Leo van Hemmen

Figure 2: Approximation of the averaged rate function ºN (left) and the density of the Žrst postsynaptic spike pŽrst (right) for various threshold values. The solid line gives the numerical result; the dashed line represents the analytic approximation. (A, B) Approximation for low-threshold value (# = 0.5) as in equations 4.15 and 4.16. (C, D), Approximation by a clipped parabola for # = 0.65, as in equations 4.9 and 4.10. (E, F) Gaussian approximation for high-threshold values (# = 0.8); see equations 4.11 and 4.12. The number of synapses is N = 100; the other parameters are as in Figure 1.

and obtain for the density of the Žrst postsynaptic spike, µ ¶ ºmax ¡h2 ( t¡t0 )2 1 ( ) ( ) ( )( ) e . pŽrst t = ºN t exp ¡N º t t ¡ t0 ¡ p 2 p h1

(4.16)

Outside the neighborhood of the threshold crossing, that is, for |t ¡ t0 | À h¡1 1 , pŽrst can be approximated by pŽrst( t) ¼ ºmax e¡ºmax ( t¡t0 ) H ( t ¡ t0 ),

|t ¡ t0 | À h¡1 1 .

(4.17)

4.3 Optimal Threshold. As can be seen in Figure 2, the width of the Žrst-spike distribution decreases with increasing threshold #. The temporal precision of the postsynaptic spike can thus be improved by increasing

Modeling Synaptic Plasticity

395

Figure 3: Reliability (A) and precision (B) of the postsynaptic Žring event as a function of the threshold #. The neuron receives input through N = 100 synapses; for details, see the caption to Figure 1. Reliability is deŽned as the R overall Žring probability ºN post = dtpŽrst (t). Precision is the inverse of the length of the interval containing 90% of the postsynaptic spikes, D t = t2 ¡ t1 with R t1 R1 dtpŽrst (t) = t dtpŽrst (t) = 0.05Nºpost . C demonstrates that there is an optimal ¡1 2 threshold in the sense that ºN post / D t exhibits a maximum at # ¼ 0.6. The short and the long dashed lines represent the results obtained through the high- and the low-threshold approximations, respectively; See equations 4.18–4.20.

the Žring threshold #. The overall Žring probability of the postsynaptic neuron, however, drops to zero if the threshold exceeds the maximum value of the membrane potential. The trade-off between the precision of the postsynaptic response and the reliability, that is, the overall Žring probability of the postsynaptic neuron, is illustrated in Figure 3. There is an optimal threshold where the expectation value of the precision, that is, the product of reliability and precision, reaches its maximum (Kempter et al., 1998). A straightforward estimate demonstrates the functional dependence of precision and reliability on the various parameters. For high-threshold values, the density pŽrst for the Žrst postsynaptic spike can be approximated by a gaussian function with variance ºN0 / 2ºN 2 . The precision of the Žring time is thus proportional to

Dt

¡1

p / ºN 2 / ºN 0 =

(

2

2h2 e¡h0 p p [1 C erf (h0 )]

!1/ 2 ,

(4.18)

which depends solely on the time course of the membrane potential and is

396

Werner M. Kistler and J. Leo van Hemmen

independent ofºmax . The reliability of the postsynaptic response amounts to ºN post

q = p ºN 03 / ºN 2 = ºmax

(

!1/ 2 p p [1 C erf ( h0 )]3 2

8h2 e¡h0

,

(4.19)

which is proportional to ºmax . For low threshold, pŽrst( t) is dominated by an exponential decay / e¡ºmax t , resulting in a constant precision, D t¡1 ¼

ºmax , 3.00

(4.20)

and reliability close to unity. 5 Synaptic Long-Term Plasticity

There is experimental evidence (Markram et al., 1997) that the strength of synapses between cortical neurons is changed according to the relative timing of pre- and postsynaptic spikes. We develop a simple model of synaptic long-term plasticity that mimics the experimental observations. Using the densities for the Žrst postsynaptic spike calculated in the previous section, we Žnd analytic expressions for the stationary synaptic weights that result from the statistics of the presynaptic spike train. 5.1 Modeling Synaptic Plasticity. Very much in the thrust of the spikeresponse model, we describe the change of a synaptic weight as the linear response to pre- and postsynaptic spikes. are formalized by P Spike trains f ), with a delta function ( sums of Dirac delta functions, S( t) = d ¡ t t f

for each Žring time t f . The most general ansatz up to and including terms bilinear in the pre- and postsynaptic spike train for the change of the synaptic weight J is µ ¶ Z d 0 0 0 J( t) = Spre (t) d pre C dt k pre ( t ) Spost ( t ¡ t ) dt µ ¶ Z C Spost (t) d post C dt0 k post ( t0 )Spre ( t ¡ t0 ) .

(5.1)

This is a combination of changes that are induced by the pre- (Spre ) and the postsynaptic (Spost ) spike train. Furthermore, d pre (d post) is the amount by which the synaptic weight is changed if a single presynaptic (postsynaptic) spike occurs. This non-Hebbian contribution to synaptic plasticity, which is well documented in the literature (Alonso, Curtis, & Llin´as, 1990; Nelson, Fields, Yu, & Liu, 1993; Salin, Malenka, & Nicoll, 1996; Urban & Barrionuevo, 1996), is necessary to ensure that the Žxed point of the synaptic

Modeling Synaptic Plasticity

397

strength can be reached from very low initial values and even zero postsynaptic activity (take 0 < d pre ¿ 1) and to regulate postsynaptic activity (take ¡1 ¿ d post < 0). Finally, k pre (s) ( k post( s)) is the additional change induced by a presynaptic (postsynaptic) spike that occurs s milliseconds after a postsynaptic (presynaptic) spike. This modiŽcation of synaptic strength depends critically on the timing of pre- and postsynaptic spikes (Dan & Poo, 1992; Bell et al., 1997; Markram et al., 1997). Causality is respected if the kernels k vanish identically for negative arguments. In order to restrict the synaptic weights to the interval [0, 1], we treat synaptic potentiation and depression separately and assume that synaptic weight changes due to depression are proportional to the current synaptic weight J( t), whereas changes due to potentiation are proportional to [1 ¡ J (t)], so that µ ¶LTP µ ¶LTD d d d ¡ J ( t) J( t) = [1 ¡ J( t)] J ( t) J ( t) dt dt dt

(5.2)

with µ

¶LTP/ D µ ¶ Z d LTP/ D LTP/ D 0 ( t )Spost (t ¡ t0 ) = Spre ( t) d pre C dt0 k pre J ( t) dt µ ¶ Z LTP/ D LTP/ D CSpost ( t) d post C dt0 k post ( t0 ) Spre ( t ¡ t0 ) , (5.3)

LTP/ D LTP/ D and d pre/ post , k pre/ post (t0 ) ¸ 0. We are interested in the net change hD Ji of the synaptic weight integrated over some time interval and averaged over an ensemble of presynaptic spike trains, ½ Z µ ¶¾ d hD Ji = dt (5.4) J ( t) . dt

If synaptic weights change only adiabatically, that is, if the change of the synaptic weight is small as compared J and (1 ¡ J), we have D E D E hD Ji = [1 ¡ J( t)] D JLTP ¡ J(t) D JLTD , (5.5) with D E D J LTP/ D =

*Z

µ dt

d J ( t) dt

¶LTP/ D +

LTP/ D

LTP/ D = d pre ºNpre Cd post ºNpost Z Z C dt dt0 º(tI t0 )k LTP/ D (tI t0 ),

(5.6)

398

Werner M. Kistler and J. Leo van Hemmen

where ºN pre,post =

R

dtºpre,post ( t) is the expected number of pre- and postsyLTP/ D LTP/ D naptic spikes, andk LTP/ D ( tI t0 ) = k pre (t¡t0 ) Ck post ( t0 ¡t) is the change of the synaptic weight due to a presynaptic spike at time t and a postsynaptic spike at time t0 . In passing we note that º( tI t0 )d tdt0 is the joint probability of Žnding a presynaptic spike in the interval [t, t C dt) and a postsynaptic spike in [t0 , t0 C dt0 ). 5.2 Stationary Weights. In the following we concentrate on calculating the stationary synaptic weights deŽned by

hD Ji = 0

()

« LTP ¬ DJ J = « LTP ¬ « LTD ¬ . DJ C DJ

(5.7)

To this end, we return to the situation described at the beginning of Section 4 where a neuron receives input through N independent synapses, each spike train being described by an inhomogeneous Poisson process with rate function ºi , and 1 · i · N. We study a single synapse and assume that this synapse conveys (on average) one action potential with a gaussian distribution centered around t = 0 so that º( t) = G (t, spre ). The synaptic weight of this synapse is denoted by J. Furthermore, we assume that the presynaptic volley of action potentials triggers at most one postsynaptic spike, as is the case, for instance, if postsynaptic action potentials are followed by strong hyperpolarizing afterpotentials. This assumption relieves us from the need to discuss the inuence of the shape of the afterpotential g because the postsynaptic spike statistic is now completely described by the Žrst-passage-time pŽrst. We discuss the case where the Žring time of a single presynaptic neuron and the postsynaptic Žring time are virtually independent. This might be considered as an approximation of the case where either the corresponding synaptic weight is small or the number of overlapping EPSPs needed to reach the Žring threshold is large. The joint probability ºi ( t, t0 ) of the Žring of the presynaptic neuron i and the Žrst postsynaptic spike is then simply the product of the probabilities, ºi ( t, t0 ) = ºi ( t) pŽrst( t0 ).

(5.8)

We investigate a learning rule that is inspired by the observation that a synapse is weakened (depressed) if the presynaptic spike arrives a short time after the postsynaptic spike and is strengthened (potentiated) if the presynaptic spike arrives a short time before the postsynaptic spike and, thus, is contributing to the postsynaptic neuron’s Žring (Markram et al., 1997). We can mimic this mechanism within our formalism by setting k LTP ( tI t0 ) = 2

LTP

h i exp ¡(t0 ¡ t)/ t LTP H ( t0 ¡ t),

(5.9)

Modeling Synaptic Plasticity

399

Figure 4: Synaptic plasticity based on the timing of pre- and postsynaptic spikes. The kernels Ck LTP (solid line) and ¡k LTD (dashed line) describe the modiŽcation of synaptic strength as it is caused by the combination of a postsynaptic spike at time 0 and a presynaptic spike arriving at time t. See equations 5.9–5.10 with 2 LTP = 2 LTD = 0.1 and t LTP = t LTD = 1.

and k LTD ( tI t0 ) = 2

LTD

h i exp ¡(t ¡ t0 )/ t LTD H ( t ¡ t0 ),

(5.10)

with 2 LTP/ D > 0 and t LTP/ D as time constants of the “synaptic memory” (see Figure 4). Our choice of adopting exponentials to describe the kernels k LTP and k LTD is motivated by both biological plausibility and the need to keep the mathematical analysis as simple as possible. With these prerequisites, we can calculate the mean potentiation and depression due to the correlation of pre- and postsynaptic spikes (cf. equation 5.6) for the limiting cases of high and low threshold. If we approximate the postsynaptic Žring by a gaussian distribution (cf. equation 4.12), pŽrst (t) = ºNpost G ( t ¡ t0 , spost ),

(5.11)

we Žnd Z Z dt dt0 º( tI t0 )k LTP ( tI t0 ) Z =2

1

LTP

¡1

1 = ºN post2 2 and

Z

Z dt

LTP

h i dt0 G ( t, spre )ºN postG ( t0 ¡ t0 , sŽrst) exp ¡(t0 ¡ t)/ t LTP

1 t

" p # µ ³ ¶ 1 s ´2 1 2s t0 t0 exp ¡ LTP erfc ¡p , 2 t LTP t 2 t LTP 2s

(5.12)

Z dt dt0 º( tI t0 )k LTD ( tI t0 ) 1 = ºN post2 2

LTD

" p # µ ³ ¶ 1 s ´2 1 2s t0 t0 exp C LTD erfc Cp , (5.13) 2 t LTD t 2 t LTD 2s

400

Werner M. Kistler and J. Leo van Hemmen

Figure 5: Stationary synaptic weights. (A) Three-dimensional plot of the stationary synaptic weight for high threshold as a function of s and t0 , where 2 C s2 s 2 = spre post is the sum of the variances of pre- and postsynaptic Žring time and t0 the time between the arrival of the presynaptic spike and the Žring of the postsynaptic action potential. (B) Contour plot of the same function as in A. (C, D) Plot of the stationary synaptic weight in the case of low-threshold # as a function of the standard deviation spre of the presynaptic Žring time and the time t0 when the expectation of the membrane potential reaches the threshold LTD = from below. The parameters used to describe synaptic plasticity ared post 0.01, LTP LTD LTP LTP LTD LTP LTD d pre = 0.001, d pre = d post = 0, 2 = 2 = 0.1, t = t = 1; See equations 5.2, 5.9, and 5.10. 2 C s 2 . Using equation 5.7, we are thus able to calculate the with s 2 = spre post stationary weight of a synapse that conveys a single action potential with 2 , given the postsynaptic spike at a gaussian distribution with variance spre 2 t = t0 with variance spost . This result is illustrated in Figure 5. If the variance 2 2 of both pre- and postsynaptic spike is small, that is, s 2 = spre C spost ¿ LTP/ D 1, the stationary weight is dominated by the kernels k : The weight is close to zero if the presynaptic spike occurs after the postsynaptic one (t0 < 0) and close to unity if the presynaptic spike arrives a short time before the postsynaptic action potential (t0 > 0). For large variances, this dependence on the timing of the spikes is smoothed, and the stationary

Modeling Synaptic Plasticity

401

weight is determined by the rates of pre- and postsynaptic spikes. We obtain a similar result for low threshold if we approximate the postsynaptic Žring time by an exponential distribution (cf. equation 4.17), Z

Z dt dt0 º( tI t0 )k LTP ( tI t0 ) Z =2 =

LTP

1

Z dt

¡1

1

dt0 G ( t, spre )ºmax e¡ºmax ( t ¡t0 ) e¡(t ¡t)/ t 0

0

LTP

max(t,t0 )

2 LTPºmax

(

ºmax C 1/ t LTP

µ ³ ¶ 1 1 spre ´2 t0 exp ¡ 2 2 t LTP t LTP " # s t0 £ erfc p ¡p 2t LTP 2spre µ ¶ ¢2 1 1¡ C exp spreºmax C t0ºmax 2 2 " #) spreºmax t0 £ erfc p Cp , 2 2spre

(5.14)

and Z

Z dt dt0 º( tI t0 )k LTD ( tI t0 ) =

2 LTDºmax

ºmax ¡ 1/ t LTD

(

µ ³ ¶ 1 1 spre ´2 t0 exp C 2 2 t LTD t LTD " # s t0 £ erfc p Cp 2t LTD 2spre µ ¶ ¢2 1 1¡ ¡ exp spreºmax C t0ºmax 2 2 " #) spreºmax t0 £ erfc p Cp . 2 2spre

(5.15)

5.3 Synaptic Weights as a Function of the Input Statistics. We saw in the previous section that the stationary synaptic weights can be calculated as a function of the parameters characterizing the distribution of the pre- and postsynaptic spikes. The synaptic weights, on the other hand, determine the distribution of the postsynaptic spike. If we are interested in the synaptic weights that are produced by given input statistics, we thus have to solve a self-consistency problem.

402

Werner M. Kistler and J. Leo van Hemmen

The self-consistency problem can be solved numerically for the limiting cases of low and high threshold, where the distribution of the postsynaptic Žring time is completely characterized by only a few parameters. That is, starting from a given vector J of synaptic couplings, we calculate the parameters that characterize the postsynaptic Žring time using the results of section 4 in the limiting cases of high- and low-Žring threshold. In a second step, we calculate the corresponding stationary synaptic weight vector J as it is determined by the statistics of pre- and postsynaptic Žring times; (cf. section 5.2). Finally, we solve the self-consistency equation J = J using standard numerics. In the simple case where all synapses deliver spikes with a common statistics, the stable solution to the self-consistency equation is unique because the mean postsynaptic Žring time is a monotone function of the synaptic weight. But if the synapses are different, say, with respect to the mean spike arrival time, the solution in general will no longer be unique. We illustrate the solution of the self-consistency problem by means of an example and consider a neuron that receives spikes via N independent synapses. Again, each presynaptic spike train is described by an inhomogeneous Poisson process with a gaussian rate G ( t, si ) centered at t = 0. In contrast to our previous examples, the synapses are different with respect to the temporal precision si with which they deliver their action potentials. Figure 6A shows the resulting synaptic weights if we start with two groups of presynaptic neurons that deliver action potentials with s1 = 0.1 and s2 = 1.0, respectively. The number of neurons contained in group 1 and group 2 are n1 = 20 and n2 = 80, respectively, but the results turned out to be insensitive to the actual numbers. (A more balanced ratio of synapses would shift the curve J1 (#) slightly to the left). We compare the results obtained from the unique solution of the self-consistency equation with simulations of a straightforward implementation of the synaptic dynamics given in equation 5.2. The number of iterations required for the synaptic weights to settle down in the stationary values depends on the amplitude of the synaptic weight change described by d LTP/ D and 2 LTP/ D and on the amount of postsynaptic activity and, thus, on the Žring threshold. With the parameters used in Figure 6, stationarity is usually reached in fewer than 1000 iterations. Finally, a brief discussion of the results shown in Figure 6 is in order. For a high Žring threshold, the synaptic weights of both groups are close to their maximum value because otherwise the neuron would not Žre at LTP > 0. For very all. This is due to the constant presynaptic potentiation d pre low threshold, the postsynaptic Žring is triggered by the Žrst spikes that arrive at the neuron. These spikes stem from presynaptic neurons with a broad Žring-time distribution (group 2). Spikes from group 1 neurons thus arrive mostly after the postsynaptic neuron has already Žred and the corresponding synapses are depressed accordingly. This explains why synapses

Modeling Synaptic Plasticity

403

Figure 6: (A) Synaptic weights for a neuron receiving input from two groups of synapses. One group (n1 = 20) delivers precisely timed spikes (spre = 0.1) and the other one (n2 = 80) spikes with a broad distribution of arrival times (spre = 1.0). The synapses are subject to a mechanism that modiŽes the synaptic strength according to the relative timing of pre- and postsynaptic spikes, see equation 5.2. The upper trace shows the resulting synaptic weight for the group of precise synapses; the lower trace corresponds to the second group. The solid lines give the analytic result for either the low-threshold (# < 0.4) or the high-threshold (# > 0.4) approximation. The dashed lines show the results of a computer simulation. The parameters for the synaptic plasticity are the same as in Figure 5. (B, C, D) precision D t¡1 , reliability ºN post , and efŽciency ºN post / D t as a function of the threshold # for the same neuron as in A. See Figure 3 for the deŽnition of ºN post and D t.

delivering precisely timed spikes are weaker than group 2 synapses, unless the Žring threshold exceeds a certain critical value. The most interesting Žnding is that in the intermediate range of the Žring threshold, synapses that deliver precisely timed spikes are substantially stronger than synapses with poor temporal precision. Furthermore, there is an optimal threshold at # ¼ 0.2 where the ratio of the synaptic strengths of precise and unprecise synapses is largest. The Žring of the postsynaptic neuron is thus driven mainly by spikes from group 1 neurons, and the jitter of the postsynaptic spikes is minimal (see Figure 6B). 6 Discussion

We have presented a stochastic neuron model that allows for a description of pre- and postsynaptic spike trains by inhomogeneous Poisson processes

404

Werner M. Kistler and J. Leo van Hemmen

provided that the number of overlapping postsynaptic potentials is not too small. In view of the huge number of synapses that a single neuron carries on its dendritic tree and the number of synchronous excitatory postsynaptic potentials required to reach threshold—104 . . . 105 synapses and several tens of EPSPs—the approximations used seem to be justiŽed. Since Žring of a neuron is triggered by a large number of synchronous presynaptic action potentials, the postsynaptic Žring time and the arrival time of a single presynaptic spike can be assumed to be independent and the correlation of pre- and postsynaptic Žring times is readily calculated. In addition to the spike-triggering mechanism, we have developed a model for synaptic long-term plasticity based on the relative timing of preand postsynaptic Žring events. The dynamics of the synaptic weights can be analyzed in connection with the former neuron model. In particular, stationary synaptic weights can be calculated as limit states for a given ensemble of presynaptic spike trains. It turns out that this mechanism is able to tune the synaptic weights according to the temporal precision of the spikes they convey. Synapses that deliver precisely timed action potentials are favored at the expense of synapses that deliver spikes with a large temporal jitter. This learning process does not rely on an external teacher signal; it is unsupervised. Furthermore, there is only one free parameter that has to be Žxed: the Žring threshold #. It is conceivable that a small, local network of inhibitory interneurons could be entrusted with setting the Žring threshold. The result concerning the tuning of synaptic weights has two consequences that are open to experimental veriŽcation. First, it shows that neurons are able to tune their synaptic weights so as to select inputs from those presynaptic neurons that provide information that is meaningful in the sense of a temporal code. Second, this tuning ensures the preservation of temporally encoded information because noisy synapses are depressed and the neuron is thus able to Žre its action potential with high temporal precision. We believe that this mechanism has clear relevance to any information processing based on temporal spike coding. Acknowledgments

W. M. K. gratefully acknowledges Žnancial support from the Boehringer Ingelheim Fonds. References Alonso, A., Curtis, M. de, & Llin a´ s, R. (1990). Postsynaptic Hebbian and nonHebbian long-term potentiation of synaptic efŽcacy in the entorhinal cortex in slices and in the isolated adult guinea pig brain. Proc. Natl. Acad. Sci. USA, 87, 9280–9284.

Modeling Synaptic Plasticity

405

Bartsch, A. P., & van Hemmen, J. L. (1998). Correlations in networks of spiking neurons. Unpublished manuscript. Munich: Physics Department, Technical University of Munich. Bell, C. C., Han, V. Z., Sugawara, Y., & Grant, K. (1997). Synaptic plasticity in a cerebellum-like structure depends on temporal order. Nature, 387, 278–281. Dan, Y., & Poo, M. (1992). Hebbian depression of isolated neuromuscular synapses in vitro. Science, 256, 1570–1573. Gerstner, W. (1995). Time structure of the activity in neural network models. Phys. Rev. E, 51, 738–758. Gerstner, W., & van Hemmen, J. L. (1992). Associative memory in a network of “spiking” neurons. Network, 3. 139–164. Gerstner, W., Kempter, R., van Hemmen, J. L. & Wagner,H. (1996). A neuronal learning rule for sub-millisecond temporal coding. Nature, 384, 76–78. Kempter, R., Gerstner, W., van Hemmen, J. L., & Wagner, H. (1998). Extracting oscillations: Neuronal coincidence detection with noisy periodic spike input. Neural Comput., 10, 1987–2017. Kistler, W. M., Gerstner, W., & van Hemmen, J. L. (1997). Reduction of the Hodgkin-Huxley equations to a single-variable threshold model. Neural Comput., 9, 1015–1045. Kistler, W. M., & van Hemmen, J. L. (1999). Short-term synaptic plasticity and network behavior. Neural Comput., 11, 1579–1594. Lamperti, J. (1966). Probability: A survey of the mathematical theory. New York: Benjamin. Markram, H., Lubke, ¨ J., Frotscher, M., & Sakmann B. (1997).Regulation of synaptic efŽcacy by coincidence of postsynaptic APs and EPSPs. Science, 275, 213– 215. Nelson, P. G., Fields, R. D., Yu, C., & Liu, Y. (1993). Synapse elimination from the mouse neuromuscular junction in vitro: A non-Hebbian activity-dependent process. J. Neurobiol., 24, 1517–1530. Rieke, F., Warland, D., de Ruyter van Steveninck, R., & Bialek W. (1997).Spikes— Exploring the neural code. Cambridge, MA: MIT Press. Salin, P. A., Malenka, R. C., & Nicoll, R. A. (1996). Cyclic AMP mediates a presynaptic form of LTP at cerebellar parallel Žber synapses. Neuron, 16, 797–803. Tuckwell, H. C. (1988).Introductionto theoreticalneurobiology, (Vol. 2.) Cambridge: Cambridge University Press. Urban, N. N., & Barrionuevo, G. (1996). Induction of Hebbian and non-Hebbian mossy Žber long-term potentiation by distinct patterns of high-frequency stimulation. J. Neurosci., 16, 4293–4299. Received August 3, 1998; accepted April 29, 1999.

LETTER

Communicated by John Platt

On-line EM Algorithm for the Normalized Gaussian Network Masa-aki Sato ATR Human Information Processing Research Laboratories, 2-2 Hikaridai, Seika-cho, Soraku-gun, Kyoto 619-0288, Japan Shin Ishii Nara Institute of Science and Technology, 8916-5 Takayama-cho, Ikoma-shi, Nara 6300101, Japan A normalized gaussian network (NGnet) (Moody & Darken, 1989) is a network of local linear regression units. The model softly partitions the input space by normalized gaussian functions, and each local unit linearly approximates the output within the partition. In this article, we propose a new on-line EM algorithm for the NGnet, which is derived from the batch EM algorithm (Xu, Jordan, & Hinton 1995), by introducing a discount factor. We show that the on-line EM algorithm is equivalent to the batch EM algorithm if a speciŽc scheduling of the discount factor is employed. In addition, we show that the on-line EM algorithm can be considered as a stochastic approximation method to Žnd the maximum likelihood estimator. A new regularization method is proposed in order to deal with a singular input distribution. In order to manage dynamic environments, where the input-output distribution of data changes over time, unit manipulation mechanisms such as unit production, unit deletion, and unit division are also introduced based on probabilistic interpretation. Experimental results show that our approach is suitable for function approximation problems in dynamic environments. We also apply our on-line EM algorithm to robot dynamics problems and compare our algorithm with the mixtures-of-experts family. 1 Introduction

The main aim of this work is to develop a fast on-line learning method that can deal with dynamic environments. We use a normalized gaussian network (NGnet) (Moody & Darken, 1989) for this purpose. The NGnet is a network of local linear regression units. The model softly partitions the input space by normalized gaussian functions, and each local unit linearly approximates the output within the partition. Since the NGnet is a local model, it is possible to change parameters of several units in order to learn a single datum. Therefore, the learning process becomes easier than that of global models such as the multilayered perceptron. In a local model, on the other hand, the number of necessary units grows exponentially as the Neural Computation 12, 407–432 (2000)

c 2000 Massachusetts Institute of Technology °

408

Masa-aki Sato and Shin Ishii

input dimension increases if one wants to approximate the input-output relationship over the whole input space. This results in a computational explosion, which is often called the curse of dimensionality. However, actual data will often distribute in a lower dimensional space than the input space, such as in attractors of dynamical systems. The NGnet, a kind of mixtures-of-experts model (Jacobs, Jordan, Nowlan, & Hinton, 1991; Jordan & Jacobs, 1994), can be interpreted as an output of a stochastic model with hidden variables. The model parameters can be determined by the maximum likelihood estimation method. In particular, the expectation-maximization (EM) algorithm for the NGnet was derived by Xu, Jordan, & Hinton (1995). In this article, we propose a new on-line EM algorithm for the NGnet. The on-line EM algorithm is derived from the batch EM algorithm (Xu et al., 1995) by introducing a discount factor. We show that the on-line EM algorithm is equivalent to the batch EM algorithm if a speciŽc scheduling of the discount factor is employed. In addition, we show that the on-line EM algorithm can be considered as a stochastic approximation method to Žnd the maximum likelihood estimator. Similar on-line EM algorithms have been proposed by Nowlan (1991) and Jordan and Jacobs (1994) for mixture models. However, there have been no convergence proofs for these on-line EM algorithms. Neal and Hinton (1998) proposed a wide variety of incremental EM algorithms and proved the convergence of their algorithms. One drawback of their algorithms is that they either need additional storage variables for all the training data or need to see all the training data at each iteration. This prevents these algorithms from being applied to realtime on-line settings where new data are supplied indeŽnitely each time. For practical applications, important modiŽcations to the on-line EM algorithm are necessary. A new regularization method is proposed in order to deal with a singular input distribution. In order to manage dynamic environments, where the input-output distribution of data changes over time, unit manipulation mechanisms such as unit production, unit deletion, and unit division are also introduced based on the probabilistic interpretation. Applicability of our new on-line EM algorithm is investigated using function approximation problems in three circumstances: a usual function approximation for a set of observed data; a function approximation in which the distribution of the input data changes over time (this experiment is performed to check the applicability of our approach to dynamic environments); and a function approximation in which the distribution of the input data is singular (i.e., the dimension of the input data distribution is smaller than the input space dimension). In this case, a straightforward application of the basic on-line EM algorithm does not work, because the covariance matrices used in the NGnet become singular. Our modiŽed on-line EM algorithm works well even in this situation. Finally, we applied our on-line EM algorithm to realistic and difŽcult problems in robot dynamics. The performance of our on-line EM algorithm is compared with those of the mixtures-of-experts family.

On-line EM Algorithm for the Normalized Gaussian Network

409

2 Normalized Gaussian Network

The NGnet model, which transforms an N-dimensional input vector x to a D-dimensional output vector y, is deŽned by the following equations: y=

M X i= 1

i ( x)

N

N

Q

i ( x) Wi xQ

´ Gi ( x)/

M X

(2.1a)

Gj ( x)

(2.1b)

j= 1

µ ¶ 1 Gi ( x) ´ (2p ) ¡N/ 2 |S i | ¡1/ 2 exp ¡ ( x ¡ m i ) 0 S i¡1 ( x ¡ m i ) . 2

(2.1c)

M denotes the number of units, and the prime (0 ) denotes a transpose. Gi ( x) is an N-dimensional gaussian function; its center is an N-dimensional vector m i , and its covariance matrix is an ( N £ N)-dimensional matrix S i . | S i | is the determinant of the matrix S i . N i (x) is the ith normalized gaussian Q i ´ ( Wi , bi ) is a ( D £ ( N C 1))-dimensional linear regression function. W 0 Q i xQ ´ Wi x Cbi . The gaussian function Gi (x) is a kind matrix, xQ ´ ( x0 , 1), and W of radial basis function (Poggio & Girosi, 1990). However, the normalized gaussian functions, N i ( x) ( i = 1, . . . , M ), do not have a radial symmetry. They softly partition the input space into M regions. The ith unit linearly Q i xQ within the corresponding region. An output approximates its output by W of the NGnet is given by a summation of these outputs weighted by the normalized gaussian functions. The NGnet (see equation 2.1) can be interpreted as a stochastic model in which a pair of an input and an output, ( x, y), is a stochastic (incomplete) event. For each event, a single unit is assumed to be selected from a set of classes fi | i = 1, . . . , Mg. The unit index i is regarded as a hidden variable. A triplet (x, y, i) is called a complete event. The stochastic model is deŽned by the probability distribution for a complete event (Xu et al., 1995), P(x, y, i|h ) = (2p ) ¡(DCN)/ 2 si¡D |S i | ¡1/ 2 M¡1 µ ¶ 1 1 Q i xQ ) 2 , £ exp ¡ (x¡m i ) 0 S i¡1 ( x¡m i ) ¡ 2 ( y¡ W 2 2si

(2.2)

Q i | i = 1, . . . , Mg is a set of model parameters. From where h ´ fm i , S i , si2 , W the probability distribution, 2.2, the following probabilities are obtained: P(i|h ) = 1/ M P(x|i, h ) = Gi ( x)

(2.3a) µ

P( y|x, i, h ) = (2p ) ¡D/ 2 si¡D exp ¡

¶

1 Q i xQ ) 2 . (y ¡ W 2si2

(2.3b) (2.3c)

410

Masa-aki Sato and Shin Ishii

These probabilities deŽne a stochastic data generation model. First, a unit is randomly selected with an equal probability (see equation 2.3a). If the ith unit is selected, the input x is generated according to the gaussian distribution (see equation 2.3b). Given the unit index i and the input x, the output y is generated according to the gaussian distribution (see equation 2.3c), Q i xQ and the variance s 2 . which has the mean W i When the input x is observed, the probability that the output value becomes y in this model turns out to be P ( y| x, h ) =

M X

N

i ( x) P( y|x,

i, h ).

(2.4)

i= 1

From this conditional probability, the expectation value of the output y for a given input x is obtained as Z M X Q i x, Q N i ( x) W (2.5) E[y|x] ´ yP( y|x, h )dy = i= 1

which is equivalent to the output of the NGnet (see equation 2.1a). The probability distribution (see equation 2.2) provides a stochastic model for the NGnet. 3 EM Algorithm

From a set of T events (observed data), (fxg, fyg) ´ f(x(t), y(t)) | t = 1, . . . , Tg, the model parameter h of the stochastic model (see equation 2.2) can be determined by the maximum likelihood estimation method. In particular, the EM algorithm (Dempster, Laird, & Rubin 1977) can be applied to models having hidden variables. The EM algorithm repeats the following E (expectation)-step and M (maximization)-step. Since the likelihood for the set of observations increases (or does not change) after an E- and M-step, the maximum likelihood estimator is asymptotically obtained by repeating the E- and M-steps: • E-step. Let hN be the current estimator value. By using hN , the posterior probability that the ith unit is selected for each observation ( x( t), y(t)) is calculated according to Bayes rule: P( i| x( t), y(t), hN ) = P(x( t), y( t), i|hN )/

M X j= 1

P( x(t), y( t), j|hN).

(3.1)

• M-step. By using the posterior probability (see equation 3.1), the expected log-likelihood Q(h |hN , fxg, fyg) for the complete events is deŽned by Q(h |hN , fxg, fyg) =

T X M X t= 1 i= 1

P(i| x( t), y(t), hN ) log P( x( t), y(t), i|h ).

(3.2)

On-line EM Algorithm for the Normalized Gaussian Network

411

On the other hand, the log-likelihood of the observed data (fxg, fyg) is given by L(h |fxg, fyg) = =

T X

log P( x(t), y( t)|h )

t= 1 T X

log

t= 1

(

M X

! P( x( t), y(t), i|h ) .

(3.3)

i= 1

Since an increase of Q(h |hN , fxg, fyg) implies an increase of the loglikelihood L(h | fxg, fyg) (Dempster et al., 1977), Q(h |hN , fxg, fyg) is maximized with respect to the estimator h . A solution of the necessity condition @Q/ @h = 0 is given (Xu et al., 1995) by m i = hxii (T )/ h1ii ( T )

(3.4a) 0

S i = h(x ¡ m i )( x ¡ m i ) ii ( T )/ h1ii ( T )

= hxx0 ii ( T )/ h1ii ( T ) ¡ m i ( T )m 0i ( T )

Q i hxQ xQ ii ( T ) = hyxQ ii (T ) W 0

0

1 Q i x| Q 2 ii (T )/ h1ii (T ) h|y ¡ W D ³ ´i 1 h 2 Q i hxy Q 0 ii ( T ) / h1ii ( T ), = h|y | ii (T ) ¡ Tr W D

(3.4b) (3.4c)

si2 =

(3.4d)

where Tr(¢) denotes a matrix trace. A symbol h¢ii denotes a weighted mean with respect to the posterior probability (see equation 3.1), and it is deŽned by h f ( x, y)ii (T ) ´

T 1X f ( x(t), y( t)) P( i|x( t), y( t), hN ). T t= 1

(3.5)

If an inŽnite number of data, which are drawn independently according to an unknown data distribution density r ( x, y), are given, the weighted mean (see equation 3.5) converges to the following expectation value, T!1

h f ( x, y)ii ( T ) ¡! E[ f (x, y)P( i|x, y, hN )]r ,

(3.6)

where E[¢]r denotes the expectation value with respect to the data distribution density r ( x, y). In this case, the M-step equation (see equation 3.4) can be written as gi ´ E[P( i|x, y, hN )]r

(3.7a)

412

Masa-aki Sato and Shin Ishii

m i = E[xP( i|x, y, hN )]r / gi

(3.7b)

S i = E[xx0 P( i|x, y, hN )]r / gi ¡ m im 0i

(3.7c)

Q i E[xQ xQ 0 P( i|x, y, hN )]r = E[yxQ 0 P( i|x, y, hN )]r W ´ 1 ³ Q i E[xy Q 0 P( i|x, y, hN )]r ) / gi , si2 = E[|y| 2 P( i|x, y, hN )]r ¡ Tr( W D

(3.7d) (3.7e)

Q i , s 2 |i = 1, . . . , Mg, is calculated where the new parameter set, h = fm i , S i , W i NQ , s 2 |i = 1, . . . , by using the old parameter set, hN = fmN i , SN i , W Mg. i Ni At an equilibrium point of this EM algorithm, the old and new parameters become the same: h = hN . The equilibrium condition of the EM algorithm (equation 3.7 along with h = hN ) is equivalent to the maximum likelihood condition, @L(h )/ @h = 0,

(3.8)

where the log-likelihood function is given by " L(h ) = E log

M X

# P( x, y, i|h )

i= 1

.

(3.9)

r

4 On-line EM Algorithm 4.1 Basic Algorithm. The EM algorithm introduced in the previous section is based on batch learning (Xu et al., 1995); the parameters are updated after seeing all the observed data (fxg, fyg). In this section, we derive an online version of the EM algorithm. Since the estimator is changed after each observation, let h (t) be the estimator after the tth observation ( x(t), y( t)). In the on-line EM algorithm, the weighted mean (see equation 3.5) is replaced by

hh f ( x, y)iii (T ) ´ g(T)

T X t= 1

(

T Y

! l(s)

s= tC1

£ f ( x( t), y(t)) P( i|x(t), y( t), h ( t ¡ 1)) g(T) ´

(

T Y T X

!¡1 l(s)

.

(4.1a) (4.1b)

t= 1 s= tC1

Here, the parameter l ( t) (0 · l ( t) · 1) is a time-dependent discount factor, which is introduced for forgetting the effect of the old posterior values employing the earlier inaccurate estimator. g(T) is a normalization coefŽcient and plays a role like a learning rate.

On-line EM Algorithm for the Normalized Gaussian Network

413

The modiŽed weighted mean hh¢iii can be obtained by the step-wise equation, hh f (x, y)iii ( t) = hh f ( x, y)iii (t ¡ 1) £ ¤ Cg(t) f ( x(t), y( t)) Pi ( t) ¡ hh f ( x, y)iii (t ¡ 1)

g(t) = (1 Cl ( t)/g( t ¡ 1))¡1 ,

(4.2a) (4.2b)

where Pi ( t) ´ P(i| x( t), y(t), h ( t ¡ 1)). The constraint, 0 · l(t) · 1, gives the constraint on g(t): 1 ¸ g(t) ¸ 1/ t.

(4.3)

If this constraint is satisŽed, equation 4.2b can be solved for l (t) as l(t) = g(t ¡ 1)(1/ g(t) ¡ 1).

(4.4)

After calculating the modiŽed weighted means, that is, hh1iii ( t), hhxiii ( t), hhxx0 iii ( t), hh|y| 2 iii (t), and hhyxQ 0 iii (t), according to equation 4.2, the new estimator h (t) is obtained by the following equations: m i ( t) = hhxiii (t)/ hh1iii ( t) S i¡1 ( t)

0

(4.5a) 0

= [hhxx iii (t)/ hh1iii ( t) ¡ m i ( t)m i ( t

)]¡1

(4.5b)

Q i ( t) = hhyxQ 0 iii ( t)[hhxQ xQ 0 iii ( t)]¡1 W ³ ´i 1 h Q i ( t)hhxy Q 0 iii ( t) / hh1iii ( t). si2 ( t) = hh|y| 2 iii (t) ¡ Tr W D

(4.5c) (4.5d)

This deŽnes the on-line EM algorithm. In this algorithm, calculations of the inverse matrices are necessary at each time-step. A recursive formula Q i can be derived using a standard method (Jurgen, for S i¡1 and W ¨ 1996), so there is no need to calculate the matrix inverse. Let us deŽne the weighted Q LQ i ( t) ´ (hhxQ xQ 0 iii (t)) ¡1 . This quantity can be inverse covariance matrix of x: obtained by the step-wise equation: LQ i ( t) =

" # 1 Pi (t) LQ i (t¡1) xQ ( t)xQ 0 ( t) LQ i ( t¡1) Q L i ( t¡1) ¡ . (4.6) 1¡g(t) (1/ g(t) ¡1) CPi ( t) xQ 0 ( t) LQ i ( t¡1) xQ (t)

S i¡1 ( t) can be obtained from the following relation with LQ i (t): LQ i ( t)hh1iii ( t) =

¡

S i¡1 (t) ¡m 0i ( t)S i¡1 ( t)

¡S i¡1 ( t)m i ( t) 1 Cm 0i ( t)S i¡1 ( t)m i (t)

¢

.

(4.7)

414

Masa-aki Sato and Shin Ishii

Q i , is given by The estimator for the linear regression matrix, W Q i ( t) = hhyxQ 0 iii (t) LQ i (t) W Q i ( t ¡ 1) Cg(t)Pi ( t)( y( t) ¡ W Q i ( t ¡ 1) xQ (t)) xQ 0 (t) LQ i (t). = W

(4.8a) (4.8b)

Thus, the basic on-line EM algorithm is summarized as follows. For the tth observation (x( t), y( t)), the weighted means—hh1iii ( t), hhxiii (t), hh|y| 2 iii (t), Q 0 iii (t)—are calculated by using the step-wise equation 4.2. LQ i (t) is and hhxy also calculated with the step-wise equation 4.6. Subsequently, the estimators for the model parameters are obtained by equations 4.5a, 4.5d, 4.7, and 4.8b. 4.2 Equivalence of On-line and Batch EM Algorithms. In this part, we show that the basic on-line EM algorithm deŽned by equations 3.1, 4.2, and 4.5 is equivalent to the batch EM algorithm deŽned by equations 3.1, 3.4, and 3.5, with an appropriate choice of l(t). It is assumed that the same set of data, f(x(t), y( t))| t = 1, . . . , Tg, is repeatedly supplied to the learning system, that is, x( t C T ) = x( t) and y( tCT ) = y( t). The time t is represented by an epoch index k (= 1, 2, . . .) and a data number index m (= 1, . . . , T ) within an epoch, that is, t = (k ¡ 1)T Cm. Let us assume that the model parameter h is updated at the end of each epoch, when m = T, and it is denoted by h ( k). Let us suppose that l ( t) is given by » 0 if t = ( k ¡ 1)T C 1 l ( t) = (4.9) 1 otherwise.

The correspondingg(t) is given byg((k¡1)T Cm) = 1/ m. Then the weighted mean hh f (x, y)iii ( t) is initialized to f (x(1), y(1)) P(i| x(1), y(1), h ( k ¡ 1)) at the beginning of an epoch. At the end of an epoch, hh f (x, y)iii ( kT ) = h f ( x, y)ii ( T ) is satisŽed for hN = h ( k ¡ 1). This shows that the on-line EM algorithm, together with l(t) given by equation 4.9, is equivalent to the batch EM algorithm. Consequently, the estimator of the on-line EM algorithm converges to the maximum likelihood estimator. It should be noted that the step-wise equation, 4.6, for LQ i ( t) cannot be used in this case, since the equation becomes singular for g(t) = 1. 4.3 Stochastic Approximation. If an inŽnite number of data, which are drawn independently according to the data distribution density r ( x, y), are available, the on-line EM algorithm can be considered as a stochastic approximation (Kushner & Yin, 1997) for obtaining the maximum likelihood estimator, as demonstrated below. Let w be a compact notation for the weighted mean: w ( t) ´ fhh1iii ( t), hhxiii ( t), hhxx0 iii ( t), hhyxQ 0 iii ( t), hh|y| 2 iii ( t)| i = 1, . . . , Mg. The on-line EM algorithm, equations 4.2 and 4.5, can be written in an abstract form:

d w ( t) ´ w (t) ¡w (t¡1) = g(t)[F(x(t), y( t), h ( t¡1)) ¡w ( t¡1)]

(4.10a)

On-line EM Algorithm for the Normalized Gaussian Network

h (t) = H(w ( t)).

415

(4.10b)

It can be easily proved that the set of equations, w = E[F( x, y, h )]r h = H (w ),

(4.11a) (4.11b)

is equivalent to the maximum likelihood condition, equation 3.8. Then the on-line EM algorithm can be written as dw (t) = g(t)(E[F(x, y, H(w ( t ¡ 1)))]r ¡ w (t ¡ 1)) Cg(t)f(x(t), y( t), w (t ¡ 1)),

(4.12)

where the stochastic noise term f is deŽned by f(x(t), y( t), w (t ¡ 1)) ´ F(x( t), y( t), H(w ( t ¡ 1)) ¡ E[F( x, y, H (w (t ¡ 1)))]r ,

(4.13)

and it satisŽes E[f ( x, y, w)]r = 0.

(4.14)

Equation 4.12 has the same form as the Robbins–Monro stochastic approximation (Kushner & Yin, 1997), which Žnds the maximum likelihood estimator given by equation 4.11. The effective learning coefŽcient g(t) should satisfy the condition t!1

g(t) ¡! 0,

1 X t= 1

g(t) = 1,

1 X t= 1

g 2 ( t) < 1.

(4.15)

Typically, g(t), which satisŽes conditions 4.3 and 4.15, is given by t!1

g(t) ¡!

1 at C b

(1 > a > 0).

(4.16)

The corresponding discount factor is given by t!1

l(t) ¡! 1 ¡

1¡a I at C ( b ¡ a)

(4.17)

l (t) is increased such that l (t) approaches 1 as t ! 1. For the convergence proof of the stochastic approximation (see equation 4.12), the boundedness of the noise variance is necessary (Kushner & Yin, 1997). The noise variance is given by E[f ( x, y, w) 2 ]r = E[F( x, y, H (w ))2 ]r ¡ E[F( x, y, H (w ))]r2 .

(4.18)

416

Masa-aki Sato and Shin Ishii

Both terms on the right-hand side in equation 4.18 are Žnite if we assume that the data distribution density r ( x, y) has a compact support. Consequently, the noise variance becomes Žnite under this assumption. This assumption is not so restrictive, because actual data always distribute in a Žnite domain. We can weaken this assumption such that r ( x, y) decreases exponentially as |x| or | y| goes to inŽnity. It should be noted that the stochastic approximation (see equation 4.12) for Žnding the maximum likelihood estimator is not a stochastic gradient ascent algorithm for the log-likelihood function (see equation 3.9). The online EM algorithm (see equation 4.12) is faster than the stochastic gradient ascent algorithm, because the M-step is solved exactly. So far we have used the on-line EM algorithm deŽned by equations 4.2 and 4.5, which is equivalent to the basic on-line EM algorithm deŽned by equations 4.2, 4.5a, 4.5d, 4.6, 4.7, and 4.8b, if g(t) 6 = 1. Therefore, the basic on-line EM algorithm is also a stochastic approximation. 5 Regularization 5.1 Regularization of Covariance Matrix. We have assumed so far that the covariance matrix of the input data is not singular in each region; the inverse matrix for every S i (i = 1, . . . , M) exists. In actual applications of the algorithm, however, this assumption often fails. A typical case occurs when the dimension of the input data distribution is smaller than the dimension of the input space. In such a case, S i¡1 cannot be calculated, and LQ i diverges exponentially with the time t. There are several methods for dealing with this problem, such as the introduction of Bayes priors, the singular value decomposition, and the ridge regression. However, these methods are not satisfactory for our purpose. Here, we propose a simple new method that performs well. Let us Žrst consider a regularization of an ( N £ N )-dimensional covariance matrix, S , for the observed data. Its eigenvalues and normalized eigenvectors are denoted by j n (¸ 0) and y n ( n = 1, . . . , N ), respectively. The set of the eigenvectors, fy n |n = 1, . . . , Ng, forms a set of orthonormal bases. The condition number of the covariance matrix S is deŽned by

u ´ jmin /j max ,

(5.1)

where j min and j max are the minimum and maximal eigenvalues of S , respectively, so that 0 · u · 1. If u = 0, the covariance matrix S is singular. If u ¼ 1, the covariance matrix S is regular and the calculation of its inverse is numerically stable. If 0 < u ¿ 1, the covariance matrix is regular and S ¡1 PN ¡1 ¡1 0 is given by S ¡1 = n= 1 jn y n y n . It is dominated by jmin , which may contain a numerical noise. As a consequence, the inverse matrix S ¡1 becomes sensitive to numerical noises.

On-line EM Algorithm for the Normalized Gaussian Network

417

In order to deal with this problem, we propose the following regularized covariance matrix S R for the data covariance matrix S : S R = S C aD 2 IN

(0 < a < 1)

2

D = Tr(S )/ N,

(5.2a) (5.2b)

where IN is an ( N £ N )-dimensional identity matrix. The data variance D 2 satisŽes the inequality jmax ¸ D 2 =

N 1 X j n ¸ jmax / N. N n= 1

(5.3)

The eigenvalues and eigenvectors of the regularized matrix S R are given by (j n CaD 2 ) and y n , respectively. From the inequality (se equation 5.3), it can be proved that the condition number of the regularized matrix S R satisŽes the condition: uR ¸ a/ ( N (1 C a)).

(5.4)

Then the condition number of S R is bounded below and controlled by the small constant a. The rate of change for the regularized eigenvalue is deŽned by R(jn ) ´ ((j n C aD 2 ) ¡ j n )/j n = a(D 2 / j n ). From the inequality (see equation 5.3), this ratio satisŽes the condition: R(j n ) ¸ R(jmax ) ¸ a/ N and R(j n ) · R(j min ) · a/ u. If all the eigenvalues of S are the same order (i.e., u ¼ O(1)), the rate of change becomes small (i.e., R(j n ) ¼ O(a)), so that the effect of the regularization is negligible. If the data covariance matrix S is singular (i.e., u = 0), the zero eigenvalue of S is replaced by aD 2 . Thus, the regularized matrix S R becomes regular, and its condition number is bounded by equation 5.4. In general, the eigenvalues, which are larger than their average, are slightly affected, and the very small eigenvalues are changed so as to be nearly equal to aD 2 . If S = 0, D 2 becomes zero and the regularization (see equation 5.2) does not work. Therefore, if D 2 is smaller than a threshold value D 2min , D 2 is set to D 2min . This prevents S R from being singular even when S = 0. By introducing a Bayes prior for a regularized matrix S B as P(S B ) = (k/ 2)N exp(¡(k / 2)Tr(S B¡1 )), one can get a similar regularization equation, S B = S Ck IN ,

(5.5)

where the regularization parameter k is a given constant. However, it is rather difŽcult to determine the k value without the knowledge of the data covariance matrix. It is especially difŽcult for the NGnet, since there are M independent local covariance matrices S i ( i = 1, . . . , M). The proposed method (see equation 5.2) automatically adjusts this constant by using the data variance D 2 , which is easily calculated in an on-line manner.

418

Masa-aki Sato and Shin Ishii

5.2 Regularization of On-line EM Algorithm. Based on the above consideration, the ith weighted covariance matrix is redeŽned in our on-line EM algorithm as

S i¡1 (t) =

h³ ´ i¡1 hhxx0 iii ( t) ¡m i ( t)m 0i ( t)hh1iii ( t) CahhD 2i iii ( t) IN /hh1iii ( t) , (5.6)

where ³ ´ hhD 2i iii (t) ´ hh|x ¡ m i (t)| 2 iii (t)/ N = hh|x| 2 iii ( t) ¡|m i ( t)| 2 hh1iii ( t) /N.

(5.7)

The regularized S i¡1 (see equation 5.6) can be obtained from relation 4.7 by using the following regularized LQ i : LQ i ( t) =

³ ´¡1 hhxQ xQ 0 iii ( t) C ahhD 2i iii ( t) IQN .

(5.8)

The (( N C 1) £ ( N C 1))-dimensional matrix, IQN , is deŽned by

¡

¢

N X 0 = eQn eQ0n , 0

I IQN ´ N 0

n= 1

where eQn is an (N C 1)-dimensional unit vector; its nth element is equal to 1, and the other elements are equal to 0. From the deŽnition 5.7, hhD 2i iii ( t) can be written as hhD 2i iii (t) = hhD 2i iii ( t ¡ 1) Cg(t)(D 2i ( t) Pi ( t) ¡ hhD 2i iii ( t ¡ 1)),

(5.9)

where D 2i ( t) is given by g(t)Pi ( t)D 2i ( t) = g(t)Pi (t)| x( t) ¡ m i ( t)| 2 / N C (1 ¡ g(t))|m i (t) ¡ m i (t ¡ 1)| 2 hh1iii ( t ¡ 1)/ N.

(5.10)

The second term on the right-hand side of equation 5.10 comes from the difference between hh|x ¡ m i (t)| 2 iii (t ¡ 1) and hh|x ¡ m i (t ¡ 1)| 2 iii ( t ¡ 1). Using equations 5.9, equation 5.8 can be rewritten as " LQ i ( t) =

(1 ¡ g(t))LQ ¡1 i ( t ¡ 1)

(

0

Cg(t) xQ (t) xQ ( t) C ºQn ( t) ´

p

aD i ( t) eQn .

N X n= 1

! 0

ºQn ( t)ºQn ( t) Pi ( t)

#¡1

(5.11a) (5.11b)

On-line EM Algorithm for the Normalized Gaussian Network

419

As a result, the regularized LQ i ( t) (see equation 5.8) can be calculated as follows. For a given input data x( t), LQ i ( t) is calculated using the step-wise equation 4.6. After that, LQ i ( t) is updated by LQ i ( t) := LQ i ( t) ¡

g(t)Pi ( t) LQ i ( t)ºQn ( t)ºQn 0 ( t) LQ i ( t) , 1 Cg(t)Pi ( t)ºQn 0 (t)LQ i (t)ºQn ( t)

(5.12)

using the virtual data fQ ºn ( t)| n = 1, . . . , Ng. After calculating the regularized LQ i ( t) (see equation 5.8), the linear regresQ i is obtained by using equation 4.8b, in which the regularized sion matrix W LQ i (t) is used. Only the observed data f(x(t), y( t))| t = 1, 2, . . .g are used in the calculation of equation 4.8b; and the virtual data fºQ n (t) | n = 1, . . . , NI t = 1, 2, . . .g, which have been used in the calculation of the regularized LQ i (t), must not be used. Therefore, equation 4.8a does not hold in our regularization method. Since the regularized LQ i ( t) is positive deŽnite, the linear Q i at an equilibrium point of equation 4.8b satisŽes the regression matrix W condition Q i E[xQ xQ 0 P(i| x, y, h )]r = E[yxQ 0 P( i|x, y, h )]r , W

(5.13)

which is identical to the maximum likelihood equation (3.7d) along with h = hN . Although the matrix E[xQ xQ 0 P( i| x, y, h )]r may not have the inverse, the relation (see equation 5.13) still has a meaning. Therefore, our method does Q i. not introduce a bias on the estimation of W . 6 Unit Manipulation

Since the EM algorithm guarantees only local optimality, the obtained estimator depends on its initial value. The initial allocation of the units in the input space is especially important for attaining a good approximation. If the initial allocation is quite different from the input distribution, much time is needed to achieve proper allocation. In order to overcome this difŽculty, we introduce dynamic unit manipulation mechanisms, which are also effective for dealing with dynamic environments. These mechanisms are unit production, unit deletion, and unit division, and they are conducted in an on-line manner after observing each datum ( x(t), y( t)). • Unit production. The probability P (x( t), y( t), i | h ( t ¡ 1)) indicates how probable the ith unit produces the datum ( x( t), y(t)) with the parameter ( ( ), ( ), ( h (t ¡ 1). Let 0 < Pproduce ¿ 1/ M. When maxM i= 1 P x t y t i | h t ¡ 1)) < Pproduce , the datum is too distant from the present units to be explained by the current stochastic model. In this case, a new unit is produced to account for the new datum. The initial parameters of the new unit are

420

Masa-aki Sato and Shin Ishii

given by: m MC1 = x( t)

(6.1a)

¡1 ¡2 S MC1 = ÂMC1 IN

M

2 = b1 min |x(t) ¡ m i | 2 / N ÂMC1 i= 1

M

(6.1b)

2 sMC1 = b2 max si2

(6.1c)

Q MC1 ´ ( WMC1 , bMC1 ) = (0, y(t)), W

(6.1d)

i= 1

where b 1 and b2 are appropriate positive constants. • Unit deletion. The weighted mean hh1iii (t), which is calculated by equation 4.2, indicates how much the ith unit has been used to account for the data until t. Let 0 < Pdelete ¿ 1/ M. If hh1iii ( t) < Pdelete , the unit has rarely been used. In this case, the ith unit is deleted. • Unit division. The unit error variance si2 ( t) (see equation 4.5d) indicates the squared error between the ith unit’s prediction and the actual output. Let Ddivide be a speciŽc positive value. If si2 ( t) > Ddivide, the unit’s prediction is insufŽcient, probably because the partition in charge is too large to make a linear approximation. In this case, the ith unit is divided into two units and the partition in charge is divided into two. The initial parameters of the two units are given by: p m i ( new) = m i ( old) C b3 j 1 y 1 p m MC1 ( new) = m i ( old) ¡ b3 j 1 y 1 (6.2a) ¡1 ( new) = S i¡1 ( new) = 4j1¡1 y 1 y 10 C S MC1

N X n= 2

j n¡1 y n y n0

(6.2b)

2 (new) = si2 ( old)/ 2 si2 ( new) = sMC1

(6.2c)

Q i ( new) = W Q MC1 ( new) = W Q i (old), W

(6.2d)

where j n and y n denote the eigenvalue and the eigenvector of the covariance matrix S i ( old), respectively, and j 1 = j max . b3 is an appropriate positive constant. Although similar unit manipulation mechanisms have been proposed (Platt, 1991; Schaal & Atkeson, 1998), these mechanisms can be conducted with a probabilistic interpretation in our on-line EM algorithm. 7 Experiments 7.1 Function Approximation in Static Environment. Applicability of our algorithm is investigated using the following function (N = 2 and

On-line EM Algorithm for the Normalized Gaussian Network

421

0.1 0.08

nMSE

0.06 0.04 0.02 0 0

10

20

30

Epochs

40

50

Figure 1: Learning processes of batch EM algorithm (dashed line) and on-line EM algorithm (solid). Five hundred training data are supplied once in each epoch. Ordinate denotes nMSE (mean squared error normalized by variance of desired outputs). Error is evaluated on 41 £ 41 mesh grids on input domain.

D = 1), which was previously used by Schaal and Atkeson (1998): n o 2 2 2 2 (¡1 · x1 , x2 · 1) . y = max e¡10x1 , e¡50x2 , 1.25e¡5(x1 Cx2 )

(7.1)

We prepared 500 input-output pairs, f(x(t), y(t))| t = 1, . . . , 500g as a training data set by uniformly sampling the input variable vector, x ´ ( x1 , x2 ), from its domain. A fairly large gaussian noise N (0, 0.1) is added to the outputs, where N (0, 0.1) denotes a gaussian distribution whose mean and standard deviation are 0 and 0.1, respectively. The problem task is for the NGnet to approximate the function 7.1 from the 500 noisy data. We compared the batch EM algorithm and the on-line EM algorithm. In this experiment, the discount factorl ( t) was scheduled to approach 1 as in equation 4.17. Figure 1 shows the time series of the normalized mean squared error, nMSE, which is normalized by the variance of the desired outputs (see equation 7.1). In the Žgure, we can see that both the batch EM algorithm and the online EM algorithm are able to approximate the target function in a small number of epochs. Although the so-called overlearning can be seen in both learning algorithms, it is more noticeable in the batch EM algorithm than in the on-line EM algorithm. The number of the units used in both algorithms was 50. The data distribution does not change over time in this task. In this case, it may be thought that the batch learning is more suitable than the on-line learning because the batch learning can process the whole data at once. However, our on-line EM algorithm has an ability similar to the batch algorithm. In addition, the on-line EM algorithm achieves

422

Masa-aki Sato and Shin Ishii

a faster and more accurate approximation for this task than the RFWR (receptive Želd weighted regression) model proposed by Schaal and Atkeson (1998). 7.2 Function Approximation in Dynamic Environments. The on-line EM algorithm is effective for a function approximation in a dynamic environment where the input-output distribution changes over time. For an experiment, the distribution of the input variable x1 in equation 7.1 was continuously changed in 500 epochs from the uniform distribution in the interval [¡1, ¡0.2] to that in the interval [0.2, 1]. At each epoch, the input variables were generated in the current domain. In the applications of the on-line EM algorithm to such dynamic environments, the learning behavior changes according to the scheduling of the discount factor l(t). When the discount factor is relatively small, the model tends to forget the past learning result and to adapt quickly to the input-output distribution. We show two typical behavioral patterns in the learning phase. In the Žrst case, l ( t) is initially set at a relatively small value, and it is slowly increased; that is, the effective learning coefŽcient g(t) is relatively large throughout the experiment. The NGnet adapts to the input distribution change by a relocation of the units’ center. During the course of this relocation, the units’ center moves to the new region, and consequently the past approximation in the old region is forgotten. Figures 2b and 2c show the center and the covariance of all the units for the 50th epoch and the 500th epoch, respectively. The NGnet adapts to the input distribution change by the drastic relocation of the units’ center. Figure 2a shows that nMSE on the current input region does not grow large throughout the task. In this experiment, the number of the units does not change. In the second case, l ( t) is rapidly increased; that is, g(t) rapidly approaches to zero. The NGnet adapts to the input distribution change without forgetting the past approximation result. Figure 3a shows this learning process. Figure 3b shows the center and the covariance of all the units at the end of the learning task. Since the unit relocation is slow, the NGnet adapts to the input distribution change mainly by the unit production mechanism. Consequently, the units in the region where no more input data appear remain, and the function approximation on the whole input domain is accurately maintained. 7.3 Singular Input Distribution. In order to evaluate our regularization method, we prepared a function approximation problem where the input variables are linearly dependent and there is an irrelevant variable. In such a case, the basic on-line EM algorithm without the regularization method obtains a poor result because the input distribution becomes singular. In this experiment, the output y is given by the same function as equation 7.1, while the input variables are given by x ´ (x1 , x2 , x3 , x4 , x5 ) = ( x1 , x2 , (x1 C x2 )/ 2, (x1 ¡ x2 )/ 2, 0.1). When the input data are generated, a gaussian noise

On-line EM Algorithm for the Normalized Gaussian Network

423

0 .0 5 0 .0 4

nM SE

0 .0 3 0 .0 2 0 .0 1

100

200

300

1

1

0.5

0.5

0

­ 0.5

­1 ­1

400

E p o ch s

X2

X2

0 0

500

0

­ 0.5

­ 0.5

0

X1

0.5

1

­1 ­1

­ 0.5

0

X1

0.5

1

Figure 2: (a) Learning process of on-line EM algorithm that quickly adapts to a dynamic environment with forgetting the past. Ordinate denotes nMSE evaluated on current input domain at each epoch. (b, c) Dynamic unit relocation for dynamic environment. Each ellipse denotes a receptive Želd of a unit; the ellipse corresponds to covariance (two-dimensional display of standard deviation). The ellipse center is set at unit center,m i . Dots denote 500 inputs in current epoch. (b) At 50th epoch. (c) At 500th epoch.

N (0, 0.05) is added to x3 , x4 , and x5 . For a training set, we prepared 500 data, where x1 and x2 were uniformly taken from their domain, and the output y was added by a gaussian noise N (0, 0.05). For evaluation, a test

424

Masa-aki Sato and Shin Ishii

Figure 3: (a) Learning process of on-line EM algorithm that slowly adapts to dynamic environment without forgetting the past. Solid line: nMSE on whole input domain. Dashed line: (nMSE ¤ 10) on current input domain at each epoch. Dash-dot line: number of units, which is normalized by its initial value 30 for convenience. (b) Receptive Želds after learning.

set was prepared by setting x1 and x2 on the 41 £ 41 mesh grids, and the other variables were generated according to the above prescription. The same training set was repeatedly supplied during the learning. In Figure 4a,

On-line EM Algorithm for the Normalized Gaussian Network

425

0 .5 0 .4

nM SE

0 .3 0 .2 0 .1 0 0

20

40

60

Epochs

80

10 0

0 .1 1

0 .0 8

0.5

X3

nM S E

0 .0 6 0

0 .0 4

­ 0.5

0 .0 2

­1 ­1

1 0

0 0

10

20

30

Epochs

40

50

0

X1

1 ­1

X2

Figure 4: (a) Learning processes when output is given by equation 7.1 for input x ´ (x1 , x2 , x3 , x4 , x5 ) = (x1 , x2 , (x1 C x2 )/ 2 C N(0, 0.05), (x1 ¡ x2 )/ 2 C N(0, 0.05), N(0.1, 0.05)). Learning curves for a = 0.23 (solid line), a = 0.1 (dashed), and a = 0 (dash-dot). (b) Learning curves for a = 0.023 (solid) and a = 0 (dashed). Output is given by equation 7.2. (c) Receptive Želds at 50th epoch, each of which is deŽned by an ellipse with axes for two principal components of the local covariance matrix S i . The ellipse center is set at unit center, m i . This Žgure shows a 3D view of a hemisphere.

the solid, dashed, and dash-dot lines are the learning curves for a = 0.23, a = 0.1, and a = 0, respectively. If no regularization method is employed (a = 0), the error becomes large after the early learning stage. Comparing

426

Masa-aki Sato and Shin Ishii

the cases of a = 0.23 and a = 0.1, the regularization effect seems fairly robust with respect to the parameter a value. Let us consider another situation, where the input data distribute on a curved manifold. Each input datum in the three-dimensional space, (x1 , x2 , x3 ), is restricted on the unit sphere, that is, (x1 , x2 , x3 ) = (cosh1 cosh2 , cosh1 sinh2 , sinh 1 ). A function deŽned on this unit sphere is given by n o 2 2 2 2 y = cos(h1 )cos(h2 / 2)max e¡10(2h1 / p ) , e¡50(h2 / p ) , 1.25e¡5((2h1 / p ) C(h2 / p ) ) , (7.2) where the range of the spherical coordinate is given by ¡p / 2 · h 1 · p / 2 and ¡p · h2 < p . The output y does not include a noise. The function 7.2 is similar to the function 7.1, but it is changed so as to satisfy the consistency of the spherical coordinate. We prepared 2000 data points uniformly on the sphere. The covariance matrix of input data is not singular. However, the covariance matrix of each local unit is nearly degenerate, so that the calculation of the inverse covariance matrix and the linear regression matrix may include a noise without the help of the regularization method. In Figure 4b, the solid and dashed lines are the learning curves for a = 0.023 and a = 0, respectively. If no regularization method is employed (a = 0), the error does not become small. Our regularization method (a = 0.023) provides a fairly good result compared to the basic method without the regularization (a = 0). The condition numbers deŽned by equation 5.1 with the regularization and without the regularization are 0.0191 § 0.0042 and 0.0071 § 0.0038, respectively. The local covariance matrix, S i , of each unit represents the local input data distribution fairly well in our regularized method. It has two principal components that span the tangential plane to the unit sphere at each local unit position. The third eigenvalue is very small and is bounded below by the regularization term. In Figure 4c, receptive Želds of the local units are shown. The receptive Želds appropriately cover the unit sphere. 7.4 Robot Dynamics. Finally, we examine realistic and difŽcult problems. The problems are taken from DELVE (Rasmussen et al., 1996), which is a new standard collection of neural network benchmark problems. The Pumadyn data sets are a family of data sets synthetically generated from a realistic simulation of the dynamics of a Puma 560 robot arm (Ghahramani, 1996; Corke, 1996). The Puma robot has six revolving joints. The state vector is deŽned by fh i , hPi |i = 1, . . . , 6g, where hi and hPi represent the angle and the angular velocity of the ith joint, respectively. The applied torque to the ith joint is denoted by ti . The task of the Pumadyn-8 family is to predict the third joint acceleration hR3 given the eight-dimensional input vector x ´ fh1 , hP1 , h2 , hP2 , h3 , hP3 , t1 , t2 g. Other variables are Žxed to zero. The input data are randomly generated by sampling the input variables uniformly from the range (|h |/ p , |hP |/ p , | t| · 0.6). The data sets are characterized by

On-line EM Algorithm for the Normalized Gaussian Network

427

nonlinearity, noise level, and size of training data. We examined six training data sets: nm64, nm256, and nm1024, which are nonlinear and medium noisy data sets consisting of 64, 256, and 1024 training data, as well as nh64, nh256, and nh1024, which are nonlinear and highly noisy data sets consisting of 64, 256, and 1024 training data. Gaussian noise with standard deviations of 0.12 and 0.4 was added to both the input and the output variables for the medium and the highly noisy data sets, respectively. The test data set size was 1024 when the training data set size was 1024 and 512 for the other cases. We compared our on-line EM (OEM) algorithm with the mixtures-ofexperts (ME) methods registered in DELVE (Waterhouse, 1997) and the Moody-Darken (MD) method (Moody & Darken, 1989). There are nine competitor methods: on-line EM algorithm (OEM), on-line EM algorithm with Neal early stopping (OEM-ese) (Neal, 1998), mixtures-of-experts trained by ensemble learning (ME-el), hierarchical mixtures-of-experts trained by ensemble learning (HME-el), mixtures-of-experts trained by committee early stopping (ME-ese), hierarchical mixtures-of-experts trained by committee early stopping (HME-ese), hierarchical mixtures-of-experts grown via committee early stopping (HME-grow), Moody-Darken (MD) method, and modiŽed Moody-Darken (MD-l) method. The committee early stopping method (Waterhouse, 1997) uses a committee of (hierarchical) ME models. The training data are randomly divided into a training set and a validation set, with two-thirds and one-third of the total data, respectively. The training is done for each member of the committee using different training and validation sets and terminated at the minimum of the validation error. The prediction is given by the average of the outputs for all committee members. The number of committee members depends on the training time. A minimum of 3 members and a maximum of 50 can be created. If the training time is greater than a given threshold, which is proportional to the amount of data, no more members can be created. The HME-grow method constructs a hierarchical ME from data, learning both the structure as well as the parameters of the model. In the growing process, one of the experts is split into two experts and a gate. The committee early stopping method is also used to control the size of the hierarchical trees. The ensemble learning estimates the posterior probability distribution for the model parameters rather than estimates the optimal parameter value. The gaussian prior distribution on the gate and the experts parameters are assumed to introduce hyperparameters. The optimal hyperparameters are estimated by an alternating minimization procedure that combines the standard EM algorithm for parameter estimation with reestimation of hyperparameters. The algorithm is computationally highly demanding. The training of the OEM method is terminated at a prespeciŽed number of epochs, which is 5% of the data size. The OEM-ese uses a simpler version of the committee early stopping proposed by Neal (1998). The training data

428

Masa-aki Sato and Shin Ishii

Table 1: Computation Time Comparison for Nine Methods. Method ME-el HME-el OEM-ese OEM ME-ese HME-ese HME-grow MD-l MD Time (sec)

16,515

23,580

395

40

2129

2171

2059

16

5

Notes: The data set used was nm1024 with 1024 training data. The computer used was Silicon Graphics Octane (CPU: R10000 175MHz £ 2).

are partitioned into four equal portions. The committee consists of four networks. Each network uses three of the four portions as a training set and the remaining portion as a validation set. The MD method uses two-stage learning. It Žrst determines the unit centers according to the input distribution by using the k-means clustering algorithm. Subsequently, the bias coefŽcients bi are determined by using the least mean squared error method while Žxing the unit centers. The NGnet used in the original MD model has isometric covariance matrices and no linear coefŽcients, that is, Wi ´ 0. The MD-l method, in contrast, uses the NGnet with linear coefŽcients. The main differences among the OEM, the ME, and the MD methods are their criterion functions. The OEM method maximizes the log-likelihood P of the joint probability distribution, Tt= 1 log P( x( t), y(t)|h ). Therefore, the units are allocated according to the input-output data distribution. The ME method maximizes the log-likelihood of the conditional probability, PT t= 1 log P( y( t)| x( t), h ). Therefore, the specialization process of each expert is done according to the output performance and does not take account of the input data distribution. The Žrst-stage learning of the MD method can be considered as the maximization of the log-likelihood of the input probP ability, Tt= 1 log P(x( t)|h ). The second-stage learning minimizes the mean squared error of the network output. Therefore, the optimality of the total algorithm is not guaranteed. The simulation results are summarized in Figure 5. Table 1 shows the computation time needed for training by these algorithms. The results depend very much on the training size. The most striking differences can be seen for the nm256 data set. The best performance is achieved by (H)ME-el, and followed by OEM (-ese), (H)ME-ese, HME-grow, and MD(-l), in that order. The performances of the ME family and the OEM family are almost the same for the nm64 and nh64 data sets, while those of the MD family are slightly worse. For the nm1024 and nh1024 data sets, (H)ME-el, OEM(-ese) show comparable performances. The performances of (H)ME-ese and MD-l are slightly worse. HME-grow and MD each shows poor performance. Some comments can be made regarding these results. The number of data is very small compared to the dimensionality of the input space; the noisy data are sparsely distributed in a high-dimensional space. Therefore, the

On-line EM Algorithm for the Normalized Gaussian Network

nM SE

nm 1024

nm 256 0 .8

0 .8

0 .6

0 .6

0 .6

0 .4

0 .4

0 .4

0 .2

0 .2

0 .2

ABCDEFGH I

0

nh1024

nMSE

nm 64

0 .8

0

ABCDEFGH I

0

nh256 0 .8

0 .8

0 .6

0 .6

0 .6

0 .4

0 .4

0 .4

0 .2

0 .2

0 .2

ABCDEFGH I

0

ABCDEFGH I

ABCDEFGH I nh64

0 .8

0

429

0

ABCDEFGH I

Figure 5: Learning results for Pumadyn data sets: nm64, nm256, nm1024, nh64, nh256, and nh1024. Bars represent the expected nMSE,which is calculated by an average of nMSE for different test data sets, for nine methods shown at the bottom of the Žgure.

reliable estimation of the optimal parameter is very difŽcult. This situation favors ensemble learning. The disadvantage of ensemble learning is that it requires a huge amount of time, as shown in Table 1. The OEM method achieves comparable learning ability with the (H)ME-el method within a signiŽcantly smaller amount of computation time for the 1024 data sets. The original MD method showed a poor performance despite being very fast. The modiŽed MD method with linear coefŽcients showed a better performance than the original MD method. However, its performance is worse than the OEM methods. The performance of the MD methods depended on the number of units. In Figure 5, the results of the best MD models are

430

Masa-aki Sato and Shin Ishii

Table 2: Learning Results for Robot Trajectory Data Sets. Method nMSE (%) Time (sec)

(H)ME-el OEM — —

1.8 1085

ME-ese HME-ese HME-grow MD-l

MD

2.3 84,626

4.4 130

2.8 63,610

22.2 36,780

3.9 220

Notes: Computation time and nMSE for six methods are shown. Ensemble learning, (H)ME-el, does not end within a week.

shown. For example, the best number of units for MD and MD-l are 140 and 33 for the nm1024 data set, respectively. The OEM method automatically selects the number of units. The Žnal numbers of units for the nm1024, nm256, and nm64 data sets are 6, 5, and 1, respectively, although the initial number of units is set to 2% of the data size. We considered a more realistic situation. For real-time robot control, the training data are generated along the robot trajectory. Therefore, we generated the training data by freely operating the Puma robot from the upright position. Due to inertia, the robot exhibited chaotic oscillations. The task was to predict the third joint acceleration hR3 given the 10-dimensional input vector x ´ fhi , hPi |i = 1, . . . , 5g. Since h 6 and hP6 were almost zero throughout the trajectory, we omitted these variables, and the applied torque was zero. The size of training data was 7000 (the ME family could not operate more than 7000 data in our computer). The 7000 test data were generated in the neighborhood of the trajectory. The results are summarized in Table 2. The OEM method achieves a better performance than the ME family with a signiŽcantly smaller amount of time. The performance of the MD method is worse than that of the (H)ME-ese method. The computation times of the OEM, the MD, and the MD-l methods scale as (N C 1) 2 M , M3 , and (( N C 1)M)3 , respectively, where M is the number of units and N is the input dimension. The best performance of MD(MD-l) was obtained with 420 (50) units, while the Žnal number of units in OEM became 230 after four epochs. When the number of units in MD(MD-l) was larger than 1000 (100), the MD(-l) method became slower than the OEM method and the performance was degraded. In summary, we conŽrmed that the on-line EM algorithm achieves good performance in a short amount of time even if the number of data become large. 8 Conclusion

In this article, we proposed a new on-line EM algorithm for the NGnet. We showed that the on-line EM algorithm is equivalent to the batch EM algorithm if a speciŽc scheduling of the discount factor is employed. In addition, we showed that the on-line EM algorithm can be considered as a stochastic approximation method to Žnd the maximum likelihood estimator.

On-line EM Algorithm for the Normalized Gaussian Network

431

A new regularization method was proposed in order to deal with a singular input distribution. In order to manage dynamic environments, unit manipulation mechanisms such as unit production, unit deletion, and unit division were also introduced based on the probabilistic interpretation. Experimental results showed that our approach is suitable for dynamic environments where the input-output distribution of data changes over time. We also applied our on-line EM algorithm to robot dynamics problems. Our algorithm achieved a comparable learning ability to the ME methods for a small number of data and achieved a better performance than the ME methods for a large number of data within a signiŽcantly smaller amount of computation time. Our method can also be applied to reinforcement learning (Sato & Ishii, 1999) and the reconstruction of nonlinear dynamics (Ishii & Sato, 1998).

Acknowledgments

We thank the referees and the editor for their insightful comments in improving the quality of this article.

References Corke, P. I. (1996).A robotics toolbox for MATLAB. IEEE Robotics and Automation Magazine, 3, 24–32. Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of Royal Statistical Society B, 39, 1–22. Ghahramani, Z. (1996). The Pumadyn dataset. Available online at: http://www.cs. toronto.edu/»delve/data/pumadyn/pumadyn.ps.gz. Ishii, S., & Sato, M. (1998). On-line EM algorithm and reconstruction of chaotic dynamics. In T. Constantinides, S.-Y. Kung, M. Niranjan, & E. Wilson (Eds.), Neural networks for siginal processing VIII (pp. 360–369). New York: IEEE. Jacobs, R. A., Jordan, M. I., Nowlan, S. J., & Hinton, G. E. (1991). Adaptive mixtures of local experts. Neural Computation, 3, 79-87. Jordan, M. I., & Jacobs, R. A. (1994). Hierarchical mixtures of experts and the EM algorithm. Neural Computation, 6, 181–214. Jurgen, ¨ S. (1996). Pattern classiŽcation: A uniŽed view of statistical and neural approaches. New York: Wiley. Kushner, H. J., & Yin, G. G. (1997). Stochastic approximation algorithms and applications. New York: Springer-Verlag. Moody, J., & Darken, C. J. (1989). Fast learning in networks of locally-tuned processing units. Neural Computation, 1, 281–294. Neal, R. M. (1998). Assessing relevance determination methods using DELVE. In C. M. Bishop (Ed.), Generalization in neural networks and machine learning (pp. 97–129). Berlin: Springer-Verlag.

432

Masa-aki Sato and Shin Ishii

Neal, R. M., & Hinton, G. E. (1998). A view of the EM algorithm that justiŽes incremental, sparse, and other variants. In M. I. Jordan (Ed.), Learning in graphical models (pp. 355–368). Norwell, MA: Kluwer Academic Press. Nowlan, S. J. (1991). Soft competitive adaptation: neural network learning algorithms based on Žtting statistical mixtures (Tech. Rep. No. CS-91-126). Pittsburgh: Carnegie Mellon University. Platt, J. (1991). A resource-allocating network for function interpolation. Neural Computation, 3, 213–225. Poggio, T., & Girosi, F. (1990). Networks for approximation and learning. Proceedings of the IEEE, 78, 1481–1496. Rasmussen, C. E., Neal, R. M., Hinton, G. E., van Camp, D., Revow, M., Ghahramani, Z., Kustra, R., & Tibshirani, R. (1996). The DELVE manual. Available online at: ftp://ftp.cs.utoronto.ca/pub/neuron/delve/doc/manual.ps.gz. Sato, M., & Ishii, S. (1999). Reinforcement learning based on on-line EM algorithm. In M. S. Kearns, S. A. Solla, & D. A. Cohn (Eds.), Advances in neural information processing systems, 11 (pp. 1052–1058).Cambridge, MA: MIT Press. Schaal, S., & Atkeson, C. G. (1998). Constructive incremental learning from only local information. Neural Computation, 10, 2047–2084. Waterhouse, S. R. (1997). ClassiŽcation and regression using mixtures of experts. Unpublished doctoral dissertation, Cambridge University. Xu, L., Jordan, M. I., & Hinton, G. E. (1995).An alternative model for mixtures of experts. In G. Tesauro, D. S. Touretzky, & T. K. Leen (Eds.), Advances in neural information processing systems, 7 (pp. 633–640). Cambridge, MA: MIT Press. Received March 23, 1998; accepted February 17, 1999.

LETTER

Communicated by Jean-Pierre Nadal

A General Probability Estimation Approach for Neural Computation Maxim Khaikine Klaus Holthausen Friedrich-Schiller-Universit¨at Jena, Ernst-Haeckel-Haus, D-07745 Jena, Germany We describe an analytical framework for the adaptations of neural systems that adapt its internal structure on the basis of subjective probabilities constructed by computation of randomly received input signals. A principled approach is provided with the key property that it deŽnes a probability density model that allows studying the convergence of the adaptation process. In particular, the derived algorithm can be applied for approximation problems such as the estimation of probability densities or the recognition of regression functions. These approximation algorithms can be easily extended to higher-dimensional cases. Certain neural network models can be derived from our approach (e.g., topological feature maps and associative networks). 1 Introduction

The concepts of probability density estimation are relevant to many aspects of neural computation (Bishop, 1995). There are two general methods to deal with this problem in the framework of statistical pattern recognition: nonparametric and parametric probability density estimation. Both recognition methods are presented and discussed in detail in Bishop (1995). The nonparametric estimation (e.g., simple histogram methods) does not assume a particular functional form for the density function. A disadvantage of this method is that it requires large amounts of processing because all data points used in the training must be retained. Moreover, it is not known a priori how many data have to be collected in order to provide a good representation. The parametric approach, on the contrary, assumes a speciŽc form for the density function, which might be very different from the actual density. The chosen functional form contains a number of adjustable parameters that are optimized by Žtting the model to the data set. For the purpose of neural network theory, an important advantage of this parametric approach is that the parameters can be adjusted locally. As local learning rules of this type, we will consider algorithms in which a hypothesis about the real distribution is changed each time after a stimulus was received without storing and processing of previous stimuli. These local recognition algorithms give quite Neural Computation 12, 433–450 (2000)

c 2000 Massachusetts Institute of Technology °

434

Maxim Khaikine and Klaus Holthausen

good results if the real probability density is not too different from some elements of the family of functions chosen for the approximation. Generally the quality of the approximation depends on the smoothness of the probability density function and the inherent degree of freedom in the applied functional form. In this article, we develop a general approach to Žnd a comprehensive family of recognition algorithms in which the class of basis functions is not limited and includes all possible smooth functions. The focus of our work is not a particular application in neural network theory, but a principled approach based on sound theoretical concepts. It allows an ideal estimation of any arbitrary probability distribution. We will show that on the basis of these learning rules, one can solve more complicated problems (e.g., an approach for the estimation of regression functions for stimulus-response pairs with arbitrary distributions). Then we discuss the general framework of our approximation theory and certain applications in neural network theory. Our Žndings support recent work on the approximation by feedforward neural networks (Scarselli & Tsoi, 1998) since they allow the focus of investigation to be turned toward the approximation capabilities of single neurons. Furthermore, earlier general approaches that combined neural computation and approximation theory (Girosi & Poggio, 1990; Kurkova, 1992) could be revised in order to gain insight into novel architecture and optimization principles in neural computation. 2 Local Algorithms for the Approximation of Probability Densities

We consider a discrete process in which a stochastic variable x adopts one of N possible states fx1 , x2 , . . . , xN g. This process can be described by the corresponding probabilities PE = ( p1 , p2 , . . . , pN ), which we will call the objective probability distribution. A priori we have no information about the objective E = ( q1 , q2 , . . . , qN ) about it. distribution, but we can make a hypothesis Q E The correEvery stimulus xk gives rise to a change of the hypothesis Q. sponding learning rule can be represented by a matrix L , whose elements E Thus, the adaptation of the hypothesis Q Et depend on the actual vector Q. after a stimulus xk was received at time t can be written with the learning E ) and learning rate 2 : rule L ( Q E t) = qtn C 2 Ln,k ( Q qtC1 n

8n.

We can calculate the expected (i.e., average) value of qtC1 n with respect to the objective probabilities pk of the stimuli xk : t hqtC1 n i = qn C 2 ¤

N X k= 1

E t ) pk . Ln,k (Q

(2.1)

General Probability Estimation Approach for Neural Computation

435

E ) are continuous functions and the coefŽIf all elements of the matrix Ln,k ( Q cient 2 is sufŽciently small, we can come to a description that is continuous E in time. Thus, the statistical dynamics of the subjective probability vector Q is given by a set of coupled differential equations: N X dq n ( t) E ) pk . = Ln,k ( Q dt k= 1

(2.2)

E ) can be derived? First, let us Which properties of the learning matrix L (Q note that the sum of all time gradients of probabilities qk in our hypothesis must be always null. This means: N N X N X X dqn ( t) E ) pk = 0 = Ln,k ( Q dt n= 1 n= 1 k= 1

E 8P.

E ) follows: Hence, the Žrst property of Ln,k (Q N X n= 1

E) = 0 Ln,k ( Q

E k. 8Q,

(2.3)

E = PE presents Second, it is a plausible assumption that a correct hypothesis Q at least a Žxed point in the phase-space and is not statistically changed with time (about convergence toward this point, see below). Together with E ): equation 2.2 we obtain the second property of Ln,k (Q N X k= 1

E ) qk = 0 Ln,k ( Q

E 8n, Q.

(2.4)

Equations 2.3 and 2.4 are obviously not sufŽcient to deŽne learning rules. Nevertheless we give an example of a possible learning matrix after simpliE ). fying the structure of L n , k ( Q 2.1 Pfaffelhuber ’s Learning Rule. Let us assume that all functions E ) in the learning matrix are divided into two principally different Ln,k ( Q classes of very simple form: » E ) = f ( qn ) if k 6 = n Ln,k (Q g( qn ) otherwise.

Equations 2.3 and 2.4 will be rewritten as follows: f ( qn )

N X k6 = n

qk C g( qn ) qn = 0

8n

436

Maxim Khaikine and Klaus Holthausen

and N X n6 = k

f ( qn ) C g( qk ) = 0

8k.

This system has only one solution if N > 2 and inŽnitely many otherwise. The general solution is f ( qn ) = ¡qn C and g(qn ) = C(1 ¡ qn ), where C is an arbitrary constant. The sign of C determines whether the dynamics will be stable. The learning matrix hence is: 0 1 1 ¡ q1 ¡q1 ... ¡q1 B ¡q2 1 ¡ q2 . . . ¡q2 C C E) = C B L( Q B .. C. .. .. @ . A . . ¡qN ¡qN . . . 1 ¡ qN This learning rule was Žrst suggested by Pfaffelhuber (1972), who also investigated its convergence. With this matrix, the statistical dynamics for this learning process follows from equation 2.2: dqn (t) = C( pn ¡ qn ). dt Thus, it has been shown that it is possible to formulate certain restrictions for our approximation theory from which a prominent learning rule can be derived. Here we investigate a broader class of approximation algorithms that could be of interest for both statistical pattern recognition and neural network theory. 3 General Mathematical Framework

The basic properties of the learning matrix (see equations 2.3 and 2.4) guarantee that the probabilities qk that constitute our hypothesis are normalized E = PE represents a Žxed and that any correctly recognized distribution Q point of the dynamics 2.2. But the convergence toward this point was not investigated. In order to solve this problem, we will apply a framework that will be more suitable than the notation of matrix functions. E (t) and the real distriFirst, we note that both vectors—the hypothesis Q E bution P—are located on a (N ¡ 1)-dimensional hyperplane—a manifold Q in the N-dimensional vector space. The manifold Q is deŽned by the normalization condition: N X n= 1

qn = 1

E 2 Q. 8Q

In the context of probability estimation, we will consider only learning processes that can be described by their dynamics on the manifold Q . These

General Probability Estimation Approach for Neural Computation

437

algorithms will be regarded as models of the real learning processes. Our aim is to deŽne a learning rule (or, better, a family of rules) so that every start hypothesis converges statistically to the objective probability distribution PE during the recognition process. E ) for every vector Q E deŽnes a linear operator The learning matrix L (Q on the space RN (cf. equation 2.2). For this linear operator, one can Žnd a set of eigenvectors fE vn n = 1, . . . , Ng with corresponding eigenvalues fl n n = 1, . . . , Ng so that E ) vEn = l n vEn . L (Q It is clear that if the eigenvectors and values are given in every point of the manifold Q , they automatically deŽne the learning rule (i.e., the learning E )) in Q . Furthermore, this method is very exible and suitable for matrix L ( Q studying the global properties of the learning process. As a simpliŽcation, we consider here only the case where alll n are real. This means that in every E on the manifold Q , the operator L (Q E ) has only one-dimensional point Q subspaces. E is an eigenvector of the operator Proposition 2.4 shows that the vector Q E L ( Q) with corresponding eigenvalue l Q = 0: E )Q E = 0¤Q E = 0. L (Q Furthermore, all other eigenvalues l n n 2 f1, . . . , N ¡ 1g of the learning E ) must be different from zero, because otherwise (for example, matrix L (Q E would be stable for every real probability l m = 0) the current hypothesis Q Q distribution P given as a normalized linear combination, PQ =

E C b vEm aQ P . a C b k vm,k

Here, the parameters a and b have to be chosen in order to satisfy the condiE vEn n = 1, . . . , N ¡ 1g form a complete tion 0 · PQk · 1. The eigenvectors fQ, vector system—the basis for the vector space—in every point of Q so that every vector xE can be expanded as a linear combination: EC xE = cQ Q

N¡1 X n= 1

cn vEn .

E ) yields: The application of the operator L ( Q E ) xE = cQ L ( Q E)Q EC L (Q

N¡1 X n= 1

E ) vEn = cn L ( Q

N¡1 X n= 1

l n cn vEn .

438

Maxim Khaikine and Klaus Holthausen

P E ) projects every vector onto the space The learning matrix L (Q n xn = 0. Therefore all eigenvectors vEn associated with nonzero eigenvalues must belong to this space: N X

vn,k = 0

k= 1

8n 2 f1, . . . , N ¡ 1g.

(3.1)

Theorem 1 introduces a broad class of learning rules that converge toward the process (2.2) and allow the study of the global properties of the E ): corresponding matrix L ( Q E ( t) always converges toward the objective probaThe hypothesis Q E ) are E bility distribution P during the learning process (see equation 2.2) if all l n (Q E real and positive and all eigenvectors of the learning matrix L ( Q) are orthogonal E 2 Q: in every point Q Theorem 1.

E ) > 0, l n(Q

vEn vEm = 0

8n, m = 1, . . . , N ¡ 1.

The eigenvalues corresponding to Pfaffelhuber ’s (1972) learning rule (section 2.1.) for the case N = 3 are given by the roots of the polynomial  (l ) = ¡l(1 ¡l) 2 . Two orthogonal eigenvectors corresponding to the eigenvalue l = C1 are given by v1 = (0, C1, ¡1) and v2 = (¡2, C1, C1). Remark.

E ( t). We choose Let us introduce the distance vector DE ( t) := PE ¡ Q the following linear combination for DE ( t): Proof.

E (t) C DE ( t) = cQ Q

N¡1 X n= 1

cn vEn .

(3.2)

E and PE belong to the manifold Q , the coefŽcient cQ in Because both vectors Q the Žrst part P of equation 3.2 must always be null so that the normalization condition k D k = 0 is always fulŽlled. In order to describe the convergence, we consider the dynamics of the distance vector:

®

®2

® ® d ®DE ( t) ® dt

= ¡2DE ( t)

E ( t) dQ E )P. E = ¡2DE (t)L ( Q dt

E)Q E = 0 we obtain: Using the property L ( Q

® ®

®2 ®

d ®DE ( t) ® dt

E ) DE ( t) = ¡2DE ( t) L ( Q

General Probability Estimation Approach for Neural Computation

= ¡2

(

N¡1 X n= 1

! cn vEn

(

N¡1 X m= 1

! l m cm vEm

= ¡2

N¡1 X n= 1

439

l n c2n | vEn | 2 .

E ) are positive for all points Q, E we can Žnd an inŽmum c so that If all l n (Q E ) ¸ c > 0 8Q, E n. Thus we obtain a simple approximation for the l n(Q dynamics of DE ( t):

® ®

®2 ®

d ®DE (t)® dt

· ¡2c

N¡1 X n= 1

®

®2

® ® c2n | vEn | 2 = ¡2 ®DE (t) ®

and

® ® ® ® ®DE ( ) ®2 · ®DE (0) ®2 exp(¡2c ). t ® t® ® ® Because the parameter c is positive, the quantity kDE ( t)k2 converges toward null. To emphasize the relationship of the results of this article with the learning matrix from the previous section, we indicate shortly how a learning matrix E ) n = 1, . . . , N ¡ 1g can be derived from the Želds of the eigenvectors fE vn ( Q E ) n = 1, . . . , N¡1g. This follows easily after adjoint and the eigenvalues fl n ( Q E ¤ , vE¤ n = 1, . . . , N ¡ 1g are introduced so that: vectors fQ n » 1 if n = k ¤ EQ E ¤ = 1, E ¤ = vE¤ Q E = 0, E E ¤ = Q vEn Q v v k n n 0 otherwise. In the case of orthogonal eigenvectors vEn , all adjoint vectors can be easily deŽned: E ¤ ( vEn Q E) vEn ¡ Q E ¤ = (1, 1, . . . , 1)I . Q vE¤n = 2 | vEn | E ) can thus be written as a product of three matrices: L( Q E ) = V( Q E ) ¤ L (Q E ) ¤ V ¤T ( Q E ), L (Q

E ) is a matrix from the components of eigenvectors including Q, E where V ( Q E ) is a matrix composed of adjoint vectors, and L (Q E ) is the transpose V ¤T ( Q a diagonal matrix whose elements are eigenvalues with the corresponding eigenvalue l Q = 0. 4 Approximation of Continuous Probability Distributions

In this section we generalize our results for the case of one continuous variable. First, we introduce and clarify the corresponding notions for our

440

Maxim Khaikine and Klaus Holthausen

description of learning processes. This will allow us to describe the convergence toward a continuous probability density. Finally, after a simpliŽcation, we arrive at possible learning rules for the approximation of probability densities. Without loss of generality, we assume a discrete stochastic process in which a continuous variable x is distributed on the interval I = [0, 2p ] with a density P( x). At every time, there is a hypothesis Qt ( x) about the objective distribution P(x). Let us suppose that both functions are smooth on I. We note that in the space of all smooth function, the hypothesis Q( x) and the real distribution P (x) always lie on a manifold Q that is deŽned by the normalization condition: Z 2p Qt ( x)dx = 1. 0

The adaptation process will be triggered by random stimuli y 2 I. We start with an arbitrary initial hypothesis Qt ( x) satisfying the normalization condition. In the tth step, a stimulus yt 2 I will be chosen according to the probability distribution P( x). Thus, the learning process can be described as a discrete adaptation of the hypothesis: QtC1 ( x) = Qt ( x) C 2 K( x, yt ).

Here, the function K( x, y) corresponds to the learning matrix from the previous section. Because we want the hypothesis Q always to belong to Q , the kernel function K(x, y) has to be smooth. In general, the learning rule K(x, y) depends on the actual hypothesis Qt ( x). To emphasize this, we write KQ (x, y). After we pass over to the continuous description, with 2 ! 0, we obtain the dynamics of Q( x): dQ( x, t) = dt

Z

2p 0

P( y)KQ ( x, y) dy.

(4.1)

In order to satisfy the normalization condition at each instance of the adaptation process, the following equation must be always fulŽlled: Z

2p 0

KQ ( x, y) dx = 0 8y and 8Q(x).

(4.2)

Moreover, the correct hypothesis Q( x, t) = P( x) must be a Žxed point of the dynamics of our adaptation process. Thus, the second condition for the kernel KQ ( x, y) is given by: Z

2p 0

KQ ( x, y) Q( y) dy = 0 8x and 8Q(x).

(4.3)

Equations 4.2 and 4.3 can be regarded as a continuous generalization of conditions 2.3 and 2.4 applied in the case of discrete approximations. Thus,

General Probability Estimation Approach for Neural Computation

441

the goal is to Žnd an appropriate integral operator L ( Q ) with smooth kernel KQ ( x, y) that is deŽned on the manifold Q , fulŽlls conditions 4.2 and 4.3, and always describes the dynamics of the hypothesis Q(x, t), dQ( x, t) = L (Q) P, dt that leads to a convergence toward the objective distribution P( x). We apply the standard deŽnition of convergence: A hypothesis Q( x, t) converges toward the distribution P(x) with the time t, if for every g > 0 one can Žnd T so that for every time tO ¸ T: |Q(x, tO) ¡ P( x)| · g. By analogy with the previous section, we introduce the set of eigenvectors fvQ,n n = 0, . . . , 1g (in this case, they are real functions on I) and corresponding eigenvalues fl Q,n n = 0, . . . , 1g for the linear operator L ( Q ) in every point of the manifold Q , and note some general properties of them. Equation 4.3 shows that every point of Q is an eigenvector for the operator L ( Q ) with corresponding eigenvalue l Q,0 = 0. With the same arguments as in section 3, one can see that all other eigenvalues fl Q,n n = 1, . . . , 1g must be different from zero to avoid the existence of an uncorrect stable hypothesis. Furthermore, all eigenvectors vQ,n (x) associated with nonzero eigenvalues have to belong to the class Z

2p 0

vQ,n ( x)dx = 0

(4.4)

and build a basis of this space. Following our constructions for the discrete case, we introduce a distance D ( x, t) between the real distribution and the hypothesis D ( x, t) = P( x) ¡ Q(x, t). The dynamics of the distance function is given by: dD ( x, t) dQ(x, t) = ¡ dt dt = ¡L ( Q ) P = ¡L ( Q )D = ¡

1 X

l Q,n cn vQ,n ( x).

(4.5)

n= 1

As a difference between two elements of Q , D ( x, t) satisŽes the condition Z 2p D ( x, t) dx = 0 8t 0

and can be expanded using the basis fvQ,n g: D ( x, t) =

1 X 1

cn vQ,n ( x).

442

Maxim Khaikine and Klaus Holthausen

And Žnally for the dynamics of kD ( x, t)k2 : dkD ( x, t)k2 = ¡2D ( x, t)L ( Q )D dt = ¡2

(

1 X n= 1

cn vQ,n ( x)

!

(

1 X

! l Q,k ck vQ,k ( x) .

k= 1

Up to this relation, our constructions for the continuous distribution approximation were the same as for the discrete case. But here the implementation of an orthogonal system of eigenvectors with corresponding positive eigenvalues in every point of Q and the proof of convergence fails. Indeed, if the kernel function KQ (x, y) of the integral operator L ( Q ) is smooth, the operator is compact. The inŽnite set of nonzero eigenvalues of such an operator has to converge toward null (limn!1 l n = 0), and thus no positive inŽmum can be found. Thus, according to theorem 1, we cannot Žnd an estimation for the convergence speed. In order to construct a class of possible learning rules, we introduce some simpliŽcations. First, we make an assumption about the form of the eigenvectors of the learning operator: in every point Q of the manifold Q , we want to construct an operator with uniform eigenvectors given as trigonometric polynomials in complex form: vn ( x) = exp(inx)

n = ¡1, . . . , 1, n 6 = 0.

All of these functions are orthogonal, independent, satisfy condition 4.4, and represent, together with the trivial eigenvector Q( x), a basis of the functional space. The second supposition is that all corresponding eigenvalues are positive, do not depend on the actual hypothesis, and satisfy the condition 1 X ¡1

|l n | < 1.

Because all vectors fvn ( x)g are Žxed, the dynamics of D (x) in equation 4.5 can be described only by the change of the coefŽcients cn : X dcn (t) X dD ( x, t) = l n c n ( t) v n ( x ) vn ( x) = ¡ dt dt n6 = 0 n6 = 0 and dcn ( t) = ¡l n cn ( t). dt

(4.6)

After this simpliŽcations we can prove the following convergence theorem.

General Probability Estimation Approach for Neural Computation

443

If the dynamics of all Fourier coefŽcients of a smooth function g( x, t) x 2 [0, 2p ] is given by Theorem 2.

dcn (t) = ¡l n cn (t), dt where alll n are positive, then the function g(x, t) converges toward null. Precisely: 8g > 0 9 T so, that 8t ¸ T: Proof.

| g( x, t)| · g

8x 2 [0, 2p ].

The absolute value of the function f ( x, t) can be estimated as fol-

lows:

­ ­ ­ ­ X 1 X ­ ­ | g( x, t)| = ­ cn ( t) exp ( inx) ­· |c (t)|. ­n ­ ¡1 n Moreover, a well-known fact from Fourier analysis is that for every smooth function on I, the relation 1 X ¡1

|cn | < 1

holds. Therefore we can Žnd a positive number L so that ¡L X n= ¡1

| cn (0)| C

1 X n= L

|cn (0)| ·

g . 2

Using equation 4.6 and l n > 0 we obtain |cn ( t)| = |cn (0)| exp(¡l n t) · |cn (0)| and ¡L X n= ¡1

| cn ( t)| C

1 X n= L

| cn ( t)| ·

g . 2

We introduce a partial sum of the coefŽcients: S L ( t) =

L X n= ¡L

| cn ( t)| .

444

Maxim Khaikine and Klaus Holthausen

From the set of positive numbers fl n n = ¡L, . . . , Lg we can always Žnd a minimum c > 0, so that all l n ¸ c . Thus, the dynamics of SL ( t) can be easily estimated: L L X X dSL ( t) = ¡ l n |cn ( t)| · ¡c |cn ( t)| = ¡c S( t). dt n= ¡L n= ¡L

This differential equation yields S( t) · S(0) exp(¡c t), and 8t ¸

1 c

) the relation S(t) · ln( 2S(0) g

| g( x, t)| ·

1 X

¡L X

| cn ( t)| =

n= ¡1

n= ¡1

g 2

holds. Finally:

|cn (t)| C

1 X n= L

| cn ( t)| C S( t) ·

g g C · g. 2 2

In order to formulate the concrete form of the kernel for the operator

L ( Q) with the given eigenvectors and eigenvalues, we introduce an adjoint

basis fQ¤ (x), v¤n ( x)g that fulŽlls the following conditions: Q( x)Q¤ ( x) = 1, » 1 v¤n ( x) vk ( x) = 0

vn (x) Q¤ (x) = v¤n ( x) Q( x) = 0, if n = k otherwise.

(4.7)

The multiplication here is deŽned via the integral: Z f ¤g=

2p 0

f ( x) g( x) dx.

Then for any arbitrary smooth function g(x) = of the operator L ( Q) can be written as: L ( Q ) g( x) =

1 X

Z

2p

n(

l n cn v x ) =

n= ¡1

0

(

P1

¡1 cn vn ( x), the application

1 X n= ¡1

! l n vn ( x) v¤n ( y)

g( y)dy.

The functions satisfying the conditions (see equation 4.7) are deŽned as follows: ¤

Q ( x) = 1,

v¤n

1 = 2p

(

Z exp (¡inx) ¡

n = ¡1, . . . , 1, n 6 = 0.

2p 0

! Q( z) exp (¡inz)dz

General Probability Estimation Approach for Neural Computation

The integral kernel is a smooth function if by: 1 X

KQ (x, y) =

ln

n= ¡1,n6 = 0

Z

¡

2p 0

(

P1

n= ¡1

|l n | · 1 and is given

exp ( in(x ¡ y)) 2p 1 X

exp ( in(x ¡ z)) ln 2p n= ¡1,n6 = 0 Z

= f ( x ¡ y) ¡

445

0

2p

Q(z) f ( x ¡ z)dz

! Q( z) dz

(4.8)

P exp ( inx) with the function f ( x) := . n6 = 0 l n 2p In this section, the detailed analysis of the dynamics of the approximation has been developed and the convergence has been proved. In order to apply this framework to discrete operating systems (e.g., computer simulations of approximation processes) we emphasize that the basic equations for the presented learning algorithm are 4.1 and 4.8. From these, a local algorithm that is based solely on adaptations of an internal hypothesis that is triggered by external stimuli can be derived. The result of a computer simulation in which the objective stimulus distribution P(x) is compared with the evolved hypothesis Q( x) after 10,000 iterations is presented in Figure 1. The start hypothesis was a uniform distribution, and the generating function for the integral kernel (see equation 4.8) was a gaussian function f ( x) = exp(¡x2 / 10). 5 Recognition of Regression Functions

One of the most important problems in neural processing and approximation theory is the recognition of statistical dependences between variables. More precisely, if we have two random variables x and w, which are correlated and distributed on the interval I = [0, 2p ], the conditional average of w for each x deŽnes a function W(x), the so-called regression function (RF), Z 2p W(x) = hw|xi = Pv ( w| x) wdw, 0

where Pv ( w|x) is a conditional probability density of w by the given value of x. We see that the deŽnition of RF is not symmetrical for w and x. To emphasize the basic difference between the variables, we will call x a stimulus and the variable w the response. The approximation of RF can be intuitively understood as a recognition of objective functional dependence W(x) on the signal with symmetrical noise. In this section we suppose that the W(x) and both probability densities— P( x) and Pv ( w| x)—are smooth on I and the distribution P( x) is not equal to null in every point of the interval I.

446

Maxim Khaikine and Klaus Holthausen

Figure 1: Simulation of a continuous density approximation for an arbitrarily chosen objective distribution P(x). Ten thousand iterations of the learning rule represented by equation (14) for the local update of the kernel function together with the local update of the hypothesis: QtC1 (x) := Qt (x) C 2 K(x, yt ) (learning reate 2 = 10¡5 ) were performed. The generating function for the kernel was the gaussian f (x) = exp(¡x2 / 10) that fulŽlls the conditions of theorem 2. As subjective initial hypothesis Q(x, 0), a uniform distribution was assumed. Due to the convergence of the algorithm, quite a good representation could be achieved.

Generally there are many possibilities for applying the approach of ideal approximation described in the previous sections for the recognition of W(x). Here we describe a simple one that consists of two simultaneous approximation processes and obviously converges toward RF. As in sections 3 and 4, we introduce a hypothesis V( x, t) about the regression function W(x). Moreover, we consider an additional function F( x, t) and the hypothesis Q( x, t) about the distribution P( x). The hypothesis Q( x, t) will be changed sequentially after the stimulus y was received, as deŽned by equation 4.1 and 4.8, so that lim Q( x, t) = P(x).

t!1

(5.1)

The local dynamics of F(x, t) is in general given by: F( x, t C D t) ¡ F(x, t) = 2 KF ( x, yt , wt ),

where KF ( x, yt , wt ) is a kernel function depending on the actual value of F(x, t). We note that compared with the previous section, the kernel has

General Probability Estimation Approach for Neural Computation

447

three parameters: the signal coordinate x, the stimulus y, and the response w. After transition to the continuous description, the average dynamics of F( x, t) can be written as: dF( x, t) = dt

Z Z

=

2p 0 2p 0

Z Z

2p 0 2p 0

P( y, w)KF ( x, y, w) dydw P( y) Pv ( w| y) KF (x, y, w) dydw.

(5.2)

If we assume the kernel function KF ( x, y, w) to be given in the form Z 2p Q ( ) ( ) KF x, y, w = w f x ¡ y ¡ F( z, t) fQ( x ¡ z)dz 0

where fQ( x) is an arbitrary function with real and positive Fourier coefŽcients. We obtain after some calculations: Z 2p Z 2p dF( x, t) = W(y)P(y) fQ( x ¡ y) dy ¡ F(z, t) fQ( x ¡ z) dz. dt 0 0 This equation describes the convolution of D ( x, t) = (W(x) P(x) ¡ F(x, t)) with the function fQ( x). Hence, the dynamics of Fourier coefŽcients of D ( x, t) satisŽes equation 4.6 and, as given in theorem 2, D ( x, t) ! 0. Finally: lim F( x, t) = W(x)P(x).

t!1

(5.3)

The hypothesis V ( x, t) about the regression function W(x) can be deŽned every time as follows: V( x, t) =

F( x, t) . Q(x, t)

And together with equations 5.1 and 5.3 and condition 2, we obtain: lim V( x, t) =

t!1

limt!1 F( x, t) = W(x). limt!1 Q( x, t)

(5.4)

In order to study the performance of this algorithm, a simultaneous approximation of the RF W(x) and the estimation of the objective distribution of the stimuli x was computed via numerical integration. The stimuli were distributed with the same probability density P( x) as in Figure 1. Figure 2 shows a comparison between the regression function W(x, t) and the corresponding approximation V( x, t) after 10,000 learning steps. The additional white noise was distributed in the interval [¡100, 100]. The start hypothesis about the RF was a constant null function.

448

Maxim Khaikine and Klaus Holthausen

Figure 2: Simulation of the approximation of a regression function W(x), which was arbitrarily chosen. The underlying probability density P(x) was the same as in the foregoing simulation (see Figure 1). The initial hypothesis V(x, 0) about the regression function was a constant null function. Again, an appropriate representation of the true regression function could be achieved.

The corresponding dynamics of the distance kW(x) ¡ V( x, t)k between the regression function and the iterated hypothesis is shown in Figure 3. We performed a series of integrations with various regression functions and usually achieved satisfactory approximations; the distance converged toward null. 6 Discussion

Neural networks can be regarded as an extension of conventional techniques in statistical pattern recognition (Bishop, 1995). And, of course, certain network models can be derived from approaches to probability density estimation. For example, a principled alternative to the widely used selforganizing map of Kohonen (1982) could be developed that has the key property that it deŽnes a probability density model (Bishop, Svensen, & Williams, 1998). Accordingly, possible generalizations and applications of the presented approach can be pointed out. Essentially we described a framework for the learning processes in neural systems that can be regarded as autonomous entities that adapt its internal structure on the basis of subjective probabilities constructed on the basis of

General Probability Estimation Approach for Neural Computation

449

Figure 3: The monotonous decrease of the distance kW(x) ¡V(x, t)k between the real regression function and the iterated hypothesis was measured during the foregoing simulation (see Figure 2). After 30,000 iterations the initial distance was reduced by a factor ¼300.

randomly received input signals (Jost, Holthausen, & Breidbach, 1997). The approximation algorithms can be easily extended to higher-dimensional cases. To do this, we have to replace all one-dimensional hypotheses with the corresponding many-dimensional ones and do the same for the kernels. All functions thus will be represented by many-dimensional Fourier series, but the main convergence properties will remain. For example, the density estimation for two-dimensional distributions can be performed in the following manner. We assume that the objective distribution P( x1 , x2 ) and the according hypothesis Q( x1 , x2 ) are given. An appropriate generating function for the kernel is deŽned by the two-dimensional gaussian f ( x1 , x2 ) = exp(¡(x1 Cx2 ) 2 / s 2 ). The adaptation of the kernel after a stimulus (y1 , y2 ) was received is given by the two-dimensional extension of equation 4.8. Thus, our approach can be applied to the generation of topological feature maps. Because of the optimal approximation capabilities enclosed in each element for the recognition of regression functions in higher-dimensional cases, our algorithms could be a good basis for the mathematical investigation of complex structures that are composed of these elements: artiŽcial neural networks. Recent work on information processing and storage at the single-cell level has revealed previously unimagined complexity (Koch, 1997). Thus, one may assume that even single neurons could perform calculations that correspond to the presented approximation model.

450

Maxim Khaikine and Klaus Holthausen

The theorems provided here concerning the convergence of the approximation algorithms are only preliminary. Especially the restriction to trigonometric polynomials presumably could be dropped in the course of more comprehensive mathematical analysis. Acknowledgments

This research was supported by the Thuringer ¨ Ministerium fur ¨ Wissenschaft, Forschung und Kultur (B 301-96028). We thank Paul Ziche for critically reading an earlier draft of the manuscript for this article. We are grateful to one of the anonymous referees who provided several detailed suggestions to clarify our derivations. References Bishop, C. M. (1995). Neural networks for pattern recognition. New York: Oxford University Press. Bishop, C. M., Svensen, M., & Williams, C. K. I. (1998). GTM: The generative topographic mapping. Neural Comp., 10, 215–234. Girosi, F., & T. Poggio (1990). Networks with the best approximation property. Biol. Cybern., 63, 169–176. Jost, J., Holthausen, K., & Breidbach, O. (1997).On the mathematical foundations of a theory of neural representation. Theory Bioscienc., 116, 125–139. Koch, C. (1997). Computation and the single neuron. Nature, 385, 207–210. Kohonen, T. (1982). Self-organized formation of topologically correct feature maps. Biological Cybernetics, 43, 59–69. Kurkova, V. (1992). Kolmogorov’s theorem and multilayer neural networks. Neural Networks, 5, 501–506. Pfaffelhuber, E. (1972). Learning and information theory. Intern. J. Neuroscience, 3, 83–88. Scarselli, F., & Tsoi, A. C. (1998). Universal approximation using feedforward neural networks: A survey of some existing methods, and some new results. Neural Networks, 11, 15–37. Received April 13, 1998; accepted January 27, 1999.

LETTER

Communicated by Richard Golden

On the Synthesis of Brain-State-in-a-Box Neural Models with Application to Associative Memory Fation Sevrani Kennichi Abe Department of Electrical Engineering, Faculty of Engineering, Tohoku University, Sendai 980-8579, Japan In this article we present techniques for designing associative memories to be implemented by a class of synchronous discrete-time neural networks based on a generalization of the brain-state-in-a-box neural model. First, we address the local qualitative properties and global qualitative aspects of the class of neural networks considered. Our approach to the stability analysis of the equilibrium points of the network gives insight into the extent of the domain of attraction for the patterns to be stored as asymptotically stable equilibrium points and is useful in the analysis of the retrieval performance of the network and also for design purposes. By making use of the analysis results as constraints, the design for associative memory is performed by solving a constraint optimization problem whereby each of the stored patterns is guaranteed a substantial domain of attraction. The performance of the designed network is illustrated by means of three speciŽc examples. 1 Introduction

In this article we consider a class of synchronous discrete-time neural networks described by the difference equation of the form x( k C 1) = g ( x (k) C a( Wx ( k) C b )),

x (0) = x 0 ,

(1.1)

where x ( k) 2 Rn is the state vector at time k, a is the step size, W = [wij ] 2 Rn£n is the connection (or weight) matrix, b = [b1 , . . . , bn ]T 2 Rn is a bias vector, and g (x ) = [g(x1 ), . . . , g( xn )]T

is a vector valued function where 8 if xi ¸ 1 <1, if ¡1 < xi < 1 g( xi ) = xi , : ¡1, if xi · ¡1. Neural Computation 12, 451–472 (2000)

c 2000 Massachusetts Institute of Technology °

452

Fation Sevrani and Kennichi Abe

The saturation nonlinearity g(¢) has the effect of restricting the allowable values of the state vector x (k) to the closed unit hypercube Hn = [¡1, 1]n . P Neural network model 1.1, which has been proposed by Hui and Zak (1992) (see also Golden, 1993), can be viewed as a generalization of Anderson’s brain-state-in-a-box (BSB) neural model (Anderson, Silverstein, Ritz, & Jones, 1977) where the connection matrix W is symmetric and the bias vector b = 0. The BSB model has been used primarily to model effects and mechanisms seen in psychology and cognitive science. (For further discussion on uses of the BSB models and additional references on this subject, see Anderson, 1993.) The application we discuss here is associative memory. In the implementation of associative memories, the network is designed to store a given set of vectors (desired memory patterns) as its asymptotically stable equilibrium points. Since the nearby points in the domain of attraction of each stored pattern can be considered as representing partial information (given a suitable metric), a design that ensures a large domain of attraction around each of the stored patterns will necessarily have the effect of increasing the capability of the network to retrieve full information given a noisy or incomplete version of it. In this regard, one is also interested in minimizing the number of extraneous memories, that is, undesirable (spurious) asymptotically stable equilibrium points of the network, which may disturb the retrieval of the desired memory patterns. Various aspects and difŽculties involved in the implementation of associative memories concerning different neural network models are discussed in Michel and Farrell (1990). Our emphasis here is on the presentation of analysis results and methods for the design of associative memory by means of the neural network model 1.1 based on the above framework. P (1992) and Hui, Lillo, and Zak P (1993) provided an analysis Hui and Zak of the dynamic behavior and examined the stability of neural networks in the form 1.1 under various assumptions of the connection matrix W and bias vector b . The analysis results were interpreted from the point of view of the implementation of associative memory using the model 1.1. However, no speciŽc information was provided on how to accomplish this. In particular, P (1992) examined the case where the connection matrix W Hui and Zak is strongly diagonally dominant and gave conditions that were necessary and sufŽcient for the network to have asymptotically stable equilibrium points at every vertex of the hypercube. This result, however, imposes severe limitations on the extent of the domain of attraction of the desired patterns to be stored as asymptotically stable equilibrium points of the network. In our approach to studying the dynamic properties of the neural network model 1.1, we do not make use of any of the assumptions in Hui and Zak (1992) and Hui et al. (1993). Instead, we show that if the matrix W has zero diagonal elements, the network can have asymptotically stable equilibria only at the vertices of the hypercube (which are normally used for storing the memory patterns), although not necessarily every vertex of

Synthesis of Brain-State-in-a-Box Neural Models

453

the hypercube is asymptotically stable (see also Perfetti, 1995) and that under mild assumptions, all other equilibria are unstable. In the analysis of the stability properties of the network, we will establish results concerning the extent of the domain of attraction of the asymptotically stable equilibrium points located at the vertices of the hypercube that are useful in the analysis of the retrieval performance of the network. Using the provided analysis results as constraints, the synthesis problem is formulated as a constraint optimization problem, which amounts to expanding the domain of attraction around each of the stored patterns and also minimizing the number of spurious asymptotically stable equilibria in the vicinity of those patterns. The proposed formulation of the synthesis problem has exibility in dealing with certain restrictions on the interconnection structure of the network, such as symmetric, nonsymmetric, and partially prespeciŽed or sparse interconnections. One can take advantage of the sparsity constraints (expressed as prespeciŽed zero elements in the connection matrix) from a computational perspective for solving the synthesis problem and in hardware implementation where the reduction in the number of connections is desirable. Furthermore, the proposed formulation is shown to have the advantages of practical efŽciency, in the sense that there exist powerful algorithms for solving the problem. What is also important is that the design can be carried out by another neural network, similar to the combined learning-production architecture proposed by Guez, Protopopsecu, and Barhen (1988). Several neural network models have been applied to several classes of constraint optimization problems and have shown promise for solving such problems. Taking advantage of the degrees of freedom and exibility of our approach of the synthesis problem, we investigate further the effect of the connection weights dispersion on the error-correction capabilities of the network in the presence of random errors. In doing so, we are motivated by the results in the case of optimal associative mappings (see Kohonen, 1987). These results suggest that the error-correction properties of such mappings are closely related to some smoothness prior, in the sense that all vectors that lie in a small neighborhood of the input vectors are mapped into a correspondingly small neighborhood of the output vectors. In the case of neural network model 1.1, a small connection weights dispersion is considered a possible way of encoding this a priori information. The effect of connection weights dispersion on the error-correction capabilities of the network is investigated by extensive simulations and will be illustrated by an example in section 4. The article is organized as follows. Section 2 presents theoretical results concerning local qualitative properties and global qualitative aspects of the considered network. In section 3, we present our approach to the synthesis problem, along with some additional heuristic arguments aiming at providing insight into the retrieval performance of the network. In this section we also discuss techniques for solving the synthesis problem. The applica-

454

Fation Sevrani and Kennichi Abe

bility of the results is demonstrated by means of three examples presented in section 4. Concluding remarks are given in section 5. 2 Analysis Results

In this section, we Žrst address the local stability properties of the equilibrium points of system 1.1. Next, we present results concerning the global stability properties of this system. Our main emphasis is on establishing results concerning the extent of the domain of attraction of the asymptotically stable equilibrium points located at the vertices of the hypercube H n , which are expressed in a form suitable for the design purposes. Recall Žrst that a point x * 2 H n is an equilibrium point of system 1.1 if and only if ¡

¡

x * = g x * C a Wx * C b

¢¢

.

(2.1)

A systematic method for locating all equilibrium points of 1.1 can be found in Hui et al. (1993). Under various assumptions on the connection matrix W and vector b , these authors provided conditions whereby the stability or instability of equilibrium points can be determined. Note that in applications of neural network model 1.1 to associative memory, asymptotically stable equilibrium points at the vertices of H n are used to store the desired memory patterns. In this regard, we shall prove the following result (see Proposition 1), which excludes the existence of other possible nonbinary stable equilibria. We begin by making the following assumption, which we shall have the occasion to use in the subsequent analysis: Assumption A1. Every principal submatrix of W of size 2 £ 2 or larger, including W itself, is hyperbolic; that is, none of their eigenvalues has a zero

real part.

Proposition 1. Let all diagonal elements of matrix W be zero and suppose that W satisŽes assumption A1. Then, for almost all b 2 Rn (in the sense of Lebesgue

measure), (i) the set of equilibrium points for system 1.1 is Žnite and (ii) any stable equilibrium point of system 1.1 is located at a vertex of the hypercube. Proof.

See the appendix.

Now, concentrating on the vertices of the hypercube, let À 2 f¡1, 1gn denote a vertex of H n . Using a continuity argument (see Hui et al., 1993), it can be shown that if ui ( W À C b )i > 0,

for i = 1, . . . , n,

(2.2)

Synthesis of Brain-State-in-a-Box Neural Models

455

then À is an asymptotically stable equilibrium point of system 1.1. Note that the above stability result is local, in the sense that there will be a sufŽciently small, open neighborhood U (À ) of À in Rn such that any trajectory of 1.1, corresponding toTan initial condition x 0 2 H n , converges to À whenever x 0 belongs to U (À ) H n . In the subsequent analysis we are concerned with a more global stability analysis for the equilibrium points of system 1.1 located at the vertices of the hypercube. More precisely, given that a vertex À of H n is asymptotically stable, depending on the positive margin with which condition 2.2 is satisŽed, we would like to know the extent of this stability—that is, for which points it is true that any trajectory of system 1.1 initiating from those points will converge to À . Let Cr (À ) ½ H n be a bounded set deŽned as Cr (À ) = fx 2 Hn | kx ¡ À k1 · 2r g,

(2.3)

where k ¢ k1 is the l1 vector norm on Rn , and r a positive scalar, 0 < r < n. We shall prove the following result: Let À be an asymptotically stable equilibrium point and let

Proposition 2.

0 < r < min i

ui ( W À C b )i , 2 maxj |wij |

i, j = 1, . . . , n.

(2.4)

Then, every trajectory of system 1.1 starting from a point in Cr ( À ) cannot escape Cr ( À) and will converge to À . Before proceeding further, it will be convenient to introduce some additional notations and geometrical considerations that will be useful in the proof. Let V = fÀ(0) , À (1) , . . . , À (n) g be a set of n C1 afŽnely independent vectors 1 in Rn such that À(0) = À and À( i) = À ¡ 2r ui e i , where e i , i = 1, . . . , n, is the ith column vector of the identity matrix. Let co(V) denote the ndimensional simplex expressed as a convex combination of the vertices À (0) , À(1) , . . . , À ( n) , that is, ( ­ ) ­ X n n X ­ ( i) co(V) = x ­ x= l iÀ , l i = 1, 0 · l i 2 R for all i . (2.5)

­

i= 0

i= 0

Equivalently, co(V) can be expressed by a set of linear inequalities fx 2 Rn | ujT x · hj , j = 1, . . . , n C 1g, where uj = uj ej , j = 1, . . . , n, u nC1 = ¡À;

hj = 1, j = 1, . . . , n, hnC1 = 2r ¡ n. Here each inequality ujT x · hj , j =

Recall that a set of n C 1 vectors fx (0) , . . . , x (n) g is afŽnely independent if and only if the translated set fx (1) ¡ x(0) , . . . , x(n) ¡ x (0) g is linearly independent. 1

456

Fation Sevrani and Kennichi Abe

1, . . . , n, deŽnes a closed half-space associated with tangent (or supporting) hyperplane to Hn at À . Accordingly, u TnC1 x · hnC1 deŽnes the closed halfspace associated with the hyperplane facing the vertex À . In this regard, each vector À ( i) lies on n linearly independent supporting hyperplanes of co(V); that is, each À ( i) satisŽes exactly n linearly independent equations ujT À ( i) = hj . Now suppose that x , | xi | · 1, i = 1, . . . , n, satisŽes u TnC1 x · 2r ¡ n. Then, u TnC1 ( x ¡ À ) · 2r or, equivalently, kx ¡ À k1 · 2r . Thus, the set Cr ( À ) can be represented as the intersection of the n-simplex, spanned by point À as well as points À(1) , . . . , À ( n) in Rn , and the hypercube Hn . In the proof of proposition 2 we shall make use of the following lemma (a proof of the lemma is given in the appendix following the approach of Golden, 1993.) The neural network model 1.1 can be represented as an uncertain linear system described by the state equation of the form Lemma 1.

x ( k C 1) = x (k) C aÀ ( k)( Wx ( k) C b ),

(2.6)

where À ( k) 2 Rn£n is a diagonal matrix with diagonal elements 0 · c i ( k) · 1. We can continue now with the proof of proposition 2. We will show Žrst that any trajectory of equation 1.1 originating in Cr ( À ) can never leave Cr (À ), that is, if x 0 2 Cr ( À), x (k) will remain in Cr ( À ) for all k > 0. Since, due to saturating nonlinearity, the state of the system is conŽned to the hypercube H n , in order to leave Cr ( À ), any trajectory originating in Cr ( À) must cross the face of Cr ( À ) opposing the vertex À , fx | hÀ , x i = n ¡ 2r , x 2 Cr (À )g, represented by the closed half-space hÀ , x i ¸ n ¡ 2r . Hence, in view of lemma 1, it sufŽces to show that if x 2 Cr ( À ), then hÀ, À (Wx C b )i ¸ 0. For this, observe that since Cr ( À) µ co(V), each point x 2 Cr ( À) is uniquely expressible as a convex combination of the form 2.5. Using the fact that l 0 C ¢ ¢ ¢ Cl n = 1, we have Proof of Proposition 2.

hÀ , À ( Wx C b )i =

n X j= 0

lj

n X i= 1

c i (¢)ui ( W À ( j) C b ) i .

(2.7)

By noting that À(0) = À and substituting for À ( j) = À ¡ 2r uj ej , we have by assumption that ui (W À ( j) C b ) i = ui ( W À C b )i ¡ 2r wij ui uj > 0,

for i, j = 1, . . . , n.

(2.8)

Since l j and c i (¢) are nonnegative, with l j adding up to 1, we conclude that hÀ , À( Wx C b )i ¸ 0.

Synthesis of Brain-State-in-a-Box Neural Models

457

To this end we have shown that Cr ( À) is an invariant set. Next we show that x (k) ! À as k ! 1 for any x0 2 Cr ( À). For this, without loss of generality, consider the case where Ài = 1, and let x ( k) 2 Cr ( À) such that xi ( k) 6 = 1. Then the ith component of x ( k) C a(Wx ( k) C b ) is 0 1 n X ( ) l j ( W À j C b ) i A > xi ( k), xi (k) C a @ j= 0

where the above inequality follows from equations 2.4 and 2.8. Thus xi ( k C 1) > xi (k). The case where ui = ¡1 is similar. Continuing these arguments for x (k C 2), x( k C 3), . . . , it can be shown that the corresponding trajectory will monotonically converge to À as k ! 1. The next result addresses the existence of other binary equilibrium points of system 1.1 in the immediate vicinity of Cr (À ). Let wii = 0 for i = 1, . . . , n. Suppose that À is an asymptotically stable equilibrium point, and let k = dmini (ui (W À C b ) i / 2 maxj |wij | )e, where dxe denotes the smallest integer ¸ x and i, j = 1, . . . , n. Then none of vertices À( k) at Hamming distance dH (À , À( k) ) · k is an equilibrium point. Proposition 3.

By assumption there exists r > 0 satisfying equation 2.4 such that from proposition 2, and in view of the deŽnition for k, none of the vertices of the hypercube within the Hamming distance k ¡ 1 from À, that is, ( ) dH ( À , ÀQ k¡1 · k¡1, ÀQ 6 = À , can be an equilibrium point. Now let À (k¡1) be an arbitrary vertex with dH ( À, À (k¡1) ) = k ¡1, and let J ½ f1, . . . , ng of cardinalP ( ) ity | J| = k ¡ 1 such that À (k¡1) = À ¡ 2 j2J uj ej . Let À ( k) = À ( k¡1) ¡ 2ui k¡1 e i Proof.

( )

such that ui k 6 = ui , where i 2 / J, i = 1, . . . , n. By assumption, we have that ³ ´ X ui W À( k¡1) C b = ui ( W À C b ) i ¡ 2ui wij uj > 0, i

for i = 1, . . . , n.

j2J

( ) Since wii = 0 8i, we obtain ui k (W À (k) C b ) i = ¡ui ( W À ( k¡1) C b ) i < 0, i 2 / J. Hence, from equation 2.1, it follows that À (k) cannot be an equilibrium point.

An immediate consequence of proposition 3 is the following: Let wii = 0 8i. Suppose that À is an asymptotically stable equilibrium point. Then none of the vertices of the hypercube at Hamming distance one from À can be an equilibrium point. Corollary 1.

458

Fation Sevrani and Kennichi Abe

In the foregoing results, no considerations regarding the global stability of system 1.1 are made. Recall that a neural network (such as network 1.1) is called globally stable if every trajectory of the network converges to some equilibrium point. This guarantees that periodic and chaotic motions cannot exist in the network. Theorem 1, which follows, provides conditions on the connection matrix W that guarantee the global stability of system 1.1. In the proof of theorem 1 we will make use of an additional assumption: Given assumption A1, system 1.1 has a Žnite number of equilibrium points. Assumption A2.

Assumption A2 seems reasonable. In fact, if matrix W satisŽes assumption A1, and, in addition, wii 6 = 0 8i, every principal matrix of W is of full rank. Then, following similar arguments as in the proof of proposition 1. (see also Hui et al., 1993), it can be shown that system 1.1 cannot have an inŽnity of equilibrium points. When wii = 0 8i, as shown in the proof of proposition 1, assumption A2 is true for almost all b 2 Rn , with the possible exception of those belonging to a set of measure zero. Suppose that system 1.1 satisŽes assumption A2 and W = W T is either positive deŽnite or a < 2/ |l min |, where l min is the smallest eigenvalue of symmetric matrix W . Then any trajectory of 1.1 starting from an arbitrary point x 0 6 = 0, x 0 2 H n , will converge to an equilibrium point. Theorem 1.

Proof.

See the appendix.

In practice, due to noise, every trajectory of 1.1 will converge to an asymptotically stable equilibrium point, which, from proposition 1, with the additional constraint of matrix W having all of its diagonal elements zero, will be located at a vertex of the hypercube H n . 3 Application to Synthesis

In this section we address the problem of synthesis of associative memory using the neural network model 1.1: Synthesis Problem. Given a set of binary vectors (desired memory patm terns), fp ( k) gk= 1 , p (k) 2 f¡1, 1gn , and given the step size a > 0, Žnd W , b ,

such that:

1. p (1) , . . . , p ( m) are stored as asymptotically stable equilibrium points (memories) of system 1.1. 2. The domain of attraction of each stored pattern is as large as possible.

Synthesis of Brain-State-in-a-Box Neural Models

459

3. The total number of spurious asymptotically stable equilibria (extraneous stored patterns) is as small as possible. 4. The system 1.1 is globally stable; that is, every trajectory of the system converges to some equilibrium point. In regard to design criterion 4, the global stability of the network is not essential for storing the information. In fact, several existing design methods do not take in consideration the global stability of the network (see, e.g., Michel, Farrell, and Porod, 1989; Michel & Farrell, 1990). However, in terms of the reliability of retrieving the information—for example, when one expects the retrieval of a stored pattern from an input that is further away from that pattern—the global stability of the network is of great interest. Based on the analysis results provided in the preceding section, we approach the synthesis problem in the following manner. According to proposition 2, by selecting an n £ n matrix W = [wij ] and an n-dimensional vector b = [b1 , . . . , bn ]T such that for each k = 1, . . . , m the inequality n X j= 1

( ) ( k)

wij pi k pj

C p(i k) bi ¸ s,

i = 1, . . . , n,

(3.1)

is satisŽed for some given s > 0, we are guaranteed that p (1) , . . . , p ( m) will be stored as memories of system 1.1. Furthermore, associated with each p (k) there will be a stability region fx 2 Hn | kx ¡ p ( k) k1 < 2r k g, ( ) where r k = mini ( pi k ( Wp ( k) C b ) i / 2 maxj | wij | ), i, j = 1, . . . , n, such that any trajectory of the system originating within this region will converge to p (k) . Assuming that |wij | · wN for all i, j, the quantity r k gives a lower bound for the extent of the domain of attraction of a stored pattern,2 and thus, it provides a reasonable measure of the error-correction capabilities of the network. From an alternative viewpoint, given a stored pattern corresponding to an asymptotically stable equilibrium point of the network, an idea of how much the domain of attraction of that pattern can be expanded may also be given by the location of other asymptotically stable equilibrium points in the vicinity—that is, spurious asymptotically stable equilibrium points as well as points used for storing the (desired) memory patterns. Hence, the requirement of excluding spurious asymptotically stable equilibria, at least within some reasonable distance from each of the stored patterns, would allow for a larger extent of the domain of attraction for those patterns. It is clear that such an extent is bounded from above by the locations of the stored patterns, and in general it decreases as their number increases. 2

Note that the domain of attraction of the stored patterns is not known in its entirety.

460

Fation Sevrani and Kennichi Abe

In regard to the location of spurious asymptotically stable equilibrium points, from proposition 1 we have that, for system 1.1, with a zero diagonal connection matrix W satisfying assumption A1, those points must be located at the vertices of the hypercube. By proposition 3, provided that condition 3.1 is satisŽed with wii = 0 8i, we are guaranteed that within a Hamming distance rk = dr k e from a stored pattern p ( k) there can be no equilibrium point apart from p (k) itself. In particular (see corollary 1), none of the vertices at Hamming distance one from any of the stored patterns can be an equilibrium point, which has the implication that none of such vertices can be used as memory locations. However, we consider this a desirable property, since patterns that differ in only one component may be too close to allow for a signiŽcant amount of error correction. At this point some observations are in order. Suppose that for some k, 1 · k · m, we have n X j= 1

( k)

wij p(i k) pj

C p(i k) bi ¡ 2

X j2J

( k)

wij p(i k) pj

> 0

(3.2)

for at least one integer i 2 J ½ f1, . . . , ng, where J is any subset of integers of cardinality | J| ¸ rk (rk ¸ 1). Stated otherwise, for at least one integer i 2 J, the ith component of W À C b , at the vertex À that differs from p ( k) in ( ) the components pi k , i 2 J, points to p ( k) . From equation 2.1, it is clear that there cannot be any equilibrium point located at such a vertex. Note that when a signiŽcant number of components of W À C b , at À , point to p ( k) , we have reason to suspect that a trajectory starting at À will approach p ( k) and eventually reach that pattern. Hence, since s in condition 3.1 provides a lower bound for the Žrst two terms in equation 3.2, if we seek to maximize s (provided that | wij | · wN for all i, j), this will certainly reduce further the number of spurious asymptotically stable equilibria, and, in addition, it will ensure good retrieval of the memory patterns even from locations (vertices) more distant than those for which we can guarantee a perfect retrieval. To this end, a Žnal observation concerns the effect of the bound wN on errorcorrection capabilities of the network in the presence of random binary errors. Let us suppose that we require the retrieval of a memory pattern in the presence of errors due to the ipping of some of its components. We anticipate that the dispersion (or discontinuity) of the wij values will affect the strength of perturbations along all other components of that pattern. The larger the dispersion of the wij values, the more the information that is lost in the presence of those errors and, hence, the less likely that the memory pattern will be correctly retrieved. The combined effect of the increase of the positive margin s in equation 3.1 and the bound wN on the error-correction capabilities of the network will be demonstrated by practical simulations results presented in section 4. Now, putting it all together, a possible strategy to approach the synthesis problem can be formulated as follows:

Synthesis of Brain-State-in-a-Box Neural Models

461 m

Given a set of binary vectors fp ( k) gk= 1 , p (k) 2 f¡1, 1gn , to be stored as memories for system 1.1, and given a > 0, wN > 0, Žnd W = [wij ] 2 Rn£n , b = [b1 , . . . , bn ]T 2 Rn such that s is maximum subject to constraints n X s > 0, k = 1, . . . , m, (3.3) wij pki pjk C pki bi ¸ s, Synthesis Strategy.

j= 1

¡wN · wij · w, N i 6 = j, wii = 0,

i, j = 1, . . . , n.

(3.4)

By making use of the condition for the smallest eigenvalue, 3 l min ( W ), provided by theorem 1, the global stability requirement imposes the additional constraint W = W T , tI n C W > 0,

t = 2/ a,

(3.5)

where In is the n £ n identity matrix and the inequality sign above means that ( tIn C W ) is positive deŽnite. In the following we point out some computational aspects of solving the synthesis problem based on the formulated strategy. Let us consider Žrst the problem of determining W and b such that s is maximum subject to constraints 3.3 and 3.4. In such a case, the synthesis problem reduces to a general linear programming (LP) problem. Provided that a solution for W and b has been found, the resulting network is not guaranteed to be globally stable; however, we have guaranteed that all given patterns are stored as asymptotically stable equilibrium points of the network, having a substantial domain of attraction and hence a signiŽcant amount of error correction. From a computational perspective, based on an LP formulation, the synthesis problem can be solved efŽciently by using any of the polynomial time algorithms for LP (see, e.g., Lustig, Martsen, & Shanno, 1994), or by employing another neural network that offers computational advantages of massive parallel processing and fast convergence (see, e.g., Xia & Wang, 1995; Xia, 1996). Note also that restrictions on the interconnection structure that require that predetermined elements of W be zero can be incorporated simply by excluding those elements from constraints 3.3 and 3.4. With the addition of the global stability constraint 3.5, the synthesis problem, that is, the problem of maximizing s over equations 3.3 through 3.5, cannot be cast as a linear program. However, the problem we wish to solve can be cast as a semideŽnite program (see Vandenberghe & Boyd, 1996) in the variables W = W T , b and s. In semideŽnite programming (SDP), one minimizes a linear function subject to the constraint that an afŽne combination of symmetric matrices is positive semideŽnite, 4 which can be regarded 3

Note that since wii = 0 8i, W = WT cannot be positive deŽnite. Many common convex constraints—for example, linear and quadratic inequalities, eigenvalue, and matrix norm inequalities—can be expressed in this way. 4

462

Fation Sevrani and Kennichi Abe

as an extension of LP where the componentwise inequalities between vectors are replaced by matrix inequalities. SemideŽnite programs are convex optimization problems, and, as in LP, they can be solved in polynomial time by effective algorithms for the solution of these problems that rapidly compute the global minimum with nonheuristic stopping criteria (Nesterov & Nemirovsky, 1994). We note here that in terms of computational requirements, it is also possible to consider a simple heuristic algorithm for solving the synthesis problem. More speciŽcally, given the speciŽcations for a and w, N one can approach the synthesis problem by Žnding a feasible solution for W and b subject to constraints 3.3 and 3.4 (with the additional constraint that wij = wji for i 6 = j, if the global stability is required), for some increasing values of s, until the solution space is empty. In such a case, simple numerical relaxation techniques can be used, such as the technique used in Perfetti (1995). 5 In order to check for global stability, one can then compute the smallest eigenvalue of the symmetric matrix W . Referring to equation 3.5, if l min < ¡2/ a, the above procedure is repeated by specifying a lower value for the bound wN until one ends up with a solution for W and b that solves the synthesis problem.6 We conclude this section with a Žnal comment concerning the stability properties of the desired memory patterns, stored as asymptotically stable equilibrium points of the network, when considering inaccuracies incurred during the implementation process or limited precision implementation of system 1.1. Denoting those parameter inaccuracies, respectively, by D W and b , and performing a similar analysis as in Liu and Michel (1994), it can be shown that the stored patterns will retain their stability properties provided that the following condition holds:7 | W k1 C kb k1 < s, where k ¢ k1 denotes the matrix norm induced by l1 vector norm. Thus, increasing s will certainly increase the robustness of the stability of the stored patterns in the presence of implementation inaccuracies or parameter perturbations.

5 Experimentation with the algorithm proposed in Perfetti (1995) shows that, in terms of numerical stability and number of iterations, this algorithm is quite inferior (as it can be expected) compared, for example, with the interior point algorithms for LPs or SDPs. 6 GerLskorin theorem applied to zero diagonal symmetric matrix W states that all eigenvaluesPof W are located inside the circular disc centered at the origin and with radius n maxi j | wij |, which is the spectral radius of W . 7 Note that, when considering the inaccuracies during the implementation process, one is more concerned with the parameter inaccuracies in the connection matrix W rather than bias vector b .

Synthesis of Brain-State-in-a-Box Neural Models

463

At this point it should be noted that since large s will usually result in a large spread or dispersion of the wij values, as we conjectured above and also as simulation results show, it may cause the error-correction capabilities of the network to deteriorate in the presence of random binary errors. Furthermore, if we require the global stability of the network, simulation results suggest that the range of possible values of s is practically small. It is clear that there will be a trade-off between robustness, global stability, error-correction capability, and speed of operation, which remain to be chosen by the designer. 4 Simulations

In this section, we demonstrate the applicability of the theoretical results as well as the constituency of the arguments presented in the previous sections by means of three speciŽc examples. 4.1 Example. In this example we compare the retrieval performance of the neural network model, which is synthesized based on the strategy given in the previous section, with the performance of the networks designed by methods of Michel, Si, & Yen (1991) and Perfetti (1995) (in the special case of system 1.1 where b = 0). For the sake of comparison with the results of Michel et al. and Perfetti, in the design presented in this example, we choose a = 1 and require the network to be globally stable. We consider the following set of binary patterns of length n = 10 (cf. Michel et al., 1991, example 2.1) to be stored as asymptotically stable equilibrium points of equation 1.1: p (1) p (2) p (3) p (4) p (5) p (6)

= = = = = =

[¡1, 1, ¡1, 1, 1, 1, ¡1, 1, 1, 1]T , [1, 1, ¡1, ¡1, 1, ¡1, 1, ¡1, 1, 1]T , [¡1, 1, 1, 1, ¡1, ¡1, 1, ¡1, 1, ¡1] T , [1, 1, ¡1, 1, ¡1, 1, ¡1, 1, 1, 1]T , [1, ¡1, ¡1, ¡1, 1, 1, 1, ¡1, ¡1, ¡1]T , [¡1, ¡1, ¡1, 1, 1, ¡1, 1, 1, ¡1, 1] T .

Specifying a = 1 and wN = 0.5, we determine a 10 £ 10 dimensional matrix W = [wij ] and a 10-dimensional vector b by maximizing s over equations 3.3 and 3.4 and the additional symmetry constraint wij = wji for i 6 = j, which is solved by using linear programming. We obtain 2

W=

0 .0000 ¡0 .0071 6¡0 .4009 6¡0 .4419 6¡0 .5000 6 6 0 .3199 6 0 .1017 4¡0 .3457 ¡0 .0071 0 .2898

¡0.0071 0.0000 0.1931 ¡0.0335 ¡0.3104 0.0000 ¡0.3300 ¡0.1920 0.4552 0.2350

¡0 .4009 0 .1931 0 .0000 0 .2238 ¡0 .3885 ¡0 .3046 0 .1568 ¡0 .2677 0 .1931 ¡0 .3830

¡0.4419 ¡0.0335 0.2238 0.0000 ¡0.4009 0.0253 ¡0.2708 0.3909 ¡0.0335 0.1719

¡0 .5000 0.3199 0 .1017 ¡0 .3104 0.0000 ¡0 .3300 ¡0 .3885 ¡0.3046 0 .1568 ¡0 .4009 0.0253 ¡0 .2708 0 .0000 0.0780 0 .1773 0 .0780 0.0000 ¡0 .5000 0 .1773 ¡0.5000 0 .0000 ¡0 .0968 0.2946 ¡0 .3584 ¡0 .3104 0.0000 ¡0 .3300 0 .3921 ¡0.5000 ¡0 .3119

¡0.3457 ¡0.1920 ¡0.2677 0.3909 ¡0.0968 0.2946 ¡0.3584 0.0000 ¡0.1920 0.4185

3

¡0 .0071 0.2898 0 .4552 0.2350 0 .1931 ¡0.3830 7 ¡0 .0335 0.1719 7 7 ¡0 .3104 0.3921 7 0 .0000 ¡0.5000 7 ¡0 .3300 ¡0.3119 7 ¡0 .1920 0.4185 5 0 .0000 0.2350 0 .2350 0.0000

464

Fation Sevrani and Kennichi Abe

Table 1: Simulation Results for Example 1. Number of Binary Inputs Converging to the Desired Pattern Pattern Number 1 2 3 4 5 6 Convergence rate (%)

HD = 1

HD = 2

HD = 3

HD = 4

10 10 10 8 10 10

35 38 31 33 35 33

65 69 39 41 45 51

62 63 10 25 20 24

96.7

75.9

43.1

16.2

HD ¸ 5

Total

40 78 0 4 7 6

212 258 90 111 117 124

10.4

89.0

Table 2: Simulation Results for the Design of Michel et al. (1991, Example 2.1). Number of Binary Inputs Converging to the Desired Pattern Pattern Number 1 2 3 4 5 6 Convergence rate (%)

HD = 1

HD = 2

HD = 3

HD = 4

10 10 10 8 10 9

25 27 32 17 30 26

16 28 34 14 26 22

4 7 12 7 5 5

95.0

58.1

19.4

3.2

HD ¸ 5

Total

0 0 2 2 0 0

55 72 90 48 71 62

0.3

38.9

and b = [ ¡0 .1071

0 .5972 ¡1.4713 0.8561 0 .4372 0 .1175 1.1485 ¡0 .1649 0 .5972 ¡0.0323 ]T ,

corresponding to smax = 0.5. Also l min ( W ) = ¡1.5 which, together with the fact that W is symmetric, according to theorem 1, guarantees the global stability of the synthesized network. Using condition 2.3 it can be veriŽed that system 1.1 with W and b given above, in addition to the desired patterns p (1) , . . . , p (6) , also has three binary patterns stored as asymptotically stable equilibrium points (spurious memories). The simulation results summarized in Tables 1–3 are intended to compare the retrieval performance of the synthesized neural network 1.1, with the performance of the networks designed by the methods of Michel et al. (1991) and Perfetti (1995). In columns 2 to 5 in each table, the data in its row, corresponding to the pattern speciŽed in the Žrst column, show the number of binary inputs converging to that pattern out of all possible binary inputs at Hamming distance HD = 1, . . . , 4 and HD ¸ 5, which are used as initial conditions for the state of the network. Referring, for example, to the Žrst

Synthesis of Brain-State-in-a-Box Neural Models

465

Table 3: Simulation Results for the Design of Perfetti (1995, Example 2). Number of Binary Inputs Converging to the Desired Pattern Pattern Number 1 2 3 4 5 6 Convergence rate (%)

HD = 1

HD = 2

HD = 3

HD = 4

10 7 10 8 10 6

31 8 15 31 33 11

46 5 19 47 42 7

30 0 8 34 28 4

85.0

47.8

23.1

8.3

HD ¸ 5

Total

8 0 0 4 7 0

125 20 52 124 120 28

1.3

45.8

Note: Special case of neural network (1) with b = 0.

row of Table 1, for the given pattern p (1) , 35 out of 45 possible initial conditions at Hamming distance HD = 2 from p (1) converged to p (1) . The data in the last row of each table show the average rate of convergence to the correct stored pattern. 8 Also, the total in the last column of each table gives an indication of the attraction capability of each given pattern. As shown in Table 1, the synthesized network (see equation 2.1) attracted 89% of the binary inputs (there are 210 = 1024 possible binary inputs) to the desired patterns. The design method (eigenstructure method) of Michel et al. (1991) (example 2.1, with the design parameters t1 = 1.5, t2 = 0.5) attracted 39% of the binary inputs to the desired patterns, and the method in Perfetti (1995) (example 2, with the design parameter d = 0.75) attracted 46% to the desired patterns. At this point we make a few additional observations. As seen from Table 2, for inputs at Hamming distance HD > 1 from the desired memory patterns, the convergence rate of the network designed by the eigenstructure method drops faster than that for two other designs. In fact, the network designed by the eigenstructure method, with the design parameters speciŽed above, has in addition 12 spurious (nonbinary) memories that affect the retrieval efŽciency of the network. This is in contrast with the design where there are only three spurious memories, as reected by the convergence rates shown in Table 1. For the design of Perfetti (1995) there are eight spurious memories; six are the negatives of the desired patterns, which are automatically stored as asymptotically stable equilibrium points of the network. Next, numerous simulations suggest that the maximization of s in equation 3.3 is critical for the performance of the network, which is in line with the theoretical results and arguments presented in the previous 8 The convergence rate is speciŽed as the ratio of the number of the initial conditions that converge to the desired pattern to the number of all possible initial conditions from a given Hamming distance HD from the desired pattern.

466

Fation Sevrani and Kennichi Abe

Table 4: Simulation Results for Table 2. Number of Binary Inputs Converging to the Desired Pattern wN

smax / 2w N

HD = 1

HD = 2

HD = 3

0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0

0.60 0.60 0.60 0.59 0.57 0.52 0.46 0.41 0.37 0.34

96.83 96.83 96.83 96.83 96.42 91.50 84.58 82.25 81.00 78.83

78.40 77.61 76.71 74.78 70.17 60.47 53.82 50.82 49.71 47.45

44.21 43.32 42.11 39.87 37.76 33.63 30.39 28.55 28.11 27.77

Number of spurious states 2.35 2.45 2.50 2.40 2.40 2.80 3.40 3.60 3.90 3.90

sections. Furthermore, by considering wN (the upper bound on the absolute values of the elements of connection matrix W ) as a design parameter, we observe that low values of w, N as the one chosen in this example, improve the retrieval efŽciency of the network in the presence of errors (see the following example). This approach also allows us to design a globally stable network by solving a linear programming problem, and thus the possibility of designing the neural network (cf. equation 1.1) by another neural network. 4.2 Example. Our objective in this example is to investigate the effect of the bound wN on the retrieval performance of the neural network model 1.1 and also to ascertain how typical are the results of example 4.1. To perform this, we used 20 randomly generated sets of binary patterns (with mutual Hamming distances greater than one but otherwise random) to be stored as asymptotically stable equilibrium points of the network. Each set contained m = 6 binary patterns of length n = 10. For each set, we synthesized the network (with n = 10 and a = 1) by solving a semideŽnite programming based on the formulation given in the preceding section for several speciŽc values for the bound w. N Thus, all the designed networks are guaranteed to be globally stable. 9 Table 4 shows the results with the obtained optimal value for s (smax ). The Žrst column speciŽes the values chosen for the bound N Columns 2–5 summarize the average results of the simulation tests. w. The simulation results show that the number of binary inputs (applied as initial conditions for the state of the network) converging to the correct stored pattern decreases as the value wN increases. In particular, this was more pronounced for inputs at Hamming distance HD = 1, 2, 3 from a stored pattern. The number of binary inputs attracted from larger Hamming distances, 9 Note that the maximization of s in equation 3.3 subject to large values for bound w N may result in connection matrices that have a large spectral radius. In such a case, without the matrix inequality constraint in equation 3.5, one cannot guarantee the global stability of the synthesized network.

Synthesis of Brain-State-in-a-Box Neural Models

467

HD ¸ 4, was in general unaffected (as it is expected) from the increase of w. N As shown in Table 4, for wN less than one, the average convergence rate for inputs at HD = 2, 3 decreases steadily despite the fact that smax / 2wN stays unchanged. Also, the convergence rate drops faster for wN ¸ 1, which may be considered due to the combined effect of the increase of wN and the decrease of smax / 2w, N 10 as predicted by our theoretical results. In experimentations with other values for n and m (not shown here) we consistently found that the retrieval performance of the network had the same trend when increasing the bound w. N This validates our conjecture that a small dispersion of wij values contributes to better error-correction capabilities of the network in the presence of random binary errors. In particular, we observe that the results of example 1 are quite typical. 4.3 Example. The purpose of this example is to examine how the retrieval performance of neural network model 1.1 is affected by the number of patterns to be stored. For this, 150 sets of binary patterns (n = 13) were chosen randomly, where each set consisted of k patterns (desired memory patterns), with k varying between 1 and 15. For each set of k patterns (for each value of k, there were 10 random sets of k patterns) we synthesized the network by solving an LP problem with speciŽcations a = 1 and wN = 0.5. All the designed networks satisŽed the global constraint 3.5. Then, for each designed network we examined the percentage of input patterns (given as an initial state of the network), at Hamming distance HD = 1, . . . , 5 and HD ¸ 6 from a stored pattern, which converge to the stored pattern. Figure 1 summarizes the average results of the simulations tests. The shape of Figure 1 is similar to the waterfall graphs in coding theory and signal processing that are used to display the degradation of system performance as the input noise increases. From this point of view, Figure 1 shows the capability of network 1.1 to handle small signal-to-noise ratios as a function of the number of stored patterns. Compared to other existing results, Figure 1 shows that, regarding the storage ability and retrieval efŽciency, the performance of the network is quite satisfactory. 5 Conclusions

We have considered the realization of associative memory by means of a class of synchronous discrete-time neural networks that can be viewed as a generalization of Anderson’s BSB neural model. We presented Žrst an analysis for the location and qualitative properties of the equilibria of the considered network. In particular, we provided results concerning the extent of the domain of attraction of asymptotically stable equilibria, which, under 10

The decrease of smax / 2w N is caused by the fact that the value of smax is bounded by the global stability constraint 3.5.

468

Fation Sevrani and Kennichi Abe

Figure 1: Average percentage of all inputs that converge from a given Hamming distance HD to a stored pattern.

appropriate assumptions, were shown to exist only at the vertices of the hypercube. Using the analysis results as constraints, the synthesis problem was formulated as a constraint optimization problem that was shown to be very exible and offering numerous degrees of freedom. Taking advantage of this formulation, the proposed approach of the synthesis problem allows a design in which the interconnection structure is predetermined with or without symmetry constraints, including a class of discrete cellular neural networks as a special case. In the case of the symmetric interconnection structure, it was shown that by properly choosing an upper bound on the absolute values of the elements of connection matrix, one can guarantee the global stability of the network by using LP techniques for solving the synthesis problem, also highlighting the possibility of the synthesis to be carried out by another neural network. In the experimentations, computer simulations veriŽed that all the desired patterns were stored as asymptotically stable equilibrium points of the designed network. Furthermore, trends observed in numerous simulations showed that the number of spurious asymptotically stable equilibrium points for the considered network is in general small. They also proved the existence of a substantial domain of attraction around the stored patterns, thus validating our theoretical results as well as heuristic arguments provided in the previous sections. Our results, compared to others reported in the literature, show that the retrieval efŽciency of the designed network is quite satisfactory.

Synthesis of Brain-State-in-a-Box Neural Models

469

Appendix Proof of Proposition 1.

integers and let

Let J ½ f1, . . . , ng denote a nonempty subset of

F = fx 2 Rn | |xi | < 1 if i 2 J, xi = § 1 if i 2/ Jg, dim(F) = | J| the cardinality of J. Let xQ 2 R| J| , xQ i = xi , i 2 J. Then xQ evolves according to the linear system 0 xQ i (k C 1) = xQ i ( k) C a

X j2J

wij xQj ( k) C a @

1 X j2J /

wij uj C bi A

8i 2 J, (A.1)

where uj , j 2/ J, is Žxed at § 1. From equation A.1, any equilibrium point of system 1.1, say, x* , must satisfy the condition Q xQ * = ¡bQ , W

(A.2)

where xQ * 2 R| J| is a vector whose components are the unsaturated compoQ 2 R| J|£| J| is a submatrix of W obtained by selecting the nents of x * , W rows and columns indicated by the indices which are in the set J, and P bQi = j2J / wij uj C bi , 8i 2 J. Let us consider Žrst the case | J| = 1, F is an edge of Hn (a face of dimension one). We have the following result. Let wii = 0 8i. Then for almost all b 2 Rn (in the sense of Lebesgue measure), there is no equilibrium point of system 1.1 on an edge of the hypercube Hn . Lemma 2.

Since wii = 0 8i, by equation A.1 the condition for P the existence of an equilibrium point on an edge of the hypercube reduces to j2J / wij uj Cbi = 0. P For Žxed W , the last equality can be satisŽed if bi = ¡ j2J w ij uj . Then, since / fbi g, i = 1, . . . , n, has one-dimensional measure zero, for all b 2 Rn except those belonging to a set of measure zero, there is no equilibrium point on an edge of Hn . Proof.

Now consider 1 < | J| · n. From the hyperbolicity assumption (assumpQ 2 R| J|£| J| is of full rank. Hence, any equation of tion A1) any submatrix W the form A.2 will have exactly one solution. Now, since the trace of W is zero (wii = 0 8i), each principal submatrix of W , including W itself, has at least one eigenvalue with positive real part (the others with negative real Q ), W Q 2 R| J|£| J| , denote the state matrix of equation A.1. By part). Let ( I| J| CaW

470

Fation Sevrani and Kennichi Abe

Q ) has at least one eigenvalue with modulus greater assumption, (fI| J| C aW than one. Hence the possible equilibrium xQ * is unstable. Consequently the only possible stable equilibria must be located at the vertices of the hypercube. Using an elementwise notation, if xi (k C 1) 6 = xi ( k), ( ) c i k can be obtained as Proof of Lemma 1.

c i ( k) =

( g (x( k) C aL(x ( k))) i ¡ xi ( k) , a(L( x ( k)))i

where, for convenience, we use the notation L( x ) = Wx C b . Now consider the following cases: Case 1. | (x ( k) C aL( x ( k)))i | · 1. Then c i ( k) =

(x ( k) C aL( x ( k)))i ¡ xi (k) = 1. a(L(x ( k))) i

Case 2. ( x (k) C aL(x ( k))) i > 1. Note that since |xi ( k)| · 1, ( L(x ( k))) i > 0. Then (x ( k) C aL( x ( k)))i ¡ xi (k) c i ( k) < = 1, a(L(x ( k))) i also c i ( k) =

1 ¡ xi ( k ) ¸ 0. a(L( x ( k)))) i

The case when ( x( k) C aL(x ( k))) i < ¡1 is similar. Proof of Theorem 1. Under the given hypotheses, that is, if the connection matrix W is symmetric and either positive deŽnite or a < 2/ |l min |, it follows

from the energy minimization theorem (see Golden, 1986, 1993; Hui et al., 1993) that the energy function E: H n ! R, deŽned by E (x ) = ¡ 12 x T Wx ¡ x T b ,

(A.3)

evaluated on the trajectories of equation is monotonically decreasing unless it encounters an equilibrium point, that is, denoting D E( x (k)) = E( x (k C1)) ¡ E( x( k)), we have that D E( x ( k)) < 0 if x ( k C1) 6 = x (k) and D E (x ( k)) = 0 if and only if x ( kC1) = x( k). Based on these considerations and by making use of assumption A2, we now show that every trajectory of system 1.1 converges to some equilibrium point as k ! 1. For this, given a trajectory fx ( k)g starting from x 0 2 Hn , k = 0, 1, 2, . . . , let V (x ) denote the positive limit set deŽned as V( x ) = fy : there is a sequence km ! C1 such that x ( km ) ! y g. By deŽnition all the trajectories of system 1.1 are bounded. Hence, by the Bolzano-

Synthesis of Brain-State-in-a-Box Neural Models

471

Weierstrass theorem V ( x) 6 = ;. Let S = fx 2 Hn : D E( x ) = 0g and let M denote the largest invariant subset of S. By deŽnition (cf. equation A.3), E is C1 and bounded on H n . Since D E( x (k)) · 0 8x ( k) 2 H n , by the invariant set theorem (see, e.g., Luenberger, 1979), every trajectory of system 1.1 approaches M as k ! 1, that is, V( x ) ½ M. Now, by assumption, the set M is the entire set of the equilibrium points of 1.1, because D E (x ( k)) = 0 iff x (k C1) = x( k). Furthermore, by assumption, the set of system equilibrium points is Žnite, that is, each equilibrium point is isolated. Since M is Žnite and V( x ) ½ M, we have that V( x) is Žnite. Thus, in order to prove that every trajectory of the system converges to an equilibrium point, it remains to be shown that V( x ) contains only a single point. By contraposition, suppose that x * , y * 2 V( x ). Then, given 2 / 2 > 0, there is a k1 such that | x ( k) ¡ x* | < 2 / 2 for k > k1 , and a k2 such that | x (k) ¡ y * | < 2 / 2 for k > k2 . Choosing kQ = max(k1 , k2 ) we have Q which contradicts that | x * ¡ y * | · | x (k) ¡ x * | C| x (k) ¡ y * | < 2 for all k > k, the fact that the equilibrium points are isolated. This contradiction shows that every trajectory of system 1.1 converges to a limit set containing a single equilibrium point of the equation. References Anderson, J. (1993). The BSB model: A simple nonlinear autoassociative neural network. In M. H. Hassoun (Ed.), Associative neural memories: Theory and implementation (pp. 77–103). Oxford: Oxford University Press. Anderson, J., Silverstein, J., Ritz, S., & Jones, R. (1977). Distinctive features, categorical perception, and probability learning: Some applications of a neural model. Psych. Rev., 84, 413–451. Golden, R. (1986). The brain-state-in-a-box neural model is a gradient descent algorithm. Journal Math. Psychol., 30(1), 73–80. Golden, R. (1993). Stability and optimization analysis of the generalized brainstate-in-a-box neural network model. J. Math. Psychol., 37, 282–298. Guez, A., Protopopsecu, V., & Barhen, J. (1988). On the stability storage capacity and design of nonlinear continuous neural networks. IEEE Trans. Systems, Man, and Cybernetic, 18, 80–87. P S. (1992). Dynamical analysis of the bain-state-in-a-box (BSB) Hui, S., & Zak, neural models. IEEE Trans. Neural Networks, 3, 86–94. P S. H. (1993). Dynamics and stability of the brainHui, S., Lillo, W. E., & Zak, state-in-a-box (BSB) neural models. In M. H. Hassoun (Ed.), Associative neural memories: Theory and implementation (pp. 212–224). Oxford: Oxford University Press. Kohonen, T. (1987). Self-organization and associative memory. Berlin: SpringerVerlag. Liu, D., & Michel, A. (1994).Dynamical systems with saturation nonlinearities: Analysis and design. New York: Springer-Verlag. Luenberger, D. (1979). Introduction to dynamic system theory; Theory, models and applications. New York: Wiley.

472

Fation Sevrani and Kennichi Abe

Lustig, I., Martsen, R., & Shanno, D. (1994). Interior point methods for linear programming: Computational state of the art. ORSA J. Computing, 6, 1–14. Michel, A., & Farrell, J. (1990). Associative memories via artiŽcial neural networks. IEEE Control Systems Magazine, 10(3), 6–17. Michel, A., Farrell, J., & Porod, W. (1989).Qualitative analysis of neural networks. IEEE Trans. Circuits and Systems, 36, 229–243. Michel, A., Si, J., & Yen, G. (1991). Analysis and synthesis of a class of discrete time neural networks described on hypercubes. IEEE Trans. Neural Networks, 2, 32–46. Nesterov, Y., & Nemirovsky, A. (1994). Interior-point polynomial methods in convex programming. Philadelphia: SIAM. Perfetti, R. (1995). A synthesis procedure for brain-state-in-a-box neural networks. IEEE Trans. Neural Networks, 6, 1071–1080. Vandenberghe, L., & Boyd, S. (1996). SemideŽnite programming. SIAM Review, 38(1), 49–95. Xia, Y. (1996). A new neural network for solving linear programming problems and its application. IEEE Trans. Neural Networks, 7(2), 525–529. Xia, Y., & Wang, J. (1995).Neural network for solving linear programming problems with bounded variables. IEEE Trans. Neural Networks, 6(2), 515–519. Received June 10, 1998; accepted January 18, 1999.

Neural Computation, Volume 12, Number 3

CONTENTS Article

Dynamics of Encoding in Neuron Populations: Some General Mathematical Features Bruce W. Knight 473 Notes

Local and Global Gating of Synaptic Plasticity Manuel A. S´anchez-Monta˜n´es, Paul F. M. J. Verschure, and Peter K¨onig

519

Nonlinear Autoassociation Is Not Equivalent to PCA Nathalie Japkowicz, Stephen Jos´e Hanson, and Mark A. Gluck

531

Letters

No Free Lunch for Noise Prediction Malik Magdon-Ismail

547

Self-Organization of Symmetry Networks: Transformation Invariance from the Spontaneous Symmetry-Breaking Mechanism Chris J. S. Webber

565

Geometric Analysis of Population Rhythms in Synaptically Coupled Neuronal Networks J. Rubin and D. Terman

597

An Accurate Measure of the Instantaneous Discharge Probability, with Application to Unitary Joint-Event Analysis Quentin Pauluis and Stuart N. Baker

647

Impact of Correlated Inputs on the Output of the Integrate-and-Fire Model Jianfeng Feng and David Brown

671

Weak, Stochastic Temporal Correlation of Large-Scale Synaptic Input Is a Major Determinant of Neuronal Bandwidth David M. Halliday

693

Second-Order Learning Algorithm with Squared Penalty Term Kazumi Saito and Ryohei Nakano

709

ARTICLE

Communicated by Carl van Vreeswijk

Dynamics of Encoding in Neuron Populations: Some General Mathematical Features Bruce W. Knight Laboratory of Biophysics, Rockefeller University, and Laboratory of Applied Mathematics, Mount Sinai Medical School, New York University,New York, NY 10021, U.S.A. The use of a population dynamics approach promises efŽcient simulation of large assemblages of neurons. Depending on the issues addressed and the degree of realism incorporated in the simulated neurons, a wide range of different population dynamics formulations can be appropriate. Here we present a common mathematical structure that these various formulations share and that implies dynamical behaviors that they have in common. This underlying structure serves as a guide toward efŽcient means of simulation. As an example, we derive the general population Žring-rate frequency-response and show how it may be used effectively to address a broad range of interacting-population response and stability problems. A few speciŽc cases will be worked out. A summary of this work appears at the end, before the appendix. 1 Population Dynamics: Preliminaries

A small patch of cortex may contain thousands of similar neurons, each communicating with hundreds or thousands of other neurons in that same patch or in other patches. By the very nature of this situation, an attempt to simulate it directly must be massive. A recently advanced and promising alternative is the simulation of the dynamical rules stating how the statistical status of the whole population evolves with time. Three recent expositions of this approach are Knight, Manin, and Sirovich (1996), Nykamp and Tranchina (1999), and Omurtag, Knight, and Sirovich (1999). Precursors with goals of a more analytic nature are Stein (1965), Johannesma (1969), Wilbur and Rinzel (1982), Kuramoto (1991), Abbott and van Vreeswijk (1993), Gerstner (1995). Here we briey give one example of the approach and then discuss the generality of its central features. Models of single neuron dynamics are typiŽed by the equations of Hodgkin and Huxley (1952). A serviceable caricature of these equations, and also of more elaborate dynamical models (see the appendix), is the generalized integrate-and-Žre equation: dx / dt D f ( x, s) Cg(x, s), s D s( t), x D 1 resets to x D 0. Neural Computation 12, 473–518 (2000)

(1.1)

c 2000 Massachusetts Institute of Technology °

474

Bruce W. Knight

Here x may be regarded as a transmembrane voltage, which depends on the accumulation of a transmembrane current f ( x, s) common to all neurons in a population, and which current, in turn, depends on the membrane voltage x and also on a synaptic input signal s(t); each neuron also feels a “noise current” (Tuckwell, 1988) g(x, s) which is statistically uncorrelated with the noise currents felt by other neurons. We may view the resetting of x D 1 to x D 0 as the occurrence of a nerve impulse. We will use equation 1.1 to illustrate particular cases of the general features. The central structural features of the population approach emerge from the consideration of a single population of noninteracting neurons. The interesting further features that emerge, for interacting subpopulations, may be addressed with little further overhead in formulation. Our early discussion consequently concerns a population of noninteracting neurons. (Interactions among subpopulations and feedback interaction among the members of one population are discussed in sections 4 and 5). The neuron population is distributed over the voltage variable x, between 0 and 1, with a distribution function r (x, t), which integrates to unity and whose evolution in time must be determined by equation 1.1. The fraction of neurons whose voltage is more than a given value x changes with time in a way that depends on the probability current, J( x, t) D f ( x, s)r ( x, t) ¡ D( x, s) D

¡

f (x, s) ¡ D( x, s)

¢

@ r ( x, t) @x

@ r (x, t), @x

(1.2)

which follows from equation 1.1. The Žrst term—the advection term—states that the probability density is swept along by the common velocity. The second term—the diffusion term—states that neurons are more likely to be shaken (by uctuations) from a voltage region, where more of them reside to an adjacent region, where they are less dense, than their converse likelihood of being shaken up the density gradient. Quantitatively, the diffusion coefŽcient D( x, s) must be derived from a more detailed speciŽcation of the noise g(x, s). Equation 1.2 is taken from Knight et al. (1996), where a brief derivation of D is given. Comparable expressions for current (specialized in different ways) are given by Abbott and van Vreeswijk (1993; equation 9.4), by Nykamp and Tranchina (1999; equations A4 and A16), and by Omurtag et al. (1999; equations 36 and 48). We note in equation 1.2 that the current is obtained from the probability density by a linear transformation on r ( x, t). From the deŽning property of the current, we easily show that its change with x gives the conservation equation (or continuity equation) for the local rate at which density accumulates: @r / @t D ¡@J / @x.

(1.3)

Dynamics of Encoding

475

(In the present context, equation 1.3 was used by Knight et al. 1996, equation 4; Abbott & van Vreeswijk, 1993, equation 3.3; Nykamp & Tranchina, 1999, equation 10.) The linear transformation from r to J, equation 1.2, may be substituted into the conservation equation, 1.3, to obtain the dynamical equation,

¡

¢

@r @ @ D ¡ f (x, s) C D( x, s) r @t @x @x

(1.4)

(Knight et al. 1996, equation 8; Abbott & van Vreeswijk, 1993, equation 9.4, Omurtag et al. 1999, equation 49), which describes how r ( x, t) evolves with the passage of time. (Such a partial differential equation for probability density is technically a Fokker-Planck equation; an extensive reference is Risken, 1996.) Equation 1.4 may be integrated over time to obtain the statistical description of the population’s evolution. This also yields J( x, t) by equation 1.2, and the current across threshold yields the per-neuron Žring rate: r( t) D J(1, t)

(1.5)

(Abbott & van Vreeswijk, 1993, equation 9.9; Nykamp & Tranchina, 1999, equation 11; Omurtag et al., equation 39). We note from equation 1.5 that r( t) is linearly related to J (x, t), and so also is related linearly to r ( x, t). Particular neurons, or particular questions concerning their dynamical behavior, or other special circumstances may require a model more detailed, or differently expressed, than what followed from equation 1.1. For example, the Hodgkin-Huxley equations describe a neuron’s momentary state in terms of four variables (v, m, h, n), including two ( m, h) that determine the conductance of sodium channels and one ( n) that governs potassium conductance. The population density in this case lives in a four-dimensional state-space, and the corresponding population current has four components that describe the ow of population density in the several coordinate directions in that space. Relay cells in the lateral geniculate nucleus (McCormick & Huguenard, 1992) reveal the presence of 10 conductance channel types, which specify a state-space with 18 dimensions. Some neurons have cell bodies that are not tightly coupled electrically with their extended dendrites, and such cells must be described by a state-space that has separate sets of dimensions for each of their multiple electrical compartments. Synapses that deliver Žnite increments of synaptic electric charge require a more detailed formulation (Nykamp & Tranchina, (1999); Omurtag et al., 1999). Synapses with a slow response dynamics add synapse dynamics dimensions to the state-space. On the other hand (as briey discussed in section 6.1), within a state-space the neuronal dynamics places a huge constraint on which probability-density conŽgurations can actually be reached; in a speciŽc case (Sirovich, Knight, & Ormutag, 1999c), this allows one to

476

Bruce W. Knight

replace the population density of equation 1.4 (where the density is a member of the inŽnite-dimensional space of functions that depend on x between 0 and 1) by a 17-entry vector, and allows the reduction of the differential operator of equation 1.4 to a 17 £17 (s-dependent) matrix. The general structural features reviewed below carry over directly to such an efŽcient matrix representation of the population dynamics. 2 Population Dynamics: Common Structure

In all of the examples cited, we Žnd the following common elements of structure: s(t): r:

an input signal. a probability density.

(2.1) (2.2)

In many of the above cases, r is a function deŽned over a state-space; the collection of such functions has the structure of a linear vector space (a weighted sum of two members yields another member). In this sense, in all of the above cases r may be regarded as a member of a linear vector space. J:

a probability current.

(2.3)

Since in many cases, such as the Hodgkin-Huxley case above, J has several components, convenience dictates that we regard J as in a space different from that which contains r . However, there are natural connections: C(s):

a current linear operator such that

(2.4) (2.5)

Cr D J.

In the case discussed above, equation 1.2 is a particular example of equation 2.5, and the linear operator that appears in equation 1.2 is the corresponding particular example of C(s): K:

a conservation operator such that

@r D ¡KJ. @t

(2.6) (2.7)

In the examples above where r is represented in a state-space, K is the familiar divergence operator of ordinary vector analysis, which in our onedimensional example reduces equation 2.7 to equation 1.3. Q( s) D ¡KC(s):

a dynamical operator.

(2.8)

Dynamics of Encoding

477

Substitution of equation 2.5 into 2.7 gives @r D Q( s( t))r : @t

a dynamical equation for

r.

(2.9)

Integration of the dynamical equation yields r at all times, and thus gives a statistical description of the evolving population. In our Žrst example, equation 2.9 becomes 1.4. r( t): R[J, s]:

the Žring rate (per neuron) of the population. a Žring-rate linear functional of J,

(2.10) (2.11)

a linear transformation from J to the scalar Žring rate r(t) such that r D R[J],

(2.12)

which implies the linear relation, r D R [(Cr ) ] ,

(2.13)

that yields the momentary Žring rate from the momentary population density. In our illustrative case above, equation 2.12 specializes to 1.5. Other cases can be more elaborate. For example, the Žring rate of a population of Hodgkin-Huxley neurons is the rate at which individual members achieve the peak values of their action potentials. At the moment of maximum voltage, the right-hand side of the Hodgkin-Huxley voltage equation equals zero, and that constraint deŽnes a 3-surface in the four-dimensional statespace. Integration of the ux of probability current across that surface yields the (per-neuron) Žring rate, and that integral is linear in J, as equation 2.11 states. Because the value of an action potential’s voltage maximum is inuenced by the input current s, the location of the 3-surface is s-dependent, which leads to the general s-dependence of R[J, s] in equation 2.11. For an additional view of the mathematical structure just presented, we may imagine ourselves endowed with a computer of mythical capability and ask how we might approach the dynamical equation, 2.9. For examples where r is naturally represented as a probability density supported by a state-space, we may subdivide that state-space into hypercubes, each sufŽciently small to ensure whatever numerical accuracy we demand. If we do this, all the numbered items above may be cast in terms of Žnite matrices, and several structural features become more clearly visible. For example, we may observe that the dynamical equation, 2.9, describes what technically is a Markov process (Bailey, 1964), and this ensures the existence of some items we use below.

478

Bruce W. Knight

Our informal discretization is introduced at this point not as an approximation but as an aid to uniform presentation. It leads us, by means of Žnite linear algebra, to some valuable general structural features that may also be obtained directly for each of the several different cases quoted above, but by means that must be customized anew for each case. In particular we note that for any speciŽed input s, the dynamical operator Q of equation 2.9 has a set of eigenvectors w n , which it maps to eigenvalue multiples l n of themselves: Qw n D l n w n .

(2.14)

All of these depend on the value of the input s: Q D Q( s), w n D w n ( s), l n D l n ( s).

(2.15)

For time-independent s, equation 2.9 has a time-independent steady-state solution r 0 ( s) for which Qr 0 D 0I

(2.16)

and time independence need not be invoked to observe that this is a distinguished eigenvector of equation 2.14 with eigenvalue l 0 D 0. This eigenvalue, of course, is independent of s in spite of equation 2.15. However, the zeroth eigenvector r 0 ( s) does depend on the input value s. The existence of a zero eigenvalue implies that all columns of the matrix Q have a common linear dependence expression satisŽed by their elements. To determine this dependence we may exploit the conservation of probability. (Only in equations 2.17, 2.18, and 2.19 will subscripts refer to the compartments of our informal discretization.) Discretized, equation 2.9 is a column of equations, the jth of which reads: X dr j D Qjkr k . dt k

(2.17)

The r j ’s are probabilities that sum to unity, and so their sum has no time dependence. Thus, if equation 2.17 is summed on j, the left side is zero: 0 1 X X @ 0D Qjk A r k . k

(2.18)

j

Equation 2.18 is satisŽed at every moment, including the moment that arbitrary initial data are imposed. Initially, we may choose any one of the

Dynamics of Encoding

479

entries r k to be unity, and choose the rest to be zero, so that equation 2.18 implies X (2.19) Qjk D 0: j

every column sums to zero. The discretized case has a natural inner product; if u and v are two column vectors, then (v, u)

(2.20)

is the sum of products of the entries of v and u if those entries are real numbers; if the entries are complex numbers, we instead take the complex conjugates of the v entries before performing the bilinear sum. (This prescription endows each of our particular cases with an inner product, a bilinear integral over state-space, if we take the small hypercubes, of our discretizing construction, to their inŽnitesimal limit.) For any Q, the inner product induces Q¤ :

the adjoint operator to Q,

which, for any u, v satisŽes ¡ ¤ ¢ Q v, u D ( v, Qu).

(2.21)

(2.22)

The entries of the matrix Q are real numbers, and its matrix transpose Q¤ satisŽes equation 2.22. The determinant of a matrix has the same value as the determinant of its transpose, so the eigenvalues of Q, which satisfy det(l I ¡ Q) D 0,

(2.23)

are the same as those of its adjoint Q¤ . We may regard equation 2.23 as a polynomial in l and must anticipate that it can have complex roots. However, l is the only possibly complex item in equation 2.23, and so complex conjugation of that equation simply replaces l by l ¤ . Thus we see l ¤ is also a root, and complex eigenvalues occur in conjugate pairs. The adjoint operator Q¤ (s) has eigenvectors b w m ( s), which satisfy w n D l ¤n b w n. Q¤ b

(2.24)

Here (for convenience, which soon will be evident), we have chosen a labeling that associates the label n with the eigenvector whose eigenvalue is the complex conjugate of l n . We now observe that ¡ ¢ ¡ ¢ ¡ ¢ ¡ ¢ ln b w m , w n D wbm , l n w n D b w m , Qw n D Q¤ b w m, w n ¡ ¢ ¡ ¢ ¡ ¢ D l ¤m b wm , w n D b w m , l m wn D l m b w m, w n . (2.25)

480

Bruce W. Knight

Subtracting the last form from the Žrst gives ¡ ¢ bm , w n D 0. (l n ¡ l m ) w

(2.26)

Since one factor or the other must vanish, we see that sets of eigenvectors, and sets of adjoint eigenvectors, which are associated with distinct eigenvalues, form biorthogonal sets. Linear algebra tells us that generically (that is, except for exceptional cases that are “unlikely” in a strict sense) there are as many distinct eigenvectors (and adjoint eigenvectors) as there are dimensions in the vector space. We may impose a normalization on these sets and get ¡

¢ b w m , w n D d mn . d mn D 1 if m D n, D 0 otherwise.

(2.27)

Moreover, the dimension count says that the eigenvectors are complete in the sense that any vector v may be represented as a sum of eigenvectors: vD

X

an w n ,

(2.28)

n

where each am may be evaluated by observing that ¡

(

¢

b wm, v D b w m,

X

! an w n

n

D

X n

¡ ¢ bm , w n D am , an w

(2.29)

where the last step followed from equation 2.27. An immediate Žrst application is the following: suppose in the dynamical equation 2.9 that s has a Žxed value and that for r (t), we have initial data r (0); then since equation 2.28 can represent any vector, we know we can represent r ( t) as r ( t) D

X

an ( t)w n ,

(2.30)

n

where ¡ ¢ an (0) D wbn , r (0) .

(2.31)

The full solution to the dynamical equation, 2.9, is given by an ( t) D an (0) el n t ,

(2.32)

whence r ( t) D

X n

an (0) el n t w n ,

(2.33)

Dynamics of Encoding

481

as substitution of equation 2.33 into 2.9 shows that the time differentiation, on the left of equation 2.9, produces on each term the same eigenvalue as does the action of Q on the corresponding eigenvector on the right. In general, if noise entered the formulation of the operator Q in equation equation 2.9 (and if s is constant), then an initial departure from equilibrium will die out, and with the exception of l 0 D 0, all the other eigenvalues will have real parts that are negative. (This is a general property of Markov matrices; Bailey, 1964.) At long times only the n D 0 term in equation 2.33 will survive, giving (with equation 2.31) ¡ ¢ b0 , r (0) r 0 . r (t ! 1) D w

(2.34)

Clearly the zeroth adjoint eigenfunction is a valuable item to understand. We have seen in the discretized format that each matrix column of Q( s) must sum to zero. As Q¤ (s) is its transpose, every row of Q¤ (s) will sum to zero. By the rules of matrix action on a column vector, for a vector whose entries are all the same, the action of Q¤ will yield the zero vector because every row-sum vanishes. Thus, the column vector whose entries are all the same constant is the eigenvector b w 0 ( s) of Q¤ ( s), which has eigenvalue zero. In fact it is not a function of s, but a vector rb0 that is universal for all s (just as is its eigenvalue zero). We have asserted both that the equilibrium probability function r 0 ( s) accumulates to a total probability of unity and that it is the eigenfunction of Q( s) with eigenvalue zero. We have also chosen the biorthonormal normalization, equation 2.27. In the particular case of m D n D 0, equation 2.27 becomes (b r 0 , r 0 ( s)) D 1,

(2.35)

which implies that we choose the value 1 for the constant entry of rb0 . The constant-entry property of rb0 shows a special feature of the eigenfunctions of Q( s). If, in the biorthogonality equation, 2.27, we choose m D 0 and n 6 D 0, we obtain (b r 0 , w n ( s)) D 0.

(2.36)

Because the entries of rb0 are constant, this says that the entries of w n sum to zero. Thus, in the eigenfunction expansion, equation 2.30, the vectors w n for n 6 D 0 do not contribute to the probability inventory, and we verify that the decay of those modes in equation 2.33 does not make probability vanish from the system. If the input s is constant, by equation 2.13 the steady-state Žring rate is given by r( s) D R [C(s)r 0 (s), s] ,

(2.37)

482

Bruce W. Knight

which speciŽes the steady-state input-output relation for the neuron population. We note from equation 2.37 that already in the steady state, r responds to s in a manner that in general is nonlinear. Evidently equation 2.37 also predicts the population response to a time-dependent s( t), which changes slowly enough. Although the estimate is informal, it is fairly clear that “slowly enough” means that the characteristic change rate of s( t) should be small compared to the slowest decay rate determined by the eigenvalues in the exponents of equation 2.33, in mathematical terms,

­ ¡ ¢­ ­ (1 / s(t)) ds( t) / dt ­< < | Re (l 1 ( s( t)))| ,

(2.38)

where the index 1 has been assigned to the nonzero eigenvalue whose absolute real part is smallest. The response to a step input also may be stated in exact closed form, because the population response is determined by the linear functional R of equation 2.13. Suppose that the population density has achieved an equilibrium distribution r 0 ( sA ) at input sA and that the input signal is jumped to a new value, sB . We set our time origin at the time of the jump. Equations 2.13, 2.31, and 2.33 give the time course of the response: r(t) D

X

¡ ¢ w n (sB ) , r 0 ( sA ) R [C ( sB ) w n (sB ) , sB ] el n ( sB ) t b

n

D R [C ( sB ) r 0 (sB ) , sB ] X ¡ ¢ C el n (sB )t wbn ( sB ) , r 0 ( sA ) R [C ( sB ) w n ( sB ) , sB ] ,

(2.39)

n6 D0

where our special information on the zeroth eigenfunctions (equation 2.35) has been used in the second line, to isolate the new steady state (compare the Žrst term with equation 2.37). The remaining terms contribute a smooth transient to what would have been an abrupt jump if equation 2.37 had been recklessly used and 2.38 ignored. The jump response (equation 2.39) has been conŽrmed in the case of equation 1.4 by comparison to both a direct integration of that equation and a direct simulation involving 90,000 integrate-and-Žre neurons (Knight, Ormutag, & Sirovich, 1999). The effect of our informal discretization above is to make the sum in equation 2.39 large and Žnite rather than inŽnite. This distinction is not of practical importance; Knight et al. (1999) show that the sum converges with a few terms. An understanding of the eigenvalues clearly is valuable in the interpretation of results such as equation 2.33. While standard algorithms are conveniently available that essentially solve equation 2.23, there are several approximate procedures, and some informal arguments as well, that give valuable systematic information concerning the eigenvalues. A case that we may expect to occur frequently is that of a neuron model that incorporates

Dynamics of Encoding

483

stochastic effects but quantitatively resembles a counterpart nonstochastic model, with the further feature that the steady input s0 is Žxed at a value where the nonstochastic counterpart Žres repetitively on a stable limit cycle. A simple example is equation equation 1.1, with s0 so chosen that f ( x, s0 ) is always positive and does not go to zero for any choice of x between 0 and 1; in the nonstochastic counterpart, the noise g is set to zero. In the more detailed example of the Hodgkin-Huxley equations, a steady input current in the most interesting range gives a limit cycle that is a one-dimensional closed curve in 4-space, around which a system point circulates at Žxed frequency. Even in most neuron models with many electrical compartments and hence a state-space with many dimensions, periodic Žring under steady input implies a system point that pursues a closed one-dimensional limit cycle curve embedded in that many-dimensional space. For the nonstochastic counterpart, we may construct a time-dependent probability density r ( x , t) that is degenerate in the sense that it is nonzero only on the limit cycle curve. But along that curve, at some initial moment a variable density of individual systems may be freely chosen. After one period, any cluster of neurons in the population will return to the same point from which they started, so that the probability density r (x , t) is periodic in time and may be represented as a Fourier series in time: r ( x , t) D

X

einv0 tr n ( x ) ,

(2.40)

n

where v0 ( s0 ) is 2p times the fundamental Žring frequency. We observe that equation 2.40 is entirely consistent with equation 2.33, with a set of eigenvalues, l n ( s) D inv0 ( s),

(2.41)

that are pure imaginary and come in complex-conjugate pairs except for the value zero. Here the integer index ranges over ¡1 < n < 1. In equation 2.40, the r n ( x ) may be regarded (aside from normalization) as the corresponding eigenfunctions. The stable limit cycle is an “attractor ” in the sense that a system point near it is drawn to the limit cycle with the passage of time. The introduction of stochastic effects will superimpose a random walk on the trajectory of a system point, but the excursions of that random walk will be limited by the greater attraction exerted on trajectories further from the limit cycle. In consequence, stochastic effects will produce an equilibrium probability density that will have appreciable size in a region of state-space that we might describe as a “fattened limit cycle.” We may expect that other eigenfunctions whose eigenvalues lie nearest zero (counterparts of the r n ( x ) in 2.40, with integer n near zero) will be conŽned to this region as well. A probability density on that fattened limit cycle feels both advection and dif-

484

Bruce W. Knight

fusion around the cycle. More detailed exploration suggests that with some generality, eigenvalues whose indices lie near zero approximate the form l n ( s) D ¡n2 C(s) C inv0 ( s),

(2.42)

where C(s) is positive. This form was found by Abbott and van Vreeswijk (1993, equation 9.15) for the case of equation 1.4 with f and D constant; by Knight et al. (1996, equation 36) for D small and constant and with f ( x, s) D s ¡ c x

(2.43)

(case of “forgetful” or “leaky” integrate-and-Žre encoder); by Sirovich, Knight, and Omurtag (1999a) for a case with Žnite synaptic jumps (both analytic and numerical conŽrmation); and further analytic evidence concerning the generality of this approximate eigenvalue rule, equation 2.42, is given in the appendix. (See also equation 3.24 and the comments following it.) The presence of diffusion tends to make the fundamental frequency v0 more rapid than it is in the nonstochastic counterpart case (Knight et al., 1996, equation 36). Finally, there are neurons that, under steady input, cyclically Žre bursts of nerve impulses. For such a neuron, the stable attractor is a two-torus embedded in the higher-dimensional state-space. The system motion exhibits two fundamental frequencies (“around and along the doughnut,” which, respectively, characterize the mean impulse rate and the burst rate), and instead of equation 2.40, we will have a sum that features the various combination frequencies of the two fundamentals. A similar but more elaborate eigenvalue theory will follow from these considerations. 3 The Frequency Response

Much valuable structural information may be learned by the study of how the neuron population responds to an input that consists of a large, steady part plus a small, superimposed part that is sinusoidal in time. A valuable slight generalization is a superimposed part that was small throughout its past history and consists of a sinusoid multiplied by an exponentially growing envelope function. The general strategy comes in two steps. The Žrst step is to observe that a small time-variable addition to steady input will cause a small timedependent addition to the steady value of any system variable that ultimately depends on the input. We will regard the relatively easy (though nonlinear) problem, of determining the steady values of the system variables, as already solved. Large, steady values we will give the index zero; small, time-varying ones will be given the index unity. In algebraic manipulations any product of terms with index unity will be dropped as small to

Dynamics of Encoding

485

second order. Following our numbered expressions of section 2, the input signal, s D s0 C s1 ( t),

(3.1)

gives rise to the probability density, r D r 0 Cr 1 (t),

(3.2)

and the probability current, J D J0 C J1 ( t).

(3.3)

The current linear operator, through Žrst order, is C0 C C1 D C (s0 C s1 ( t)) D C (s0 ) C s1 (t) Cs ,

(3.4)

where the subscript s implies differentiation with respect to s of the sdependent linear operator C. In the illustrative case of the operator C in equation 1.2 (also see the particular example, equation 2.43), both f and D are to be differentiated with respect to s. We note in equation 3.4 that the zeroth-order terms on the left and right sides are equal and may be subtracted, leaving only the evaluation of the Žrst-order term (C1 D s1 Cs ). Below, the same sort of steps will be taken repeatedly. The mapping from density to current, equation 2.5 now gives a Žrst-order part, C0r 1 C s1 Csr 0 D J1 ,

(3.5)

so that the perturbed time-dependent current arises from two pieces, the second and less obvious of which depends directly on small changes in the input. The conservation operator K is not s-dependent; as it is linear, we have K ( J0 C J1 ) D KJ0 C KJ1 ,

(3.6)

and by equation 2.7, probability conservation at Žrst order is @r 1 D ¡KJ1 . @t

(3.7)

By equation 2.8, the Žrst-order departure (from equilibrium) of the dynamical operator Q is Q1 D ¡KC1 D ¡s1 KCs D s1 Qs ,

(3.8)

486

Bruce W. Knight

where the last form establishes a more convenient notation. From equation 2.9 the Žrst-order dynamical equation is @r 1 D Q0r 1 C s1 Qsr 0 . @t

(3.9)

The Žnal term, which depends directly on the input, involves only known quantities and makes the equation linear inhomogeneous in the Žrst-order probability density and also linear in the Žrst-order input. In principle these observations provide the means to solve equation 3.9 analytically for r 1 (for example by the construction of a Green’s function solution). By equation 2.11 the Žring rate satisŽes r0 C r1 (t) D R [J0 , s0 ] C R [J1 , s0 ] C s1 Rs [J0 , s0 ] .

(3.10)

With the two terms from the J1 equation, 3.5, this gives the Žrst-order response, r1 D R [C0r 1 , s0 ] C s1 (R [Csr 0 , s0 ] C Rs [C0r 0 , s0 ]) b[r 0 ] , D R [C0r 1 ] C s1 R

(3.11)

where the second line simpliŽes notation a bit and reminds us that the departure in Žring rate can show a direct dependence on input, as well as a response through the departure in population density r 1 . The second step in constructing a frequency response is to assume that the small time-dependent input is sinusoidal: s1 ( t) D s1 (0) eivt .

(3.12)

Instead we at once make the slightly more general specialized assumption that s1 is a sinusoid multiplied by an exponentially growing envelope: s1 ( t) D s1 (0) est where s D a C ivI

(3.13)

here a determines the growth rate of the growing envelope exp(at). Because all of the Žrst-order relations above are both linear and time invariant, the other Žrst-order dynamical quantities generically will respond sympathetically with a similar time dependence: J1 (t) D J1 (0) est , r 1 (t) D r 1 (0) est , r1 ( t) D r1 (0) est .

(3.14)

Here the coefŽcients at time zero will depend on s and are to be determined by the Žrst-order equations. Since s is generally complex, they will be complex (and, in fact, analytic functions of s). Their complex nature will

Dynamics of Encoding

487

indicate how their phases in the sinusoidal cycle differ from the phase of the input. Since equation 3.13 is a specialization of the general time-dependent input perturbation above, all the linear Žrst-order equations remain intact. However the dynamical equation, 3.9, reduces to a simpler form: (sI ¡ Q0 ) r 1 D s1 Qsr 0 ,

(3.15)

and this equation has the formal solution r 1 D s1 (s I ¡ Q0 ) ¡1 Qsr 0 .

(3.16)

Now the right-hand expression Qsr 0 is to be regarded as a vector, which can be represented as a sum of eigenvectors of the operator Q0 , as in equation 2.28. We further observe that (s I ¡ Q0 ) ¡1 w n D

1 wn . s ¡ ln

(3.17)

(Multiply both sides by sI ¡ Q0 and use the eigenvector equation 2.14; equation 3.17 becomes an identity.) This enables us explicitly to solve equation 3.16. With equations 2.28 and 2.29, we have X¡ ¢ b w n , Qsr 0 w n , (3.18) Qsr 0 D n

which brings equation 3.16 to the form r 1 D s1

X n

¢ 1 ¡b w n , Qsr 0 w n . s ¡ ln

(3.19)

If we let s D iv here, this expression tells us the frequency response of the population density to a small sinusoidal oscillation in input. If our eigenvalues are of the form expected commonly (see equation 2.42), we see that as we move the driving frequency v, every time it passes a value nv0 , there will be a resonant response, and the eigenfunction w n will become prominent in the population density function. In equation 3.19 there is no resonance for s D 0 even though l 0 D 0 : the n D 0 term in the numerator vanishes, as we now show. By equation 2.16, the equilibrium distribution satisŽes 0D

@ @r 0 ( Qr 0 ) D Qsr 0 C Q . @s @s

(3.20)

Thus the n D 0 numerator in equation 3.19 is

¡

(b r 0 , Qsr 0 ) D rb0 , ¡Q

@r 0 @s

¢ ¡

D Q¤rb0 , ¡

@r 0 @s

¢

D 0,

(3.21)

488

Bruce W. Knight

the last because rb0 is the zeroth eigenfunction of Q¤ . Consequently we can reduce the sum in equation 3.19 to one over n 6 D 0. The population Žring-rate frequency response, or transfer function, is the (typically complex) ratio of output departure to input departure: T 0 (s) D r1 / s1 .

(3.22)

Because r1 and s1 are proportional to the same time course, equations 3.13 and 3.14, their ratio is time independent; because they are linearly related, their ratio is amplitude independent: the transfer function depends on input only through the value of s. If we substitute equation 3.19 for r 1 into 3.11 for r1 , we see that r1 is indeed proportional to s1 , and the transfer function, equation 3.22, becomes b[r 0 ] C T 0 (s) D R

¡ ¢ bn , Qsr 0 X w R [C0 w n ] . s ¡ln n6 D0

(3.23)

We will loosely call this the population rate frequency response, which more properly is T0 ( iv), for the more special case of purely oscillatory input. We see again, in the output response, equation 3.23, the appearance of resonance when imaginary s is brought close to a complex eigenvalue l n . The b of equation 3.23 is independent of frequency, and represents leading term R accurate tracking even at high frequencies. (However, in a detailed model b term goes to zero, as whose state-space includes synaptic dynamics, the R explicated by Gerstner, 1999, and its effect is taken over by large-eigenvalue terms in the sum.) An example of the frequency response, equation 3.23, was evaluated by a quite different route some years ago (Knight, 1972a). In the case of equation 2.43 (“forgetful integrate-and-Žre” model) let the “capacitor discharge rate” c be small and also let stochastic effects be small. Then encoders, which have simultaneously reset to zero, will disperse only slightly before reaching threshold, and stochastic input current may be replaced by an effective stochastic threshold, which slightly disperses Žring times with a small standard deviation t . The frequency response in this model (Knight, 1972a, equation 8.22, adapted to present notation) is T 0 (s) D

¡

v0 2p s0

¢

C

X n6 D0

c (v0 / 2p s0 ) » ¼ v03 t 2 2 s ¡ ¡n C inv0 4p

¡ ¢

.

(3.24)

Comparison with equation 3.23 identiŽes the various terms and relates those of equation 3.23 to “back of the envelope” model parameters. Amplitude and phase dependence on s D iv were in agreement with a population frequency response measured (and v0 and t also measured) in an experiment

Dynamics of Encoding

489

(Knight, 1972b). The form of the bracketed expression in the denominator of equation 3.24 is in agreement with the expected form of the eigenvalues in equation 2.42. (For this model, as c and t go to zero, the radian frequency is v0 D 2p s0 .) For subsequent convenience, we notationally simplify equation 3.23 to bC T0 (s) D R

X n6 D0

bn . s ¡ ln

(3.25)

For a multidimensional neuron model, the expedient evaluation of the various constants in equation 3.25 (deŽned in more detail in equation 3.23) remains a challenge for the future. However, experience to date (Omurtag et al., 1999; Knight et al., 1999; Sirovich et al. 1999) indicates that for onedimensional neuron models, such evaluations are quite straightforward. In the Žrst section of the appendix, a procedure is outlined to reduce multidimensional neuron models approximately to fairly realistic one-dimensional generalized integrate-and-Žre counterparts. Also in section 6.2 a method (so far not implemented) is outlined for the efŽcient direct numerical evaluation of the coefŽcients in equations 6.14, which express the population density dynamics in a “moving basis” representation. If these equations are specialized to the sinusoidal perturbation input of equations 3.1 and 3.12, equation 3.25 emerges together with a prescription for numerical evaluation of its various constants. 4 Applications of the Frequency Response: Stability of Interacting Subpopulations and of Feedback

The deliberations above are major tools for the investigation of how interacting subpopulations of neurons behave in consort. In time-independent equilibrium the kth subpopulation will satisfy a generalized form of the population-response relation, equation 2.37: £ ¤ (4.1) rk (s 0 ) D Rk Ck ( s 0 ) r k0 ( s0 ) , s0 . Here s0 stands for a list of inputs that may be derived from external sources or from the outputs of other subpopulations. Equation 4.1 has been stated in a way that leaves room for differences not only in the detailed dynamics of each subpopulation but also in the way different input channels deliver synaptic signals to any particular subpopulation. Clearly the dynamical equation, 2.9, may be generalized in a manner similar to equation 4.1: @r k D Qk ( s (t)) r k . @t

(4.2)

Again the subscript k refers to the kth subpopulation. The procedures of section 3 may be applied to this more general case, to Žnd the frequency

490

Bruce W. Knight

response of the kth population to the mth, input, as in equation 3.22: rk1 D Tkm (s) sm1 .

(4.3)

(Looking back to equations 3.16 and 3.17, we note that the resonant eigenvalues in this expression sensibly depend on the kth dynamical equation but not on the mth input source.) We still have to specify how the response of one population is to affect the input to another. We will work through some informative simple cases, but initially let us leave this relation very general and write £ ¡ ¢ ¤ sm ( t) D s( ext) m ( t) C um r t0 , t .

(4.4)

The Žrst right-hand term is to be regarded as speciŽed external input from other neuron subpopulations, which can be as numerous as the populations under our direct investigation. The second term is a general nonlinear functional, where the prime on t0 indicates that this functional can look back across past times t0 at earlier outputs of all of the different subpopulations. Thus um is general enough to include, for example, distributed propagation time lags from other subpopulations. In the special state of time-independent equilibrium, equation 4.4 reads sm0 D s( ext) m0 C um [r0 ] ,

(4.5)

where the s( ext) m0 are given. These equations simply state how, through back coupling, the time-independent total inputs must depend in a speciŽed way on the time-independent outputs. They are to be solved together with equation 4.1 for the two sets of constants s0 , r 0 . Simple particular cases suggest particular simple strategies, but we note that one standard method for solving equations 4.5 and 4.1 is to replace um by bum and repeatedly solve while advancing b from b D 0, by small steps to b D 1. There remains the question of the stability of the feedback-coupled solution. The technical means for evaluating a transfer function, discussed in the previous section, are broadly applicable. In particular if we are given a speciŽed um [r( t0 ), t], and if we assume that the nth input component has the time dependence rn ( t) D rn0 C rn1 (t) D rn0 C rn1 (0) est ,

(4.6)

with rn1 small, and we further assume that the remaining components of r( t) are at their equilibrium values, the general steps of the previous section yield £ ¡ ¢ ¤ um r t0 , t D um [r0 ] C Umn (s) rn1 ,

(4.7)

Dynamics of Encoding

491

and the transfer function Umn (s) is straightforward to evaluate in particular cases. If instead we assume that all the rn ( t) are of the form of equation 4.6, then equation 4.7 generalizes to X £ ¡ ¢ ¤ u m r t0 , t D u m [r 0 ] C Umn (s) rn1 ,

(4.8)

n

which is a component of the matrix equation £ ¡ ¢

¤

u r t0 , t D u [r0 ] C U (s) r1 .

(4.9)

Again in matrix notation, if we assume that the external driving in equation 4.4 is of the form s( ext) ( t) D s( ext)0 C s ( ext)1 (0) est ,

(4.10)

then the Žrst-order part of the feedback equation, 4.4, becomes s1 D s ( ext)1 C U (s) r 1 .

(4.11)

The subpopulation input-output relation, equation 4.3, in matrix form becomes r1 D T 0 (s) s1 .

(4.12)

If we now act on the previous equation with the population-response transfer function matrix of equation 4.12, we may express the Žrst-order Žring rates of all the subpopulations in terms of those of the external inputs: r1 D T 0 (s) s( ext)1 C T 0 (s) U (s) r1 .

(4.13)

This may be solved for the departures from equilibrium of the population rates by a matrix inversion: r1 D ( I ¡ T 0 (s) U (s)) ¡1 T 0 (s) s ( ext)1 .

(4.14)

When this expression is written out in components, it tells us the transfer function that expresses the modulation of Žring rate, near equilibrium, of any subpopulation, in response to modulation of any one of the external inputs, and includes the effects of cross-talk communicated between different subpopulations as well as the effect of any self-feedback. It is time to observe the connection between the analytic properties of the transfer function and the question of stability. In general, a modulated output r1 is related to a modulated input b s1 by a (generally complex) ratio

492

Bruce W. Knight

T (s) (as in equation equation 3.22), which allows us to Žnd what input is required to achieve a speciŽed output: b s1 D ( T (s)) ¡1 r1 .

(4.15)

If we move s on the complex plane, the required value of b s1 changes. In particular if we move s so that (T (s)) ¡1 D 0,

(4.16)

we have an output in the absence of input, which is to say that a root of equation 4.16 gives us a free-running behavior of our system, which has a time course proportional to exp(st). It may be technically simpler to observe that the free-running exponents of the system occur where the transfer function itself becomes inŽnite. A case in point is the population Žring-rate response transfer function of equation equation 3.25, which goes to inŽnity at the eigenvalues s D l n . We note from equation 2.39 that the eigenvalues l n are the free-running exponents of the system. We see from equation 4.14 that the free-running exponents for the coupled subpopulations are given by the values of s, where the inverse matrix goes to inŽnity, and these are the roots of the equation, det ( I ¡ T 0 (s) U (s)) D 0.

(4.17)

This expression is a polynomial in the known transfer functions of the various component pieces of the overall coupled system. If any root of equation 4.17 has a real part that is positive, then the equilibrium determined by equations 4.1 and 4.5 is unstable. In typical cases, the features of the speciŽc problem may be exploited to Žnd the informative roots of equation 4.17 by means more efŽcient than a direct determinental expansion. The principal virtue of equation 4.17 is to show that in general, there is a well-deŽned route to a well-deŽned goal that answers the question about stability. 5 Stability: SpeciŽc Cases 5.1 One Population, Self-Feedback with Fixed Lag. In this case we choose the feedback functional in equation 4.4 to be the linear relation

£ ¡ ¢ ¤ ¡ ¢ u r t0 , t D gr t ¡ b t ,

(5.1)

where g is the gain from output to feedback input, and b t is the Žxed lag time, so that the feedback equation, 4.4, becomes s(t) D s( ext) (t) C gr( t ¡ b t).

(5.2)

Dynamics of Encoding

493

For time-independent equilibrium, equation 4.5 becomes s0 D s(ext)0 C gr0 .

(5.3)

This is to be solved together with the steady-state population-response equation, 2.37: r0 D R [C(s0 )r 0 ( s0 ), s0 ] ´ R0 (s0 ),

(5.4)

which is a nonlinear relation between r0 and s0 . Thus the determination of s0 in terms of s( ext)0 cannot be done in a simple and exact closed form. But to understand the basic facts about the static behavior of our population response, we should already have on hand the evaluation of the function R0 ( s0 ) over a range of s0 values, and so we can solve equations 5.3 and 5.4 together to express the s0 versus s( ext)0 relation as s( ext)0 D s0 ¡ gR0 ( s0 ).

(5.5)

Values of s( ext)0 that are outside the range of positive values given by equation 5.5 will not allow solution of the steady-state equations 5.3 and 5.4. We make the cautionary observation that in equation 5.5, any steady-state total input s0 may be excluded from the solvable region by making the positive feedback g strong enough. Clearly our discussion applies to the remaining broad range of cases where the time-independent solution exists. If we let r( t) D r0 C r1 ( t) D r0 C r1 (0) est ,

(5.6)

our specialization equation, 5.1, becomes ¡ ¢ £ ¡ ¢ ¤ s t¡b t t ( ), D gr0 C ge ¡sb u r0 C r1 t0 , t D gr0 C gr 1 (0) e r1 t

(5.7)

so in this case the transfer function of equation 4.7 is t . U (s) D ge ¡sb

(5.8)

The Žrst-order part of equation 5.2 is s1 D s(ext)1 C U (s) r1 ,

(5.9)

while from equation 3.22, T0 (s) s1 D r1 .

(5.10)

494

Bruce W. Knight

We multiply equation 5.9 by T0 (s) and Žnd r1 D

1 T0 (s) s( ext)1 1 ¡ T0 (s) U (s)

(5.11)

for the population rate frequency response to external input. (Note that this is the 1 £ 1 matrix case of the matrix equation, 4.14.) By equation 4.16, the free-running response exponents of the system will be those values of s for which the denominator vanishes, so that T 0 (s) U (s) D 1,

(5.12)

or in our particular case (equation 5.8), t e¡sb T 0 (s) D

1 , g

(5.13)

a transcendental equation to be solved for s, where T0 (s) is the explicit expression (equation 3.25). A specialized case will show some qualitative features. In the simple case of self-feedback without delay (b t D 0), equation 5.13 becomes T 0 (s) D

1 . g

(5.14)

If the feedback gain g is small, then a root s must make T0 (s) large; this will happen if s lies near a pole l n of T0 (s). As a Žrst approximation we may truncate the sum in equation 3.25 to the term that is large because s lies near its pole: T 0 (s) » D

bn , s ¡ln

(5.15)

and equation 5.14 may now be solved to give s D l n C gbn .

(5.16)

If bn is real and positive (as in the explicit example of T0 (s) given by equation 3.24) then, for positive feedback g, equation 5.16 shows that the real part of s is less negative than that of l n , and also that increasing g moves the real part of s in the positive direction and eventually past zero and beyond the threshold of instability. More generally, to Žnd the border of instability, we may set the real part of s to zero and take the real part of equation 5.16 to Žnd the critical feedback value, gcrit D |Rel n | / Re bn ,

(5.17)

Dynamics of Encoding

495

to within the approximation of equation 5.15. The system will be unstable if any root of equation 5.14 has a positive real part, so in equation 5.17, we must choose the index n that makes gcrit smallest. Equation 2.42 suggests that commonly this will occur for n D 1. Returning to equation 5.13, for Žnite lag b t the “small g-single pole” approximation, equation 5.15, still makes good sense; instead of equation 5.16, it now gives us the implicit relation t s D l n C e¡sb bn g.

(5.18)

Again let us observe that a critical value of g yields a purely imaginary s at the edge of instability. At that value take the real part of equation 5.18, and instead of equation 5.17 we Žnd ¡¢ gcrit b t D

1 | Rel n | ¡ ¢ , cos |s| b t Re bn

(5.19)

and (because the cosine cannot exceed unity) the presence of the lag tends to stabilize positive feedback. Equation 5.18 may be used in this way if gcrit (b t) is still small, which should hold if | Rel n | is small, which should be the case if the inuence of stochastic effects is modest. 5.2 One Population, Distributed Feedback Lags. Suppose that instead of a single Žxed lag b t as in equation 5.1, we have a distribution of lags weighted according to a distribution function P( t0 ), which integrates to unity. In place of equation 5.1, our feedback functional now is

£ ¡ ¢¤ u r t0 D g

Z1 0

¡ ¢ ¡ ¢ dt0 P t0 r t ¡ t0 ,

(5.20)

which is still linear in r, and where g again is the feedback gain. Substitution of a constant r0 in equation 5.20 retrieves exactly the same results as in the previous exercise, since P( t0 ) integrates to unity. If we assume a superimposed small modulation, r1 (t) D r1 (0) est ,

(5.21)

then equation 5.20 gives £ ¡ ¢ ¤ u r1 t0 , t D g

Z1

¡ ¢ 0 dt0 P t0 r1 (0) es (t¡t )

0

D gr1 (t)

Z1 0

¡ ¢ 0 dt0 e¡st P t0 D U (s) r1 ( t),

(5.22)

496

Bruce W. Knight

and the explicit feedback transfer function, U (s) D g

Z1 0

¡ ¢ 0 b(s) , dt0 e¡st P t0 D gP

(5.23)

may be substituted into equation 5.12, whose roots give us the free-running frequencies. An example of such a situation is given by Abbott and van Vreeswijk (1993). If the feedback coupling g is small enough, we may again use the “small g-single pole” approximation, equation 5.15. Since in this case s lies near l n , to lowest order in g we may replace U (s) by U(l n ), and equation 5.12 may be solved for the free-running exponents, b(l n ) . sn D l n C bn U (l n ) D l n C gbn P

(5.24)

Here the Žrst right-hand form makes no assumption concerning the nature of the feedback transfer function U(s), except that its effect should be small enough to permit the single-pole approximation. In the second right-hand form, we may exploit the fact that P (t0 ) is positive in equation 5.23. If, further, P( t0 ) is concentrated around small values of t0 , then for small integer |n|, equation 5.23 tells us that U(l n ) will have a real part that is positive. Equation 3.24 suggests (as do some detailed analytic arguments, as well as numerical evidence; Sirovich et al., 1999a) that the real part of bn (for small |n|) should be positive as well. Consequently the second right-hand form of equation 5.24 indicates that, again in this case, more positive feedback leads to real parts of the sn that are less negative, and so leads to more persistent free-running modes of motion. Again, the borderline of instability may be estimated. If we assume that s1 is pure imaginary and take the real part of equation 5.24, we obtain gcrit D

| Rel 1 | ¡ ¢, b(l 1 ) Re b1 P

(5.25)

which should have quantitative accuracy if gcrit is small enough to fulŽll the single-pole approximation. 5.3 A Multipopulation Generalization for Small Feedback. If feedbacks are small among interacting subpopulations of neurons, the freerunning exponent equation, 4.17, has an instructive and valuable approximate analytic solution that can be obtained by the single-pole approximation. The determinant of a matrix vanishes if that matrix has a zero eigenvalue, and so equation 4.17 will be solved by any value s that solves the matrix equation,

( I ¡ T 0 (s) U (s)) w D 0 .

(5.26)

Dynamics of Encoding

497

A typical matrix element of T 0 (s) is Tkm (s) D

X n6 D0

(km) bn , (k) s ¡ln

(5.27)

where the row k indexes the kth subpopulation and the column m the mth input, while n indexes the nth free-running mode of one subpopulation in isolation. (The fact that a subpopulation’s free-running eigenvalues depend on which subpopulation, but not on which input, is reected in equation 5.27.) There is a pole for every eigenvalue of every subpopulation. If the matrix elements of U are small, then T 0 must have large elements to compensate, ( ) and so, in equation 5.27, s must lie near some l nk , let us say for the qth subpopulation, and all other poles may be ignored. The matrix elements (see equation 5.27) become Tkm (s) D

( qm) bn vm D ( q) ( q) s ¡ ln s ¡ln

if k D q, D 0 otherwise,

(5.28)

and the second right-hand form deŽnes the elements of the speciŽed vector: ( qm)

v : vm D bn

.

(5.29)

Let us specify another vector: b q: b qk D 1

if k D q, D 0 otherwise.

(5.30)

Then (with the superscript T for transpose) equation 5.28 may be expressed as the outer-product matrix, T 0 (s) D

1 ( q) s ¡ ln

³ ´ b q ­ vT .

(5.31)

( q)

If s lies close to l n , then, in equation 5.26, to lowest order U (s) may be (q ) replaced by U (l n ), and equation 5.26 becomes w¡

1 ( q) s ¡ ln

³ ´³ ³ ´´ ( q) b q ­vT U l n w D0.

(5.32)

Performing the right-hand operations and multiplying by the denominator: ³ ´ ³ ³ ´ ´ ( q) ( q) s ¡ l n w Db q v, U l n w ,

(5.33)

498

Bruce W. Knight

where the right-hand parenthesis is the scalar product of v with Uw . Equation 5.33 shows us that the components of the unknown vector w are proportional to the components of the known vector b q , so the eigenvector of equation 5.26 is w Db q,

(5.34)

and we may equate the coefŽcients of equation 5.33 to obtain the exponent s: ³ ³ ´ ´ ³ ´ ( q) ( q) ( q) X ( qm) ( q) s D l n C v, U l n b q D ln C bn Umq l n ,

(5.35)

m

where equations 5.29 and 5.30 were used, in the last step, to evaluate the scalar product. Equation 5.35 tells us how far the exponent s is moved from ( q) the nearby eigenvalue l n , by the interpopulation coupling Umq . This subsection has shown how the single-population results of the previous subsection generalize naturally to a situation with interacting subpopulations. In a single small patch of cerebral cortex, we Žnd subpopulations of neurons of different dynamical design, and we also Žnd, in separated patches, subpopulations of neurons with the same dynamical design. This subsection was intended, together with the next subsection, as a guide to show how a model that is a caricature of cortex, with subpopulations that interact and have separate inputs, but are of only a single dynamical type, may be generalized to the more realistic situation, which involves several different dynamical types. 5.4 Caricature of an Orientation Hypercolumn. In primary visual cortex, nearby patches of neurons typically respond best to linear visual stimuli that have similar orientations. There is neural interaction between the patches, and this interaction is weaker between pairs of patches whose preferred orientations are more dissimilar. Approximately, patches for all preferred orientations are on an equal footing, with no preferred orientation exciting a larger subpopulation of neurons than others do or differing from others in design characteristics. A set of nearby patches that includes all orientations is referred to as a hypercolumn. If a hypercolumn contained neurons of only one dynamical type, and contained N subpopulations with equally spaced preferred orientations, then the following caricature would be accurate:

A common dynamical equation, @r k D Q (sk ( t)) r k , @t for the kth subpopulation.

(5.36)

Dynamics of Encoding

499

A common Žring-rate rule rk ( t) D R [C (sk (t)) r k , sk ( t) ] .

(5.37)

An even-handed feedback rule, X £ ¡ ¢ ¤ bn u rk¡n t0 , t . sk ( t) D s( ext) k ( t) C

(5.38)

n

Here the summed index is understood to be modulo N, so no subpopulation has preference. Since u has been left general, we may assume that the weightings bn sum to unity: X n

bn D 1.

(5.39)

Let us also reasonably assume that bn is always positive and is even (D b¡n ). For simplicity we assume that u is a linear functional, as in the earlier subsections. If we assume a time-independent baseline external input that also is evenhandedly independent of k, then it is consistent to assume that the resulting total inputs and the subpopulation responses are independent of k as well, and we retrieve, for the N subpopulations the equilibrium equations, 5.3, 5.4, and 5.5, which held for a single population with feedback. Proceeding as before with the small-departure analysis, from our earlier results we easily derive from equations 5.36 and 5.37 that rk,1 D T0 (s) sk,1 ,

(5.40)

while equation 5.38 gives us sk,1 D s( ext)k,1 C U (s)

X

bn rk¡n,1 .

(5.41)

n

To relate population response to external input, we may multiply equation 5.41 by T0 (s) and use equation 5.40: rk,1 D T0 (s) s( ext) k,1 C T 0 (s) U (s)

X n

bn rk¡n,1 .

(5.42)

This is a special case of the matrix equation, 4.13. It is much simpliŽed because the single population dynamics results in a population transfer function matrix, which is a single scalar transfer function that multiplies the identity matrix, while the feedback matrix is a single scalar function of s which multiplies a discrete convolution. The presence of a discrete convolution suggests that a discrete-Fouriertransform strategy should yield major simpliŽcations, and this is indeed

500

Bruce W. Knight

the case. In fact discrete Fourier transformation of our nonlinear starting equations, 5.36–5.38, gives a reduction in numerical work by isolating a collection of “nonlinear dynamical modes,” which can be much less than N in number and also carry a valuable physiological interpretation. The stability question, which we address here, is expedited by a small bit of the discrete-Fourier machinery. We have seen that the free-running exponents of the coupled system may be found as the values of s that produce a zero eigenvalue and so solve the matrix equation, 5.26. Corresponding to equation 5.42, that matrix eigenvalue equation specializes to wk ¡ T0 (s) U (s)

X n

b n wk¡n D 0.

(5.43)

Because of the discrete convolution, we can anticipate that the eigenvector w will be a discrete-Fourier basis vector: wk D eiqk

where q D 2p j / N,

(5.44)

and where j can be any integer in the range that includes zero to N ¡ 1. To verify this, substitute equation 5.44 into 5.43 and at once reduce that equation to ¡ ¢ 1 ¡ T 0 (s) U (s) be q D 0,

(5.45)

where ¡ ¢ X ¡iqn be q D bn e

(5.46)

n

is the discrete Fourier transform of b n . We note for j D 0 that be(0) D 1,

(5.47)

by equation 5.39. Because we chose the bn real, positive, and even in n, we see that be is a real number and ¡ ¢ be q

1.

(5.48)

Thus equation 5.45, which is to be solved for s, is identical in form to equation 5.12, which was for a single-neuron population with the same population dynamics and the same feedback dynamics. For q 6 D 0, the presence e in equation 5.45 is simply to reduce the strength of the effective of b(q) feedback coupling according to equation 5.48. Thus, we anticipate that the

Dynamics of Encoding

501

least stable exponent will go with q D 0, which gives us exactly the singlepopulation case, equation 5.12), discussed earlier. Three further points should be made here. The Žrst is that discrete subpopulations appeared in equations 5.36–5.38 in anticipation of numerical work to come, while a more realistic hypercolumn model would feature a continuous gradation in best-orientation sensitivity. But a free-running exponent equation, 5.45, has emerged as an effective single-population relation whose form is independent of how Žnely we discretize the system. We could have started equally well with a continuously graded system, and the same effective one-dimensional result would have followed. The simpliŽcation of equation 5.45 was a consequence of an underlying symmetry of the system. We could have started with the symmetry of the circle, which is what the problem gave to us, rather than reverting to the less natural symmetry of an N-sided regular polygon, which is what the discrete Fourier transform addresses. The second point is that we could have undertaken a more realistic hypercolumn model involving several different cell types, each with a population dynamics differing in its details. All the steps above would have their counterparts in this more realistic case. At the Žnal step we would have, instead of the single equation 5.45, a matrix eigenvalue equation like 5.26. But the number of entries in the eigenvector would be not the large number of subpopulations but rather the small number of dynamically different cell types. Thus, the stability problem still is very tractable for this case. The third point is in a more speculative vein. Recent evidence (Everson et al. 1998) suggests that primary visual cortex has a single processing strategy and even-handed allocation of resources for the processing of patches of retinal image that differ not only in orientation but also in magniŽcation scale over a fair range of magniŽcations, to reasonable approximation. The combined two-parameter (“twist” and “zoom”) symmetry has a set of two-parameter eigenvectors analogous to the “discrete twist” eigenvector (see equation 5.44). These mathematical constructs might not only assist more realistically detailed modeling of visual cortex, but perhaps also might provide some new insight into the nature of cortical processing. 6 Possibilities for EfŽcient Simulation 6.1 Principal Components. Section 1 concluded with the mention of a means to reduce the dynamical equation, 2.9, and the Žring-rate equation, 2.13, to matrix equations of fairly modest size. This may be achieved, once we have on hand a sample solution of equation 2.9 in response to a span of “typical input” s( t), by principal component analysis (Sirovich & Everson, 1992) (also called “singular value decomposition” or “representation by empirical eigenfunctions”), which now will be briey described. For rea-

502

Bruce W. Knight

sonable input s(t), some dynamical equations of the form 2.9 prove to be almost unable to carry the population density function r ( t) outside a subspace of modest dimension that is embedded in its full function space. If at a time t¤ we have a snapshot r ( t¤ ) of the density function, we may create from it the singular operator ¡ ¢ ¡ ¢T P t¤ D r t¤ ­r t¤ ,

(6.1)

which is a projection operator in the sense that its action on any vector u gives ¡ ¢¡ ¡ ¢ ¢ P t¤ u D r t¤ r t¤ , u ,

(6.2)

which is a vector in the one-dimensional manifold deŽned by the snapshot vector r ( t¤ ). From equation 6.1 we immediately show that P t¤ is self-adjoint. Because ¡ ¢ ¡ ¢ ¡ ¡ ¢ ¡ ¢¢ P t¤r t¤ D r t¤ r t¤ , r t¤ ,

(6.3)

it has only nonnegative eigenvalues and is positive-semideŽnite. If we have a collection of snapshots, taken at times t¤ , we may form the operator AD

X t¤

Pt¤ ,

(6.4)

which, according to equation 6.2, maps any vector into the modestdimensional manifold accessible to the dynamical equation. The self-adjoint property, and the positive-semideŽnite property, are inherited by A, and this ensures a set of orthonormal eigenvectors in the eigenvalue problem Awn D l n wn ,

(6.5)

whose eigenvectors we can number in the order of their decreasing nonnegative eigenvalues. Because the eigenvectors are orthonormal, we have ( wm , wn ) D d mn .

(6.6)

Because the effectively nonnull subspace of A is of modest dimension, to good approximation we may span it with a modest subset of the eigenvectors to equation 6.5. Now we can expand a solution r ( t) as r ( t) D

X n

an ( t) wn ,

(6.7)

Dynamics of Encoding

503

where the sum extends over a modest number of terms (17 terms in the quoted example). This we now can substitute into the dynamical equation, 2.9. We take the inner product of that equation with wm and (employing equation 6.6 on the left-hand side) directly conclude that X dam (wm , Q (s(t)) wn ) an , D dt n

(6.8)

which may be integrated in time for the am (t). With these am (t) on hand, substitution of equation 6.7 into the Žring-rate equation, 2.13, explicitly evaluates the Žring rate as rD

X

R [Cwn ] an .

(6.9)

n

We note that the an may be regarded as the components of a vector, and this endows equations 6.8 and 6.9 with an inherited structure identical to that of equations 2.9 and 2.13. Equation 6.9 is an explicitly speciŽed linear functional of its input vector. Equation 6.8 expresses the evolution of that vector in terms of the action on it, of an s-dependent operator—here an s-dependent matrix. If the operator in equation 2.9 can be expressed as a polynomial in s (as, for example, by substituting equation 2.43 into equation 1.4), then the matrix in equation 6.8 can be expressed as a corresponding polynomial in which the same set of powers of s multiplies corresponding component constant matrices. These considerations translate directly into a practical and efŽcient computational scheme. However, this procedure requires caution at one point: the operator Q has conservation of probability built into it, but its representation by a truncated matrix in equation 6.8 may slightly miss perfect conservation because of the truncation; the formerly zero eigenvalue may be slightly perturbed, and this will give to the total probability a spurious slow exponential drift as time progresses. Conservation of probability is easily restored to the matrix of equation 6.8. In discretized format, conservation for the original matrix Q was assured by the fact that every column of Q was orthogonal to the zeroth adjoint eigenvector rb0 , which had constant entries. In the present representation the nth component of this vector is given by b an D (wn , rb0 ) .

(6.10)

Without truncation, the admixture of this vector in each column of the matrix in equation 6.8 would be zero. Let us inner-product the truncated equation 6.10, with a column of the truncated matrix, to obtain the (very small) coefŽcient of actual admixture; we then multiply the vector (equation 6.10) by this admixture coefŽcient and subtract the result from the matrix column.

504

Bruce W. Knight

Conservation of probability is now fulŽlled. In cases tested (Sirovich et al., 1999c) the resulting matrix applied in equations 6.8 and 6.9 gives excellent agreement with the much longer direct calculation of equations 2.9 and 2.13. The choice of 17 dimensions rendered graphical results indistinguishable from those of the longer direct calculation. (This was not quite achieved with 15 dimensions. The dimension count jumps by twos because the natural subspaces relate to consecutive pairs of complex conjugate eigenvalues, as the next section will elucidate.) 6.2 Moving Basis. We conclude with a brief comment on a set of possibilities that might lead to extremely efŽcient simulation and give an additional viewpoint on mathematical structure. Once the efŽcient principal components representation has been constructed, it is not a heavy computation to solve the eigenvalue problem and tabulate the results, parametrically in the input s, for the eigenvalues and eigenfunctions that are within the subspace that the dynamical equation is able to access. In cases examined, these have proved to be in good practical agreement with their counterparts derived from the more laborious direct calculation (Sirovich et al., 1999c). Thus, it is reasonable to envision a representation of the dynamics expressed in terms of the input-dependent dynamical eigenvectors of equations 2.14 and 2.15:

r ( t) D

X

e an ( t)w n (s(t)) .

(6.11)

n

This is a representation in terms of a “moving basis” that at each time t remains optimized for the moving present value of the input s(t). (The overhead mark is to distinguish these coefŽcients from those of equation 6.7.) Effective convergence thus should be achieved by a sum of fewer terms in equation 6.11 than were required in equation 6.7, where the Žxed basis was a compromise that had to span the Žrst several eigenvectors of Q( s) over the entire range of s( t). Once the e an (t) in equation 6.11 have been determined, the population Žring rate is given by the short sum rD

X n

e an R [C( s)w n (s) , s] D

X

bn ( s)e R an (t).

(6.12)

n

To determine the e an ( t), we may work from the differential equations that they satisfy. To do this we may inner-product r ( t) in equation 6.11 with the adjoint eigenvector b w m (s), to obtain ¡ ¢ bm (s (t)) , r ( t) . e a m ( t) D w

(6.13)

Dynamics of Encoding

505

Differentiation now yields de am D dt

¡b

w m,

@r @t

¢ ¡b

C w m,s

ds ,r dt

¢

¡ ¢ ds ¡ ¢ b D Q¤ b w m, r C w m,s , r dt ¼ X »¡ ¢ ds ¡ ¢ b e D w m,s , w n an l ¤m wbm , w n C dt n D l m ( s( t))e am C

ds X Mmn (s( t))e an . dt n

(6.14)

In the second line we used the dynamical equation and the adjoint operator relation; in the third we substituted equation 6.11 for r ; in the fourth line we abbreviated by deŽning the matrix ¡ ¢ w m ( s) / @s, w n ( s) M mn (s) D @b

(6.15)

(whose m D 0 entries are all zero, because of the constant-entry property of rb0 , as discussed at equations 2.35 and 2.36). The fourth line of equation 6.14 gives an explicit system of ordinary differential equations to be integrated for the e am ( t). A check on the equations 6.14 is that if s is constant, they may be solved analytically, and their solution is equation 2.32. Because for m 6 D 0 the eigenvectors w m do not carry any probability (see the discussion at equation 2.36) we must have e a0 ( t) ´ 1.

(6.16)

(by equation 6.11) and this is consistent with equation 6.14 even when s(t) is time dependent, since, when m D 0, both l m and Mmn are zero. If s(t) changes very slowly (see equation 2.38), then r ( t) will pass through a progression of steady states, and we will have r ( t) D r 0 (s(t)) .

(6.17)

The remaining members of equation 6.14 furnish a sequence of corrections to this result. We can estimate how many terms we will need in the sums in equations 6.11 and 6.14. If our eigenvalues are of the “damped limit-cycle” form (see equation 2.42), then the equations 6.14 are those for a sequence of “tuned Žlters,” each tuned for a radian frequency given by the imaginary part of the eigenvalue, and each driven by the right-hand-most term in the equation. We see that the pair of equations for m and ¡m are needed to furnish the part of the response spectrum in a frequency range near m times the limit cycle frequency. If we wish to compute an output response that

506

Bruce W. Knight

is accurate through frequency components that are N times as high as the highest single-neuron Žring rate, then the estimated number of terms in the sums in the Žring rate and dynamical equations 6.12 and 6.14 will be 2N C1. Thus, for example, seven terms should yield a computed output with a short-term-estimated spectrum that remains accurate through frequencies about three times the momentary single-neuron Žring rate. Direct application of the moving basis, as just discussed, to a simulation in which the value of the input s( t) passes from the limit-cycle regime of an individual neuron to the regime of the stable equilibrium point, will encounter a technical nuisance. (This situation will arise in the simulation of primary visual cortex.) In the region of passage between the two regimes, each pair of early eigenvalues l§ m ( s), at some critical value of s, will merge by having their imaginary parts move to zero, and further change in s will give rise to two negative real eigenvalues that correspond to a faster and a slower pure relaxation. The members of equation 6.14 for § m together may be written ¡ ¢ de a Cm / dt D (l Cm ¢ e a Cm C 0 ¢e a¡m ) C ds / dt L Cm (e a C, e a¡ , s) ¡ ¢ de a¡m / dt D (0 ¢e a Cm Cl ¡m ¢e a¡m ) C ds / dt L¡m (e a C, e a¡ , s)

(6.18)

to emphasize that here we are dealing with a 2 £ 2 diagonal matrix. The L§ terms are the remaining linear sums in equation 6.14. Clearly we can pursue equation 6.18 into the pure-relaxation regime by assigning Cm and ¡m, respectively, to the less and to the more negative real eigenvalues there. But the transition is not smooth at the merger value, as the motion of each eigenvalue (for example) on the complex plane takes a right-angle turn at that value of s. The two invariants of the matrix, its determinant 4 and its trace T, are related to the eigenvalues by 4m ( s) D l Cm ( s) ¢ l ¡m (s)

(6.19)

T m ( s) D l Cm (s) Cl ¡m ( s)

(6.20)

» ¼ q 1 2 l§ m ( s) D Tm ( s) § (Tm ( s)) ¡ 44m (s) . 2

(6.21)

The critical condition occurs at the value of s where the argument of the root passes through zero and then changes sign. We observe that the rightangle-turn behavior of the eigenvalues is not shared by either of the two invariants, and this indicates that the passage may be made smooth by a

Dynamics of Encoding

507

linear transformation upon equation 6.18 that carries it to the form ¡ ¢ ¡ ¢ ¡ ¢ dxm / dt D 0 ¢ xm C 1 ¢ ym C ds / dt Xm x, y, s

¡ ¢ ¡ ¢ ¡ ¢ dym / dt D ¡4m ¢ xm C Tm ¢ ym C ds / dt Ym x, y, s

(6.22) (6.23)

whose 2 £ 2 matrix has the same determinant and trace as in equation 6.18. Equation 6.22 may be solved for ym and substituted into equation 6.23 to remove explicit dependence on ym from that equation’s right-hand side. An additional differentiation of equation 6.22 then carries the form of the dynamics to that of the fairly familiar damped and forced harmonic oscillator. The critical value of s corresponds to critical damping in the oscillator, which does not introduce singular behavior into the oscillator ’s forced motion. The moving-basis equations in the real-component form, 6.22 and 6.23, may be forward integrated across the critical damping condition without encountering embarrassment. The introduction of the moving basis in terms of sharpening the results of principal component analysis at the start of this subsection was convenient but inessential. The evaluated pieces that go into equations 6.12 bn (s), l m ( s), and Mmn ( s), are all to be evaluated at conand 6.14, which are R stant input s. If state-space is divided up into small compartments, then the only compartments visited by a neuron at constant s will be those that lie within a “fattened limit cycle” or in the case of an s so chosen that the neuron without noise would be quiescent, will lie analogously within the “stochastically fattened neighborhood of a stable equilibrium point.” These small compartment subsets of the state-space offer a numerically tractable eigenvalue problem and direct evaluation of the pieces just mentioned. The standard sinusoidal small perturbation assumption, as stated in equations 3.1 and 3.12, can be substituted into the moving-basis dynamical equations 6.14. Since equation 6.14 contains no prior approximation, the transfer function (see equation 3.25) must follow, and does so with a little thoughtful manipulation. Hence the numerically tractable eigenvalue problem will in principle lead to a similarly tractable numerical evaluation of the various constants that appear in equation 3.25. Equations 6.12 and 6.14 summarize the dynamics of a neuron population in a sort of a compact canonical form, in the sense that in principle, two dynamically distinct neuron designs can lead to numerically identical examples of equations 6.12 and 6.14, but if they do then both alternative designs would play dynamically identical roles as subpopulations in a larger array of interacting neuron subpopulations. If the effect on a speciŽc neuron population by an elsewhere-directed pharmacological agent is to be minimal, or if the effect of a base-pair change in a mouse gene is to be maximal, then this more properly could be measured by minimal or maximal numerical changes in equations 6.12 and 6.14 rather than in the underlying dynamical equations. Regarding efŽcient simulation, we can

508

Bruce W. Knight

envision an elaborate neuron model with multiple electrical compartments, and a highly simpliŽed neuron of the general integrate-and-Žre variety, giving rise to numerically almost identical instances of equations 6.12 and 6.14. 7 Summary

We may anticipate that a fairly broad range of useful neuron-population simulations will exhibit a common mathematical structure. The main ingredient of that structure is a dynamical equation that describes the momentary evolution of a population density. That density exists in a state-space whose points represent the possible states of a single model neuron. The population density dynamical equation follows from the postulated dynamics of the single neuron, including the consequences of an input noise uncorrelated among different neurons of the population. The population dynamics equation is nonlinear in the common inputs from other neuron populations (which nonlinearity reects the nonlinear dynamics of the identical individual neurons in the population) but is linear in the population density. This linearity endows the population dynamics with a structure whose broad outlines are familiar from applied mathematics and which gives insight regarding the ways the neuron population will respond to different input situations. The second major ingredient of a neuron population dynamics model is an expression that speciŽes its momentary per-neuron output rate of impulses. This output rate expression depends linearly on the momentary disposition of the population density. (In the sense that the impulse train from each neuron is a point process in time, the merged impulse trains from a large population of noninteracting neurons will be a time-dependent Poisson point process.) Because the population dynamical equation is linear in the population density, it may be viewed in terms of an input-dependent population dynamical linear operator that acts on the population density. At a speciŽed input, that operator has a set of eigenfunctions in terms of which the population density can be expanded as a linear weighted sum and a corresponding set of eigenvalues that include zero and complex conjugate pairs with negative real parts. Under common circumstances, the eigenvalue pairs nearest zero may be approximated by a simple orderly progression, and we may anticipate that the corresponding eigenfunctions will dominate in the expansion of the population density. We may examine how such a population dynamics model responds to input that is a small sinusoidal oscillation around an operating point. This leads to an input-output frequency response that is a simple analytic function of frequency with “resonance denominators” at the complex eigenvalues that, under common circumstances, lie near integer multiples of the population’s mean Žring rate.

Dynamics of Encoding

509

The output of one population can be used as input to another. If several neuron subpopulations are thus coupled together, commonly a set of equilibrium Žring rates may be found. A frequency response function can be assigned to each input-output pair; from these we may derive an analytic function that is a polynomial combination of those frequency responses, and whose complex roots specify the possible free-running behaviors near equilibrium of the coupled set of subpopulations. A root whose real part is positive indicates a free-running behavior that is a sinusoidal oscillation times an exponentially growing envelope; hence the equilibrium is unstable in this case. In particular cases this stability analysis is quite tractable. The question of the stability of a model of an orientation hypercolumn of neurons in primary visual cortex may be reduced by symmetry to the tractable stability analysis of a much smaller system, which contains only one subpopulation of each cortical cell type. Finally, because only fairly few early eigenfunctions proved important in exploring the analytic methods discussed above, we are led to a preliminary examination of how this feature might enable extremely efŽcient simulations of neuron population dynamics. One such method involves examination of the solution of the full population dynamics equation in response to a sample of typical variable input. The small subspace of function space, in which the population density always resides, may be determined by principal component analysis. The dynamical equation may be partially diagonalized into this subspace, where it becomes a modest system of linear ordinary differential equations coupled by a matrix that changes in time with the changing input. A second approach, which currently is speculative but promising, is to recast the dynamical equation in terms of coefŽcients of a “moving basis” of the input-dependent eigenfunctions of the dynamical operator. This approach has the potentially useful additional feature that it reduces neuron population models to a single canonical form. This should lead, for example, to a quantitative estimate of how well an elaborate neuron model may be approximated by a simpler model. We certainly can envision models of interacting cortical neuron subpopulations whose mathematical features will include some that lie beyond those presented here. The eventual accurately descriptive model may well fall in the broader set. Still, what is presented here may help furnish a bit of guidance for theoretical developments to come. Appendix: Estimate of the Eigenvalues A.1 The Model. The generalized integrate-and-Žre model, of equation 1.1, can be made to mimic a more detailed model fairly well, for driving inputs s(t) that do not change with unnatural abruptness. In the more detailed case of the Hodgkin-Huxley neuron, for example, the voltage equation is of

510

Bruce W. Knight

the form dv / dt D F( v, m, h, n, s),

(A.1)

where m, h, n satisfy differential equations of their own. For a typical Žxed value of s, these variables follow a limit cycle, and on that s-dependent limit cycle, the other variables can be expressed as functions of the concurrent value of v, whence F( v, m, h, n, s) D F( v, m( v, s), h( v, s), n( v, s), s) D b F( v, s).

(A.2)

The model equation, dv / dt D b F( v, s),

(A.3)

when s is held constant, will integrate to yield the exact voltage time course on the limit cycle, for each s-value. More generally, for time-dependent s, the momentary exact m, h, n values will not depart far from their limit-cycle values for present v and s, so long as s does not change by a major fraction of its value during a single cycle of v. The other variables relate to v in different ways on the fast descending side of the voltage cycle and on the much slower ascending side. To avoid double-valuedness, we may deŽne our variable x in equation 1.1 as xD

1 2 ( vmax ¡ vmin ) ( (for downward motion) vmax ¡ v £ (for upward motion). vmax ¡ vmin C ( v ¡ vmin )

(A.4)

If x is taken modulo unity, it is periodic on the limit cycle. Substitution of v in terms of x in equation A.3 now gives the f ( x, s) of equation 1.1. If we proceed in this way, the boundary condition induced on the consequent dynamical equation, 1.4, for the population density r ( x, t), is that r ( x, t) should be smoothly periodic in x, with period 1. The more familiar boundary condition, for integrate-and-Žre dynamics with diffusion, is an absorbing (or “no-return”) upper boundary. We note the same effect is built in here, as on the Žnal fast voltage rise of the nerve impulse, forward advection overwhelms backward diffusion. The form of equation 1.4, as a good approximation to more elaborate and realistic dynamics, may be reached by an entirely different path. Suppose that in a state-space of higher dimension, we have properly formulated the dynamical operator in diffusion approximation. Suppose further that stochastic effects are modest enough so that the “fattened limit cycle” (discussed in section 2, equations 2.40 and 2.41), which supports the population

Dynamics of Encoding

511

density function, is reasonably thin. Then our partial differential equation eigenvalue problem may be separated into a preliminary transverse eigenvalue problem that may be solved locally at each cross-section across the limit cycle, and an axial eigenvalue problem that follows from assembling the solutions to the transverse problem. (This is the same methodology that is natural in addressing wave propagation in a waveguide of variable crosssection.) The resulting axial dynamical operator inherits both advective and diffusive terms, and is of the form given in equation 1.4, although in this case f ( x, s) includes the effects of transverse diffusion. A.2 Small Stochastic Effects. If the diffusion term in equation 1.4 is small, we may approach the eigenvalue problem,

Qw n D l n w n ,

(A.5)

by straightforward perturbation theory. In equation 1.4, let Q D Q0 C Q1 ,

(A.6)

where Q0 D ¡

@ @ @ f (x) , Q1 D D (x) @x @x @x

(A.7)

(we drop the s dependence from the notation, as input is held Žxed in the eigenvalue problem), and we assume that inclusion of the small Q1 induces changes away from both the eigenvalues and eigenvectors of Q0 : (0)

(0) (0)

Q0 w n D l n w n

(A.8)

(0)

(1)

(A.9)

(0)

(1)

(A.10)

wn D w n C wn

l n D l n Cl n .

Substitution of equations A.6, A.9, and A.10 into A.5 (and use of equation A.8) gives the Žrst-order part of equation A.5: (1)

(0)

(0) (1)

(1) (0)

Q0 w n C Q1 w n D l n w n Cl n w n ,

(A.11)

where we regard terms with index zero as known, and those with index one as to be found. The perturbed eigenfunction can be expanded in terms of the unperturbed eigenfunctions: (1)

wn D

X m6 Dn

(0) . am w m

(A.12)

512

Bruce W. Knight (0)

It is notable that the m D n term, which would include an admixture of w n (1) in w n , is absent from the sum. This term is absent because its inclusion in equation A.9 would serve simply to rescale the unperturbed eigenvector, and the eigenvector property is not affected by a scale change. Essentially equation A.11 asks, How much must the old eigenvector be redirected to be an eigenvector of the modiŽed operator? and a simple scale change is irrelevant to answering this question. So every term of equation A.12 is (0) biorthonormal to the adjoint eigenvector b w n . If all we want from equa(1) tion A.11 is the Žrst-order correction to the eigenvalue, l n , we may inner(0) product that equation with b w n , and by use of equations 2.22 and 2.27, we at once Žnd ³ ´ (1) (0) (0) l n D wbn , Q1 w n . (A.13) The inner product for the state-space of equation 1.4 is Z1 ( v, u) D

dxv¤ ( x) u ( x) .

(A.14)

0

We observe that Z1 ( v, Q0 u) D ¡

dxv ¤ (x) 0

Z1 D

dx 0

¡

f (x)

¢ @ ¡ f ( x) u ( x) @x

@ v (x) @x

¢

¡ ¢ D Q¤0 v, u ,

¤

u (x) (A.15)

where for the second line we integrated by parts, and used the periodic boundary condition discussed in section A.1. By equation A.15 the zerothorder adjoint operator is Q¤0 D f ( x)

@ , @x

and the adjoint eigenvalue equation, ³ ´¤ (0) (0) (0) b wn D l n wn , Q¤0 b

(A.16)

(A.17)

is f ( x)

d b(0) ³ (0) ´¤ b(0) wn D l n wn , dx

(A.18)

Dynamics of Encoding

513

which (by direct substitution back into equation A.18) has the solution

8 9 <³ ´¤ Z x = ¡ ¡ ¢¢ (0) (0) b w n ( x) D exp l n . dx0 1 / f x0 : ;

(A.19)

0

Periodicity demands that (0) (0) b w n (1) D b w n (0) ,

(A.20)

and this requires us to choose the eigenvalues (0) ln

8 1
: 0

9¡1 ¢= D inv0 , dx 1 / f (x) ; ¡

(A.21)

which is of a form in agreement with equation 2.41. If we now go back to equation 1.1, in the absence of noise it gives dt D dx / f ( x) ,

(A.22)

which we may immediately integrate from x D 0 to x D 1 to obtain the transit time, or reciprocal Žring frequency: Z1 TD

dx / f ( x) .

(A.23)

0

As Žring period and radian frequency are related by v0 D 2p / T,

(A.24)

this is in exact agreement with the fundamental radian frequency that appears in the eigenvalue evaluation, A.21. In the same manner as what we have just done, from equation A.7, the unperturbed dynamical operator ’s eigenvalue equation is ¡

´ d ³ (0) (0) (0) f (x) w n D l n w n , dx

(A.25)

which, as back-substitution again conŽrms, has the solution (0) wn

8 9 Zx < = ¡ ¡ ¢¢ ( x) D C 1 / f ( x) exp ¡l n dx0 1 / f x0 . : ; ¡

¢

0

(A.26)

514

Bruce W. Knight

If in equation A.26 we choose the constant C D T ¡1 ,

(A.27)

then the eigenfunctions and their adjoints are indeed biorthonormal, as equation 2.27 becomes

8 9 Z1 Zx ³ ´ < 0 = 2p i dx dx (0) (0) b ( n ¡ m) wm , wn D C exp . : T f ( x) f ( x0 ) ; 0

(A.28)

0

In fact equation A.22 serves as a crafty substitution that reduces equation A.28 to an easy integral and conŽrms equation 2.27 for this case. We may now evaluate the integral, equation A.13, for the eigenvalue’s perturbation due to diffusion. We Žrst make the observation that Z1 (v, Q1 u) D

dxv ¤ (x) 0

Z1 D¡

dx

d d D (x) u (x) dx dx

¡d dx

0

v ( x)

¢

¤

D ( x)

d u ( x) , dx

(A.29)

again by parts integration and periodicity. Thus equation A.13 becomes

(1) ln

Z1 D¡

dx 0

¡ db

(0)

dx

w n ( x)

¢

¤

D ( x)

d (0) w n ( x) . dx

(A.30)

bn(0) we can simplify directly from the adjoint The derivative that involves w (0) eigenvalue equation, A.18, to obtain a term with a factor l n . To evaluate the right-most derivative in equation A.30, we may bring equation A.25 to the form ³ ´ (0) (0) l n C f 0 ( x) dw n (0) D¡ wn . dx f ( x)

(A.31)

From equations A.19 and A.26, we have further that 1 1 (0)¤ (0) wbn ( x) w n ( x) D . T f ( x)

(A.32)

Dynamics of Encoding

515

When these ingredients are substituted into equation A.30, we obtain the evaluated form, (1) ln

Z1 ³ ´2 Z1 dx D (x) dx f 0 (x) D ( x) (0) (0) D ln ¡ ¢3 Cl n ¡ ¢3 T f ( x) T f ( x) 0

0

³ ´2 (0) (0) D ln A Cl n B,

(A.33)

where A is clearly positive and the two integrals A and B are independent of the eigenvalue number n. (However, they do depend on the input s, which controls the fundamental radian frequency v0 .) The eigenvalue without (0) diffusion,l n , is given by equation A.21; substitute that into equations A.33 and A.10, which gives the eigenvalue through zeroth and Žrst orders, and we have l n D ¡n2 v02 A C inv0 (1 C B) .

(A.34)

Here (unlike in equation 2.42) v0 is the fundamental radian Žring frequency in the absence of diffusion, and A and B, assumed small, are the two integrals that appear in equation A.33, which are linear in the strength of diffusion. The presence of diffusion modiŽes the fundamental Žring frequency, through the term B in equation A.34; and equation 2.42 is conŽrmed under the often reasonable assumption of small diffusion and in the context of the fairly accurate model presented here. A.3 Rapidly Oscillating Eigenfunctions. The eigenfunction problem

given by equation 1.4

¡

¢

d d ¡ f ( x) C D ( x) w n D l n wn , dx dx

(A.35)

may be expected to yield solutions w n (x) that oscillate more and more rapidly across the range of x, as |n| and consequently |l n | become larger. This offers us another approximation procedure, the “WKB” approximation, that does not hinge on small diffusion. We assume that the solution locally looks like a sinusoidal wave, but with a wavelength and amplitude that change gradually with position: 8 x 9 < Z = ¡ ¢ w n ( x) D A ( x) exp i dx0 k x0 . (A.36) : ; 0

In the very special case where f and D are constant, equation A.36 solves A.35 exactly with A and k constant, and where k must satisfy ¡if k ¡ Dk2 D l n .

(A.37)

516

Bruce W. Knight

If we now insist on solutions with unit period, we must have k D 2p n, whereupon equation A.37 agrees exactly with equation 2.42. For variable f and D, substitution of equations A.36 into A.35 yields the result that, so long as k( x) is very large compared to the Žrst derivatives of f ( x) and D( x), neglect of the relatively much smaller terms again leads to equation A.37 locally for every value of x, or equivalently » ¼ q ¡i 2 (A.38) kD f § f C 4Dl n . 2D If we let the diffusion D go to zero in equation A.38 (with the choice of the minus sign), then substitution of A.38 into A.36 again gives an exact solution, as we retrieve the zeroth-order exact solution of the previous subsubsection, which in fact was in the form of equation A.36. Substitution of k( x) from equation A.38 into A.36 yields a condition that speciŽes the eigenvalue once we insist that w n (x) be periodic, as in equation A.20: Z1 dx 0

¡1 2D ( x)

»

q f ( x) §

¡

f ( x)

¢2

¼ C 4D (x) l n D ¡2p in.

(A.39)

This is a transcendental equation to be solved forl n . If n is large, then clearly |l n | must be chosen large to solve the equation. But if |l n | is large, then we can approximate k in equation A.38 by k» D

¡i 2D

¡

»

f C 2l n / D1 / 2 1 C 1 2

1 f2 8 Dl n

¢¼

(A.40)

(where l 1 / 2 implies both positive and negative signs) and the eigenvalue equation, A.39, becomes Z1 ¡in D

0

dx f 1 2 Cl n/ 2p 2D

D a Cl n/

1 2

¡

bC

Z1 0

c ln

¢

1 dx 1 C 2p D1 / 2 l 1n / 2

Z1 0

dx 1 f 2 2p 8 D3 / 2 (A.41)

where a, b, c are the three speciŽc integrals, which do not depend on l n or n. Carry a to the left side by subtracting it from equation A.41, and square:

¡

¡n2 C 2ina C a2 D l n b 2 C

2bc c2 C 2 ln ln

D b 2l n C 2bc C

c2 . ln

¢

(A.42)

Dynamics of Encoding

517

If |l n | is large, the right-hand term with the coefŽcient 1 /l n may be dropped, and the rest may be solved for l n : ln D ¡

¡

1 2 a a2 2c C ¢ 2 C ¡ n i n 2 2 2 b b b b

¢

.

(A.43)

This result, which should hold for large n, is again in agreement with equation 2.42, except for the right-hand asymptotic off-set term, which does not grow with n. Acknowledgments

I owe a particular debt to Lawrence Sirovich. Others who contributed valuable ideas are Daniel Amit, Dmitrii Manin, and Ahmet Omurtag. This work has been supported by NIH/NEI EY11276, NIH/NIMH MH50166, ONR N00014-96-1-0492. Publications from the Laboratory of Applied Mathematics may be found at http://camelot.mssm.edu/. References Abbott, L. F., & van Vreeswijk, C. (1993). Asynchronous states in networks of pulse-coupled oscillators. Phys. Rev. E., 48, 1483–1490. Bailey, N. T. J. (1964). The elements of stochastic processes with applications to the natural sciences. New York: Wiley. Everson, R. M., Prashanth, A. K., Gabbay, M., Knight, B. W., Sirovich, L., & Kaplan, E. (1998). Representation of spatial frequency and orientation in the visual cortex. Proceedings of the National Academy of Science, 95, 8334–8338. Gerstner, W. (1995). Time structure of the activity in neural network models. Phys. Rev. E., 51, 738–758. Gerstner, W. Population dynamics of spiking neurons: Fast transients, asynchronous states and locking. In press. Neural Computation. Hodgkin, A. L., & Huxley, A. F. (1952). A quantitative description of membrane current and its application to conduction and excitation in nerve. J. Physiol., 117, 500–544. Johannesma, P. I. M. (1969). Stochastic neural activity: A theoretical investigation. Unpublished Ph.D. dissertation, University of Nijmegen. Knight, B. W. (1972a). Dynamics of encoding in a population of neurons. J. Gen. Physiol., 59, 734–766. Knight, B. W. (1972b).The relationship between the Žring rate of a single neuron and the level of activity in a population of neurons: Experimental evidence for resonant enhancement in the population response. J. Gen. Physiol., 59, 767–778. Knight, B. W., Manin, D., & Sirovich, L. (1996). Dynamical models of interacting neuron populations. In Symposium on Robotics and Cybernetics: Computational Engineering in Systems Applications (Gerf, E.C. ed) Cite ScientiŽque, Lille, France.

518

Bruce W. Knight

Knight, B. W., Omurtag, A., & Sirovich, L. (2000). The approach of a neuron population Žring rate to a new equilibrium: An exact theoretical result. In press. Neural Computation. Kuramoto, Y. (1991).Collective synchronization of pulse-coupled oscillators and excitable units. Physica D, 50, 15–30. McCormick, D. A., & Huguenard, J. R. (1992). A model of the electrophysiological properties of thalamocortical relay neurons. J. Neurophysiol.,68, 1384–1400. Nykamp, D., & Tranchina, D. A population density approach that facilitates large-scale modeling of neural networks: Analysis and an application to orientation tuning. In press. J. Comp. Neurosci. Omurtag, A., Knight, B. W., & Sirovich, L. (1998). On the simulation of large populations of neurons. In press. J. Comp. Neurosci. Risken, H. (1996). The Fokker–Planck equation. Methods of solution and applications. Berlin: Springer-Verlag. Sirovich, L., & Everson, R. (1992). Management and analysis of large scientiŽc datasets. International Journal of Supercomputer Applications, 6, 50–68. Sirovich, L., Everson, R., Kaplan, E., Knight, B. W., O’Brien, E., & Orbach, D. (1996). Modeling the functional organization of the visual cortex. Physica D, 96, 355–366. Sirovich, L., Knight, B. W., & Omurtag, A. (2000). Dynamics of neuronal populations: The equilibrium solution. In press. SIAM. Sirovich, L., Knight, B. W., & Omurtag, A. (1999a). Dynamics of neuronal populations: Eigentheory. In preparation. Sirovich, L., Knight, B. W., & Omurtag, A. (1999b). Dynamics of neuronal populations: Stability theory. In preparation. Sirovich, L., Knight, B. W., & Omurtag, A. (1999c). Dynamics of neuronal populations: Low dimensional approximations. In preparation. Stein, R. B. (1965). A theoretical analysis of neuronal variability. Biophys. J., 5, 173–194. Tuckwell, H. C. (1988). Introduction to theoretical neurobiology (Vol. 2). Cambridge: Cambridge University Press. Wilbur, W. J., & Rinzel, J. (1982). An analysis of Stein’s model for stochastic neuronal excitation. Biol. Cybern., 45, 107–114. Received January 11, 1999; accepted April 12, 1999.

NOTE

Communicated by P. Read Montague

Local and Global Gating of Synaptic Plasticity Manuel A. S a´ nchez-Monta n˜ e´ s ¨ ¨ Institute of Neuroinformatics, ETH/University Zurich, 8057 Zurich, Switzerland, and Grupo de Neurocomputaci´on Biol´ogica, ETS de Inform´atica, Universidad Aut´onoma de Madrid, 28049 Madrid, Spain Paul F. M. J. Verschure Peter K onig ¨ Institute of Neuroinformatics, ETH/University Zurich, ¨ 8057 Zurich, ¨ Switzerland Mechanisms inuencing learning in neural networks are usually investigated on either a local or a global scale. The former relates to synaptic processes, the latter to unspeciŽc modulatory systems. Here we study the interaction of a local learning rule that evaluates coincidences of pre- and postsynaptic action potentials and a global modulatory mechanism, such as the action of the basal forebrain onto cortical neurons. The simulations demonstrate that the interaction of these mechanisms leads to a learning rule supporting fast learning rates, stability, and exibility. Furthermore, the simulations generate two experimentally testable predictions on the dependence of backpropagating action potential on basal forebrain activity and the relative timing of the activity of inhibitory and excitatory neurons in the neocortex. 1 Introduction

Research on neuronal networks has been very much motivated by the ability of these systems to learn from experience (Alkon et al., 1991). In these systems information is stored in the pattern of synaptic efŽcacies. Among the processes inuencing the modiŽcation of synaptic connections local and global mechanisms can be distinguished (Montague, Dayan, & Sejnowski, 1996). Examples of global signals involved in learning can be found in gating mechanisms (Abbott, 1990). In the brain this might be compared to the inuence of modulatory signals, arising from subcortical structures, on cortical plasticity. Here, the cholinergic system of the basal forebrain projecting to the cerebral cortex is of particular interest. It is a necessary ingredient for the induction of cortical representations following monocular deprivation (Singer & Rauschecker, 1982). In addition, it may switch between storage and recall modes in the hippocampus (Hasselmo, 1993), and it gates the plasticity of receptive Želds of neurons in the primary auditory cortex during Neural Computation 12, 519–529 (2000)

c 2000 Massachusetts Institute of Technology °

520

M. A. S´anchez-Montan´ ˜ es, P. F. M. J. Verschure, and P. Konig ¨

classical conditioning (Weinberger, 1993; Bakin & Weinberger, 1996; Kilgard & Merzenich, 1998). These results support the suggestion that modulatory substances can act as a “print now” signal gating synaptic plasticity (Singer, von Grunau, ¨ & Rauschecker, 1979). Because the inuence of such a signal onto the cortical network is homogeneous and is not related to the speciŽcs of a given stimulus, it can be called a global mechanism. Local mechanisms in learning and memory refer back to the classical work of Hebb (1949). Since then, many variations and physiological implementations have been suggested (Stent, 1973; Sejnowski, 1977; Buonomano & Merzenich, 1998). Recently Stuart and Sakmann (1994) observed that action potentials can propagate backward into the dendritic tree . This signal provides a possible explanation for the dependence of long-term potentiation (LTP) and long-term depression (LTD) on the relative timing of preand postsynaptic action potentials (Gerstner, Ritz, & van Hemmen, 1993; Markram, Lubke, Frotscher, & Sakmann, 1997). The changes in synaptic efŽcacy are speciŽc to the neurons involved and may be described by rules local in space and time. In spite of the enormous amount of work on local and global mechanisms of synaptic plasticity, their relationship has been little investigated, and, implicitly, their action is assumed to be independent of each other. Here, we investigate an alternative model, where the global and local mechanisms are intimately related. The global mechanism inuences the fraction of action potentials that successfully invade the dendritic tree of the stimulated neurons, a signal exploited by the local process. We demonstrate that such an integrated mechanism allows the combination of the positive properties of its constituents, such as stability, speciŽcity, and exibility, in one learning rule. 2 Methods

We studied local and global mechanisms of synaptic plasticity in a neuronal network consisting of excitatory and inhibitory cortical neurons, subcortical sensory input, and input from the basal forebrain. The sensory afferents target the excitatory neurons and deŽne their receptive Želds. Initially, the synaptic efŽcacy of these connections is randomly distributed and updated according to the learning rule described below. The excitatory cortical neurons project to inhibitory interneurons, which in turn project back to the excitatory neurons, forming a negative feedback loop. The simulated basal forebrain afferents target the inhibitory cortical neurons and have an inhibitory effect (Freund & Gulyas, 1991; Freund & Meskenaite, 1992). All neurons are simulated as integrate-and-Žre units (for quantitative details, see the appendix). The synaptic efŽcacy of the afferent sensory projections to the excitatory neurons evolves according to a modiŽcation of a recently proposed learning rule that uses a backpropagating action potential (Kording ¨ & K¨onig,

Local and Global Gating of Synaptic Plasticity

521

1998). First, when a “backpropagating” action potential arrives at a synapse simultaneously (i.e., within a small symmetrical temporal window) with an action potential in the presynaptic afferent Žber the efŽcacy of the respective synapse is increased (Gerstner et al., 1993; Markram et al., 1997; Magee, Hoffman, Colbert, & Johnston, 1998; Bi & Poo, 1998). Furthermore, we studied the effects of using an asymmetrical temporal window, thus including a dependency of the synaptic modiŽcations not only on the absolute delay of pre- and postsynaptic signals, but also on the temporal order of their arrival, which matches the physiological results more faithfully. Second, activation of inhibitory synapses located at dendritic sites proximal to the soma tree may attenuate the retrograde propagation of the action potential in the dendritic tree (Spruston, Schiller, Stuart, & Sakmann, 1995; Tsubokawa & Ross, 1996). In this case the efŽcacies of activated synapses are decreased. In addition, we implemented heterosynaptic LTD in the simulation: synaptic efŽcacy is decreased in case of postsynaptic activity without coincident presynaptic activity. Thus, in this learning rule, the changes of synaptic efŽcacy are crucially dependent on the temporal dynamics of the neuronal network, in particular on the relative timing of the excitatory and inhibitory inputs to the cortical neurons. We used two kinds of stimulation protocols to investigate the interaction of global and local mechanisms of synaptic plasticity in analogy to experiments on auditory conditioning (Kilgard & Merzenich, 1998). In the Žrst protocol, a pseudo-random sequence of 500 exemplars, repeatedly selected from a pool of 10 different stimuli, was presented alone. In the second protocol, one of the stimuli was paired with activity of the basal forebrain. For both protocols, we analyzed 40 simulations, each starting with random initial connections between sensory and excitatory populations. To control for dependencies on the details of the implementation, we conducted control runs using neurons without a refractory period. All simulations were performed using IQR 427 (Verschure, 1997). 3 Results

As a Žrst step, we investigated the relationship between the formation of representations of input stimuli and the frequency of their occurrence. Two of the stimuli were presented four times more often than the other eight (probability of occurrence of stimuli 1 and 2: 0.2500 versus stimuli 3 to 10: 0.0675). After about 200 stimulus presentations, the network had converged and stabilized. The resulting speciŽcity of the excitatory cortical units showed a bimodal distribution. Neurons either received comparable input upon presentation of any stimulus, and thus were completely untuned (44.8%), or they responded highly speciŽcally to only 1 of the 10 stimuli (50.5%). Few neurons showed intermediate degrees of speciŽcity (4.7%). This is in good accord with our previous results (K¨ording & Konig, ¨ 1998). Next, we investigated the distribution of stimuli speciŽcity for the whole popula-

522

M. A. S´anchez-Montan´ ˜ es, P. F. M. J. Verschure, and P. Konig ¨

tion of neurons. Figure 1A shows that the number of neurons responding to the stimuli presented often (stimuli 1 and 2, red and orange, respectively) is similar to the size of the representation of stimuli presented rarely (stimuli 3–10, yellow to violet). Compiling results from 40 runs, we did not Žnd a signiŽcant inuence of presentation frequency onto the size of the representation (3.0§ 0.4, 3.0§ 0.6, mean§ standard deviation; Figure 1C green curve). Allowing a stronger competition for representing neurons between different stimuli, by reducing the size of the network, leads to a weak dependence of the size of their representation on presentation frequency (data not shown). Thus, the local mechanism alone leads to the formation of stable and speciŽc representations of stimuli which are largely independent of the frequency of stimulus presentation. Next, we investigated the role of the global mechanism. The activation of the nucleus basalis neuron leads to a long-lasting hyperpolarization of the inhibitory neurons. This necessitates a longer integration period of the inhibitory neurons until the threshold is reached. Nevertheless, with the chosen set of parameters, the inhibitory neurons do not Žre in bursts, and those excitatory inputs, which without nucleus basalis activation were falling into the refractory period, now contribute to the activation of the inhibitory neurons. Thus, the mean activity of the inhibitory neurons does not change. This is consistent with experimental results, which found highly complex effects of basal forebrain and direct ACh application, which cannot be characterized by straightforward changes in mean Žring rates of cortical neurons (Sato et al., 1987; Francesconi, Muller, ¨ & Singer, 1988; Murphy & Sillito, 1991; Jimenez-Capdeville, Dykes, & Myasnikov, 1997). The main effect of nucleus basalis stimulation in the model is delay to the inhibitory activity relative to the excitatory activity in the cortical circuit by about 3.3 milliseconds (see Figure 1B). As a result, a much larger fraction of the action potentials generated by the excitatory neurons invade the dendritic tree (21 § 16% and 87 § 17% without and with nucleus basalis activation, respectively). In this case the number of neurons representing the stimulus that is paired with nucleus basalis activation (stimulus 8, mid-blue) increases about 10-fold (see Figure 1A, lower panel). The representations of the other stimuli, however, remain constant. Again, compiling results from a total of 40 runs, we did not Žnd a signiŽcant difference between the size of the representations of stimuli shown at different frequencies (3.0 § 0.4 versus 2.9 § 0.6, high and low rate, respectively). However, the representation of the paired stimulus was signiŽcantly increased (31.7§ 19.8; see Figure 1C, blue curve). Thus, the global mechanism biases the size of the representation toward the paired stimulus, while allowing the map to preserve a stable representation of all stimulus patterns. As a control, a modiŽed learning rule was studied. In case the backpropagating action potential successfully invades the dendritic tree, the sign of change of synaptic efŽcacy is dependent on the sequence of arrival of preand postsynaptic action potentials. An interesting difference was found.

Local and Global Gating of Synaptic Plasticity

523

Figure 1: (A) The tuning of the 100 input neurons is shown in the upper panel using a false color code. Low frequencies correspond to reddish colors, high frequencies to violet, and intermediate frequencies follow a rainbow pattern. The distribution of the preferred frequency of the cortical excitatory units after training, without nucleus basalis activity, is shown in the middle panel using the same color code. Hue indicates the preferred stimulus frequency, while saturation indicates the tuning strength. Each square corresponds to one neuron. For better visibility, they are arranged in Žve rows in order of increasing preferred stimulus frequency. Thus, the graph might be compared to a top view onto the cortex as used by Kilgard and Merzenich (1998). In the lower panel, the tuning of cortical excitatory neurons is shown, while one stimulus (8, mid-blue) was paired with nucleus basalis activity. (B) Temporal relation of the activity of cortical inhibitory neurons in relation to activity of excitatory neurons with (blue) and without (green) simultaneous nucleus basalis activity. (C) Size of the representation of all 10 stimuli with (blue) or without nucleus basalis activity (green) paired with stimulus 8. The error bars indicate standard error of the mean.

524

M. A. S´anchez-Montan´ ˜ es, P. F. M. J. Verschure, and P. Konig ¨

Those cortical neurons representing a particular stimulus did not contact all of the respective input units equally, but a symmetry breaking occurred, and each cortical unit connected only to a subset of afferents sufŽcient for its activation. This can be interpreted as an effect, where a neuron cuts down on a set of inputs sufŽcient for its activation. With respect to the central question of the study, however, no difference was found. Each stimulus was learned by about three to four neurons, and the size of the representation was independent on presentation frequency. Stimulation of the basal forebrain unit lead to a huge increase of the representation of the paired stimulus as before. Thus, in the investigated system, the symmetry or asymmetry of the temporal window for coincidence detection of pre- and postsynaptic action potentials is not relevant for the size of cortical representations. 4 Discussion

In this study, we investigated the interaction of local and global mechanisms regulating synaptic efŽcacy. This relates to previous work in the auditory system of the rat and guinea pig (Weinberger, 1993; Kilgard & Merzenich, 1998). In these experiments, the animals were exposed to acoustic stimuli either with or without paired electrical stimulation of the basal forebrain. The frequent presentation of a stimulus on its own did not lead to an enlargement of its representation in primary auditory cortex. In contrast, presentation of an auditory stimulus paired with electrical stimulation of the basal forebrain induced a dramatic increase of the cortical area devoted to that particular stimulus. This matches well the behavior of the model presented here. By making the propagation of action potentials into the dendritic tree subject to attenuation by inhibitory afferents, the local learning rule implements an inhibitory mechanism (K¨ording & Konig, ¨ 1998). However, this mechanism is acting on the change of synaptic efŽcacy, and not directly on the neuronal activity level. Recent physiological data show that in primary visual cortex, optimally activated neurons tend to Žre before suboptimally activated neurons (Konig, ¨ Engel, Roelfsema, & Singer, 1995). Therefore, the optimally activated neurons can block synaptic plasticity in the suboptimally activated ones. As a consequence, this mechanism allows neurons that are optimally tuned to a stimulus to prevent the recruitment of more and more neurons for its representation. The Žrst protocol demonstrates that the local learning rule leads to a size of neuronal representations that is independent of the stimulus presentation frequency. Nevertheless, particular stimuli, whether rarely or frequently presented, can be emphasized and allocated an increased cortical representation through the global mechanisms, as shown in the second protocol. In our model, the substantial GABA-ergic projection originating in the basal forebrain and terminating on cortical inhibitory neurons (Freund & Guylas, 1990; Freund & Meskenaite, 1992) increases the proportion of sucessfully backpropagating action potentials in the cortical excitatory neurons.

Local and Global Gating of Synaptic Plasticity

525

Interestingly, the much better investigated basal forebrain cholinergic projection (Singer & Rauschecker, 1982) increases the fraction of backpropagating action potentials in cortical neurons (Tsubokawa & Ross, 1997). Thus, these two subcortical projections may act synergistically, enhancing each others effect. Of course, these Žndings do not preclude additional actions of acetylcholine, for example, the modulation of effective intracortical connectivity (Verschure & Konig, ¨ 1999). In the deŽnition of the model, several design decisions had to be made. Increasing the number of parameters in a model to make it more biologically plausible may obscure basic principles. Therefore, we decided not to address the two-dimensional topographic arrangement of feature preferences. These seem to involve independent mechanisms, and several good reviews are available on this topic (e.g., Erwin, Obermayer, & Schulten, 1995). Most notably, the learning rule evaluates only the absolute difference of arrival times of pre- and postsynaptic action potentials. In contrast to the experimental results (Markram, Lubke, Frotscher, & Sakmann, 1997; Bi & Poo, 1998) the sequence of their arrival is not taken into account. This simpliŽcation was made for several reasons. We did not want to create the impression that the observed effects depend on this particular feature. Indeed, in control experiments using a modiŽed learning rule sensitive to the sequence of pre- and postsynaptic action potentials, qualitatively identical results were obtained. The requirement for temporal contiguity is sufŽcient to generate the observed effects. This is not surprising, as the activity statistics of the input units follows a Poisson distribution, and thus did contain additional information. In a complete large-scale model, including feedback of the “cortical” neurons onto units representing neurons in the thalamus, new and interesting properties might emerge. These are, however, subject to current investigation and outside the scope of this article. In the model, the action of the basal forebrain is mediated by a subtle inuence on the temporal relation of inhibitory and excitatory activity in the cortex, inuencing the backpropagation of action potentials. This gives rise to two speciŽc predictions. First, using classical electrophysiological techniques to measure the activity of single excitatory and inhibitory neurons, the spike-triggered average of local Želd potentials can be determined. This measure gives an indication of the relative timing of the recorded neurons with respect to the population activity. Therefore, it is a sensitive indicator for the predicted delay of inhibitory activity during basal forebrain stimulation. Second, the fraction of retrogradely propagating action potentials (and their resulting effects on the calcium concentration) should be increased when the basal forebrain is stimulated. This can be assessed in vivo using the recently introduced two-photon imaging technique (Denk et al., 1994). Thus, a test of these two speciŽc predictions of the proposed mechanisms seems possible with currently available techniques. By allowing a global mechanism to affect synaptic plasticity via a local learning rule, a single integrated mechanism can be deŽned that combines

526

M. A. S´anchez-Montan´ ˜ es, P. F. M. J. Verschure, and P. Konig ¨

continuous learning, stability, speciŽcity, and exibility. The global component, which was previously interpreted as a “print now” signal, might actually be best described as a “print bold” signal. Appendix

The network contains an input layer, a layer of excitatory cells, a layer of inhibitory cells, and one unit representing the activity of the nucleus basalis. The input layer consists of 100 neurons and projects in an all-to-all manner to the set of excitatory cortical neurons. This connectivity is initially random, with weights homogeneously distributed between 0.3 and 0.7 in units of the postsynaptic threshold. Hence, the receptive Želds of the excitatory neurons are initially unspeciŽc. The input neurons are activated for 30 ms and generate spikes following a Poisson process. For physiological realism, we implemented a refractory period for these neurons (5 ms). As a control, simulations without such a refractory period were performed. The spatial distribution of their Žring rate follows a gaussian shape, with a peak of 100 Hz and dispersion of 3 units. The excitatory neurons are modeled as integrate-and-Žre neurons with a time constant of 9.5 ms. The synapses of the projection between input and excitatory neurons are modiŽed according to the learning rule described below. During all the simulations, the weights are kept in the 0–1 range. The set of excitatory units projects in a one-to-one manner to the inhibitory neurons. This projection is subject to a transmission delay of 2 ms. The inhibitory units project all-to-all back to the excitatory neurons. Here as well, a transmission delay of 2 ms is taken into account. The dynamics of the inhibitory neurons are identical to those of the excitatory neurons described above. In addition to excitatory input, they receive inhibitory afferents from the basal forebrain. The learning rule is a modiŽcation of one proposed in Kording ¨ and Konig ¨ (1998). The change of synaptic efŽcacy is contingent on the relative timing of pre- and postsynaptic action potentials. If these two signals are coincident on a timescale t0 D 10 ms (Markram et al., 1997), the synaptic efŽcacy is increased (see equation A.1, aLTP > 0). In case the action potential is attenuated while propagating through the dendritic tree by inhibitory input within 3 ms after its initiation, the synaptic efŽcacy is decreased (see equation A.2, aLTD < 0). The relative timing is evaluated as before. Furthermore, the introduction of heterosynaptic depression leads to a weakening of inactive synapses in the case of postsynaptic activity (see equation A.3, aheteroLTD < 0): D v D aLTP D v D aLTD

t0 t0 C | tpost ¡ tpre| t0 t0 C | tpost ¡ tpre|

(A.1) (A.2)

Local and Global Gating of Synaptic Plasticity

D v D aheteroLTD

527

(A.3)

with t0 D 10 ms and tpost and tpre timing of the pre- and postsynaptic action potential. For control simulation we used a modiŽed learning rule, where the sign of synaptic modiŽcation is dependent on the sequence of pre- and postsynaptic action potentials: D v D aLTP

sign (tpost ¡ tpre)t0 t0 C |tpost ¡ tpre|

with parameters as before. For each stimulus presentation, every synapse is updated only once, even if there are several presynaptic spikes generated by the presynaptic neuron. Acknowledgments

We are grateful to Konrad K¨ording and Mike Merzenich for valuable discussions of the previous work on the learning rule and the experimental data and Daniel Kiper for comments on a previous version of the manuscript. We are happy to acknowledge the support of SPP Neuroinformatics (grants 5002–44888/2&3 to P. F. M. J. V.), SNF (grant 31-51059.97, awarded to P. K.), and an FPU grant from MEC (M. A. S.-M., Spain). References Abbott, L. F. (1990). Learning in neural network memories. Network, 1, 105–122. Alkon, D. L., Amaral, D. G., Bear, M. F., Black, J., Carew, T. J., Cohen, N. J., Disterhoft, J. F., Eichenbaum, H., Golski, S., Gorman, L. K., Lynch, G., Mcnaughton, B. L., Mishkin, M., Moyer, J. R., Olds, J. L., Olton, D. S., Otto, T., Squire, L. R., Staubli, U., Thompson, L. T., & Wible, C. (1991). Learning and memory. Brain Res. Rev., 16, 193–220. Bakin, J. S., & Weinberger, N. M. (1996). Induction of a physiological memory in the cerebral cortex by stimulation of the nucleus basalis. Proc. Natl. Acad. Sci. USA, 93, 11219–11224. Bi, G. Q., & Poo, M. M. (1998). Synaptic modiŽcations in cultured hippocampal neurons: Dependence on spike timing, synaptic strength, and postsynaptic cell type. J. Neurosci., 18, 10464–10472. Buonomano, D. V., & Merzenich, M. M. (1998).Cortical plasticity: From synapses to maps. Annu. Rev. Neurosci., 21, 149–186. Denk, W., Delaney, K. R., Gelperin, A., Kleinfeld, D., Strowbridge, B. W., Tank, D. W., & Yuste, R. (1994). Anatomical and functional imaging of neurons using 2-photon laser scanning microscopy. J. Neurosci. Methods, 54, 151–162. Erwin, E., Obermayer, K., & Schulten, K. (1995). Models of orientation and ocular dominance columns in the visual cortex: A critical comparison. Neural Comput., 7, 425–468.

528

M. A. S´anchez-Montan´ ˜ es, P. F. M. J. Verschure, and P. Konig ¨

Francesconi, W., Muller, ¨ C. M., & Singer, W. (1988). Cholinergic mechanisms in the reticular control of transmission in the cat lateral geniculate nucleus. J. Neurophysiol., 59, 1690–1718. Freund, T. F., & Gulyas, A. I. (1991). GABAergic interneurons containing calbindin D28K or somatostatin are major targets of GABAergic basal forebrain afferents in the rat neocortex. J. Comp. Neurol., 314, 187–199. Freund, T. F., & Meskenaite, V. (1992). Gamma-aminobutyric acid- containing basal forebrain neurons innervate inhibitory interneurons in the neocortex. Proc. Natl. Acad. Sci. USA, 89, 738–742. Gerstner, W., Ritz, R., & van Hemmen, J. L. (1993). Why spikes? Hebbian learning and retrieval of time-resolved excitation patterns. Biol. Cybern., 69, 503– 515. Hasselmo, M. E. (1993).Acetylcholine and learning in a cortical associative memory. Neural Comput., 5, 32–44. Hebb, D. O. (1949). The organization of behavior. New York: Wiley. Jimenez-Capdeville, M. E., Dykes, R. W., & Myasnikov, A. A. (1997). Differential control of cortical activity by the basal forebrain in rats: A role for both cholinergic and inhibitory inuences. J. Comp. Neurol., 381, 53–67. Kilgard, M. P., & Merzenich, M. M. (1998). Cortical map reorganization enabled by nucleus basalis activity. Science, 279, 1714–1718. Konig, ¨ P., Engel, A. K., Roelfsema, P. R., & Singer, W. (1995). How precise is neuronal synchronization? Neural Comput., 7, 469–485. Kording, ¨ K. P., & Konig, ¨ P. (1998). A single learning rule for the formation of efŽcient representations and for the segmentation of multiple stimuli. Soc. Neurosci. Abst., 24, 323.16 Magee, J., Hoffman, D., Colbert, C., & Johnston, D. (1998). Electrical and calcium signaling in dendrites of hippocampal pyramidal neurons. Annu. Rev. Physiol., 60, 327–346. Markram, H., Lubke, J., Frotscher, M., & Sakmann, B. (1997). Regulation of synaptic efŽcacy by coincidence of postsynaptic APs and EPSPs. Science, 275, 213–215. Montague, P. R., Dayan, P., & Sejnowski, T. J. (1996). A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J. Neurosci., 16, 1936–1947. Murphy, P. C., & Sillito, A. M. (1991). Cholinergic enhancement of direction selectivity in the visual cortex of the cat. Neuroscience, 40, 13–20. Sato, H., Hata, Y., Masui, H., & Tsumoto, T. (1987). A functional role of cholinergic innervation to neurons in the cat visual cortex. J. Neurophysiol., 58, 765– 780. Sejnowski, T. J. (1977). Storing covariance with nonlinearly interacting neurons. J. Math. Biology, 4, 303–321. Singer, W., & Rauschecker, J. P. (1982). Central core control of development plasticity in the kitten visual cortex. II: Electrical activation of mesencephalic and diencephalic projections. Exp. Brain Res., 47, 223–233. Singer, W., von Grunau, ¨ M. W., & Rauschecker, J. P. (1979). Requirements for the disruption of binocularity in the visual cortex of strabismic kittens. Brain Res., 171, 536–540.

Local and Global Gating of Synaptic Plasticity

529

Spruston, N., Schiller, Y., Stuart, G., & Sakmann, B. (1995). Activity-dependent action potential invasion and calcium inux into hippocampal CA1 dendrites. Science, 268, 297–300. Stent, G. S. (1973). A physiological mechanism for Hebb’s postulate of learning. Proc. Natl. Acad. Sci. USA, 70, 997–1001. Stuart, G. J., & Sakmann, B. (1994). Active propagation of somatic action potentials into neocortical pyramidal cell dendrites. Nature, 367, 69–72. Tsubokawa,H., & Ross, W. N. (1996).IPSPs modulate spike backpropagation and associated Ca2+i changes in the dendrites of hippocampal CA1 pyramidal neurons. J. Neurophysiol., 76, 2896–2906. Tsubokawa, H., & Ross, W. N. (1997).Muscarinic modulation of spike backpropagation in the apical dendrites of hippocampal CA1 pyramidal neurons. J. Neurosci., 17, 5782–5791. Verschure, P. F. M. J. (1997). Xmorph: A software tool for the synthesis and analysis of neural systems. Technical Report ETH-UZ, Institute of Neuroinformatics. Verschure, P. F. M. J., & Konig, ¨ P. (1999). On the role of biophysical properties of cortical neurons in binding and segmentation of visual scenes. Neural Comp., 11, 1113–1138. Weinberger, N. M. (1993).Learning induced changes of auditory receptive Želds. Curr. Opin. Neurobiol., 3, 570–577. Received March 11, 1999; accepted August 4, 1999.

NOTE

Communicated by Garrison Cottrell

Nonlinear Autoassociation Is Not Equivalent to PCA Nathalie Japkowicz Faculty of Computer Science, Dalhousie University, Halifax, Nova Scotia, Canada, B3H 1W5 Stephen Jos e´ Hanson Department of Psychology, Rutgers University, Newark, NJ 07102, U.S.A. Mark A. Gluck Center for Molecular and Behavioral Neuroscience, Rutgers University, Newark, NJ 07102, U.S.A. A common misperception within the neural network community is that even with nonlinearities in their hidden layer, autoassociators trained with backpropagation are equivalent to linear methods such as principal component analysis (PCA). Our purpose is to demonstrate that nonlinear autoassociators actually behave differently from linear methods and that they can outperform these methods when used for latent extraction, projection, and classiŽcation. While linear autoassociators emulate PCA, and thus exhibit a at or unimodal reconstruction error surface, autoassociators with nonlinearities in their hidden layer learn domains by building error reconstruction surfaces that, depending on the task, contain multiple local valleys. This interpolation bias allows nonlinear autoassociators to represent appropriate classiŽcations of nonlinear multimodal domains, in contrast to linear autoassociators, which are inappropriate for such tasks. In fact, autoassociators with hidden unit nonlinearities can be shown to perform nonlinear classiŽcation and nonlinear recognition.

1 Introduction

An autoassociator is a feedforward connectionist device whose goal is to reconstruct the input at the output layer. When used with a hidden layer smaller than the input or output layer and linear activations only, the autoassociator implements a compression scheme that was shown to be equivalent to principal component analysis (PCA)—also known as singular value decomposition—by Baldi and Hornik (1989). Another interesting and related result was obtained by Bourlard and Kamp (1988), who claim, “For Neural Computation 12, 531–545 (2000)

c 2000 Massachusetts Institute of Technology °

532

N. Japkowicz, S. J. Hanson, and M. A. Gluck

autoassociation with linear output units, the optimal weight values can be derived by standard linear algebra, consisting essentially in singular value decomposition (SVD) and making thus the nonlinear functions at the hidden layer completely unnecessary.” They backed their claim using theoretical considerations, while Cottrell and Munro (1988) arrived at the same conclusion independently, using an experimental methodology. Paradoxically, nonlinear autoassociators are also famous for their capacity to solve the encoder problem (Rumelhart, Hinton, & Williams, 1986), a problem that cannot be solved by PCA because of the singularity of the principal components. 1 Moreover, recent research suggests that nonlinear autoassociators are well suited for classiŽcation of certain types of nonlinearly separable and multimodal domains (Japkowicz, Myers, & Gluck, 1995; Petsche et al., 1996), another task for which PCA and linear autoassociation are not appropriate because of their linear restriction. In spite of the obvious contradiction exhibited by these facts, there has been little effort at explaining the source of this inconsistency. The purpose of this article is to demonstrate experimentally that Bourlard and Kamp’s (1988) claim does not necessarily hold and to analyze the differences between PCA and nonlinear autoassociators when it does not. In particular, we test the performance of nonlinear autoassociators relative to PCA and linear autoassociation on a nonlinear multimodal classiŽcation problem inspired by the projected topology of a real-world domain. After showing that there is indeed a difference in the classiŽcation accuracy obtained by the linear and nonlinear devices on this domain, we explain this difference by comparing the reconstruction error surfaces constructed by each system over the test domain. More speciŽcally, our experiments demonstrate that nonlinear sigmoidal single-hidden-layer autoassociators may sometimes interpolate sets of points differently from PCA. In particular, we show that on the test domain considered, while PCA (or equivalently, linear autoassociation) exhibits at or unimodal reconstruction error surfaces, nonlinear sigmoidal autoassociators build error reconstruction surfaces that contain multiple local valleys. In fact, the reconstruction error surfaces built by the sigmoidal autoassociators are closely related to the reconstruction error surface built by a radial basis function autoassociator, thus suggesting that when the sigmoidal autoassociator does not operate like PCA—computing a globalized solu-

1 Kruglyak (1990) and Phatak, Choi, and Koren (1993) demonstrate that for any N, the N-input, N-output encoder problem can, actually, be represented by an autoassociator of only two nonlinear hidden units. The fact that large encodings can actually be learned by two-hidden-unit autoassociators was demonstrated by Zipser (1989), who showed that such devices are capable of learning how to encode the position of a spot (or cluster) of normally distributed points appearing at random locations on a 10 £ 10 squared array.

Nonlinear Autoassociation

533

tion to the interpolation problem—then it operates like a radial basis function autoassociator—computing a localized solution to the interpolation problem.2 We begin our study by discussing the limitations of Bourlard and Kamp’s (1988) claim. We then demonstrate experimentally that these limitations can indeed be exceeded by standard problems within the context of classiŽcation and we analyze these results, concluding with some speculations as to why sigmoidal autoassociators sometimes do and sometimes do not operate like PCA. 2 Limitation of the Conventional Wisdom

When considering the results of Baldi and Hornik (1989), Bourlard and Kamp (1988), and Cottrell and Munro (1988) and the evidence that nonlinear autoassociators can solve problems that cannot be solved by PCA (Rumelhart et al., 1986; Japkowicz et al., 1995; Petsche et al., 1996), an important question comes to mind: Does the Bourlard and Kamp (1988) claim hold in all cases, or does it depend on certain assumptions concerning the underlying domain or the autoassociator? A careful look at the discussion laid out in Bourlard and Kamp (1988) reveals that the claim actually does depend on a particular condition. More speciŽcally, let F be the nonlinear function present at the output of the hidden units of a nonlinear autoassociator (F is the function that makes the autoassociator nonlinear). The assumption Bourlard and Kamp (1988) make in order to carry out their demonstration is that for small values of x, F( x) can be approximated as closely as desired by the linear part of its power series expansion. However, this means that x D h, the vector of presynaptic hidden unit activations (i.e., the hidden unit activations prior to their transformation by F), must be in the linear range of the squashing function. This suggests that nonlinear autoassociators do not necessarily emulate PCA when the net inputs are outside the linear range of the squashing function. The remainder of the article demonstrates experimentally that there are indeed differences between the linear and nonlinear schemes and that these differences have practical consequences for the task of classiŽcation. 3 Experiments

We conducted two sets of experiments for this article.

2 Although both the sigmoidal and radial basis function autoassociators are nonlinear, all further references to nonlinear autoassociators correspond to references to sigmoidal autoassociators.

534

N. Japkowicz, S. J. Hanson, and M. A. Gluck

3.1 SpeciŽcation of the Devices and Description of the Task.

3.1.1 Devices. The devices we used in our experiments are PCA and three autoassociators. From an analytical point of view, the four devices differ: two of them are purely linear, and the other two have nonlinear capabilities. The three autoassociators are connectionist methods that differ from one another with regard to the type of hidden and output layer they use. In more detail, we compare a one-hidden-layer autoassociator whose hidden and output layers are both linear (L-L); a one-hidden-layered autoassociator whose hidden layer is nonlinear but whose output layer is linear (NL-L); and a one-hidden-layered autoassociator whose hidden layer and output layer are both nonlinear (NL-NL).In each of these devices, the function used in the nonlinear units is the usual logistic function. 3.1.2 Task. The operation of the four devices was contrasted on the task of classiŽcation. A formal deŽnition of the classiŽcation problem typically involves an input vector x and a discrete response vector y, which are such that the pair (x, y) belongs to some unknown joint probability distribution, P. The goal of classiŽcation is to induce a function f ( x) from a set (x1 , y1 ), ( x2 , y2 ), . . . , ( xN , yN ) of training examples, so that f ( x) predicts y. Here we assume that y is a binary vector. Although classiŽcation by neural networks is typically performed using a discrimination network, 3 it can also be performed using a recognition approach implemented by PCA or the autoassociator. This approach is discussed in Oja (1983) for PCA and in Hanson and Kegl (1987) for autoassociators and operates as follows.4 During the training phase, the autoassociator is taught how to reconstruct examples of the concept. Once training is completed, classiŽcation is performed on a new vector xTest by comparing its reconstruction error5 to a threshold, and assigning it to the conceptual class if the reconstruction error is smaller than this threshold and to the other class otherwise. The idea behind this recognition-based classiŽcation scheme is that since the autoassociator is trained to compress and decompress examples of the conceptual class only, 3 Using such a scheme, yi , the response variable associated with input vector xi , takes on (in the simplest case) a value of 1 or 0, which is interpreted to mean that xi belongs or does not belong to the conceptual class of the problem. After training the network to approximate the function f (x) from which the training examples are believed to have been generated, it is expected that if the training set was appropriately designed, the network will be able to compute the appropriate label (1 or 0) for any input vector of size d, the size of the input layer, even if that vector did not appear in the training set. 4 We describe the method for autoassociators, taking into consideration that it is similar for PCA. j 5 The reconstruction error corresponds to S d [x j )]2 , where g is the function jD1 Test ¡g(xTest j

implemented by the autoassociator and xTest and f j (xTest ) represent input unit j and output unit j of the network, respectively.

Nonlinear Autoassociation

535

Figure 1: Transition from the sonar detection domain to the artiŽcial nonlinear domain.

when tested on a novel data point, it will compress and decompress it appropriately if this example belongs to the conceptual class, but it will not do so appropriately if the example does not belong to the conceptual class. 3.2 SpeciŽcation of the Test Domain. In order to compare and explain the classiŽcation performance of the four devices considered in this study, two experiments were conducted for each classiŽer on a two-dimensional artiŽcial domain. This problem was inspired from real-world data as shown in Figure 1, which illustrates the transition from real-world to artiŽcial data. SpeciŽcally, Figure 1a displays a two-dimensional representation of sonar target recognition data available from the University of California at Irvine Repository for Machine Learning. These data were compressed from 60

536

N. Japkowicz, S. J. Hanson, and M. A. Gluck

features to 2 using PCA. In this plot, black triangles and white circles correspond to actual points (mines and rocks, respectively), and data clusters are indicated by polygons. Figure 1b shows the same two-dimensional representation of sonar target recognition data, but this time with means replacing the clusters. 6 Although this Žgure is just a two-dimensional projection of the original data set, it clearly indicates the typical kind of problem that the autoassociator has to deal with: multimodality. We chose a simple twodimensional abstraction of the multimodality problem for benchmarking purposes and explaining the capabilities of autoassociators. This artiŽcial domain, illustrated in Figure 1c, consists of four conceptual data clusters with means represented by black triangles and nine counterconceptual data clusters with means represented by white circles. In more detail, the conceptual clusters are located at points (.2, .2), (.2, .8), (.8, .2), and (.8, .8), respectively; the counterconceptual clusters are located at points (.1, .1), (.1, .9), (.2, .5), (.5, .2), (.5, .5), (.5, .8), (.8, .5), (.9, .1), and (.9, .9), respectively. Each cluster is composed of 50 points normally distributed around these means and with variance s 2 D .01. The counterconceptual clusters are further divided into internal and external clusters. Internal counterconceptual clusters correspond to the counterconceptual data that are located inside the convex hull deŽned by the conceptual data, while external counterconceptual clusters correspond to the counter-conceptual data located outside this convex hull. 3.3 Experiment Set 1. The Žrst experiment we conducted consisted of comparing the classiŽcation accuracy of the four systems on the test domain of Figure 1c. After having been tuned to their optimal settings, the four devices were trained on conceptual data, and a threshold was established by Žtting the reconstruction error obtained on additional conceptual data to a gaussian and setting a boundary corresponding to a prespeciŽed conŽdence interval. Subsequently, the systems were tested on a testing set containing both instances and counterinstances of the concept.

3.3.1 Tuning the Devices. The number of hidden units that the nonlinear connectionist systems used was determined by running each network with 1, 2, 4, 8, 16, 32, and 64 hidden units on Žve cross-validation sets containing 25 points per cluster. It was concluded that in order to reach a good classiŽcation performance, the two nonlinear autoassociators had to be trained with 16 hidden units. It was also shown that the networks had converged by Epoch 2000, which was thus selected as a stopping point. Optimization in this experiment was performed using the backpropagation procedure with standard learning rate and momentum of 0.05 and 0.9, respectively. For PCA, since the test domain is two-dimensional, only one or 6

Clusters of fewer than three points were ignored.

537

Average Percent Error Over Five Trials

30

Average Percent Error

40

Nonlinear Autoassociation

0

10

20

Pos Misclass Neg Misclass

PCA

L-L

NL-L NL-NL

Figure 2: ClassiŽcation error rate of the four classiŽcation schemes. Positive misclassiŽcation rates are indicated in gray; negative ones are indicated in black.

two principal components could be used. It was determined that one principal component yields a better classiŽcation rate than two, and therefore the PCA experiment was conducted using a single principal component. Linear autoassociation performed better as well with a single hidden unit than with two or more; therefore, it too was tested with a single hidden unit. The stopping criterion, learning rate, and momentum that the linear autoassociator used were the same as those used by the nonlinear devices. In the four systems, the threshold was set so as to allow for a 97% conŽdence interval. 3.3.2 Accuracy Rates. The results we obtained in this experiment are presented in Figure 2. This Žgure plots the average classiŽcation error rates over Žve trials obtained by the four systems on the test domain of Figure 1c. The graph shows that PCA and L-L obtained very large classiŽcation error rates, whereas the two nonlinear autoassociators obtained low error rates. This demonstrates that while PCA and L-L can technically be used for classiŽcation, they are not that useful on the problem of Figure 1c, whereas both

538

N. Japkowicz, S. J. Hanson, and M. A. Gluck

NL-L and NL-NL yield acceptable classiŽcation rates on that problem.7 This result thus conŽrms that there can be a difference in the computations performed by PCA (or linear autoassociation) and nonlinear autoassociation and that this difference has practical consequences for the task of classiŽcation. In order to analyze this difference in more depth, we conducted further experiments. 3.4 Experiment Set 2. These experiments sought to explain the results obtained in the previous set of experiments by computing and plotting the reconstruction error surfaces constructed by PCA and the three connectionist systems after being trained on the conceptual data (the data summarized by black triangles in Figure 1c). In other words, we were interested in Žnding out how the interpolation strategy that the various devices considered in this study used differ from one another.

3.4.1 Tuning the Devices. For experiment set 2, two different hidden unit settings were tested: expansion and compression. This was done so as to Žnd out whether the systems operate qualitatively differently when used in a nonbottleneck or in a bottleneck fashion. In the expansion setting, the same 16-hidden-unit setting was used for the nonlinear systems as those used in the Žrst experiment, while PCA and the linear autoassociator were tested with two principal components or hidden units. In other words, the optimal nonbottleneck setting for the nonlinear systems was compared to the only nonbottleneck setting possible for the linear systems.8 All the other parameters remained the same as in experiment set 1, except for the learning rate of L-L, which had to be increased from 0.05 to 0.1 for full convergence to take place. In the compression setting, the number of principal components and hidden units were restricted to one for all the systems since it is the only bottleneck setting available given that the test domain is two-dimensional. All the other parameters remained the same as in experiment 1. 3.4.2 Expansion Results. The results we obtained using the expansion setting of experiment set 2 are presented in Figure 3, which displays threedimensional plots of the error ratio surfaces constructed by NL-L and NLNL, respectively. The results for PCA and linear autoassociation are not displayed since they exhibit a at reconstruction error surface. In the graphs of Figures 3a and 3b, the plots are drawn along the x-, y-, and z-axes where x corresponds to the x-axis of the input domain, y to its y-axis, and z is the

7 Nonlinear autoassociation-based classiŽcation was previously used in practical settings involving a CH-46 helicopter gearbox and motor monitoring (Japkowicz et al., 1995; Petsche et al., 1996) and yielded accurate results. 8 For the linear autoassociator, hidden layers greater than two are actually possible, but they are equivalent to hidden layers of size two.

Nonlinear Autoassociation

539

Figure 3: (a) Reconstruction errors of NL-L and (b) NL-NL, with 16 hidden units on the artiŽcial domain generated from Figure 1c.

error ratio at every point considered in the space such that: z( x, y) D

1 (( A( x) ¡ x)2 C ( A( y) ¡ y) 2 ). c

(3.1)

In the formula, x and y represent the two dimensions of each data point, A() is the function realized by the trained autoassociators, and c is a modeldependent scale factor used for plotting purposes. The plots of Figures 3a and 3b are particularly helpful for understanding the nature of the solutions computed by the nonlinear autoassociators and contrasting them to the solution obtained by the linear autoassociator and PCA. In particular, they show that these error surfaces are qualitatively different from the ones computed by PCA and the linear autoassociator since, while the nonlinear autoassociators build multiple-local-valley representations of the underlying domain (see Figures 3a and 3b), PCA and the linear autoassociator learn how to reconstruct the domain perfectly and exhibit a at reconstruction error surface. This suggests that the solutions computed by autoassociators with nonlinearities in their hidden layer use a multimodal interpolation bias that the linear methods do not use. 9 Our results of Figure 3 can be used to ex9 As expected from Baldi and Hornik (1989), our results show that the linear autoassociator is capable of emulating PCA. However, in order for the two paradigms to be equivalent, the linear autoassociator had to be trained with a learning rate of 0.1 (instead of the learning rate of 0.05 used by the nonlinear systems). When a learning rate of 0.05 was used, the linear autoassociator got stuck in a saddle point (see Baldi & Hornik, 1989) and returned a constant output corresponding to the mean of the four training clusters. This observation is interesting in its own right because it underlines the difference between linear and nonlinear autoassociators even prior to the linear autoassociator ’s full convergence. Indeed, it suggests that the linear autoassociator tackles the reconstruction problem globally, using a unimodal bias, whereas the nonlinear autoassociators use local

540

N. Japkowicz, S. J. Hanson, and M. A. Gluck

plain the nonlinear autoassociators (NL-L and NL-NL) results of Figure 2, which show that nonlinear autoassociators are capable of classifying the nonlinearly separable and multimodal domain of Figure 1c. Indeed, since the reconstruction error surfaces of the nonlinear autoassociators include several local valleys centered at each of the positive clusters (see Figures 3a and 3b), the error ratios of the counterconceptual data can all be appropriately high relative to their conceptual counterparts, whether they are located in the interior, the sides, or the exterior of the square underlying the conceptual components. Thus, all the counterconceptual data get appropriately differentiated with respect to the conceptual data. The classiŽcation results obtained by PCA and linear autoassociation in Figure 2 will be explained next, since they were obtained with bottleneck devices. 3.4.3 Compression Results. The results obtained in the previous set of experiments demonstrate that nonlinear autoassociators are not equivalent to linear autoassociators or PCA in the case where the number of hidden units exceeds the number of input units or when the number of principal components is equal to the domain dimensionality. Autoassociators, however, are most typically used as compression devices with a number of hidden units smaller than the number of input units. We now discuss whether the same result also holds in this case. Figures 4a, 4b, and 4c display the reconstruction error surfaces obtained by PCA or L-L (Figure 4a), NL-L (Figure 4b), and NL-NL (Figure 4c) using a single principal component or hidden unit. These reconstruction errors were also obtained using equation (3.1). The reconstruction error surfaces obtained by these devices show that none is appropriate for full classiŽcation of the test domain of Figure 1c since they can only classify all (or most) of the data (whether conceptual or counterconceptual) in one diagonal of the input space as conceptual and all (or most) of the other data (whether conceptual or counterconceptual, again) as counterconceptual. In other words, because they do not interpolate the training set fully, these devices are not useful for classiŽcation, and this explains the high classiŽcation error rate obtained by the linear systems in Figure 2.10 Nevertheless, these results are interesting since, as in the nonbottleneck case, they do illustrate the difference between the linear and the nonlinear schemes. Indeed, while the linear systems display an undistorted surface with no saddle points (see Figure 4a), the nonlinear ones have saddle

strategies characterized by their multimodal interpolation biases. 10 While for the test domain used in this study, classiŽcation can be achieved only if using a number of hidden units greater than the number of input units, in the work of (Japkowicz et al. (1995) and Petsche et al. (1996), the number of hidden units is smaller than the number of input units. This suggests that the hidden-to-input unit ratio does not have any bearing on the behavior of nonlinear autoassociators in classiŽcation tasks, since what really matters is their capacity to interpolate the training set regardless of how many hidden units that may take.

Nonlinear Autoassociation

541

Figure 4: Reconstruction errors of (a) PCA or L-L (with one principal component or one hidden unit), (b) NL-L, and (c) NL-NL (with one hidden unit) on the artiŽcial domain generated from Figure 1c.

points (see Figures 4b and 4c). Thus, the compression results demonstrate that nonlinear autoassociation is not equivalent to linear autoassociation or PCA even when the number of hidden units or principal components is smaller than the number of input units. 4 Discussion

We now analyze the results we obtained and explain why our observations depart from the expectations derived from the past literature. In addition, we compare the results of the nonlinear autoassociators to the result of another well-known paradigm, radial basis functions, and speculate on what nonlinear autoassociators compute and why they do so. 4.1 Nonlinear Autoassociation Versus PCA and Linear Autoassociation. The difference between the linear and nonlinear systems can be ex-

542

N. Japkowicz, S. J. Hanson, and M. A. Gluck

plained by the fact that for the particular test domain considered,11 Bourlard and Kamp’s (1988) assumption has been violated. Indeed, their discussion was shown to hold only in the case where the inputs to the nonlinear activation function of the hidden units remain in the linear range of the squashing function. An observation of the presynaptic activations of the 16 hidden units of NL-NLreveals that, on average, these values are equal to m D 0.6953 with variance s 2 D 0.1919. Similarly, for NL-L, we found that m D .3222 and s 2 D .0548. Although for input values in the [¡1, 1] interval, the sigmoid function is close to linear, our results seem to suggest that a small violation of the Bourlard and Kamp (1988) assumption can make a big difference in certain contexts. While this difference might be marginal and thus overlooked when considering the hidden layer alone, it is magniŽed in the output layer, as suggested by the plots of Figures 3a and 3b, which do not display a at surface of the sort computed by PCA or autoassociation. This remark also holds for the encoder problem of Rumelhart et al. (1986), which can be solved by an autoassociator with nonlinear hidden units but not by a linear one. In this problem, nonlinearities in the hidden layer are also shown to have an impact. 4.2 What Do Nonlinear Autoassociators Compute? A Comparison with RBF Autoassociators. We will now attempt to establish the nature of the

computations taking place in the nonlinear autoassociators by comparing their reconstruction error surfaces to the reconstruction error surface obtained by a radial basis function (RBF) autoassociator with gaussian activations in the hidden units and linear output activations. The reconstruction error surface obtained by such a device on the domain of Figure 1c is plotted in Figure 5. The particular RBF network that yielded this Žgure has a capacity of four hidden units, and the variance of the gaussian functions is Žxed at and set to 2.8. The reconstruction error surface of this autoassociator was computed in the same fashion as that of the other autoassociators, using equation (3.1). The similarity between the reconstruction obtained by the nonbottleneck, nonlinear autoassociators (especially NL-L, which also has linear output activations) and the RBF autoassociator suggests that, like the RBF network, the nonlinear autoassociators use their 16 hidden units to put centers down over the four clusters representing the training data, and evaluate the distance of the testing data to these centers. 12 This result, together with the results of Bourlard and Kamp (1988), suggests that nonlinear autoassociators can resort to one of two modes of operation: either they operate like PCA, seeking a globalized solution to the reconstruction problem they are trying to solve, or they operate like an RBF autoassociator, seeking a localized solution.

11 12

For other classes of domains, this might not be the case. We thank Gary Cottrell for suggesting this comparison.

Nonlinear Autoassociation

543

Figure 5: Reconstruction errors of the RBF autoassociator on the artiŽcial domain generated from Figure 1c. The network has a capacity of four hidden units, and the variance of its gaussian functions is Žxed and set at 2.8.

4.3 On the Duality of Nonlinear Autoassociators: Our Speculation.

The question we now ask is: When does a sigmoidal autoassociator resort to one or another of the two modes of action described in the previous section? Because of the current lack of technology available for analyzing feedforward neural networks fully, we can only speculate as to when each phenomenon will take place. In particular, we suggest that the sigmoidal autoassociator will make use of its nonlinearities when the data set it attempts to reconstruct is multimodal and the different clusters that compose it are very spread out. 13 Indeed, if the data set is unimodal or if it is multimodal but can easily be confused for unimodal data, then every point in the training set activates hidden unit values located in the same vicinity that increase or decrease monotonically. Such data can thus be approximated linearly, in the same fashion as they would be by a linear autoassociator. On the other hand, if the data set is multimodal and the different clusters constituting it are very spread out, then approximation of their hidden unit activation values by a linear function will fail because of the discontinuities introduced by the multimodal nature of the domain. In such cases, the sigmoidal part of the activation function is very useful since it permits the local processing of separate clusters in the same fashion as RBF autoassociators. Indeed, by using a sigmoidal function, a hidden unit can map all the data contained in different clusters to approximately the same default activation value (of 0 or 1) while it can map the data in some other clusters, which fall in the linear part of the sigmoidal function, to more interesting values. Each hidden unit can then use the same strategy for different clusters, and thus a 13 Support for this speculation can be found in Japkowicz (1999), which shows a clear decrease in classiŽcation accuracy by NL-NL as the conceptual clusters of the domain in Figure 1c are moved closer to one another.

544

N. Japkowicz, S. J. Hanson, and M. A. Gluck

global solution can be found by the overall network by compounding the localized solution computed by each hidden unit. Because in the encoder problem, the autoassociator is expected to reconstruct data points located far away from each other—since they are located at extreme corners of the input space—this speculation also explains why the nonlinear autoassociator resorts to a localized solution not available to PCA in that case.

5 Summary

The purpose of this article was to address the paradox raised by the claim of Bourlard and Kamp (1988) that, in autoassociation, nonlinearities in the hidden units are completely unnecessary, given that nonlinear autoassociators are known to perform tasks that cannot be solved by PCA or linear autoassociation. We began by demonstrating that although both PCA (or linear autoassociation) and nonlinear sigmoidal autoassociation can be used for classiŽcation, the linear systems are not appropriate for certain types of multimodal and nonlinear domains, whereas the nonlinear systems are capable of classifying such domains accurately. An explanation of this result is provided by the observation of the interpolation surfaces constructed by the linear and nonlinear systems when trained on the conceptual examples of our classiŽcation test domain. The difference in the interpolation and, as a result, the classiŽcation scheme used by the linear and nonlinear devices thus contradicts the claim by Bourlard and Kamp (1988) initially considered. This contradiction is resolved by extracting the assumption on which the claim is based and showing that this assumption is not necessarily always veriŽed. In a last step, we note that when the nonlinear autoassociator does not operate in a linear fashion, it emulates an RBF autoassociator and thus computes a localized solution to the problem it attempts to solve. Speculations as to when nonlinear autoassociators do resort to a linear operative mode and when they do not do so are then formulated and conclude the article.

Acknowledgments

We express our gratitude to Gary Cottrell and an anonymous reviewer for their extremely helpful comments. We also thank Inna Stainvas for useful discussions about autoassociators and for her careful reading of a version of the manuscript for this article. This research was supported by the OfŽce of Naval Research through grant N0014-88-K-0112 to M. A. G. We are also grateful to the School of Mathematical Science of Tel-Aviv University for its Žnancial support and the Cognitive Science Laboratory of Princeton University for the use of computing facilities.

Nonlinear Autoassociation

545

References Baldi, P., & Hornik, K. (1989). Neural networks and principal component analysis: Learning from examples without local minima. Neural Networks, 2, 53–58. Bourlard, H., & Kamp, Y. (1988). Auto-association by multilayer perceptrons and singular value decomposition. Biological Cybernetics, 59, 291–294. Cottrell, G. W., & Munro, P. (1988). Principal component analysis of images via back propagation. In Proceedings of the Society of Photo-Optical Instrumentation Engineers. Cambridge, MA. Hanson, S. J., & Kegl, J. (1987). PARSNIP: A connectionist network that learns natural language grammar from exposure to natural language sentences. In Proceedings of the Ninth Annual Conference on Cognitive Science. Japkowicz, N. (1999). Concept-learning in the absence of counter-examples: An autoassociation-based approach to classiŽcation. Technical Report DCS-TR-390, Dept. of Computer Science, Rutgers Univ. Japkowicz, N., Myers, C., & Gluck, M. A. (1995). A novelty detection approach to classiŽcation. In Proceedings of the Fourteenth Joint Conference on ArtiŽcial Intelligence (pp. 518–523). Kruglyak, L. (1990). How to solve the N bit encoder problem with just two hidden units. Neural Computation, 2, 399–401. Oja, E. (1983). Subspace methods of pattern recognition. New York: Wiley. Petsche, T., Marcantonio, A., Darken, C., Hanson, S. J., Kuhn, G. M., & Santoso, I. (1996). A neural network autoassociator for induction motor failure prediction. In D. Toueretzky, M. Mozer, & M. Hasselmo (Eds.), Advances in neural information processing systems, 8. Cambridge, MA: MIT Press. Phatak, D. S., Choi, H., & Koren, I. (1993). Construction of minimal N-2-N encoders for any N. Neural Computation, 5, 783–794. Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning internal representations by error propagation. In D. E. Rumelhart & J. L. McClelland (Eds.), Parallel distributed processing (pp. 318–364). Cambridge, MA: MIT Press. Zipser, D. (1989). Programming neural nets to do spatial computations. In N. Sharkey (Ed.), Models of cognition: A review of cognitive science. Norwood, NJ: Ablex. Received August 8, 1997; accepted May 14, 1999.

LETTER

Communicated by David Wolpert

No Free Lunch for Noise Prediction Malik Magdon-Ismail California Institute of Technology, Pasadena, CA 91125, U.S.A. No-free-lunch theorems have shown that learning algorithms cannot be universally good. We show that no free funch exists for noise prediction as well. We show that when the noise is additive and the prior over target functions is uniform, a prior on the noise distribution cannot be updated, in the Bayesian sense, from any Žnite data set. We emphasize the importance of a prior over the target function in order to justify superior performance for learning systems. 1 Introduction

Already established in the learning theory literature are a set of “no-freelunch” (NFL) theorems for various types of learning systems (Wolpert, 1996a, 1996b; Zhu, 1996; Cataltepe, Abu-Mostafa, & Magdon-Ismail, 1999) and for optimization methods (Wolpert & Macready, 1997). The essential content of these theorems is that no learning algorithm can be universally good. A learning algorithm that performs exceptionally well in certain situations will perform comparably poorly in other situations. Thus, in particular, random algorithms that always yield average performance, cross validation, bootstrapping, and anti–cross validation are all, in some sense, on an equal footing. Stopping early at a higher training error than the minimum achievable gains nothing if the hypotheses yielding that training error are chosen with equal probability. A search technique that performs well on one cost function will perform poorly on another cost function. These existing results demonstrate that for certain problems, given a certain criterion of performance, it is not possible to do well consistently in the most general setting. Thus, one needs to make some assumptions about the underlying structure of the problem to be able to claim any kind of superior performance. Here we look at the problem of inferring the noise distribution from a data set. The existing NFL results look at a problem of learning a dependency (not necessarily deterministic) from a Žnite data set and making statements about the out-of-sample performance. The problem we treat here is to deduce properties of the noise distribution (for example, the noise variance) from a Žnite data set that consists of input-output pairs. The outputs have a deterministic dependence on the inputs (the target function) and a noisy component. One approaches the problem with some prior belief about the noise distribution, and one hopes that the data will enable a Žne-tuning of that prior Neural Computation 12, 547–564 (2000)

c 2000 Massachusetts Institute of Technology °

548

Malik Magdon-Ismail

Figure 1: Two sample noisy data sets. The sets were created by taking two functions and adding noise to them. The x-values are the same in both cases.

belief to the extent that more precise statements can be made about the properties of the noise. An accurate estimate of the noise distribution is useful because the noise sets a fundamental limit on the performance of a learning system (Cortes, Jackel, & Chiang, 1994). Noise may play other roles as well: for example, in Žnancial markets, the noise level is itself a tradable quantity. Our criterion for selection of a noise distribution will be to maximize its posterior probability. We will show that having a uniform prior on the possible realizations of the target function leads to no update for a wellbehaved prior on the noise distributions (in the Bayesian sense) from any Žnite data set. Thus the posterior over noise distributions will equal the prior, and we may as well have not even looked at the data. One can compare this to the analogous NFL result of Wolpert: if the prior over target functions is uniform, then the out-of-sample error is constant, and thus one need not have even looked at the data before picking a hypothesis function. Let us consider the question posed in Figure 1. Which data set has more noise? Figure 2 shows the same data, but this time with the function that created

No Free Lunch for Noise Prediction

549

Figure 2: Noisy data and the functions that generated them. The same noisy data sets that were shown in Figure 1 and the functions that were used to produce them. In (b), the noise variance is roughly six times larger.

the data. Now which data set has more noise? The task is considerably easier given this extra information. We will show that some extra information of this form is essential. This article is organized as follows. We start with an example in section 2, where we attempt to predict a noise variance when learning using linear models. In this section we show that one method for predicting the noise variance using a bootstrap technique does not work in the naive form presented. As a result, one might therefore spend a lot of effort trying to Žnd a more sophisticated technique that might work. That this would be fruitless (for this speciŽc case and the more general case) is the purpose of the remainder of the article. In section 3, we set up the problem of noise prediction in general; section 4 presents the main results, beginning with the simple case of boolean functions and continuing to the more general case. We show that information about the noise distribution cannot be obtained from a Žnite data set if the target distribution is assumed uniform. We conclude with section 5, where we also discuss the implications of the NFL theorems.

550

Malik Magdon-Ismail

2 An Example Using Linear Models

In this section we illustrate the conclusions of this chapter for the case of linear learning models on the input space R d . It is not our goal to be strictly rigorous, but rather to illustrate the point. Let the hypothesis functions be of the form gw ( x) D w ¢ x C w 0 ,

(2.1)

where w , w 0 are called the weights. We are given a data set D D fxa , ya D f ( x a ) Cna gN aD1 . f (¢) is some deterministic function (we place no other restrictions on the function) and na is noise, drawn independently and identically distributed with zero mean and variance s 2 . We use mean squared deviation as the measure of error. DeŽne the training error as

Etr D

1 X ( w ¢ x a C w0 ¡ ya ) 2 . N a

(2.2)

DeŽne w ¤ , w ¤0 as those weights that minimize Etr . It can then be shown that at this minimum, the expected training error is given by (theorem 4 in the appendix)1 hEtr¤ i

,n

D E0 C s 2 ¡

¡ ¢

s 2 ( d C 1) C B 1 CO 3 N N2

.

(2.3)

We use h¢i to denote expectations, and the expectation here is over the possible data sets (the input space and the noise, where we have made explicit the expectation over the noise).2 E0 is the bias, where the bias is the expected training error of the best hypothesis function in our learning model. B is a constant given by theorem 4. For the purposes of this section, the exact expression for B is not essential. It sufŽces to know that B is related to the variance of a sample statistic around its population value. The goal here is to estimate the noise variance, s 2 . The training error used as an estimate of the noise variance is asymptotically biased. For large N, it overpredicts the noise variance by the bias, E0 . Since we know the form of hEtr¤ i, perhaps we can do better. We can sample N1 data points from the N 1

We use standard order notation: N!1

f (N) D o(g(N)) ) f (N) / g(N) ¡! 0 f (N) D O(g(N)) ) f (N) · Cg(N) for some C > 0. 2

In the language of the NFL theorems (Wolpert, 1996b), hE tr¤ i , n is E(C| f, m) where the on-training-set likelihood and the quadratic loss function are used.

No Free Lunch for Noise Prediction

551

data points and estimate the training error. We use bootstrapping (Shao & Tu, 1996) to estimate hEtr¤ ( N1 )i, and by varying N1 , we can Žt the resulting dependence of hEtr¤ ( N1 )i on 1 / N1 using equation 2.3. Let the slope of this O From equation 2.3, we see that AO is an estimate for s 2 ( d C 1) C B; Žt be A. therefore, AO / ( d C 1) used as an estimate of s 2 is off by B / ( d C 1). Perhaps we can estimate B by the bootstrap as well. Given a bootstrapped estimate for B, we can combine it with AO to obtain an estimate for s 2 . A more careful analysis presented in the appendix shows that this too leads to failure. Thus, to get information on s 2 , perhaps a more subtle approach such as looking at terms with higher order in 1 / N would work. Perhaps some method other than simply using the training error would yield a better result. Instead of using linear models, why not use a more complex model for which E 0 is zero? Then at least asymptotically, the training error will approach s 2 (provided that the model is not too complex). However, for any Žnite N, we run the risk of overŽtting the data, perhaps even obtaining an estimate s 2 D 0. Perhaps there is a systematic way of choosing the model complexity as a function of N to get an optimal noise variance estimate. The next few sections show that such an exhaustive search will necessarily be fruitless unless we make some statement about f . In this case it would sufŽce to tell us “how good our model is at estimating f ” (i.e., E0 ). Rather than explore every other potential method for estimating s 2 in this linear scenario, we will set up a more general framework for the estimation of noise distributions and show that unless some assumptions are made about the target function, all noise models are just as probable, given the data. The intuition is that the data points are consistent with any noise level if we do not restrict the target function. We pursue these issues next. 3 Problem Set-Up

A Žnite data set D D fx a, ya glaD1 is given, where x a 2 R d , ya D f ( xa ) C na, f : R d ! R and the data set consists of l data points. 3 We concentrate on the case where the noise distribution is Žxed. The results can be modiŽed to accommodate noise distributions that vary with x. It will always be understood that a indexes points in the data set. We concentrate on compact spaces and assume that the input space is the unit hypercube in d dimensions. Let the output space be the bounded interval [¡C, C]. Discretize the input and output spaces as follows. Let the input variable, x , take on values in the set

X 3

D f0, D x , 2D x , . . . , 1 ¡ D x , 1gd ,

(3.1)

We use l rather than the customary N to avoid confusions with Nx , Ny , to be deŽned shortly.

552

Malik Magdon-Ismail

Figure 3: Discretized function space. The space has only a Žnite number of functions. An arbitrary precision can be obtained by making the grid Žner and Žner.

so there are Nx D 1 C D1 possible values for each input variable. Let the x output variable, y, take a value in the set © ª Y D ¡C, ¡C C D y , . . . , 0, D y , 2D y , . . . , C ¡ D y , C . There are Ny D

2C Dy

(3.2)

C 1 possible y values. A function ( f ) will be a mapping, d

f : X ! Y . There are C D Ny Nx different functions. By making D x , D y as small as we choose, we can achieve an arbitrary precision (for example, see Figure 3). In all applications, only a Žnite number of functions are available (computers are Žnite precision machines). Due to this Žniteness, we can deŽne a probability distribution on the function space—a prior over the function space. One natural prior is to take this distribution to be uniform. Therefore to every function we assign a probability of C1 . We assume that C can be made as large as we wish. Further, without loss of generality, we can scale the outputs by D1 ; therefore we can assume that y Q ¡CQ C 1, ¡CQ C 2, . . . , CQ ¡ 1, Cg, Q Ny D 2CQ C 1 and CQ D C / D y . With y 2 f¡C, this representation, the noise n 2 f0, § 1, § 2, . . .g has a distribution given by

No Free Lunch for Noise Prediction

553

P a vector P , where Pi D Pr[n D i] and i Pi D 1. Assume a prior probability measure on the possible noise distributions g( P ). We are interested in the Bayesian posterior on these noise distributions, Pr[D| P ]g( P ) , (3.3) Pr[D ] P where Pr[D ] D P Pr[D | P ]](P ). We can calculate Pr[D| P ] as follows: g( P |D) D

Pr[D | P , f ] D

l Y aD1

Pr[na D ya ¡ f ( x a )].

(3.4)

Note that the right-hand side implicitly depends on P through the probabilities for the noise realizations na . Multiplying by P[ f ] and summing over f , we have Pr[D| P ] D

l X Y 1 XX ¢¢¢ Pr[na | f , D ] C f ( x ) f (x ) f ( x ) aD1 1

l 1 Y

( a)

D

Ny l

2

Nx

C X

Pr[na | f , D ],

(3.5)

aD1 f ( xa )D¡C

where we have used the notation Pr[na| f, D ] to mean Pr[na D ya ¡ f ( x a )]; (a) follows when we sum over all points not in the data set, and we have used the fact that Pr[ f ] D 1 / C ¡ uniform prior. 4 No Free Lunch for Noise Prediction 4.1 Boolean Functions. We use the following notation: x 2 f0, 1gd

f : x ! f0, 1g (1 ¡ p) 2 [0, 1] D probability of ip. d

There are C D 22 possible functions. Let g( p) be the prior distribution for the noise parameter, p. If the prior distribution on the possible functions is uniform then g( p| D ) D g( p) . Theorem 1.

Let mi be the number of times fi agrees with D where we use i to index functions. Then Pr[D |p, fi ] D pmi (1 ¡ p) l¡mi , so Proof.

Pr[D , fi |p] D Pr[D | p, fi ]Pr[ fi | p] D

1 mi p (1 ¡ p) l¡mi . C

(4.1)

554

Malik Magdon-Ismail

Letr ( mi ) be the number of functions agreeing exactly mi times with the data ¡ ¢ d ¡ ¢ set D . r (mi ) D ml 22 ¡l D ml 2¡l C. Summing equation 4.1 over fi , we get i i X Pr[D | p] D Pr[D , fi | p] fi

D D

l 1 X pmi (1 ¡ p) l¡mir (mi ) C m D0 i 1 Pl mi (1 ¡ p) l¡mi mi D0 p 2l

¡

D2

¢

lmi

¡l

independent of p. Therefore, g( p| D ) D

Pr(D | p) g( p) D g( p). Pr( D )

4.2 Extension to More General Functions. We would like to extend the previous analysis to the case where the output is not binary and the input is the discretized version of [0, 1]d . Figure 4 illustrates the conclusions of this section on a particular discretized grid. This section is somewhat technical. Its essential content is contained in theorem 3, which formalizes the intuition that if every target function value is equally likely, then each noise realization for every data point is equally likely. Hence nothing can be deduced about the relative likelihoods of different noise realizations.

4.2.1 Cyclic Noise. When the noise is additive modulo Ny , then P D [P0 , P1 , . . . , P Ny ¡1 ] fully speciŽes the noise distribution. The binary case with probability of ip 1 ¡ p we have Ny D 2, P0 D p, P1 D 1 ¡ p Example.

Lemma 1.

P f ( xa )

Pr[na | f , D ] D 1 for all x a 2 D .

Proof. C X f ( xa )D¡C

Pr[na | f, D ] D

yX a ¡C na Dya CC

Pr[na | f , D ] D

N y ¡1 X na D0

Pna D 1.

Combining lemma 1 with equation 3.5, we have the following theorem: Let g( P ) be the prior distribution for the cyclic noise. If the prior distribution for the functions is uniform, then g( P | D) D g( P ). Theorem 2.

No Free Lunch for Noise Prediction

555

Figure 4: Illustration of no noisy free lunch (NNFL). Suppose that the two functions shown are both equally likely (probability 1/2) to be the function that created the data. Then the two hypotheses, noise D 0 and noise > 0, are still equally likely to be true if they were a priori equally likely to be true. Proof.

By lemma 1 and equation 3.5, Pr[D| P ] D

g( P |D) D

1 Ny l

. Therefore,

Pr[D| P ]g( P ) D g( P ). Pr[D ]

Note that the boolean case is a special case of the cyclic noise case. In the language of information theory (Cover & Thomas, 1991), the noise forms a symmetric channel between the function value and the output value. A uniform distribution for the function values induces a uniform distribution on the output values, independent of the details of the symmetric noisy channel. Hence, observing the output conveys no information about the noisy channel. This is essentially the content of theorem 2. 4.2.2 Thresholded Additive Noise. Noise is added to the target, and then the resultant value is thresholded, so, yi D

» minfC, f ( x i ) C ni g, f (x i ) C ni ¸ 0 maxf¡C, f (x i ) C ni g, f ( x i ) C ni < 0.

556

Malik Magdon-Ismail

P D [P0 , P§ 1 , P§ 2 , . . .]. Unlike the previous case, edge effects make Pr[D| P ] dependent on P . These effects can be made as small as we please by allowing

C to be as large as we please. The intuition is that the data set is Žnite (yi is bounded), and so, because the noise distribution must decay to zero, if C is large enough, the probability that f ( xa ) is close to the edge and ya so far away becomes negligible. This is formalized in the next lemma. Given 2 > 0, 9C¤ such that for all C ¸ C¤ ,

Lemma 2.

1 Ny

l

(1 ¡ 2 ) · Pr[D| P ] ·

1 Ny l

.

(4.2)

(Ny D 2C C 1 and intuitively speaking, 2

C!1

¡! 0).

P Because fPi g is summable, there exists g such that |i|> g Pi < 2 l . Let ymax D maxfyi g and ymin D minfyi g. Choose C¤ > maxf|ymax Cg|, | ymin ¡g|g and let C ¸ C¤ . We then have Proof.

C X f ( xa )D¡C

Pr[na | f, D ] D

yX a CC

Pr[na | f , D ]

na Dya ¡C

D 1¡

X

na < ya ¡C

2

Pr[na | f , D ] ¡

X

Pr[na | f, D ]

na > ya CC

¸ 1¡ , l where the last inequality follows because by construction ymax ¡ C < ¡g and ymin C C > g. Using equation 3.5, we have 1 Ny

l

¸ Pr[D| P ] ¸

1 ³ 2 ´l 1¡ . l l Ny

The lemma now follows by using the inequality (1 ¡ x) n ¸ 1 ¡ nx for 0 · x · 1. Thus, we see that by choosing C, the bound on our function f to be large enough, the noise distribution has almost no effect on the probability of obtaining a particular data set. All data sets become equally probable, in the limit of large C. This tells us that the data should not be telling us any more about the noise distribution than we already know. The prior distribution for the noise is unaffected given any Žnite data set. To formalize this intuition, we need to impose a technical condition on the prior over the noise distribution space.

No Free Lunch for Noise Prediction

557

Let 0 < 2 · 1. DeŽne g P (2 ) by

DeŽnition 1.

( gP (2 ) D min g:

X |i|> g

) Pi < 2

.

For all 0 < 2 · 1, g P (2 ) < 1, by the summability of P . Let 0 < 2 · 1. DeŽne g( N 2 ) by

DeŽnition 2.

g( N 2 ) D sup fgP (2 )g P

where the sup is taken over P 2 supportfg( P )g. We say that a prior on the space in which P is deŽned, g(P ) , is totally bounded if DeŽnition 3.

g( N2 )< 1 for all 2 > 0. Essentially this means that the support set for the prior on the noise distributions is not “too large.” For example, a prior over the noise distribution space that has support over only a Žnite number of points (i.e., that is nonzero only for a Žnite number of noise distributions) is totally bounded. We are now in a position to make precise the intuition mentioned above. Theorem 3 (No Noisy Free Lunch (NNFL)). Let the prior over noise distributions, g( P ) , be totally bounded. For all 0 < 2 < 1, 9C¤ > 0 such that for all

C ¸ C¤ ,

(1 ¡ 2 ) g(P ) · g( P |D) ·

g( P ) 1 ¡2

(4.3)

Therefore, lim g( P | D) D g(P )

C!1

because 2 ! 0 can be attained by letting C ! 1. Following lemma 2, now let C¤ > maxf| ymax C g( N 2 )| , | ymin ¡ g( N 2 )| g and let C ¸ C¤ . Then lemma 1 applies uniformly to every P 2 supportfg( P )g. Proof.

558

Malik Magdon-Ismail

Therefore equation 4.2 holds for every P 2 supportfg( P )g. Pr[D| P ]g(P ) Pr[D ] Pr[D| P ]g( P ) D P . P Pr[D| P ]g( P )

g(P | D) D

(4.4)

Using equation 4.2, an upper bound can be obtained by substituting Pr[D| P ] with 1 / Ny l in the numerator and (1 ¡ 2 ) / Ny l in the denominator. To get a P lower bound, we do the opposite. Then, using the fact that P g( P ) D 1, we get equation 4.3. Therefore, having no bound on f and allowing all f ’s to be equally likely does not allow anything new to be deduced about the noise distribution given any Žnite data set. It should also be clear how to extend these results to the case where the noise distribution has x dependence. One looks at each point independently. The nonthresholded additive noise model is asymptotically obtained by letting C become arbitrarily large. Further, the results apply to arbitrary D x , D y , so by allowing D x , D y ! 0, these results can be applied to the nondiscretized model in this asymptotic sense. 5 Discussion

The posterior g(P | D) is given by g(P | D) / g( P )

X

Pr[D | P , f ]P[ f ],

(5.1)

f

where P[ f ] is the prior over target functions. We have demonstrated that when P[ f ] is “uniform,” it is not possible to update a prior on the noise distribution from a Žnite data set, provided that a certain technical condition is satisŽed. A nontrivial P[ f ] is needed for g(P | D) to be different from g(P ). Exactly how P[ f ] will factor into g( P |D) is given by equation 5.1. The result is more useful than it appears at Žrst sight, for one might argue that no one ever has a uniform prior. A restatement of the result is: Given two noise distributions, P 1 and P 2 , there are just as many (nonuniform) priors that favor P 1 over P 2 (when one weights by the amount by which P 1 is favored) as vice versa. This result Žts into a set of NFL theorems that might be considered a collection of negative results, offering no hope for learning theory. The situation is considerably more positive, however. One is never in a situation with a uniform P[ f ]. Further, it is rare that no information about P[ f ] is known (excepting that it is nonuniform). Therefore, incorporating P[ f ] into

No Free Lunch for Noise Prediction

559

the noise prediction or learning algorithm is necessary in order to claim superior performance. Distribution-independent bounds such as the VC bounds (Vapnik, 1995) say that with high probability, the training error is close to the test error given some Žxed target function f . But VC results do not guarantee that the training error will be low with high probability; therefore they are of little practical use (for a deeper discussion of the subtleties underlying the connection between NFL and VC theory, see Wolpert (1995, 1996b). 4 How could one guarantee that a training error close to zero is achievable? One has to use the prior information available on the target function. Along a similar vein, results that exemplify the superiority of algorithm A over algorithm B on a certain benchmark problem are also of little use, for the reverse will be true on some other “benchmark” problem, and we never know which “benchmark” problem we will run into. Similarly, looking at a data set alone, one cannot produce any reliable estimate for the noise variance. Perhaps we “feel” as though we ought to be able to (for how often can it be heard: “This data set is clearly noisy”), but only because we already have a smoothness prior built into our neural network. Why? Because the types of functions we tend to encounter in practice are smooth. NFL focuses our attention on the importance of the prior information available. One should not attempt to Žnd learning systems that are universally good. Instead, one should study the possibilities that various priors present. Unfortunately, identifying and justifying the prior for the problem at hand is already a daunting task, let alone incorporating it into the learning problem. One way would be through the use of hints (Abu-Mostafa, 1995; Sill, 1998; Sill & Abu-Mostafa, 1997). A possible starting point would be to identify priors, P[ f ], with learning systems that perform well with respect to them (for example, Barber & Williams, 1997; Zhu & Rohwer, 1996). One might then be able to relate real problems to these priors and thus choose an appropriate learning system, or at least it might become possible to make precise the assumptions behind a belief that a certain learning system is appropriate for a certain problem. A useful result would be of the form, “Algorithm A [or learning system A] performs better than average on problems drawn from the (useful) class B.” Appendix

O an A.1 Estimating Noise Variance Using Linear Models. We have A, estimate of s 2 ( d C 1) C B, and we would like to estimate B by bootstrap4 VC results make statements about Pr[training error| test error] by making the statement that the probability of a low training error given a high test error is extremely unlikely. In practice, what we would really like to make statements about is the Pr[test error|training error] for which we really need to make some statement about the a priori probablity of a certain test error; that is, we need to make some statement about a prior over target functions.

560

Malik Magdon-Ismail

ping. Suppose we draw NB data points and estimate B (using the bootstrap O It can be shown that (see theorem 5) technique) by B.

¡

O n D BQ C s 2 ( d C 1) 1 ¡ NB hBi N

¢

C

s2 trhS ¡1 V S ¡1 V i, {z } N |

(A.1)

O( dC1)

where BQ is an unbiased estimate of B and the expectation in the last term is with respect to the bootstrapping. The deŽnitions of V and S are given in the next section and are unimportant for our purpose here. Thus, we now O an estimate of B C s 2 ( d C 1), and B, O an estimate of have two quantities: A, 2 ¡1 ¡1 (( ) ). B C s d C 1)(1 ¡ NB / N C trhS V S V i / N Solving for s 2 , we have s2 D

(d

AO ¡ BO

C 1) NNB

¡

trhS ¡1 V S ¡1 V i N

.

(A.2)

However, we are bootstrapping a quantity B, which is related to the deviation of a statistical quantity of the sample from its population value. This estimate is good only as long as N represents the population—that is, NB / N ! 0. In this case, BO D B C s 2 ( d C 1) and thus we cannot combine this with AO to solve for s 2 (the expression in equation A.2 becomes indeterminate). We are faced with a dilemma. For Žnite NB / N we can get an estimate of s 2 , but this estimate is not very good. However, we have no way to make it better because by making the bootstrap more effective for B, the Žnal result cannot be solved for s 2 . Technically speaking, we cannot in this way get a consistent estimate of s 2 , even as N ! 1. A.2 Proof of Theorems. We use the notation X D [ x 1 « x2 ¬ . . . S ´ xxT x ,

xN ] , y D f Cn q D hx f (x )ix .

X is a (dC1)£N matrix where the x i are the data points (we use the convention

that the Žrst component is set to 1 and the remaining d components are the coordinates of the data point). f is an N £ 1 vector of output values at the data points, and n is a vector of noise realizations. If hx i D 0, then S is the covariance matrix for the input distribution. The law of large numbers gives us that XX T / N ¡! S and Xf / N ¡! q , where we assume that the N!1

N!1

conditions for this to happen are satisŽed. We thus deŽne V and a by XX T

N

V ( X) DS C p , N

Xf

N

a (X ) DqC p . N

Note that hV iX D ha iX D 0 and Var( V ) and Var(a ) are O(1).

(A.3)

No Free Lunch for Noise Prediction

561

Let the learning model be the set of linear functions w ¢ x and let the learning algorithm be minimization of the squared error. Then Theorem 4.

hEtr¤ i

D E0 C s 2 ¡

,n

¡ ¢

s 2 ( d C 1) C B 1 CO 3 N N2

,

(A.4)

where E 0 and B are constants given by E0 D h f 2 i ¡ q T S ¡1 q BD Proof.

(A.5)

hq T S ¡1 V S ¡1 V S ¡1 q C a T S ¡1 a ¡ 2a T S ¡1 V S ¡1 q i . N

(A.6)

The least squares estimate of w is given by

w O D ( XX T ) ¡1 Xy ,

(A.7)

from which we calculate the expected training error as ½ hEtr i D D

¾ ½ ¾ 1 T 1 (X w ((X T ( XX T ) ¡1 X ¡ 1) y ) 2 O ¡ y) 2 D N N « T ¬ « T ¬ T ¡1 f f ¡ f X ( XX ) Xf C hn T n i ¡ hn T X T ( XX T ) ¡1 Xn i

N T X T ( XX T ) ¡1 Xf i h f s 2 ( d C 1) D h f 2i ¡ Cs 2 ¡ . N | {z N } T1

Using equation A.3 and the identity [1 Cl A ]¡1 D 1 ¡ l A Cl 2 A 2 C O(l 3 ), we evaluate T1 as T1 D h f 2 i ¡ q T S ¡1 q ¡

¡ ¢

hq T S ¡1 V S ¡1 V S ¡1 q C a T S ¡1 a ¡ 2aT S ¡1 V S ¡1 q i 1 CO 3 N N2

.

The theorem now follows. This result is similar to results obtained by Amari, Murata, Muller, ¨ and Yang (1997) and Moody (1991). From theorem 4, we have that 0

1

1 B T ¡1 ¡1 ¡1 ¡1 ¡1 ¡1 C BD @hq S V S V S q i ChaT S a i ¡2 ha T S V S q iA . {z } | {z } | {z } N | T1

T2

T3

(A.8)

562

Malik Magdon-Ismail

We wish to estimate T1 , T2 , T3 . Suppose that we try to bootstrap them as follows. For T 1 , we take qO D Xy / N D X ( f C n ) / N and SO Dp( XX T / N ). We estimate V by sampling NB of the N data points and compute NB ((X B X TB / NB )¡ (XX T / N )). We then estimate T1 by TO 1 , a bootstrapped expectation of qO T S O¡1 V S O¡1 V S O¡1 qO . This requires NB / N to be small. Doing all this, we Žnd that E Xf E fT X T D O¡1 s 2 D O¡1 S V S O¡1 V S O¡1 C TO 1 D tr S V S O¡1 V , N N N

(A.9)

where we have an expectation with respect to the noise and the remaining expectation is with respect to the bootstrapping. Notice that the Žrst term is the unbiased estimator of T1 . Denote this term by TQ 1 . p For TO 2 D ha T S ¡1 a i we use a D NB (X B y B / NB ¡ qO ). Partitioning the input matrix and noise vector into the part in the sampled NB points and the remaining part, write X D [X B XQ B ] and n D [n TB nQ TB ]T . Noting that n B and nQ B are independent, we have for TO 2 : 1 (NB Xn ¡ N X B n B )S ¡1 ( NB Xn ¡ N X B n B ) N2 NB " # " # 2( ¡ Q B nQ B Q B nB Q B nQ B Q B nB )2 N N N X X X X B ¡1 B D TQ2 C ¡ S ¡ N 2 NB N ¡ NB NB N ¡ NB NB

TO 2 D TQ2 C

¡

NB D TQ2 C s 2 (d C 1) 1 ¡ N

¢ ¡ 1¢ Co

N

,

(A.10)

where we have taken the expectation with respect to the noise and TQ 2 is an unbiased estimate of T2 . Performing the same kind of analysis for TO 3 , we Žnd that TO 3 D TQ 3 , an unbiased estimate of T3 . Now we estimate BO by TO 1 C TO 2 ¡ 2TO 3 , which is the unbiased estimate of B plus some correction terms. Therefore we have proved the following theorem. For N ! 1, NB / N ! 0, the bootstrapped estimate of B, which O has an expectation given by we call B, Theorem 5.

O n D BQ C hBi

s 2 ( d C 1) N

¡

1¡

NB N

¢

C

s2 trhS ¡1 V S ¡1 V iX , N

(A.11)

where BQ is an unbiased estimate of B and the remaining expectation is with respect to the bootstrapping.

No Free Lunch for Noise Prediction

563

Acknowledgments

We thank Yaser Abu-Mostafa and the members of the Caltech Learning Systems Group for helpful comments. In addition, two anonymous referees provided useful comments.

References Abu-Mostafa, Y. (1995). Hints. Neural Computation, 4(7), 639–671. Amari, S., Murata, N., Muller, ¨ K., & Yang, H. (1997).Asymptotic statistical theory of overtraining and cross validation. IEEE Transactions on Neural Networks, 8(5), 985–996. Barber, D., & Williams, C. K. I. (1997). Gaussian processes for Bayesian classiŽcations via hybrid Monte Carlo. In M. C. Mozer, M. I. Jordan, & T. Petsche (Eds.), Advances in neural information processing systems, 9 (pp. 340–346). San Mateo, CA: Morgan Kaufmann. Cataltepe, Z., Abu-Mostafa, Y. S., & Magdon-Ismail, M. (1999).No free lunch for early stopping. Neural Computation, 11, 995–1001. Cortes, C., Jackel, L. D., & Chiang, W.-P. (1994). Limits on learning machine accuracy imposed by data quality. In G. Tesauro, D. S. Touretzky, & T. K. Leen (Eds.), Neural information processing systems, 7 (pp. 239–246). Cambridge, MA: MIT Press. Cover, T. M., & Thomas, J. A. (1991). Elements of information theory. New York: Wiley. Moody, J. E. (1991). The effective number of parameters: An analysis of generalization and regularization in nonlinear learning systems. In J. E. Moody, S. J. Hanson, & R. P. Lippmann (Eds.), Advances in neural information procesing systems, 4 (pp. 847–854). San Mateo, CA: Morgan Kaufmann. Shao, J., & Tu, D. (1996).The jackknife and the bootstrap. New York: Springer-Verlag. Sill, J. (1998). Monotonic networks. In M. Kearns, M. Jordan, & S. Solla (Eds.), Advances in neural information processing systems, 10. Cambridge, MA: MIT Press. Sill, J., & Abu-Mostafa, Y. S. (1997). Monotonicity hints. In M. C. Mozer, M. I. Jordan, & T. Petsche (Eds.), Advances in neural information processing systems, 9 (pp. 634–640). Cambridge, MA: MIT Press. Vapnik, V. N. (1995). The nature of statistical learning theory. New York: SpringerVerlag. Wolpert, D. H. (1995). The mathematics of generalization. Reading, MA: Addison Wesley. Wolpert, D. H. (1996a). The existence of a priori distinctions between learning algorithms. Neural Computation, 8(7), 1391–1420. Wolpert, D. H. (1996b). The lack of a priori distinctions between learning algorithms. Neural Computation, 8(7), 1341–1390. Wolpert, D. H., & Macready, W. G. (1997). No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation, 1, 67–82.

564

Malik Magdon-Ismail

Zhu, H. (1996). No free lunch for cross validation. Neural Computation, 8(7), 1421–1426. Zhu, H., & Rohwer, R. (1996). Bayesian regression Žlters and the issue of priors. Neural Computation Applications, 4, 130–142. Received May 29, 1998; accepted March 31, 1999.

LETTER

Communicated by Peter Foldiak

Self-Organization of Symmetry Networks: Transformation Invariance from the Spontaneous Symmetry-Breaking Mechanism Chris J. S. Webber Defence Evaluation and Research Agency, Malvern WR14 3PS, U.K. Symmetry networks use permutation symmetries among synaptic weights to achieve transformation-invariant response. This article proposes a generic mechanism by which such symmetries can develop during unsupervised adaptation: it is shown analytically that spontaneous symmetry breaking can result in the discovery of unknown invariances of the data’s probability distribution. It is proposed that a role of sparse coding is to facilitate the discovery of statistical invariances by this mechanism. It is demonstrated that the statistical dependences that exist between simplecell-like threshold feature detectors, when exposed to temporally uncorrelated natural image data, can drive the development of complex-cell-like invariances, via single-cell Hebbian adaptation. A single learning rule can generate both simple-cell-like and complex-cell-like receptive Želds. 1 Introduction

Symmetry networks (Shawe-Taylor, 1989) are neural networks in which some synaptic weights are equal to others, to some degree of approximation. In other words, the weight vectors are invariant under certain permutations among the weights. For example, the simple permutation that swaps over the ith and i0 th components of weight vector w will leave w invariant if the weights wi and wi0 are equal. These symmetries have the consequence that the scalar product with an input vector x will be invariant under corresponding permutations of the input nodes: 0

Sii...... ( w ) D w

)

0

w ¢ Sii...... (x ) D w ¢ x .

(1.1)

This follows because w ¢ S( x) D S¡1 (w ) ¢ x and because S( w ) D w implies S¡1 ( w ) D w . This is true for any permutation, including those that cycle more than two weights and those that cycle several distinct equivalence classes of weights simultaneously. Because the length | w | of a vector is invariant under a permutation, invariance of w ¢ x implies invariance of the distance | x ¡ w | , so that Sii...... ( w ) D w 0

)

­0 ­ ­ ­ Sii...... ( x ) ¡ w ­D | x ¡ w |. ­

Neural Computation 12, 565–596 (2000)

(1.2)

c British Crown copyright 2000 °

566

Chris J. S. Webber

Any neuron whose response is a function of w ¢ x or | x ¡ w | only will respond invariantly under those input-node permutations that correspond to its weight vector ’s symmetries. The converse of equation 1.1 implies that in general, such a neuron will discriminate between a pair of permutationrelated patterns if its weight vector is not symmetric under that particular permutation. Depending on the weight vector ’s speciŽc symmetries, the response will be invariant for some input-node permutations but not for others; the neuron’s invariances are selective for speciŽc groups of transformations. A neuron with a high-dimensional weight vector can be invariant for many distinct permutations, while retaining the capacity to discriminate for very many other permutations. These arguments are also valid for the case of approximate symmetries, if the neural response is a continuous function of w ¢ x or | x ¡ w | , because S(w ) ¼ w implies w ¢ S(x ) ¼ w ¢ x . Actually the arguments based on equations 1.1 and 1.2 apply to a more general group of linear transformations than the permutations: the continuous orthogonal1 group of proper and improper rotations in the weight-vector space, of which the coordinate permutations form a discrete subgroup. (The orthogonal transformations are those linear transformations that leave vector lengths invariant: |S( w )| D | w |.) The condition for w ¢ x and | x ¡ w | to be invariant under such a transformation is that the weight vector lies in the invariant subspace of the transformation, which is deŽned to be the subspace of vectors w for which S(w ) D w . The invariant subspace is the multidimensional analog of a mirror plane or axis of rotation. Minsky and Papert (1969) originated the idea that invariant response can be related to weight symmetries in this way: their group-invariance theorem proves that if one can construct a perceptron for computing a given symmetric function, then one can also construct a perceptron with symmetric weights for computing the same function. Many results on the subject of transformation invariance have depended on the design of networks having equalities among weights. Such networks have been successfully applied to the problem of graph isomorphism (Shawe-Taylor, 1993). The rotationinvariant network of Fukumi, Omatu, Takeda, and Kosaka (1992), the handwritten zip code recogniser of Le Cun et al. (1989), and the neocognitron (Fukushima & Miyake, 1982) all implicitly use built-in weight equalities to achieve their transformation invariances. Instead of imposing prior knowledge of a desired invariance directly onto the network architecture in this way, one can incorporate that prior knowledge into the learning algorithm, in the form of prior constraints on the learning equations. The use of weight sharing to distinguish T and C shapes independently of translation and 90 degree rotation (Rumelhart, Hinton, and Williams, 1986) provides one ex-

1 Results obtained for orthogonal transformations can be generalized straightforwardly to apply to unitary transformations if we were to consider complex-valued weights and data.

Self-Organization of Symmetry Networks

567

ample. Rao and Ballard (1998) and Wood (1995) provide others. Supervised networks have even been explicitly trained to be invariant under inŽnitesimal transformations of a known invariance group (Hinton, 1987; Simard, Victorri, Le Cun, & Denker, 1991). This article addresses the question of what kind of intrinsic invariances of the data might be able to drive the development of weight symmetries during unsupervised adaptation, and proposes a generic mechanism by which speciŽc invariances of the data can drive the development of transformation-invariant response. In principle, this generic mechanism could be used to generate invariances with respect to a wide range of transformations, in a range of learning rules. As a speciŽc example, the Hebbian rule of Webber (1994) is studied and is shown analytically to give rise to weight symmetries when trained on data having statistical invariances under permutations. Webber (1994) used an analytical model, and simulations involving synthetic images, to demonstrate that this single-cell learning rule can discover independent components in kurtotically distributed data. The component-Žnding behavior of this learning rule has also been demonstrated using natural data: the rule discovers phones and words when exposed to continuous speech (Webber, 1997). Blais, Intrator, Shouval, and Cooper (1998) have shown that various other single-cell rules also belonging to the general class of projection pursuit algorithms (Friedman, 1987) can discover independent components in natural images, which resemble the oriented receptive Želds of cortical simple cells and appear localized to some extent. Such single-cell learning rules are very different from multicell independent component analysis algorithms such as those of Olshausen and Field (1996) and Bell and Sejnowski (1997), in that they are not derived from any kind of minimum mutual entropy or statistical independence principle, such as was originally proposed by Barlow (1989). In this article the Webber (1994) rule is also trained on natural images, and it generates simple-cell-like feature detectors having oriented and highly localized receptive Želds as expected. The patterns of output activity induced in a set of these “simple cells,” when natural images are presented at their inputs, are then used to train other similar, higherlevel Webber (1994) neurons. These higher-level neurons develop weight symmetries that give them complex-cell-like translation invariances. Thus, key response properties of both simple and complex cells can be derived within the framework of the Hebbian single-cell learning rule, without the need to rely on temporal correlations to explain transformation invariance, as others have done (Foldiak, 1991). 2 The Spontaneous Symmetry-Breaking Mechanism

If weight equalities are not explicitly constructed by prior constraints, how can they arise through self-organization? What properties of the data, unknown prior to learning, can determine the permutations under which the

568

Chris J. S. Webber

weight vector is to become invariant? In other kinds of dynamical system, the symmetry of the convergent state is often determined by spontaneous breaking of dynamical symmetries. Examples are widespread in physical systems. This article proposes that spontaneous symmetry-breaking mechanisms can also determine the self-organization of invariant neural response. The term dynamical symmetry refers to an invariance of the equations of motion, under a transformation of the system’s dynamical coordinates. In the neural network context, these coordinates are a set of adaptive weight vectors W ´ ( w 1 , . . . , w m ), and the equations of motion are the adaptation rules, dw j D vj ( w 1 , . . . , w m ) dt

( j D 1, . . . , m),

(2.1)

or, deŽning V ´ ( v 1 , . . . , v m ), dW D V ( W ). dt

(2.2)

Equations of motion that are Žrst order in t include most learning rules, notable exceptions being rules with second-order “momentum” terms. The velocity functions vj will depend in some way on the distribution of the training data Pr(x ). The class of rules of the form 2.2 includes the familiar rules in which the weight vector is updated smoothly on the basis of some average h. . .ifx g over Pr(x ): Z vj ( w 1 , . . . , w m ) D Àj ( x, w 1 , . . . , w m ) Pr( x ) dx ( j D 1, . . . , m).

(2.3)

In these ensemble-average rules, the order in which data patterns are drawn from Pr( x) has no inuence on the learning dynamics: it is assumed that the statistics of Pr( x ) are stationary over time. The velocities vj are usually derived from a Lyapunov function U, called the objective function in the neural network context, Z (2.4) U ( w 1 , . . . , w m ) D u( x, w 1 , . . . , w m ) Pr( x) dx , often via gradient ascent or descent: vj ( w 1 , . . . , w m )

§

@U ( w 1 , . . . , w m ) @w j

( j D 1, . . . , m).

(2.5)

Consider a linear transformation of the coordinate system from old coordinates W to new coordinates W 0 , W 0 ´ S( W ).

(2.6)

Self-Organization of Symmetry Networks

569

In the new coordinates, the equations of motion (equation 2.2) become dS¡1 ( W 0 ) D V (S¡1 ( W 0 )) dt

i.e.

dW 0 D S( V (S¡1 ( W 0 ))), dt

(2.7)

because linearity of the transformation implies S¡1 ( W 0o C dW 0 ) ¡ S¡1 ( W 0o ) dS¡1 ( W 0 ) D lim D S¡1 dt dt!0 dt

¡ dW ¢ 0

dt

.

(2.8)

Note that although we have assumed here that the transformation is linear, we do not have to assume that the dynamics are linear—that V ( W ) is a linear function of W . The transformation S on the dynamical coordinates W is said to be a symmetry of the equations of motion if the equations have the same functional form, dW 0 D V ( W 0 ), dt

(2.9)

in the new coordinate system. For linear transformations, such dynamical symmetry is deŽned by V ( W 0 ) D S(V (S¡1 (W 0 )))

i.e.

V (S(W )) D S(V ( W )),

(2.10)

as can be seen by comparing equation 2.7 with 2.9. In cases where the equations of motion just perform gradient ascent or descent on an objective function, any orthogonal transformation that leaves the objective function invariant, U( W 0 ) D U( W ),

(2.11)

will also be a symmetry (see equation 2.10) of the equations of motion, because the gradient in the transformed coordinate system is (using the chain rule and the deŽnition of orthogonality 2 ) @U @W T @U @U @U D D (S¡1 ) T DS . @W 0 @W 0 @W @W @W

(2.12)

However, the concept and deŽnition (see equation 2.10) of symmetries of the equations of motion apply even to dynamics for which no objective function U exists. If the equations of motion are invariant under some group of transformations, this has implications for the invariances of the weight vectors when the 2

S¡1 D ST for orthogonal matrices.

570

Chris J. S. Webber

learning dynamics have brought them to convergence. From equation 2.2, it can be seen that the Žrst of two conditions for a conŽguration of weight 1) vectors to be a convergent state W 1 ´ (W 1 1 , . . . , W m is that it should be a stationary point, V ( W 1 ) D 0,

(2.13)

even for dynamics for which no Lyapunov function exists. This condition is satisŽed for unstable stationary states as well as for stable or metastable convergent states, so, in addition, convergent states must satisfy a second condition,

²T ( @V T / @W ) ² · 0,

(2.14)

for any small perturbation W D W 1 C ² away from the stationary state. This says that the “restoring force” V ( W ) should oppose3 perturbations ² away from the stationary point, so ² ¢ V (W ) should not be positive for any ²: equation 2.14 can be derived by Taylor expansion of V (W ) about W 1 in the standard way. In the special case of gradient ascent, the matrix @V T / @W is the objective function’s Hessian matrix, and equation 2.14 is just the condition for the stationary point deŽned by equation 2.13 to be a local maximum of the objective function. If S( W ) is an orthogonal symmetry of the equations of motion (equation 2.2) and if the particular state W 1 is one of the stable convergent states satisfying equations 2.15 and 2.16, then the symmetry deŽnition (equation 2.10) ensures that the state S(W 1 ) will also be one of the stable convergent states satisfying equations 2.13 and 2.14. Clearly S( W 1 ) will also be a stationary point because V(W1) D 0

implies

S( V ( W 1 )) D S( 0 ) D 0 ,

(2.15)

and, by symmetry of the objective function under S, S( W 1 ) will obviously be a local maximum of U if W 1 is. Stability (see equation 2.14) of W 1 implies stability of S( W 1 ) even when no objective function U exists. If S is an orthogonal symmetry (see equation 2.10) of the equations of motion, a proof analogous to equation 2.12 shows that T @V T ( W 0 ) ¡1 @V ( W ) D S S. @W 0 @W

(2.16)

The matrices @V T ( W 0 ) / @W 0 and @V T ( W ) / @W are thus related by a similarity transformation.4 For every possible perturbation ²0 in the W 0 coordinate system, there is a corresponding direction ² D S( ²0 ) in the W coordinate 3 4

Or not further enhance it, in the case of metastability. It can be shown that the stability condition, equation 2.14, is equivalent to the re-

Self-Organization of Symmetry Networks

571

system for which the left-hand side of equation 2.14 has identical value (and vice versa),

²0 T

@V T ( W 0 ) 0 @V T ( W ) ² D ²T ², 0 @W @W

(2.17)

so the condition (see equation 2.14) that these values are nonpositive for every possible perturbation is independent of the coordinate transformation: S(W 1 ) must be stable if W 1 is stable or metastable if W 1 is metastable. Orthogonal symmetries of the equations of motion map convergent states onto convergent states. Two possibilities arise when the equations of motion are invariant under an orthogonal transformation S( W ). Either convergent states W 1 and S(W 1 ) are distinct from one another, in which case S( W 1 ) 6 D W 1 and S( W ) is said to be a spontaneously broken, or hidden, symmetry, or else they represent identical states, in which case S( W 1 ) D W 1 ,

(2.18)

and S( W ) is said to be a preserved, or manifest, symmetry. The preserved symmetries always form a subgroup of the equation of motion’s symmetry group, because they are elements of the equation of motion’s symmetry group and because symmetries form groups by deŽnition. It is not helpful to think of spontaneous symmetry breaking as a temporal process that begins at the initial state and ends with convergence, and to regard it as being caused by asymmetry in the initial state. It simply describes the situation in which the objective function as a whole is symmetric under some transformation group, but in which the optima of the objective function are mapped onto one another, rather than onto themselves, by the action of that group. It is important to note that spontaneous symmetry breaking is not caused by slight asymmetries in the equations of motion. There are cases in physics where the dominant component of the Lyapunov function may be symmetric but some perturbation effect introduces a small asymmetry into the Lyapunov function, with the result that its optima no longer map onto themselves exactly. This situation is also called symmetry breaking, but the symmetry is then not hidden and the breaking is not called spontaneous. This article is concerned exclusively with spontaneous symmetry breaking. In the neural network context, whether a particular symmetry of the equations of motion is preserved or broken will depend on the distribution quirement that @V T / @W has no positive eigenvalues. Matrices H and H0 ´ S¡1 H S related by a similarity transformation have the same set of eigenvalues: H e D c e implies H0 S¡1 e D c S¡1 e. The condition that @V T ( W ) / @W has no positive eigenvalues is therefore independent of the transformation S( W ) if S( W ) is an orthogonal symmetry of the equations of motion.

572

Chris J. S. Webber

Pr( x ) and on the particular functional form of the Àj in equation 2.3 and on any Žxed symmetry-breaking parameters implicit in the functions Àj . In other words, whether a symmetry is preserved or broken depends on the particular learning rule; there is no guarantee that an arbitrary learning rule will preserve any symmetries, except the trivial identity transformation. The values of any symmetry-breaking parameters will control how large a subgroup of the dynamical symmetries is preserved; section 5 provides a simple example. Under the action of a learning rule that preserves a symmetry, the weight-vector conŽguration will become symmetric (see equation 2.18) on convergence, even though the learning starts from a random and asymmetric initial conŽguration in general. 3 Symmetry Preservation and Invariant Neural Response

The principles proved in the previous section will be familiar to those who have come across the spontaneous symmetry-breaking mechanism in physics. Although these principles have not elsewhere been set down and worked through explicitly for the neural network context, other authors have made references to symmetry breaking in neural networks. For example, Yamagishi (1994) refers to the breaking of left-right ocular symmetry in the visual cortex, which he suggests takes place when dominance columns are formed. However, other authors have not considered the signiŽcance of the preservation of symmetries for transformation-invariant neural response. Considering this leads us to new results, which connect the subjects of symmetry networks and unsupervised learning quite intimately. Because we are interested in the response invariances of individual neurons in symmetry networks, we concentrate on transformations that permute the coordinates of individual weight vectors, operating in the same way on all the weight vectors: ( w 1 , . . . , w m ) ¡! (S( w 1 ), . . . , S( w m )).

(3.1)

Where the transformation S permutes all the weight vectors in the same way, the general symmetry deŽnition, equation 2.10, simpliŽes to vj (S(w 1 ), . . . , S( w m )) D S( vj ( w 1 , . . . , w m ))

( j D 1, . . . , m).

(3.2)

S( W ) will be an orthogonal transformation in the W -space if S( w ) is an orthogonal transformation on all the individual weight vectors. 5 If S(W ) D (S( w 1 ), . . . , S( w m )) is a preserved permutation symmetry, the symmetry-network conditions, 1.1 and 1.2, for the neurons’ responses to be invariant under S( x ) are automatically satisŽed. If S( W ) D (S( w 1 ), . . . , 5

Because the mn £ mn S-matrix that transforms the W -space is assembled from m repetitions of the n £ n S-matrix that transforms each w -space, in block-diagonal form.

Self-Organization of Symmetry Networks

573

S(w m )) is a spontaneously broken permutation symmetry, however, neurons will instead discriminate between data patterns related by the transformation S(x ). (These results are valid not just for permutations but hold for all orthogonal transformations in the weight-vector space.) If the dynamics are invariant under a whole group of symmetries, some of them may be preserved while others are spontaneously broken, so a neuron may have invariant response under some transformations but discriminate for others. We observe that symmetry preservation provides a mechanism for neurons to acquire invariant response through self-organization, and that symmetry breaking provides a mechanism for the response invariances to be restricted to a speciŽc subgroup of transformations. 4 Invariances of the Data Determine the Dynamical Symmetries

Whether a given permutation is actually a symmetry of the equations of motion in the Žrst place depends on Pr( x ). Consider the familiar ensembleaverage learning rules in which u is a continuous function of m scalar products wj ¢ x , m distances | w j ¡ x| , and / or m lengths | w j | , and for which the m neurons see the same input space x and so belong to the same layer of the network. S( w j ) ¢ S( x) D wj ¢ x

and

|S( wj ) ¡ S( x )| D | wj ¡ x |,

(4.1)

0

for any permutation Sii...... , so the objective function, equation 2.4, will be symmetric, U(S(w 1 ), . . . , S(w m )) D U( w 1 , . . . , w m ),

(4.2)

under any permutation that satisŽes Pr(S( x )) D Pr( x )

(4.3)

because Z

Z u( x , S( W )) Pr( x ) dx D D

u(S¡1 ( x ), W ) Pr( x ) dx Z u(x , W ) Pr(S( x )) dx .

(4.4)

For many ensemble-average learning rules (see equation 2.3), we can derive analogous results even if no objective function exists. Consider the learning rules of the form

Àj ( x , w 1 , . . . , w m ) D vj x ¡ w j wj

( j D 1, . . . , m),

(4.5)

574

Chris J. S. Webber

where each of the 2m scalars vj and wj is a continuous but not necessarily linear function of the m scalar products w j ¢ x , the m distances | wj ¡ x| , and / or the m lengths | w j |. Their dynamics will be symmetric (see equation 2.10) under any permutation that satisŽes equation 4.3 because all the vj and w j are invariants (see equation 4.1) under combined S( W ) and S( x ), so that Z Z ( )) ) Àj x , S( W Pr( x dx D S( Àj (S¡1 ( x), W )) Pr( x ) dx Z D S( Àj ( x , W )) Pr(S( x )) dx. (4.6) These new results state that any permutation that is an invariance of the data’s distribution will therefore be a dynamical symmetry of these learning dynamics. Those permutations that are invariances of the data’s distribution will therefore give rise to corresponding weight equalities if they are preserved unbroken in the convergent state. We therefore propose that such invariances of the data, unknown prior to learning, can drive the self-organization of transformation invariances in the response of individual neurons, through symmetry preservation. (Again, all the arguments of this section hold for the wider group of orthogonal6 transformations.) The theory of spontaneous symmetry breaking applies to real physical systems, which are only ever deemed symmetric to a Žnite approximation. For u( x , w 1 , . . . , w m ) that are continuous functions, approximate symmetries of Pr( x) will give rise to approximate symmetries of the objective function U (W ). Any Pr(x ) can be decomposed into a sum of symmetric and antisymmetric components 7 under a given permutation S(x ), and any U( W ) can be decomposed under the corresponding S( W ). As one perturbs Pr( x) away from exact symmetry by introducing an inŽnitesimal antisymmetric component, U (W ) will acquire an inŽnitesimal antisymmetric component. As this component grows, any symmetry-preserving stable optima W 1 will be perturbed away from the invariant subspace deŽned by S( W ) D W , remaining approximately invariant under S but to an increasingly inaccurate degree. 8 Thus, approximate symmetries of Pr( x) can give rise to approximate weight symmetries in the convergent state. The example of section 6 demonstrates that approximate invariances of the data’s distribution can indeed drive the development of approximate symmetries among synaptic weights. 6 The arguments could also be generalized straightforwardly to unitary transformations if we were to consider complex-valued weights and data. 7 The decomposition Pr( x ) D Pr C ( x ) C Pr¡ ( x ) is merely an analytical device. The ) are not themselves probability densities and are not required to satisfy functions Pr§ ( xR Pr§ ( x) ¸ 0 and Pr§ ( x ) dx D 1. 8 A large enough antisymmetric component may eventually break the approximate symmetry of these stable optima, when they abruptly cease to be stable, or, alternatively, it may move them so far from the invariant subspace that their approximate symmetry becomes washed out.

Self-Organization of Symmetry Networks

575

5 Hebbian Adaptation of Threshold Neurons as an Example

So far, only a few properties of learning rules have been assumed, but remarkably general predictions have been made. To demonstrate that spontaneous symmetry breaking gives rise to self-organized symmetry in practice, a speciŽc learning rule is cited. Webber (1994) studied the Hebbian adaptation of a scalar product neuron whose continuous threshold nonlinearity approximates the limiting case µ a¡# lim r( a) D 0 s!0

if a ¸ # if a · #,

µ dr 1 lim D 0 s!0 da

if a ¸ # if a · #,

(5.1)

where a ´ w ¢ x is the scalar product. s represents a “softness” parameter, which determines how closely the threshold nonlinearity approximates this sharp piece-wise linear limit in practice: a convenient soft parameterization (Webber, 1997, 1998) is

¡

¢

µ ¶ a¡# , r( a) D s log 1 C exp s

¡

¢

µ ¶ dr a ¡ # ¡1 D 1 C exp ¡ . (5.2) da s

The neuron’s weight vector rotates under the effect of Hebbian (Hebb, 1949) adaptation to a distribution Pr( x ) of training patterns, dw dt

hx r( w ¢ x )ifxg ¡ (length constraint term parallel to w ),

(5.3)

subject to a Lagrange length constraint | w | D 1. The Lagrange term in equation 5.3 must operate parallel (or antiparallel) to w , because it is derived from the gradient of | w | 2 (Webber, 1994). The only effect of using a soft threshold, equation 5.2 instead of 5.1, is that the adaptation can boot-strap itself if all the training patterns initially lie below threshold. The convergent states for the two forms (equations 5.1 and 5.2) are clearly no different if s is small (Webber, 1997). (Actually, the soft form, equation 5.2, is used in the simulations here only because it is generally preferable to the original Webber (1994) form, equation 5.1: the simulation results can also be obtained using the s D 0 form (equation 5.1). Because u( x , w ) D r( x ¢ w ) 2 , a function of x ¢ w only, the objective function U (w ) will be symmetric under any permutation symmetry of Pr( x ). Hebbian adaptation optimizes this objective function by gradient ascent (subject to the length constraint), seeking out one of its stable maxima. Maxima related to one another by permutation symmetries of Pr( x ) have basins of attraction of equal area. A large enough population of neurons, adapting independently of one another, will converge to populate such maxima in equal numbers, provided their initial states were distributed randomly and uniformly over the unit hypersphere in w -space.

576

Chris J. S. Webber

Appendix A shows analytically that the threshold # is a symmetrybreaking parameter, which determines how large a subgroup of Pr( x )’s symmetries will leave the response r( w ¢ x) invariant when w has converged to a stable state. For low enough #, the weight vector will develop symmetry under every permutation that is a symmetry of Pr( x ). As # is raised, various symmetry-breaking bifurcations will be passed, which depend quantitatively on Pr( x ). At each bifurcation, a previously stable state becomes unstable, to be replaced by a set of new stable states having a smaller symmetry group.9 The new stable states will be capable of invariant response for only a smaller group of transformations. The broken symmetries no longer map a stable state onto itself, but instead map the set of new stable states onto one another. The sequence in which the permutation symmetries of Pr( x ) are progressively broken into smaller and smaller symmetry groups as # is raised can be determined analytically (see appendix A). In this way, the symmetry-breaking parameter can be used to control the degree of invariance or discrimination in the neuron’s response. Among a collection of neurons adapting with various values for the symmetrybreaking parameter, those with lower values will develop invariant responses for a larger group of transformations, while those with higher values will discriminate to a greater extent. If all permutation symmetries are preserved so that the weight vector just ends up at the mean of the training ensemble, one obtains a useless neuron with no capacity to discriminate. If the spontaneous symmetry breaking is so extensive that only the identity symmetry is preserved, no invariances will develop. It is the ability of the Webber (1994) rule to control the symmetry breaking under the inuence of a symmetry-breaking parameter, in such a way that an intermediate-sized subgroup of the data’s permutation symmetries is preserved, which makes possible the development of response invariances in a discriminating neuron. It is clear from section 4 that a wide range of unsupervised algorithms will constitute spontaneous symmetrybreaking systems when trained on data distributions that possess symmetries, but this does not mean that such systems preserve their symmetries in general. Only the single-cell rule of Webber (1994) has so far been proved analytically to be able to preserve speciŽc subgroups up to and including the group of all the permutation symmetries of the data and to be able to break them in a well-deŽned and controlled sequence under the inuence of a symmetry-breaking parameter. (It is also possible to replace the threshold function r( a) in equations 5.1 and 5.2 by the more conventional and more biologically plausible sigmoid, and to use the same single-cell Hebbian rule and 9 If the symmetry is exact and discrete, the transition to states of lower symmetry will be abrupt. However, if Pr( x) has an almost continuous sequence of progressively less accurate symmetries, the stable states can appear to change continuously as these approximate symmetries are progressively broken. Section 6’s synthetic data set has examples of both of these kinds of symmetry.

Self-Organization of Symmetry Networks

577

symmetry-breaking parameter of Webber (1994) to reproduce this article’s empirical demonstrations, though at the expense of no longer being able to characterize the symmetry-preserving behavior analytically so easily.) 6 A First Demonstration Using Completely DeŽned Statistics

Webber (1991) provided an empirical demonstration of self-organized permutation invariance, using a synthetic data set having completely deŽned statistics. The data set was designed to model the problem of Žnding statistical dependences among sparse-coded features. The same data set is used here to demonstrate that such invariances arise through the spontaneous symmetry-breaking mechanism. The statistics of this example data set are made simple to deŽne, so that one can perceive clearly the invariances of the data and the self-organized symmetries of the weight vectors. However, the statistical dependences are nontrivial and cannot be revealed by principal component analysis or algorithms designed to Žnd localized clusters, as is explained below. This example addresses the problem of revealing the statistical dependences between sparse, nonoverlapping, sharp spikes of neural activity, and is described in greater length, with the beneŽt of diagrams, in Webber (1991). Every pattern of this data set consists of exactly four spikes of unit height on a zero background. Each spike occurs in one of four separate one-dimensional (1D) coordinate spaces labeled A to D, and there is always exactly one spike in each of the four spaces. The location of the spike within each space is determined by a single position-coordinate t, which can take integer values modulo n / 4; that is, the four 1D spaces tA , . . . , tD are each discretized into n / 4 locations and have circular boundary conditions. The spikes can be thought of as representing the responses of n localized feature detectors, arranged in four rows, each containing n / 4 detectors. Each row detects localized features of one of four types in hypothetical 1D images, within which the location of the feature is measured by the positioncoordinate t. Together the four rows of feature detectors encode those images as n-dimensional code vectors x , using a sparse 4-of-n code. The analogy with rows of feature detectors is taken further in the more realistic but less precisely deŽnable example of section 7. p The n-dimensional x -vectors of this data set all subtend an angle arccos 4 / n with the uniform n-dimensional vector (1, 1, . . . , 1). Positive-valued sparse codes have the property that their code vectors subtend large angles with the uniform vector. Every x-vector lies on one of the many vertices of p the unit hypercube that lie 4 length-units from the origin. p pNonidentical p p patterns are separated from one another by distances of 1, 2, 3, or 4 units in x-space, depending on the number of spikes the patterns happen to share in common at corresponding locations. No two patterns are farther than 2 units from each other, and no two patterns lie closer than 1 unit.

578

Chris J. S. Webber

Individually, the spikes are uniformly distributed over the four 1D spaces: Pr(tA ) D Pr(tB ) D Pr(tC ) D Pr(tD ) D 4 / n.

(6.1)

Despite their uniform marginal distributions, the locations of the spikes are statistically dependent on one another, with two signiŽcantly different correlation lengths s2 > s1 À 1. The statistics Pr( x) of these training data are determined completely from the joint distribution Pr(tA , tB , tC , tD ) D

4 N( tA ¡ tB , s1 ) N( tC ¡ tD , s1 ) n t A C tB tC C tD ¢N ¡ , s2 . 2 2

¡

¢

(6.2)

Although (discretised) univariate gaussians N ( t, s), of zero mean and standard deviation s, appear in Pr(tA , tB , tC , tD ), Pr( x) is clearly not a multivariate gaussian and is not localized into distinct clusters. Algorithms designed to Žnd localized clusters are inappropriate for revealing the statistical dependences between these nonoverlapping spikes, because there are no distinct, localized clusters in x -space to Žnd. However, both the pair of spikes ( A, B ) and the pair (C, D) appear as relatively localized structures in imagespace t, as is evident from the marginal distributions Pr(tA , tB ) D

4 N ( tA ¡ tB , s1 ) n

and

Pr(tC , tD ) D

4 N (tC ¡ tD , s1 ). (6.3) n

In image-space t, each pair is a composite structure that has an internal degree of freedom (tA ¡ tB or tC ¡ tD ). Webber (1991) points out that internal degrees of freedom of such composite image-space structures are analogous to shape degrees of freedom. The “centers of gravity” of the two pairs are uniformly distributed over image-space t, Pr

¡t

A

C tB 2

¢

D Pr

¡t

C

C tD 2

¢

D 4 / n,

(6.4)

although the two centers are locally correlated with each other: Pr

¡t

A

C tB tC C t D , 2 2

¢

D

4 N n

¡t

A

¢

C tB tC C t D ¡ , s2 . 2 2

(6.5)

The two pairs are intercorrelated more weakly than they are intracorrelated, because s2 > s1 . In the limit s2 ! 1, the pairs would cease to be intercorrelated at all. The statistical dependences deŽne composite structures on two size scales: the strongly correlated ( A, B) and ( C, D) pairs on the Žner scale, and the weaker associations between those ( A, B) and (C, D) pairs

Self-Organization of Symmetry Networks

579

on the coarser scale. The coarser-scale composite structures have a shape degree of freedom ( tA C tB ) / 2 ¡ ( tC C tD ) / 2. The statistics Pr(tA , tB , tC , tD ) are invariant under any translation t ! tCt applied to all four t-coordinates (corresponding to translationally invariant statistics of the hypothetical images that contain the features). These global translation invariances give rise to permutation symmetries of Pr( x ). The distribution is also symmetric under the two global exchanges tA $ tB and tC $ tD , permutations that map the four t-coordinates onto one another, and also under the global exchange (tA , tB ) $ ( tC , tD ). More interesting are the approximate symmetries that permute individual pairs of neighboring xi , corresponding to similar t-values, that is, t $ t C t where |t | ¿ s1 . These approximate symmetries result from the fact that N( t, s) changes only appreciably over a scale length | t | » s, so that a pair of xi corresponding to neighboring t -locations within the same t-space will have very similar statistics. These symmetries would correspond to approximate invariances of the statistics of the hypothetical 1D images, under local shifts of individual features. Principal component analysis (PCA) would be inadequate to deduce the localized nature of the structures deŽned by these correlations, because the eigenvectors of Toeplitz matrices are periodic, not localized. (The distinction between the two correlation lengths would result in periodic eigenvectors of distinct wavelengths, corresponding to distinct eigenvalues, however.) Because of its threshold nonlinearity, without which the Hebbian neuron could do no more than the PCA neuron of Oja (1982), the neuron described by equation 5.1 is sensitive to higher-order statistics—that is, properties of Pr( x ) other than its mean and covariance. Its nonlinearity allows its convergent states to undergo symmetry-breaking bifurcations under the control of a neural parameter. This is demonstrated in Figure 1. Through the preservation of some permutation symmetries of Pr( x ) and the breaking of others, this Hebbian neuron is able to reveal the two scales of localized composite structure present in the data. If the bifurcation parameter # is high enough, the neuron develops into a detector for either ( A, B) pairs or ( C, D) pairs—the Žner-scale structures. Because the global translation symmetry of Pr( x ) has been broken, a pair detector can resolve its pair’s rough location. Because the approximate symmetry of Pr( x) under exchange of neighboring t-locations has been preserved, the responses of these pair detectors are tolerant to variation in the shape degrees of freedom tA ¡ tB and tC ¡ tD . If # lies between the two bifurcation points, the neuron resolves the coarser-scale structure but can no longer resolve the Žner-scale structure. Again, because the global translation symmetry of Pr( x ) has been broken, the neuron can resolve the rough location of the coarser-scale structure. Again, the approximate symmetry of Pr( x ) under exchange of neighboring t-locations has been preserved, so the neural response is tolerant to the shape degree of freedom ( tA CtB ) / 2¡(tC CtD ) / 2. If the bifurcation parameter # is low enough, none of

580

Chris J. S. Webber

Figure 1: The weight vector w of the threshold neuron (see equation 5.1), having converged under Hebbian adaptation (see equation 5.3) to patterns drawn from the permutation-symmetric distribution Pr( x ) deŽned by equation 6.2, is plotted for each of three values of the bifurcation parameter #. Permutation-symmetric receptive Želds have developed from randomly initialized, asymmetric initial states. The approximate symmetry of Pr( x ) under individual swaps of closely neighboring t-locations is preserved unbroken into the convergent states. The receptive Želds vary smoothly with t, resulting in local position tolerance of the neural response, and tolerance to the “shape” degree of freedom. Above the second bifurcation, two kinds of convergent state can be obtained: the one depicted and another, related to it by the symmetry transformation (tA , tB ) $ (tC , tD ), which is broken at the second bifurcation. The global translation symmetry of Pr( x ) is broken at the Žrst bifurcation, and the resulting localized convergent states are related to one another via those broken translation symmetries. The special choice of origin used for this Žgure shows the states depicted as centered, although there are equally many such states at every location, identical up to a translation. Broken symmetries always relate convergent states that have equal basins of attraction, so such states are equally likely to develop if the initial states were uniformly distributed. (The very small value 10¡3 used for s means that r( w ¢ x ) is a very close approximation to the sharp nonlinearity (see equation 5.1).)

Self-Organization of Symmetry Networks

581

the permutation symmetries of Pr( x) is broken, and w O converges to the mean of Pr(x ), as predicted by the theory. When no symmetries are broken, the neuron has no capacity to discriminate between patterns. Each bifurcation breaks some symmetries but not others; the symmetry-breaking mechanism gives rise to self-organized detectors that can discriminate between structures but generalise over their internal degrees of freedom. A different kind of structure is detected in each distinct regime of the symmetry-breaking parameter. A large enough population of such neurons, with randomly initialized weights and randomly set #-values, will converge to span the full variety of convergent states available. This demonstration gives examples of both discrete, exact symmetries (the global exchanges tA $ tB , tC $ tD and (tA , tB ) $ ( tC , tD )) and continuous, approximate symmetries (the symmetries under local pair-wise swaps t $ t C t, which become progressively less accurate with increasing |t |). The breaking of exact, discrete symmetries involves abrupt transitions to states of lower symmetry, as is the case for the two bifurcations illustrated in Figure 1. The breaking of an almost continuous sequence of progressively less accurate symmetries involves an apparently continuous process of transition, however. Although this is not illustrated, the smooth weight proŽles of Figure 1 become progressively narrower as the bifurcation parameter # is raised. This narrowing process corresponds to the progressive breaking of the continuous sequence of approximate symmetries under local pair-wise swaps t $ t C t , in which less accurate, larger |t | symmetries are broken before more accurate, smaller |t |, symmetries. Preservation of symmetry under local permutations of neighboring input coordinates has given rise to locally smooth weight-vector proŽles, even though the individual training patterns are sharply spiked and not smooth. This does not happen with every equation of motion that has dynamical symmetries, however, because arbitrary adaptation rules are not in general guaranteed to preserve permutation symmetries as is the Hebbian rule of Webber (1994). If the local permutation symmetries of this data set were not preserved, the converged weight-vector proŽles would instead appear asymmetric, spiky, and disordered, and would not provide translation-tolerant response. 7 Hebbian Adaptation to Simple-Cell-Encoded Natural Images

The statistical dependences present in temporally uncorrelated, natural image data can also give rise to local weight symmetries. This demonstration also shows that the mechanism is no less applicable if the input vectors are real valued, rather than binary valued as in the previous idealized example. However, for nonsynthetic data, the distribution Pr( x ) is not constructed with symmetries deŽned a priori, so it is not so easy to determine precisely which local permutation symmetries of Pr( x) are responsible for the approximate local weight symmetries that develop. These statistical symmetries

582

Chris J. S. Webber

are not necessarily simple pair-wise swaps as in the previous synthetic example, but may involve larger subsets of the set of feature detectors. When exposed to a temporally uncorrelated ensemble of spectrally attened natural images that has rotationally invariant statistics, the Hebbian threshold neurons of Webber (1994) develop localized, oriented, bandpass receptive Želds that appear similar to those of cortical simple cells. Two examples are shown in Figure 2, obtained under simulation conditions described in appendix B. (The two examples chosen for Figure 2 are rather special in that they are horizontally and vertically oriented. Actually one can obtain receptive Želds having a range of orientations, positions, and size scales, depending on the random initialization of the weight vector.) This result raises the questions of what is gained by transforming images into this sparse, nonlinear simple-cell representation, what happens if similar, higher-level, Hebbian neurons are trained on input that has already been preprocessed by a lower network layer into this simple-cell-encoded form, and whether the naturally occurring statistical dependences between the simple-cell outputs can drive the development of weight symmetries in higher-level neurons. These questions have to be addressed using much larger natural images than those that other authors have used to derive simple-cell receptive Želds (160 £ 160 pixels as opposed to 16 £ 16). This is because one has to simulate a signiŽcant number of simple cells simultaneously, which collectively cover an image area substantially larger than the individual receptive Želds. Only then can one demonstrate the emergence of envelopes of translation tolerance that are substantially larger than the simple cells’ receptive Želds. The simulations required are much more time-consuming, because more training patterns are required to characterize the greater number of degrees of freedom that exist in higher-dimensional pattern-vector spaces. Consequently, constraints of computing resources have restricted the number of simple-cell orientations simulated here to two (horizontal and vertical), and restricted all the simple cells to having the same receptive Želd size. This is because one can efŽciently simulate many receptive Želds that share the same orientation and size by exploiting the convolution theorem (see appendix B). A more realistic demonstration of complex cell self-organization would require the simulation of many more simple cells, spanning a full range of orientations, sizes, and modal spatial frequencies, and modal Fourier phases. However, these simulations are sufŽcient to demonstrate the self-organization of weight symmetries driven by the statistical dependences within temporally uncorrelated natural data. Figure 2 describes the Žrst, simple-cell layer of a two-layer network, in which all neurons adapt according to the Webber (1994) Hebbian rule. Such a two-layer network of these Hebbian neurons was originally studied in Webber (1994), analytically and using synthetic data. All neurons in the simple-cell layer have the same threshold function (see equations 5.1

Self-Organization of Symmetry Networks

583

Figure 2: Grids of 512 “simple cells,” whose outputs form the input vector for the “complex cell” whose weight vector is represented in subsequent Žgures. These simple cells have either horizontal orientation selectivity (left) or vertical (right). The numbers in the Žgure represent pixel coordinates within the input image. The 256 horizontal cells differ only in the location of their receptive Želd’s centroid within the image plane (as do the 256 vertical cells). The centroids of the two kinds of simple cell are located on the lattice points of two corresponding grids (lower). These simple cells sample the image plane anisotropically; they are most densely packed along the dimension perpendicular to their orientation axis. The simple cells have no lateral connections with each other in this model, so the complex cell must deduce their topological relationships purely on the basis of the joint statistics of their responses. The receptive Želds are bandpass; their 2D power spectra all have bandwidths of 0.082 fN and 0.14 fN measured parallel to and perpendicular to the orientation axis, respectively. ( fN denotes the Nyquist frequency.) Their mean spatial frequency perpendicular to the orientation axis is 0.26 fN . A single receptive Želd size is used for all the simple cells so that the convolution theorem can be exploited in order to compute many scalar products efŽciently (see appendix B).

584

Chris J. S. Webber

and 5.2) and threshold value, 10 and collectively transform the spectrally attened input images into a nonlinear sparse representation. The array of real-valued, nonnegative, simple-cell outputs so generated for each image is used as the input vector x for a single higher-level neuron, which adapts by the same rule. It is this “complex cell” whose self-organization we are primarily concerned with. The array of simple cells described in Figure 2 can be conceptually divided into two grids of neurons. One grid consists of similar receptive Želds all oriented vertically, and the other of similar receptive Želds all oriented horizontally. The centroids of the vertically oriented receptive Želds are more closely packed along the horizontal image dimension than along the vertical, and the converse is true for the horizontally oriented Želds. Field (1987) has proposed, on the grounds of code completeness and economy, that simple cells should not overlap with other similarly oriented simple cells to a greater extent along their orientation axes than perpendicular to their orientation axes, and therefore that their centroids may in reality be distributed anisotropically in this way with respect to orientation axis. With the topographic arrangement of simple cells that is used here (in Figure 2), the distinct receptive Želds are roughly orthogonal to one another. 11 The simple cells have no lateral connections with each other in this model, so the complex cell must deduce their topological relationships purely on the basis of the joint statistics of their responses. Each 160 £ 160-pixel training image is excised from a larger landscape photograph, which has been subjected to a different and uniformly distributed random rotation and then spectrally attened (see appendix B). The ensemble of training images therefore has rotationally invariant statistics. Consequently, the joint distribution Pr(x ) of simple-cell outputs will be invariant under the global permutation that swaps over the two grids of Figure 2—that is, the permutation that exchanges the set of all vertically oriented simple cells with the set of all horizontally oriented simple cells, while maintaining the topographic relationships within each set. Neglecting edge effects, the training ensemble is also invariant under global transla-

10 The simple cell’s thresholds # are set at 0.012, the same value as was used during the training process that generated their receptive Želds. (The training used a very small but nonzero value of the softness parameter s D 5 £ 10¡4 , so the hard-threshold limit [see equation 5.1] is approximated very closely.) # and s are quoted in input-vector length units in which the whitened input images have pixel variance measured as 4 £ 10¡5 squared length units. The variance of the projection x ¢ w along each simple cell’s unit weight vector is also 4 £ 10¡5 (because the image ensemble is on average spectrally at, and so must have an isotropic covariance matrix). 11 The scalar product between the weight vectors of immediate neighbors in the same row is 0.0027, corresponding to a w -space angle of 89.8 degrees. The corresponding values for nearest neighbors in adjacent rows of the same orientation grid are 0.29 and 73 degrees, and 0.15 and 81 degrees for nearest neighbors separated by two rows. The values for colocated receptive Želds of perpendicular orientations are 0.17 and 80 degrees.

Self-Organization of Symmetry Networks

585

Figure 3: A self-organized conŽguration of input synapses for the second-layer “complex cell.” Heights represent values of synaptic weights (scaled by a factor of 1000). The 512 synapses provide input from the 512 “simple cells” located at the 512 grid points depicted in the previous Žgure. (Although not constrained a priori to be nonnegative, all the complex cell’s synaptic weights have developed nonnegative values, because the outputs of the simple cells that feed these synapses are always nonnegative.) In the example, the global rotation symmetry of the data’s statistics happens to have been broken in such a way that this complex cell will only be triggered by activity in the horizontally oriented simple cells: the complex cell depicted has horizontal orientation selectivity. The weight proŽle is smooth—approximately symmetric under permutations of neighboring weights. The complex cell will have translation-tolerant response as a result. In this simulation, the threshold parameter # was set at 0.02 (measured in the same length-units as the simple cells’ thresholds).

tions, because the training images were selected randomly from uniformly distributed locations within larger landscape photographs and because the training images are presented repeatedly to the network, translated to every possible position offset with respect to the network’s input window (see appendix B). Pr(x ) will therefore be approximately invariant under those global cycles among simple cells that correspond to these translations. These symmetries are only approximate because the periodic boundary condition implied by global cycling is an imperfect model of image edge effects. If Pr(x ) also has approximate symmetries under more local permutations, which involve smaller subsets of simple cells as in the synthetic example of section 6, those symmetries will give rise to localized weight symmetries when the symmetry-preserving Hebbian rule is trained on these simple-cell outputs. Further details of how this Hebbian rule is simulated here are given in appendix B. Figure 3 depicts a self-organized conŽguration of the higher-level Hebbian “complex cell,” trained on the outputs of Figure 2’s simple-cell grid as images from the training ensemble are presented. After adapting from

586

Chris J. S. Webber

a random initial conŽguration, the input synapses of this higher-level cell vary smoothly with the grid location of the corresponding simple cell. The approximate weight symmetries that have developed in the complex cell’s receptive Želd mean that the complex cell will be tolerant to the position of the simple cells that trigger it. The complex cell’s translation tolerance is substantially greater than that of the simple cells, which will be of the order of the separation between peak and trough in the simple cells’ receptive Želds (3 pixels), or less. This observation is consistent with the experimental result that real complex cells are position tolerant over a distance greater than the width of the barlike features for which they are optimally sensitive (Hubel & Wiesel, 1962). One can see from Figure 3 that the “complex cell” has developed so as to become sensitive to activity in simple cells of one orientation only; the global rotation symmetry has been broken. The complex cell’s orientation selectivity has developed because the statistical dependences between neighboring simple cells of the same orientation are stronger than the statistical dependences between simple cells of very different orientations. Long image features typically trigger several similarly oriented neighboring simple cells simultaneously, but neighboring simple cells of very different orientations are simultaneously triggered less frequently, by rarer features such as corners, which have both horizontal and vertical components. (Because rotational symmetry has been broken, the translation invariance result cannot be attributed to rotational symmetry, and so the credibility of that result is not undermined by the fact that rotational invariance of the training statistics has been reinforced by applying random rotations.) Figure 4 illustrates that broken permutation-symmetries map distinct stable states onto one another, and that raising the value of the Webber (1994) threshold parameter # breaks symmetry, as the analytical treatment of appendix B predicts. At the lower of the two threshold values depicted, the stable weight vectors are approximately invariant (i.e., invariant up to edge effects) under global permutations that correspond to translations of the whole image. The 90 degree rotation symmetry of Pr(x ) is already broken at this #-value; this rotation symmetry maps the two stable conŽgurations obtainable at this #-value onto one another. When # is raised to the higher value, the global translation symmetry is broken, and longer-range translations now map distinct localized stable conŽgurations onto one another. Even at this higher #-value, the translation-tolerance of the “complex cell” is still 10 times wider than that of the simple cells that provide its input; its weight vector remains approximately symmetric under the most localized permutations of neighboring simple cells. 8 Discussion

Webber (1991) proposed that a role of sparse feature coding might be to map the raw data into a representation in which transformations of fea-

Self-Organization of Symmetry Networks

587

tures, such as local shift and deformation, are rendered as permutations of the output values among the various feature-coding units. That hypothesis can be extended by observing that any invariances of the raw data’s statistics under such transformations would give rise to homomorphic symmetries of the sparse code’s statistics under permutations among the coding units. Such mapping of invariances onto permutation symmetries would be a useful function for sparse coding to perform, because permutation symmetries can drive the development of invariant response in neurons of the next network layer, via the spontaneous symmetry-breaking mechanism. The examples have illustrated how translations (of individual features or sets of features) can be mapped onto permutations by a sparse feature code. When the features are moved, a different subset of feature detectors Žres, which corresponds to the new feature locations. By analogy, one can see how scale transformations might be mapped onto permutations by a sparse code based on self-similar wavelets of various size scales; when a feature or set of features is enlarged or reduced, a different subset of wavelet detectors will Žre, corresponding to the new feature sizes. Because the principles proposed here do not constrain the coding units of sparse codes to have any particular topological relationships (such as the 2D topology of the image plane, for example), it is conceivable that rotations in three dimensions could be represented as permutations of some more abstract, higher-level, sparse code in which 3D features having very different orientations in 3D space are encoded by different feature detectors. (However, it would be inefŽcient to encode certain very simple transformation groups by homomorphism onto the permutations. For example, image illumination variation can be effectively removed by simple preprocessing operations such as local normalization or local background subtraction on a logarithmic scale.) Sparse feature codes represent individual features of the data on individual coding units, or feature detectors. If a sparse feature code has permutation symmetries Pr(S( x)) ¼ Pr( x ), what does this imply about the subset of features exchanged under the permutation? If exchanging one feature with another (or one subset of features with another subset) is to transform every training pattern into a roughly equiprobable pattern, then the exchanged features (or subsets of features) must have similar statistics, so that the exchanged features always arise in similar contexts with similar likelihood. (“Similar context” means that apart from the features exchanged, the same combination of features is present in the two equiprobable training patterns; the pattern of responses of the rest of the code’s feature detectors is the same for the two patterns.) Thus, if a feature code has permutation symmetries, this means that the exchanged features are indistinguishable from each other on the basis of their statistical dependences on the rest of the features that make up the code. In the spontaneous symmetry-breaking mechanism, it is this property of statistical indistin-

588

Chris J. S. Webber

guishability among features that can drive the development of invariant neural response. The example of section 6 illustrates how discrete local permutations can represent effectively continuous transformations like local position shifts and shape deformations. However, permutations can also represent absolutely discrete transformations, because permutations are inherently discrete transformations themselves. One example of an absolutely discrete transformation is the global exchange of all visual cells having left-ocular dominance with all cells having right-ocular dominance. Yamagishi (1994) has already argued that the breaking of this obvious symmetry may be responsible for the formation of ocular dominance columns. This leads us to speculate that the symmetry preservation mechanism proposed here could explain how invariance with respect to ocularity might be reacquired by cells involved in higher stages of visual processing. Another example of a discrete transformation is the replacement of a word in a sentence by a synonymous word. If the exchanged words have always arisen in similar contexts in the listener ’s experience and have similar frequencies of occurrence, then they are statistically indistinguishable by deŽnition. Another example is the difference in pronunciation of the vowels between one speaking accent and another. In principle, the statistical indistinguishability model could allow invariance with respect to pronunciation to be learned, because the different pronunciations of the vowels are always experienced in the same contexts.

Figure 4: Facing page. The effect of varying the complex cell’s threshold parameter # and the effect of starting from different random initial weight conŽgurations. Neglecting edge effects, the two weight vectors obtainable at # D 0.02 (a and b) are approximately symmetric under long-range translations, but not under exchange of horizontal with vertical; the “complex cell” is translation tolerant but orientation speciŽc. The global translation symmetry of the data’s statistics (made approximate by edge effects) has been preserved, but the 90 degree rotation symmetry has been broken. The rotation symmetry instead maps the two distinct convergent states a and b onto one another. As the threshold is raised to # D 0.04, the receptive Želds become more localized (c and d); the global translation symmetry has now been broken, but the weights are still approximately symmetric under more localized permutations. As a result, the translation tolerance of the complex cell’s response will have become more short range but not eliminated. Even at the higher #-value, the width of the tolerance envelope is still about 20 pixels, 10 times wider than the simple cells’ tolerance width. With the long-range translation symmetry broken at the higher #-value, translations as well as 90 degree rotations map distinct convergent states onto one another (e.g., c onto d). Although not depicted, the rotation symmetry may be restored by lowering # (to 0.01); then the weights all become equal (edge effects make the equality approximate).

Self-Organization of Symmetry Networks

589

Invariant neurons in symmetry networks achieve invariance by discarding information about the transformation’s degrees of freedom. For example, a translation-invariant complex cell does not retain precise information about its feature’s position degree of freedom. The discarded degreeof-freedom information is not destroyed; it is still obtainable from other neurons—for example, from the simple cells from which the complex cell takes its input. However, nowhere is there stored a feature-independent generator of the transformation, which could be used to generate transformed copies of arbitrary features from prototypical templates. The transformation is not represented independently of the object on which it acts. There is an

590

Chris J. S. Webber

advantage to this apparent omission in that there is no binding problem: no ambiguity in deciding which, of the many objects that may be present, each particular transformation applies to. Single-cell learning rules such as the Webber (1994) rule do not attempt to minimize statistical dependences between neurons as Barlow (1989) proposed. Indeed, clear statistical dependences between simple cells are needed to drive the development of complex-cell invariances in the symmetrypreservation theory, so the elimination of statistical dependence between simple cells would not be a useful objective from the point of view of this theory. Other theories have been proposed to account for the unsupervised discovery of unknown invariances, based on the temporal association of sequences of transformation-related patterns over short memory timescales (Foldiak, 1991; Wallis et al., 1993; Bartlett & Sejnowski, 1995). Using simple synthetic training patterns rather than natural images, Foldiak (1991) argued that complex cells could discover short-timescale correlations between neighboring simple cells if they were to average the simple-cell activity pattern over those short timescales. Although the explanation of complex-cell development offered here does not exploit time dependence, the general principle of invariance discovery proposed here is not incapable of making use of time dependence. However, it would do so in other contexts and in a different way. For example, it could discover statistical indistinguishabilities among the statistics of sets of motion detectors. Temporal association theories of invariance discovery have problems in principle that are not shared by the symmetry-preservation theory. If a neuron’s memory timescale is to be short enough for it to be sensitive to the temporal associations in short transformation sequences, it is hard to see why it would not forget those associations just as quickly if its stimulus temporarily became stationary and become matched to the stationary stimulus instead, so losing its transformation invariance. By exposure to rotation sequences, complex cells would Žnd associations between simple cells of different orientations as easily as they Žnd associations between simple cells of different positions from translation sequences. The temporal association model does not explain why complex cells are orientation speciŽc but translation tolerant, except by assuming that because translation sequences are (presumably) so much more prevalent than rotation sequences, cells would quickly forget any rotation sequences that they happened to learn. That assumption raises a stability-plasticity dilemma, however; if cells forget some transformation sequences in order to learn newer ones, it is not obvious that their response properties would ever settle down. Temporal association models are not relevant to discrete transformations where neurons never experience a temporal sequence—for example, the transformations that relate different speakers’ pronunciations of the same vowel, or the transformation that exchanges ocularity between left and right. Association is just as important in the symmetry-preservation theory as in temporal asso-

Self-Organization of Symmetry Networks

591

ciation theories, but the association is not on the basis of similarity over time, but similarity of context: statistical indistinguishability. In principle, symmetry preservation provides a generic mechanism for generating neurons that respond similarly to different features that arise in similar contexts. Appendix A: Symmetry-Breaking Bifurcation in the Hebbian Threshold Neuron

Webber (1994) showed that the condition for the weight vector of the neuron (see equation 5.1) to be stationary under the Hebbian dynamics is D

x xT

E1 H( w O 1)

w O 1 ¡ #hx iH( wO 1 ) D b w O1

(positive scalar b),

(A.1)

where the average is now taken over just that subset H of patterns for which the neuron’s response r rises above threshold: x¢w O > #

,

x 2 H( w O ).

(A.2)

In addition, for w O 1 to be a stable stationary state, any small perturbation 1 O Dw O C ² perpendicular to w O 1 must satisfy w D E ²O T x x T

H (wO 1 )

D E O 1T x x T ²O < w

O 1) H( w

O1 w

O 1T hx iH( wO 1 ) ¡ #w

(²O ´ | ²| ¡1 ²).

(A.3)

O 1 are permitted by the constraint Only perturbations ² perpendicular to w O | D 1. This stability condition |w « ¬ is an upper limit on the eigenvalues of that part N H( wO 1 ) of the matrix x xT H( wO 1 ) that lies in the subspace of permissible « ¬ perturbations ²—the part of x x T H( wO 1 ) that projects normal to w O 1 (Webber, 1994). Those conditions can be used to deŽne the symmetry-breaking behavior of this Hebbian neuron. When the threshold # is very large and negative, the entire training ensemble will belong to the set H, and the only stable state that will satisfy equations A.1 and A.3 is O1D w

hx ifxg |hx ifxg |

for # ! ¡1.

(A.4)

So when # is very negative, the weight vector inevitably converges to the mean of the ensemble. Because the mean is invariant under the entire group G of Pr( x )’s permutation symmetries, no permutation symmetry will be broken if # is low enough.

592

Chris J. S. Webber

As # is raised, the right-hand side of equation A.3 will get gradually 12 smaller, until it drops below the left-hand side, at which point the stationary state w O 1 will cease to be stable. It will bifurcate along the direction of the principal eigenvector of N H( wO 1 ) (or along any direction within its principal subspace if its largest eigenvalue is degenerate). Regardless of Pr( x ), the indicatrix ellipsoid of this matrix always has reection and rotation symmetries about a number of invariant subspaces, each containing a number of its eigenvectors. The symmetries that will be broken by the bifurcation are those whose invariant subspaces do not contain the principal eigenvector(s) of N H( wO ) . Permutation symmetries correspond to rotations and reections that map the axes of x -space onto one another and are special cases of proper and improper rotations: the orthogonal transformations. If Pr( x ) is symmetric O1 under a group G of orthogonal transformations, every convergent state w 0 will lie in the invariant subspaces of some group G of orthogonal « unbroken ¬ symmetries, where G0 is a subgroup of G. hx iH( wO 1 ) , x x T H( wO 1 ) and N H( wO 1 ) will all be symmetric under the unbroken orthogonal symmetries G0 of Pr( x), because w O 1 lies in the invariant subspaces of all these unbroken symmetries, and because the above-threshold subset H will therefore be symmetric if the whole of Pr( x ) is. Thus, every unbroken orthogonal symmetry of Pr( x ) will be a symmetry of the indicatrix of N H (wO 1 ) . Whichever unbroken symmetries have invariant subspaces that do not contain the principal eigenvector(s) of N H( wO 1 ) will be the ones that will be broken when the next bifurcation point is passed. Appendix B: Details of Algorithm Implementation for the Natural Image Example

The Žrst network layer, represented in Figure 2, consists of “simple-cell” receptive Želds of just two orientations but many different locations in the image plane. All the receptive Želds of a single orientation can be mapped onto one another by translations. The correlation function between the image and a receptive Želd, computable using the convolution theorem, is equivalent to an array of scalar products, between the image and an array of similar receptive Želds translated to all possible image locations. O required to compute the Consequently, the array of scalar products x ¢ w outputs of each grid of “simple cells” can be computed very efŽciently by fast Fourier transform, as is done in Webber (1998), for example, which also explains how to compute the weight updates d w O due to an array of cell 12 The left-hand and right-hand sides may both change discontinuously if the set H loses members as a result of the raising of the threshold #, although these discontinuous changes turn out not to break the inequality, because they affect both sides equally. This is because any pattern that is just about to pass below the x ¢ w D # plane will make only a negligible contribution u( x , w ) to the objective function.

Self-Organization of Symmetry Networks

593

outputs equally efŽciently. This trick can be used to train both the simplecell layer and the complex cell. It is only by exploiting the huge speed enhancement afforded by the convolution theorem that the simulations presented here are feasible for a contemporary serial computer in reasonable time. Computing a pair of correlation functions, between a 160 £ 160-pixel input image and a pair of 64 £ 64-pixel “simple-cell” receptive-Želd kernels (one horizontally and the other vertically oriented), yields a pair of square correlation functions of side length 160 ¡ 64 C 1 D 97 pixels, if the artifact of image wraparound is to be avoided. That artifact would introduce undesirable horizontal and vertical correlations between the simple-cell outputs, due to the long discontinuities at the input-image edges. This pair of 97 £ 97-pixel correlation functions forms the basis for the training input to the single Hebbian “complex cell” that forms the second layer of the network. The “simple-cell” receptive Želds are not located at 1-pixel intervals, but sample only at the grid points illustrated in Figure 2, so the “complex cell” is allowed to receive input from the two correlation functions only through a limited number of synapses, connecting it only to those coordinates of the correlation functions corresponding to these grid points. As illustrated, these grid points extend over a region of the image 64 pixels square, but there are only 512 of them, so the “complex cell” has 512 synaptic inputs, half of them from vertically oriented “simple cells” and half from horizontally oriented “simple cells.” The size of the training set is augmented by updating the complex cell’s synapses with each training image presented repeatedly, at every possible location translated with respect to the input synapses. This amounts to reinforcing the natural translational invariance of the image statistics. The convolution theorem allows this to be done very efŽciently (Webber, 1998). In this way, one can augment the training set by a factor of (97 ¡ 64 C 1)2 if one wishes to avoid wraparound of the correlation functions. Augmenting the size of the training set in this way is necessary for two closely related reasons. First, one needs a sufŽciently large training set to characterize the relevant degrees of freedom of natural images adequately; without such augmentation, too few photographs would have been available to populate the high-dimensional pattern space, so there would have been very strong statistical biases due to gross undersampling of the space. Second, one needs to present at each iteration a sufŽciently large random subsample of this training set to ensure that the adaptation converges smoothly and is not disrupted by statistical uctuations introduced by undersampling bias. Each 200-iteration simulation required about six weeks’ CPU time on a 266 MHz Alpha, benchmarked at 90 SPECfp rate95. At each iteration, 16,000 training images were presented to the network, each presented at each of the (97 ¡ 64 C 1) 2 position-offsets computable simultaneously using the convolution theorem. Each set of 16,000 training images was selected randomly from a total ensemble of 392,400 images, which were

594

Chris J. S. Webber

obtained by rotating 13 109 large landscape photographs by 1800 different angles covering the full 360 degrees, duplicating these by reections, then applying a spectral whitening Žlter (the same Žlter 14 is applied to all photographs), and then excising a randomly chosen 160£160 pixel region within the rotated photograph (avoiding the edges of the photograph where the whitening produces artifacts). Together with the translations, the reections and rotations are necessary to make the training set large enough to eliminate statistical bias due to undersampling of the high-dimensional pattern space. In all simulations depicted in Figures 3 and 4, the complex cell was trained with the very small but nonzero softness parameter s D 5 £ 10¡4 . The fractional rate constant l 0 , which is related to the time-varying Webber (1994) rate-parameter l (the constant of proportionality in equation 5.3) as deŽned in Webber (1997), was set at 0.3 for all cells in all these simulations. Acknowledgments

I am grateful to Steve Luttrell for discussions. References Barlow, H. B. (1989). Unsupervised learning. Neural Computation, 1, 295–311. Bartlett, M. S., & Sejnowski, T. (1995). Unsupervised learning of invariant representations of faces through temporal association. In J. M. Bower (Ed.), The neurobiology of computation. Norwell, MA: Kluwer. Bell, A. J., & Sejnowski, T. J. (1997). The “independent components” of natural scenes are edge Žlters. Vision Research, 37, 3327–3338. Blais, B. S., Intrator, N., Shouval, H. V., & Cooper, L. N. (1998). Receptive Želd formation in natural scene environments: Comparison of single-cell learning rules. Neural Computation, 10, 1797–1813. Field, D. J. (1987). Relations between the statistics of natural images and the response properties of cortical cells. Journal of the Optical Society of America, 4, 2379–2394. Foldiak, P. (1991). Learning invariance from transformation sequences. Neural Computation, 3, 194–200. Friedman, J. H. (1987). Exploratory projection pursuit. Journal of the American Statistical Association, 82, 249–266. Fukumi, M., Omatu, S., Takeda, F., & Kosaka, T. (1992). Rotation-invariant neural pattern recognition system with application to coin recognition. IEEE Transactions on Neural Networks, 3, 272–279.

13

The rotation uses a conventional bilinear interpolation algorithm. The whitening Žlter has been optimized so as to make the average power spectrum over the entire training ensemble at. 14

Self-Organization of Symmetry Networks

595

Fukushima, K., & Miyake, S. (1982). Neocognitron: A new algorithm for pattern recognition tolerant of deformations and shifts in position. Pattern Recognition, 15, 455–469. Hebb, D. O. (1949). The organisation of behaviour. New York: Wiley. Hinton, G. E. (1987). Learning translation invariant recognition in a massively parallel network. In G. Goos & J. Hartmanis (Eds.), PARLE: Parallel architectures and languages europe (pp. 1-13).New York: Springer-Verlag. Hubel, D. H., & Wiesel, T. N. (1962). Receptive Želds, binocular interaction and functional architecture in the cat’s visual cortex. Journal of Physiology, 160, 106–154. Le Cun, Y., Boser, B., Denker, J., Henderson, D., Howard, R., Hubbard, W., & Jackel, L. (1989). Backpropagation applied to handwritten zip code recognition. Neural Computation, 1, 541–551. Minsky, M. L., & Papert, S. A. (1969). Perceptrons. Cambridge, MA: MIT Press. Oja, E. (1982). A simpliŽed neuron model as a principal component analyser. Journal of Mathematical Biology, 15, 267–273. Olshausen, B. A., & Field, D. J. (1996). Emergence of simple-cell receptive Želd properties by learning a sparse code for natural images. Nature, 381, 607– 609. Rao, R. P. N., & Ballard, D. H. (1998).Development of localized oriented receptive Želds by learning a translation-invariant code for natural images. Network: Computation in Neural Systems, 9, 219–234. Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning internal representations by error propagation. In D. E. Rumelhart & J. L. McClelland (Eds.), Parallel distributed processing: Explorations in the microstructures of cognition (Vol. 1, pp. 318–362). Cambridge, MA: MIT Press. Shawe-Taylor, J. (1989).Building symmetries into feedforward networks. In First IEEE Conference on ArtiŽcial Neural Networks (pp. 158–162). Shawe-Taylor, J. (1993). Symmetries and discriminability in feedforward network architectures. IEEE Transactions on Neural Networks, 4, 816–826. Simard, P., Victorri, B., Le Cun, Y., & Denker, J. (1991).Tangent prop—a formalism for specifying selected invariances in an adaptive network. Neural Information Processing Systems, 4, 895–903. Wallis, G., Rolls, E., & Foldiak, P. (1993). Learning invariant responses to the natural transformations of objects. Proceedings of the 1993 International Joint Conference on Neural Networks, 2, 1087–1090. Webber, C. J. S. (1991). Self-organisation of position- and deformation-tolerant neural representations. Network: Computation in Neural Systems, 2, 43–61. Webber, C. J. S. (1994). Self-organisation of transformation-invariant detectors for constituents of perceptual patterns. Network: Computation in Neural Systems, 5, 471–496. Webber, C. J. S. (1997). Generalisation and discrimination emerge from a selforganising componential network: A speech example. Network: Computation in Neural Systems, 8, 425–440. Webber, C. J. S. (1998). Emergent componential coding of a handwriting-image database by neural self-organisation. Network: Computation in Neural Systems, 9, 433–447.

596

Chris J. S. Webber

Wood, J. J. (1995). A model of adaptive invariance. Unpublished doctoral dissertation, University of London. Yamagishi, K. (1994). Spontaneous symmetry breaking and the formation of columnar structures in the primary visual cortex. Network: Computation in Neural Systems, 5, 61–73. Received March 23, 1998; accepted April 26, 1999.

LETTER

Communicated by Bard Ermentrout

Geometric Analysis of Population Rhythms in Synaptically Coupled Neuronal Networks J. Rubin D. Terman Department of Mathematics, Ohio State University, Columbus, Ohio 43210, U.S.A. We develop geometric dynamical systems methods to determine how various components contribute to a neuronal network’s emergent population behaviors. The results clarify the multiple roles inhibition can play in producing different rhythms. Which rhythms arise depends on how inhibition interacts with intrinsic properties of the neurons; the nature of these interactions depends on the underlying architecture of the network. Our analysis demonstrates that fast inhibitory coupling may lead to synchronized rhythms if either the cells within the network or the architecture of the network is sufŽciently complicated. This cannot occur in mutually coupled networks with basic cells; the geometric approach helps explain how additional network complexity allows for synchronized rhythms in the presence of fast inhibitory coupling. The networks and issues considered are motivated by recent models for thalamic oscillations. The analysis helps clarify the roles of various biophysical features, such as fast and slow inhibition, cortical inputs, and ionic conductances, in producing network behavior associated with the spindle sleep rhythm and with paroxysmal discharge rhythms. Transitions between these rhythms are also discussed. 1 Introduction

Neuronal networks often exhibit a rich variety of oscillatory behavior. The dynamics of even a single cell may be quite complicated; it may, for example, Žre repetitive spikes or bursts of action potentials, each followed by a silent phase of near-quiescent behavior (Rinzel, 1987; Wang & Rinzel, 1995). The bursting behavior may wax and wane on a slower timescale (Destexhe, Babloyantz, & Sejnowski, 1993; Bal & McCormick, 1996). Examples of population rhythms include synchronous behavior, in which every cell in the network Žres at the same time, and clustering, in which the entire population of cells breaks up into subpopulations or blocks; the cells within a single block Žre synchronously while different blocks are desynchronized from each other (Golomb & Rinzel, 1994; Kopell & LeMasson, 1994). Of course, much more complicated population rhythms are also possible (Traub & Miles, 1991; Terman & Lee, 1997). Activity may also propagate through the Neural Computation 12, 597–645 (2000)

c 2000 Massachusetts Institute of Technology °

598

J. Rubin and D. Terman

network in a wavelike manner (Kim, Bal, & McCormick, 1995; Destexhe, Bal, McCormick, & Sejnowski, 1996; Golomb, Wang, & Rinzel, 1996; Rinzel, Terman, Wang, & Ermentrout, 1998). A network’s population rhythm results from interactions among three separate components: the intrinsic properties of individual neurons, the synaptic properties of coupling between neurons, and the architecture of coupling (i.e., which neurons communicate with each other). These components typically involve numerous parameters and multiple timescales. The synaptic coupling, for example, can be excitatory or inhibitory, and its possible turn-on and turn-off rates can vary widely. Neuronal systems may include several different types of cells, as well as different types of coupling. An important and typically challenging problem is to determine the role each component plays in shaping the emergent network behavior. In this article we consider recent models for thalamic oscillations (see, for example, Destexhe, McCormick, & Sejnowski, 1993; Steriade, McCormick, & Sejnowski, 1993; Golomb, Wang, & Rinzel, 1994; Terman, Bose, & Kopell, 1996; Destexhe & Sejnowski, 1997). The networks consist of several types of cells and include excitatory as well as both fast and slow inhibitory coupling. One interesting property of these networks is that they exhibit very different rhythms for different parameter ranges. For some parameter values, the network behavior resembles that of the spindle sleep rhythm: one population of cells is synchronized at the spindle frequency, while another population of cells exhibits clustering. If a certain parameter, corresponding to the strength of fast inhibition, is varied, then the entire network becomes synchronized. This resembles paroxysmal discharge rhythms associated with spike-and-wave epilepsy. In other parameter ranges, the network behavior is similar to that associated with the delta sleep rhythm; in this case, each cell exhibits an entirely different behavior from before. We develop geometric dynamical systems methods to analyze the mechanisms responsible for each of these rhythms and the transitions between them. This approach helps determine each component’s contribution to the network behavior and to clarify how the behavior changes with respect to parameters. We are particularly interested in analyzing the role of inhibitory coupling in generating different oscillatory behaviors. This is done by considering a series of networks with increasing levels of complexity. Our analysis demonstrates, for example, how networks with distinct architectures can make different uses of inhibition to produce different rhythms. The techniques we develop are quite general and do not depend on the details of the speciŽc systems. For a given network, however, these techniques lead to rather precise conditions for when a particular rhythm is possible. Numerous work has considered the role of inhibition in synchronizing oscillations (for example, Wang & Rinzel, 1992, 1993; Golomb & Rinzel, 1993; van Vreeswijk, Abbott, & Ermentrout, 1994; Whittington, Traub, & Jefferys, 1995; Bush & Sejnowski, 1996; Gerstner, van Hemmen, & Cowan, 1996; Rowat & Selverston, 1997; Terman, Kopell, & Bose, 1998). Many of

Population Rhythms

599

these articles used model neurons with very short spiking times. These include integrate-and-Žre models along with alpha function–type dynamics for the synapses. More biophysically based models for bursting neurons were considered by, among others, Wang and Rinzel (1993) and Terman et al. (1998). One conclusion of those works is that inhibition can lead to synchrony only if the inhibition decays at a sufŽciently slow rate; in particular, the rate of decay of the synapses must be slower than the rate at which the neurons recover in their refractory period. These theoretical and numerical studies, however, considered rather idealized networks: Each cell contained only one channel state variable, and the network architecture was simply two mutually coupled cells. By considering more realistic biophysical models, we demonstrate that fast inhibitory coupling can indeed lead to synchronous rhythms. We show that this is possible in networks with complicated cells but simple architectures and in networks with more complicated architectures but simple cells. Geometric singular perturbation methods have been used previously to study the population rhythms of neuronal networks (for example, Somers & Kopell, 1993; Skinner, Kopell, & Marder, 1994; Terman & Wang, 1995; Terman & Lee, 1997; Terman et al., 1998; LoFaro & Kopell, in press). Each of the relevant networks possess multiple timescales; this allows one to dissect the full system into fast and slow subsystems. Often, however, there is no clear-cut separation of timescales, so it is not obvious how one decides which variables should be considered as fast or slow. This is particularly the case when there are multiple intrinsic channel state variables and the synaptic variables turn on and turn off at different rates. A primary goal of this article is to demonstrate how one can make the singular reduction in order to understand mechanisms responsible for the thalamic rhythms. Two crucial issues are related to the geometric analysis. The Žrst is concerned with the existence of a singular solution corresponding to a particular pattern. We assume that individual cells, without any coupling, are unable to oscillate. The existence of network oscillatory behavior then depends on whether the singular trajectory is able to “escape” from the silent phase. An important point will be that greater cellular or network complexity enhances each cell’s opportunity to escape the silent phase when coupled. The second issue is concerned with the stability of the solution. To demonstrate stability of a perfectly synchronous state, for example, we must show that the trajectories corresponding to different cells are brought closer together as they evolve in phase space. As we shall see, this compression is usually not controlled by a single factor; it depends on the underlying architecture as well as nontrivial interactions between the intrinsic and synaptic properties of the cells (see also Terman et al., 1998). Our analysis demonstrates, for example, why thalamic networks are well suited to use inhibitory coupling to help synchronize oscillations and produce other, clustered, rhythms. In the next section we describe in detail the types of models to be considered for individual neurons. We distinguish between basic and compound

600

J. Rubin and D. Terman

cells. As a concrete example, we consider recent conductance-based models for thalamocortical relay (TC) cells (Golomb et al., 1994; Destexhe & Sejnowski, 1997 and references therein). Compound cells can be realized as a model for a TC cell that includes three ionic currents: a low-threshold calcium current (IT ), the nonselective sag current (Isag ), and a leak. A basic cell does not include Isag . In this context, our results help to explain the role of Isag in generating network activity; we shall see that this role depends on the architecture of the network. We also discuss different forms of synaptic coupling and different architectures. In section 3, we show that synchronization is possible in mutually coupled networks that include compound cells and fast synapses. This analysis is similar to that done by Terman et al. (1998), which showed that synchronization is possible in networks with basic cells and slow inhibitory coupling. Those networks contain two types of slow processes: one corresponds to an intrinsic ionic current and the other to a synaptic slow variable. The main conclusion of the analysis here is that what is crucial for synchronization is that the network possess at least two slow processes; one may be intrinsic and the other synaptic, or both may be intrinsic. In section 4, we consider networks with architectures motivated by recent models for the thalamic spindle sleep rhythm. The more complex architectures allow the network to use inhibition in different ways to produce different population rhythms. In particular, inhibition can play an important role in synchronizing the cells in a much more robust way than in the mutually coupled networks. We demonstrate how tuning various parameters allows the network to control the effect of inhibition and thereby control the emergent behavior. Consequences of these results for the full thalamic networks are presented in section 5. We consider the roles of various biophysical parameters associated with fast and slow inhibition, the sag current, cortical inputs, and other currents. These results help clarify the mechanisms responsible for spindle and paroxysmal discharge rhythms and the ways that changes in biophysical parameters lead to transitions between different rhythms. We conclude with a discussion in section 6.

2 The Models

We begin by describing the equations corresponding to individual cells and then describe the synaptic coupling between two cells. It is necessary to explain which parameters determine whether the synapse is excitatory or inhibitory and which other parameters determine whether the synapse is fast or slow. It will also be necessary to distinguish between direct synapses and indirect synapses. Finally, we describe the types of architectures to be considered.

Population Rhythms

601

Figure 1: Nullclines of basic and compound cells. (A) The v- and w-nullclines of a basic cell intersect at p0 , on the middle branch of the v-nullcline, in the oscillatory case. The closed curve indicates a singular periodic orbit, with double arrows denoting fast pieces and single arrows denoting slow pieces. (B) The vnullcline and a singular solution of an excitable compound cell with a stable critical point p0 . 2.1 Single Cells. We model a basic cell as the relaxation oscillator

v0 D f ( v, w) w0 D 2 g( v, w),

(2.1)

d where 0 D dt . Here 2 is assumed to be small; hence, w represents a slowly evolving quantity. The active phase of the oscillation can be viewed as the envelope of a burst of spikes. We assume that the v-nullcline, f (v, w) D 0, deŽnes a cubic-shaped curve, as shown in Figure 1A, and the w-nullcline, g D 0, is a monotone decreasing curve that intersects f D 0 at a unique point p0 . We also assume that f > 0 ( f < 0) above (below) the v¡nullcline and

602

J. Rubin and D. Terman

g > 0 (< 0) below (above) the w-nullcline. If p0 lies on the middle branch of f D 0, then equation 2.1 gives rise to a periodic solution for all 2 sufŽciently small, and we say that the system is oscillatory. In the limit 2 ! 0, one can construct a singular solution as shown in Figure 1A. If p0 lies on the left branch of f D 0, then the system is said to be excitable; p0 is a stable Žxed point, and there are no periodic solutions for all 2 small. Note that the thalamic cells we seek to model are typically excitable during the sleep state. For some of our results, it will be necessary to make some more technical assumptions on the nonlinearities f and g. We will sometimes assume that fw > 0,

gv < 0

and

gw < 0

(2.2)

near the singular solutions. By a compound cell we mean one that contains at least two slow processes. We consider compound cells that satisfy equations of the form v0 D f ( v, w, y) w0 D 2 g( v, w) y0 D 2 h(v, y).

(2.3)

Precise assumptions required on the nonlinear functions in equation 2.3 are given later. For now, we assume that for each Žxed value of y, the functions f ( v, w, y) and g( v, w) satisfy the conditions described for a basic cell. Then f f ( v, w, y) D 0g deŽnes a cubic-shaped surface. The system (see equation 2.3) is said to be excitable if there exists a unique Žxed point, which we denote by p0 , and this lies on the left branch of the cubic-shaped surface. One can construct singular solutions of this equation, and one of these is shown in Figure 1B. The singular solution shown begins in the silent phase, or left branch, of the surface. It evolves there until it reaches the curve of jump-up points that correspond to the left knees of the cubic surface. The singular solution then jumps up to the active phase, or right branch, of the surface. It evolves in the active phase until it reaches the jump-down points or right knees of the surface. It then evolves in the silent phase, approaching the stable Žxed point at p0 . A more formal description of certain singular solutions is given in section 3.2. 2.2 Synaptic Coupling. Consider the network of two mutually coupled cells: E1 $ E2 . The equations corresponding to this network are

v01 D f ( v1 , q1 ) ¡ gsyns2 ( v1 ¡ vsyn) q01 D 2 L ( v1 , q1 ) v02 D f ( v2 , q2 ) ¡ gsyns1 ( v2 ¡ vsyn) q02 D 2 L ( v2 , q2 ).

(2.4)

Population Rhythms

603

Here, qi D wi and L D g if the cells are basic, while qi D ( wi , yi ) and L D ( g, h) if the cells are compound. In equation 2.4, gsyn > 0. It is the parameter vsyn that determines whether the synapse is excitatory or inhibitory. If vsyn < v along each bounded singular solution, then the synapse is inhibitory. The coupling depends on the synaptic variables si , i D 1, 2. We consider two choices for the si . Each si may satisfy a Žrst-order equation of the form s0i D a(1 ¡ si ) H (vi ¡ h syn) ¡ bsi .

(2.5)

Here, a and b are positive constants, H is the Heaviside step function, and hsyn is a threshold above which one cell can inuence the other. Note that a and b are related to the rates at which the synapses turn on or turn off. For fast synapses, we assume that a is O(1) with respect to 2 . It may seem natural also to assume that b is O(1), and this is, in fact, what we will do in the next section. However, when we consider the thalamic networks in sections 4 and 5, it will be necessary to assume that b D K2 , where K is a large constant. The reason for choosing the fast synaptic variables in this way is discussed in more detail later. (The model for slow synapses is discussed in section 5.2.) If the synaptic variables satisfy equation 2.5, then we say that the synapse is direct. We will also consider indirect synapses. These are modeled by introducing a second synaptic variable xi (see Golomb et al., 1994; Terman et al., 1998). The equations for ( xi , si ) are: x0i D 2 ax (1 ¡ xi ) H (vi ¡ hsyn) ¡ 2 bx xi s0i D a(1 ¡ si )H ( xi ¡ hx ) ¡ bsi .

(2.6)

Here, ax and bx are positive constants. Note that indirect synapses have the effect of introducing a delay in the synaptic action, and this delay takes place on the slow timescale. If, say, the cell E1 Žres, then x1 will activate once v1 crosses the threshold h syn . The activation of s1 must wait until x1 crosses the second threshold hx . Note also that an indirect synapse can be fast if a and b are O(1), as discussed above. 2.3 Globally Inhibitory Networks. Besides mutually coupled networks, we also consider networks with the architecture shown in Figure 2. This network contains two different types of cells, labeled E-cells and J-cells. Each E-cell sends fast excitation to some of the J-cells, and each J-cell sends inhibition to some of the E-cells. The inhibition may be fast or slow (or both). There is no communication among different E-cells; however, the J-cells communicate with each other via fast inhibitory coupling. This network is motivated by recent models for the thalamic sleep rhythms discussed in section 1. The E- and J-cells correspond to thalamocortical relay (TC) and thalamic reticularis (RE) cells, respectively.

604

J. Rubin and D. Terman

Figure 2: Network of inhibitory J-cells and excitatory E-cells.

3 Mutually Coupled Compound Cells with Fast Inhibitory Synapses

Rubin and Terman (1998) proved that stable synchronous oscillations are not possible in mutually coupled networks with fast inhibitory coupling and only one slow variable corresponding to each cell. This holds regardless of whether the cells are excitable or oscillatory and whether the synapses are direct or indirect. These results are for networks with relaxation-type neurons, as discussed in the previous section. Here we demonstrate that when the mutually coupled cells are compound, they can exhibit synchronized oscillations when connected with fast inhibitory coupling. The synchronous solution may exist if the synapses are direct, but as in Terman et al. (1998), it can be stable only if the synapses are indirect (see section 3.3). We assume that each cell is excitable for Žxed levels of input; however, there is no problem in extending the analysis if this does not hold. We analyze the network by constructing singular solutions, done by piecing together solutions of reduced fast and slow subsystems. Since the cells are compound, there will be at least two slow variables corresponding to each cell. The multiple slow variables are needed for both the existence and the stability of the synchronous solution. For existence, the multiple slow variables allow the singular trajectory to escape from the silent phase, despite the fact that each cell is excitable. The multiple slow variables also allow for several mechanisms that lead to compression of cells as they evolve in phase-space. We do not give precise conditions on the parameters and nonlinearities in the equations to specify when the synchronous solution is stable, as was done in Terman et al. (1998). This will be done elsewhere. Here we describe the geometric mechanisms that allow the cells to escape the silent phase so a synchronous solution is possible and then characterize the compression mechanisms that act to stabilize the synchronous solution. A primary aim of this article is to compare how inhibition is used in different networks with different architectures to produce stable synchrony. By identifying the compression mechanisms here, we are able to evaluate the robustness of the

Population Rhythms

605

synchronous solution. As we shall see, the compression mechanisms for the mutually coupled architecture are considerably less robust than those that arise in the globally inhibitory thalamic networks. The analysis here is similar to but more general than that in Terman et al. (1998), where it is demonstrated that stable synchrony can arise in networks with basic cells but slow synapses. It is assumed in Terman et al. (1998) that the synapses activate on the fast timescale and that the evolution of the cells in the active phase does not depend on the level of synaptic input. These assumptions imply that the slow system corresponding to the active phase is one-dimensional. We do not make these assumptions here and demonstrate that the resulting richer dynamics may lead to additional compression mechanisms for stabilizing the synchronous solution. 3.1 Singular Solutions. We analyze the network by treating 2 as a small, singular perturbation parameter and constructing singular solutions. These consist of various pieces, each piece corresponding to a solution of either fast or slow equations. The fast equations are obtained by simply letting 2 D 0 in equation 2.4 and in either equation 2.5 or 2.6. The slow equations are obtained by replacing t with t D 2 t as the independent variable and then letting 2 D 0. We assume here that both a and b are independent of 2 . There is no problem in extending this analysis if b D O(2 ); this is actually an easier case since the additional slow variable then provides additional opportunities for compression. Here we derive the reduced slow equations valid when the synapses are direct. The equations for indirect synapses are similar, but there are more cases to consider. The slow equations are

0 D f (vi , wi , yi ) ¡ gsyn sj ( vi ¡ vsyn ) wP i D g( vi , wi ) yP i D h(vi , yi ) 0 D a(1 ¡ si ) H (vi ¡ hsyn) ¡ bsi ,

(3.1)

where P D dtd . One can reduce this system to equations for just the slow variables ( yi , wi ). There are several cases to consider depending on whether both cells are silent, both are active, or one is silent and the other is active. We assume that the solution of the Žrst equation in 3.1 deŽnes a cubic-shaped surface C s , and the left and right branches of this surface can be expressed as vi D W L ( wi , yi , si ) and vi D W R ( wi , yi , si ), respectively. If both cells are silent, then each vi < h syn and si D 0. Let GL ( w, y, s) ´ g(W L ( w, y, s), w) and HL ( w, y, s) ´ h(W L ( w, y, s), y). Then each ( wi , yi ) satisŽes the equations wP D GL (w, y, 0)

yP D HL ( w, y, 0).

(3.2)

606

J. Rubin and D. Terman

If both cells are active, then each vi > h syn and the last equation in 3.1 implies that si D sA ´ a / (a C b). Let GR ( w, y, s) ´ g(W R (w, y, s), w) and HR ( w, y, s) ´ h(W R ( w, y, s), y). Then each (wi , yi ) satisŽes the equations wP D GR ( w, y, sA )

yP D HR ( w, y, sA ).

(3.3)

Finally suppose that one cell, say cell 1, is silent and cell 2 is active. Then the slow variables satisfy the reduced equations wP1 D GL ( w1 , y1 , sA ) yP1 D HL ( w1 , y1 , sA )

wP2 D GR ( w2 , y2 , 0) yP2 D HR ( w2 , y2 , 0).

(3.4)

We may view the singular solution as two points moving around in the ( y, w) slow phase-space. Each point corresponds to the projection of one of the cells onto the slow phase plane. The points evolve according to one of the reduced slow systems until one of the points reaches a jump-up or jump-down curve. The cells then jump in the full phase-space; however, the slow variables remain constant during the fast transitions. The points then “change directions” in slow phase-space and evolve according to some other reduced slow equations. Since the cells are excitable, the reduced system, equation 3.2, with s D 0 has a stable Žxed point, which we denote by P0 . The slow phase-space corresponding to equation 3.2 is illustrated in Figure 3A. Note that while some of the trajectories are attracted toward P0 , others are able to reach the jump-up curve. That is, although the uncoupled cells are excitable, it is possible for a cell to begin in the silent phase and still Žre. This will be important in the next section, when we discuss the existence of the synchronous solution. The following lemma characterizes the left and right folds (or jumpup and jump-down curves) of C s . We assume here that fy > 0 on the left branch of C s while fy < 0 on the right. This assumption is justiŽed, based on biophysical considerations, in remark 4 (in appendix A). The left and right folds of C s can be expressed as JL D f(vL ( y, s), L L R ( ), )g < 0, @w > 0, @w > 0, wL y, s y and JR D f(vR ( y, s), wR ( y, s), y)g where @w @y @s @y Lemma.

and

@wR @s

> 0.

Since vL ( y, s) D W L (wL ( y, s), y, s), it follows from equation 3.1 and the deŽnition of folds that Proof.

0 D f (W L (wL ( y, s), y, s), wL ( y, s), y) ¡ gsyn s(W L ( wL ( y, s), y, s) ¡ vsyn ) 0 D fv (W L ( wL ( y, s), y, s), wL ( y, s), y) ¡ gsyns (3.5)

Population Rhythms

607

Figure 3: Singular solutions for compound cells. (A) The slow phase-space of an uncoupled compound cell, bounded by the jump-up curve JL and the jumpdown curve JR of the slow manifold Cs . The solid (dashed) curves represent evolution in the silent (active) phase. P0 is a stable Žxed point. (B) Numerically generated synchronous solution for mutually coupled compound cells that are separately excitable, in (y, w)-space. The inset (v versus t) shows the voltage traces as two cells approach synchrony. These curves, as well as those in other Žgures, were generated using the program XPPAUT, developed by G. B. Ermentrout, with parameter values given in appendix A. The solid curve is the synchronous solution, the dashed curves are the jump-up (labeled) and jump-down (approximately horizontal, unlabeled) curves for s D 0.2, and the dash-dotted curves (partially obscured) are those for s D sA D 0.8. Note that the curves for s D 0.8 lie at larger w-values than those for s D 0.2. Since 2 6 D 0, the synchronous solution does not jump up [y0 D 0, near (y, w) D (0.08, 0.07)] immediately on reaching the jump-up curve.

608

J. Rubin and D. Terman

Differentiating the Žrst equation in 3.5 with respect to w and using the second equation, we obtain 0D

@f @wL ¡ gsyn(W L ( wL ( y, s), y, s) ¡ vsyn). @w @s

Hence, gsyn(W L (wL ( y, s), y, s) ¡ vsyn) @wL D . @s fw

(3.6)

The right-hand side of this expression has a positive numerator because the coupling is inhibitory and a positive denominator from equation 2.2, so @wL R > 0. Analogously, @w > 0. @s @s Similarly, differentiating with respect to y in equation 3.5 yields 0 D f @f @wL L C @f , or @w D ¡ f y , with fy @w @y @y @y w fy @wR D ¡ , with , evaluated f f y w @y fw

and fw evaluated on JL . Analogously,

on JR . The above assumptions on fy , together with equation 2.2, yield the desired result. In the thalamic networks of interest, the jump-down curve JR is nearly horizontal (see Figure 3B). This holds because the y current has a much smaller reversal potential and maximal conductance than the w current; when a cell is in the active phase, this implies that | fy | ¿ | fw |, so R | @w | is quite small. (See remark 5 in appendix A.) @y Remark 1.

3.2 Existence of the Synchronous Solution. Here we illustrate why it is possible for a synchronous solution to exist in a network of mutually coupled compound cells even when the individual cells are excitable. A numerically generated picture of such a solution, projected onto the slow variables ( y, w), is shown together with certain jump-up and jump-down curves in Figure 3B. We also show a solution with initial conditions slightly perturbed from the synchronous solution. The precise equations that this solution satisŽes and the parameter values used numerically are given in appendix A. One constructs the singular synchronous solution as follows. We begin when the cells are in the silent phase, just after they have jumped down. The slow variables then evolve according to equation 3.2. If they are able to reach the jump-up curve, then they jump up according to the fast equations. While in the active phase, the slow variables satisfy equation 3.3 until they reach the jump-down curve. They then jump down according to the fast equations, and this completes one cycle of the synchronous solution. Note that w and y are decreasing in the active phase and then increase just after jump-down; we use this in the next subsection.

Population Rhythms

609

It is not clear how to choose the starting point ( yi (0), wi (0)) so that the singular orbit returns precisely to this point after one cycle. Note, however, that the variables yi relax very close to yi D 0 during the active phase. If we suppose that yi ¼ 0 at the jump-down point, then the value of wi is determined; that is, for the coupled cells, wi ¼ wR (0, sA ). A straightforward Žxed-point argument shows that the synchronous solution will therefore exist if the solutions of equation 3.2, which begin near ( yi , wi ) D (0, wR (0, sA )), are able to reach the curve of jump-up points. The reason that a synchronous solution can exist even when the uncoupled cells cannot oscillate is that the synchronous solution lies on a different cubic during the active phase than the uncoupled cells. For this reason, the synchronous solution jumps down along a different curve than the uncoupled cells do. From the lemma, the jump-down curve JR (sA ) has larger values of w than the jump-down curve JR (0) (see Figure 3B). It is therefore possible for the coupled cells to jump down to a point from which they are able eventually to escape the silent phase, although the uncoupled cells jump down to a point from which they cannot escape. There is a nice biophysical interpretation for why coupled excitable cells may be able to oscillate. Recall that a thalamocortical relay cell is an example of a compound cell. (See section 5 for a more detailed discussion.) Then w corresponds to the inactivation variable of the IT current. A larger value of w means that this current is more deinactivated. This implies that if the cells jump down at a larger value of w, then it is easier for the cells to become sufŽciently depolarized so they can reach threshold and Žre. The construction of the synchronous solution for indirect synapses is very similar. The only difference is that after the cells jump up or jump down, there is a delay until the inhibition either turns on or turns off. The cells switch their cubic surface while in the silent and active phases, assuming that the delay is shorter than the time that the cells spend in each of their silent and active phases. We will assume that this is the case throughout the remainder of this article. 3.3 Stability of the Synchronous Solutions. We assume throughout this section that the synapses are indirect. The synchronous solution cannot be stable if the synapses are direct for the following reason. Suppose we start with both oscillators in the silent phase and assume that cell 1 jumps up. If the synapse is direct, then s2 jumps instantly (with respect to the slow timescale) to s2 D 1. The effect of this is to move cell 2 instantly away from its Žring threshold, thus destabilizing the synchronous solution. Indirect synapses are needed for stability since they provide a window of opportunity for both cells to jump up during the same cycle. However, this does not guarantee that the synchronous solution is stable. One must still show that cells that are initially close together are brought closer together, or compressed, as they evolve in phase-space.

610

J. Rubin and D. Terman

It is not at all obvious how to deŽne compression. We need to demonstrate that the cells are brought closer together; however, this requires that we have a notion of distance between the cells. There are several possible metrics, each with certain advantages on different pieces of the solution. One obvious metric is the Euclidean distance between the points in phase-space corresponding to the cells. It is sometimes convenient to work with a time metric; the “distance” between the cells is then the time it takes for one cell to reach the initial position of the other cell. This was used previously in Somers & Kopell (1993), Terman & Wang (1995), Terman et al. (1998), and LoFaro & Kopell (in press). Here, we describe several mechanisms for compression, each corresponding to a different piece of the singular solution. These mechanisms illustrate how the geometry of the two-dimensional slow subsystem, notably its curves of knees, allows for compression in the presence of inhibitory coupling. By identifying the compression mechanisms, we can then understand how changing parameters in the equations inuences the stability of solutions. We can also compare the robustness of the compression mechanisms for this network with that for other networks with other architectures. 3.3.1 The Jump Up. Suppose that cell 1 lies on the jump-up curve when t D 0. After cell 1 Žres, there is a delay in the onset of inhibition. We assume that cell 2 begins in the silent phase so close to the jump-up curve that it Žres before it feels this inhibition. Suppose that cell 2 Žres when t D T0 . We now need to make an assumption on the nonlinearities. Let ( y¤ , w¤ ) be the point where the synchronous solution jumps up. We assume that (A) |GL ( y¤ , w¤ , 0)| < |GR ( y¤ , w¤ , 0)|

and

| HL ( y¤ , w¤ , 0)| < |HR ( y¤ , w¤ , 0)|

Note that this assumption implies that the w and y coordinates of both cells change at a faster rate in the active phase after the jump up than in the silent phase before the jump up. This is certainly satisŽed for the example described in appendix A and is similar to assumptions in previous work (see, for example, Somers & Kopell, 1993, where the notion of fast threshold modulation is introduced). There are now several cases to consider depending on the orientation of the cells both before and after they jump up. We work out two of these in detail. These are the cases that arise most often for the system described in appendix A. Similar analysis applies to other cases. Assume that w1 (0) < w2 ( T0 ) < w2 (0) and y2 (0) < y1 ( T0 ) < y1 (0) (see Figure 4A). These assumptions, together with the lemma, imply that |y1 (T 0 )¡ y2 (T0 )| < |y1 (0)¡ y2 (0)| so there is compression in the y-coordinates after the jumps. From assumption A, there is also compression in a time metric corresponding to the y-coordinate. For each t0 , let r y (t0 ) be the time it takes for cell 2 to evolve from its position at t D t0 until its y-coordinate is that of cell 1 when t D t0 . It then follows that r y ( T0 ) < r y (0).

Population Rhythms

611

Figure 4: Compression mechanisms for mutually coupled compound cells. (A) Compression in the time metric r w can occur between mutually coupled compound cells in the jump-up. Note that w1 (0) < w2 (T0 ) < w2 (0) and y2 (0) < y1 (T0 ) < y1 (0). The Euclidean distances dw (0), dw (T0 ) are used to compute the time metrics r w (0), r w (T0 ), respectively. (B) A reversal of orientation in the active phase (solid lines) can lead to compression in the jump-down; the dashed line indicates evolution of cell 1 in the silent phase. (C) Numerically computed trajectories of a pair of mutually coupled compound cells undergoing order reversal. Cells 1, 2 correspond to c1 , c2 respectively in (B). Parameter values are given in appendix A.

612

J. Rubin and D. Terman

We now show that there is also a compression in the time metric corresponding to the w-coordinate. This is denoted by r w (t ). Let a¡ D |GL ( y¤ , w¤ , 0)| and a C D |GR ( y¤ , w¤ , 0)|. Then w2 (0) ¡ w1 (0) w2 (0) ¡ w2 (T0 ) w2 ( T0 ) ¡ w1 (0) D C a¡ a¡ a¡ w2 ( T0 ) ¡ w1 (0) w2 ( T0 ) ¡ w1 (0) ¼ T0 C > T0 C a¡ aC w1 (0) ¡ w1 ( T0 ) w2 (T 0 ) ¡ w1 (0) ¼ C aC aC ¼ r w ( T0 ).

r w (0) ¼

(3.7)

Now suppose that w1 (0) < w2 ( T0 ) < w2 (0) and y1 ( T0 ) < y2 (0). The exact same calculation given in equation 3.7 shows that there is compression in the time metric r w across the jump up. A simple calculation also shows that there is compression in r y . 3.3.2 The Active Phase. Next assume that the cells are active with xi > hx . Then each ( yi , wi ) satisŽes equation 3.3. It is easy to see why the cells are compressed in the Euclidean metric if we make some simplifying assumptions concerning the nonlinear functions g and h. These assumptions arise naturally if one considers the network in appendix A; it is also a simple matter to extend this analysis to more general systems. Suppose that g and h are of the form g( v, w) D ( w1 (v) ¡ w) / tw ( v) and h(v, y) D ( y1 ( v) ¡ y) / ty (v). Note that while in the active phase, y1 ( v) and w1 ( v) are very small. Moreover, ty ( v) and tw ( v) are nearly constant. We assume here that while in the active phase, g(v, w) D ¡w / tw and h( v, y) D ¡y / ty , where tw and ty are positive constants. It follows that each ( wi , yi ) satisŽes simple linear equations. If we ignore the jump-down curve, then each slow variable decays to 0 at an exponential rate. In particular, the distance between the cells decays exponentially. Actually, more is true. Each ( yi , wi ) approaches the origin tangent to the weakest eigendirection. Now suppose that the jump-down curve passes close to the origin. The Euclidean distance between the cells still decreases exponentially, and both cells jump down at nearly the same point. This is the point where the jumpdown curve crosses the weakest eigendirection. We note that there is also a more subtle source of compression while the cells are active. There will be some period of time when cell 2 receives inhibition but cell 1 does not; that is, s1 D sA but s2 D 0. During this time, the ( yi , wi ) satisfy different equations. It is then possible that the trajectories ( yi (t ), wi (t )) cross in the slow phase-space. This leads to a reversal of orientation between the cells, as shown in Figure 4B. Next, we discuss why a reversal of orientation can lead to compression in the cells’ trajectories.

Population Rhythms

613

3.3.3 The Jump Down. We now show that if the cells reverse their orientation while in the active phase, this can lead to a form of compression after the cells jump down. Let Ti be the time when cell i jumps down. As above, assume that cell 1 jumped up Žrst with w1 (0) < w2 (0). We will assume that w1 (t ) < w2 (t) as long both cells are active. Moreover, from remark 1, the jump-down curve is nearly horizontal. Hence, cell 1 jumps down Žrst; that is, T1 < T2 . If the cells’ trajectories cross while in the active phase, then y1 ( T1 ) < y2 (T 2 ). This is shown in Figure 4B. Let r yA ( T1 ) be the time it would take for the solution of equation 3.3 starting at ( y2 ( T1 ), w2 ( T1 )) to reach the ycoordinate y1 ( T1 ). It follows that r yA ( T1 ) > T2 ¡ T1 . For T1 < t < T2 , cell 1 evolves in the silent phase with y increasing, while cell 2 evolves in the active phase with y decreasing. If y1 (T2 ) < y2 ( T2 ), then | y2 ( T2 ) ¡ y1 (T 2 )| < | y2 ( T1 ) ¡ y1 ( T1 )| so there is compression in the ycoordinates of the cells across the jump. Now suppose that y1 ( T2 ) > y2 ( T2 ), as shown in Figure 4B. Let r yS (T2 ) be the time it would take for the solution of equation 3.2 starting at ( y2 ( T2 ), w2 (T 2 )) to reach the y-coordinate of cell 1. Since y2 ( T1 ) > y1 ( T1 ), it follows thatr yS ( T2 ) < T2 ¡T1 . We have now demonstrated that r yS (T2 ) < T2 ¡ T1 < r yA ( T1 ). That is, there is compression in the time metric corresponding to the y-coordinate. We note that since the jumpdown curve is nearly horizontal, any compression in the w-coordinates is, to Žrst order, neutral. A numerical example of orientation reversal in two cells’ trajectories in ( y, w)-space is shown in Figure 4C. At the top of the Žgure, the cells are in the silent phase. Each cell jumps up where the corresponding y0i D 0; chronologically, cell 1 jumps up Žrst. In the active phase, the trajectories cross, because the cells experience different levels of inhibition; cell 2 receives inhibition Žrst. The paths cross again at the bottom left of the Žgure after the leading cell, cell 1, falls down to the silent phase. 3.3.4 The Silent Phase. Suppose that both cells lie in the silent phase with xi < h syn . Then each ( yi , wi ) satisŽes equation 3.2 until one of them reaches the jump-up curve. We now deŽne a metric between the cells and, in appendix B, we analyze how to choose parameters to guarantee that the metric decreases as the cells evolve in the silent phase. This metric is similar to that introduced by Terman et al. (1998). Suppose that cell 1 reaches the jump-up curve Žrst, and this is at the point ( y¤1 , w¤1 ). (See Figure 13 in appendix B.) Fix some time t0 , and let t wL0 be the physical translate of the jump-up curve such that ( y¤1 , w¤1 ) is translated to the point ( y1 (t0 ), w1 (t0 )). Then the “distance” between ( y1 (t0 ), w1 (t0 )) and ( y2 (t0 ), w2 (t0 )) is the time it takes for the solution of equation 3.2, which begins at ( y2 (t0 ), w2 (t0 )), to cross wtL0 . This is certainly well deŽned as long as the two cells are sufŽciently close to each other.

614

J. Rubin and D. Terman

One can compute explicitly how this metric changes as the cells evolve in the silent phase. (The computation, rather technical, is in appendix B.) A more complete discussion of how this metric is used to prove the stability of the synchronous solution of two mutually coupled basic cells with slow synapses is given in Terman et al. (1998). 3.4 Further Remarks. The compression of cells can take place as the cells evolve along the multidimensional slow manifold or as they jump up or down. The compression during the jumping process depends on the geometry (or slope) of the curve of knees and the orientation of the cells both before and after the jumps. Parameters that determine this slope may therefore have subtle effects on the stability of the synchronous solution; gsyn is one such parameter (see equation 3.6). The results in Terman et al. (1998) provide precise conditions on combinations of parameters that ensure that the synchronous solution is stable when the inhibition is slow. Increasing gsyn, for example, may sometimes stabilize the synchronous solution; however, when other parameters satisfy a different relationship, increasing gsyn may destabilize the synchronous solution. The size of the domain of attraction of the synchronous solution is, to a large extent, determined by the delay in the onset of inhibition. The two cells are able to Žre together if the trailing cell lies within the window of opportunity determined by this delay. If the trailing cell lies outside this window, then the network typically exhibits antiphase behavior in which the cells take turns Žring, although other network behavior is possible. The system may crash, for example, since the completely quiescent state is asymptotically stable. In our analysis, we assumed that the cells and coupling are homogeneous. The effect of heterogeneities on mutually coupled basic cells with slow synapses was studied in several papers (Golomb & Rinzel, 1993; White, Chow, Ritt, Soto, & Kopell, 1998; Chow, 1998). These found that the synchronous solution is not very robust to mild levels of heterogeneities; a 5% variation in parameters was sufŽcient to destroy synchronous behavior. We have done a number of numerical simulations in order to study the effects of heterogeneities on the network considered in this section. Our numerical results are consistent with those in previous studies. 4 Globally Inhibitory Networks

We now consider the network described in section 2.3. Recall that in this network, E-cells excite J-cells, which in turn inhibit E-cells. The thalamic networks involved in sleep rhythms, discussed in section 5, are examples of such a network, with compound cells, to which the results of this section apply. We assume for now that there are just two E-cells, denoted by E1 and E2 , and there is one J-cell, which we denote as J. Larger networks are considered later. Initially, each cell is assumed to be a basic cell; generalization

Population Rhythms

615

for compound cells is discussed in section 4.4. The E-cells are identical to each other, but they may be different from the J. Each cell is also assumed to be excitable for Žxed levels of input. The system of equations corresponding to each Ei is v0i D f (vi , wi ) ¡ ginh sJ ( vi ¡ vinh ) w0i D 2 g( vi , wi ) s0i D a(1 ¡ si ) H (vi ¡ h ) ¡ bsi ,

(4.1)

while the equation for J is 1 v0J D fJ ( v J , wJ ) ¡ ( s1 C s2 ) gexc ( vJ ¡ vexc ) 2 w0J D 2 g J ( v J , wJ ) s0J D aJ (1 ¡ sJ ) H( v J ¡ h J ) ¡ 2 KJ sJ .

(4.2)

Here, each synapse is direct. Indirect synapses will be needed when we discuss the stability of solutions. Note that the inhibitory variable s J turns off on the slow timescale. The reason that we write the equations this way will become clear in the analysis. We assume that b D O(1); however, there is no a problem in extending the analysis if b D O(2 ). If vi > h , then si ! sA ´ aCb on the fast timescale. Two types of network behavior are shown in Figures 8 and 9. A synchronous solution, in which each cell Žres during every cycle, is shown in Figure 9. In Figure 8, each excitatory cell Žres every second cycle, while J Žres during every cycle. This type of solution is referred to as a clustered solution. In the following sections, we construct singular orbits corresponding to each of these solutions and then analyze their stability. The constructions then lead to conditions for when the different solutions exist and are stable. 4.1 Existence of the Synchronous Solution. We now construct a singular trajectory corresponding to a synchronous solution in phase space. As before, the trajectory for each cell lies on the left or right branch of a cubic nullcline during the silent and active phases. Which cubic a cell inhabits depends on the total synaptic input that the cell receives. Nullclines for the Ei are shown in Figure 5A and those for J in Figure 5B. Note in Figure 5A that the s J D 1 nullcline lies above the sJ D 0 nullcline, while in Figure 5B, the stot ´ 12 ( s1 C s2 ) D sA nullcline lies below the stot D 0 nullcline. This is because the Ei receive inhibition from J while J receives excitation from the Ei . We will make several assumptions concerning the ow in the following construction. These are justiŽed later. We begin with each cell in the active phase just after it has jumped up. These are the points labeled P0 and Q0 in Figure 5. Each Ei evolves down

616

J. Rubin and D. Terman

Figure 5: Nullclines for (A) E-cells and (B) J-cells in a globally inhibitory network with basic cells. The closed curves and points Pi , Q i correspond to the singular synchronous solution discussed in the text. Note that s J decays on the slow timescale.

the right branch of the sJ D 1 cubic, while J evolves down the right branch of the stot D sA cubic. We assume that the Ei have a shorter active phase than J, so each Ei reaches the right knee P 1 and jumps down to the point P2 before J jumps down. We also assume that at this time, J lies above the right knee of the stot D 0 cubic. J must then jump from the point Q1 to the point Q2 along the stot D 0 cubic. On the next piece of the solution, J moves down the right branch of the stot D 0 cubic while the Ei move up the left branch of the s J D 1 cubic. When J reaches the right knee Q3 , it jumps down to the point Q4 along the left branch of the stot D 0 cubic. Now the inhibition sJ to the E i starts to turn off on the slow timescale. Thus, the Ei do not jump to another cubic. Instead, the trajectory for the Ei moves upward, with increasing wi , until it crosses the w nullcline. Then each wi starts to decrease. If this orbit is able to reach a left knee, it jumps up to

Population Rhythms

617

Figure 6: Slow phase plane for an E-cell. The curve wL (sJ ) is the jump-up curve, which trajectories reach if K J is large enough (¡ ¢ ¡ ¢ ¡ path; cell jumps up from the point marked *). The dotted curve WF (s J ) consists of zeros of GL (w, sJ ) in system 4.3; trajectories tend to the stable critical point WF (0) as s J ! 0 for small K J (¡ ¢ ¢ ¡ ¢ ¢ ¡ path). Note that w0 < 0 for w > WF .

the active phase, and this completes one cycle of the synchronous solution. When the Ei jump up, J also jumps up if it lies above the left knee of the stot D sA cubic. We now derive more quantitative conditions for when the singular synchronous solution exists. It is not at all obvious, for example, why we needed to assume that the active phase of J is longer than that for the E i . It is also not clear what conditions are needed to ensure that the Ei are able to reach a jump-up curve and escape once they are released from inhibition. These two issues are actually closely related. We Žrst discuss how the Ei can reach the jump-up curve. For this, it is convenient to derive equations for the evolution of the slow variables (wi , sJ ) as was done in section 3.1. Let t D 2 t, denote the left branch of the cubic f (v, w) ¡ ginh s(v ¡ vinh ) D 0 by v D W L (w, s) and let GL ( w, s) ´ g(W L ( w, s), s). Then each ( wi , sJ ) satisŽes the slow equations, wP D GL ( w, sJ ) sPJ D ¡KJ sJ .

(4.3)

The phase plane corresponding to this system is illustrated in Figure 6. There are two important curves shown in the Žgure. The Žrst is the jumpup curve w D wL ( sJ ); this is the curve of “left knees.” The second curve, denoted by WF ( s J ), corresponds to the Žxed points of the Žrst two equations

618

J. Rubin and D. Terman

in 4.1 with the input s J held constant. This corresponds to the w-nullcline of equation 4.3. We need to determine when a solution (w(t), sJ (t )) of equation 4.3 beginning with s J (0) D 1 and w(0) < W F (1) can reach the jump-up curve wL ( sJ ). This is clearly impossible if WF (1) < wL (0), so we shall assume that WF (1) > wL (0). If w(0) > wL (0) and K J is sufŽciently large, the solution will certainly reach the jump-up curve; this is because the solution will be nearly vertical, as shown in Figure 6. If, on the other hand, KJ is too small, the solution will never be able to reach the jump-up curve. This is because the solution will slowly approach the curve WF (s J ) and lie very close to this curve as s J approaches zero. This is also shown in Figure 6. We conclude that the cells are able to escape the silent phase if the inhibitory synapses turn off sufŽciently quickly and the w-values of the cells are sufŽciently large when this deactivation begins. Escape is not possible for very slowly deactivating synapses (although it would be possible with slow deactivation if the cells were oscillatory for some levels of synaptic input). A biophysical interpretation of this is that escape is possible for GABAA synapses and will occur if the cell’s IT current is sufŽciently deinactivated when inhibition begins to wear off. We assume that KJ is large enough so that escape is possible. Choose Wesc so that the solution of equation 4.3 that begins with sJ (0) D 1 will be able to reach the jump-up curve only if w(0) > Wesc . The existence of the singular synchronous solution now depends on whether the Ei lie in the region where wi > Wesc when J jumps down to the silent phase. We claim that this requires that the active phase of J be sufŽciently long. One can give a simple estimate on how long this active phase must be as follows. Suppose that all the cells jump up when t D 0, the Ei jump down when t D tE , and J jumps down when t D tJ . We require that wi (tJ ) > W esc . Since the time the Ei spend in the silent phase before they are released from inhibition is tJ ¡tE , this implies that tJ ¡tE must be sufŽciently large. Hence, J’s active phase must be sufŽciently longer than the Ei ’s. More precisely, let wRK be the value of w at the right knee of the sJ D 1 cubic, and let tL be the time it takes for the solution of the Žrst equation in 4.1 with sJ D 1 to go from w D wRK to w D Wesc . We require that t J ¡ tE > tL .

(4.4)

4.2 Stability of the Singular Synchronous Solution. We now demonstrate that the synchronous solution is stable if the synapse s J is indirect and the active phase of J is sufŽciently long. We start with the Ei a small distance apart, just after both have jumped up to the active phase. Assume that this causes J to Žre. We will show that after one cycle, both of the Ei are so close that they must Žre together again. Moreover, there is a contraction in the distance between the E-cells during each cycle. The analysis proceeds as in the previous section. We assume that the active phases of the Ei are shorter than that of J, so that the Ei return to

Population Rhythms

619

the silent phase and proceed up the left branch of the sJ D 1 cubic before J jumps down. As the Ei move up this left branch, they approach the point PL where the s J D 1 cubic intersects the w-nullcline. (See Figure 5A.) If the active phase of J is sufŽciently long, then the Ei lie as close as we please to PL , and therefore to each other, when J jumps down. This is precisely what is required to guarantee that both will Žre together during the next cycle. While in the silent phase, the Ei approach PL at an exponential rate (in the slow timescale). This leads to a very strong compression of Euclidean distance between the cells while in the silent phase. This compression is certainly stronger than any possible expansion over the remainder of the cycle. After the J-cell falls down, s J decays on the slow timescale. This allows the J-cell to recover so that it can Žre when excited by the Žring of the E-cells, and the whole cycle repeats. We need to assume that sJ corresponds to an indirect synapse for the same reason as we did previously. When one of the E-cells Žres, this causes J to Žre, which sends inhibition back to the other E-cell. If s J is direct, this causes the second E-cell to be “stepped on” on the fast timescale, and the synchronous solution cannot be stable. Note that the time between the Žrings of the Ecells is determined by K J , the rate at which sJ decays. If KJ is large, the time between Žrings is short; it is then easier for the second cell to pass through the window of opportunity provided by the indirect synapse. Our analysis has shown that the dynamics of the J-cell can inuence the domain of attraction of the synchronous solution in several ways. If J’s active phase is long, then the E-cells lie close to each other, near PL , when J jumps down and releases them from inhibition. Moreover, if J recovers quickly in the silent phase, then KJ can be chosen to be large. Both factors make it easier for the E-cells, once they are released from inhibition, to pass through a window of opportunity and Žre during the same cycle. Hence, both enlarge the domain of attraction of the synchronous solution. There are important differences between the ways in which mutually coupled and globally inhibitory networks use inhibition to synchronize oscillations. In mutually coupled networks, a second slow variable is required for the existence of the synchronized solution; it allows the cells to escape from the silent phase. The second slow variable is also required for the compression of the cells as they evolve in phase space. The existence and stability of the synchronous solution in globally inhibitory networks, on the other hand, are controlled by the dynamics of the J-cell. If the Jcell’s active phase is long enough, then this pushes the E-cells, in their silent phase, to a position from which they can escape; moreover, this provides a strong compression of the E-cells. For this network, we require that the inhibition decays on the slow timescale; however, the reason is so that the J-cell can recover sufŽciently. The slow recovery is not needed to allow the E-cells to escape or for compression. In fact, the domain of stability of the Remark 2.

620

J. Rubin and D. Terman

synchronous solution is increased if the recovery of the J-cell and the decay of the synapses occur quickly, on the slow timescale. 4.3 Clustered Solution. We now describe the geometric construction of the singular antiphase, or clustered, solution. It sufŽces to consider half of a complete cycle. During this half-cycle, E1 Žres and returns to the initial position of E2 , J Žres and returns to its initial position, and E 2 evolves in the silent phase to the initial position of E1 . By symmetry, we can then continue the solution for another half-cycle with the roles of E1 and E2 reversed. When E1 jumps up, it forces J to jump up to the right branch of the stot D 12 sA cubic. Then E1 moves down the right branch of the sJ D 1 cubic, while J moves down the right branch of the stot D 12 sA cubic and E2 moves up the left branch of the s J D 1 cubic. We assume, as before, that E1 ’s active phase is shorter than J’s active phase, so E1 jumps down before J does so. It is possible that J lies below the right knee of the stot D 0 cubic at this time, in which case J also jumps down. If J lies above this right knee, then it moves down the right branch of the stot D 0 cubic until it reaches the right knee and then jumps down. During this time, both E1 and E2 move up the left branch of the s J D 1 cubic. After J jumps down, s J (t ) slowly decreases. If E 2 is able to reach the jump-up curve, then it Žres, and this completes the Žrst half-cycle of the singular solution. Suppose that t D tF when this occurs. For this to be onehalf of an antiphase solution, we need w2 (tF ) D w1 (0), w1 (tF ) D w2 (0), and wJ (tF ) D wJ (0). We now derive conditions for when the antiphase solution exists. These will imply that the active phase of J cannot be too long or too short, compared with the active phase of the Ei . If J’s active phase is too long, then the network exhibits synchronous behavior as described before. If J’s active phase is too short, then the system approaches the stable quiescent state. Suppose that E1 and J jump up when t D 0, E1 jumps down when t D tE , and J jumps down when t D t J . Let tL and Wesc be as deŽned in the previous section. To have a clustered solution, we require that

w1 (tJ ) < Wesc < w2 (tJ ).

(4.5)

The second inequality is necessary to allow E2 to Žre during the second halfcycle. The Žrst inequality guarantees that E1 does not Žre during this halfcycle. It follows from the deŽnitions that the Žrst inequality is equivalent to t J ¡ tE < tL .

(4.6)

Next we derive a similar expression for the second inequality in equation 4.5. For w0 < w1 , let r ( w0 , w1 ) to be the time it takes for a solution of the

Population Rhythms

621

Žrst equation in 4.3 with sJ D 1 to go from w0 to w1 . The second inequality is equivalent to r ( wRK , w2 (tJ )) > tL . Now, r ( wRK , w2 (tJ )) D r (wRK , w2 (0)) Cr (w2 (0), w2 (tJ )). Moreover, wRK D w1 (tE ) and w2 (0) D w1 (tF ). Hence, r ( wRK , w2 (tJ )) D r (w1 (tE ), w1 (tF )) Cr ( w2 (0), w2 (tJ )). Clearly, r ( w2 (0), w2 (tJ )) D tJ , because E 2 lies on the sJ D 1 cubic for 0 < t < tJ . It is not true that E1 lies on the sJ D 1 cubic for tE < t < tF ; however, if the Žrst equation in 4.3 is weakly dependent on sJ , then we have that r ( w1 (tE ), w1 (tF )) ¼ tF ¡ tE . In this case, the second inequality in equation 4.3 is equivalent to tF ¡ tE C t J > tL .

(4.7)

One can simplify this formula if the parameter KJ is rather large. In this case, tF ¡ tJ is small; that is, E2 escapes the silent phase as soon as it is released from inhibition. Then equation 4.7 is approximately equivalent to tJ >

1 (tE C tL ). 2

(4.8)

Combining equation 4.6 and 4.8 leads to the following condition for the existence of a clustered solution if the synaptic variable sJ turns off quickly: 1 (tE C tL ) < tJ < tE C tL . 2

(4.9)

It is possible for both stable synchrony and stable clustering to exist for the same parameter values. Note that the domain of stability of the synchronous solution is controlled to a large extent by the size of the delay caused by the indirect inhibitory synapses. If this delay is small, the domain of stability will also be small. In this case, the synchronous solution will still be stable; however, most solutions will converge to a stable clustered solution. Remark 3.

4.4 Globally Inhibitory Networks with Compound Cells. The discussion in the previous subsections generalizes to globally inhibitory networks with compound cells, such as the model thalamic network in the next section. The primary difference is that each cell contains an additional slow variable, yi , so it is necessary to consider a higher-dimensional slow phasespace. As a consequence, the jump-up curves of the previous subsections are

622

J. Rubin and D. Terman

Figure 7: Three-dimensional slow phase-space for a singular synchronous periodic orbit of the globally inhibitory network. Double (single) arrows denote evolution on the fast (slow) timescale, solid (dashed) lines indicate the silent (active) phase, and points Pi are as discussed in the text. The shaded region represents the jump-up surface w D wL (y, sJ ).

replaced by jump-up surfaces fw D wa ( y, sJ )g, a D L or R, where wa ( y, sJ ) is as in the lemma in section 3. Figure 7 illustrates the evolution of the slow variables ( wi , yi , sJ ) for the singular synchronous solution. We begin at the point labeled P1 on the jumpup surface w D wL ( y, sJ ). The E-cells then jump up, and this forces J to jump up. Hence, sJ ! 1. This corresponds to the segment in Figure 7 that connects P1 to the point P2 on the s J D 1 surface. Each cell then evolves in the active phase with s J D 1. As before, we assume that the active phases of the E-cells are shorter than that of J. Hence, the E-cells jump down when the ( wi , yi ) reach the jump-down curve wi D wR ( yi , 1). This is at the point labeled P3 in Figure 7. While J lies in the active phase, the E-cells evolve in the silent phase, but sJ D 1 still holds. At P4 , J jumps down and the ( wi , yi , sJ ) evolve

Population Rhythms

623

until they reach the jump-up curve. This then completes one cycle of the singular solution. The stability analysis proceeds just as in section 4.2. What is crucial for stability is that J remains active long enough. The E-cells then approach the stable Žxed point on the left branch of the sJ D 1 surface while J is active. This provides the compression needed for stability. The construction of a clustered solution is also very similar to that described in section 4.3. We do not describe the construction here; some comments are given in the next section. 4.5 Further Remarks. The geometric constructions of the synchronous and clustered solutions extend in a straightforward manner to globally inhibitory networks with an arbitrary number of excitatory cells Ei . Of course, in a larger network there are more possibilities for clustered solutions; however, if each cluster contains (approximately) the same number of cells, then inequalities similar to equation 4.9 must be satisŽed. This is similar to analysis of Terman and Wang (1995), which yields precise conditions for the existence of clustered states in a locally excitatory and globally inhibitory network model for scene segmentation. Further analysis of clustered solutions in inhibitory networks is given in Rubin and Terman (1999). The analysis leads to simple formulas for the periods of the synchronous and clustered solutions. Let tJ be, as above, the time cell J spends in the active phase, and let tS be the time for the E-cells to reach the jump-up curve after the J-cell jumps down. Then the period of the synchronous solution is simply t J C tS . Now tJ is determined by the dynamics of the J-cell, while tS is primarily controlled by the rate at which the synapses turn off; this is the parameter KJ in equation 4.2. Also see Figure 11. Other parameters play a secondary role. Note, for example, that the parameter gsyn inuences the period by controlling the slope of the jump-up curve, as shown in equation 3.6. The synchronous and clustered solutions can exist only if trajectories are able to escape from the silent phase. Previous work on mutually inhibitory neurons has emphasized the distinction between “release” and “escape” in producing both synchronous and antiphase solutions (Wang & Rinzel, 1993). “Release” refers to the case in which the active phase of one cell ends and this releases another silent cell from inhibition. “Escape” refers to the case in which the dynamics of the inactive cell allows it to Žre even if that cell receives inhibition. As pointed out in Terman et al. (1998), there is no clear distinction between escape and release when there are multiple slow processes. A silent E-cell is in some sense released when the J-cell jumps down to the silent phase. The rhythms can continue, however, only if this E-cell is able to escape the silent phase. Here we are assuming that each E-cell is excitable for constant levels of inhibition. Both escape and release are therefore needed to maintain oscillations.

624

J. Rubin and D. Terman

5 Thalamic Network

The model for the thalamic spindle sleep rhythm falls into the framework of the network analyzed in the preceding section. The two populations of cells in the model are the thalamocortical relay (TC) cells and the thalamic reticularis (RE) cells, corresponding to the E-cells and J-cells, respectively. One difference between the spindle model and those considered earlier is that the spindle model contains numerous RE as well as TC cells. The RE cells communicate with each other through fast inhibitory synapses, as illustrated in the network shown in Figure 2. During spindling, the network exhibits behavior similar to the clustered solution discussed in the previous section. The RE population is synchronized, while the TC cells break up into groups; cells within each group are synchronized, while cells within different groups are desynchronized. The network also exhibits completely synchronized rhythms. The synchronized rhythms arise, for example, when fast inhibition is removed from the entire network or from between the RE cells only. Recent results have also shown that the synchrony can arise if the RE population receives additional phasic excitation, corresponding to cortical input, at the delta frequency. Hence, the network can transform from clustering to synchronized behavior without any change in the inhibitory synapses. In this section, we demonstrate how the geometric analysis helps to explain the dynamical mechanisms responsible for these rhythms and transitions between them. We begin by presenting a concrete model for the sleep rhythms and then present results of numerical simulations of this model. The numerical simulations clearly show that solutions of the model behave as predicted by the singular perturbation analysis; in particular, solutions jump up and jump down when they reach a curve (or surface) of knees. This conŽrms that the decomposition into fast and slow variables, as described in the previous section, provides the correct singular perturbation framework for analyzing these rhythms. We also demonstrate how the geometric analysis leads to quantitative statements concerning the behavior of solutions. In particular, the analysis predicts correctly how the frequency of oscillations depends on parameters. The analysis also leads to precise conditions for when the network exhibits either synchronous or clustered solutions. For example, factors that enable the TC cells to synchronize are a long RE active phase, a relatively fast RE recovery, and a fast decay of inhibition. Some of these factors are inconsistent with mechanisms for synchronization in mutually coupled inhibitory networks. Finally, the analysis clariŽes the roles of the various intrinsic and synaptic currents in generating a particular rhythm. We illustrate this in section 5.3, where we consider the role of the sag current, which also differs from the role of the second slow intrinsic current in mutually coupled networks.

Population Rhythms

625

5.1 Model. The following model contains many parameters and nonlinear functions (these are given in appendix A). The cells are modeled using the Hodgkin-Huxley formalism (Hodgkin & Huxley, 1952); the equations are very similar to those in Golomb et al. (1994). The equations of each TC cell are:

v0i D ¡IT (vi , hi ) ¡ Isag ( vi , ri ) ¡ IL ( vi ) ¡ IA ¡ IB h0i D ( h1 ( vi ) ¡ hi ) / th (vi ) r0i D ( r1 ( vi ) ¡ ri ) / tr ( vi ).

(5.1)

This describes a compound cell, with hi and ri corresponding to w and y, respectively, and 2 absorbed in th , tr rather than mentioned explicitly. The terms IT , Isag , and IL are intrinsic currents; they are given by IT ( v, h) D gCa m21 ( v) h( v ¡ vCa ), Isag ( v, r) D gsag r( v ¡ vsag ), and IL ( v) D gL (v ¡ vL ). The terms IA and IB represent the fast (GABAA ) and slow (GABAB ) inhibitory input from the RE cells. We model the fast inhibition IA as in previous P j sections; that is, IA D gA ( vi ¡ vA ) N1TR sA , where g A and vA are the maximal conductance and the reversal potential of the synaptic current. The sum is over all RE cells that send input to the ith TC cell and NTR represents the maximum number of RE cells that send inhibition to a single TC cell. Each j

synaptic variable sA satisŽes the Žrst-order equation j 0

j

j

j

sA D aR (1 ¡ sA ) H( vR ¡ h R ) ¡ bR sA ,

(5.2)

j

where vR is the membrane potential variable of the jth RE cell. Motivated by recent experiments (Destexhe, Bal, McCormick, & Sejnowski, 1996; Destexhe & Sejnowski, 1997), we model the slow inhibition IB somewhat differently from IA . We Žrst discuss, however, the model for the RE cells. The equations of each RE cell are: 0 viR D ¡IRT ( viR , hiR ) ¡ IAHP ( viR , mi ) ¡ IRL ( viR ) ¡ IRA ¡ IE 0 hiR D (hR1 ( viR ) ¡ hiR ) / tRh (viR )

mi 0 D m 1 [Ca]i (1 ¡ mi ) ¡ m 2 mi

[Ca]0i D ¡ºIRT ¡ c [Ca]i .

(5.3)

The IRT , IAHP , and IRL represent intrinsic currents. These are given by IRT ( v, h) D gRCa m2R1 (v) h( v ¡ vRCa ), IAHP (v, m) D gAHP m( v ¡ vK ), and IRL ( v) D gRL ( v ¡ vRL ). More details concerning the biophysical signiŽcance of each term are given in Golomb et al. (1994) and Terman et al. (1996). In equation 5.3, IRA represents the inhibitory input from other RE cells. It P j is modeled as IRA D gRA (viR ¡ vRA ) N1 sRA where the sum is over all RE RR

j

cells that send input to the ith RE cell. Each synaptic variable sRA satisŽes a

626

J. Rubin and D. Terman

Žrst-order equation similar to 5.2. The term IE represents excitatory (AMPA) P j input from the TC cells and is expressed as IE D gE ( viR ¡ vE ) N1 sE , where RT the sum is over all TC cells that send excitatory input to the ith RE cell. The j

synaptic variables sE are fast and also satisfy Žrst-order equations similar to 5.2. It remains to discuss how we model the slow inhibitory current IB . Sims4

ilarly to Destexhe et al. (1996), we assume that IB D gB s4 biCl (vi ¡ vB ) where sbi , along with variable xbi , satisŽes

bi

s0bi D k1 H( xbi ¡ hxb )(1 ¡ sbi ) ¡ k2 sbi i k3 hX x0bi D H( viR ¡ h Rb ) (1 ¡ xbi ) ¡ k4 xbi . NTR The parameters are such that xbi can become activated (i.e., exceed hxb) only if a sufŽciently large number of RE cells have their membrane potentials viR above the thresholdh Rb . The threshold is chosen rather large so the RE bursts must be sufŽciently powerful to activate xbi . Once xbi becomes activated, it turns on the synaptic variable sbi ; the expression s4bi in IB further delays the effect of the inhibition on the postsynaptic cell. 5.2 Numerical Simulations. A clustered solution is shown in Figure 8. There are three RE cells in this example, and they oscillate in synchrony at about 12.5 Hz; one of the RE cells is shown in Figure 8A. The RE cells synchronize due to excitation from the TC population. (This is discussed in more detail in section 5.3.) There are six TC cells, and they form two clusters, each oscillating at half of the RE oscillation frequency, as shown in Figure 8B. In Figure 8C, we show the time courses of the fast (sA ) and slow (sb ) inhibitory synaptic variables, respectively. Note that the fast inhibition activates during every cycle, providing the hyperpolarizing current needed to deinactivate each TC cell’s IT current. The fast inhibition is also needed to desynchronize the TC cells so they can form clusters. The slow current IB never activates during this solution because the RE cells do not Žre powerful enough bursts; that is, the membrane potentials viR do not rise above the threshold hRb D ¡25 mV long enough to activate the variables xbi . A synchronous solution is shown in Figure 9. The parameters are exactly as in Figure 8 except we set gRA D 0; that is, we have turned off the fast inhibition between RE cells. Note in Figure 9 that each TC cell Žres during every cycle along with the RE cells. The slow inhibitory current IB now activates during every cycle. Comparing the slow inhibitory variable with the fast inhibitory variable in Figure 9C, we see that the slow inhibition stays on longer and both turns on and turns off more gradually. Geometric analysis is useful in understanding why removing fast inhibition allows slow inhibition to activate and why this leads to synchronization of the network. This is discussed in the next subsection.

Population Rhythms

627

Figure 8: Numerical solution, with two TC clusters, of the thalamic network (parameter values are given in appendix A). Voltages are in mV and time is in msec. (Top) RE cell (the time course of which matches that of the RE population of three cells). (Middle) TC population of six cells, forming two clusters of three cells each. (Bottom) Inhibitory synaptic variables sA (dashed) and sb (solid). Note that sb ´ 0, since the RE cell bursts are not powerful enough to activate slow inhibition in this case.

In Figure 10 we plot the trajectory ( hi (t ), ri (t ), sb (t)) corresponding to the synchronous solution shown in Figure 9. We also plot the numerically computed surface of knees corresponding to the jump-up points; there is a similar jump-down surface, but it is not shown. The behavior of the trajectory is consistent with that discussed in the previous section. For example, the cells jump up when the trajectory crosses the jump-up surface of knees (approximately, since 2 6 D 0 numerically) (see Rubin & Terman, 1998, for a similar plot for the clustered solution). Figure 11 shows a plot of the period of synchronous TC oscillations as the rate of decay of inhibition, the rate of change of h, and the parameter gB are

628

J. Rubin and D. Terman

Figure 9: Numerical synchronous solution of the thalamic network (parameter values are given in appendix A). Voltages are in mV and time is in msec. (Top) RE cell (the time course of which matches that of the RE population of three cells). (Middle) TC population of six cells (synchronized). (Bottom) Inhibitory synaptic variables sA (dashed) and sb (solid). Note that sA turns on and off faster than sb , which in turn stays on longer.

separately varied. These relations can be derived from our analysis, as can predictions about the dependence of the period on other model parameters. As noted earlier, the period is given by the duration of the RE active phase (tJ ) plus the time it takes for inhibition to decay sufŽciently that the TC cells can jump up (tS ); hence, the period drops relatively sharply as the inhibitory decay rate increases. The same strong dependence, which our analysis explains, was found in the modeling work of Destexhe (1998). As the rate of change of h increases, there is essentially no change in the length of the TC silent phase (not shown) or in the period. This reects the fact that over the parameter range considered, the TC cells are compressed close to their silent phase rest state while the RE cells are active, and then the TC cells

Population Rhythms

629

Figure 10: Numerical trajectory of a TC cell, together with jump-up surface of knees (shaded), in the (h, r, sb ) slow phase-space. The TC cell shown belongs to the synchronized population of six TC cells shown in Figure 9. Note that sb does not immediately increase when the cell jumps up, since we have taken slow inhibitory synapses to be indirect.

evolve with little change in h after the RE cells jump down. Interestingly, gB has little effect on period because it has little inuence on tJ , tS . The mild effect that does occur is due to the subtle inuence of gB on the slope of the curve of knees for the TC silent phase. This actually causes the period to increase as gB , and hence the strength of slow inhibitory coupling, increases (see also Golomb et al., 1994). Our numerical studies show that the synchronous and clustered solutions are robust to moderate levels of heterogeneities and variations in the parameters. This is due to the strong TC compression provided by the RE cells. For example, these solutions were not affected by heterogeneities of about 20% in sag conductances and about 5% in the TC IT conductance.

630

J. Rubin and D. Terman

Figure 11: Period of synchronous TC oscillations versus rate of decay of inhibition (k2 D bR , solid curve), rate of change of h (dashed line), and gB (dotted line) for the population shown in Figure 9. The parameters k2 , bR were varied from 0.04 to 0.14 and were held at 0.1 for the other two curves. A parameter w was factored out from th0 , th1 in the h-equation and was varied from 1.5 to 2.5; w was held at 2.0 for the other curves. The parameter gB was varied from 0.03 to 0.07 and was held at 0.05 for the other two curves. Other parameter values as given in appendix A.

Stronger TC heterogeneities tend to promote TC clustering in this model. In the resultant patterns, TC cells with similar properties Žre together, independent of the way they are initially perturbed from synchrony. 5.3 Further Implications of the Analysis.

5.3.1 Removing Fast Inhibition. Fast inhibition occurs in two places: the RE cells inhibit themselves as well as the TC cells. Removing fast inhibition has different consequences for each of these synaptic connections, and

Population Rhythms

631

both of these help to synchronize the TC cells (see also Golomb et al., 1994; Destexhe & Sejnowski, 1997). Removing the RE-TC fast inhibition is clearly helpful for synchronization among the TC cells. This holds because the fast inhibition has a very short rise time, which follows since the fast inhibitory synapses are direct in equations 5.1 and 5.2. This short rise time corresponds to a small domain of attraction of the synchronous solution. In fact, it is precisely this inhibition that is responsible for desynchronizing the TC cells during the clustered solution. Removing the RE-RE fast inhibition appears to be even more crucial for synchronizing the TC cells (see also Steriade, McCormick, & Sejnowski, 1993; Huguenard & Prince, 1994; Destexhe, Bal, McCormick, & Steriade, 1996; Destexhe, 1998). This allows the RE cells to Žre longer, more powerful bursts, which activate the slow inhibitory current IB . The analysis in section 4 demonstrates that long, powerful bursting of the RE cells is needed for the TC cells to synchronize, unless the desynchronizing effect of inhibition is somehow removed as discussed in Terman et al. (1996). Our numerical simulations (e.g., Figures 9–12) show, in fact, that the TC cells will synchronize even if fast inhibition is removed from within the RE population but not from the RE-TC connections. Why removal of inhibition leads to stronger RE bursts can be easily understood by our analysis of trajectories in phase-space. This removal forces the RE cells to lie on the right branch of a different cubic while in the active phase. The cubic of the disinhibited cells lies below the cubic of the cells with inhibition. The disinhibited cells therefore jump up to larger values of membrane potential; moreover, their jump-down point (right knee) lies below the jump-down point of the inhibited cells. The disinhibited RE cells therefore have a longer active phase. Note that the slow inhibitory current IB , activated when RE-RE fast inhibition is removed, has slower rise and decay times than IA . The slow rise time enhances the domain of attraction of the synchronous solution by expanding the window of opportunity. The slow decay time can help to bring the cells closer together while in the silent phase, as discussed in section 3.3 and appendix B. Hence, both effects improve the ability of the TC cells to synchronize in the absence of fast inhibition. 5.3.2 Role of the Sag Current. We view the TC cells as examples of compound cells. If one removes the sag current Isag or keeps it constant, then they become basic cells. Hence, questions concerning the differences between networks with basic or compound cells are closely related to issues pertaining to the role of the sag current. The analysis in section 3 shows that in mutually coupled networks, whether the cells are basic or compound is signiŽcant. Added complexity allows the cells to escape from the silent phase and also leads to possible compression mechanisms. In globally inhibitory networks, the dynamical mechanisms responsible for the synchronous and clustered solutions are basically the same for basic or compound cells. We

632

J. Rubin and D. Terman

therefore do not view the sag current as playing a direct role in the generation of these rhythms. Recall that oscillations arise when the cells are capable of escaping from the silent phase. The sag current helps modulate the intrinsic properties of the TC cells, and this can determine whether escape is even possible. From a more biophysical viewpoint, the sag current depolarizes the TC cells while they are silent. Hence, a stronger sag current raises the TC’s resting potential. If this resting potential is too large, then inhibitory input from the RE cells may not be capable of hyperpolarizing the TC cells sufŽciently to deinactivate the IT current. If this is the case, then the TC cells will not be able to Žre (i.e., escape the silent phase). We note that this relates closely to some theories about mechanisms responsible for waxing and waning behavior during the spindle rhythm (Destexhe, Babloyantz, & Sejnowski, 1993; Bal & McCormick, 1996). Within the range where escape is possible, increasing the sag current tends to enhance the TC synchronization. This occurs because the increased sag current tends to depolarize the TC cells more rapidly, leading to stronger compression. Note that while the sag current is depolarizing during the silent phase, it is hyperpolarizing in the active phase. Hence, increasing the sag conductance lowers the left branch of the cubic corresponding to the TC cells but raises the right branch of the cubic. This decreases the voltage level to which TC cells jump up and hence the overall amplitude of TC oscillations (see Figure 12); at jump-down, the sag current is essentially deactivated, so the size of gsag has little effect there. Increasing gsag also tends to decrease the level of deinactivation of IT at Žring, as we can show using arguments similar to the proof of the lemma in section 3, and to decrease the period of the TC cells (see Figure 12). To analyze this dependence of the period more carefully, note that the evolution of the inactivation variable h for IT is almost linear for most of the silent phase; we can model this by h0 D w (1 ¡ h), independent of the inhibition level. From this equation, we can approximate the time tk it takes for the cell to reach the curve of knees. This depends on gsag , since the location of the curve of knees does. We Žnd that tk ¼ ¡ w1 ln(1 Ch f ¡ hk ), where h f is the h-value when the cell falls down and hk is the h-value when it hits the curve of knees to jump up. Incorporating the dependence of h f and hk on gsag implies that @tk / @gsag < 0; in fact, since h f ¡ hk is small relative to 1 and h f , hk depend close to linearly on gsag , the dependence of tk on gsag is almost linear as well. An analogous argument shows that the time spent in the active phase also has a roughly linear relation to gsag . These nearly linear dependencies sum to yield a nearly linear relationship between overall oscillation frequency and gsag , as appears in Figure 12. 5.3.3 Synchronization Among the RE Cells. The RE cells are synchronized during the spindle rhythm, primarily due to the excitation they receive from

Population Rhythms

633

Figure 12: The effect of gsag on synchronous TC oscillations. Six TC cells and three RE cells were simulated with parameter values given in appendix A.

the TC cells. Although only about half of the TC cells Žre during each cycle, if each TC cell sends excitation to several RE cells, then every RE cell will receive a sufŽcient amount of excitation to Žre during every cycle. Experiments have also shown that the RE cells can sustain synchronized rhythms when these cells are completely isolated (Steriade, Domich, Oakson, & Deschˆenes, 1987; Steriade, Jones, & Llin´as, 1990). In fact, these are the experiments that motivated many of the theoretical studies concerning how synchronization can arise in a population of inhibitory cells. If one considers the RE cells to be modeled as basic cells, then the conclusion of these studies is that synchronization is possible only if the decay of inhibition is sufŽciently slow. However, modeling the RE cells as in equation 5.3, we consider them to be compound cells; the slow variables are hiR and mi . We then conclude from our analysis of mutually coupled inhibitory cells in section 3 that synchronization is possible even if the inhibition decays quickly.

634

J. Rubin and D. Terman

5.3.4 Cortical Inputs. Our results clearly demonstrate that globally inhibitory networks provide a natural framework for analyzing RE-TC interactions in the generation of thalamic sleep rhythms. These results are also consistent with recent work about the impact of cortical inputs in the development and manifestation of the sleep rhythms (Steriade, Curro´ Dossi, & Nunez, ˜ 1991; Steriade, Nunez, ˜ & Amzica, 1993; Steriade, 1994; Contreras, Destexhe, Sejnowski, & Steriade, 1996; Timofeev & Steriade, 1996; Contreras, Destexhe, & Steriade, 1997; Destexhe, Contreras, & Steriade, 1998). Here we review some of this work and briey describe how it relates to our analysis. Recent work has emphasized the importance of the cortex in the transformation of spindle oscillations into spike-and-wave-like (SW) epileptiform oscillations in the thalamus (Steriade, 1994; Steriade, Contreras, & Amzica, 1994; Contreras, Destexhe, & Steriade, 1996; Destexhe, Contreras, Sejnowksi, & Steriade, 1996; Destexhe & Sejnowski, 1997; Steriade, Contreras, & Amzica, 1997; Destexhe, 1998). The experiments and modeling described in these works have suggested that this can arise without the removal of fast inhibition from the thalamus. Recall that in the mechanism we have discussed, disinhibition of the RE cells leads to more powerful RE bursts, and this permits the TC cells to synchronize. Our analysis supports the Žnding (Contreras, Destexhe & Steriade, 1996; Destexhe et al., 1996; Steriade et al., 1997; Destexhe, 1998) that if one does not remove the fast inhibition among the RE cells but instead induces sufŽciently strong excitation from the cortex, then this will have the same effect: the RE cells will Žre more powerful bursts because of the additional excitation (their cubics are lowered). Hence, GABAB inhibition of the TC cells ensues and the TC cells can synchronize, even though they receive fast inhibition from the RE cells. In particular, this explains the mechanism behind the results of Destexhe (1998), in which sufŽciently strong corticothalamic excitation (achieved by blocking only cortical GABAA ) is found to be crucial for triggering powerful RE bursts. These bursts in turn lead to the activation of GABAB in the thalamus and the generation of synchronized » 3 Hz oscillations in TC and RE cells. It is interesting to note that SW discharges of pyramidal cells and interneurons in the cortex, as modeled by Destexhe, apparently result from a related interaction: excitation from TC cells induces the “spike” of prolonged Žring of pyramidal cells and interneurons in the cortex, and resulting GABAB inhibition from the interneurons to the pyramidal cells enhances synchronization and maintains the subsequent “wave” of inactivity until the TC rebound and restart the cycle. Our analysis has shown that it is possible for the TC cells to synchronize even in the presence of fast inhibition if the RE cell bursts are powerful enough. This does not, however, contradict the argument in Terman et al. (1996) that the effect of fast inhibition to the TC cells must be removed during delta. Since the RE cells Žre at the slow rhythm in delta, their bursts

Population Rhythms

635

are completely absent (after the Žrst cycle) while the TC cells Žre. Hence, the RE cells do not produce the powerful bursts that would be needed to synchronize the TC cells if the effect of the fast inhibition remained. 5.3.5 Variations in Synchronous Oscillations. The thalamic network may exhibit other types of solutions than those discussed. This is because assumptions required for the geometric constructions of the solutions may not be satisŽed for some ranges of parameter values. Subtle variations in network behavior can arise as parameters that change the underlying geometry of phase space are varied. One possibility that arises, for certain biophysically relevant parameter values, is that the right knees of each RE cell’s family of cubics lie very close to the left knees of the RE cell’s cubics. In this case, each RE cell recovers almost instantly on jump-down (after a long, gradual decrease in vR during the active phase). If the TC cells approach sufŽciently close to the stable equilibrium in the silent phase while the RE cells are still active, then they can all Žre as soon as they are disinhibited by RE jump-down. Hence, stable synchrony can occur here without activation of slow inhibition, even if fast inhibition decays on the fast timescale, since slow decay of inhibition was needed only to allow the RE cells to recover; this holds even though RE-TC fast inhibitory synapses are direct. (For more details, see Rubin & Terman, 1998.) 6 Discussion

Numerous articles have considered models for thalamic oscillations and mechanisms for synchronization. The RE-TC model studied here is closely related to that in Golomb et al. (1994); however, those results are based primarily on numerical studies. Here we develop, for the Žrst time, a systematic approach for analytically studying models as complex as the thalamic networks. Related articles that deal with the role of inhibition in synchronizing bursting-like oscillations are Wang and Rinzel (1993) and Terman et al. (1998). Their conclusion is that a slow decay of inhibition is needed to obtain synchrony. This does not account, however, for synchronization of the RETC network in the presence of GABAA inhibition. By extending the analysis in Terman et al. (1998) to more complex networks, we are able to understand the mechanisms responsible for this. Several other articles, including van Vreeswijk et al. (1994) and Gerstner et al. (1996), have also shown that inhibition can lead to synchrony. These articles considered integrate-and-Žre type models and therefore do not directly apply to the thalamic networks. One of their conclusions is that the synchronous solution must be unstable if the synapses have instantaneous onset. This is consistent with our result (also in Terman et al., 1998) that indirect synapses are needed for synchrony. Our analysis shows, however, that parameters not in the integrate-and-Žre models, such as the time on the excited branch or the geometry of the curves of knees, can strongly inuence when synchrony arises.

636

J. Rubin and D. Terman

Our results clarify the multiple roles that inhibition can play in producing different rhythms. Inhibition may help to synchronize or desynchronize oscillations, depending on several factors. Fast onset of inhibition tends to desynchronize the cells, because when one cell Žres, it then quickly “steps on” other cells. For this reason, we generally need to assume that the synapses are indirect for stable synchronization to occur. Slow offset of inhibition may help to synchronize the oscillations; it can lead to compression of the cells while they evolve in the silent phase. Compression (or expansion) can also take place as the cells jump up or down between the silent and active phases. This depends on the geometry of curves (or surfaces) of knees, which in turn depends on both the intrinsic and the synaptic parameters, such as gsyn and gsag . A more powerful source of compression arises in the globally inhibitory networks. While the inhibitory cell J is active, it produces sustained inhibition to the E-cells. This forces the E-cells close to a stable Žxed point and therefore close to each other. In these networks, fast offset of inhibition also helps to synchronize the cells; it helps to get cells through the narrow window of opportunity for Žring. We have demonstrated that mutually coupled networks of excitable cells with indirect fast inhibitory coupling can produce synchronized rhythms. This is not possible if the network contains only basic cells. The geometric approach helps explain why additional cellular complexity allows for synchronized rhythms. The additional complexity translates, within the framework of the geometric approach, to higher-dimensional slow manifolds. This allows the cells to escape from the silent phase. It also leads to additional sources of compression. We conclude that synchronization is possible in mutually coupled inhibitory networks if there are at least two slow variables in the intrinsic or synaptic dynamics. This has relevance for isolated RE cell populations, which can synchronize even though they include fast inhibition (Steriade et al., 1990). Globally inhibitory networks can produce different rhythms depending on the intrinsic dynamics of the J-cells, which controls the amount of inhibition sent back to the E-cells. A rapid rate of synchronization, and a large domain of attraction for the synchronous solution, are achieved when the following factors are present: indirect synapses to provide a window of opportunity, a long J-cell active phase to enhance compression among E-cells, and a fast J-cell recovery coupled with relatively fast synaptic decay. Less powerful inhibition, or a smaller window of opportunity relative to synaptic decay rate, results in clustering among the E-cells. The network crashes if the amount of inhibition is too small. In the models for sleep rhythms, there are several possible ways to control the RE cells’ bursts and therefore the emergent network behavior. More powerful RE bursts result from removal of fast inhibition from among the RE cells (Steriade, McCormick, & Sejnowski, 1993; Huguenard & Prince, 1994; Destexhe, Bal, McCormick, & Sejnowski, 1996; Destexhe, 1998) or from the addition of excitatory input from the cortex (Contreras, Destexhe, & Steriade, 1996; Destexhe, Contreras, Sejnowski,

Population Rhythms

637

& Steriade, 1996; Steriade et al., 1997; Destexhe, 1998). Other intrinsic RE parameters, such as a leak conductance, may also greatly inuence the RE cells’ dynamics. We have seen how such considerations help explain the transition between spindling, delta, and paroxysmal discharges in RE-TC networks. The geometric analysis can lead to precise statements for when a particular rhythm is possible. For example, a TC cell can Žre only if it is sufŽciently hyperpolarized; this deinactivates the IT current. A geometric interpretation is that a TC cell can Žre only if, during the silent phase, it lies in the region where trajectories are able to reach the jump-up curve of knees. By considering the slow equations corresponding to the TC cells (the compound analogue of equation 4.1), one can then derive conditions for when a TC will Žre. In this sense, the geometric analysis helps clarify how different parameters inuence the rhythms. For example, we have seen how the analysis demonstrates that the sag current plays different roles in generating synchronized rhythms in mutually coupled and globally inhibitory networks. We note that the geometric approach used here (see also Terman & Wang, 1995; Terman & Lee, 1997; Terman et al., 1998) is somewhat different from that used in many dynamical systems studies. All of the networks considered here consist of many differential equations, especially for larger networks. Traditionally, one would interpret the solution of this system as a single trajectory evolving in a very large-dimensional phase-space. Instead, we consider several trajectories, one corresponding to each cell, moving around in a much lower-dimensional phase-space. After reducing the full system to a system for just the slow variables, the dimension of the lowerdimensional phase-space equals the number of slow intrinsic variables and slow synaptic variables corresponding to each cell. In the worst case considered here, there are two slow variables for each compound cell and one slow synaptic variable; hence, we never have to consider phase-spaces with dimension more than three. Of course, the particular phase-space we need to consider may change, depending on whether the cells are active or silent and also depending on the synaptic input that a cell receives. We assumed throughout that the networks were completely homogeneous. The analysis certainly extends to network models with mild heterogeneities. In this case, cells within each cluster will no longer be perfectly synchronized. They may lie on different branches of different cubic surfaces. If the cubics are sufŽciently close to each other, however, the jumps will be very close to each other and almost synchrony will result. Appendix A

The equations for the TC and RE cells in the thalamic network are given in section 5.1. As our biophysical mutually coupled network of compound cells, we considered a pair of TC cells, each governed by equation 5.1; note that in the case of basic cells without slow inhibition, these could be thought

638

J. Rubin and D. Terman

of as simpliŽed RE cells or TC cells. The synaptic variables in this model satisfy equation 2.6. For numerical simulations, we approximated Heaviside functions by functions of the form H1 ( v) D

1 . 1 C exp(¡k (v ¡ h ))

The functions h1 ( v), m1 ( v), r1 ( v), hR1 ( v), and mR1 ( v) are assumed to be of 1 the same form: if j D h, m, r, hR or mR , then j 1 ( v) D 1Cexp((vCh . Further, j ) / sj ) we take th ( v) D th0 C

th1 1 C exp((v C vt h ) / st h )

with tRh ( v) having an analogous form, and tr ( v) D tr0 C

tr1 . exp((v C vt r0 ) / st r0 ) C exp(¡(v C vt r1 ) / st r1 )

To generate our numerical Žgures, we started cells under a slight perturbation from a synchronous state. For Figures 3B and 4C, we used the following parameter values, based on Golomb et al. (1994) and Terman et al. (1996). IT : gCa D 2.5, h m D 57.0, sm D ¡6.0, vCa D 140.0, hh D 81.0, sh D 4.0, th0 D N vt h D 78.0, st h D 3.0; Isag : gsag D 0.2, vsag D ¡50.0, hr D 10.0, th1 D 73.3, 75.0, sr D 5.5, tr0 D 20.0, tr1 D 1000.0, vt r0 D 71.5, st r0 D 14.2, vt r1 D 89.0, st r1 D 11.6; IL : gL D 0.025, vL D ¡75.0; IA : gA D 0.4, vA D ¡79.0, a D 16.0, b D 4.0, hx D 0.1, k D 100.0, 2 ax D 0.3, 2 bx D 0.1, h syn D ¡50.0. We did not include slow inhibition here. To generate the clustered solution displayed in Figure 8, we used the following parameter values, also based on Golomb et al. (1994) and Terman et al. (1996). Six TC cells: IT : gCa D 1.5, hm D 59.0, sm D ¡9.0, vCa D N th1 D 333.3, N vt h D 78.0, st h D 1.5; Isag : 90.0, h h D 82.0, sh D 5.0, th0 D 66. 6, gsag D 0.15, vsag D ¡40.0, h r D 75.0, sr D 5.5, tr0 D 20.0, tr1 D 1000.0, vt r0 D 71.5, st r0 D 14.2, vt r1 D 89.0, st r1 D 11.6; IL : gL D 0.2, vL D ¡76.0; IA : g A D 0.1, vA D ¡84.0, aR D 8.0, bR D 0.05, h R D ¡50.0, k A D 2.0; IB : gB D 0.05, vB D 95.0, l D 10¡4 , k1 D 0.1, k2 D 0.05, hxb D 0.8, k xb D 0.02, k3 D 0.5, k4 D 0.005, hRb D ¡25.0, k b D 2.0. Three RE cells: IRT : R D gRCa D 2.0, hmR D 52.0, smR D ¡9.0, vRCa D 90.0, h hR D 72.0, shR D 2.0, th0 N t R D 333. 3, N vR D 78.0, s R D 1.0; IAHP : gAHP D 0.1, vK D ¡90.0, m 1 D 66. 6, th th h1 0.02, m 2 D 0.025, º D 0.01, c D 0.08; IRL : gRL D 0.3, vRL D ¡76.0; IRA : gRA D 0.25, vRA D ¡84.0, other parameters as in IA for the TC cells; IE : gE D 0.6, vE D 0, aE D 2.0, bE D 0.05, h E D ¡35.0, k E D 2.0. [Note that in j

the current IE , the variables sE satisfy an equation of the form (5.2), with the parameters aE , . . . replacing aR , . . . .]

Population Rhythms

639

To generate the synchronous solution displayed in Figures 9 and 10, we used the same parameter values except a D 16, b D 0.2, st h D 3.0, stRh D 2.0, gRA D 0. By setting gRA D 0, we removed the RE-RE fast inhibition; for aR D 16, bR D 0.2 with the same initial conditions and gRA D 0.25, the solution forms two clusters, but slow inhibition is slightly activated and it eventually destabilizes, while for gRA D 0.5, the solution forms two clusters without activation of slow inhibition. Finally, for Figures 11 and 12, we used the above parameter values except certain parameters were varied as discussed in the text and Žgure captions and th0 D 100.0, th1 D 500.0, st h D R D 100.0, t R D 500.0, s R D 2.0. 3.0, th0 th h1 We assumed in the proof of the lemma in section 3 that for compound cells, fy > 0 in the silent phase, while fy < 0 in the active phase. This is justiŽed for the TC cell model for the following reason. Note that y corresponds to the variable r; hence, fy D ¡gsag (v ¡ vsag ). Since vsag ¼ ¡40 mV typically, while v ranges from around ¡80 mV in the silent phase to at least ¡30 mV in the active phase, the result follows. Remark 4.

R We claimed in remark 1 that | @w | is quite small. This is also @y shown numerically in Figure 3B. We can understand analytically why this

Remark 5.

f

a is so by recalling, from the proof of the lemma, that @w D ¡ f y for a D L or R @y w corresponding to the silent or active phase respectively. From equation 5.1,

gsag ( v ¡ vsag ) @wa D¡ . @y gCa m21 (v)(v ¡ vCa )

(A.1)

Typical values are gsag ¼ .04 up to 2.0, vsag ¼ ¡40 mV, gCa ¼ 2.5, vCa ¼ 140 mV. In the silent phase, m1 ( v) is small, so the numerator and denominator in equation A.1 have similar magnitudes, even though v ¼ ¡80 mV. R In the active phase, however, m1 ( v) ¼ 1 while v ¼ 0, typically. Hence, | @w | @y is quite small. Appendix B

Consider a pair of compound cells in the active phase, with mutual inhibitory coupling. Suppose that the cells are perturbed from synchrony such that the lead cell falls down to the silent phase at time t1 and the following cell falls down at time t2 > t1 . Assume slow decay of inhibition (b D 2 K), although the analysis simpliŽes otherwise (see below). During the silent phase, for t ¸ t2 , the slow dynamics of the two cells are given by wP D g( v, w) yP D h(v, y) sP D ¡Ks,

(B.1)

640

J. Rubin and D. Terman

Figure 13: The set-up for analysis of compression in the silent phase in two dimensions (i.e., assuming s is Žxed). The picture generalizes naturally to 3D when s is included as another slowly evolving variable. Cell 1 jumps up Žrst, with t D T1 , at (y¤1 , w¤1 ). The larger dotted square shows a blow-up of the smaller dotted square, in which the vectors V and (g, r ) are deŽned. d where v satisŽes v D W L ( w, y, s) andP D dt . Let V (t ) D ( g( w, v), h( y, v), ¡Ks). The equation of variations that describes the evolution of tangent vectors to the ow of equation B.1 is

d Pw D ¡awd w ¡ bwdy ¡ cwd s P D ¡ayd w ¡ bydy ¡ cyds dy dPs D ¡Kds,

(B.2)

where aw D ¡@g / @w, bw D ¡@g / @y D ¡@g / @v ¢ @W L / @y, and so on. Fix the cell that jumps up out of the silent phase Žrst as cell 1 and call the other cell 2. Let Ti denote the jump-up time of cell i and restrict to t 2 [t2 , T1 ]. Let (g(t ), r (t), s(t)) denote the vector from the position of cell 1 to that of cell 2, such that (g, r , s) satisŽes equation B.2; see Figure 13 for a two-dimensional representation of this set-up. Next, let wL ( y, s) denote the projection of the left surface of knees to (w, y, s)-space and let (a, b, c) D r[wL ( y1 ( T1 ), s( T1 )) ¡ w( T1 )] D (¡1, @wL ( ( ), ( )), @wL ( ( ), ( ))). Let wtL denote the physical translate @y y1 T1 s T1 @s y1 T1 s T1

Population Rhythms

641

of wL ( y, s) to the position of cell 1 at time t; for example, at time t2 , cell 1 lies in wtL2 . Again, see Figure 13. We deŽne the time T (t ) as the time for cell 2 to ow to its Žrst intersection with wtL . This satisŽes "

Z

(a, b, c) ¢ (g, r , s) C

T (t )

0

# V (t Cj ) dj

D 0,

(B.3)

where V is evaluated along the path of cell 2 (see Figure 13), but Z

T(t ) 0

Z V (t Cj ) dj D

T (t ) 0

( V (t) Cj VP (t ) C O(j 2 ))dj

D V (t )T (t ) C O(T 2 ).

(B.4)

For small perturbations from synchrony, if the vector Želd of equation B.1 is O(1), then the O(T 2 ) term in equation B.4 can be neglected. Substituting the approximation equation B.4 into B.3 yields, at leading order, T (t ) D ¡

( a, b, c) ¢ (g, r , s) g ¡ br ¡ cs D , ( a, b, c) ¢ V ¡g(w2 , v2 ) C bh( y2 , v2 ) ¡ cKs2

(B.5)

since a D ¡1. Henceforth, we omit the arguments indicating evaluation along the path of cell 2 when this is clear. A sufŽcient condition for compression in the silent phase is that TP (t ) < 0 for t 2 [t2 , T1 ]. Thus, we proceed to compute TP (t ). Since (g, r , s) and V (t ) satisfy equation B.2, we differentiate, substitute from B.2, and simplify to obtain [V(t ) £ (g, r , s)] ¢ [((a, b, c) ¢ DV ) £ ( a, b, c)]† , TP (t ) D (( a, b, c) ¢ V (t ) † ) 2

(B.6)

where † denotes transpose. Geometrically, the determinant of three vectors gives the volume of the parallelepiped they bound. The numerator of TP (t) consists of such a determinant, with the three edges of the parallelepiped given by the vector Želd, the vector from the position of cell 1 to that of cell 2, and a third vector. This last vector relates the linearization of the vector Želd to the gradient of the translate of the surface of knees. Next, consider Z ´ (Z1 , Z2 , Z3 ) :D V £ (g, r , s) D (s h Cr Ks, ¡sg ¡ gKs, r g ¡ gh)

D (( h, ¡Ks) ^ (r , s), (g, s) ^ ( g, ¡Ks), ( g, h) ^ (g, r )).

(B.7)

642

J. Rubin and D. Terman

Differentiating equation B.7 and using B.2 gives, in system form, ZP 1 D ¡(by C K)Z1 C ay Z2 ZP 2 D bw Z1 ¡ (aw C K) Z2 ZP 3 D cw Z1 C cy Z2 ¡ ( aw C by ) Z3 .

(B.8)

For basic cells, without the current y, equation B.6 simpliŽes to sg CgKs ¡Z2 ( c( aw ¡ K) C cw ) D (c( aw ¡ K) C cw ). TP (t ) D ( g C cKs) 2 ( g C cKs) 2 Moreover, Z1 ´ 0, so ZP 2 D ¡(aw CK)Z2 . Hence, the sign of Z2 (t ) is invariant for t 2 [t2 , T1 ]. In Terman et al. (1998), Z2 (t2 ) D ( w1 (t2 ) ¡w2 (t2 )) Ks2 (t2 ) < 0, which implies that ¡Z2 (t2 ) > 0. Thus, the sign of TP (t) matches that of (c(aw ¡ K) C cw ) (Terman et al. use l 1 to denote c). By showing that this quantity is negative for all relevant t for K < a¡ (case I in Terman et al., 1998), Terman et al. achieve a sufŽcient condition for compression in the silent phase. In general, one can compute the signs of a, b, c as well as aw , bw , . . . (see, e.g., the lemma in section 3) and then can use equations B.6 and B.8 to derive compression conditions for the silent phase. SimpliŽcations facilitate this process in certain cases even for compound cells. For example, s D sP D 0 during the silent phase for E cells in a globally inhibitory network as well as for much of the silent phase for mutually coupled cells with fast synaptic decay. In the latter case, however, an adjustment must be made to compensate for the fact that one cell loses inhibition before the other; we omit details and explicit computations here. Acknowledgments

We thank N. Kopell for her very useful comments. Research for this article was supported in part by the NSF grants DMS-9423796 and DMS-9804447. Some of the research was done while the authors were visiting the IMA at the University of Minnesota. References Bal, T., & McCormick, D. A. (1996). What stops synchronized thalamocortical oscillations? Neuron, 17, 297–308. Bush, P., & Sejnowski T. (1996). Inhibition synchronizes sparsely connected cortical neurons within and between columns of realistic network models. J. Comp. Neurosci., 3, 91–110. Chow, C. C. (1998). Phase-locking in weakly heterogeneous neuronal networks. Physica D, 118, 343–370.

Population Rhythms

643

Contreras, D., Destexhe, A., Sejnowski, T. J., & Steriade, M. (1996).Control of spatiotemporal coherence of a thalamic oscillation by corticothalamic feedback. Science, 274, 771–774. Contreras, D., Destexhe, A., & Steriade, M. (1996). Cortical and thalamic participation in the generation of seizures after blockage of inhibition. Soc. Neurosci. Abst., 22, 2099. Contreras, D., Destexhe, A., & Steriade, M. (1997). Spindle oscillations during cortical spreading depression in naturally sleeping cats. Neuroscience, 77, 933– 936. Destexhe, A. (1998) Spike-and-wave epileptic oscillations based on the properties of GABA B receptors. J. Neurosci., 18, 9099–9111. Destexhe, A., Babloyantz, A., & Sejnowski, T. J. (1993). Ionic mechanisms for intrinsic slow oscillations in thalamic relay and reticularis neurons. Biophys. J., 65, 1538–1552. Destexhe, A., Bal, T., McCormick, D. A., & Sejnowski, T. J. (1996). Ionic mechanisms underlying synchronized oscillations and propagating waves in a model of ferret thalamic slices. J. Neurophysiol., 76, 2049– 2070. Destexhe, A., Contreras, D., Sejnowski, T. J., & Steriade, M. (1996). Cortical projections to the thalamic reticular nucleus may control the spatiotemporal coherence of spindle and epileptic oscillations. Soc. Neurosci. Abst., 22, 2099. Destexhe, A., Contreras, D., & Steriade, M. (1998). Mechanisms underlying the synchronizing action of corticothalamic feedback through inhibition of thalamic relay cells. J. Neurophysiol., 79, 999–1016. Destexhe, A., McCormick, D. A., & Sejnowski, T. J. (1993). A model for 8–10 Hz spindling in interconnected thalamic relay and reticularis neurons. Biophys. J., 65, 2474–2478. Destexhe, A., & Sejnowski, T. J. (1997). Synchronized oscillations in thalamic networks: Insights from modeling studies. In M. Steriade, E. G. Jones, & D. A. McCormick (Eds.), Thalamus. Amsterdam: Elsevier. Gerstner, W., van Hemmen, J. L., & Cowan, J. (1996). What matters in neuronal locking? Neural Comp., 8, 1653–1676. Golomb, D., & Rinzel, J. (1993).Dynamics of globally coupled inhibitory neurons with heterogeneity. Phys. Rev. E, 48, 4810–4814. Golomb, D., & Rinzel, J. (1994). Clustering in globally coupled inhibitory neurons. Physica D, 72, 259–282. Golomb, D., Wang, X.-J., & Rinzel, J. (1994). Synchronization properties of spindle oscillations in a thalamic reticular nucleus model. J. Neurophysiol., 72, 1109–1126. Golomb, D., Wang, X.-J., & Rinzel, J. (1996) Propagation of spindle waves in a thalamic slice model. J. Neurophysiol., 75, 750–769. Hodgkin, A. L., & Huxley, A. F. (1952). A quantitative description of membrane current and its application to conduction and excitation in nerve. J. Physiol. (London), 117, 500–544. Huguenard, J., & Prince, D. (1994). Clonazepam suppresses GABAB -mediated inhibition in thalamic relay neurons. J. Neurophysiol., 71, 2576–2581.

644

J. Rubin and D. Terman

Kim, U., Bal, T., & McCormick, D. A. (1995). Spindle waves are propagating synchronized oscillations in the ferret LGNd in vitro. J. Neurophysiol., 74, 1301–1323. Kopell, N., & LeMasson, G. (1994).Rhythmogenesis, amplitude modulation and multiplexing in a cortical architecture. Proc. Nat. Acad. Sci. (USA), 91, 10586– 10590. LoFaro, T., & Kopell, N. (In press). Time regulation in a network reduced from voltage-gated equations to a one-dimensional map. J. Math. Biology. Rinzel, J. (1987). A formal classiŽcation of bursting mechanisms in excitable systems. In A. M. Gleason (Ed.), Proceedings of the International Congress of Mathematics (pp. 1578–1593). Providence, RI: AMS. Rinzel, J., Terman, D., Wang, X.-J., & Ermentrout, B. (1998). Propagating activity patterns in large-scale inhibitory neuronal networks. Science, 279, 1351– 1355. Rowat, P., & Selverston, A. (1997). Oscillatory mechanisms in pairs of neurons connected with fast inhibitory synapses. J. Comp. Neurosci., 4, 103–127. Rubin, J., & Terman, D. (1998). Geometric analysis of population rhythms in synaptically coupled neuronal networks. Unpublished manuscript, IMA Preprint Series 1596, University of Minnesota. Rubin, J., & Terman, D. (1999). Analysis of clustered Žring patterns in synaptically coupled networks of oscillators. Manuscript submitted for publication. Skinner, F., Kopell, N., & Marder, E. (1994). Mechanisms for oscillation and frequency control in networks of mutually inhibitory relaxation oscillators. J. Comp. Neurosci., 1, 69–87. Somers, D., & Kopell, N. (1993). Rapid synchronization through fast threshold modulation. Biol. Cyber., 68, 393–407. Steriade, M. (1994). Coherent activities in corticothalamic networks during resting sleep and their development into paroxysmal events. In G. Buzs´aki, R. Llin a´ s, W. Singer, A. Berthoz, & Y. Chrtisten (Eds.), Temporal coding in the brain. New York: Springer-Verlag. Steriade, M., Contreras, D., & Amzica, F. (1994). Synchronized sleep oscillations and their paroxysmal developments. TINS, 17, 199–208. Steriade, M., Contreras, D., & Amzica, F. (1997). The thalamocortical dialogue during wake, sleep, and paroxysmal oscillations. In M. Steriade, E. G. Jones, & D. A. McCormick (Eds.), Thalamus. Amsterdam: Elsevier Steriade, M., Curr´o Dossi, R., & Nunez, ˜ A. (1991). Network modulation of a slow intrinsic oscillation of cat thalamocortical neurons implicated in sleep delta waves: Cortically induced synchronization and brainstem cholinergic suppression. J. Neurosci., 11, 3200–3217. Steriade, M., Domich, L., Oakson, G., & Deschˆenes, M. (1987). Abolition of spindle oscillations in thalamic neurons disconnected from nucleus reticularis thalami. J. Neurophysiol., 57, 260–273. Steriade, M., Jones, E. G., & Llin a´ s, R. R. (1990). Thalamic oscillations and signaling. New York: Wiley Steriade, M., McCormick, D. A., & Sejnowski, T. J. (1993). Thalamocortical oscillations in the sleeping and aroused brain. Science, 262, 679–685.

Population Rhythms

645

Steriade, M., Nunez, ˜ A., & Amzica, F. (1993). Intracellular analysis of relations between the slow (< 1 Hz) neocortical oscillation and other sleep rhythms of the electroencephalogram. J. Neurosci., 13, 3266–3283. Terman, D., Bose, A., & Kopell, N. (1996). Functional reorganization in thalamocortical networks: Transition between spindling and delta sleep rhythms. Proc. Natl. Acad. Sci. (USA), 93, 15417–15422. Terman, D., Kopell, N., & Bose, A. (1998). Dynamics of two mutually coupled inhibitory neurons. Physica D, 117, 241–275. Terman, D., & Lee, E. (1997). Partial synchronization in a network of neural oscillators. SIAM J. Appl. Math., 57, 252–293. Terman, D., & Wang, D. L. (1995). Global competition and local cooperation in a network of neural oscillators. Physica D, 81, 148–176. Timofeev, I., & Steriade, M. (1996). Low-frequency rhythms in the thalamus of intact-cortex and decorticated cats. J. Neurophysiol., 76, 4152–4168. Traub, R. D., & Miles, R. (1991). Neuronal networks of the hippocampus. New York: Cambridge University Press van Vreeswijk, C., Abbott, L., & Ermentrout, G. B. (1994). When inhibition, not excitation synchronizes neural Žring. J. Comp. Neurosci., 313–321. Wang, X.-J., Golomb, D., & Rinzel, J. (1995). Emergent spindle oscillations and intermittent burst Žring in a thalamic model: SpeciŽc neuronal mechanisms. Proc. Natl. Acad. Sci. (USA), 92, 5577–5581. Wang, X.-J., & Rinzel, J. (1992). Alternating and synchronous rhythms in reciprocally inhibitory model neurons. Neural Comp., 4, 84–97. Wang, X.-J., & Rinzel, J. (1993). Spindle rhythmicity in the reticularis thalamic nucleus: Synchronization among mutually inhibitory neurons. Neuroscience, 53, 899–904. Wang, X.-J., & Rinzel, J. (1995). Oscillatory and bursting properties of neurons. In M. A. Arbib (Ed.), The handbook of brain theory and neural networks (pp. 686– 691). Cambridge, MA: MIT Press. Whittington, M. A., Traub, R. D., & Jefferys, J. G. R. (1995). Synchronized oscillations in interneuron networks driven by metabotropic glutamate receptor activation. Nature, 373, 612–615. White, J., Chow, C. C., Ritt, J., Soto, C., & Kopell, N. (1998). Synchronization and oscillatory dynamics in heterogeneous, mutually inhibited neurons. J. Comp. Neurosci., 5, 5–16. Received July 20, 1998; accepted February 23, 1999.

LETTER

Communicated by George Gerstein

An Accurate Measure of the Instantaneous Discharge Probability, with Application to Unitary Joint-Event Analysis Quentin Pauluis Laboratory of Neurophysiology, School of Medicine, Universit´e Catholique de Louvain Brussels, Belgium Stuart N. Baker Department of Anatomy, University of Cambridge, Cambridge, CB2 3DY, U.K. We present an estimate for the instantaneous discharge probability of a neurone, based on single-trial spike-train analysis. By detecting points where the neurone abruptly changes its Žring rate and treating them specially, the method is able to achieve smooth estimates yet avoid the blurring of signiŽcant changes. This estimate of instantaneous discharge probability is then applied to the method of unitary event analysis. We show that the unitary event analysis as originally conceived is highly sensitive to Žring-rate nonstationarities and covariations, but that it can be considerably improved if calculations of statistical signiŽcance use an instantaneous discharge probability instead of a Žring-rate estimate based on averaging across multiple trials.

1 Introduction

A large number of new techniques have been developed for the analysis of single-unit neuronal discharge. The increasing mathematical complexity of these techniques permits maximal information to be extracted about how these signals are being used by the brain. However, most new techniques are developed and tested on the assumption that neurone spikes resemble stationary Poisson processes. This often yields mathematical tractability, but at the expense of grossly oversimplifying the statistics of the target spike trains. Nonstationarity in cell discharge can occur in a number of ways. First, the Žring rate often changes following delivery of a stimulus or performance of a behavioral task by the experimental animal. These changes can be profound (from no Žring to > 100 Hz) and rapid. In order to account for such rate modulation, many analysis techniques estimate the Žring rate on a single trial from the peristimulus time histogram (PSTH; Gerstein & Kiang, 1960; Lemon, 1984), derived by summing together the discharge on a number of repeats of the same trial. The underlying model for the spike train then Neural Computation 12, 647–669 (2000)

c 2000 Massachusetts Institute of Technology °

648

Quentin Pauluis and Stuart N. Baker

becomes a Poisson process with instantaneous Žring rate on any one trial modulating as the all-trial PSTH (Aertsen, Gerstein, Habib, & Palm, 1989; Grun, ¨ 1996). More complex nonstationarities occur when the Žring-rate modulation is not the same from one trial to the next. First, there may be a jitter in the time of occurrence of a rise in rate relative to the alignment event; this is common in work on awake, behaving animals, where a trial lasting several seconds must be aligned to one point in the performance (Seidemann, Meilijson, Abeles, Bergmann, & Vaadia, 1996; Munoz & Wurtz, 1993). Second, the overall shape could differ from one trial to the next, depending on uncontrolled external or random events such as attentional shifts. These two forms of nonstationarity may show no covariation between simultaneously recorded cells. However, since such variation is often not simply noise in the system, but instead reects genuine variation in the task performance of the animal from trial to trial, it is more likely that the times, the magnitudes, or the overall shape of rate changes will be correlated. The difference between these two situations is of great importance for methods that seek to estimate the expected number of synchronous discharges between two cells; unfortunately, the PSTH can only reect the mean Žring-rate proŽle and gives no clue to its variability or correlation structure from trial to trial. Brody (1997, 1999a, 1999b) proposed a method of overcoming these problems by Žtting the cell discharge on a single trial to a function that had the same shape as the mean PSTH but was shifted by an adjustable latency and scaled by a multiplicative gain. This estimate of the single-trial Žring rates permitted a better prediction of the cross-correlation histogram between two units than one derived from the PSTH alone. However, the all-trial PSTH reects only the mean change in rate, taking no account of the variability of this proŽle among trials, so that this technique is probably limited in its applicability to experimental data. In this article, we present a method for estimating the instantaneous discharge probability density of a neurone in a single trial. We then apply it to the method of unitary event analysis pioneered by Grun ¨ (1996) and recently applied to data recorded in the motor cortex of awake monkeys (Riehle, Grun, ¨ Diesmann, & Aertsen, 1997). We show that this method, as originally conceived, is highly sensitive to the nonstationarities described above, but that it can be considerably improved using single-trial Žring-rate estimates. 2 Measurement of Instantaneous Discharge Probability

We Žrst draw a distinction between instantaneous discharge probability density and the instantaneous Žring rate. We view the latter as an output, which may be read by a neurone farther downstream. By contrast, we view the discharge probability density as an input function that could be fed into a neuronal model to generate a spike train similar to the one observed. Gen-

Instantaneous Discharge Probability

649

eration of such a measure requires allowance to be made for the delayed response of the neuronal integrator to changes in its inputs. The discharge probability density should be smooth, and not replicate the variability typically seen in interspike intervals (Softky & Koch, 1992; Shadlen & Newsome, 1994). However, in conict with this, the function should also be able to identify and vary abruptly with sudden Žring changes. Finally, since the only data available are the spike train, the method has to start with a measure of the instantaneous Žring rate. Two distinct methods exist in the literature for estimation of the instantaneous Žring rate of a spike train. In one, pulses produced at the spike times are low pass Žltered (French & Holden, 1971), either using analog Žlters (Houk, Dessem, Miller, & Sybirska, 1987), or by convolution with a smoothing function (often a gaussian; MacPherson & Aldridge, 1979; Richmond, Optican, Podell, & Spitzer, 1987). The closer together the spikes, the more summation of the smoothed waveforms that occurs, and hence the larger the output signal. However, this method requires that the cutoff of the low-pass Žlter be adjusted to be appropriate for the Žring rate under study. If it is too low, a highly smoothed signal is obtained, which cannot follow rapid rate changes; if it is too high, periods of zero estimated Žring rate are interposed between spikes with long intervals. Adaptive kernel methods (Silverman, 1986; Richmond, Optican, & Spitzer, 1990) represent an attempt to adjust the amount of smoothing depending on the local density, so that these two extremes are avoided. More sophisticated techniques such as orthogonal series estimates and maximum penalized likelihood estimates (Silverman, 1986) can suffer from the need to tune smoothing parameters to Žnd the optimal representation of the data. An alternative is to use the reciprocal of the interspike interval (ISI) as an estimate of the Žring rate during the time spanned by that interval. Such a method is often implicitly employed using “frequencygram” plots, where a point is plotted at the time of each spike with a height of the reciprocal of the preceding interval (Matthews, 1963; Bessou, Laporte, & Pag e` s, 1968; Awiszus, 1988). It is similar to the use of “fractional intervals” (Schwartz, 1992). This technique has the advantage that it never has periods of zero estimated rate; however, in its raw form, the estimate shows great variability. Any subsequent smoothing will necessarily be a trade-off between the need to reduce this variability without removing rapid uctuations. A means of avoiding this trade-off would be to detect rapid changes in Žring rate and treat them as a special case. This is essentially the same problem as detection of bursts in a spike train, and a number of techniques have been suggested. The Žrst was the CUMSUM technique (Hurst, 1950), which has been applied to both averaged PSTH (Ellaway, 1978) and also to single-trial analysis (Butler, Horne, & Hawkins, 1992; Butler, Horne, & Rawson, 1992). The CUMSUM involves subtraction of a control reference level from a series of datum points and adding cumulatively. However, when the goal is the formation of a continuous estimate of discharge probability,

650

Quentin Pauluis and Stuart N. Baker

it is not clear where the control level should be placed. CUMSUM methods are also very sensitive to small variations during the control period and require excessive parameter tuning (Churchward et al., 1997). An alternative has been proposed by Cocatre-Filgien and Delcomyn (1992), who Žrst identiŽed ISI subpopulations using an ISI histogram, and then tested identiŽed bursts against the mean Žring frequency with a chi-square test. This method is well suited to a spike train where Žring is at a constant level for most of the time, with occasional high-frequency bursts; it is much less suitable for neurones where the Žring rate changes continually over a wide range. Other methods in the literature suffer from similar problems: ISIs are always compared with the mean ISI for the whole recording, making the assumption that intervals are Poisson distributed (Leg´endy & Salcman, 1985; Eggermont, Smith, & Bowman, 1993). Our proposed technique is a variation of these methods. The Žrst difference is that we work with ISIs locally. In this way, a change in Žring rate is identiŽed relative to the Žring of the cell close to the time of interest, rather than by comparison to the global ISI statistics. Three consecutive intervals are used, since a single interval shorter than the preceding one could simply represent the tail of the ISI distribution, even if Žring rate were constant. Two consecutive ISIs signiŽcantly shorter than the preceding ISI are much less likely without a change in Žring rate. The same limit to the smallest detectable burst has been used by Cocatre-Zilgien and Delcomyn (1992). Second, we assume that the ISIs follow a gamma density. This allows for an absolute refractory period and for the relative refractory period produced by afterhyperpolarization, and is a more realistic description of interval statistics than the more commonly used Poisson process (for discussion, see Berry & Meister, 1998; de Ryuter van Steveninck, Lewen, Strong, Koberle, & Bialek, 1997). Moreover, by changing the order parameter of the gamma density, it can be adjusted to Žt interval statistics from a wide range of experimental data. The Žrst step is to generate an instantaneous Žring-rate function F(t), sampled at an appropriately Žne temporal resolution, whose value at any point is simply the reciprocal of the ISI that spans it (see Figure 1B). The series of ISIs is then scanned to determine times at which the Žring rate abruptly changes. To do this, we make the assumption that, at constant Žring rate, ISIs are distributed according to a gamma density. The probability that if the mean interval is m , an interval i will be larger than some value I is then: Z P(i > I | m ) D

1 I

¡ a¢ m

a

1 ta¡1 e¡ta /m dt, ( a ¡ 1)!

(2.1)

where a is the order of the gamma process used to model the intervals. In the remainder of this article, we use a D 4, although this can be adjusted to Žt the interval statistics for a given cell (a D 1 would correspond to a Poisson process). As illustrated in Figure 1A, the nth interval was accepted as the

Instantaneous Discharge Probability

651

Figure 1: Reconstruction of the instantaneous frequency. (A) Original spike train. (B) Inverse of the interspike intervals (ISIs). Statistical tests detect points of rate increase (empty arrow) or decrease (Žlled arrow). (C) The high Žring rates are extended out half an interval, to the best estimate of the time of the rate change. In order to maintain the overall mean rate, the Žring rate before or after these rate change points is reduced. (D) The signal is smoothed by convolution with a gaussian. No smoothing is performed across the rate change points. Full details are given in the text.

onset of an increase in rate if the following two criteria were both fulŽlled: P(i > In | m D InC1 ) < 0.05 P(i > In | m D InC2 ) < 0.05.

(2.2)

Likewise, the nth interval was accepted as the onset of a decrease in rate if both of the following two criteria were met: P(i > In | m D In¡1 ) < 0.05 P(i > In | m D In¡2 ) < 0.05.

(2.3)

The longest of the three intervals under consideration is always the one tested. As noted, the requirement that the interval should be significantly larger than both of the intervals preceding it (for a rate fall) or

652

Quentin Pauluis and Stuart N. Baker

succeeding it (for a rate rise) ensures that only genuine rate changes are detected, and not anomalous intervals arising from the tail of the ISI distribution. Changes in rate can thus be detected. The next stage is to determine at what precise time the change occurs. One possibility would obviously be at the time of the Žrst spike contributing to the changed interval. However, this is likely to be biased. A neurone that suddenly receives a higher rate of input will not Žre immediately; the moment of Žring probability rise is therefore likely to be some time prior to the Žrst spike of the shortened interval. The best solution can be appreciated by the following formalism. Let the long interval In be divided into two parts: one lasting TLo in which the discharge probability density is FLo , and another lasting T Hi in which the discharge probability density is FHi . We have therefore that: T Lo C THi D In .

(2.4)

Since one spike in total resulted from the interval In , the sum of the Žring probability density must be 1 over this time: T Lo FLo C THi FFi D 1.

(2.5)

Finally, in the absence of any more information about the timing of the change in discharge probability, we assume that the change occurs independently from the spike process. Had the Žring probability density not changed, the interval would have been 1 / FLo. We therefore assume that on average, TLo is half of this value: 1 2FLo T Lo FLo D 0.5 D T Hi FHi . T Lo D

(2.6)

FHi is assumed to be equal to the reciprocal of the short interval IShort either preceding (for a rate fall) or succeeding (for a rate rise) In . Hence equations 2.4–2.6 provide a unique solution, which is: T Hi D

1 1 D IShort 2FHi 2

TLo D In ¡ FLo D

1 1 D In ¡ Ishort 2FHi 2

1 . 2TLo

(2.7)

The high discharge probability density calculated from the short interval is therefore extended out for half of this interval into the adjoining long

Instantaneous Discharge Probability

653

interval, as shown in Figure 1C, for both rate increase and rate decrease. This is compensated for by reducing the discharge probability density in the remainder of the long interval to a value close to half of the original (see Figure 1C). A probability density function of this form is exactly normalized, in the sense that its integral from time zero to a given time, rounded down, gives the number of spikes which have occurred to that point. This function can therefore be seen as the input to an IF neurone, which would thereby generate the observed spike train. The discharge probability density of Figure 1C still exhibits high variability, due to high variance in intervals even if the underlying rate remains constant (Shadlen & Newsome, 1994; Softky & Koch, 1992). In order to remove this unwanted variability partially, intervals between abrupt Žring changes are then smoothed by convolution with a gaussian kernel (standard deviations: 10 ms, chosen to be in the range of the somatic time constant; Koch, Rapp, & Segev, 1996). This smoothing is not carried out across boundaries of rate changes as deŽned above, producing a measure that varies smoothly between rate change points but abruptly at them. This is illustrated in Figure 1D. It is the explicit identiŽcation and localization of abrupt Žring probability changes that confer on this measure the most tangible advantage over other techniques. Without such a step, there is necessarily a trade-off between oversmoothing rapid changes and high variability, which cannot be satisfactorily resolved. The method described produces an estimate of the discharge probability density function that can be computed without averaging across trials. The next section investigates an application of this method to the assessment of the signiŽcance of unitary joint events. 3 Application to Unitary Joint Event Analysis

The aim of the unitary joint event method is to detect discharges of two neurones that are synchronous within an allowed jitter and occur more often than expected by chance. The technique, similar to that proposed by Grun ¨ (1996; UEMWA method) but with minor changes to avoid unnecessary binning, is reviewed in Figures 2A–G. From the repeated trials (see Figures 2A and 2C), PSTHs for the two cells are built up, and these are smoothed with a moving window (here chosen to be 101 ms wide; see Figures 2B and 2D). Spikes from the two cells occurring at times t1 and t2 are designated a coincident event (CE) if |t1 ¡ t2 | < D ,

(3.1)

where D in this study was taken as 5 ms. We conventionally place the occurrence time of the CE halfway between the two coincident spikes, at ( t1 Ct2 ) / 2. The number of CEs is counted in successive bins, and the count is plotted as dots in Figure 2E. The number of these raw CEs rises as the cell discharge

654

Quentin Pauluis and Stuart N. Baker

rates increase. The expected number of coincidences is approximately E (CEs in bin T ) D 2wD NF1 ( T ) F2 ( T ),

(3.2)

where N is the number of trials, w is the bin width used to count the CEs (here 1 ms), and F1 , F2 represent the Žring rates of the cells estimated from the smoothed PSTHs. Computation of the expected number of CEs is made more complex than equation 3.2 because the bin width w is in general larger than the precision to which spike times are recorded. If a CE falls into the bin centered on time T, then

­ ­ ­ ­ w ( t1 ¡ t2 ) ­ ¡ T­ ­ 2 ­< 2 .

(3.3)

Suppose as an example that t1 is in the bin centered on T ¡w, and t2 is the bin centered on T Cw. For some possible combinations of t1 and t2 , equation 3.3

Figure 2: Facing page. Comparison of signiŽcance calculation for coincident events using the method of Grun ¨ (1996) and the novel method proposed here. One hundred trials, each lasting 500 ms, were generated with a central 250 ms burst of activity (Žring rate rose from 25 Hz to 75–125 Hz) which began with a temporal jitter of 100 ms. (A–G) Methods based on the peristimulus time histograms (PSTH). For each cell (A and C), the mean Žring rates are computed by compiling PSTHs across trials; these are smoothed by a 101 ms moving window (B and D). (E) The expected coincidence rate (solid line) is calculated by multiplying the two mean Žring rates, with a correction for the size of the window within which coincidences are detected. Dots show the actual coincidences detected in this simulation. (F) Actual and expected coincidence rates are smoothed with a 101 ms moving window. (G) The probability P to Žnd as many or more CEs as observed is calculated from the expected rate assuming a Poisson distribution. P is displayed as log10 ((1 ¡ P) / P), facilitating visualization of high and low values of P. (H–P) Method based on the single-trial Žring-rate estimate. For each cell (rasters for the Žrst trial shown in H and J), the instantaneous frequency is reconstructed on a trial-by-trial basis (I and K ; Žlter SD between rate change points D 10 ms). Then the rate of expected CEs is calculated (L: Žrst trial; M: four other trials). A bin-per-bin summation of the single-trial CE expectancy curves produces the global CE expectancy (N, solid line), plotted with the number of CEs detected (dots). (O) Both rates are then Žltered using a 101 ms moving window. (P) SigniŽcance of departures from the expected rate is assessed as in (G) and plotted in the same way. Vertical bar: (B and D) 100 sp.s¡1 ; (E, F, N, O) 100 CE.s¡1 .trial¡1 ; (I, K) 500 sp.s¡1 ; (L, M) 1667 CEs.s¡1 ; (G, P) 5.

Instantaneous Discharge Probability

655

could still be left unfulŽlled (t1 D T ¡ 0.7w and t2 D T C1.3w). This is shown schematically by Figure 3A: CEs generated by spikes each conŽned to one bin may fall into two adjacent bins. Extending these considerations to all possible combinations of bin occupancy for t1 and t2 , which could fulŽll equations 3.1 and 3.2, leads us to an exact solution for the case of D D 5 ms shown in Figure 3B and described numerically as: 0 !1 C3 C3 X X 2 (3.5) E(CEs in T ) D w N @ F1 ( T C jw) Mj,k F2 ( T C kw) A jD¡3

(

kD¡3

656

Quentin Pauluis and Stuart N. Baker

2

MjD¡3...3,kD¡3...3

0 6 0 6 6 0 6 D6 6 0 6 0 6 41 / 4 0

0 0 0 0 2/4 4/4 1/4

0 0 0 2 /4 4 /4 2 /4 0

0 0 2/4 4/4 2/4 0 0

0 2 /4 4 /4 2 /4 0 0 0

1/4 4/4 2/4 0 0 0 0

3 0 1 / 47 7 0 7 7 0 7 7. 0 7 7 0 5 0

(3.6)

Figure 2E plots the expected number of coincidences E as a solid line. The number of observed coincidences is then smoothed with a 101 ms moving window, yielding the plot in Figure 2F. Finally, the probability that the number of coincidences c will be at least as many as the number observed, C, given that E are expected, is calculated assuming that they follow a Poisson distribution: P (c ¸ C) D 1 ¡

iDC¡1 X e¡E Ei

i!

iD0

.

(3.7)

This probability is plotted in Figure 2G as log10 ((1 ¡ P) / P ), thereby permitting bins with a signiŽcant excess, or a signiŽcant deŽcit, of CEs to stand out. The horizontal lines mark the 99% conŽdence limits (two tailed). The modiŽcation to this technique that we propose is illustrated in Figures 2H–2P. For each trial (see Figures 2H and 2J), the instantaneous discharge probability density is calculated (see Figures 2I and 2K). This is used to calculate the expected number of coincidences in a single bin, in one trial; summing over all trials yields the approximate number of coincidences in a given bin: E (CEs in T ) D 2wD

N X nD1

Fn1 ( T ). Fn2 ( T ),

(3.8)

where Fn1 , Fn2 represent the instantaneous discharge probability density during trial n. Once again, however, the situation is made more complex by the binning used. By similar considerations to those above, we Žnd the exact formula for when D D 5w to be: 0 !1 C3 C3 N X X X @ E(CEs in T ) D w2 . Fn1 ( T C jw). Mj,k Fn2 ( T C kw) A , (3.9) nD1

jD¡3

(

kD¡3

where Mj,k is as deŽned in equation 3.5. This measure is plotted as a solid line in Figure 2N, overlaid with the observed number of coincidences (dots). The moving-window-smoothed traces are shown in Figure 2O, and the signiŽcance plot, calculated according to equation 3.6, is illustrated in Figure 2P. The crucial difference between the two methods lies in the order in which averaging across trials is performed. In the original method proposed by

Instantaneous Discharge Probability

657

Figure 3: Schematic illustration of CE binning. Calculation of the CEs before the binning step ensures their exact determination but renders complex the calculation of their expectancy. Black squares represent a bin containing spikes from one unit; white squares represent a bin containing spikes from the other. The gray squares code on a gray scale the density of CEs resulting from spikes in the black and white squares. White bins with dotted outlines indicate a situation where not all of the possible combinations of spikes will lie within the coincidence detection interval D . (A) The CE distribution resulting from the combination of spikes in two bins has a triangular shape. If bins are separated by (D ¡ 1)w, w being odd, half the spike combinations do not verify the coincidence criterion of equation 3.1 and the number of CEs is reduced by two. (B) The problem is solved extensively for D D 5w. Figures on the right represent the fraction of spikes from these bins, which would form CEs within the central bin of interest (shown with thick vertical lines).The values given are used to construct the matrix Mj,k in equation 3.5.

658

Quentin Pauluis and Stuart N. Baker

Table 1: Evaluation of the Numbers of False-Positive CEs Occurring in Randomly Simulated Independent Spike Trains. Frequency

IDPM

PSTHM

Difference

(Hz)

a D 0.005

a D 0.025

a D 0.005

a D 0.025

a D 0.005

a D 0.025

10 30 50 70

6 2 0.5 0

26.3 15.8 8.3 2.3

6.5 3.3 0.5 0.3

26 20 12.5 4

¡0.5 ¡1.3¤ 0 ¡0.3

0.3 ¡4.3¤ ¡4.3¤ ¡1.8¤

Notes: One hundred trials, 1500 ms long, were simulated at a constant Žring frequency for the two cells. CEs were counted within the central 500 ms of the simulation. The table shows the percentage of simulations with signiŽcant CEs in a sample of 400 simulations with the same Žring frequency. Two values for the criterion level a were used. Results for the methods using instantaneous discharge probability (IDPM) and normalization by the PSTH (PSTHM) are presented. The PSTH normalization method gives more false positives, although false-positive rates are high for both methods with a D 0.025. ¤ D a signiŽcant difference between the two methods (binomial test, p < 0.05).

Grun ¨ (1996), Žring rate is Žrst averaged across trials, and then the expected number of coincidences is calculated as a product of Žring rates. In the current method, the expectation product is calculated on a single trial, and this is then summed. These will produce different results if the Žring rates show any covariation from trial to trial about their mean proŽles. In agreement with this, Table 1 presents data showing that both methods are indeed similar if data are stationary. Two spike trains were generated as independent gamma processes, order 4, with constant Žring rate. Simulations were run for 1500 ms, with 100 trials per simulation; 400 simulations were run for each set of condi-

Figure 4: Facing page. Performance of the two methods on spike trains with Žring-rate changes that covary in their timing. One hundred trials were generated following the sequence of Žring rate changes shown in (A). The black bars indicate the time range over which the rate changes began (uniform probability distribution). The Žrst column shows the results obtained with the single-trial expectancy calculation, and the second column with the method based on the mean Žring rate. (G) Rasters of activity over the 100 trials for both cells. (B) Reconstructed instantaneous frequency for both cells during the Žrst trial (Žlter SD between rate change points: 10 ms) and (H) mean Žring rates assessed over all trials. (C and I) CE rasters. (D and J) CE expectancy (solid) and CE observed rate (dots) smoothed with a 101 ms moving window. (E and K) Probability plot (same display as Figures 3G and 3P). (F and L) CE rasters. The CEs in windows that exceed the 0.5% conŽdence limits are marked by a square. A large number of false positives occur with the mean Žring-rate method.

Instantaneous Discharge Probability

659

tions. The number of simulations with signiŽcant CEs in the central 500 ms was counted using both of the methods described above, for two different signiŽcance criterion levels a. At the more relaxed criterion of a D 0.025, a large number of false positives occur with each method, although the

660

Quentin Pauluis and Stuart N. Baker

situation is slightly better for the instantaneous discharge probability technique. With the more rigorous criterion of a D 0.005, the two methods give comparable numbers of false positives. Figure 4 illustrates the performance of the two algorithms on simulated data, which have a covariation in the timing of the Žring-rate modulation. The spike trains were independent, apart from the rate covariation, so that no signiŽcant CEs should be detected. For a particular trial, the underlying Žring-rate proŽle of the two cells was the same. Figure 4A shows how the underlying Žring rate changed with time; the onset of the two periods of increased Žring rate was uniformly distributed over the times shown by the black bars; the timing of the two bursts was independent. One hundred such trials were simulated. Figure 4G shows rasters of the unit activities simulated according to the Žring-rate proŽle illustrated in Figure 4A. The estimated instantaneous discharge probability density of the Žrst trial (see Figure 4B) can be compared with the Žring rate derived by averaging across all trials in Figure 4H. Spikes that were coincident within 5 ms are shown in Figures 4C and 4I. The number of CEs, counted in a 101 ms wide moving window are plotted as dots in Figures 4D and 4J. Overlaid as a solid line is the expected number of coincidences, calculated according to the instantaneous discharge probability method (see Figure 4D) or the PSTH method (see Figure 4J). While the two plots closely follow one another in Figure 4D, there are discrepancies in Figure 4J associated with the changes in Žring rate. These differences reach signiŽcance on the calculation of equation 2.5, such that the signiŽcance plot of Figure 4K goes outside the 0.5% conŽdence intervals. In Figures 4F and 4L, following the display convention established by Grun ¨ (1996), we display rasters of the CEs in which all events falling within a window that tested as signiŽcant are highlighted by a square. It is clear that the use of a trial-averaged Žring-rate estimate can lead to errors in signiŽcance testing of unitary events when the neurone Žring rates show a temporal jitter relative to the analysis alignment point, and that jitter is not independent between the two units. Our method based on single-trial discharge probability estimates is capable of alleviating this problem. The

Figure 5: Facing page. Performance of the two methods on spike trains that show covariant Žring rates. (A) Schematic indicating the simulated Žring-rate proŽles. After an initial period of constant rate, the rate changed to a value chosen at random from the range shown. Three further rate changes occurred. The organization of the panels is the same as in Figure 4, with the methods based on the single-trial expectancy in the left column, and the mean Žring-rate approximation on the right. Since the mean Žring rate does not change in these simulations, the PSTH-based method cannot predict the increased number of CEs expected, and a large number of false positives result.

Instantaneous Discharge Probability

661

improvement primarily results from the changed order of the operators (multiply, then sum, versus sum, then multiply). It is likely that almost as much of an improvement would be obtained with simpler estimates of the

662

Quentin Pauluis and Stuart N. Baker

instantaneous discharge probability, such as the ISI reciprocal (smoothed or not) or a low-pass Žltered spike train. Our method, however, is capable of following both fast discharge probability changes (via insertion of breakpoints) and slow ones (via the smoothing of the ISI reciprocal); it is therefore preferred. Figure 5 illustrates the results of analysis on spike trains simulated to display variation in Žring rate from trial to trial. The underlying rates of the two neurones on a given trial were the same, allowing the effect of rate covariation to be examined. The simulated rate proŽles are shown schematically in Figure 5A. The remainder of Figure 5 follows the same display conventions as Figure 4. It can be seen that the trial-averaged Žring rates (smoothed PSTH) in Figure 5H show no modulation: this is expected from Figure 5A, since the mean Žring rates do not change. However, the increase in Žringrate variability brings about a rise in the number of CEs, shown by the dots in Figures 5D and 5J. This cannot be predicted by the trial-averaged measure, which therefore assigns a high signiŽcance level to these extra events. By contrast, the expected and observed curves show good agreement using the instantaneous discharge probability method of Figure 5D, such that no CEs reach signiŽcance. Figure 6 examines the detection of genuine excess CEs by the new algorithm. Figures 6A–6F follow the same format as Figures 4 and 5. One spike train was simulated as previously by a simple gamma process (order 4). Some of these spikes were selected at random to become “imposed CEs.”

Figure 6: Facing page. Detection of injected CEs by the method based on the single-trial CE expectation, and comparison of the sensitivity of the two methods. One hundred trials were simulated with a constant Žring rate of 30 Hz; between ¡0.4 and 0.25 s, 371 coincident events were imposed on the spike train of the second cell (20% of the spikes of cell 1; this yielded 893 CEs in total, 250 more than the 643 CEs expected from a control simulation). (A–F) Detection of imposed CEs (same display conventions as Figures 4 and 5, left column). (G) Data from simulations of 100 trials at a constant Žring rate, which investigated the sensitivity of the two methods. Trials lasted 500 ms; CEs were imposed during the central 150 ms. The number of windows that exceeded signiŽcance is plotted versus the number of excess CEs. Different curves show results for different simulated background Žring rates. Circles represent the method based on single-trial expectancy (instantaneous discharge probability method, IDPM) and triangles the method based on the mean Žring rates (PSTHM). Bars indicate one standard deviation in a sample of 20 simulations. The methods show similar sensitivity; the decrease seen for high Žring rates is due to the high level of chance coincidences expected. (H) Number of CEs within windows that tested as signiŽcant versus the number of excess CEs, for the same simulations as shown in (G). The correlation is good, indicating that many of the injected CEs are retrieved. Filter SD between rate change points: 10 ms.

Instantaneous Discharge Probability

663

The second spike train was also initially simulated as a gamma process; the spike closest to each imposed CE was then moved to be at the same time as this event. This produced an increased number of CEs above the chance

664

Quentin Pauluis and Stuart N. Baker

level but maintained the Žring rate of the two units. In Figures 6E and 6F, the CEs injected between ¡0.4 and 0.25 s were accurately detected. During this period, the Žring rate was constant at 30 Hz, and there were 38% more CEs than expected by chance (893 CEs versus 643 in a control simulation; 2.5 extra CEs per trial spread over 650 ms). The right-hand side of Figure 6 compares both methods for CE detection in stationary spike trains at different Žring rates. This was investigated by simulations similar to the one illustrated in Figure 6A, except that the Žring rate and number of imposed CEs were varied systematically. Figure 6G plots the number of bins that tested as having a signiŽcant excess of CEs versus the number of excess CEs (calculated as the difference between the mean number of CEs observed and the mean number of CEs during control simulation without imposed CEs). The curves for the two methods lie close together at all tested Žring rates, indicating a similar sensitivity for each method. The decline of apparent sensitivity at high Žring rates is due to the high number of CEs that would be expected by chance at these rates; larger numbers of excess CEs are therefore necessary before signiŽcance is reached Figure 6H plots the number of detected CEs (the number of CEs in the signiŽcant bins) versus the number of excess CEs; for both methods there is a good correlation. Points lie above the identity line because signiŽcance is attributed to all the CEs in the 101 ms window; there is no means of dissociating the injected CEs from those that occurred by chance. The similar performance when conditions of Žring-rate stationarity are fulŽlled indicates that the different results in more complex situations cannot be simply explained by a difference in overall sensitivity between the two methods. 4 Discussion 4.1 Instantaneous Discharge Probability Density Calculation. The instantaneous discharge probability density function has a number of attractive features, which make it a useful measure for single-trial analysis. It is able to cope well with a large range of Žring rates, unlike methods that smooth pulses derived from the spike train (French & Holden, 1971; Houk et al., 1987). It is locally smooth, except at a few points of rapid change and does not show great uctuations simply due to the variable nature of ISIs. However, by explicitly detecting points of rate change, blurring of rapid changes is avoided. The probabilistic framework for the detection of rate changes means that it is an entirely objective method in application. Finally, the use of a general gamma distribution as the underlying spike train model permits accurate representation of neurones with many different Žring characteristics simply by changing the order parameter a. 4.2 Unitary Event SigniŽcance Testing. The results presented demonstrate that the current methods used for assessing the signiŽcance of uni-

Instantaneous Discharge Probability

665

tary events can lead to errors when there is variability in the Žring rate of the neurones from trial to trial. Such variation in responses is less likely in sensory systems where the response to a stimulus is measured under anesthesia, since relatively few synapses are interposed between peripheral receptor and recorded neurones. By contrast, in awake animals subject to attentional uctuations, trial-to-trial variation is the norm. Animals can never be trained to produce precisely reproducible behavior, and even if the overt output appears similar, it is likely that there are many different redundant patterns of neural activity that will lead to the same goal. Lee, Port, Kruse, and Georgopoulos (1998) showed that single-trial Žring rates often show correlation (noise correlation in their terminology), which is strongest for cells close together. The situations presented in Figures 4 and 5 are therefore not artiŽcially contrived to test current methods to their limits, but rather represent realistic possibilities common in experimental recordings. Other methods that rely on trial-averaged measures, such as the joint peristimulus time histogram (JPSTH) of Aertsen et al. (1989), or that assume intertrial constancy, such as the widely used shufe predictor (Perkel, Gerstein, & Moore, 1967; Engel, Konig, Gray, & Singer, 1990; Lee et al., 1998) are also vulnerable to error in such situations. When there is little variation in Žring rate from trial to trial, the proposed method based on instantaneous discharge probability has similar sensitivity to use of the PSTH (see Table 1 and Figure 6). The novel method is therefore prefereable when analyzing a wide range of empirical data. Any technique for signiŽcance testing of CEs will make assumptions (the null hypothesis); a signiŽcant excess of CEs indicates a region of the data where these assumptions do not hold. Normalization using the PSTH makes the assumption that each trial is a realization of a random process whose discharge probability changes as the PSTH. This will detect as signiŽcant all correlation between the two spike trains, over both the short term, and longer time course covariation of the Žring rates about the trialaveraged level. Our method assumes that the neurones Žre at random in response to an input that either changes smoothly over time or makes abrupt jumps in magnitude. CEs that are assigned signiŽcance using this approach result from transient input uctuations correlated between the two neurones and occur over a timescale briefer than that implied by the assumed smooth changes in input. We consider that this is appropriate, since we are interested in short-term synchronization, which may reect processes of temporal coding rather than longer-term comodulation of Žring rate. Artifactual CEs could be produced, however, if other assumptions of the method are violated. For example, consider the situation where the neurone is more regular in its response than assumed in the algorithm used to position breakpoints (e.g., the cell Žres as a gamma process of order 4, but we test it against a Poisson process). In this case, the algorithm will fail to detect most of the step changes in input undergone by the cell; if these input changes covary between the two neurones investigated, an artifactual ex-

666

Quentin Pauluis and Stuart N. Baker

cess of CEs will be detected. It is therefore important to adjust the assumed order of the gamma distribution to agree with the observed ISI distribution in experimental data. There is one interesting circumstance in which our method will fail to assign signiŽcant CEs; it is debatable whether this is desirable. If a neurone Žres brief bursts of only a few spikes, breakpoints will be set to delimit the bursts. The instantaneous discharge probability will therefore show a sharp increase closely surrounding the burst. If two neurones burst simultaneously, the expected CE count will rise sharply during the burst, such that any CEs that do occur are unlikely to reach signiŽcance. Whether such CEs should be treated as signiŽcant is an open question. Some other instantaneous discharge probability estimates could be proposed that would smooth across the burst limits; they would result in a measure of burst coincidence between the two trains rather than a measure of spike coincidence inside synchronous bursts. Alternative measures more suited to the data, such as the correlation between neurones of burst onset latency measured on single trials, would probably be simpler to interpret. The one report to date that has applied unitary joint-event analysis to experimental data demonstrated that signiŽcant CEs were seen in association with rate changes and during periods of constant Žring rate (Riehle et al., 1997). No data were presented on the variability of Žring rate around rate changes, but it is likely that some of these CEs would not be signiŽcant using our improved technique. In the instances where signiŽcant CEs occurred without a modulation in Žring rate, it is possible that there was nevertheless a change in the variability of discharge, as simulated in Figure 5. This could also lead to erroneous acceptance of chance CEs as signiŽcant. The unitary joint-event method has received considerable interest, since it appears to allow the detection of synchronous Žring on single trials (Fetz, 1997). However, as made clear by section 3, determination of the signiŽcance of CEs requires averaging across trials; this article has been concerned primarily with the best way in which to do this. Formally, therefore, the unitary joint-event method is similar to time-resolved cross-correlation methods, such as the JPSTH (Aertsen et al., 1989), except that it concentrates only on the central portion of the cross-correlation histogram, and then identiŽes the moment and the trials where synchronous Žrings occurred that contributed to signiŽcant cross-correlation peaks. Whether this trial identiŽcation is useful depends on our underlying model for how synchrony occurs and is used in information processing in the brain. If we postulate that synchrony between the two recorded neurones is itself an important event, then knowing the trials on which it occurs is of obvious value in investigating the relationship between the CEs and, for example, task precision or reaction time. However, we may alternatively assume that the brain is composed of many small assemblies of neurones and that information is carried by the synchronous Žring of a random subset of cells in one assembly (e.g., the synŽre chain hypothesis of Abeles, 1991; Pauluis, Baker, & Olivier, 1999).

Instantaneous Discharge Probability

667

This seems likely, since at any moment some cells of a given assembly may be refractory or otherwise nonexcitable due to their recent participation in other assemblies. If this is the case, trials on which the two cells whose discharge is monitored Žre synchronously are those trials where not only this assembly was active, but also where both cells were (by chance) able to participate in it. Knowing which trials contained the synchronous events is, on this model, less useful than knowing where, within the task, there was on average an excess of CEs. We have shown that modiŽcation of unitary event analysis to use an instantaneous discharge probability density measure can compensate accurately for changes in the occurrence rate of CEs produced by Žring-rate covariation. It is anticipated that this technique, or others based on the instantaneous discharge probability density, will Žnd wide applicability in the analysis of synchronization among neural populations recorded from awake behaving animals. Acknowledgments

We thank Sonia Grun ¨ for helpful discussions. This work was supported by a British Council–CGRI travel grant. S.N.B. is supported by the UK MRC and Christ’s College, Cambridge. References Abeles, M. (1991). Corticonics. Cambridge: Cambridge University Press. Aertsen, A. M. H. J., Gerstein, G. L., Habib, M. K., & Palm, G. (1989). Dynamics of neuronal Žring correlation: Modulation of “effective connectivity.” J Neurophysiol., 61, 900–917. Awiszus, F. (1988). Continuous functions determined by spike trains of a neuron subject to stimulation. Biol. Cybern., 58, 321–327 Berry, M. J., & Meister, M. (1998).Refractoriness and neural precision. J. Neurosci., 18, 2200–2211. Bessou, P., Laporte, Y., & Pag`es, B. (1968). A method of analysing the response of spindle primary endings to fusimotor stimulation. J. Physiol., 196, 37–45. Brody, C. D. (1997).Latency, excitability, and spike timing correlations. Soc. Neurosci. Abstr., 23, 14. Brody C. D. (1999a).Disambiguating different covariation types. Neural Comput., 11, 1527–1535. Brody, C. D. (1999b). Latency, excitability, and spike timing correlations. Neural Comput., 11, 1537–1551. Butler, E. G., Horne, M. K., & Hawkins, N. J. (1992). The activity of monkey thalamic and motor cortical neurones in a skilled, ballistic movement. J. Physiol., 42, 25–48. Butler, E. G., Horne, M. K., & Rawson, J. A. (1992). Sensory characteristics of monkey thalamic and motor cortex neurones. J. Physiol., 442, 1–24.

668

Quentin Pauluis and Stuart N. Baker

Churchward, P. R., Butler, E. G., Finkelstein, D. I., Aumann, T. D., Sudbury, A., & Horne, M. K. (1997). A comparison of methods used to detect changes in neuronal discharge patterns. J. Neurosci. Meth., 76, 203– 210. Cocatre-Filgien, J. H., & Delcomyn, F. (1992). IdentiŽcation of bursts in spike trains. J. Neurosci. Meth., 41, 19–30. de Ruyter van Steveninck, R. R., Lewen, G. D., Strong, S. P., Koberle, R., & Bialek, W. (1997). Reproducibility and variability in neural spike trains. Science, 275, 1805–1808. Eggermont, J. J., Smith, G. M., & Bowman, D. (1993). Spontaneous Žring in cat primary auditory cortex: Age and depth dependence and its effect on neural interaction measures. J. Neurophysiol., 69, 1292–1313. Ellaway, P. H. (1978). Cumulative sum technique and its application to the analysis of peristimulus time histograms. Electroenceph. Clin. Neurophys., 45, 302– 304. Engel, A. K., Konig, P., Gray, C. M., & Singer, W. (1990). Stimulus-dependent neuronal oscillations in cat visual-cortex—intercolumnar interaction as determined by cross-correlation analysis. Europ. J. Neurosci., 2, 588–606. Fetz, E. E. (1997). Temporal coding in neural populations? Science, 278, 1901– 1902. French, A. S., & Holden, A. V. (1971). Alias-free sampling of neuronal spike trains. Kybernetik, 8, 165–171. Gerstein, G. L., & Kiang, N. Y. S. (1960). An approach to the quantitative analysis of electrophysiological data from single neurons. Biophys. J., 1, 15–28. Grun, ¨ S. (1996). Unitary joint-events in multiple-neuron spiking activity. Thun, Frankfurt am Main: Harri Deutsch. Houk, J. C., Dessem, D. A., Miller, L. E., & Sybirska, E. H. (1987). Correlation and spectral analysis of relations between single unit discharge and muscle activities. J. Neurosci. Methods, 21, 201–224. Hurst, H. E. (1950). Long-term storage capacity of reservoirs. Proc. Amer. Soc. Civil Engineering, 76, 1–30. Koch, C., Rapp, M., & Segev, I. (1996). A brief history of time (constants). Cereb. Cortex, 6, 93–101. Lee, D., Port, N. L., Kruse, W., & Georgopoulos, A. P. (1998). Variability and correlated noise in the discharge of neurons in motor and parietal areas of the primate cortex. J. Neurosci., 18, 1161–1170. Leg´endy, C. R., & Salcman, M. (1985). Bursts and recurrences of bursts in the spike trains of spontaneously active striate cortex neurons. J. Neurophysiol., 53, 926–939. Lemon, R. N. (1984). Methods for neuronal recording in conscious animals. London: Wiley. MacPherson, J. M., & Aldridge, J. W. A. (1979). A quantitative method of computer analysis of spike train data collected from behaving animals. Brain Res., 175, 183–187. Matthews, P. B. C. (1963). Apparatus for studying the responses of muscle spindles to stretching. J. Physiol., 169, 58–60P. Munoz, D. P., & Wurtz, R. H. (1993).Fixation cells in monkey superior colliculus.

Instantaneous Discharge Probability

669

I. Characteristics of cell discharge. J. Neurophysiol., 70, 559–575. Pauluis, Q., Baker, S. N., & Olivier, E. (1999). Emergent oscillations in a realistic network: The role of inhibition and the effect of spatiotemporal distribution of the input. J. Comput. Neurosci., 6, 27–48. Perkel, D. H., Gerstein, G. L., & Moore, G. P. (1967). Neuronal spike trains and stochastic point processes. II. Simultaneous spike trains. Biophys. J., 7, 419– 440. Richmond, B. J., Optican, L. M., Podell, M., & Spitzer, H. (1987).Temporal encoding of two-dimensional patterns by single units in primate inferior temporal cortex. I. Response characteristics. J. Neurophysiol., 57, 132–146. Richmond, B. J., Optican, L. M., & Spitzer, H. (1990). Temporal encoding of two-dimensional patterns by single units in primate primary visual cortex. I. Stimulus-response relations. J. Neurophysiol., 64, 351–369. Riehle, A., Grun, ¨ S., Diesmann, M., & Aertsen, A. (1997). Spike synchronization and rate modulation differentially involved in motor cortical function. Science, 278, 1950–1953. Seidemann, E., Meilijson, I., Abeles, M., Bergman, H., & Vaadia, E. (1996).Simultaneously recorded single units in the frontal cortex go through sequences of discrete and stable states in monkeys performing a delayed localization task. J. Neurosci., 16, 752–768. Schwartz, A. B. (1992). Motor cortical activity during drawing movements: Single-unit activity during sinusoid tracing. J. Neurophysiol., 68, 528–541. Shadlen, M. N., & Newsome, W. T. (1994). Noise, neural codes and cortical organization. Curr. Opin. Neurobiol., 4, 569–579. Silverman, B. W. (1986). Density estimation for statistics and data analysis. London: Chapman & Hall. Softky, W. R., & Koch, C. (1992). Cortical cells should Žre regularly, but do not. Neural Comput., 4, 643–646. Received August 31, 1998; accepted March 1, 1999.

LETTER

Communicated by Anthony Zador

Impact of Correlated Inputs on the Output of the Integrate-and-Fire Model Jianfeng Feng David Brown Computational Neuroscience Laboratory, Babraham Institute, Cambridge CB2 4AT, U.K. For the integrate-and-Žre model with or without reversal potentials, we consider how correlated inputs affect the variability of cellular output. For both models, the variability of efferent spike trains measured by coefŽcient of variation (CV) of the interspike interval is a nondecreasing function of input correlation. When the correlation coefŽcient is greater than 0.09, the CV of the integrate-and-Žre model without reversal potentials is always above 0.5, no matter how strong the inhibitory inputs. When the correlation coefŽcient is greater than 0.05, CV for the integrateand-Žre model with reversal potentials is always above 0.5, independent of the strength of the inhibitory inputs. Under a given condition on correlation coefŽcients, we Žnd that correlated Poisson processes can be decomposed into independent Poisson processes. We also develop a novel method to estimate the distribution density of the Žrst passage time of the integrate-and-Žre model. 1 Introduction

Although the single-neurone model has been widely studied using computer simulation, most of these studies have made the assumption that inputs are independent (Brown & Feng, 1999; Feng, 1997; Feng & Brown, 1998a, 1998b; Tuckwell, 1988) both spatially and temporally. This assumption obviously contradicts the physiological data, which clearly show that nearby neurones usually Žre in a correlated way (Zohary, Shadlen, & Newsome, 1994)—what we term spatial correlation—and neurones with similar functions frequently form groups and Žre together. In fact “Žring together, coming together ” is a basic principle in neuronal development (Sheth, Sharma, Rao, & Sur, 1996). Furthermore data in Zohary et al. (1994) show that even a weak correlation within a population of neurones can have a substantial impact on network behavior, which suggests that when comparing simulation results with experimental data, it is of vital importance to investigate the effects of correlated input signals on the output of single cells. We address the following two important issues: how correlation between the inputs affects the output of single-neurone models and when and how Neural Computation 12, 671–692 (2000)

c 2000 Massachusetts Institute of Technology °

672

Jianfeng Feng and David Brown

correlated inputs can be transformed into equivalent independent inputs. The second issue is interesting since it is usually not easy to model or analyze a correlated system, except when it is gaussian distributed. Theoretically we do not have general analytical tools to handle correlated systems, even to Žnd analytical expressions for the distribution. It is even difŽcult numerically to generate a correlated random vector. Therefore, a transformation from correlated to independent inputs greatly simpliŽes the problem theoretically and numerically and, furthermore, provides insights on the functional role of the correlation. The technique can be applied to both simulation of biophysical models and to experiments in order to generate correlated Poisson inputs. The neuronal model we use is the integrate-and-Žre model with or without reversal potentials. Positive correlation in input signals usually enlarges the coefŽcient of variation (CV) of the integrate-and-Žre model. When the correlation coefŽcient is greater than 0.09, we Žnd that CV for the leaky integrate-and-Žre model without reversal potentials is always greater than 0.5. When the correlation coefŽcient is greater than 0.05, CV for the equivalent model with reversal potentials is always greater than 0.5. There has been much research devoted to Žnding a neuronal mechanism that can generate spike trains with a CV greater than 0.5 (see Brown & Feng, 1999; Feng, 1999; Feng & Brown, 1998b, 1999a; Konig, Engel, & Singer, 1996 and references therein). To the best of our knowledge, this is the Žrst article to provide a possible mechanism for doing this for the integrate-and-Žre model independently of the ratio between the inhibitory and excitatory inputs. Independently and experimentally, Stevens and Zador (1998) found that correlation between inputs was necessary in order to generate a sufŽciently large CV in output spike trains of neurons in neocortical slices. Our analysis and simulations therefore provide a theoretical explanation of their Žndings. We also show that it is only under a restrictive condition that dependent inputs can be decomposed into independent inputs. Finally we also propose a novel method of Žnding the distribution density of the Žrst passage time of the integrate-and-Žre model. Correlation usually imposes a strong restriction on the system and has a profound impact on its output (Gawne & Richmond, 1993; Zohary et al., 1994). Imagining a neurone exclusively receiving excitatory postsynaptic potentials (EPSPs) from a population of p excitatory neurones with mutually positive correlation coefŽcient c, then the output CV of the perfect integrateand-Žre neurone (see section 3) is proportional to [1 C ( p2 ¡ p) c / p] / Nth D [1 C ( p ¡ 1)c] / Nth , where Nth is the number of EPSPs required for the cell to Žre. The Žrst term 1 / Nth is well known (Softky & Koch, 1993; Abbott, Varela, Sen, & Nelson, 1997) and is responsible for what is usually termed the central limit theorem effect. The important difference between the Žrst and second terms is that the latter depends on p, which is usually a large number. The appearance of this factor here and not in the Žrst term is due to the fact that

Impact of Correlated Inputs

673

the correlation is a second-order statistical quantity. Hence, even when c is small, say 0.1, the quantity ( p ¡ 1)c can be substantially greater than the Žrst term, approaching or exceeding Nth , thus resulting in a high CV. This is the second of our series of articles aiming to elucidate how more realistic inputs, in contrast to conventional independently and identically distributed Poisson inputs, which have been intensively studied in the literature, affect the outputs of simple neuronal models. We aim to describe as completely as possible the full spectrum of the behavior inherent in these models, documenting more thoroughly their restrictions and potential. Feng and Brown (1998b) have considered the behavior of the integrate-and-Žre model subject to independent inputs with different distribution tails. In the near future, we will report on more complicated biophysical models with realistic inputs as we develop here and in Feng and Brown (1998b). 2 Models

In this section we deŽne the models used and then discuss the effects of including reversal potentials on the dynamic behavior of the leaky integrateand-Žre model, attempting to elucidate the essential difference this makes to its behavior. Suppose that a cell receives EPSPs at p excitatory synapses and inhibitory postsynaptic potentials (IPSPs) at q inhibitory synapses. The activities among excitatory synapses and inhibitory synapses are correlated, but for simplicity of notation here, we assume that the activities of the two classes are independent of each other. When the membrane potential Vt is between the resting potential Vrest and the threshold Vthre , p q X X 1 dV t D ¡ Vt dt C a dEi ( t) ¡ b dIj (t), c iD1 jD1

(2.1)

where 1 /c is the decay rate, Ei ( t), Ii (t) are Poisson processes with ratel E and l I , respectively, and a, b are magnitude of each EPSP and IPSP (Thomson, 1997). Once Vt crosses Vthre from below, a spike is generated, and Vt is reset to Vrest . This model is termed the integrate-and-Žre model. The interspike interval of the efferent spike process is T D infft: Vt ¸ Vthre g. Without loss of generality, we assume that the correlation coefŽcient between ith excitatory (inhibitory) synapse and jth excitatory (inhibitory) synapse is c(i, j) D q

h(Ei ( t) ¡ hEi (t)i)( Ej ( t) ¡ hEj ( t)i)i

h(Ei ( t) ¡ hEi (t)i)2 i ¢ h(Ej ( t) ¡ hEj (t)i)2 i

D r (ki ¡ jk),

674

Jianfeng Feng and David Brown

where r is a nonincreasing function. More speciŽcally, we consider two structures for the correlation: block-type correlation and gaussian-type correlation. Here block-type correlation means that N cells are divided into N / k blocks, and cell activities inside each block are correlated, whereas between blocks they are independent; gaussian-type correlation implies that the correlation between cells is a decreasing function of their geometrical distance (we impose a periodic boundary condition). A slightly more general model than the integrate-and-Žre model deŽned above is the integrate-and-Žre model with reversal potentials deŽned by p

dZt D ¡

q

X X Zt ¡ Vre dt CNa(VE ¡ Zt ) dEi (t) CbN(VI ¡Zt ) dIj (t), c iD1 jD1

(2.2)

where Vre is the resting potential, aN , bN are the magnitude of single EPSP and IPSP, respectively, and VE and VI are the reversal potentials. Zt (membrane potential) is now a birth-and-death process with boundaries VE and VI . Once Zt is below Vre , the decay term Zt ¡ Vre will increase membrane potential Zt ; whereas when Zt is above Vre , the decay term will decrease it. A detailed analysis might clarify the essential difference between the models with and without reversal potentials. Rewrite the model with reversal potentials, equation 2.2, in the following way, p

dZt D ¡

q

X X Zt ¡ Vre dt C aN (VE ¡ Vre ) dEi ( t) C bN( VI ¡ Vre ) dIj ( t) c iD1 jD1

¡ aN (Zt ¡ Vre )

p X iD1

dEi ( t) ¡ bN(Zt ¡ Vre )

q X

dIj ( t).

(2.3)

jD1

We see that the term ¡(Zt ¡ Vre ) /c dt C aN ( VE ¡ Vre )

p X iD1

dEi ( t) C bN(VI ¡ Vre )

q X

dIj ( t)

jD1

is exactly the same as the integrate-and-Žre model without reversal potentials; the coefŽcients of the diffusion term are Žxed and independent of time. Compared to the integrate-and-Žre model without reversal potentials, there is an additional term, ¡Na(Zt ¡ Vre )

p X iD1

dEi (t) ¡ bN(Zt ¡ Vre )

q X jD1

dIj ( t),

Impact of Correlated Inputs

675

which is a pure decay term. Hence equation 2.3 can be expressed in the following way: 2

3 p q X X 1 dZ t D ¡(Zt ¡ Vre ) 4 dt C aN dEi (t) C bN dIj (t) 5 c iD1 jD1 C aN ( VE ¡ Vre )

p X iD1

dEi ( t) C bN( VI ¡ Vre )

q X

dIj (t).

(2.4)

jD1

Therefore, the model with reversal potentials essentially increases the decay rate of membrane potential, resulting in the system’s forgetting its dynamic history more quickly. This property is important when we consider how a group of cells synchronize their behavior (Feng & Brown, 1999b) and is also important for understanding numerical differences in the behavior of models with and without reversal potentials as presented below. Furthermore equation 2.4 might help us reach a theoretical conclusion on a debate (Tuckwell, 1988, p. 165; Wilbur & Rinzel, 1983) that has existed in the literature for decades. 3 Analytical Results

We Žrst use the technique of martingale decomposition to approximate the input to the integrate-and-Žre model. Together with the result in the appendix, this leads in theorem 1 to a formula for the distribution of interspike interval for the perfect integrate-and-Žre model subjected to correlated inputs. We then obtain (in lemma 1) the interspike interval distribution of the leaky integrate-and-Žre model for the special case when the leakage rate is such that the attractor of the deterministic part of the dynamics (Feng & Brown, 1999a) coincides with the spiking threshold. From this result, we Žnd an expression in theorem 2 for the distribution density of interspike interval for the general leaky integrate-and-Žre model, which is then discussed. Finally the model with reversal potentials is considered. First, martingale decomposition is used to approximate the integrateand-Žre model with or without reversal potentials. We do not discuss the approximation accuracy since it has been done by many authors previously (Tuckwell, 1988; Ricciardi & Sato, 1990) (see section 5). The Poisson point processes representing EPSP and IPSP input involve perturbations of membrane potential with the following two properties. First, they are either Žxed in size (for the model without reversal potentials) or of a size dependent on membrane potential (for the model with reversal potentials). Second, they arrive irregularly in time according to Poisson processes. Using martingale decomposition, we effectively rewrite this process as a Wiener or Brownian motion process, which is simulated as a sum of an equivalent continuous current input representing the drift that occurs with unbalanced EPSP/IPSP

676

Jianfeng Feng and David Brown

input and unbiased random input involving independently varying, small, random perturbations of membrane potential, arriving at regularly spaced, very short intervals of time. The martingale decomposition reads p E i (t) » l E t C l E BEi (t), and, similarly, p Ii (t) » l I t C l I BIi ( t), where B Ei ( t) and BIi ( t) are standard Brownian motions. For Žxed time t, the above decomposition is equivalent to the central limit theorem. Therefore the integrate-and-Žre model without reversal potentials can be approximated by p q p X X p X 1 l E dt ¡ b l I dt C a l E dvt D ¡ vt dt C a dBEi ( t) c iD1 iD1 iD1 q p X ¡ b lI dBIi (t). iD1

Since the summation of Brownian motions is again a Brownian motion, we can rewrite the equation above as follows: 1 dvt D ¡ vt dt C (apl E ¡ bql I )dt c v u p X p q X q X X u C ta2l E cE ( i, j) C b2l I cI (i, j)dB( t) iD1 jD1

(3.1)

iD1 jD1

1 D ¡ vt dt C (apl E ¡ bql I )dt c . v u p q X X u C ta2 pl E C b2 ql I C a2l E cE ( i, j) C b2l I cI ( i, j) dB( t) i6 D j

i6 D j

When cE ( i, j) D cI (i, j) D 0, equation 3.1 gives rise to the same results as in the literature (Tuckwell, 1988) for independent inputs. When q D 0 and cE ( i, j) D c, the signal-to-noise ratio of inputs given by equation 3.1 is SNR D p

pl E , pl E C p( p ¡ 1)l E c

which coincides with that in Zohary et al. (1994, Žg. 3a).

Impact of Correlated Inputs

677

Combining the results above with the conclusions in the appendix, we arrive at the following theorem for the perfect integrate-and-Žre model. Let T be the Žrst passage time of vt . If c D 1 the distribution density of T is

Theorem 1.

p( t ) D q

2p ( a2 pl 2

E

Cb2 ql

I

Ca2l

Vthre Pp E P ( ) Cb2l I qi6D j cI ( i, j))t3 E i6 D j c i, j 3

6 6 ( Vthre ¡ (apl E ¡ bql I ) t) 2 exp 6 ¡ Pp 6 2(a2 pl E Cb2 ql I Ca2l E i6D j cE (i, j) 4 Pq C b2l I i6D j cI ( i, j))t

7 7 7 7 5

(3.2)

and in particular

8 > >
Vthre apl E ¡ bql I Pp Pq a2l E i6D j cE ( i, j) C b2l I i6D j cI (i, j) (3.3) > a2 pl E C b2 ql I > :CV D C . Vthre ( apl E ¡ bql I ) Vthre ( apl E ¡ bql I ) Theorem 1 tells us that for the perfect integrate-and-Žre model, correlation between inputs does have an impact on the variation (CV) of output interspike interval, but not on the mean Žring rate. Compared to the perfect integrate-and-Žre model without correlation, the CV of efferent interspike intervals falls between 0.5 and 1 for greater degrees of imbalance between EPSPs and IPSPs (Feng & Brown, 1998b). Furthermore, the output distribution is a long-tailed distribution if and only if an exact balance between EPSPs and IPSPs is reached, although in general correlated inputs will generate efferent spikes with a larger CV. Although theorem 1 is a special case of theorem 2, we prefer to write it as a separate theorem since we apply it in the next section. In appendix A, we present a new and simple proof of theorem 1 without resorting to the classic Laplace transformation (Tuckwell, 1988), which we hope opens up new possibilities for obtaining an analytical formula for general cases—the integrate-and-Žre model without reversal potentials. The idea of our approach is to use Girsanov’s theorem (Protter, 1980), which connects two diffusion processes with a displacement transformation. Once we know the properties of the following diffusion process (s > 0, a constant), dX t D b(Xt ) dt C sdBt ,

678

Jianfeng Feng and David Brown

we can, by Girsanov’s theorem, infer properties of the following diffusion process: dYt D b( Yt ) dt Cm dt C sdBt . For the problem we consider here, we can Žnd the distribution density of the following case, dvN t D ¡vN t /c dt Cm 0 dt C sdBt , with m 0 D Vthre /c ,

(3.4)

the special case that the attractor of the deterministic dynamics coincides with the threshold. (For further interesting properties of the dynamics under the condition equation 3.4, see Feng, 1999; Feng & Brown, 1999a.) Let TN be the Žrst passage time of vN t . Lemma 1.

When m D Vthre /c the distribution density of TN is given by

2s 2 Vthre exp(¡t /c ) p0 ( t) D p p [s 2c (1 ¡ exp(¡2t /c ))]3 " # 2 exp(¡2t /c ) Vthre ¢ exp ¡ 2 . s c (1 ¡ exp(¡2t /c ))

(3.5)

Hence, by using Girsanov’s theorem, as we do in the appendix, we Žnd an analytical formula for the general case, dvt D ¡vt /c dt Cm 0 dt C (m ¡ m 0 )dt C sdBt , as described below. Theorem 2.

The distribution density of the Žrst passage time T of vt is

¡

¡

¢

µ ¶ (m ¡m 0 ) 2 t ¡(m ¡ m 0 )m 0 t Vthre (m ¡m 0 ) C exp p( t) D p0 ( t) exp ¡ 2 2 2s s s2 * ! + Z TN N m ¡m 0 Vt exp ¢ . (3.6) dt| FTN 2 s 0 c N

(

TDt

Impact of Correlated Inputs

679

As in the appendix, we only need to calculate the Radon-Nikodyn derivative in Girsanov’s theorem, which is given by F(t) D exp D exp

¡Z

t 0

1 1 (m ¡ m 0 ) dBs ¡ s 2

¡ m ¡m s

0

Bt ¡

Z

1

t 0

)2

¢

(m ¡ m 0 ) 2 ds s2

¢

1 (m ¡ m 0 t . 2 s2

Therefore we have hg(T)i D

Z

¡

(m ¡ m 0 ) 2 t 2s 2 0 ½ ¾ m ¡m 0 exp BTN | FTN dt, s N TDt g( t) p0 ( t) exp ¡

¡

¢

¢

(3.7)

where g is a measurable, nonnegative function, T is the Žrst passage time, and Ft is the s-algebra up to time t. Now the only remaining task to prove theorem 2 is to Žnd an analytical formula for BTN . From dvN t D ¡vN t /c dt Cm 0 dt C sdBt . we have Z vN TN D ¡

TN 0

vN t dt /c Cm 0 TN C sBTN .

Equation 3.7 thus becomes

¡

¢

µ ¶ (m ¡ m 0 ) 2 t Vthre (m ¡ m 0 ) C g( t) p0 ( t) exp ¡ 2s 2 s2 0 * ! + Z TN ¡(m ¡ m 0 )m 0 t m ¡m 0 vN t exp exp ¢ dt | FTN s2 s2 0 c N Z

hg(T)i D

1

¡

which gives the desired results.

¢

(

dt,

TDt

RN Now let us consider the meaning of the term 0T vN t dt. It is the area under the curve visited by the process vN t , before it hits the threshold, as shown in Figure 1. It is easily seen that we are able to use different ways to apR TN N lemma 1), which gives proximate 0 vN t dt (we know the distribution of T, rise to various approximations of the distribution density of T. To Žnd an analytical formula for the distribution density of T is a problem studied for decades, and a number of different methods, mainly in terms of PDEs, have been employed (see, for example, Ricciardi & Sato, 1990). Here we present a novel and probably the most natural way—by virtue of the probability

680

Jianfeng Feng and David Brown

Figure 1: Dark area above the resting potential ¡ dark area below the resting potential D

R TN 0

vN t dt (Vrest D 0 mV).

method—to approximate the distribution density. (A study on how to improve the accuracy of the approximation is an interesting problem, but it is outside the scope of this article and will be published elsewhere.) In fact, from theorem 2 we can obtain some informative conclusions. For example, we claim that as soon as m ¸ m 0 , then p( t) goes to zero (t ! 1) faster than or equal to µ ¶ (m ¡ m 0 ) 2 1 exp ¡ C t , 2s 2 c

¡

¢

an exponential decreasing function. This is because vN t /c · Vthre /c D m 0 , and therefore exp

¡ ¡(m ¡ m

0 )m 0 t

s2

¢*

(

m ¡m 0 exp ¢ s2

Z

TN 0

! + vN t dt | FTN c N

TDt

· 1.

Now we turn to the model with reversal potentials. Similar to the above treatment of the model without reversal potentials, we can rewrite the model in the following form, p

q

X X 1 l E ( VE ¡ zt ) dt C b l I ( VI ¡ zt ) dt dz t D ¡ zt dt C a c iD1 iD1 p q p X p X C a lE |VE ¡ zt |dBEi ( t) C b l I |VI ¡ zt | dBIi ( t), iD1

iD1

Impact of Correlated Inputs

681

which gives 1 dzt D ¡ zt dt C (ap( VE ¡ zt )l E ¡ bq( zt ¡ VI )l I )dt C sr ( t)dBt , c

(3.8)

where, sr2 ( t) D a2 ( zt ¡ VE ) 2l E

p X p X

cE ( i, j)

iD1 jD1

C b2 ( zt ¡ VI ) 2l I

q X q X

cI ( i, j)

D a2 ( zt ¡ Ve ) 2 pl E C b2 ( zt ¡ VI ) 2 ql I C a2 ( zt ¡ VE ) 2l E C b2 ( zt ¡ VI ) 2l I

(3.9)

iD1 jD1

q X

p X

cE ( i, j)

i6 D j

cI (i, j).

i6 D j

There are other forms for the diffusion terms in equation 3.8 to approximate the original process (Musila & L´ansky, ´ 1994). For simplicity of notation we conŽne ourselves to equation 3.9. 4 Numerical Results

From now on we assume that cE (i, j) D cI (i, j) D c( i, j). In this section we present and discuss the results of numerical simulations of equations 3.1 and 3.8 showing the effect of block-type correlation on mean (ISI) and CV of the leaky integrate-and-Žre model without (see Figure 2) and with reversal potentials (see Figure 4) for moderate EPSP/IPSP sizes, and also for very small EPSP/IPSP sizes (see Figure 3). We also demonstrate the effect of a correlation that falls off as an inverse square law (see Figure 5). 4.1 Block-type Correlation. For simplicity of notation, we consider only the case that c( i, j) D c for i 6 D j, i, j D 1, . . . , p and i 6 D j, i, j D 1, . . . , q. It is reported in the literature (Zohary et al., 1994) that the correlation coefŽcient between cells is around 0.1 in V5 of rhesus monkeys in vivo. In human motor units of a variety of muscles, the correlation coefŽcients are usually in the range 0.1 to 0.3 (Matthews, 1996). In Figure 2 we show numerical simulations for q D 0, 10, 20, . . . , 100, p D 100, Vrest D 0, Vthre D 20 mV, a D b D 0.5 mV, c D 20.2 msec, l E D l I D 100 Hz, a set of parameters as in the literature (Brown & Feng, 1999; Feng, 1999; Feng & Brown, 1998b, 1999a), and c D 0, 0.01, 0.02, . . . , 0.1. When c D 0, inputs without correlation, there

682

Jianfeng Feng and David Brown

Figure 2: Mean Žring time (left) and CV (right) versus q for c(i, j) D 0., 0.01, 0.05, 0.09, 0.1 of the integrate-and-Žre model without reversal potentials. Ten thousand spikes are generated for calculating the mean Žring time and CV. Parameters are speciŽed in the context. As soon as c(i, j) ¸ 0.09, we see that the CV of efferent spike trains is greater than 0.5.

are many numerical and theoretical investigations (see, for example, Softky & Koch, 1993; Ricciardi & Sato, 1990; Feng & Brown, 1998b). As one might expect, the larger the input correlation, the larger the output CV. In particular we note that when c ¸ 0.09, then CV > 0.5 for any value of q. When the ratio q / p is small, the mean Žring time is independent of the correlation, a phenomenon we have pointed out for the perfect integrateand-Žre model (see theorem 1). However when q / p is large (i.e., q ¸ 50), the mean Žring time with weak correlation is greater than that with large correlation. It can be easily understood from the properties of the process since the larger the correlation, the larger the diffusion term, which will drive the process to cross the threshold more often. However, as in the perfect integrate-and-Žre model, CV of efferent spike trains does depend on the ratio and the correlation. The larger the correlation and the ratio, the larger the CV. Many studies have been devoted to the problem of how to generate spike trains with CV between 0.5 and 1 (see, for example, Brown & Feng, 1999; Feng, 1999; Feng & Brown, 1998b, 1999a; Softky & Koch, 1993; Shadlen & Newsome, 1994). In particular, Softky and Koch (1993) point out that it is impossible for the integrate-and-Žre model to generate spike trains with CV

Impact of Correlated Inputs

683

Figure 3: Mean Žring time (left) and CV (right) versus q for c(i, j) D 0., 0.01, 0.05, 0.09, 0.1 of the integrate-and-Žre model without reversal potentials and a D b D 0.05, p D 1000. Ten thousand spikes are generated for calculating the mean Žring time and CV. Parameters are speciŽed in the context. For the case of c(i, j) D 0 and q D 0, the CV is 0.05. Note that we calculate q D 0, 100, . . . , 800 only for c(i, j) D 0 since the mean Žring time is too large when q D 900, 1000.

between 0.5 and 1 if the inputs are exclusively excitatory. A phenomenon referred to as central limit effect is widely cited in the literature (Abbott et al., 1997) as justiŽcation of this statement. Many approaches have been proposed to get around this problem (Shadlen & Newsome, 1994). Here we clearly demonstrate that even with exclusively excitatory inputs, the integrate-and-Žre model is capable of emitting spike trains with CV greater than 0.5. No matter what the ratio between the excitatory and inhibitory synapses, the CV of efferent spike trains is always greater than 0.5, provided that the correlation coefŽcient between the inputs is greater than or equal to 0.09. Furthermore, we want to emphasize that in Zohary et al. (1994) the authors claim, based on their physiological data, that c > 0.1 is the most plausible case. For further conŽrming that a small correlation plays an important role in neuronal output, in Figure 3 we simulate an extreme case with EPSP and IPSP size of 0.05 mV. Shadlen and Newsome (1994) have reported that the size of EPSP is between 0.05 mV and 2 mV. This choice implies that 400 EPSPs are needed to drive a cell to Žre. To have a reasonable comparison with the numerical results of a D 0.5, m is kept as a constant when the ratio

684

Jianfeng Feng and David Brown

Figure 4: Mean Žring time (left) and CV (right) versus q for c(i, j) D 0., 0.01, 0.05, 0.09, 0.1 of the integrate-and-Žre model with reversal potentials. Ten thousand spikes are generated for calculating the mean Žring time and CV. Parameters are speciŽed in the context. As soon as c(i, j) ¸ 0.05, we see that the CV of efferent spike trains is greater than 0.5.

q / p is the same and hence p D 1000 in Figure 3. When c( i, j) D 0 and q D 0, CV is about 0.05 (see Figure 3). Figure 3 clearly shows that a large, efferent CV is again obtained with a small correlation. Now we turn to the integrate-and-Žre model with reversal potentials. As we already pointed out, the essential difference between this model and that without reversal potentials is that the latter has a large-membrane potential decay rate. Therefore in this case, it is natural to expect that a larger efferent spike train CV is obtained since the decay term plays a predominant role. We assume that Vre D ¡50 mV, VI D ¡60 mV,Vth D ¡30, and VE D 50 mV, values that match experimental data. As in the literature (Musila & Lansky, 1994; Feng & Brown, 1998b) we impose a local balance condition on the magnitude of EPSPs and IPSPs: aN ( VE ¡ Vre ) D bN( Vre ¡ VI ) D 1 mV; starting from the resting potential, a single EPSP or a single IPSP will depolarize or hyperpolarize the membrane potential by 1mV (see the discussion in section 3). Figure 4 clearly shows that efferent spike trains of the integrateand-Žre model with reversal potentials are more variable than for the model without reversal potentials (except for high q and c( i, j)). When the correlation coefŽcient is greater than 0.05, CV > 0.5, no matter what the ratio

Impact of Correlated Inputs

685

Figure 5: Mean Žring time (left) and CV (right) versus q for s D 1, 2, 5, 9, 10 of the integrate-and-Žre model without reversal potentials for a gaussian-type correlation pattern. Ten thousand spikes are generated for calculating the mean Žring time and CV. Parameters are speciŽed in the context. As soon as s ¸ 5, we see that the CV of efferent spike trains is greater than 0.5.

between inhibitory and excitatory inputs. Due to this effect (large CV) we see that mean Žring rates are also wider spread than for the model without reversal potentials. When q ¸ 20, the difference between mean Žring times become discernible (see Figure 4). From Figures 2 and 4 we can see another interesting phenomenon. When q / p approaches one, the CV of the integrate-and-Žre model without reversal potential is greater than that of the model with reversal potentials (which itself is less than or equal to one) (see Tuckwell, 1988, p. 165; Wilbur & Rinzel, 1983). The reason can be understood from our analyses in section 3. Note that the decay term will return the system to the resting potential. The model with reversal potentials is equivalent to one with a large decay rate, and so Zt generally remains closer to the resting potential than Vt . When q gets close to 100, the deterministic forces tending to increase membrane potential become weak, and so the process starting from the resting potential will tend to fall below the resting potential more and more often. However, the decay terms together with the reversal potentials in the integrate-andŽre model with reversal potentials will prevent the process from going far below the resting potential. The process Zt is thus more densely packed in a compact region, and a smaller range of CV is observed.

686

Jianfeng Feng and David Brown

4.2 Gaussian-Type Correlation. We suppose that c (i, j) D exp(¡(i ¡ j)2 / s 2 ), i, j D 1, . . . , p with periodic boundary conditions. Hence the larger that s is, the more widely spread the correlation for the ith cell. We simulate the integrate-and-Žre model using the Euler scheme (Feng, Lei, & Qian, 1992) with s D 1, 2, . . . , 10. Similar phenomena as in the previous subsection are observed (see Figure 5): there is a threshold value sc , such that as soon as s > sc , CV of efferent trains becomes greater than 0.5, independent of the number of inhibitory inputs. Similar results are obtained for the integrateand-Žre model with reversal potentials (see the previous subsection) as well (not shown). 5 From Dependent to Independent Synapses

We now consider how we can realize a constant correlated Poisson process. DeŽne Ei ( t) D Ni ( t) C N ( t) where Ni ( t), i D 1, . . . , p are independently and identically distributed Poisson processes with rate (1 ¡ c)l E , and N (t) is an independent Poisson process with rate cl E . It is then easily seen that the correlation coefŽcient between Ei and Ej is c. Therefore, in Pterms of our conclusions above, we can rewrite the total synaptic inputs i E i (t) in the following way: X X (5.1) E i ( t) D Ni ( t) C pN ( t). i

This conclusion tells us that constant correlated Poisson inputs are equivalent to the case where there is a common source—the term N (t) in equation 5.1—for all synapses. We usually prefer independent inputs, which are much easier to deal with than correlated inputs, and we do so again here. Let us rewrite the integrate-and-Žre model in the following way: p

dV t D ¡

q

X X Vt dt C a dNiE ( t) C apdN E ( t) ¡ b dNiI (t) ¡ bqdNI ( t) c iD1 iD1

Vt dt Cm dt c . q C a2 p(1 ¡ c)l E C b2 q(1 ¡ c)l I C a2 p2 cl E C b2 q2 cl E dBt

»¡

The independent term contributes to the variance term with an amount of a2 pl E (1 ¡ c), but the common source contributes with an amount of a2 p2l E c. Since p usually is large, the common source thus plays a predominant role in the model behavior when we employ physiological parameters—a D 0.5, c D 0.1—in the integrate-and-Žre model. We thus ask whether a given series of correlated Poisson processes (suppose that c(i, j), j D 1, . . . , p is not a constant) can be decomposed into a

Impact of Correlated Inputs

687

series of independent Poisson processes. Furthermore, without loss of generality, we assume that l E D 1. Decomposition Theorem. Suppose that p is large and

Theorem 3. p X jD2

c(1, j) ·

1 , 2

(5.2)

the Poisson process E i (t) deŽned below, E i ( t) D N i ( t) C

i¡1 X jD1

Nj,i¡j (t) C

p¡i X

Ni, j ( t)

jD1

is correlated with a correlation between Ei ( t) and Ej ( t) equal to c(1, |i¡ j| ), i 6 D j, j§ 1, where Ni ( t), Ni, j ( t), i D 1, . . . , pI j D 1, . . . , p ¡ i are independent Poisson Pi¡1 Pp¡i processes with rate l Ni D t(1 ¡ jD2 c(1, j) ¡ jD2 c(1, j)) and l Ni, j D tc(1, j). Proof . For i 6 D j, j C 1, j ¡ 1, we obtain

hEi ( t) ¡ hEi ( t)i, Ej (t) ¡ hEj ( t)ii

D hNi ( t) ¡ hNi (t)i, Nj ( t) ¡ hNj ( t)ii C C C C

j¡1 i¡1 X X kD1 lD1 p¡j i¡1 X X kD1 lD1 p¡i X j¡1 X kD1 lD1 p¡i X p¡j X kD1 lD1

hNk,i¡k ( t) ¡ hNk,i¡k ( t)i, Nl, j¡l ( t) ¡ hNl, j¡l ( t)ii hNk,i¡k ( t) ¡ hNk,i¡k ( t)i, Nj,l ( t) ¡ hNj,l ( t)ii hNi,k ( t) ¡ hNi,k ( t)i, Nl, j¡l ( t) ¡ hNl, j¡l ( t)ii hNi,k ( t) ¡ hNi,k ( t)i, Nj,l (t) ¡ hNj,l (t)ii

1 1 (1 C sgn( i ¡ j)) c(1, |i ¡ j| )t C (1 C sgn( j ¡ i))c(1, |i ¡ j| ) t 2 2 D c(1, | i ¡ j| ) t. D

Using a similar algebraic calculation, we have hEi ( t)¡hEi (t)i, Ei (t)¡hEi ( t)ii D t. Therefore the correlation coefŽcient between Ei ( t) and Ej ( t) is given by c(1, |i ¡ j| ).

688

Jianfeng Feng and David Brown

It is obvious that condition 5.2 is rather restrictive since c(i, j) D c with c > 1 / p violates it. However, the following example shows that the condition in theorem 3 is a necessary and sufŽcient condition for a linear decomposition: Example. Consider four cells i D 1, 2, 3, 4 and assume that c(2, 3) > 0 and c(2, 4) D 0. Since we have c(1, 3) D 0, the most efŽcient way to construct E2 is E 2 ( t) D N1 (t) C N3 (t), with hN1 (t)i D c(2, 3) D hN3 ( t)i. Therefore if and only if 2c(1, 2) · 1 do we have a linear decomposition of Ei ( t). 6 Discussion

We have shown that a weak correlation between neurones can have a dramatic inuence on model output. In the integrate-and-Žre model without reversal potentials, when the correlation coefŽcient is above 0.09, the CV of efferent spike trains is above 0.5. In the integrate-and-Žre model with reversal potentials, when the correlation coefŽcient is above 0.05, the CV of efferent spike trains is greater than 0.5 already. These properties are independent of the numbers of inhibitory inputs and therefore resolve an existing problem in the literature: how the integrate-and-Žre model generates spike trains with CV greater than 0.5 when there is a low ratio between inhibitory and excitatory inputs. Note that Zohary et al. (1994) have claimed that a correlation greater than 0.1 is the most plausible value. The integrate-and-Žre model is one of the most widely used singleneuronal models. The advantage when studying it is that there are some useful analytical formulas, although there is a fundamental question open: to Žnd the distribution density of the Žring time. We have proposed a novel approach based on Girsanov’s theorem, which opens up new possibilities for answering this question. Analytical approaches of this type are a useful—and, wherever possible, essential—supplement computer simulation results, since analytically derived expressions are usually very general rather than applicable just to the speciŽc parameter values used in the case of simulation results. We illustrate this by deriving an algebraic form for the interspike interval distribution tail. This could be important for Žtting to experimental data, and hence obtaining parameter estimates, but also has implications explored in our previous work (Feng & Brown, 1998a) when the output of the current layer of neurones is the input to a subsequent layer. When the correlation coefŽcient between neurones is a constant, we can easily decompose them into a linear summation of independent Poisson

Impact of Correlated Inputs

689

processes, and therefore all results in the literature are applicable. Under a sufŽcient condition, we also study the possibility of decomposing a correlated Poisson process into independent Poisson processes. The results can be widely used in biophysical models and experiments. Finally, we discuss the implication of random inputs—Poisson process inputs—in our model. This is a puzzling issue, and a solid answer can be provided only in terms of experiments (Abeles, 1982, 1990). Mainen and Sejnowski (1995) pointed out that “Reliability of spike timing depended on stimulus transients. Flat stimuli led to imprecise spike trains, whereas stimuli with transients resembling synaptic activity produced spike trains with timing reproducible to less than one millisecond.” However, it must be emphasized that their experiments were carried out in neocortical slices. It is interesting to see that the variability of spike trains depends on the nature of inputs, but CV of efferent spike trains in vivo might be very different from those in slices due to the absence of other inputs in slice experiments. For example, it has been reported that CV is between 0.5 and 1 for visual cortex (V1) and extrastriate cortex (MT) (Softky & Koch, 1993; Marsalek, Koch, & Maunsell, 1997) even in human motor cells CV is between 0.1 and 0.25 (Matthews, 1996, p. 597). There are also examples that demonstrate a group of cells in vivo behaving totally differently from in slices. Oxytocin cells Žre synchronously in vivo, but this property is totally lost in slices (see Brown & Moos, 1997, and references). Very recently, it has been reported that random noise of similar amplitude to the deterministic component of the signal plays a key role in neural control of eye and arm movements (Harris & Wolpert, 1998; Sejnowski, 1998). Thus randomness is present—and appears to have a functionally important role—in a number of physiological functions. Appendix

We know that in terms of the reective principle, the distribution density of the Žrst passage time from (¡1, Vthre ) of a Brownian motion sBt is given by p( t ) D p

1 2p t3 s 2

2 Vthre exp(¡Vthre / (2s 2 t)).

We intend to Žnd the distribution density of the process Xt D m t C sBt . For any measurable positive function g, we have hg(T))i D hhg(t)F(t)| F ( TO )ii,

(A.1)

690

Jianfeng Feng and David Brown

where F(t) is the Radon-Nikodyn derivative given by the Girsanov’s theorem, as follows, F( t) D exp

¡Z

t 0

m / s dBs ¡ 2

D exp(m / sBt ¡ m t /

1 2s 2

Z

t 0

s 2 ds

¢

(2s 2 )),

and TO is the Žrst passage time of Bt . Inserting the expression of F( t) into equation A.1 and using the fact that BTO D Vthre / s, we obtain hg(T))i D hg( TO ) exp(m / sB TO ¡ m 2 TO / (2s 2 ))i Z 1 D g(t) exp(m Vthre / s 2 ¡ m 2 t / (2s 2 )) 0

¢p Z D Z D

1 2p t3 s 2

1 0 1 0

2 Vthre exp(¡Vthre / (2s 2 t)) dt

g( t ) p g( t ) p

Vthre 2p s 2 t3 Vthre 2p s 2 t3

2 exp(m Vthre / s 2 ¡ m 2 t / (2s 2 ) ¡ Vthre / (2s 2 t)) dt

exp(¡(m t ¡ Vthre ) 2 / (2s 2 t)) dt.

Therefore the distribution density of T, the Žrst exiting time of Xt , is given by p

Vthre 2p s 2 t3

exp[¡(m t ¡ Vthre )2 / (2s 2 t)].

Acknowledgments

We are grateful to the anonymous referees for bringing references (Stevens & Zador, 1998; Gawne & Richmond, 1993) to our attention and their valuable comments on the manuscript of this article The work was partially supported by BBSRC and an ESEP grant of the Royal Society. References Abeles, M. (1982). Local cortical circuits: An electrophysiological study. New York: Springer-Verlag. Abeles, M. (1990). Corticonics. Cambridge: Cambridge University Press. Abbott, L. F., Varela, J. A., Sen, K., & Nelson, S. B. (1997). Synaptic depression and cortical gain control. Science, 275, 220–223. Brown, D., & Moos, F. (1997). Onset of bursting in oxytocin cells in suckled rats. J. of Physiology, 503, 652–634.

Impact of Correlated Inputs

691

Brown, D., & Feng, J. (1999).Is there a problem matching model and real CV(ISI)? Neurocomputing, 26–27, 117–122. Feng, J. (1997). Behaviours of spike output jitter in the integrate-and-Žre model. Phys. Rev. Lett., 79, 4505–4508. Feng, J. (1999). Origin of Firing varibility of the integrate-and-Žre model. Neurocomputing, 26–27, 87–91. Feng, J., & Brown, D. (1998a).Spike output jitter, mean Žring time and coefŽcient of variation. J. Phys. A: Math. Gen., 31, 1239–1252. Feng, J., & Brown, D. (1998b). Impact of temporal variation and the balance between excitation and inhibition on the output of the perfect integrate-andŽre model. Biol. Cybern., 78, 369–376. Feng, J., and Brown, D. (1999a). CoefŽcient of variation greater than .5 how and when? Biol. Cybern., 80, 291–297. Feng, J., & Brown, D. (1999b). Synchronisation due to common pulsed input in stein’s model. Accepted by Phys. Rev. E. Feng, J., Lei, G., & Qian, M. (1992). Second-order algorithms for SDE. Journal of Computational Mathematics, 10, 376–387. Gawne, T. J., & Richmond, B. J. (1993). How independent are the messages carried by adjacent inferior temporal cortical neurons? J. Neurosci., 13, 2758– 2771. Harris, C. M., & Wolpert, D. M. (1998).Signal-dependent noise determines motor planning. Nature, 394, 780–784. Konig P., Engel A. K., & Singer W. (1996). Integrator or coincidence detector? The role of the cortical neuron revisited. TINS, 19, 130–137. Mainen, Z. F., & Sejnowski, T. J. (1995). Reliability of spike timing in neocortical neurones. Science, 268, 1503–1506. Marsalek, P., Koch, C., & Maunsell, J. (1997).On the relationship between synaptic input and spike output jitter in individual neurones. Proceedings of the National Academy of Sciences U.S.A., 94, 735–740. Matthews, P. B. C. (1996). Relationship of Žring intervals of human motor units to the trajectory of post-spike after-hyperpolarization and synaptic noise. J. Physiology, 492, 597–628. Musila, M., & L´ansky, ´ P. (1994). On the interspike intervals calculated from diffusion approximations for Stein’s neuronal model with reversal potentials. J. Theor. Biol., 171, 225–232. Protter, P. (1980). Stochastic integration and differential equations: A new approach. Berlin: Springer-Verlag. Ricciardi, L. M., & Sato, S. (1990).Diffusion process and Žrst-passage-times problems. In L. M. Ricciardi (Ed.), Lectures in applied mathematics and informatics. Manchester: Manchester University Press. Sejnowski, T. J. (1998). Neurobiology—making smooth moves. Nature, 394, 725– 726. Shadlen, M. N., & Newsome, W. T. (1994). Noise, neural codes and cortical organization. Curr. Opin. Neurobiol., 4, 569–579. Sheth, B. R., Sharma, J., Rao, S. C., & Sur, M. (1996).Orientation maps of subjective contours in visual cortex. Science, 274, 2110–2115. Softky, W., & Koch, C. (1993). The highly irregular Žring of cortical-cells is incon-

692

Jianfeng Feng and David Brown

sistent with temporal integration of random EPSPs. J. Neurosci., 13, 334–350. Stevens, C. F., & Zador, A. M. (1998). Input synchrony and the irregular Žring of cortical neurons. Nature Neurosci., 1, 210–217. Thomson, A. M. (1997). Activity-dependent properties of synaptic transmission at two classes of connections made by rat neocortical pyramidal. J. Physiology, 502, 131–147. Tuckwell, H. C. (1988). Stochastic processes in the neurosciences. Philadelphia: Society for Industrial and Applied Mathematics. Wilbur, W. J., & Rinzel, J. (1983). A theoretical basis for large coefŽcient of variation and bimodality in neuronal interspike interval distributions. J. Theor. Biol., 105, 345–368. Zohary, E., Shadlen, M. N., & Newsome, W. T. (1994). Correlated neuronal discharge rate and its implications for psychophysical performance. Nature, 370, 140–143. Received September 10, 1998; accepted March 4, 1999.

LETTER

Communicated by Raphael Ritz

Weak, Stochastic Temporal Correlation of Large-Scale Synaptic Input Is a Major Determinant of Neuronal Bandwidth David M. Halliday Division of Neuroscience and Biomedical Systems, Institute of Biomedical and Life Sciences, University of Glasgow, Glasgow, G12 8QQ, U.K. We determine the bandwidth of a model neurone to large-scale synaptic input by assessing the frequency response between the outputs of a two-cell simulation that share a percentage of the total synaptic input. For temporally uncorrelated inputs, a large percentage of common inputs are required before the output discharges of the two cells exhibit signiŽcant correlation. In contrast, a small percentage (5%) of the total synaptic input that involves stochastic spike trains that are weakly correlated over a broad range of frequencies exert a clear inuence on the output discharge of both cells over this range of frequencies. Inputs that are weakly correlated at a single frequency induce correlation between the output discharges only at the frequency of correlation. The strength of temporal correlation required is sufŽciently weak that analysis of a sample pair of input spike trains could fail to reveal the presence of correlated input. Weak temporal correlation between inputs is therefore a major determinant of the transmission to the output discharge of frequencies present in the spike discharges of presynaptic inputs, and therefore of neural bandwidth.

1 Introduction

Mammalian central nervous system neurones have extensive dendritic trees (Mel, 1994). The combined effects of large-scale synaptic input to such structures act in several ways, which contribute directly and indirectly to the output discharge of the cell. Bernander, Douglas, Martin, and Koch (1991) and Rapp, Yarom, and Segev (1992) demonstrated that large-scale synaptic activity can alter the integrative properties of a cell by acting to reduce the input resistance and time constant. Bernander et al. (1991) further demonstrated that large-scale background synaptic activity results in an increased sensitivity to highly synchronized inputs. Murthy and Fetz (1994) and Bernander, Koch, and Usher (1994) showed that highly synchronized inputs can modulate the mean Žring rate of a cell. Dendrites are known to act as lowpass Žlters (Rinzel & Rall, 1974), which in terms of populations of synaptic inputs results in a Žltering out of higher-frequency components present in Neural Computation 12, 693–707 (2000)

c 2000 Massachusetts Institute of Technology °

694

David M. Halliday

input spike trains (Farmer, Bremner, Halliday, Rosenberg, & Stephens, 1993; Halliday, 1995). This study examines the response of cells to different strengths of temporal correlations within subpopulations of the total synaptic input. Particular attention is given to an important aspect of neuronal systems: the widespread presence of noise at all levels (Holden, 1976), which results in the random uctuations in membrane potential observed in intracellular recordings (Calvin & Stevens, 1968) and the stochastic nature of both interspike intervals and the temporal correlation between spike trains. This study includes weak stochastic temporal correlation between presynaptic inputs, in contrast to the highly synchronized form of temporal correlation used in the above studies, where all synchronized inputs always Žre together within a narrow time window. The aim is to match natural patterns of correlation more closely (Singer, 1993; Gray, 1994). We consider Žrst the effects of uncorrelated inputs, where a large percentage of common inputs are required before the two cells exhibit signiŽcant correlation. In contrast, for inputs with weak temporal correlation, a small percentage (5–10%) of the total synaptic input can exert a clear inuence on the output discharge at frequencies where the inputs are correlated. Preliminary results are presented in Halliday (1998b). 2 Model Description and Parameters

The model is based on a class of cells with extensive dendrites whose morphology and electrophysiology have been widely studied (Cullheim, Fleshman, Glenn, & Burke, 1987; Fleshman, Segev, & Burke, 1988; Rall et al., 1992): spinal motoneurones. Based on these studies, the membrane resistivity, Rm , is 11, 000V ¢ cm2 , the ctyoplasmic resistivity, Ri , is 70V ¢ cm, and the membrane capacitance, Cm , is 1.0m F / cm2 . Each cell has a spherical soma of50m m diameter with 12 tapered dendrites, 4 each of three types: short, medium, and long. A uniform taper of 0.5m m per 100m m length is used for dendrite diameters in the distal direction. For each dendrite type, the initial diameters at the soma are 5, 7.5, and 10m m; the physical lengths are 766, 1258, and 1904m m; the electrotonic lengths are L D 0.7, 1.0, and 1.5; and the total membrane areas are 8100, 18,500 and 33, 500m m2 , respectively. Each dendrite is modeled as a sequence of connected cylinders, of Žxed electrotonic length of 0.1 unit; the complete cell model has 129 compartments, representing a total membrane area of 248200m m2 , with 97% of this area in the 12 dendrites. The input resistance of the complete model neurone (including soma and tapered dendrites) to hyperpolarizing current injected at the soma is 4.92MV, and the time constant is 9.7 msec. The compartmental model provides a model of current ow in dendrites accurate to second order in both space and time (Segev, Fleshman, & Burke, 1989; Mascagni, 1989). Action potential generation and afterhyperpolarization (AHP) currents are based on the model of Baldissera and Gustafsson (1974). This incorpo-

Dynamic Modulation of Neuronal Bandwidth

695

rates a threshold-crossing mechanism in conjunction with a time-dependent potassium conductance to represent AHP currents. The three-term model for potassium conductances proposed by Baldissera and Gustafsson is reduced to its single dominant term of an exponential function with a time constant of 14 msec. At the low Žring rates used in this study, it is not necessary to include the faster-acting conductances. Output spikes are generated when the membrane potential at the soma exceeds the Žxed threshold of ¡55 mV, and the AHP conductance is activated after each output spike. This conductance produces a rapid hyperpolarization of the membrane potential toward the reversal potential of ¡75 mV. With constant injected current sufŽcient to induce an output Žring rate of 12 spikes/sec, the membrane potential at the soma following each output spike decreases by approximately 12 mV. Individual excitatory synaptic inputs use a time-dependent conductance change, speciŽed by an alpha function: gsyn( t) D Gs t / t e¡t / t (Jack, Noble, & Tsien, 1975), with the time constant for this conductance change set at t D 2e ¡ 04 sec (Segev, Fleshman, & Burke, 1990), and scaling factor Gs D 1.19e ¡ 08, giving a peak conductance change of 4.38 nS at t D 0.2 msec. The membrane resting potential is ¡70 mV, and the reversal potential for excitatory postsynaptic potentials (EPSPs) is ¡10 mV. When activated at the soma from rest, this conductance generates an EPSP with peak magnitude 100m V at t D 0.525 msec, and a rise time (10% to 90%) and half-width of 0.275 msec, 2.475 msec. The EPSP parameters at the soma when activated in the most distal compartment of the short dendrite (L D 0.7) are 52.8m V at t D 2.00 msec, and 0.900, 7.05 msec. When activated in the most distal compartment of the medium dendrite (L D 1.0), the EPSP parameters are 40.4m V at t D 3.08 msec, and 1.35, 9.35 msec. Activation in the most distal compartment of the long dendrite (L D 1.5) gives EPSP parameters at the soma of 27.1m V at t D 4.75 msec, and a rise time and half-width of 2.08, 11.38 msec. Presynaptic inputs are spatially distributed uniformly by membrane area over the soma and dendrites. The level of excitation necessary to induce a constant Žring rate of around 12 spikes/sec in the model cell is 31,872 EPSPs/sec. This can be achieved by 996 uniformly distributed inputs Žring randomly at 32 spikes/sec. In this conŽguration the soma receives 32 inputs, and each dendrite type (short, medium, and long) receives 33, 74, and 134 synaptic inputs each, distributed by area over the surface of the dendrite. In the examples below, where periodic inputs form part of the total synaptic input, the mean rate and number of inputs are adjusted in tandem to provide the same number of EPSPs/sec acting on each compartment of the model. 3 Data Analysis and Inference of Neural Bandwidth

The simulations consist of two cells in which a percentage of the total synaptic input is common to both cells. The cells therefore exhibit a tendency for

696

David M. Halliday

correlated discharge, which we use to estimate the range of frequencies in the common input spike trains that are transmitted to the output discharge of the cells. The data analysis methods are those set out in Rosenberg, Amjad, Breeze, Brillinger, and Halliday (1989) and Halliday et al. (1995) for frequency domain analysis of stochastic neuronal data. We use estimates of coherence functions and cumulant density functions to characterize the correlation between the output discharges. Cumulant density functions provide a measure of correlation between spike trains as a function of time; coherence functions provide a measure of correlation as a function of frequency. The cumulant density funct