Neural Information Processing: 14th International Confernce, ICONIP 2007, Kitakyushu, Japan, November 13-16, 2007, Revised Selected Papers, Part I

Lecture Notes in Computer Science Commenced Publication in 1973 Founding and Former Series Editors: Gerhard Goos, Juris Hartmanis, and Jan van Leeuwen

Editorial Board David Hutchison Lancaster University, UK Takeo Kanade Carnegie Mellon University, Pittsburgh, PA, USA Josef Kittler University of Surrey, Guildford, UK Jon M. Kleinberg Cornell University, Ithaca, NY, USA Alfred Kobsa University of California, Irvine, CA, USA Friedemann Mattern ETH Zurich, Switzerland John C. Mitchell Stanford University, CA, USA Moni Naor Weizmann Institute of Science, Rehovot, Israel Oscar Nierstrasz University of Bern, Switzerland C. Pandu Rangan Indian Institute of Technology, Madras, India Bernhard Steffen University of Dortmund, Germany Madhu Sudan Massachusetts Institute of Technology, MA, USA Demetri Terzopoulos University of California, Los Angeles, CA, USA Doug Tygar University of California, Berkeley, CA, USA Gerhard Weikum Max-Planck Institute of Computer Science, Saarbruecken, Germany

4984

Masumi Ishikawa Kenji Doya Hiroyuki Miyamoto Takeshi Yamakawa (Eds.)

Neural Information Processing 14th International Conference, ICONIP 2007 Kitakyushu, Japan, November 13-16, 2007 Revised Selected Papers, Part I

13

Volume Editors Masumi Ishikawa Hiroyuki Miyamoto Takeshi Yamakawa Kyushu Institute of Technology Department of Brain Science and Engineering 2-4 Hibikino, Wakamatsu, Kitakyushu 808-0196, Japan E-mail: {ishikawa, miyamo, yamakawa}@brain.kyutech.ac.jp Kenji Doya Okinawa Institute of Science and Technology Initial Research Project 12-22 Suzaki, Uruma, Okinawa 904-2234, Japan E-mail: [email protected]

Library of Congress Control Number: Applied for CR Subject Classification (1998): F.1, I.2, I.5, I.4, G.3, J.3, C.2.1, C.1.3, C.3 LNCS Sublibrary: SL 1 – Theoretical Computer Science and General Issues ISSN ISBN-10 ISBN-13

0302-9743 3-540-69154-5 Springer Berlin Heidelberg New York 978-3-540-69154-9 Springer Berlin Heidelberg New York

This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. Springer is a part of Springer Science+Business Media springer.com © Springer-Verlag Berlin Heidelberg 2008 Printed in Germany Typesetting: Camera-ready by author, data conversion by Scientific Publishing Services, Chennai, India Printed on acid-free paper SPIN: 12282845 06/3180 543210

Preface

These two-volume books comprise the post-conference proceedings of the 14th International Conference on Neural Information Processing (ICONIP 2007) held in Kitakyushu, Japan, during November 13–16, 2007. The Asia Pacific Neural Network Assembly (APNNA) was founded in 1993. The first ICONIP was held in 1994 in Seoul, Korea, sponsored by APNNA in collaboration with regional organizations. Since then, ICONIP has consistently provided prestigious opportunities for presenting and exchanging ideas on neural networks and related fields. Research fields covered by ICONIP have now expanded to include such fields as bioinformatics, brain machine interfaces, robotics, and computational intelligence. We had 288 ordinary paper submissions and 3 special organized session proposals. Although the quality of submitted papers on the average was exceptionally high, only 60% of them were accepted after rigorous reviews, each paper being reviewed by three reviewers. Concerning special organized session proposals, two out of three were accepted. In addition to ordinary submitted papers, we invited 15 special organized sessions organized by leading researchers in emerging fields to promote future expansion of neural information processing. ICONIP 2007 was held at the newly established Kitakyushu Science and Research Park in Kitakyushu, Japan. Its theme was “Towards an Integrated Approach to the Brain—Brain-Inspired Engineering and Brain Science,” which emphasizes the need for cross-disciplinary approaches for understanding brain functions and utilizing the knowledge for contributions to the society. It was jointly sponsored by APNNA, Japanese Neural Network Society (JNNS), and the 21st century COE program at Kyushu Institute of Technology. ICONIP 2007 was composed of 1 keynote speech, 5 plenary talks, 4 tutorials, 41 oral sessions, 3 poster sessions, 4 demonstrations, and social events such as the Banquet and International Music Festival. In all, 382 researchers registered, and 355 participants joined the conference from 29 countries. In each tutorial, we had about 60 participants on the average. Five best paper awards and five student best paper awards were granted to encourage outstanding researchers. To minimize the number of researchers who cannot present their excellent work at the conference due to financial problems, we provided travel and accommodation support of up to JPY 150,000 to six researchers and of up to JPY to eight students 100,000. ICONIP 2007 was jointly held with the 4th BrainIT 2007 organized by the 21st century COE program, “World of Brain Computing Interwoven out of Animals and Robots,” with the support of the Japanese Ministry of Education, Culture, Sports, Science and Technology (MEXT) and Japan Society for the Promotion of Science (JSPS).

VI

Preface

We would like to thank Mitsuo Kawato for his superb Keynote Speech, and Rajesh P.N. Rao, Fr´ed´eric Kaplan, Shin Ishii, Andrew Y. Ng, and Yoshiyuki Kabashima for their stimulating plenary talks. We would also like to thank Sven Buchholz, Eckhard Hitzer, Kanta Tachibana, Jung Wang, Nikhil R. Pal, and Tetsuo Furukawa for their enlightening tutorial lectures. We would like to express our deepest appreciation to all the participants for making the conference really attractive and fruitful through lively discussions, which we believe would tremendously contribute to the future development of neural information processing. We also wish to acknowledge the contributions by all the Committee members for their devoted work, especially Katsumi Tateno for his dedication as Secretary. Last but not least, we want to give special thanks to Irwin King and his students, Kam Tong Chan and Yi Ling Wong, for providing the submission and reviewing system, Etsuko Futagoishi for hard secretarial work, Satoshi Sonoh and Shunsuke Sakaguchi for maintaining our conference server, and many secretaries and graduate students at our department for their diligent work in running the conference.

January 2008

Masumi Ishikawa Kenji Doya Hiroyuki Miyamoto Takeshi Yamakawa

Organization

Conference Committee Chairs General Chair Organizing Committee Chair Steering Committee Chair Program Co-chairs

Tutorials Chair Exhibitions Chair Publications Chair Publicity Chair Local Arrangements Chair Web Master Secretary

Takeshi Yamakawa (Kyushu Institute of Technology, Japan) Shiro Usui (RIKEN, Japan) Takeshi Yamakawa (Kyushu Institute of Technology, Japan) Masumi Ishikawa (Kyushu Institute of Technology, Japan), Kenji Doya (OIST, Japan) Hirokazu Yokoi (Kyushu Institute of Technology, Japan) Masahiro Nagamatsu (Kyushu Institute of Technology, Japan) Hiroyuki Miyamoto (Kyushu Institute of Technology, Japan) Hideki Nakagawa (Kyushu Institute of Technology, Japan) Satoru Ishizuka (Kyushu Institute of Technology, Japan) Tsutomu Miki (Kyushu Institute of Technology, Japan) Katsumi Tateno (Kyushu Institute of Technology, Japan)

Steering Committee Takeshi Yamakawa, Masumi Ishikawa, Hirokazu Yokoi, Masahiro Nagamatsu, Hiroyuki Miyamoto, Hideki Nakagawa, Satoru Ishizuka, Tsutomu Miki, Katsumi Tateno

Program Committee Masumi Ishikawa, Kenji Doya Track Co-chairs

Track 1: Masato Okada (Tokyo Univ.), Yoko Yamaguchi (RIKEN), Si Wu (Sussex Univ.) Track 2: Koji Kurata (Univ. of Ryukyus), Kazushi Ikeda (Kyoto Univ.), Liqing Zhang (Shanghai Jiaotong Univ.)

VIII

Organization

Track 3: Yuzo Hirai (Tsukuba Univ.), Yasuharu Koike (Tokyo Institute of Tech.), J.H. Kim (Handong Global Univ., Korea) Track 4: Akira Iwata (Nagoya Institute of Tech.), Noboru Ohnishi (Nagoya Univ.), SeYoung Oh (Postech, Korea) Track 5: Hideki Asoh (AIST), Shin Ishii (Kyoto Univ.), Sung-Bae Cho (Yonsei Univ., Korea)

Advisory Board Shun-ichi Amari (Japan), Sung-Yang Bang (Korea), You-Shou Wu (China), Lei Xu (Hong Kong), Nikola Kasabov (New Zealand), Kunihiko Fukushima (Japan), Tom D. Gedeon (Australia), Soo-Young Lee (Korea), Yixin Zhong (China), Lipo Wang (Singapore), Nikhil R. Pal (India), Chin-Teng Lin (Taiwan), Laiwan Chan (Hong Kong), Jun Wang (Hong Kong), Shuji Yoshizawa (Japan), Minoru Tsukada (Japan), Takashi Nagano (Japan), Shozo Yasui (Japan)

Referees S. Akaho P. Andras T. Aonishi T. Aoyagi T. Asai H. Asoh J. Babic R. Surampudi Bapi A. Kardec Barros J. Cao H. Cateau J-Y. Chang S-B. Cho S. Choi I.F. Chung A.S. Cichocki M. Diesmann K. Doya P. Erdi H. Fujii N. Fukumura W-k. Fung T. Furuhashi A. Garcez T.D. Gedeon

S. Gruen K. Hagiwara M. Hagiwara K. Hamaguchi R.P. Hasegawa H. Hikawa Y. Hirai K. Horio K. Ikeda F. Ishida S. Ishii M. Ishikawa A. Iwata K. Iwata H. Kadone Y. Kamitani N. Kasabov M. Kawamoto C. Kim E. Kim K-J. Kim S. Kimura A. Koenig Y. Koike T. Kondo

S. Koyama J.L. Krichmar H. Kudo T. Kurita S. Kurogi M. Lee J. Liu B-L. Lu N. Masuda N. Matsumoto B. McKay K. Meier H. Miyamoto Y. Miyawaki H. Mochiyama C. Molter T. Morie K. Morita M. Morita Y. Morita N. Murata H. Nakahara Y. Nakamura S. Nakauchi K. Nakayama

Organization

K. Niki J. Nishii I. Nishikawa S. Oba T. Ogata S-Y. Oh N. Ohnishi M. Okada H. Okamoto T. Omori T. Omori R. Osu N. R. Pal P. S. Pang G-T. Park J. Peters S. Phillips

Y. Sakaguchi K. Sakai Y. Sakai Y. Sakumura K. Samejima M. Sato N. Sato R. Setiono T. Shibata H. Shouno M. Small M. Sugiyama I. Hong Suh J. Suzuki T. Takenouchi Y. Tanaka I. Tetsunari

N. Ueda S. Usui Y. Wada H. Wagatsuma L. Wang K. Watanabe J. Wu Q. Xiao Y. Yamaguchi K. Yamauchi Z. Yi J. Yoshimoto B.M. Yu B-T. Zhang L. Zhang L. Zhang

Sponsoring Institutions Asia Pacific Neural Network Assembly (APNNA) Japanese Neural Network Society (JNNS) 21st Century COE Program, Kyushu Institute of Technology

Cosponsors RIKEN Brain Science Institute Advanced Telecommunications Research Institute International (ATR) Japan Society for Fuzzy Theory and Intelligent Informatics (SOFT) IEEE CIS Japan Chapter Fuzzy Logic Systems Institute (FLSI)

IX

Table of Contents – Part I

Computational Neuroscience A Retinal Circuit Model Accounting for Functions of Amacrine Cells . . . Murat Saglam, Yuki Hayashida, and Nobuki Murayama

1

Global Bifurcation Analysis of a Pyramidal Cell Model of the Primary Visual Cortex: Towards a Construction of Physiologically Plausible Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tatsuya Ishiki, Satoshi Tanaka, Makoto Osanai, Shinji Doi, Sadatoshi Kumagai, and Tetsuya Yagi

7

Representation of Medial Axis from Synchronous Firing of Border-Ownership Selective Cells . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yasuhiro Hatori and Ko Sakai

18

Neural Mechanism for Extracting Object Features Critical for Visual Categorization Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mitsuya Soga and Yoshiki Kashimori

27

An Integrated Neuro-mechanical Model of C. elegans Forward Locomotion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jordan H. Boyle, John Bryden, and Netta Cohen

37

Applying the String Method to Extract Bursting Information from Microelectrode Recordings in Subthalamic Nucleus and Substantia Nigra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Pei-Kuang Chao, Hsiao-Lung Chan, Tony Wu, Ming-An Lin, and Shih-Tseng Lee

48

Population Coding of Song Element Sequence in the Songbird Brain Nucleus HVC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jun Nishikawa, Masato Okada, and Kazuo Okanoya

54

Spontaneous Voltage Transients in Mammalian Retinal Ganglion Cells Dissociated by Vibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tamami Motomura, Yuki Hayashida, and Nobuki Murayama

64

Region-Based Encoding Method Using Multi-dimensional Gaussians for Networks of Spiking Neurons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lakshmi Narayana Panuku and C. Chandra Sekhar

73

Firing Pattern Estimation of Biological Neuron Models by Adaptive Observer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kouichi Mitsunaga, Yusuke Totoki, and Takami Matsuo

83

XII

Table of Contents – Part I

Thouless-Anderson-Palmer Equation for Associative Memory Neural Network Models with Fluctuating Couplings . . . . . . . . . . . . . . . . . . . . . . . . Akihisa Ichiki and Masatoshi Shiino Spike-Timing Dependent Plasticity in Recurrently Connected Networks with Fixed External Inputs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Matthieu Gilson, David B. Grayden, J. Leo van Hemmen, Doreen A. Thomas, and Anthony N. Burkitt A Comparative Study of Synchrony Measures for the Early Detection of Alzheimer’s Disease Based on EEG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Justin Dauwels, Fran¸cois Vialatte, and Andrzej Cichocki Reproducibility Analysis of Event-Related fMRI Experiments Using Laguerre Polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hong-Ren Su, Michelle Liou, Philip E. Cheng, John A.D. Aston, and Shang-Hong Lai

93

102

112

126

The Effects of Theta Burst Transcranial Magnetic Stimulation over the Human Primary Motor and Sensory Cortices on Cortico-Muscular Coherence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Murat Saglam, Kaoru Matsunaga, Yuki Hayashida, Nobuki Murayama, and Ryoji Nakanishi

135

Interactions between Spike-Timing-Dependent Plasticity and Phase Response Curve Lead to Wireless Clustering . . . . . . . . . . . . . . . . . . . . . . . . Hideyuki Cˆ ateau, Katsunori Kitano, and Tomoki Fukai

142

A Computational Model of Formation of Grid Field and Theta Phase Precession in the Entorhinal Cells . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yoko Yamaguchi, Colin Molter, Wu Zhihua, Harshavardhan A. Agashe, and Hiroaki Wagatsuma Working Memory Dynamics in a Flip-Flop Oscillations Network Model with Milnor Attractor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . David Colliaux, Yoko Yamaguchi, Colin Molter, and Hiroaki Wagatsuma

151

160

Corticopetal Acetylcholine: Possible Scenarios on the Role for Dynamic Organization of Quasi-Attractors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hiroshi Fujii, Kazuyuki Aihara, and Ichiro Tsuda

170

Tracking a Moving Target Using Chaotic Dynamics in a Recurrent Neural Network Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yongtao Li and Shigetoshi Nara

179

A Generalised Entropy Based Associative Model . . . . . . . . . . . . . . . . . . . . . Masahiro Nakagawa

189

Table of Contents – Part I

The Detection of an Approaching Sound Source Using Pulsed Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kaname Iwasa, Takeshi Fujisumi, Mauricio Kugler, Susumu Kuroyanagi, Akira Iwata, Mikio Danno, and Masahiro Miyaji

XIII

199

Sensitivity and Uniformity in Detecting Motion Artifacts . . . . . . . . . . . . . Wen-Chuang Chou, Michelle Liou, and Hong-Ren Su

209

A Ring Model for the Development of Simple Cells in the Visual Cortex . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Takashi Hamada and Kazuhiro Okada

219

Learning and Memory Practical Recurrent Learning (PRL) in the Discrete Time Domain . . . . . Mohamad Faizal Bin Samsudin, Takeshi Hirose, and Katsunari Shibata

228

Learning of Bayesian Discriminant Functions by a Layered Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yoshifusa Ito, Cidambi Srinivasan, and Hiroyuki Izumi

238

RNN with a Recurrent Output Layer for Learning of Naturalness . . . . . . J´ an Dolinsk´y and Hideyuki Takagi

248

Using Generalization Error Bounds to Train the Set Covering Machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Zakria Hussain and John Shawe-Taylor

258

Model of Cue Extraction from Distractors by Active Recall . . . . . . . . . . . . Adam Ponzi

269

PLS Mixture Model for Online Dimension Reduction . . . . . . . . . . . . . . . . . Jiro Hayami and Koichiro Yamauchi

279

Analysis on Bidirectional Associative Memories with Multiplicative Weight Noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chi Sing Leung, Pui Fai Sum, and Tien-Tsin Wong

289

Fuzzy ARTMAP with Explicit and Implicit Weights . . . . . . . . . . . . . . . . . . Takeshi Kamio, Kenji Mori, Kunihiko Mitsubori, Chang-Jun Ahn, Hisato Fujisaka, and Kazuhisa Haeiwa Neural Network Model of Forward Shift of CA1 Place Fields Towards Reward Location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Adam Ponzi

299

309

XIV

Table of Contents – Part I

Neural Network Models A New Constructive Algorithm for Designing and Training Artificial Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Md. Abdus Sattar, Md. Monirul Islam, and Kazuyuki Murase

317

Effective Learning with Heterogeneous Neural Networks . . . . . . . . . . . . . . Llu´ıs A. Belanche-Mu˜ noz

328

Pattern-Based Reasoning System Using Self-incremental Neural Network for Propositional Logic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Akihito Sudo, Manabu Tsuboyama, Chenli Zhang, Akihiro Sato, and Osamu Hasegawa

338

Effect of Spatial Attention in Early Vision for the Modulation of the Perception of Border-Ownership . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Nobuhiko Wagatsuma, Ryohei Shimizu, and Ko Sakai

348

Effectiveness of Scale Free Network to the Performance Improvement of a Morphological Associative Memory without a Kernel Image . . . . . . . Takashi Saeki and Tsutomu Miki

358

Intensity Gradient Self-organizing Map for Cerebral Cortex Reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cheng-Hung Chuang, Jiun-Wei Liou, Philip E. Cheng, Michelle Liou, and Cheng-Yuan Liou Feature Subset Selection Using Constructive Neural Nets with Minimal Computation by Measuring Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . Md. Monirul Kabir, Md. Shahjahan, and Kazuyuki Murase Dynamic Link Matching between Feature Columns for Different Scale and Orientation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yasuomi D. Sato, Christian Wolff, Philipp Wolfrum, and Christoph von der Malsburg

365

374

385

Perturbational Neural Networks for Incremental Learning in Virtual Learning System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Eiichi Inohira, Hiromasa Oonishi, and Hirokazu Yokoi

395

Bifurcations of Renormalization Dynamics in Self-organizing Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Peter Tiˇ no

405

Variable Selection for Multivariate Time Series Prediction with Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Min Han and Ru Wei

415

Table of Contents – Part I

XV

Ordering Process of Self-Organizing Maps Improved by Asymmetric Neighborhood Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Takaaki Aoki, Kaiichiro Ota, Koji Kurata, and Toshio Aoyagi

426

A Characterization of Simple Recurrent Neural Networks with Two Hidden Units as a Language Recognizer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Azusa Iwata, Yoshihisa Shinozawa, and Akito Sakurai

436

Supervised/Unsupervised/Reinforcement Learning Unbiased Likelihood Backpropagation Learning . . . . . . . . . . . . . . . . . . . . . . Masashi Sekino and Katsumi Nitta

446

The Local True Weight Decay Recursive Least Square Algorithm . . . . . . Chi Sing Leung, Kwok-Wo Wong, and Yong Xu

456

Experimental Bayesian Generalization Error of Non-regular Models under Covariate Shift . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Keisuke Yamazaki and Sumio Watanabe

466

Using Image Stimuli to Drive fMRI Analysis . . . . . . . . . . . . . . . . . . . . . . . . David R. Hardoon, Janaina Mour˜ ao-Miranda, Michael Brammer, and John Shawe-Taylor

477

Parallel Reinforcement Learning for Weighted Multi-criteria Model with Adaptive Margin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kazuyuki Hiraoka, Manabu Yoshida, and Taketoshi Mishima

487

Convergence Behavior of Competitive Repetition-Suppression Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Davide Bacciu and Antonina Starita

497

Self-Organizing Clustering with Map of Nonlinear Varieties Representing Variation in One Class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hideaki Kawano, Hiroshi Maeda, and Norikazu Ikoma

507

An Automatic Speaker Recognition System . . . . . . . . . . . . . . . . . . . . . . . . . P. Chakraborty, F. Ahmed, Md. Monirul Kabir, Md. Shahjahan, and Kazuyuki Murase Modified Modulated Hebb-Oja Learning Rule: A Method for Biologically Plausible Principal Component Analysis . . . . . . . . . . . . . . . . . Marko Jankovic, Pablo Martinez, Zhe Chen, and Andrzej Cichocki

517

527

Statistical Learning Algorithms Orthogonal Shrinkage Methods for Nonparametric Regression under Gaussian Noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Katsuyuki Hagiwara

537

XVI

Table of Contents – Part I

A Subspace Method Based on Data Generation Model with Class Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Minkook Cho, Dongwoo Yoon, and Hyeyoung Park

547

Hierarchical Feature Extraction for Compact Representation and Classification of Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Markus Schubert and Jens Kohlmorgen

556

Principal Component Analysis for Sparse High-Dimensional Data . . . . . . Tapani Raiko, Alexander Ilin, and Juha Karhunen

566

Hierarchical Bayesian Inference of Brain Activity . . . . . . . . . . . . . . . . . . . . . Masa-aki Sato and Taku Yoshioka

576

Neural Decoding of Movements: From Linear to Nonlinear Trajectory Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Byron M. Yu, John P. Cunningham, Krishna V. Shenoy, and Maneesh Sahani

586

Estimating Internal Variables of a Decision Maker’s Brain: A Model-Based Approach for Neuroscience . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kazuyuki Samejima and Kenji Doya

596

Visual Tracking Achieved by Adaptive Sampling from Hierarchical and Parallel Predictions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tomohiro Shibata, Takashi Bando, and Shin Ishii

604

Bayesian System Identification of Molecular Cascades . . . . . . . . . . . . . . . . Junichiro Yoshimoto and Kenji Doya

614

Use of Circle-Segments as a Data Visualization Technique for Feature Selection in Pattern Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Shir Li Wang, Chen Change Loy, Chee Peng Lim, Weng Kin Lai, and Kay Sin Tan

625

Extraction of Approximate Independent Components from Large Natural Scenes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yoshitatsu Matsuda and Kazunori Yamaguchi

635

Local Coordinates Alignment and Its Linearization . . . . . . . . . . . . . . . . . . . Tianhao Zhang, Xuelong Li, Dacheng Tao, and Jie Yang

643

Walking Appearance Manifolds without Falling Off . . . . . . . . . . . . . . . . . . . Nils Einecke, Julian Eggert, Sven Hellbach, and Edgar K¨ orner

653

Inverse-Halftoning for Error Diffusion Based on Statistical Mechanics of the Spin System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yohei Saika

663

Table of Contents – Part I

XVII

Optimization Algorithms Chaotic Motif Sampler for Motif Discovery Using Statistical Values of Spike Time-Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Takafumi Matsuura and Tohru Ikeguchi

673

A Thermodynamical Search Algorithm for Feature Subset Selection . . . . F´elix F. Gonz´ alez and Llu´ıs A. Belanche

683

Solvable Performances of Optimization Neural Networks with Chaotic Noise and Stochastic Noise with Negative Autocorrelation . . . . . . . . . . . . . Mikio Hasegawa and Ken Umeno

693

Solving the k-Winners-Take-All Problem and the Oligopoly Cournot-Nash Equilibrium Problem Using the General Projection Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Xiaolin Hu and Jun Wang Optimization of Parametric Companding Function for an Efficient Coding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Shin-ichi Maeda and Shin Ishii A Modified Soft-Shape-Context ICP Registration System of 3-D Point Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jiann-Der Lee, Chung-Hsien Huang, Li-Chang Liu, Shih-Sen Hsieh, Shuen-Ping Wang, and Shin-Tseng Lee Solution Method Using Correlated Noise for TSP . . . . . . . . . . . . . . . . . . . . Atsuko Goto and Masaki Kawamura

703

713

723

733

Novel Algorithms Bayesian Collaborative Predictors for General User Modeling Tasks . . . . Jun-ichiro Hirayama, Masashi Nakatomi, Takashi Takenouchi, and Shin Ishii

742

Discovery of Linear Non-Gaussian Acyclic Models in the Presence of Latent Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Shohei Shimizu and Aapo Hyv¨ arinen

752

Efficient Incremental Learning Using Self-Organizing Neural Grove . . . . . Hirotaka Inoue and Hiroyuki Narihisa

762

Design of an Unsupervised Weight Parameter Estimation Method in Ensemble Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Masato Uchida, Yousuke Maehara, and Hiroyuki Shioya

771

Sparse Super Symmetric Tensor Factorization . . . . . . . . . . . . . . . . . . . . . . . Andrzej Cichocki, Marko Jankovic, Rafal Zdunek, and Shun-ichi Amari

781

XVIII

Table of Contents – Part I

Probabilistic Tensor Analysis with Akaike and Bayesian Information Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dacheng Tao, Jimeng Sun, Xindong Wu, Xuelong Li, Jialie Shen, Stephen J. Maybank, and Christos Faloutsos Decomposing EEG Data into Space-Time-Frequency Components Using Parallel Factor Analysis and Its Relation with Cerebral Blood Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Fumikazu Miwakeichi, Pedro A. Valdes-Sosa, Eduardo Aubert-Vazquez, Jorge Bosch Bayard, Jobu Watanabe, Hiroaki Mizuhara, and Yoko Yamaguchi

791

802

Flexible Component Analysis for Sparse, Smooth, Nonnegative Coding or Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Andrzej Cichocki, Anh Huy Phan, Rafal Zdunek, and Li-Qing Zhang

811

Appearance Models for Medical Volumes with Few Samples by Generalized 3D-PCA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Rui Xu and Yen-Wei Chen

821

Head Pose Estimation Based on Tensor Factorization . . . . . . . . . . . . . . . . . Wenlu Yang, Liqing Zhang, and Wenjun Zhu

831

Kernel Maximum a Posteriori Classification with Error Bound Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Zenglin Xu, Kaizhu Huang, Jianke Zhu, Irwin King, and Michael R. Lyu Comparison of Local Higher-Order Moment Kernel and Conventional Kernels in SVM for Texture Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . Keisuke Kameyama

841

851

Pattern Discovery for High-Dimensional Binary Datasets . . . . . . . . . . . . . . V´ aclav Sn´ aˇsel, Pavel Moravec, Duˇsan H´ usek, Alexander Frolov, ˇ Hana Rezankov´ a, and Pavel Polyakov

861

Expand-and-Reduce Algorithm of Particle Swarm Optimization . . . . . . . . Eiji Miyagawa and Toshimichi Saito

873

Nonlinear Pattern Identification by Multi-layered GMDH-Type Neural Network Self-selecting Optimum Neural Network Architecture . . . . . . . . . Tadashi Kondo

882

Motor Control and Vision Coordinated Control of Reaching and Grasping During Prehension Movement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Masazumi Katayama and Hirokazu Katayama

892

Table of Contents – Part I

Computer Simulation of Vestibuloocular Reflex Motor Learning Using a Realistic Cerebellar Cortical Neuronal Network Model . . . . . . . . . . . . . . Kayichiro Inagaki, Yutaka Hirata, Pablo M. Blazquez, and Stephen M. Highstein Reflex Contributions to the Directional Tuning of Arm Stiffness . . . . . . . . Gary Liaw, David W. Franklin, Etienne Burdet, Abdelhamid Kadi-allah, and Mitsuo Kawato

XIX

902

913

Analysis of Variability of Human Reaching Movements Based on the Similarity Preservation of Arm Trajectories . . . . . . . . . . . . . . . . . . . . . . . . . Takashi Oyama, Yoji Uno, and Shigeyuki Hosoe

923

Directional Properties of Human Hand Force Perception in the Maintenance of Arm Posture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yoshiyuki Tanaka and Toshio Tsuji

933

Computational Understanding and Modeling of Filling-In Process at the Blind Spot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Shunji Satoh and Shiro Usui

943

Biologically Motivated Face Selective Attention Model . . . . . . . . . . . . . . . . Woong-Jae Won, Young-Min Jang, Sang-Woo Ban, and Minho Lee

953

Multi-dimensional Histogram-Based Image Segmentation . . . . . . . . . . . . . . Daniel Weiler and Julian Eggert

963

A Framework for Multi-view Gender Classification . . . . . . . . . . . . . . . . . . . Jing Li and Bao-Liang Lu

973

Japanese Hand Sign Recognition System . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hirotada Fujimura, Yuuichi Sakai, and Hiroomi Hikawa

983

An Image Warping Method for Temporal Subtraction Images Employing Smoothing of Shift Vectors on MDCT Images . . . . . . . . . . . . . Yoshinori Itai, Hyoungseop Kim, Seiji Ishikawa, Shigehiko Katsuragawa, Takayuki Ishida, Ikuo Kawashita, Kazuo Awai, and Kunio Doi

993

Conflicting Visual and Proprioceptive Reflex Responses During Reaching Movements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1002 David W. Franklin, Udell So, Rieko Osu, and Mitsuo Kawato An Involuntary Muscular Response Induced by Perceived Visual Errors in Hand Position . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1012 David W. Franklin, Udell So, Rieko Osu, and Mitsuo Kawato Independence of Perception and Action for Grasping Positions . . . . . . . . . 1021 Takahiro Fujita, Yoshinobu Maeda, and Masazumi Katayama

XX

Table of Contents – Part I

Handwritten Character Distinction Method Inspired by Human Vision Mechanism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1031 Jumpei Koyama, Masahiro Kato, and Akira Hirose Recent Advances in the Neocognitron . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1041 Kunihiko Fukushima Engineering-Approach Accelerates Computational Understanding of V1–V2 Neural Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1051 Shunji Satoh and Shiro Usui Recent Studies Around the Neocognitron . . . . . . . . . . . . . . . . . . . . . . . . . . . 1061 Hayaru Shouno Toward Human Arm Attention and Recognition . . . . . . . . . . . . . . . . . . . . . 1071 Takeharu Yoshizuka, Masaki Shimizu, and Hiroyuki Miyamoto Projection-Field-Type VLSI Convolutional Neural Networks Using Merged/Mixed Analog-Digital Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1081 Osamu Nomura and Takashi Morie Optimality of Reaching Movements Based on Energetic Cost under the Influence of Signal-Dependent Noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1091 Yoshiaki Taniai and Jun Nishii Influence of Neural Delay in Sensorimotor Systems on the Control Performance and Mechanism in Bicycle Riding . . . . . . . . . . . . . . . . . . . . . . . 1100 Yusuke Azuma and Akira Hirose Global Localization for the Mobile Robot Based on Natural Number Recognition in Corridor Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1110 Su-Yong An, Jeong-Gwan Kang, Se-Young Oh, and Doo San Baek A System Model for Real-Time Sensorimotor Processing in Brain . . . . . . 1120 Yutaka Sakaguchi Perception of Two-Stroke Apparent Motion and Real Motion . . . . . . . . . . 1130 Qi Zhang and Ken Mogi Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1141

A Retinal Circuit Model Accounting for Functions of Amacrine Cells Murat Saglam, Yuki Hayashida, and Nobuki Murayama Graduate School of Science and Technology, Kumamoto University, 2-39-1 Kurokami, Kumamoto 860-8555, Japan [email protected], {yukih,murayama}@cs.kumamoto-u.ac.jp

Abstract. In previous experimental studies on vertebrates, high level processes of vision such as object segregation and spatio-temporal pattern adaptation were found to begin at retinal stage. In those visual functions, diverse subtypes of amacrine cells are believed to play essential roles by processing the excitatory and inhibitory signals laterally over a wide region on the retina to shape the ganglion cell responses. Previously, a simple "linear-nonlinear" model was proposed to explain a specific function of the retina, and could capture the spiking behavior of retinal output, although each class of the retinal neurons were largely omitted in it. Here, we present a spatio-temporal computational model based on the response function for each class of the retinal neurons and the anatomical intercellular connections. This model is not only capable of reproducing filtering properties of outer retina but also realizes high-order inner retinal functions such as object segregation mechanism via wide-field amacrine cells. Keywords: Retina, Amacrine Cells, Model, Visual Function.

1 Introduction The vertebrate retina is far more than a passive visual receptor. It has been reported that many high-level vision tasks begin in the retinal circuits although they are believed to be performed in visual cortices of the brain [1]. One important task among those is the discrimination of the actual motion of an object from the global motion across the retina. Even in the case of perfect stationary scene, eye movements cause retinal image-drifts that hinder the retinal circuit to have a stationary global input at the background [2]. To handle this problem, retinal circuits are able to distinguish the object motions better when they have different patterns than background motion. The synaptic configuration of diverse types of retinal cells plays an essential role during this function. It was reported that wide-field polyaxonal amacrine cells can drive inhibitory process between the surround and the object regions (receptive field) on the retina [1, 2, 4, 5, 6]. Those wide-field amacrine cells are known to use inhibitory neurotransmitters like glycine or GABA [7, 8]. A previous study reported that glycine-mediated wide-field inhibition exists in the salamander retina and proposed a simple “linear-nonlinear” model, consisting of a temporal filter and a threshold function [2]. However that model does not include the details of any retinal neurons M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 1–6, 2008. © Springer-Verlag Berlin Heidelberg 2008

2

M. Saglam, Y. Hayashida, and N. Murayama

accounting for that inhibitory mechanism although it is capable of predicting the spiking behavior of the retinal output for certain input patterns. On the other hand, different temporal models that include the behavior of each class of retinal neurons exist in the literature [9, 10]. Even though those models provide high temporal resolution, they lack spatial information of the retinal processing. Here, we present a spatio-temporal computational model that realizes wide-field inhibition between object and surround region via wide-field transient on/off amacrine cells. The model considers the responses of all major retinal neurons in detail.

2 Retinal Model The retina model parcels the stimulating input into a line of spatio-temporal computational units. Each unit consists of main retinal elements that convey information forward (photoreceptors, bipolar and ganglion cells) and laterally (horizontal and amacrine cells). Figure 1 illustrates the organization and the synaptic connections of the retinal neurons.

Fig. 1. Three computational units of the model are depicted. Each unit includes: PR: Photoreceptor, HC: Horizontal Cell, onBC: On Bipolar Cell, offBC: Off Bipolar Cell, onAC: Sustained On Amacrine Cell, offAC: Sustained Off Amacrine Cell, on/offAC: Fast-transient Wide-field On/Off Amacrine Cell, GC: On/Off Ganglion Cell. Excitatory/Inhibitory synaptic connections are represented with black/white arrowheads, respectively. Gap junctions within neighboring HCs are indicated by dotted horizontal lines. Wide-field connections are realized between wide-field on/offAC and GCs, double-s-symbols point to distant connections within those neurons.

A Retinal Circuit Model Accounting for Functions of Amacrine Cells

3

Each neuron’s membrane dynamics is governed by a differential equation (eqn.1) which is adjusted from push-pull shunting models of retinal neurons [9]. n dvc (t ) = − Avc (t ) + [ B − vc (t )]e(t ) − [ D + vc (t )]i (t ) + ∑Wck vk (t ) dt k =1

(1)

Here υc(t) stands for the membrane potential of the neuron of interest. A represents the rate of passive membrane decay toward the resting potential in the dark. B and D are the saturation levels for the excitatory, e(t) and inhibitory i(t) inputs, respectively. Those excitatory/inhibitory inputs correspond to the synaptic connections (solid lines in fig.1) of different neurons in a computational unit. υk(t) is the membrane potential of a different neuron belonging to another unit making synapse or gap-junction to the neuron of interest, υc(t). The efficiency of that link is determined by a weight parameter, Wck. In the current model, spatial connectivity is present within horizontal cells as gap junctions (dashed lines in fig.1) and between on/off amacrine cells and ganglion cells as wide-field inhibitory process (thin solid lines in fig1). As for the other neurons Wck is fixed to zero since we ignore the lateral spatial connection within them. A compressive nonlinearity (eqn. 2) is cascaded prior to the photoreceptor input stage in order to account the limited dynamic range of the neural elements. Therefore the photoreceptor is fed by a hyperpolarizing input, r (t), representing the compressed form of light intensity, f (t).

⎛ f (t ) ⎞ ⎟⎟ r (t ) = G⎜⎜ ⎝ f (t ) + I ⎠

n

(2)

Here G denotes the saturation level of the hyperpolarizing input to the photoreceptor. I represents the light intensity yielding half-maximum response and n is a real constant. Although ganglion cell receptive field size is diverse among different animals, we defined that each unit corresponds to 500μm which is in well accordance with experiments on salamander [2]. 32 computational units are interconnected as a line. Wck values are determined as a function of distance between computational units. Parameter set given in [9] is calibrated to reproduce the temporal dynamics of all neuron classes. Spatial parameters of the model are selected to meet the spatial ganglion cell response profile given in [2]. All differential equations in the model are solved sequentially using fixed-step (1ms) Bogacki-Shampine solver of the MATLAB/SIMULINK software package (The Mathworks-Inc., Natick, MA).

3 Results First we confirmed that responses of each neuron agree with the physiological observations [9]. Figure 2 illustrates the response of all neurons to 150ms-long flash light stimulating the whole model retina. At the outer retina, photoreceptor responds with a transient hyperpolarization followed by a less steep level and reaches to the resting potential with a small overshoot. Essentially horizontal cell has the smoothened form of the photoreceptor response due to its low-pass filtering feature. On- and offbipolar cells are depolarized during on/off-set of the flash light, respectively. Since

4

M. Saglam, Y. Hayashida, and N. Murayama

Fig. 2. Responses of retinal neurons (labeled as fig.1) to 150ms-long full field light flash. Dashed horizontal lines indicate dark responses(resting potentials) of each neuron. Note that the timings of on/off responses of wide-field ACs and GC spike generating potentials (GC gen. pot.) match each other. This phenomenon drives the wide-field inhibition.

Fig. 3. GC spike generating potential responses (top row) of the center unit at incoherent (left column) and coherent (right column) stimulation case. Stimulation timing and position are depicted on the x and y axes, respectively (bottom row, white bars indicate the light onset). Under coherent stimulation condition, off responses are significantly inhibited and on responses all disappeared.

those cells build negative feedback loop with sustained on- and off-amacrine cells, their responses are more transient than photoreceptors and horizontal cells as expected. Eventually bipolar cells transmit excitatory, wide-field transient on/off amacrine cells

A Retinal Circuit Model Accounting for Functions of Amacrine Cells

5

convey inhibitory inputs to ganglion cells. Significant inhibition at ganglion cell level only happens when wide-field amacrine cell signal matches the excitatory input. Figure 3 demonstrates how inhibitory process differs when the peripheral (surround) and the object regions are stimulated coherently or incoherently. In both cases object region invades 3 units (750μm radius) and stimulated identically. When the surround is stimulated incoherently, depolarized peaks of wide-field amacrine cells do not coincide with the ganglion cell peaks so that spike generating potentials are evident. However when the surround region is simulated coherently, inhibition from amacrine cells cancels out the big portions of ganglion cell depolarizations. This leads to maximum inhibition of the spike generating potentials of ganglion cells (Fig.3, right column).

4 Discussion In the current study we realized the basic mechanism of an important retinal task which is discriminating a moving object from moving background image. The coherent stimulation (Fig.2) can be linked to the global motion of the retinal image that takes place when the eye moves. However when there is a moving object in the

Fig. 4. Relative generating potential response of GC as a function of object size. Dashed line represents the model response with the original parameter set. Triangle and square markers indicate data points for ‘wide-field AC blocked’ and ‘control’ cases, respectively. Maximum GC response is observed when the object radius is 250μm (1 unit stimulation, 2nd data point). For the sake of symmetry 3rd data point represents 3 unit stimulation (750μm radius as in Fig.3), similarly each interval after 2nd data point corresponds to 500μm increment in the radius of object. As the object starts to invade the surround region, GC response decreases. When the weight of the interconnections among wide-field on/off ACs are set to zero (STR application), inhibition process is partially disabled (solid line).

6

M. Saglam, Y. Hayashida, and N. Murayama

scene, its image would be reflected on the receptive field as a different stimulation pattern than the global pattern (Incoherent stimulation, Fig.3). Experimental results revealed that blocking glycine-mediated inhibition by strychnine (STR) disables the wide-field process [2]. Therefore this glycine-ergic mechanism could be accounted to wide-field amacrine cells [7]. In our model, STR application can be realized by turning off synaptic weight parameters between wide-field amacrine cells and ganglion cells. Figure 4 demonstrates how STR application can affect ganglion cell response. As the object invades the background ganglion cell response is expected to be inhibited as in the control case however STR prevents this phenomenon to occur. This behavior of the model is in very well accordance with the experimental results in [2]. Note that the model is flexible enough to fit to another wide-field inhibitory process such as GABA-ergic mechanism [8]. Spike generation of the ganglion cells is not implemented in the current model in order to highlight the role of wide-field amacrine cells only. A specific spike generator can be cascaded to the model to reproduce spike responses and highlight retinal features more. Since the model covers on/off pathways and all major retinal neurons, it can be flexibly adjusted to reproduce other functions. Although we deduced the retina into a line of spatio-temporal computational units, the model was able to reproduce a retinal mechanism. This deduction can be bypassed and more precise results can be achieved by creating a 2-D mesh of spatiotemporal computational units.

References 1. Masland, R.H.: Vision: The retina’s fancy tricks. Nature 423(6938), 387–388 (2003) 2. Olveczky, B.P., Baccus, S.A., Meister, M.: Segregation of object and background motion in the retina. Nature 423(6938), 401–408 (2003) 3. Volgyi, B., Xin, D., Amarillo, Y., Bloomfield, S.A.: Morphology and physiology of the polyaxonal amacrine cells in the rabbit retina. J. Comp. Neurol. 440(1), 109–125 (2001) 4. Lin, B., Masland, R.H.: Populations of wide-field amacrine cells in the mouse retina. J. Comp. Neurol. 499(5), 797–809 (2006) 5. Solomon, S.G., Lee, B.B., Sun, H.: Suppressive surrounds and contrast gain in magnocellular pathway retinal ganglion cells of macaque. J. Neurosci. 26(34), 8715–8726 (2006) 6. van Wyk, M., Taylor, W.R., Vaney, D.: Local edge detectors: a substrate for fine spatial vision at low temporal frequencies in rabbit retina. J. Neurosci. 26(51), 250–263 (2006) 7. Hennig, M.H., Funke, K., Worgotter, F.: The influence of different retinal subcircuits on the nonlinearity of ganglion cell behavior. J. Neurosci. 22(19), 8726–8738 (2002) 8. Lukasiewicz, P.D.: Synaptic mechanisms that shape visual signaling at the inner retina. Prog Brain Res. 147, 205–218 (2005) 9. Thiel, A., Greschner, M., Ammermuller, J.: The temporal structure of transient ON/OFF ganglion cell responses and its relation to intra-retinal processing. J. Comput. Neurosci. 21(2), 131–151 (2006) 10. Gaudiano, P.: Simulations of X and Y retinal ganglion cell behavior with a nonlinear pushpull model of spatiotemporal retinal processing. Vision Res. 34(13), 1767–1784 (1994)

Global Bifurcation Analysis of a Pyramidal Cell Model of the Primary Visual Cortex: Towards a Construction of Physiologically Plausible Model Tatsuya Ishiki, Satoshi Tanaka, Makoto Osanai, Shinji Doi, Sadatoshi Kumagai, and Tetsuya Yagi Division of Electrical, Electronic and Information Engineering, Graduate School of Engineering, Osaka University, Yamada-Oka 2-1, Suita, Osaka, Japan [email protected]

Abstract. Many mathematical models of different neurons have been proposed so far, however, the way of modeling Ca2+ regulation mechanisms has not been established yet. Therefore, we try to construct a physiologically plausible model which contains many regulating systems of the intracellular Ca2+ , such as Ca2+ buffering, Na+ /Ca2+ exchanger and Ca2+ pump current. In this paper, we seek the plausible values of parameters by analyzing the global bifurcation structure of our temporary model.

1

Introduction

Complex information processing of brain is regulated by the electrical activity of neurons. Neurons transmit an electrical signal called action potential each other for the information processing. The action potential is a spiking or bursting and plays an important role in the information processing of the brain. In the visual system, visual signals from the retina are processed by neurons in the primary visual cortex. There are several types of neurons in the visual cortex, and pyramidal cells compose roughly 80% of the neurons of the cortex. Pyramidal cells are connected each other and form a complex neuronal circuit. Previous physiological and anatomical studies [1] revealed the fundamental structure of the circuit. However, it is not completely understood how visual signals propagate and function in the neuronal circuit of the visual cortex. In order to investigate the neuronal circuit, not only physiological experiments but also simulations by using a mathematical model of neuron are necessary. Many mathematical models of neurons have been proposed so far [2]. Though there are various models of neurons, the way of modeling the regulating system of the intracellular calcium ions (Ca2+ ) has not been established yet. The regulating system of the intracellular Ca2+ is a very important element because the intracellular Ca2+ plays crucial roles in cellular processes such as hormone and neurotransmitter release, gene transcription, and regulations of synaptic plasticity. Therefore, it is important to establish the way of modeling the regulating system of the intracellular Ca2+ . M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 7–17, 2008. c Springer-Verlag Berlin Heidelberg 2008 

8

T. Ishiki et al.

In this paper, we try to construct a model of pyramidal cells by regarding previous physiological experimental data, especially focusing on the regulating systems of the intracellular Ca2+ , such as Ca2+ buffering, Na+ /Ca2+ exchanger and Ca2+ pump current. In order to estimate the value of the parameters which cannot be determined by physiological experiments solely, we analyze the global bifurcation structure based on the slow/fast decomposition of the model. Thus we demonstrate the usefulness of such nonlinear analyses not only for the analysis of an established model but also for the construction of a model.

2

Cell Model

The well-known Hodgkin-Huxley (HH) equations [3] describe the temporal variation of membrane potential of neuronal cells. Though there are many neuron models based on the HH equations, the way of modeling the regulating system of the intracellular Ca2+ has not been established yet. Thus, we construct a pyramidal cell model by using several physiological experiments data [4]-[11]. The model includes Ca2+ buffer, Ca2+ pump, Na+ /Ca2+ exchanger in order to describe the regulating system of the intracellular Ca2+ appropriately. The model also includes seven ionic currents through the ionic channels. The equations of the pyramidal cell model are as follows: −C

dV = Itotal − Iext , dt dy 1 = (y∞ − y) , (y = M 1, · · · , M 6, H1, · · · , H6), dt τy

d[Ca2+ ] −S · ICatotal = + k− [CaBuf] − k+ [Ca2+ ][Buf], dt 2·F d[Buf] = k− [CaBuf] − k+ [Ca2+ ][Buf], dt d[CaBuf] = −(k− [CaBuf] − k+ [Ca2+ ][Buf]), dt

(1a) (1b) (1c) (1d) (1e)

where V is the membrane potential, C is the membrane capacitance, Itotal is the sum of all currents through the ionic channels and Na+ /Ca2+ exchanger, and Iext is the current injected to the cell externally. The variable y denotes gating variables (M 1, · · · , M 6, H1, · · · , H6) of the ionic channels, y∞ is the steady state function of y, and τy is a time constant. [Ca2+ ] denotes the intracellular Ca2+ concentration, ICatotal is the sum of all Ca2+ ionic currents, S is the surface to volume ratio, and F is the Faraday constant. [Buf] and [CaBuf] are the concentrations of the unbound and the bound buffers, and k− and k+ are the reverse and forward rate constants of the binding reaction, respectively. Details of all equations and parameter values of this model can be found in Appendix. First we show the simulation results when a certain external stimulus current is injected to the cell model (1). Figure 1 shows an action potential waveform, and a change of [Ca2+ ] when a narrow external stimulus current (length 1ms,

Global Bifurcation Analysis of a Pyramidal Cell Model

20

9

[B]

[A]

0.0018

V (mV)

[Ca2+] (mM)

0 -20 -40 -60 -80 3500

4000

4500

5000

t (ms)

5500

6000

0.0014 0.001 0.0006 0.0002 3500 4000

4500

5000 5500 6000

t (ms)

Fig. 1. Waveforms of [A] the membrane potential and [B] [Ca2+ ] in the case of a 1ms narrow pulse injection [A]

[B]

V (mV)

20 0

-20 -40 -60 -80

4000

5000

t (ms)

6000

7000

Fig. 2. [A] A waveform of the membrane potential under a long pulse injection. [B] A typical waveform of the membrane potential of pyramidal cell in physiological experiments [12].

density 40μA/cm2 ) is injected at t = 4000ms. The waveforms of the membrane potential and [Ca2+ ] are not so different from the physiological experimental data qualitatively [1]. In contrast, as shown in Fig. 2A, in the case that a long pulse (length 1000ms, density 40μA/cm2 ) is injected, the membrane potential keeps a resting state after one action potential is generated. Though the membrane potential is spiking continuously in the physiological experiment (Fig. 2B), the membrane potential of the model does not show such a behavior. In general, it is well known that the membrane potential of pyramidal cell is resting in the case of no external stimulus, and spiking or bursting when the external stimulus is added. The aim of this paper is a reconstruction or a parameter tuning of the model (1) which can reproduce such a behavior of the membrane potential.

3

Bifurcation Analysis

The characteristics of the membrane potential vary with the change of the value of some parameters, therefore we investigate the bifurcation structure of the model (1) to estimate the values of the parameters. For the bifurcation analysis in this paper, we used the bifurcation analysis software AUTO [13].

10

T. Ishiki et al.

V (mV)

-35 -40 -45 -50 -55 -60 -65 -70 0

20

40

60

80

100

Iext (µA/cm2)

Fig. 3. One-parameter bifurcation diagram on the parameter Iext . The solid curve denotes stable equilibria of eq. (1).

3.1

The External Stimulation Current I ext

We analyze the bifurcation structure of the model to understand why the continuous spiking of the membrane potential is not generated when a long pulse is injected. In order to investigate whether the spiking is generated or not when Iext is increased, we vary the external stimulation current Iext as a bifurcation parameter. We show the one-parameter bifurcation diagram on the parameter Iext (Fig. 3) in which the solid curve denotes the membrane potential at stable equilibria of eq. (1). The one-parameter bifurcation diagram shows the dependence of the membrane potential on the parameter Iext . There is no bifurcation point in Fig. 3. Therefore the stability of the equilibrium point does not change, and thus the membrane potential of the model keeps resting even if Iext is increased. This result means that we cannot reproduce the physiological experimental result in Fig. 2B by varying Iext , thus we have to reconsider the parameter values of the model which cannot be determined by physiological experiments only. 3.2

The Maximum Conductance of Ca2+ -Dependent Potassium Channel GKCa

The current through the Ca2+ -dependent potassium channel is involved in the generation of spiking or bursting of the membrane potential. Therefore, we select the maximum conductance of Ca2+ -dependent potassium channel GKCa as a bifurcation parameter, and show the one-parameter bifurcation diagram when Iext = 10 (Fig. 4A). There are two saddle-node bifurcation points (SN1, SN2), three Hopf bifurcation points (HB1-HB3) and two torus bifurcation points (TR1, TR2). An unstable periodic solution which is bifurcated from HB1 changes its stability at the two torus bifurcation points and merges into the equilibrium point at HB3. Only in the range between HB1 and HB3, the membrane potential can oscillate. In order to investigate the dependence of the oscillatory range on the Iext value, we show the two-parameter bifurcation diagram (Fig. 4B) in which the horizontal and vertical axes denote Iext and GKCa , respectively. The twoparameter bifurcation diagram shows the loci where a specific bifurcation occurs.

Global Bifurcation Analysis of a Pyramidal Cell Model [A]

14

V (mV)

-10

-40

HB3

-60

2.50

SN1 HB2 HB1

2.75

3.00

3.25

HB SN

10

SN2

-50

[B]

12

TR1

-20 -30

Iext =10

TR2

G K Ca (mS/cm 2 )

0

11

3.50

G K Ca (mS/cm 2 )

8 6 4 2

3.75

4.00

0 -15 -10 -5

0

5

10 15 20 25 30

Iext (µA/cm2)

Fig. 4. [A] One-parameter bifurcation diagram on the parameter GKCa . The solid and broken curves show stable and unstable equilibria, respectively. The symbols • and ◦ denote the maximum value of V of the stable and unstable periodic solutions, respectively. [B] Two-parameter bifurcation diagram in the (Iext , GKCa )-plane.

In the diagram, the gray colored area separated by the HB and SN bifurcation curves corresponds to the range between HB1 and HB3 in Fig. 4A where the periodic solutions appear. Increasing Iext , the gray colored area shrinks gradually and disappears near Iext = 25. This result means that the membrane potential of the model cannot present any oscillations (spontaneous spiking) for large values of Iext even if we changed the value of the parameter GKCa . 3.3

The Maximum Pumping Rate of the Ca2+ Pump Apump

The Ca2+ pump plays an important role in the regulation of the intracellular Ca2+ . We also investigate the effect of varying the Apump value, which is the maximum pumping rate of the Ca2+ pump, on the membrane potential. Figure 5A is the one-parameter bifurcation diagram when Iext = 10. There are two Hopf bifurcation points (HB1, HB2) and four double-cycle bifurcation points (DC1DC4). A stable periodic solution generated at HB1 changes its stability at the four double-cycle bifurcation points and merges into the equilibrium point at HB2. Similarly to the case of GKCa , we show the two-parameter bifurcation diagram in the plane of two parameters Iext and Apump (Fig. 5B) in order to examine the dependence of the oscillatory range between HB1 and HB2 on Iext . In Fig. 5B, the gray colored area, where the membrane potential oscillates, shrinks and disappears as Iext increases. The result shows that the membrane potential of the model cannot present any oscillations (spontaneous spiking) for large values of Iext even if we changed the value of Apump similarly to the case of GKCa . 3.4

Slow/Fast Decomposition Analysis

In this section, in order to investigate the dynamics of our pyramidal cell model in more detail, we use the slow/fast decomposition analysis [14].

T. Ishiki et al.

30 20 10 0 -10 -20 -30 -40 -50 -60 0

[A]

1000 DC3

Iext =10

DC4 DC2 HB2 DC1 HB1

300

500

800

1000

Apump (pmol/s/cm2)

1300

Apump (pmol/s/cm2)

V (mV)

12

[B] HB

800 600 400 200 0

0

20

40

60

80

100 120 140

Iext (µA/cm2)

Fig. 5. [A] One-parameter bifurcation diagram on the parameter Apump . [B] Twoparameter bifurcation diagram in the (Iext , Apump )-plane.

A system with multiple time scales can be denoted generally as follows: dx = f (x, y), x ∈ Rn , y ∈ Rm , dt dy = g(x, y),   1. dt

(2a) (2b)

Equation (2b) is called a slow subsystem since the value of y changes slowly while equation (2a) a fast subsystem. The whole eq. (2) is called a full system. So-called slow/fast analysis divides the full system into the slow and fast subsystems. In the fast subsystem (2a), the slow variable y is considered as a constant or a parameter. The variable x changes more quickly than y and thus x is considered to stay close to the attractor (stable equilibrium points, limit cycle, etc.) of the fast subsystem for a fixed value of y. The variable y changes slowly with a velocity g(x, y) in which x is considered to be in the neighborhood of the attractor. The attractor of the fast subsystem may change if y is varied. The problem of analysis of the dependence of attractor on the parameter y is a bifurcation problem. Thus the slow/fast analysis reduces the analysis of full system to the bifurcation problem of the fast subsystem with a slowly-varying bifurcation parameter. In the case of the pyramidal cell model (1), under the assumption that the change of the intracellular Ca2+ concentration [Ca2+ ] is slower than the other variables, the slow/fast analysis can be made. Thus, we consider [Ca2+ ] as a bifurcation parameter and eq. (1c) as a slow subsystem, and all other equations of eqs. (1a,b,d,e) are considered as a fast subsystem. We show the bifurcation diagram of the fast subsystem by varying the value of [Ca2+ ] as a parameter (Fig. 6). The figure shows the stable and unstable equilibria of the fast subsystem with Iext = 0 (thick solid and broken curves, resp.), and the slow subsystem (thin curve). The point at the intersection of the equilibrium curve of the fast subsystem with the nullcline of the slow subsystem is the equilibrium point of the f ull system. The stability of the full system is determined whether the intersection point is on the stable or unstable branch of the equilibrium curve of the f ast subsystem. Therefore, the stability of the full system is stable in the case of

Global Bifurcation Analysis of a Pyramidal Cell Model

13

10

0

V (mV)

-10 -20 -30 -40 -50 -60 -70 -80

0

0.0001 0.0002 0.0003 0.0004 0.0005

[Ca2+] (mM)

Fig. 6. Bifurcation diagram of the fast subsystem (Iext = 0) with Ca2+ as a bifurcation parameter and the slow-nullcline of the slow subsystem

Fig. 6. In addition, when Iext is increased, the bifurcation diagram (equilibrium curve) of the fast subsystem shifts upward and the stability of the full system keeps stable and no oscillation appear, as will be shown in Fig. 7. By changing the parameter of the slow subsystem, the shape of the slownullcline changes and the intersection point is also shifted. First, we select some parameters of the slow subsystem. Because the Ca2+ pump is included only in the slow subsystem, we select the Apump and the dissociation constant Kpump which are both parameters contained in Ca2+ pump as the parameters of the slow subsystem. Second, we change the values of Apump and Kpump in order to change the shape of the nullcline. Figure 7A shows the slow-nullclines (thin solid or broken curves) with varying Apump and also the equilibria of the fast subsystem (thick solid and broken curves) with Iext = 0, 20 and 40. By the increase of Apump , the nullcline of the slow subsystem shifts upward, and the intersection point of the equilibrium curve of the fast subsystem (Iext = 0) with the slow-nullcline is then located at an unstable equilibrium. Therefore, the membrane potential of the full system is spiking when Iext = 0. Figure 7B is the similar diagram to Fig. 7A, where the value of Kpump is varied (Apump is varied in Fig. 7A). By the change of Kpump value, the shape of the slow-nullcline is not changed much, therefore the intersection point of the equilibrium curve of the fast subsystem with the nullcline keeps staying at stable equilibria and the full system remains stable at a resting state. Next, in Fig. 8, we show an example of spontaneous spiking induced by an increase of Apump (Apump = 20). The gray colored orbit in Fig. 8A is the projected trajectory of the oscillatory membrane potential and the waveform is shown in Fig. 8B. Because the equilibrium curve of the fast subsystem intersects with the slow-nullcline at the unstable equilibrium, the membrane potential oscillates even though Iext = 0. The projected trajectory of the full system follows the stable equilibrium of the fast subsystem (the lower branch of thick curve) for a long time, and this prolongs the inter-spike interval. After the trajectory passes through the intersection point, the membrane potential makes a spike. When the trajectory passes through the intersection point, the trajectory winds around the intersection point. This winding is possibly caused by a complicated nonlinear

14

T. Ishiki et al.

10

[A] Iext=0 Iext=20 Iext=40 Apump=5(default) Apump=50 Apump=100 Apump=150

0 -10 -30 -40 -50

-20 -30 -40 -60 -70

0.0002

-80

0.0004 0.0006 0.0008 0.001

or or or

-50

-70 0

Iext=0 Iext=20 Iext=40 Kpump=0.2 Kpump=0.32 Kpump=0.4(default) Kpump=1.2 Kpump=2.0 Kpump=3.2

-10

-60 -80

[B]

0

V (mV)

V (mV)

-20

10

or or or

[Ca2+] (mM)

0

0.0002

0.0004 0.0006 0.0008 0.001

[Ca2+] (mM)

Fig. 7. Variation of the equilibria of the fast subsystem and the nullcline of the slow subsystem, by the change of the parameters of slow subsystem: [A] Apump , [B] Kpump [A]

20

0

0

-20

-20

V (mV)

V (mV)

20

-40

-40 -60

-60 -80

[B]

0

0.0005

0.001

[Ca2+] (mM)

0.0015

-80 20000

20400

20800

21200 21600 22000

t (ms)

Fig. 8. [A] An oscillatory trajectory of the full system (gray curve) with bifurcation diagram of the fast subsystem (Iext = 0, thick solid and broken curve) and the nullcline of the slow subsystem (Apump = 20, thin curve), [B] The oscillatory waveform of the membrane potential

dynamics [14] and makes the subthreshold oscillation of the membrane potential just before the spike in Fig. 8B. However, this subthreshold oscillation cannot be observed in physiological experiments of pyramidal cell (Fig. 2B).

4

Conclusion

In this research, we tried to construct a model of pyramidal cells in the visual cortex focusing on Ca2+ regulation mechanisms, and analyzed the global bifurcation structure of the model in order to seek the physiologically plausible values of its parameter. We analyzed the global bifurcation structure of the model using the maximum conductance of Ca2+ -dependent potassium channel (GKCa ) and the maximum pumping rate of the Ca2+ pump (Apump) as bifurcation parameters. According to the two-parameter bifurcation diagrams we showed that the range where the spontaneous spiking occurs shrinks as the external stimulation current Iext

Global Bifurcation Analysis of a Pyramidal Cell Model

15

increases. Therefore, the membrane potential of the model cannot oscillate for large values of Iext even if both values of GKCa and Apump were changed. We also investigated the effect of Apump and the dissociation constant Kpump on the nullcline of slow subsystem based on the slow/fast decomposition analysis. If Apump is increased, the membrane potential is spiking when Iext = 0 because the nullcline shifts upward and the stability of the full system becomes unstable. When Kpump is varied, the membrane potential keeps a resting state because the full system remains stable. Unfortunately, no expected behavior was obtained by the change of values of the parameters considered in this paper. We have, however, demonstrated the usefulness of such nonlinear analyses as the bifurcation and slow/fast analyses to examine parameter values and construct a physiological model. More detailed study using the other parameters is necessary for the construction of the appropriate model as a future subject.

References 1. Osanai, M., Takeno, Y., Hasui, R., Yagi, T.: Electrophysiological and optical studies on the signal propagation in visual cortex slices. In: Proc. of 2005 Annu. Conf. of Jpn. Neural Network Soc., pp. 89–90 (2005) 2. Herz, A.V.M., Gollisch, T., Machens, C.K., Jaeger, D.: Modeling single-neuron dynamics and computations: a balance of detail and abstraction. Science 314, 80– 85 (2006) 3. Hodgkin, A.L., CHuxley, A.F.: A quantitative description of membrane current and its application to conduction and excitation in nerve. J. Physiol(Lond) 177, 500–544 (1952) 4. Brown, A.M., Schwindt, P.C., Crill, W.E.: Voltage dependence and activation kinetics of pharmacologically defend components of the high-threshold calcium current in rat neocortical neurons. J. Neurophysiol. 70, 1516–1529 (1993) 5. Peterson, B.Z., Demaria, C.D., Yue, D.T.: Calmodulin is the Ca2+ sensor for Ca2+ dependent inactivation of L-type calcium channels. Neuron. 22, 549–558 (1999) 6. Cummins, T.R., Xia, Y., Haddad, G.G.: Functional properties of rat and human neocortical voltage-sensitive sodium currents. J. Neurophysiol. 71, 1052–1064 (1994) 7. Korngreen, A., Sakmann, B.: Voltage-gated K + channels in layer 5 neocortical pyramidal neurons from young rats: subtypes and gradients. J. Neurophy. 525, 621–639 (2000) 8. Kang, J., Huguenard, J.R., Prince, D.A.: Development of BK channels in neocortical pyramidal neurons. J. Neurophy. 76, 188–198 (1996) 9. Hayashida, Y., Yagi, T.: On the interaction between voltage-gated conductances and Ca+ regulation mechanisms in retinal horizontal cells. J. Neurophysiol. 87, 172–182 (2002) 10. Naraghi, M., Neher, E.: Linearized buffered Ca+ diffusion in microdomains and its implications for calculation of [Ca+ ] at the mouth of a calcium channel. J. Neurosci. 17, 6961–6973 (1997) 11. Noble, D.: Influence of Na/Ca exchanger stoichiometry on model cardiac action potentials. Ann. N.Y, Acad. Sci. 976, 133–136 (2002)

16

T. Ishiki et al.

12. Yuan, W., Burkhalter, A., Nerbonne, J.M.: Functional role of the fast transient outward K+ current IA in pyramidal neurons in (rat) primary visual cortex. J. Neurosci. 25, 9185–9194 (2005) 13. Doedel, E.J., Champeny, A.R., Fairgrieve, T.F., Kunznetsov, Y.A., Sandstede, B., Xang, X.: Continuation and bifurcation software for ordinary differential equations (with HomCont). Technical Report, Concordia University (1997) 14. Doi, S., Kumagai, S.: Generation of very slow neuronal rhythms and chaos near the Hopf bifurcation in single neuron models. J. Comp. Neurosci. 19, 325–356 (2005)

Appendix Itotal = INa + IKs + IKf + IK−Ca + ICaL + ICa + Ileak + Iex ICatotal = ICaL + ICa − 2Iex + Ipump 2 INa = GNa · M 1 · H1 · (V − ENa ), GNa = 13.0(mS/cm ), ENa = 35.0(mV) V +29.5 τM1 (V ) = 1/ 0.182 1−expV +29.5 − 0.124 1−exp [− V +29.5 ] [ V +29.5 ] 6.7 6.7   V +124.955511   V +10.07413  τH1 (V ) = 0.5 + 1/ exp − 19.76147 + exp − 20.03406 2 IKs = GKs  · M 2 · H2 · (V − EK ), GKs = 0.66(mS/cm ), EKs = −103.0(mV) 1.25 + 115.0 exp[0.026V ], (V < −50mV) τM2 (V ) = 1.25 + 13.0 exp[−0.026V ], (V ≥ −50mV)   τH2 (V ) = 360.0 + (1010.0 + 24.0(V + 55.0)) exp (−(V + 75.0)/48.0)2 IKf = GKf · M 3 · H3 · (V − EKf ), GKf = 0.27(mS/cm2 ), EKf = EKs   τM3 (V ) = 0.34 + 0.92 exp (−(V + 71.0)/59.0)2   τH3 (V ) = 8.0 + 49.0 exp (−(V + 37.0)/23.0)2 IKCa = GKCa · M 4 · (V − EKCa ), GKCa = 12.5(mS/cm2 ), EKCa = EKs M 4∞ (V, [Ca2+ ]) = ([Ca2+ ]/([Ca2+ ] + Kh )) · (1/(1 + exp[−(V + 12.7)/26.2])) Kh = 0.15(μM) 1.25 + 1.12 exp[(V + 92.0)/41.9] (V < 40mV) τM4 (V ) = 27.0 (V ≥ 40mV)

2+  (2F )2 [Ca ] exp[2V F /RT ]−[Ca2+ ]o ICaL = PCaL · M 5 · H5 · RT · V · exp[2V F /RT ]−1 4

PCaL = 0.225(cm/ms), H5∞ = KCa 4 /(KCa 4 + [Ca2+ ] ), KCa = 4.0(μm) τM5 (V ) = 2.5/(exp[−0.031(V + 37.1)] + exp[0.031(V + 37.1)]) τH5 = 2000.0

2+  )2 exp[2V F /RT ]−[Ca2+ ]o ICa = PCa · M 6 · H6 · (2F · V · [Ca ] exp[2V R·T F /RT ]−1 PCa = 0.155(cm/ms), τH6 = 2000.0 τM6 (V ) = 2.5/(exp[−0.031(V + 37.1)] + exp[0.031(V + 37.1)]) Ileak = V /30.0      Iex = k [Na+ ]3i · [Ca2+ ]o · exp s · VRTF − [Na+ ]3o · [Ca2+ ] · exp −(1 − s) · VRTF −5 2 4 k = 6.0 × 10 (μA/cm /mM ), s = 0.5 Ipump = (2 · F · Apump · [Ca2+ ])/([Ca2+ ] + Kpump ) Apump = 5.0(pmol/s/cm2 ), Kpump = 0.4(μM) M i∞ = 1/(1 + exp (−(V − αM i )/βM i ), i = 1, 2, 3, 5, 6 Hj∞ = 1/(1 + exp ((V − αHj )/βHj ), j = 1, 2, 3, 6 i 1 2 3 5 6 j 1 2 3 6 αM i (mV ) −29.5 −3.0 −3.0 −18.75 18.75 βM i 6.7 10.0 10.0 7.0 7.0

αHj (mV ) −65.8 −51.0 −66.0 −12.6 βHi 7.1 12.0 10.0 18.9

Global Bifurcation Analysis of a Pyramidal Cell Model

17

C = 1.0(μF/cm2 ), S = 3.75(/cm), k− = 5.0(/ms), k+ = 500.0(/mM · ms) [Ca2+ ]o = 2.5(mM), [Na+ ]i = 7.0(mM), [Na+ ]o = 150.0(mM)

T and R denote the absolute temperature and gas constant, respectively. In all ionic currents of the model, the powers for gating variables (M 1, · · · , M 6, H1, · · · , H6) are approximately set at one in order to simplify the equations. Leak current is assumed to have no ion selectivity and follow Ohm’s law. Thus, the reversal potential of leak current is set at 0 (mV), though it might be unusual.

Representation of Medial Axis from Synchronous Firing of Border-Ownership Selective Cells Yasuhiro Hatori and Ko Sakai Graduate School of Systems and Information Engineering, University of Tsukuba, 1-1-1 Tennodai, Tsukuba, Ibaraki, 305-8573 Japan [email protected] [email protected] http://www.cvs.cs.tsukuba.ac.jp/

Abstract. The representation of object shape in the visual system is one of the most crucial questions in brain science. Although we can perceive figure shape correctly and quickly, without any effort, the underlying cortical mechanism is largely unknown. Physiological experiment with macaque indicated the possibility that the brain represents a surface with Medial Axis (MA) representation. To examine whether early visual areas could provide basis for MA representation, we constructed the physiologically realistic, computational model of the early visual cortex, and examined what constraint is necessary for the representation of MA. Our simulation results showed that simultaneous firing of BorderOwnership (BO) selective cells at the stimulus onset is a crucial constraint for MA representation. Keywords: Shape, representation, perception, vision, neuroscience.

1 Introduction Segregation of figure from ground might be the first step in the cortex toward the recognition of shape and object. Recent physiological studies have shown that around 60% of neurons in cortical areas V2 and V4 are selective to Border Ownership (BO) that tells which side of a contour owns the border, or the direction of figure, even about 20% of V1 neurons also showed the BO selectivity [1]. These reports also give an insightful idea on coding of shape in early- to intermediate- level vision. The coding of shape is a major question in neuroscience as well as in robot vision. Specifically, it is of great interest that how the visual information in early visual areas is processed to form the representation of shape. Physiological studies in monkeys [2] suggest that shape is coded by medial axis (MA) representation in early visual areas. The MA representation is the method that codes a surface by a set of circles inscribed along the contour of the surface. An arbitrary shape would be reproduced from the centers of the circles and their diameters. We examined whether neural circuits in early- to intermediate-level visual areas could provide a basis for MA representation. We propose that the synchronized responses of BO-selective neurons could evoke the representation of MA. The physiological study [2] showed that V1 neurons responded to figure shape around 40 ms after the stimulus onset, while the latency of the cells responded to MA was about 200 ms after the onset. Physiological study on M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 18–26, 2008. © Springer-Verlag Berlin Heidelberg 2008

Representation of Medial Axis from Synchronous Firing of BO Selective Cells

19

BO-selective neurons reported that their latency is around 70 ms. These results give rise to the proposal that the neurons detect contours first, then the BO is determined from the local contrast detected, and finally MA representation is constructed. It should be noted that BO could be determined from local contrast surrounding the classical receptive field (CRF), thus a single neuron with surrounding modulation would be sufficient to yield BO selectivity [3]. To examine the proposal, we constructed physiologically realistic, firing model of Border-ownership (BO) selective neuron. We assume that the onset of a stimulus with high contrast evokes simultaneous, strong responses of the neurons. These fast responses will propagate retinotopically, thus the neurons at MA (equidistance to the contour) will be activated. Our simulation results show that even relatively small facilitation from the propagated signals yield the firing of the model-cells at the MA because of the synchronization, indicating that simultaneous firing of BO-selective cells at the stimulus onset could enable the representation of MA.

2 The Proposed Model The model is comprised of three stages: (1) contrast detection stage, (2) BO detection stage, and (3) MA detection stage. The first stage extracts luminance contrast as similar to V1 simple cells. The model cells in the second stage mimic BO-selective neurons, which determine the direction of BO with respect to the border at the CRF based on the modulation from surrounding contrast up to 5 deg in visual angle from the CRF. The third stage spatially pools the responses from the second stage to test the MA representation from simultaneous firing of BO cells. A schematic diagram of the model is given in Figure 1. The following sections describe the functions of each stage. 2.1 Model Neurons To enable the simulations of precise spatiotemporal properties, we implemented single-component firing neurons and their connections through their synapses on NEURON simulator [4]. The cell body of the model cell is approximated by a sphere. We set the radius and the membrane resistance to 50μm and 34.5Ωcm, respectively. The model neurons calculate the membrane potential following the Hodgkin-Huxley equation [5] with constant parameter as shown in table 1. We used the biophysically realistic spiking neuron model because we need to examine exact timing of the firing of BO-selective cells as well as the propagtation of the signals, which cannot be realized by integrate-and-fire neuron model or any other abstract models. 2.2 Contrast Detection Stage The model cells in the first stage have response properties similar to those of V1 cells, including the contrast detection, dynamic contrast normalization and static compressive nonlinearity. The model cells in this stage detect the luminance contrast from oriented Gabor filters with distnct orientations. We limited ourselves to have four orientations for the sake of simplicity; vertical (0 and 180 deg) and horizontal (90 and 270 deg). Gabor filter is defined as follows:

Gθ (x , y ) = cos(2πω

(x sin θ + y cos θ )) × gaussian(x , y , μ x , μ y , σ x , σ y ),

(1)

20

Y. Hatori and K. Sakai

Fig. 1. A schematic illustration of the proposed model. Luminance contrast is detected in the first stage which is then processed to determine the direction of BO by surrounding modulation. F and S represent excitatory and inhibitory regions, respectively, for the surrounding modulation. The last stage detects MA based on the propagations from BO model cells.

where x and y represent spatial location, μx and μy represent central coordinate of Gauss function, σx and σy represent standard deviation of Gauss function, θ and ω represent orientation and spatial frequency, respectively. We take a convolution of an input image

Representation of Medial Axis from Synchronous Firing of BO Selective Cells

21

with the Gabor filters, with dynamic contrast normalization [6] including a static, compressive nonlinear function. For the purpose of efficient computation, the responses of the vertical pathways (0 and 180 deg) are integrated to form vertical orientation, and so do the horizontal pathways (90 and 270 deg), which will be convenient for the computation of iso-orientation suppression and cross-orientation facilitation in the next stage: O θ (x , y ) = (I * G θ )(x , y ) ,

(2)

1 (x , y ) = O0 (x , y ) + O180 (x , y ) , Oiso

(3)

1 (x , y ) = O90 (x , y ) + O270 (x , y ) , Ocross

(4)

where I represents input image, Oθ(x,y) represents the output of convolution (*), O1iso (O1cross) represents integrated responses of vertical (horizontal) pathway. Table 1. The constant values for the model cells used in the simulations

Parameter

Value

Cm

1( F/cm2) 50(mv) -77(mv) -54.3(mv) 0.120(S/cm2) 0.036(S/cm2) 0.0003(S/cm2)

ENa EK El gNa gK gl

μ

2.3 BO Detection Stage The second stage models surrounding modulation reported in early- to intermediatelevel vision for the determination of BO [3]. The model cells integrate surrounding contrast information up to5 deg in visual angle from the CRF center. We modeled the surrounding region with two Gaussians, one for inhibition and the other for facilitation, that are located asymmetrically with respect to the CRF center. If a part of the contour of an object is projected onto the excitatory (or inhibitory) region, the contrast information of the contour that is detected by the first stage is transmitted via a pulse to generate EPSP (or IPSP) of the BO-selective model-cell. In other words, the projection of a figure within the excitatory region facilitates the response of the BO model cell. Conversely, if a figure is projected onto the inhibitory region, the response of the model cell is suppressed. Therefore, surrounding contrast signals from the excitatory and inhibitory regions modulate the activity of BO model cells depending on the direction of figure. In this way, we implemented BO model cells based on the surrounding modulation. Note that Jones and her colleagues reported the orientation dependency in the surrounding modulation in monkeys' V1 cells [7]. The suppression is limited to similar orientations to the preferred orientation of the CRF (iso-orientation suppression), and facilitation is dominant for other orientations (cross-orientation facilitation). We implemented this orientation dependency for surround modulation.

22

Y. Hatori and K. Sakai

Taking into account the EPSP and IPSP from the surrounds, we compute the membrane potential of a BO-selective model cell at time t as follows: O 2 ( x1 , y1 , t ) = input ( x1 , y1 ) + c ∑{E iso (x, y , t - d x1, y1 (x, y ) ) + E cross (x, y , t - d x1, y1 (x, y ))} (5) x, y

,where x1 and y1 represent the spatial position of the BO-selective cell, input(x1,y1) represents the output of the first stage that is O1iso or O1cross, c represents a weight of synaptic connection, Eiso(x,y,t-dx1,y1(x,y)) represents EPSP (or IPSP) that is triggered by the pulse which is generated at t-dx1,y1. Ecross(x,y,t-dx1,y1(x,y)) is the same except for input orientation. And, dx1,y1(x,y) shows time delay in proportion to the distance between the BO-selective cell whose coordinate is (x1,y1) and the connected cell whose coordinate is (x,y). We defined Eiso(x,y,t-dx1,y1(x,y)) and dx1,y1(x,y) as:

(

)

(

)

E iso x , y ,t - d x1 , y1 (x , y ) = gaussian x , y , μ x1 , μ y1 ,σ x1 ,σ y1 × exp d x1 , y1 (x , y ) = ctime

(x

1

((

)

- t - d x1 , y1 ( x , y )

τ

)× (v - e) ,

2 2 - x ) + (y1 - y ) ,

(6) (7)

where τ represents time constant, v represents membrane potential, e represents reversal potential, ctime is a constant that converts distance to time. We set c, τ, e and ctime to 0.6 (or 1.0), 10ms, 0mv and 0.2ms/μm, respectively. And Ecross(x,y,t - dx1,y1(x,y)) is also calculated similarly to eq.6. 2.4 MA Detection Stage The third stage integrates BO information from the second stage to detect the MA. A MA model cell has a single excitatory surrounding region that is represented by a Gaussian. The membrane potential of a MA model cell is given by: O 3 (x2 , y 2 ,t )= cmedial

∑gaussian(x , y , μ

(x ,y )∈(x1 ,y1 )

x2

)

(

)

, μ y2 ,σ x2 ,σ y2 × O 2 x , y ,t - d x2 , y2 (x , y ) ,

(8)

where x2 and y2 represent the spatial position of the MA model cell, μx2 and μy2 represent the center of gauss function, cmedial is a constant that we set to 1.8 (or 10.0), dx2,y2(x,y) is calculated similarly to eq.7. Note that MA model cells receive EPSP only from BO model cells. When a BO model cell is activated, the model cell transmits a pulse to MA model cells, with its magnitude and time delay depending on the distance between the two. If a MA model cell was located equidistance to some parts of the contours, the pulses from the BO model cells on the contours reach the MA cell at the same time to evoke strong EPST that will generate a spike. On the other hand, MA model cells that are located not equidistance to the contours will never evoke a pulse. Therefore, the model cells that are located equidistance to the contours are activated based on simultaneous activation from BO model cells, the neural population of which will represent the medial axis of the object.

3 Simulation Result We carried out the simulations of the model to test whether the model shows the representation of MA. As typical examples, the results for thee types of stimuli are

Representation of Medial Axis from Synchronous Firing of BO Selective Cells

23

shown here, a square, a C-shaped figure, and a natural image of an eagle [8], as shown in Fig 1. Note that the model cells described in the previous sections are distributed retinotopically to form each layer with 138 × 138 cells.

(A)

(B)

(C)

Fig. 2. Three examples of stimuli used for the simulations. A square (A), a C-shaped figure (B), and a natural image of an eagle from Berkeley Segmentation Dataset [8] (C).

3.1 A Single Square First, we tested the model with a single square similar to that used in corresponding physiological experiment [2], as shown in Fig.1(A). Although we carried out the simulations retinotopically in 2D, for the purpose of graphical presentation, the responses of a horizontal cross section of the stimulus indicated by Fig. 3(A) are shown here in Fig. 3(B). The figure exhibits the firing rates for two types of the model cells, BO model cells responding to contours (solid lines at the horizontal positions of -1 and 1) and MA model cells (dotted lines at the horizontal position of 0). We observe a clear peak corresponding to MA at the center, as similar to the results of physiological experiments by Lee, et al. [2]. Although we tested the model along the horizontal cross section, the response of a vertical is identical. This result suggests that simultaneous firing of BO cells is capable of generating MA without any other particular constraints. 14 12 c 10 e s m 0 8 0 4 / s 6 e ki p s

4 2 0

(A)

-1 0 Horizontal position

1

(B)

Fig. 3. The simulation results for a square. (A) We show the responses of the cells located along the dashed line (the number of neurons is 138). Horizontal positions -1 and 1 represent the places on the vertical edges of the square. Zero represents the center of the square. (B) The responses of the model cells in firing rate along the cross-section as a function of the horizontal location. A clear peak at the center corresponding to MA is observed.

24

Y. Hatori and K. Sakai

3.2 C-Shape We tested the model with a C-shaped figure that has been suggested to be difficult shape for the determination of BO. Fig. 4 shows the simulation result for the C-shape. The responses of a horizontal and vertical cross sections, as indicated in Fig.3(A), are shown here in Fig. 4(B) and (C), respectively, with the conventions same as Fig.1. The figure exhibits the firing rates for two types of the model cells, BO model cells responding to contours (solid lines) and MA model cells (dotted lines). MA representation predicts a strong peak at 0 along the horizontal cross section, and distributed strong responses along the vertical cross section. Although we observe the maximum responses at the centers corresponding to MA, the peak along the horizontal was not significant, and the distribution along the vertical was peaky and uneven. It appears that MA cells cannot integrate properly the signals propagated from BO cells. This distributed MA response comes from complicated propagations of BO signals from the concaved shape. Furthermore, as has been suggested [3], the C-shaped figure is a challenging shape for the determination of BO. Therefore, the BO signals are not clear before the propagation begins. We would like to note that the model is still capable of providing a basis for MA representation for a complicated figure.

(A) 14

14

12

12

c10 e s m 0 8 0 4 / s 6 e ik p s 4

c10 e s m 0 8 0 4 / s 6 e ik p s

2

2

0

4

-1

0 1 Horizontal position

(B)

0

-1

0 Vertical position

1

(C)

Fig. 4. The simulation results for the C-shaped figure. (A) Positions of the analysis along horizontal and vertical cross-sections as indicated by dashed line. The responses of the model cells in firing rate along the horizontal cross section (B), and that along the vertical cross section (C). Solid and dotted lines indicate the responses of BO and MA model cells, respectively. Although the maximum responses are observed at the centers, the responses are distributed.

3.3 Natural Images The model has shown its ability to extract a basis for MA representation for not only simple but also difficult shapes. To further examine the model for arbitrary shapes, we

Representation of Medial Axis from Synchronous Firing of BO Selective Cells

25

tested the model with natural images taken from Berkeley Segmentation Dataset [8]. Fig.2(C) shows an example, an eagle stops on a tree branch. Because we are interested in the representation of shape, we extracted its shape by binarizing the gray scale, as shown in Fig. 5(A). The simulation results of BO and MA model cells are shown in Fig. 5(B). We plotted the responses along the horizontal cross-section indicated in Fig. 5(A). Although the shape is much more complicated, detailed and asymmetric, the results are very similar to that for a square as shown in Fig.2(B). The BO model cells responded to the contours (horizontal positions at -1 and 1), and MA model cells exhibited a strong peak at the center (horizontal position at 0). This result indicates that the model detects MA for figures with arbitrary shape. The aim of the simulations with natural images is to test a variety of stimulus shape and configurations that are possible in natural scenes. Further simulations with a number of natural images is expected, specifically with images including occulusion, multiple objects and ambiguous figures. 12 10 c e s 8 m 0 0 4 6 / s e ik 4 p s

2 0

(A)

-1 0 1 Horizontal position

(B)

Fig. 5. An example of simulation results for natural images. (A) The binary image of an eagle together with the horizontal cross-section for the graphical presentation of the results. (B) The responses of the model cells in firing rate along the cross-section as a function of the horizontal location. Horizontal positions -1 and 1 represent the places on the vertical edges of the eagle. Zero represents the center of the bird. A clear peak at the center corresponding to MA is observed. Although the shape is much more complicated, detailed and asymmetric, the results are very similar to that for a square as shown in Fig. 2(B), indicating robustness of the model.

4 Conclusion We studied whether early visual areas could provide basis for MA representation, specifically what constraint is necessary for the representation of MA. Our results showed that simultaneous firing of BO-selective neurons is crucial for MA representation. We implemented the physiologically realistic firing model neurons that have connections from BO model cells to MA model cells. If a stimulus is presented at once so that BO cells along the contours fire simultaneously, and if a MA cell is located equidistant from some of contours of the stimulus, then the MA cell fires because of synchronous signals from the BO cells that give rise to strong EPSP. We showed three typical examples of the simulation results, a simple square, a difficult C-shaped figure, and a natural image of an eagle. The simulation results showed that the model provides a basis for MA representation for all three types of stimuli. These

26

Y. Hatori and K. Sakai

results suggest that the simultaneous firing of BO cells is an essence for the MA representation in early visual areas.

Acknowledgment We thank Dr. Haruka Nishimura for her insightful comments and Mr. Satoshi Watanabe for his help in simulations. This work was supported by Grant-in-aid for Scientific Research from the Brain Science Foundation, the Okawa Foundation, JSPS (19530648), and MEXT of Japan (19024011).

References 1. Zhou, H., Friedman, H.S., Heydt, R.: Coding of Border Ownership in Monkey Visual Cortex. The Journal of Neuroscience 86, 2796–2808 (2000) 2. Lee, T.S., Mumford, D., Romero, R., Lamme, V.A.F.: The role of the primary visual cortex in higher level vision. Vision Research 38, 2429–2454 (1998) 3. Sakai, K., Nishimura, H.: Surrounding Suppression and Facilitation in the determination of Border Ownership. The Journal of Cognitive Neuroscience 18, 562–579 (2006) 4. NEURON: http://www.neuron.yale.edu/neuron/ 5. Johnston, D., Wu., S. (eds.): Foundations of Cellular Neurophysiology. MIT Press, Cambridge (1999) 6. Carandini, M., Heeger, D.J., Movshon, J.A.: Linearity and Normalization in Simple Cells of the Macaque Primary Visual Cortex. The Journal of Neuroscience 21, 8621–8644 (1997) 7. Jones, H.E., Wang, W., Silito, A.M.: Spatial Organization and Magnitude of Orientation Contrast Interaction in Primate V1. Journal of Neurophysiology 88, 2796–2808 (2002) 8. The Berkeley Segmentation Dataset: http://www.eecs.berkeley.edu/Research/ Projects/CS/vision/grouping/segbench/ 9. Engel, A.K., Fries, P., Singer, W.: Dynamic predictions: oscillations and synchrony in topdown processing. Nature Reviews Neuroscience 2, 704–716 (2001)

Neural Mechanism for Extracting Object Features Critical for Visual Categorization Task Mitsuya Soga1 and Yoshiki Kashimori1,2 1

2

Dept. of Information Network Science, Graduate school of Information Systems, Univ. of Electro-Communications, Chofu, Tokyo 182-8585, Japan Dept. of Applied Physics and Chemistry, Univ. of Electro-communications, Chofu, Tokyo 182-8585, Japan

Abstract. The ability to group visual stimuli into meaningful categories is a fundamental cognitive process. Some experiments are made to investigate the neural mechanism of visual categorization. Although experimental evidence is known that prefrontal cortex (PFC) and inferiortemporal (IT) cortex neurons sensitively respond in categorization task, little is known about the functional role of interaction between PFC and IT in categorization task To address this issue, we propose a functional model of visual system, and investigate the neural mechanism for the categorization task of line drawings of faces. We show here that IT represents similarity of face images based on the information of the resolution maps of early visual stages. We show also that PFC neurons bind the information of part and location of the face image, and then PFC generates a working memory state, in which only the information of face features relevant to the categorization task are sustained.

1

Introduction

Visual categorization is fundamental to the behavior of higher primates. Our raw perceptions would be useless without our classification of items such as animals and food. The visual system has the ability to categorize visual stimuli, which is the ability to react similarity to stimuli even when they are physically distinct, and to react differently to stimuli that may be similar. How does the brain group stimuli into meaningful categories? Some experiments have been made to investigate the neural mechanism of visual categorization. Freedman et al.[1] examined the responses of neurons in the prefrontal cortex(PFC) of monkey trained to categorize animal forms(generated by computer) as either “doglike” or “catlike”. They reported that many PFC neurons responded selectively to the different types of visual stimuli belonging to either the cats or the dogs category. Sigala and Logothetis [2] recorded from inferior temporal (IT) cortex after monkey learned a categorization task, and found that selectivity of the IT neurons was significantly increased to features critical for the task. The numerous reciprocal connections between PFC and IT could allow the necessary interactions to select the best diagnostic features of stimuli [3]. However, little is known about the role of interaction between IT M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 27–36, 2008. c Springer-Verlag Berlin Heidelberg 2008 

28

M. Soga and Y. Kashimori

and PFC in categorization task that gives the category boundaries in relation to behavioral consequences. To address this issue, we propose a functional model of visual system in which categorization task is achieved based on functional roles of IT and PFC. The functional role of IT is to represent features of object parts, based on different resolution maps in early visual system such as V1 and V4. In IT, visual stimuli are categorized by the similarity based on the features of object parts. Posterior parietal (PP) encodes the location of object part to which attention is paid. The PFC neurons combine the information about feature and location of object parts, and generate a working memory of the object information relevant to the categorization task. The synaptic connections between IT and PFC are learned so as to achieve the categorization task. The feedback signals from PFC to IT enhance the sensitivity of IT neurons that respond to the features of object parts critical for the categorization task, thereby enabling the visual system to perform quickly and reliably task-dependent categorization. In the present study, we present a neural network model which makes categories of visual objects depending on categorization task. We investigated the neural mechanism of the categorization task about line drawings of faces used by Sigala and Logothetis [2]. Using this model we show that IT represents similarity of face images based on the information of the resolution maps in V1 and V4. We also show that PFC generates a working memory state, in which only the information of face features relevant to the categorization task are sustained.

2

Model

To investigate the neural mechanism of visual categorization, we made a neural network model for a form perception pathway from retina to prefrontal cortex(PFC). The model consists of eight neural networks corresponding to the retina, lateral geniculate nucleus(LGN), V1, V4, inferior temporal cortex (IT), posterior parietal(PP), PFC, and premotor area, which are involved in ventral and dorsal pathway [4,5]. The network structure of our model is illustrated in Fig. 1. 2.1

Model of Retina and LGN

The retinal network is an input layer, on which the object image is projected. The retina has a two-dimensional lattice structure that contains NR x NR pixel scene. The LGN network consists of three different types of neurons with respect to the spatial resolution of contrast detection, fine-tuned neurons with high spatial frequency (LGNF), middle-tuned neurons with middle spatial frequency (LGNM), and broad-tuned neurons with low spatial frequency (LGNB). The output of LGNX neuron (X=B, M, F) of (i, j) site is given by ILGN (iR , jR ; x) =

 i

j

IR (iR , jR )M (i, j; x),

(1)

Neural Mechanism for Extracting Object Features

29

Fig. 1. The structure of our model. The model is composed of eight modules structured such that they resemble ventral and dorsal pathway of the visual cortex, retina, lateral geniculate nucleus (LGN), primary visual cortex (V1), V4, inferior temporal cortex (IT), posterior parietal (PP), prefrontal cortex (PFC), and premotor area. a1 ∼ a3 mean dynamical attractors of visual working memory.

    (i − iR )2 + (j − jR )2 (i − iR )2 + (j − jR )2 M (i, j; x) = A exp − −B exp − , 2 2 σ1x σ2x (2) where, IR (iR , jR ) is the gray-scale intensity of the pixel of retinal site (iR , jR ), and the function M (i, j; X) is Mexican hat-like function that represents the convergence of the retinal inputs with ON center-OFF surrounding connections between retina and LGN. The parameter values were set to be A = 1.0, B = 1.0, σ1x = 1.0, and σ2x = 2.0. 2.2

Model of V1 Network

The neurons of V1 network have the ability to elemental features of the object image, such as orientation and edge of a bar. The V1 network consists of three different types of networks with high, middle, and broad spatial resolutions, V1B, V1M, and V1F, each of which receives the outputs of LGNB, LGNM, and LGNF, respectively. The V1X network (X=B, M, F) contains MX × MX hypercolumns, each of which contains LX orientation columns. The neurons in V1X (X=B, M, F) have receptive field performing a Gabor transform. The output of V1X neuron of (i, j) site is given by  IV 1 (i, j, θ; x) = ILGN (p, q; x)G(p, q, θ; x), (3) p

q

30

M. Soga and Y. Kashimori

  1 (i − p)2 (j − q)2 G(p, q, θ; x) = exp − + 2 2 2πσxX σyX 2 σGx σGy π

× sin 2πfx p cos θ + 2πfy q sin θ + , 2 1



(4)

where fx , fy are spatial frequencies of x- and y- coordinate, respectively. The parameter values were σGx = 1, σGy = 1, fx = 0.1Hz, and fy = 0.1Hz. 2.3

Model of V4 Network

The V4 network consists of three different networks with high, middle, and low spatial resolutions, which receive convergent inputs from the cell assemblies with the same tuning in V1F, V1M, and V1B. The convergence of outputs of V1 neurons enables V4 neurons to respond specifically to a combination of elemental features such as a cross and triangle represented on the V4 network. 2.4

Model of PP

The posterior parietal(PP) network consists of NP P x NP P neurons, each of which corresponds to a spatial position of each pixel of the retinal image. The functions of PP network are to represent the spatial position of a whole object and the spatial arrangement of its parts in the retinotopic coordinate system and to mediate the location of the object part to which attention is paid. 2.5

Model of IT

The network of IT consists of three subnetworks, each of which receives the outputs of V4F, V4M, and V4B maps, respectively. Each network has neurons tuned to various features of the object parts depending on the resolutions of V4 maps and the location of the object parts to which attention is directed. The first sub-network detects a broad outline of a whole object, the second subnetwork detects elemental figures of the object that represent elemental outlines of the object, and the third subnetwork represents the information of the object parts based on the fine resolution of V4F map. Each subnetwork was made based on Kohonen’s self-organized map model. The elemental figures in the second subnetwork may play an important role in extracting the similarity to the outlines of objects. 2.6

Model for Working Memory in PFC

The PFC memorizes the information of spatial positions of the object parts as dynamical attractors. The functional role of the PFC is to reproduce a complete form of the object by binding the information of the object parts memorized in the second and third subnetworks of ITC and the their spatial arrangements represented by PP network. The PFC network model was made based on the dynamical map model [6,7]. The network model consists of three types of neurons, M, R, and Q neurons. Q neuron is connected to M neuron with inhibitory

Neural Mechanism for Extracting Object Features

31

synapse. M neurons are interconnected to each other with excitatory and inhibitory synaptic connections. M neuron layer of the present model corresponds to an associative neural network. M neurons receive inputs from three IT subnetworks. Dynamical evolutions of membrane potentials of the neurons, M, R, and Q, are described by

τm

tm   dumi = −umi + Wmm,ij (t, Tij )Vm (t − τij ) + Wmq U qi dt j τij =0   +Wmr Uri + WIT,ik XkIT + WP P,il XlP P , R

(5)

l

duqi = −uqi + Wqm Vmi , dt duir τri = −uri + Wrm Vmi , dt τqi

(6) (7)

where um,i , uq,i , and ur,i are the membrane potentials of ith M neuron, ith Q neuron, and ith R neuron, respectively. τmi , τqi , τri are the relaxation times of these membrane potentials. τij is the delay time of the signal propagation from jth M neuron to ith one, and τmax is the maximum delay time. The time delay plays an important role in the stabilization of temporal sequence of firing pattern. Wmm,ij (t, Tij ) is the strength of the axo-dendric synaptic connection from jth M neuron to ith M neuron whose propagation delay time is τij . Wmq (t), Wmr (t), and Wqm (t), and Wrm (t) are the strength of the dendro-dendritic synaptic connection from Q neuron to M neuron, from R neuron to M neuron, from M neuron to Q neuron, and from M neuron to R neuron, respectively. Vm is the output of ith M neuron, Uqi and Uri are dendritic outputs of ith Q neuron and ith R neuron, respectively. Outputs of Q and R neurons are given by sigmoidal functions of uqi and uri , respectively. M neuron has a spike output, the firing probability of which is determined by a sigmoidal function of umi . WIT,ik and WP P,il are the synaptic strength from kth IT neuron to ith M neuron and that from lth PP neuron to ith M neuron, respectively. XkIT and XlP P are the output of k th IT neuron and lth PP neuron, respectively. The parameters were set to be τm = 3ms, τqi = 1ms, τri = 1ms, Wmq = 1, Wmr = 1, Wqm = 1, Wqr = −10, and τm = 38ms. Dynamical evolution of the synaptic connections are described by τw

dWmm,ij (t, Tij ) = −Wmm,ij (t, Tij ) + λVmi (t)Vmj (t − Tij ), dt

(8)

where τw is a time constant, and λ is a learning rate.The parameter values were τw = 1200ms, and λ = 28. The PFC connects reciprocally with IT, PP, and the premotor networks. The synaptic connections between PFC and IT and those between PFC and PP are learned by Hebbian learning rule.

32

2.7

M. Soga and Y. Kashimori

Model of Premotor Cortex

The model of premotor area consists of neurons whose firing correspond to action relevant to the categorization task, that is, pressing right or left lever.The details of mathematical description are described in Refs. [8-10].

3 3.1

Neural Mechanism for Extracting Diagnostic Features in Visual Categorization Task Categorization Task

We used the line drawings of faces used by Sigala and Logothetis [2] to investigate the neural mechanism of categorization task. The face images consist of four varying features, eye height, eye separation, nose length, and mouse height. The monkeys were trained to categorize the face stimuli depending on two diagnostic features, eye height and eye separation. The two diagnostic features allowed separation between classes along a linear category boundary as shown in Fig.2b. The face stimuli were not linearly separable by using the other two, non-diagnostic features, or nose length and mouth height. On each trial, the monkeys saw one face stimulus and then pressed one of two levers to indicate category. Thereafter, they received a reward only if they chose correct category. After the training, the monkeys were able to categorize various face stimuli based on the two diagnostic features. In our simulation, we used four training stimuli shown in Fig.2a and test stimuli with varying four features.

Fig. 2. a) The training stimulus set consisted of line drawing of faces with four varying features: eye separation, eye height, nose length and mouth height. b) In the categorization task, the monkyes were presented with one stimulus at a time. The two categories were linearly separable along the line. The test stimuli are illustrated by the marks, ‘x‘ and ‘o‘. See Ref.2 for details of the task.

Neural Mechanism for Extracting Object Features

3.2

33

Neural Mechanism for Accomplishing Visual Categorization Task

Neural mechanism for accomplishing visual categorization task is illustrated in Fig.3. Object features are encoded by hierarchical processing at each stage of ventral visual pathway from retina to V4. The IT neurons encode the information of object parts such as eyes, nose, and mouth. The PP neurons encode the location of object parts to which attention should be directed. The information of object part and its location are combined by a dynamical attractor in PFC. The output of PFC is sent to two types of premotor neurons, each firing of which leads to pressing of the right or left lever. When monkeys exhibit the relevant behavior for the task and then receive a reward, the attractor in PFC, which represents the object information relevant to the task, is gradually stabilized by the facilitation of learning across the PFC neurons and that between PFC and premotor neurons. On the other hand, when monkeys exhibit the irrelevant behavior for the task and then receive no reward, the attractor associated with the irrelevant behavior is destabilized, and then eliminated in the PFC network. As a result, the PFC retains only the object information relevant to the categorization task, as working memory. The feedback from PFC to IT and PP makes the responses of IT and PP neurons strengthened, thereby enabling the visual system to rapidly and accurately discriminate between object features belonging to different categories. When monkey pays attention to a local region of face stimulus, the attention signal from other brain area such as prefrontal eye field increases the activity of PP neurons encoding the location of face part to which attention is directed. The PP neurons send their outputs back to V4, and thereby increasing the activity of V4 neurons encoding the feature of the face parts which the attention is paid, because V4 has the same retinotopic map as PP. Thus the attention to PP allows V4 to send IT only the information of face part to which attention is directed,

Fig. 3. Neural mechanism for accomplishing visual categorization task. The information about object part and it’s location are combined to generate working memory, indicated by α and β. The solid and dashed lines indicate the formation and elimination of synaptic connection, respectively.

34

M. Soga and Y. Kashimori

leading to generation of IT neurons encoding face parts. Furthermore, as the training proceeds, the attention relevant to the task is fixed by the learning of synaptic connections between PFC and PP, allowing monkey to perform quickly the visual task.

4 4.1

Results Information Processing of Visual Images in Early Visual Areas

Figure 4 shows the responses of neurons in early visual areas, LGN, V1, and V4, to the training stimulus, face 1 shown in Fig. 2a. The visual information is processed in a hierarchical manner, because the neurons involved in the pathway from LGN to V4 have progressively larger receptive fields and prefer more complex stimuli. At the first stage, the contrast of the stimulus is encoded by ON center-OFF surrounding receptive field of LGN neurons, as shown in Fig. 4b. Then, the V1 neurons, receiving the outputs of LGN neurons, encode the information of directional features of short bars contained in the drawing face, as shown in Fig. 4c. The V4 network was made by Kohonen’s self-organized map so that the V4 neurons could respond to more complex features of the stimulus. Figure 4d shows that the V4 neurons respond to the more complex features such as eyes, nose, and mouth.

Fig. 4. Responses of neurons in LGN, V1, and V4. The magnitude of neuronal responses in these areas is illustrated with a gray scale, in which the response magnitude is increased with increase of gray color. a) Face stimulus. The image is 90 x 90 pixel scale. b) Response of LGN neurons. c) Responses of V1 neurons tuned to four kinds of directions. d) Responses of V4 neurons. The kth neurons (k=1-3) encode the stimulus features such as eyes, nose, and mouth, respectively.

4.2

Information Processing of Visual Images in IT Cortex

Figure 5a shows the ability of IT neurons encoding eye separation and eye height to categorize test stimuli of faces. The test stimuli with varying the two features were categorized by the four ITC neurons learned by the four training stimuli shown in Fig.2a, suggesting that the ITC neurons are capable for separating test stimuli into some categories, based on similarity to the features of face parts.

Neural Mechanism for Extracting Object Features

35

Fig. 5. a) Ability of IT neurons to categorize face stimuli for two diagnostic features. The four IT neurons were made by using the four kinds of training stimuli shown in Fig. 2a, whose features are represented by four kinds of symbols (circle, square, triangle, cross). The test stimuli, represented by small symbols, are categorized by the four IT neurons. The kind of small symbols means the symbol of IT neuron that categorizes the test stimulus. The solid lines mean the boundary lines of the categories. b) Temporal variation of dynamic state of the PFC network during the categorization task. The attractors representing the diagnostic features are denoted by α ∼ δ, and the attractors representing non-diagnostic feature is denoted by . A mark on the row corresponding to α ∼  indicates that the network activity stays in the attractor. The visual stimulus of face 1 was applied to the retina at 300m ∼ 500ms.

Similarly, the IT neurons encoding nose length and mouth height separated test stimuli into other categories on the basis of similarity of the two features. However, the classification in the IT is not task-dependent, but is made based on the similarity of face features. 4.3

Mechanism for Generating Working Memory Attractor in PFC

The PFC combines the information of face features and that of location of face parts to which attention is directed, and then makes memory attractors about the information. Figure 5b shows temporal variation of the memory attractors in PFC. The information about face parts with the two diagnostic features is represented by attractors X (X= α, β, γ, δ), in which X represents the information about eye separation and eye height of the four training stimuli and the location around eyes. The attractors X are dynamically linked in the PFC. As shown in Fig. 5b, the information about face parts with the diagnostic features are memorized as working memory α ∼ δ, because the synaptic connections between PFC and premotor area are strengthened by a reward signal given by the choice of correct categorization. On the other hand, the information about face parts with non-diagnostic features are not memorized as a stable attractor, as shown by  in Fig.5b, because the information of non-diagnostic features does

36

M. Soga and Y. Kashimori

not lead to correct categorization behavior. Thus, the PFC can retain only the information required for the categorization task, as working memory.

5

Concluding Remarks

In the present study, we have shown that IT represents similarity of face images based on the resolution maps of V1 and V4, and PFC generates a working memory state, in which the information of face features relevant to categorization task are sustained. The feedback from PFC to IT and PP may play an important role in extracting the diagnostic features critical for the categorization task. The feedback from PFC increases the sensitivity of IT and PP neurons which encode the relevant object feature and location to the task, respectively. This allows the visual system to rapidly and accurately perform the categorization task. It remains to see how the feedback from PFC to IT and PP makes the functional connections across the three visual areas.

References 1. Freedman, D.J., Riesenhube, M., Poggio, T., Miller, E.K.: Categorical representation of visual stimuli in the primate prefrontal cortex. Science 291, 312–316 (2001) 2. Sigala, N., Logothetis, N.K.: Visual categorization shapes feature selectivity in the primate temporal cortex. Nature 415, 318–320 (2002) 3. Hagiwara, I., Miyashita, Y.: Categorizing the world: expert neurons look into key features. Nature Neurosci. 5, 90–91 (2002) 4. Marcelija, S.: Mathematical description of the responses of simple cortical cells. J. Opt. Soc. Am. 70, 1297–1300 (1980) 5. Rolls, E.T., Deco, G.: Computational Neuroscience of Vision. Oxford University Press, Oxford (2002) 6. Hoshino, O., Inoue, S., Kashimori, Y., kambara, T.: A hierachical dynamical map as a basic frame for cortical mapping and its application to priming. Neural Comput. 13, 1781–1810 (2001) 7. Hoshino, O., Kashimori, Y., Kambara, T.: An olfactory recognition model of spatiotemporal coding of odor quality in olfactory bulb. Biol. Cybernet 79, 109–120 (1998) 8. Suzuki, N., Hashimoto, N., Kashimori, Y., Zheng, M., Kambara, T.: A neural model of predictive recognition in form pathway of visual cortex. Biosystems 79, 33–42 (2004) 9. Ichinose, Y., Kashimori, Y., Fujita, K., Kambara, T.: A neural model of visual system based on multiple resolution maps for categorizing visual stimuli. In: Proceedings of ICONIP 2005, pp. 515–520 (2005) 10. Kashimori, Y., Suzuki, N., Fujita, K., Zheng, M., Kambara, T.: A functional role of multiple spatial resolution maps in form perception along the ventral visual pathway. Neurocomputing 65-66, 219–228 (2005)

An Integrated Neuro-mechanical Model of C. elegans Forward Locomotion Jordan H. Boyle, John Bryden, and Netta Cohen School of Computing, University of Leeds, Leeds LS2 9JT, United Kingdom

Abstract. One of the most tractable organisms for the study of nervous systems is the nematode Caenorhabditis elegans, whose locomotion in particular has been the subject of a number of models. In this paper we present a first integrated neuro-mechanical model of forward locomotion. We find that a previous neural model is robust to the addition of a body with mechanical properties, and that the integrated model produces oscillations with a more realistic frequency and waveform than the neural model alone. We conclude that the body and environment are likely to be important components of the worm’s locomotion subsystem.

1

Introduction

The ultimate aim of neuroscience is to unravel and completely understand the links between animal behaviour, its neural control and the underlying molecular and genetic computation at the cellular and sub-cellular levels. This daunting challenge sets a distant goal post in the study of the vast majority of animals, but work on one animal in particular, the nematode Caenorhabditis elegans, is leading the way. This tiny worm has only 302 neurons and yet is capable of generating an impressive wealth of sensory-motor behaviours. With the first fully sequenced animal genome [1], a nearly complete wiring diagram of the nervous circuit [2], and hundreds of well characterised mutant strains, the link between genetics and behaviour never seemed more tractable. To date, a number of models have been constructed of subcircuits within the C. elegans nervous system, including sensory circuits for thermotaxis and chemotaxis [3,4], reflex control such as tap withdrawal [5], reversals (from forward to backward motion and vice versa) [6] and head swing motion [7]. Locomotion, like the overwhelming majority of known motor activity in animals, relies on the rhythmic contraction of muscles, which are controlled or regulated by neural networks. This system consists of a circuit in the head (generally postulated to initiate motion and determine direction) and an additional subcircuit along the ventral cord (responsible for propagating and sustaining undulations, and potentially generating them as well). Models of C. elegans locomotion have tended to focus on forward locomotion, and in particular, on the ability of the worm to generate and propagate undulations down its length [8,9,10,11,12]. These models have tended to study either the mechanics of locomotion [8] or the forward locomotion neural circuit [9,10,11,12]. In this paper we present simulations of an M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 37–47, 2008. c Springer-Verlag Berlin Heidelberg 2008 

38

J.H. Boyle, J. Bryden, and N. Cohen

integrated model of the neural control of forward locomotion [12] with a minimal model of muscle actuation and a mechanical model of a body, embedded in a minimal environment. The main questions we address are (i) whether the disembodied neural model is robust to the addition of a body with mechanical properties; and (ii) how the addition of mechanical properties alters the output from the motor neurons. In particular, models of the isolated neural circuit for locomotion suffer from a common limitation: the inability to reproduce undulations with frequencies that match the observed behaviour of the crawling worm. To address this question, we have limited our integrated model to a short section of the worm, rather than modelling the entire body. We find that the addition of a mechanical framework to the neural control model of Ref. [12] leads to robust oscillations, with significantly smoother waveforms and reduced oscillation frequencies, matching observations of the worm.

2 2.1

Background C . elegans Locomotion

Forwards locomotion is achieved by propagating sinusoidal undulations along the body from head to tail. When moving on a firm substrate (e.g. agarose) the worm lies on its side, with the ventral and dorsal muscles at any longitudinal level contracting in anti-phase. With the exception of the head and neck, the worm is only capable of bending in the dorso-ventral plane. Like all nematode worms, C. elegans lacks any form of rigid skeleton. Its roughly cylindrical body has a diameter of ∼ 80 μm and a length of ∼ 1 mm. It has an elastic cuticle containing (along with its intestine and gonad) pressurised fluid, which maintains the body shape while remaining flexible. This structure is referred to as a hydrostatic skeleton. The body wall muscles responsible for locomotion are anchored to the inside of the cuticle. 2.2

The Neural Model

The neural model used here is based on the work of Bryden and Cohen [11,12,13]. Specifically, we use the model (equations and parameters) presented in [12] which is itself an extension of Refs. [11,13]. The model simplifies the neuronal wiring diagram of the worm [2,14] into a minimal neural circuit for forward locomotion. This reduced model contains a set of repeating units, (one “tail” and ten “body” units) where each unit consists of one dorsal motor neuron (of class DB) and one ventral motor neuron (of class VB). A single command interneuron (representing a pair of interneurons of class AVB in the biological worm) provides the “on” signal to the forward locomotion circuit and is electrically coupled (via gap junctions) to all motor neurons of classes DB and VB. In the model, motor neurons also have sensory function, integrating inputs from stretch-receptors, or mechano-sensitive ion channels, that encode each unit’s bending angle. Motor neurons receive both local and – with the exception of the tail – proximate sensory input, with proximate input received from the adjacent posterior unit.

An Integrated Neuro-mechanical Model of C. elegans Forward Locomotion

39

Fig. 1. A: Schematic diagram of the physical model illustrating nomenclature (see Appendix B for details). B: The neural model, with only two units (one body, one tail). AVB is electrically coupled to each of the motor neurons via gap junctions (resistor symbols).

The sensory-motor loop for each unit gives rise to local oscillations which phase lock with adjacent units. Equations and parameters for the neural model are set out in Appendix A. This neural-only model uses a minimal physical framework to translate neuronal output to bending. Fig. 1B shows the neural model with only two units (a tail and one body unit), as modelled in this paper. In the following section, we outline a more realistic physical model of the body of the worm.

3

Physical Model

Our physical model is an adaptation of Ref. [8], a 2-D model consisting of two rows of N points (representing the dorsal and ventral sides of the worm). Each point is acted on by the opposing forces of the elastic cuticle and pressure, as well as muscle force and drag (often loosely referred to as friction or surface friction [8]). We modify this model by introducing simplifications to reduce simulation time, in part by allowing us to use a longer time step. Fig. 1A illustrates the model’s structure. The worm is represented by a number rigid beams, connected to each of the adjacent beams by four springs. Two horizontal (h) springs connect points on the same side of adjacent beams and resist both elongation and compression. Two diagonal (d) springs connect the dorsal side of the ith beam to the ventral side of the i + 1st , and vice versa. These springs strongly resist compression and have an effect analogous to that of pressure, in that they help to maintain reasonably constant area in each unit. The model was implemented in C++, using a 4th order Runge-Kutta method for numerical integration, with a time step of 0.1 ms.1 Equations and parameters 1

The original model [8] required a time step of 0.001 ms with the same integration method.

40

J.H. Boyle, J. Bryden, and N. Cohen

of the physical model are given in Appendix B. The steps taken to interface the physical and neuronal models are described in Appendix C.

4

Results

Using our integrated model we first simulated a single unit (the tail), and then implemented two phase-lagged units (adding a body unit). In what follows, we present these results, as compared to those of the neural model alone. 4.1

Single Oscillating Segment

The neural model alone produces robust oscillations in unit bending angle (θi ) with a roughly square waveform, as shown in Fig. 2A. The model unit oscillates at about 3.5 Hz, as compared to frequencies of about 0.5 Hz observed for C. elegans forward locomotion on an agarose substrate. It has not been possible to find parameters within reasonable electrophysiological bounds for the neural model that would slow the oscillations to the desired time scales [12]. Oscillations of the integrated neuro-mechanical model of a single unit are shown in Fig. 2B. All but four parameters of the neuronal model remain unchanged from Ref. [12]. However, parameters used for the actuation step caused a slight asymmetry in the oscillations when integrated with a physical model, and were therefore modified. As can be seen from the traces in the figure, the frequency of oscillation in the integrated model is about 0.5 Hz for typical agarose drag [8], and the waveform has a smooth, almost sinusoidal shape. Faster (and slower) oscillations are possible for lower (higher) values of drag. Fig. 2C shows a plot of oscillation frequencies as a function of drag for the integrated model.

Fig. 2. Oscillations of, A, the original neural model [12] and, B, the integrated model (with drag of 80 × 10−6 kg.s−1 ). Note the different time scales. C: Oscillation frequency as a function of drag. The zero frequency point indicates that the unit can no longer oscillate.

An Integrated Neuro-mechanical Model of C. elegans Forward Locomotion

4.2

41

Two Phase-Lagged Segments

Parameters of the neural model are given in Table A-1 for the tail unit and in Table A-2 for the body unit. Fig. 3 compares bending waveforms recorded from a living worm (Fig. 3A), simulated by the neural model (Fig. 3B) and simulated by the integrated model (Fig. 3C).

Fig. 3. Phase lagged oscillation of two units. A: Bending angles extracted from a recording of a forward locomoting worm on an agarose substrate. The traces are of two 1 points along the worm (near the middle and 12 of a body length apart). B: Simulation of two coupled units in the neural model. C: Simulation of the integrated model. Take note of the faster oscillations in subplot B.

5

Discussion

C. elegans is amenable to manipulations at the genetic, molecular and neuronal levels but with such rich behaviour being produced by a system with so few components, it can often be difficult to determine the pathways of cause and effect. Mathematical and simulation models of the locomotion therefore provide an essential contribution to the understanding of C. elegans neurobiology and motor control. The inclusion of a realistic embodiment is particularly relevant to a model of C. elegans locomotion. Sensory feedback is important to the locomotion of all animals. However, in C. elegans, the postulated existence of stretch receptor inputs along the body (unpublished communication, L. Eberly and R. Russel, reported in [2]) would provide direct information about body posture to the motor neurons themselves. Thus, the neural control is likely to be tightly coupled to the shape the worm takes as it locomotes. Modelling the body physics is therefore particularly important in this organism. Here we have presented the first steps in the implementation of such an integrated model, using biologically plausible parameters for both the neural and mechanical components. One interesting effect is the smoothing of the waveform from a square-like waveform in the isolated neural model to a nearly sinusoidal waveform in the integrated model. The smoothing can be attributed to the body’s resistance to bending (modelled as a set of springs), which increases with the bending angle.

42

J.H. Boyle, J. Bryden, and N. Cohen

By contrast, in the original neural model, the rate of bending depends only on the neural output. The work presented here would naturally lead to an integrated neuromechanical model of locomotion for an entire worm. The next step toward this goal, extending the neural circuit to the entire ventral cord (and the corresponding motor system) is currently underway. The physical model introduces long range interactions between units via the body and environment. In a real worm, as in the physical model, for bending to occur at some point along the worm, local muscles must contract. However, such contractions also apply physical forces to adjacent units, and so on up and down the worm, giving rise to a significant persistence length. For this reason the extension of the neuro-mechanical model from two to three (or more) units will not be automatic and will require parameter changes to model an operable balance between the effects of the muscle and body properties. In fact, the worm’s physical properties (and, in particular, the existence of long range physical interactions along it) could set new constraints on the neural model, or could even be exploited by the worm to achieve more effective locomotion. Either way, the physics of the worm’s locomotion is likely to offer important insights that could not be gleaned from a model of the isolated neural subcircuit. We have shown that a neural model developed with only the most rudimentary physical framework can continue to function with a more realistic embodiment. Indeed, both the waveform and frequency have been improved beyond what was possible for the isolated neural model. We conclude that the body and environment are likely to be important components of the subsystem that generates locomotion in the worm.

Acknowledgement This work was funded by the EPSRC, grant EP/C011961. NC was funded by the EPSRC, grant EP/C011953. Thanks to Stefano Berri for movies of worms and behavioural data.

References 1. C. elegans Sequencing Consortium: Genome sequence of the nematode C. elegans: A platform for investigating biology. Science 282, 2012–2018 (1998) 2. White, J.G., Southgate, E., Thomson, J.N., Brenner, S.: The structure of the nervous system of the nematode Caenorhabditis elegans. Philosophical Transactions of the Royal Society of London, Series B 314, 1–340 (1986) 3. Ferr´ee, T.C., Marcotte, B.A., Lockery, S.R.: Neural network models of chemotaxis in the nematode Caenorhabditis elegans. Advances in Neural Information Processing Systems 9, 55–61 (1997) 4. Ferr´ee, T.C., Lockery, S.R.: Chemotaxis control by linear recurrent networks. Journal of Computational Neuroscience: Trends in Research, 373–377 (1998) 5. Wicks, S.R., Roehrig, C.J., Rankin, C.H.: A Dynamic Network Simulation of the Nematode Tap Withdrawal Circuit: Predictions Concerning Synaptic Function Using Behavioral Criteria. Journal of Neuroscience 16, 4017–4031 (1996)

An Integrated Neuro-mechanical Model of C. elegans Forward Locomotion

43

6. Tsalik, E.L., Hobert, O.: Functional mapping of neurons that control locomotory behavior in Caenorhabditis elegans. Journal of Neurobiology 56, 178–197 (2003) 7. Sakata, K., Shingai, R.: Neural network model to generate head swing in locomotion of Caenorhabditis elegans. Network: Computation in Neural Systems 15, 199–216 (2004) 8. Niebur, E., Erd¨ os, P.: Theory of the locomotion of nematodes. Biophysical Journal 60, 1132–1146 (1991) 9. Niebur, E., Erd¨ os, P.: Theory of the Locomotion of Nematodes: Control of the Somatic Motor Neurons by Interneurons. Mathematical Biosciences 118, 51–82 (1993) 10. Niebur, E., Erd¨ os, P.: Modeling Locomotion and Its Neural Control in Nematodes. Comments on Theoretical Biology 3(2), 109–139 (1993) 11. Bryden, J.A., Cohen, N.: A simulation model of the locomotion controllers for the nematode Caenorhabditis elegans. In: Schaal, S., Ijspeert, A.J., Billard, A., Vijayakumar, S., Hallam, J., Meyer, J.A. (eds.) Proceedings of the eighth international conference on the simulation of adaptive behavior, pp. 183–192. MIT Press / Bradford Books (2004) 12. Bryden, J.A., Cohen, N.: Neural control of C. elegans forward locomotion: The role of sensory feedback (Submitted 2007) 13. Bryden, J.A.: A simulation model of the locomotion controllers for the nematode Caenorhabditis elegans. Master’s thesis, University of Leeds (2003) 14. Chen, B.L., Hall, D.H., Chklovskii, D.B.: Wiring optimization can relate neuronal structure and function. Proceedings of the National Academy of Sciences USA 103, 4723–4728 (2006)

Appendix A: Neural Model Neurons are assumed to have graded potentials [11,12,13]. In particular, motor neurons (VB and DB) and are modelled by leaky integrators with a transmembrane potential V (t) following: C

dV = −G(V − Erev ) − I shape + I AVB , dt

(A-1)

where C is the cell’s membrane capacitance; Erev is the cell’s effective reversal potential;  and G is the total effective membrane conductance. Sensory input n stretch I shape = )Gstretch σjstretch (θj ) is the stretch receptor input j j=1 (V − Ej from the shape of the body, where Ejstretch is the reversal potential of the ion channels, θj is the bending angle of unit j and σjstretch is a sigmoid response function of the stretch receptors to the local bending. The stretch receptor activation function is given by σ stretch (θ) = 1/ [1 + exp (−(θ − θ0 )/δθ)] where the steepness parameter δθ and the threshold θ0 are constants. The command input current I AVB = GAVB (VAVB − V ) models gap junctional coupling with AVB (with coupling strength GAVB and denoting AVB voltage by VAVB ). Note that in the model, AVB is assumed to have a sufficiently high capacitance, so that the gap junctional currents have a negligible effect on its membrane potential.

44

J.H. Boyle, J. Bryden, and N. Cohen

Segment bending in this model is given as a summation of an output function from each of the two neurons: dθ out = σVout B (V ) − σDB (V ) , dt

(A-2)

where σ out (V ) = ωmax /[1 + exp (−(V − V0 )/δV )] with constants ωmax , δV and V0 . Note that dorsal and ventral muscles contribute to bending in opposite directions (with θ and -θ denoting ventral and dorsal bending, respectively). Table A-1. Parameters for a self-oscillating tail unit (as in Ref. [12]) Parameter Value Parameter Value Parameter Value Erev −60mV VAVB −30.7mV C 5pF GVB 19.07pS GDB 17.58pS GAVB 35.37pS VB GAVB 13.78pS Gstretch 98.55pS Gstretch 67.55pS DB VB DB Estretch 60mV θ0,VB −18.68o θ0,DB −19.46o δθVB 0.1373o δθDB 0.4186o ωmax,VB 6987o /sec ωmax,DB 9951o /sec V0,VB 22.8mV V0,DB 25.0mV δVVB 0.2888mV/sec δVDB 0.0826mV/sec

Table A-2. Parameters for body units and tail-body interactions as in Ref. [12]. All body-unit parameters that are not included here are the same as for the tail unit. Parameter GVB  Gstretch DB  θ0,DB

Value Parameter Value Parameter Value  26.09pS GDB 25.76pS Gstretch 16.77pS VB   18.24pS E stretch 60mV θ0,VB −19.14o   −13.26o δθVB 1.589o /sec δθDB 1.413o /sec

Appendix B: Physical Model The physical model consists of N rigid beams which form the boundaries between the N − 1 units. The ith beam can be described in one of two ways: either by the (x, y) coordinates of the centre of mass (CoMi in Fig. 1) and angle φi , or by the (x, y) coordinates of its two end points (PiD and PiV in Fig. 1). Each formulation has its own advantages and is used where appropriate. B.1 Spring Forces The rigid beams are connected to each of their neighbours by two horizontal (h) springs and two diagonal (d) springs, directed along the vectors k k Δh k,i = Pi+1 − Pi for k = D, V k Δdm,i = Pi+1 − Pil for k = D, V , l = V, D and m = 1, 2 ,

(B-1)

An Integrated Neuro-mechanical Model of C. elegans Forward Locomotion

45

for i = 1 : N − 1, where Pik = (xki , yik ) are the coordinates of the ends of the ith beam. The spring forces F(s) depend on the length of these vectors, Δjk,i = |Δjk,i | and are collinear to them. The magnitude of the horizontal and diagonal spring forces are piecewise linear functions ⎧ h κ (Δ − Lh2 ) + κhS1 (Lh2 − Lh0 ) : Δ > Lh2 ⎪ ⎪ ⎨ S2 κhS1 (Δ − Lh0 ) : Lh2 > Δ > Lh0 h F(s) (Δ) = , (B-2) h h κ (Δ − L1 ) + κhC1 (Lh1 − Lh0 ) : Δ < Lh1 ⎪ ⎪ ⎩ C2 h h κC1 (Δ − L0 ) : otherwise ⎧ d ⎨ κC2 (Δ − Ld1 ) + κdC1 (Ld1 − Ld0 ) : Δ < Ld1 d F(s) (Δ) = κdC1 (Δ − Ld0 ) : Ld1 < Δ < Ld0 , (B-3) ⎩ 0 : otherwise where spring (κ) and length (L) constants are given in Table B-1. Table B-1. Parameters of the physical model. Note that values for θ0 and θ0 differ from Ref. [12] and Table A-1. Parameter Value Parameter Value Parameter Value D 80μm Lh0 50μm Lh1 0.5Lh0  h h d d h2 2 L2 1.5L1 L0 L0 + D L1 0.95Ld0 h −1 h h h κS1 20μN.m κS2 10κS1 κC1 0.5κhS1 h h d h d κC2 10κC1 κC1 50κS1 κC2 10κdC1 fmuscle 0.005Lh0 κhC1 c = c⊥ 80 × 10−6 kg.s−1 θ0,VB −29.68o   θ0,DB −8.46o θ0,VB −22.14o θ0,DB −10.26o

B.2

Muscle Forces

Muscle forces F(m) are directed along the horizontal vectors Δh k,i with magnitude F(m)k,i = fmuscle Ak,i for k = D, V and i = 1 : N − 1 ,

(B-4)

where fmuscle is a constant (see Table B-1) and Ak,i are scalar activation functions for the dorsal and ventral muscles, determined by  (θi (t), 0) if θi (t) ≥ 0 (AD,i , AV,i ) = (B-5) (0, −θi (t)) if θi (t) < 0 , where θi (t) = B.3

t

dθi 0 dt dt

is the integral over the output of the neural model.

Total Point Force

With the exception of points on the outer beams, each point i is subject to forces F D,i and F V,i , given by differences of the spring and muscle forces from the corresponding units (i and i − 1):

46

J.H. Boyle, J. Bryden, and N. Cohen

F D,i = (F h(s)D,i − F h(s)D,i−1 ) + (F d(s)1,i − F d(s)2,i−1 ) + (F (m)D,i − F (m)D,i−1 ) F V,i = (F h(s)V,i − F h(s)V,i−1 ) + (F d(s)2,i − F d(s)1,i−1 ) + (F (m)V,i − F (m)V,i−1 ) . (B-6) Since the first beam has no anterior body parts, and the last beam has no posterior body parts, all terms with i = 0 or i = N are taken as zero. B.4

Equations of Motion

Motion of the beams is calculated from the total force acting on each of the 2N points. Since the points PiD and PiV are connected by a rigid beam, it is convenient to convert F(t)k,i to a force and a torque acting on the beam’s centre of mass. y x Rotation by φi converts the coordinate system of F(t)k,i = (F(t)k,i , F(t)k,i ) 

 ⊥ to a new system F(t)k,i = (F(t)k,i , F(t)k,i ) with axes perpendicular to (⊥) and parallel with () the beam: y ⊥ x F(t)k,i = F(t)k,i cos(φi ) + F(t)k,i sin(φi ) 

y x F(t)k,i = F(t)k,i cos(φi ) − F(t)k,i sin(φi ) .

(B-7)

The parallel components are summed and applied to CoMi , resulting in pure translation. The perpendicular components are separated into odd and even parts (giving rise to a torque and force respectively) by Fi⊥,even = Fi⊥,odd =

⊥ ⊥ (F(t)D,i + F(t)V,i )

2 ⊥ ⊥ (F(t)D,i − F(t)V,i ) 2

.

(B-8)

As in Ref. [8] we disregard inertia, but include Stokes’ drag. Also following Ref. [8], we allow for different constants for drag in the parallel and perpendicular directions, given by c and c⊥ respectively. The motion of CoMi is therefore 

1   (F + F(t)V,i ) c (t)D,i 1 = (2Fi⊥,even) c⊥ 1 = (2Fi⊥,odd ) , rc⊥

V(CoM),i = ⊥ V(CoM),i

ω(CoM),i

(B-9) 

where r = 0.5D is the radius of the worm. Finally we convert V(CoM),i and ⊥ V(CoM),i back to (x, y) coordinates with 

x ⊥ V(CoM),i = V(CoM),i cos(φi ) − V(CoM),i sin(φi ) 

y ⊥ V(CoM),i = V(CoM),i cos(φi ) + V(CoM),i sin(φi ) .

(B-10)

An Integrated Neuro-mechanical Model of C. elegans Forward Locomotion

47

Appendix C: Integrating the Neural and Physical Model In the neural model, the output dθi (t)/dt specifies the bending angles θi (t) for each unit. In the integrated model, θi (t) are taken as the input to the muscles. Muscle ouputs (or contraction) are given by unit lengths. The bending angle αi is then estimated from the dorsal and ventral unit lengths by αi = 36.2

h |Δh D,i | − |ΔV,i |

Lh0

,

(C-1)

where Lh0 is the resting unit length. (For simplicity, we have denoted the bending angles of both the neural and integrated models by θ in the Figures).

Applying the String Method to Extract Bursting Information from Microelectrode Recordings in Subthalamic Nucleus and Substantia Nigra Pei-Kuang Chao1, Hsiao-Lung Chan1,4, Tony Wu2,4, Ming-An Lin1, and Shih-Tseng Lee3,4 1

Department of Electrical Engineering, Chang Gung University 2 Department of Neurology, Chang Gung Memorial Hospital 3 Department of Neurosurgery, Chang Gung Memorial Hospital 4 Center of Medical Augmented Virtual Reality, Chang Gung Memorial Hosipital 259 Wen-Hua First Road, Gui-Shan, 333, Taoyuan, Taiwan [email protected]

Abstract. This paper proposes that bursting characteristics can be effective parameters in classifying and identifying neural activities from subthalamic nucleus (STN) and substantia nigra (SNr). The string method was performed to quantify bursting patterns in microelectrode recordings into indexes. Interspike-interval (ISI) was used as one of the independent variables to examine effectiveness and consistency of the method. The results show consistent findings about bursting patterns in STN and SNr data across all ISI constraints. Neurons in STN tend to release a larger number of bursts with fewer spikes in the bursts. Neurons in SNr produce a smaller number of bursts with more spikes in the bursts. According to our statistical evaluation, 50 and 80 ms are suggested as the optimal ISI constraint to classify STN and SNr’s bursting patterns by the string method. Keywords: Subthalamic nucleus, substantia nigra, inter-spike-interval, burst, microelectrode.

1 Introduction Subthalamic nucleus (STN) is frequently the target to study and to treat Parkinson’s disease [1, 2]. Placing a microelectrode to record neural activities in deep brain nuclei provides useful information for localization during deep brain stimulation (DBS) neurosurgery. DBS has been approved by FDA since 1998[3]. The surgery implants a stimulator to deep brain nuclei, usually STN, to alleviate Parkinson’s symptoms, such as tremor and rigidity. To search for STN in operation, a microelectrode probe is often used to acquire neural signals from outer areas to the specific target. With assistance of imagery techniques, microelectrode signals from different depth are read and recorded. Then, an important step to determine STN location is to distinguish signals of STN from its nearby areas, e.g. subtantia nigra (SNr) (which is a little ventral and medial to STN). Therefore, characterizing and quantifying firing patterns of STN and M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 48–53, 2008. © Springer-Verlag Berlin Heidelberg 2008

Applying the String Method to Extract Bursting Information

49

SNr are essential. Firing rate defined as the number of neural spikes within a period is the most common variable used for describing neural activities. However, STN and SNr have a broad range of firing rate and mostly overlapped [4] (although SNr has a slightly higher mean firing rate than STN). This makes it difficult to depend on firing rate to target STN. Bursting patterns may provide a better solution to separate signals from different nuclei. Bursting, defined as clusters of high-frequency spikes released by a neuron, is believed storing important neural information. To establish long-term responses, central synapses usually require groups of action potentials (bursts) [5,6]. Exploring bursting information in neural activities has recently become fundamental in Parkinson’s studies [1,2]. Also, that spike arrays are more regular in SNr than in STN signals is observed [7]. However, the regularity of firing or grouping of spikes in STN and SNr potentials has not been investigated thoroughly. This study aims to extract bursting information from STN and SNr. A quantifying method for bursting, the string method, will be applied. The string method quantifies bursting information relying on inter-spike-interval (ISI) and spike number [8]. Although some other methods for quantifying bursts exist [9,10], the string method is the one which can provide information about what spike members contributing to the detected bursts. In addition, because various ISIs have been used in research [8,9] to define bursts, this study will also evaluate the effect of ISI constraints on discriminating STN and SNr signals.

2 Method The neuronal data used in this study were acquired during DBS neurosurgery in Chang Gung Memorial Hospital. With assistance of imagery localization systems [11], trials (10s for each) of microelectrode recordings were collected at a sampling

a

b

Fig. 1. MRI images from one patient: a. In the sagittal plane – the elevation angle of the probe (yellow line) from the inter-commissural line (green line) was around 50 to 75°; b. In the frontal plane – the angle of the probe (yellow lines) from the midline (green line) was about 8 to 18° to right or left.

50

P.-K. Chao et al.

rate of 24,000 Hz. Based on several observations, e.g. magnetic resonance imaging (MRI), computed topography (CT), motion/perception-related responses and probe location according to a stereotactic system, experienced neurologists diagnosed 18 trials as neural signals from STN, and the other 23 trials as from SNr. The trials which were collected outside STN and SNr and/or confused between STN and SNr were excluded. In this paper, the data are from 3 Parkinson’s patients (2 females, 1 male, age=73.3±8.3 y/o) who received DBS treatment. Due to the patients’ individual difference, e.g. head size, the depth of STN from the scalp was found varied between 15 and 20 cm. During surgery, the elevation angle of the probe from the intercommissural line was around 50 to 75° (Fig 1a) and the angle between the probe and the midline was about 8 to 18° toward either right or left (Fig 1b). 2.1 Spike Detection Each trial of microelectrode recordings includes 2 types of signals, spikes and background signals. The background signals are interference from nearby neural areas or environment. Because background signals can be interpreted as a Gaussian distribution, signals which are 3 standard deviations (SD) above or below mean can be treated as non-background signals or spikes. Therefore, a threshold in the level of mean plus 3 SD is applied in this study to detect spikes (Fig 2).

100 90 80

amplitude

70 60 50 40 30 20 10 5.175

5.18

5.185

5.19 5.195 time (s)

5.2

5.205

5.21

Fig. 2. A segment of microelectrode recording – the red horizontal line is threshold; the green stars indicate the found spikes; most signals around the baseline are background signals

Applying the String Method to Extract Bursting Information

51

2.2 The String Method Every detected spike was plotted as a circle in a spike sequential number versus spike occurring time figure (Fig 3). The spike sequential number is starting at 1 for the first spike in a trial. The spikes which are closer to each other are labeled as strings [8] and defined as bursts. Two parameters were controlled and manipulated to determine bursts: (1) the minimum number of spikes to form a burst was 5; (2) the maximum ISI of the adjacent spikes in a burst was set as 20ms, 50ms, 80ms, and 110ms separately to find an optimal condition to distinguish STN and SNr bursting patterns.

spike sequential number

180

160

140

120

100

80 5

5.5

6

6.5

7

time (s)

Fig. 3. A segment of strings plot – each blue circle means a spike; the red triangles mean the starting spikes of a burst; the black triangles mean the ending spikes of a burst

2.3 Dependent Variables Three dependent variables were computed: (1) Firing rate (FR) was calculated as total spike number divided by trial duration (10 s). (2) Number of bursts (NB) was determined by the string method as the total burst number in a trial. (3) The average of spike number in each burst (SB) was also counted in every trial. 2.4 Statistical Analysis Independent sample’s t-test was applied to test firing rate difference between STN and SNr signals. MANOVA was performed to evaluate NB and SB separately among different ISI constraints (α=.05).

52

P.-K. Chao et al.

3 Results The signals from STN and SNr showed similar firing rate but different bursting patterns. There is no significant difference between STN and SNr in firing rate (STN: 57.0±22.1; SNr: 68.8±23.5) (p>.05). The results of NB and SB are listed in Table 1 and Table 2. In NB, SNr has significantly fewer bursts than STN while ISI setting is 50ms and 80 ms (p<.05). In SB, SNr has significant more spikes in bursts than STN while ISI setting is 20ms, 50ms and 80 ms (p<.05). Based on the findings, 2 points can be addressed: First, STN and SNr have different bursting patterns, although their total numbers of spikes (firing rate) are similar. Comparing to STN, SNr releases fewer bursts but each burst contains more spikes. Second, setting ISI constraints around 50~80 ms in the string method can be effective to distinguish the difference between STN and SNr signals. Table 1. NB results in SNr and STN ISI constraint

SNr

STN

p

20ms

32.1±15.7

38.2±23.1

>.05

50ms

8.9±6.8

23.9±7.7

<.05*

80ms

5.2±5.8

10.7±7.5

<.05*

110ms

3.1±3.4

4.9±4.4

>.05

Table 2. SB results in SNr and STN ISI constraint

SNr

STN

p

20ms

21.6±27.8

8.5±2.2

<.05*

50ms

202.5±309.9

28.1±24.1

<.05*

80ms

396.0±373.9

124.2±141.3

<.05*

110ms

515.1±367.6

337.7±342.8

>.05

4 Discussion and Conclusion Microelectrode recordings from STN and SNr show valuable information in bursting patterns which may be useful to assist neurosurgery in the future. From the results, STN and SNr show very different patterns in bursting. Neurons in STN tend to release more “small” bursts which contain fewer spikes. Neurons in SNr tend to produce “giant” bursts which contain a big number of spikes. These bursting characteristics are quantified into NB and SB which may assist in making decision about localizing stimulation probes during DBS operations. Also, the simplicity of the string method can offer quick information and be efficient in real-time analysis.

Applying the String Method to Extract Bursting Information

53

Different bursting characterisitcs in STN and SNr are revealed across all ISI constraints (20, 50, 80, 110 ms) in both variables, although statistical significance only shows in 50 and 80ms. Because there is no “gold standard” to determine ISIs of adjacent spikes in bursts, several ISI constraint settings were tested in this study. No matter in which setting, STN has a larger NB and smaller SB than what SNr has. Statistical significance only shows in both dependent variables in 50 and 80 ms settings. Therefore, we suggest the optimal ISI setting for identifying bursts in STN and SNr signals should be around 50 and 80 ms. For further studies, signals from other deep brain nuclei may be analyzed to enrich the application of bursting information. Also, since the bioelectrical signals from deep brain nuclei are non-stationary, non-linear methods, e.g. complexity, would be performed and compared with current results to provide more nucleus-identifying clues in the future. Acknowledgments. The authors would like to express sincere appreciation to the grant support from the Ministry of Economic Affairs in Taiwan, under contract 95-EC-17-A-19-S1-035.

References 1. Baufreton, J., Zhu, Z.-T., Garret, M., Bioulac, B., Johnson, S.W., Taupignon, A.I.: Dopamine Receptors Set the Pattern of Activity Generated in Subthalamic Neurons. FASEB J. 19, 1771–1777 (2005) 2. Magarinos-Ascone, C.M., Figueiras-Mendez, R., Riva-Meana, C., Cordoba-Fernadez, A.: Subthalamic Neuron Activity Related to Tremor and Movement in Parkinson’s Disease. Eur. J. Neurosci. 12, 2597–2607 (2000) 3. Deushl, G., Volkmann, J., Krack, P.: Deep Brain Stimulation for Movement Disorders. Mov. Disord. 17, S1-S1 (2002) 4. Sterio, D., Zonenshayn, M., Mogilner, A.Y., Rezai, A.R., Kiprovski, K., Kelly, P.J., Beric, A.: Neurophysiological Refinement of Subthalamic Nucleus Targeting. Neurosurg 50, 58–69 (2002) 5. Zucker, R.S.: Frequency Dependent Changes in Excitatory Synaptic Efficacy. In: Dichter, M.A. (ed.) Mechanisms of Epiletogenesis, pp. 153–157. Plenum Press, New York (1988) 6. Lisman, L.E.: Bursts as a Unit of Neural Information: Making Unreliable Synapses Reliable. TINS 20, 38–43 (1997) 7. Benazzouz, A., Breit, S., Koudsie, A., Pollak, P., Krack, P., Benabid, A.L.: Intraoperative Microrecordings of the Subthalamic Nucleus in Parkinson’s Disease. Mov. Disord. 17, S145S149 (2002) 8. Turnbull, L., Dian, E., Gross, G.: The String Method of Burst Identification in Neuronal Spike Trains. J. Neurosci. Methods 145, 23–35 (2005) 9. Mulloney, B.: A Method to Measure the Strength of Multi-unit Bursts of Action Potentials. J. Neurosci. Methods 146, 98–105 (2005) 10. Favre, J., Taha, J.M., Baumann, T., Burchiel, K.J.: Computer Analysis of the Tonic, Phasic, and Kinesthetic Activity of Pallidal Discharges in Parkinson Patients. Surg. Neurol. 51, 665– 673 (1999) 11. Lee, J.-D., Huang, C.-H., Lee, S.-T.: Improving Stereotactic Surgery Using 3-D Reconstruction. IEEE Eng. Med. Biol. Mag. 21, 109–116 (2002)

Population Coding of Song Element Sequence in the Songbird Brain Nucleus HVC Jun Nishikawa1 , Masato Okada1,2 , and Kazuo Okanoya1 1

2

RIKEN Brain Science Institute, 2-1 Hirosawa, Wako, Saitama 351-0198, Japan Graduate School of Frontier Sciences, The University of Tokyo, 5-1-5 Kashiwanoha, Kashiwa, Chiba 277-8561, Japan

Abstract. Birdsong is a complex vocalization composed of various song elements organized according to sequential rules. To reveal the neural representation of song element sequence, we recorded the neural responses to all possible element pairs of stimuli in the Bengalese finch brain nucleus HVC. Our results show that each neuron has broad but differential response properties to element sequences. We calculated the time course of population activity vectors and mutual information between auditory stimuli and neural activities. The clusters of population vectors responding to second elements had a large overlap, whereas the clusters responding to first elements were clearly divided. At the same timing, confounded information also significantly increased. These results indicate that the song element sequence is encoded in a neural ensemble in HVC via population coding.

1

Introduction

Songbirds have a complex learned vocalization composed of various song elements with a typical sequential rule. In Bengalese finches, these rules follow individually distinctive finite state syntax [1]. Songbirds have been intensively studied as a model for the syntactical properties of human language [2]. It is important to reveal the neural representation of complex song element sequences in the songbird brain. Based on the finding of sequential selective neurons [3,4], it has been thought that the song element sequence is encoded in a chain of the rigid selective neurons [5]. Alternatively, it can be encoded in a neural ensemble of relatively broadly selective neurons in a distributed manner [6,7]. We attempted to determine which neural representation actually occurs in the songbird brain. Songbirds have a specialized brain area for generating and learning complex vocalizations, and the area is called as a song system. From the importance of auditory feedback in song learning, numerous studies have investigated auditory neural representation in the song system [3,4]. Especially, HVC is one of the major sensory-motor integration sites in the song system; these neurons selectively respond to the bird’s own song (BOS) in a time-locked manner. Margoliash and Fortune found the neurons selectively responded to only typical element pair stimuli included in their own song [3,4]. This type of neuron was named the temporal combination selective neuron (TCS neuron). Since the discovery of M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 54–63, 2008. c Springer-Verlag Berlin Heidelberg 2008 

Population Coding of Song Element Sequence

55

TCS neurons, it has been thought that the song element sequence is encoded in a series of different types of TCS neurons. However, TCS neurons were found only in a small portion of the recorded data. Many song-selective neurons lack TCS properties. In addition, the stimuli used in these experiments were only partial presentations of the entire sequence, such as EE, EF, FE, or FF within ABCDEFGHI. This design did not test for responses to other element pairs, such as AB, GC, or any other combination. To more fully understand the neural representation of song element sequences, we must evaluate activity in response to all possible song element pairs within ABCDEFGHI. In this study, we recorded single-unit activities of HVC neurons driven by all possible song element pair stimuli in anesthetized Bengalese finches. Then, we used sequential response distribution analysis, population dynamics analysis, and information-theoretic analysis to show that the song element sequence is encoded within a neural ensemble in HVC neurons by population coding. These findings led us postulate an alternative scheme for encoding the song element sequence in the songbird HVC, with distributed neural representation rather than the chain model of rigid selective TCS neurons.

2 2.1

Material and Methods Animals

Twenty-three adult Bengalese finches (> 180 days post-hatch) were used in this study. All experimental procedures were performed according to established animal care protocols approved by the animal care and use committee at RIKEN. 2.2

Stimuli

Undirected songs were recorded in a quiet soundproof box using a microphone and amplifier connected to a computer with a sampling rate of 44.1 kHz and 16-bit resolution. We calculated sonograms from the recorded song using sound analysis software (SASLab Pro; Avisoft, Berlin, Germany). A birdsong consists of a series of discrete song elements with silent intervals among them. Song elements were divided into distinct types by visual inspection of the spectrotemporal structure of each sonogram. The transition matrix, representing the transition probability between each song element, was then calculated. We can evaluate the syntactical structure of song in individual Bengalese finches using this transition matrix. We prepared five different types of sound stimuli: BOS, REV, OREV, element, and element pair. BOS is the forward playback of the bird’s own song, while REV is the reversed playback of the same song. OREV is a modified version of the song, in which the spectro-temporal composition of each song element is retained, but the order of the song elements has been reversed. Element stimuli are isolated playbacks of each song element. For example, if the song has nine elements, element stimuli are A, B, C, and so on. Element pair stimuli are combinations of all possible element pairs. In this case, we prepared 81 stimuli, including AA, AB, · · ·, IH, and II.

56

2.3

J. Nishikawa, M. Okada, and K. Okanoya

Recording Procedure

Before electrophysiological recording sessions, birds were anesthetized with 4 to 7 doses of 10% urethane (40 μl per dose) at 20-min intervals. The birds were restrained in a custom-made chamber on a stereotaxic device (David Kopf Instruments, Tujunga, CA, USA). The birds were fixed with ear-bars and a beak-holder that positioned the beak tip at an angle of 45 degrees below the horizontal plane. The head was treated with Xylocaine gel and the feathers and skin were removed. A custom-made three-point fixation device (Narishige, Tokyo, Japan) was attached to the rostral part of the skull surface with dental cement. Small holes were made in the skull just above the HVC. Finally, the dura was removed, and tungsten electrodes were set on the surface. The ear-bars were removed before making physiological recordings. The birds were located in an electromagnetically shielded sound-attenuation box while in the stereotaxic device. The electrodes were lowered into the brain using a hydraulic micropositioner (MODEL640, David Kopf Instruments), and extracellular signals from HVC were recorded. The signals from the electrodes were amplified (gain 10,000) and filtered (100 Hz-10 kHz bandpass) using an extracellular recording amplifier (ER-91, Cygnus Technology, Water Gap, PA, USA). The data were digitized at 20 kHz with 16-bit resolution using the data acquisition system (Micro1401, Cambridge Electronic Design, Cambridge, UK) and the associated software (Spike2, Cambridge Electronic Design). The data were stored in a computer disk for off-line analysis. During the neural recording session, sound stimuli were presented at a peak sound pressure of 70 dB. At first, we presented BOS, REV, OREV, and the silent stimuli. Next, each of the elements and silent stimuli were delivered. Finally, we presented element pairs and silent stimuli. Each sound stimulus was presented 20 times in a random order with an interstimulus interval of 3 to 5 s. The computer for neural recording and that for stimulus presentation were synchronized by a trigger-signal generated simultaneously with the stimuli. 2.4

Data Analysis

Analyses were performed using custom-made programs written by MATLAB (Mathworks, Natick, MA, USA). The mean spontaneous firing rate was calculated from the baseline activity registered during the silent stimulus. The response strength RS was calculated by subtracting the spontaneous rate from the firing rate R registered during stimulus presentation. R and RS were measured 2 in each 10 ms bin, from which we calculated the average R, RS, and variance σR , 2 σRS across the stimulus presentation period. To determine the selectivity of each neuron, we calculated the psychophysical measure , as previously described [8]. 2(xA − xB ) d (xA /xB ) =  2 . σxA + σx2B

(1)

In this equation, xA is the response to stimulus A, and xB is the response to stimulus B. d (xA /xB ) represents the response selectivity to stimulus A relative to stimulus B based on the mean and variation of the responses. We considered a neuron to be selective for a stimulus when the selectivity satisfies d > 1.0 [9].

Population Coding of Song Element Sequence

2.5

57

Population Dynamics Analysis

To analyze neural activity at the population level, we performed a population dynamics analysis [10]. With our experimental design, we were not able to combine the data from different individuals because their songs and elements thereof were completely different from each other. Therefore, we presented the data for one typical bird in which we could register activity from six distinct single units throughout the presentation of all stimuli. Note that the qualitative property for the obtained results was similar in the other birds. For each stimulus, we calculated a population activity vector, which is the set of instantaneous mean firing rates for each neuron in a 50 ms time window. In this case, each population activity vector had six dimensions. Since this typical bird had four song elements, we calculated 16 population activity vectors for 16 element pair stimuli, within the 50 ms time window. The time window shifted by increments of 1 ms from -200 to 600 ms (stimulus onset = 0 ms). The data were smoothed using a Gaussian filter with a variance of 10 ms. These procedures enabled us to observe the temporal aspects of the neuronal population. The multidimensional scaling method (MDS) [11] is a dimension-reduction method that rearranges data from a high-dimensional space into a lower-dimensional space, while preserving as much of the information as possible. MDS was applied to the set of population activity vectors for each time window. Finally, the population response to each stimulus was represented in two-dimensional MDS space, and the clustering of these responses was analyzed. 2.6

Information-Theoretic Analysis

To evaluate how much information is transmitted by each neuron, we calculated the mutual information between the stimulus and the neural response [12]. Mutual information was quantified as the decrease in entropy of the stimulus occurrence: I(S; R) = H(S) − H(S|R),     =− p(s) log p(s) − − p(s|r) log p(s|r) . s

s

(2)

r

In this equation, S is the set of stimuli s, and R is the set of neural responses r, i.e., spike count. p(s|r)is the conditional probability of stimulus s given an observed spike count r , and p(s) is the a priori probability of stimulus s. The brackets indicate an average of the signal distribution p(r). To examine the time course of the information, the response was evaluated using a 50 ms sliding window. The center of the window was moved in 10 ms steps, beginning 200 ms before the stimulus onset and lasting until 600 ms after the stimulus. To test the statistical significance, we estimated the mean and standard deviation of the information during the 200 ms period before the stimulus onset. If the value exceeded the mean + 3SD, we considered the information significant (P < 0.001).

58

3 3.1

J. Nishikawa, M. Okada, and K. Okanoya

Results Selective Auditory Response to BOS

We used a spike sorting procedure to classify the signals recorded from the Bengalese finch HVC, which yielded well identified single units (n = 104, 23 birds). We analyzed these data using the psychophysical measure d . In total, 86% of HVC neurons selectively responded to BOS compared to the silent condition (d (RS BOS /RS Baseline ) > 1.0, 89/104 cells). In addition, 63% of neurons were more responsive to BOS than to REV (d (RS BOS /RS REV ) > 1.0, 65/104 cells), and 28% of neurons were more responsive to BOS than to OREV (d (RS BOS /RS OREV ) > 1.0, 29/104 cells). These results are consistent with past studies [4]. The mean of d (RS BOS /RS REV ) was 1.25, and the mean of d (RS BOS /RS OREV ) was 0.63. These results indicate that BOS-selective neurons in HVC are largely variable, especially in terms of sequential response properties. A. Elements PSTH (Hz)

Raster

20 0 100 0 -0.2

0

0.5 (s)

A

B

C

A B

A C

D

PSTH (Hz)

Raster

B. Element pairs 20 0 100 0 -0.2

0 A A

B A

0.5 (s)

B

B

B

AD

C

B D

C A

C B

C C

CD

D A

D B

D C

D D

Fig. 1. An example of the auditory response to song elements (A) and element pairs (B) in a single unit from HVC

Population Coding of Song Element Sequence

59

Fig. 2. Song transition matrices of self-generated songs (first row) and sequential response distribution matrices of each single unit (second to seventh rows)

3.2

Responses to Song Element Pair Stimuli

To investigate the neural selectivity to song element sequences, we recorded neural responses to all possible element pair stimuli. Because the playback of these stimuli is extremely time-consuming, we could only maintain 34% of the recorded single units stable throughout the entire presentation (35/104 cells, 12/23 birds). In total, 70% of the stable single-units were BOS-selective (d (RS BOS /RS REV ) > 1.0, 27/35 cells, 12/12 birds). Thereafter, we focused on these data. A typical example of neural responses to each song element is shown in Fig. 1 (A). The neuron responded to a single element A or C with single phasic activity, but it did not respond to element B. It responded to element D with double phasic activity. These results indicate that the neuron has various response properties even during single element presentation. In addition, the neuron exhibited more complex response properties during the presentation of element pairs (Fig. 1(B)). The neuron responded more strongly to most of the element pairs when the second element was A or C, compared to single presentation of each element. However, the response was weaker when the first and second elements were the same. When the second element was B, no differences were observed between single and paired stimuli. When the second element was D, we measured single

60

J. Nishikawa, M. Okada, and K. Okanoya

phasic responses, and a strong response to BD. These response properties were not correlated with the element-to-element transition probabilities in the song structure. The dotted boxes indicate the sequences included in BOS. However, the neuron responded only weakly to some sequences that were included in BOS (brack arrows). In contrast, the neuron responded strongly to other sequences that were not included in BOS (white arrows). Thus, the neuron had broad response properties to song element pairs beyond the structure of self-generated song. To quantitatively evaluate sequential response properties, we calculated the response strength measure d (RS S /RS Baseline ) to the element pair stimuli S. The sequential response distributions were created for each neuron in two individuals with more than five well identified single units. Song transition matrices and sequential response distributions are shown in Fig. 2. The response distributions were not correlated with the associated song transition matrices. However, each HVC neuron in the same individual had broad but different response distribution properties. This tendency was consistent among individuals. This result indicates that the song element sequence is encoded at the population level, within broadly but differentially selective HVC neurons. 3.3

Population Dynamics Analysis

To analyze the information coding of song element sequences at the population level, we calculated the time course of population activity vectors, which is the set of instantaneous mean firing rates for each neuron in a 50 ms time window. Snapshots of population responses to stimuli are shown in eight panels of Fig. 3A (n = 6, bird 2 of Fig. 2). Each point in the panel represents the population vector toward each stimulus on the MDS space. The ellipses in the upper four panels indicate the group of vectors whose stimuli have the same first element, while the ellipses in the lower four panels indicate the group of vectors whose stimuli have the same second element. Note that the population activity vectors in the upper four panels are identical to those in the bottom four panels, and only the ellipses differ. Before the stimulus presentation ([-155 ms: -105 ms], upper and lower panels), only spontaneous activities were observed around the origin. After the first element presentation ([50 ms: 100 ms], upper panel), groups with the same first elements split. After the second element presentation ([131 ms: 181 ms] of the lower panel), groups with the same second elements were still largely overlapping. In the next section, we will show that confounded information, which represents the relation between first and second elements, increased significantly in this timing. After sufficient time ([480 ms: 530 ms], upper and lower panels), the neurons returned to spontaneous activity. The result indicates that the population response to the first and second element is drastically different. Subsequently, we will show that this overlap is derived from the information in the song element sequence. 3.4

Information-Theoretic Analysis

To determine the origin of the overlap in the population response, we calculated the time course of mutual information between the stimulus and neural activity.

Population Coding of Song Element Sequence

61

Fig. 3. Responses of HVC neurons at the population level (A) and encoded information (B)

The mutual information for first elements I(S1 ; R), the second elements I(S2 ; R), and that of element pairs I(S1 , S2 ; R) was calculated within each time window; the window was shifted to analyze the temporal dynamics of information coding (left upper 3 graphs in Fig. 3B). Narrow lines in each graph indicate the cumulative trace of mutual information in each neuron. The thick line is the cumulative trace of all neurons in the individual. The bottom-left graph in Fig. 3B shows the probability of stimulus presentation. After the presentation of the first elements, mutual information for the first elements increased, showing a statistically significant peak (P < 0.001). After the presentation of the second elements, mutual information for the second elements significantly increased (P < 0.001). At the

62

J. Nishikawa, M. Okada, and K. Okanoya

same time, mutual information for element pairs also showed a significant peak (P < 0.001). Intuitively, information for element pairs I(S1 , S2 ; R) would consist of information for the first elements I(S1 ; R) and second elements I(S2 ; R). However, the consecutive calculation of I(S1 , S2 ; R) − I(S1 ; R) − I(S2 ; R) in each time window causes a statistical peak after the presentation of element pairs (P < 0.001; forth graph from left in Fig. 3B). The difference C represents the conditional mutual information between the first and second elements for a given neural response, otherwise known as confounded information [13]. Therefore, confounded information represents the relationship between the first and second elements encoded in the neural responses. The I(S1 ; R) peak occurred at the same time that groups of population vectors with the same first elements were splitting ([50 ms: 100 ms]). The peaks for I(S2 ; R), I(S1 , S2 ; R), and C occurred during the same time that groups with the same second elements were still largely overlapping ([131 ms: 181 ms]). This indicates that the sequential information causes an overlap in the population response. In the population dynamics analysis, we cannot combine the data from different birds because each bird has a different number and types of song elements. However, in the mutual information analysis, we can combine and average the data from different birds. The five graphs on the right in Fig. 3B show the time courses for I(S1 ; R), I(S2 ; R), I(S1 , S2 ; R), C, and the stimulus presentation probability, which were calculated from all stable single units with BOS selectivity (n = 27, 12 birds). The combined mutual information for first elements was very similar to that from one bird, showing a significant peak after the presentation of the first elements (P < 0.001). Mutual information for second elements, element pairs, and confounded information also had significant peaks after the presentation of the second elements (P < 0.001). These results show that the song element sequence is encoded into a neural ensemble in HVC by population coding.

4

Conclusion

In this study, we recorded auditory responses to all possible element pair stimuli from the Bengalese finch HVC. By determining the sequential response distributions for each neuron, we showed that each neuron in HVC has broad but differential response properties to song element sequences. The population dynamics analysis revealed that population activity vectors overlap after the presentation of element pairs. Using mutual information analysis, we demonstrated that this overlap in the population response is due to confounded information, namely, the sequential information of song elements. These results indicate that the song element sequence is encoded into the HVC microcircuit at the population level. Song element sequences are encoded in a neural ensemble with broad and differentially selective neuronal populations, rather than the chain-like model of differential TCS neurons.

Population Coding of Song Element Sequence

63

Acknowledgment This study was partially supported by the RIKEN Brain Science Institute, and by a Grant-in-Aid for young scientists (B) No. 18700303 from the Japanese Ministry of Education, Culture, Sports, Science, and Technology.

References 1. Okanoya, K.: The Bengalese finch: a window on the behavioral neurobiology of birdsong syntax. Ann. N.Y. Acad. Sci. 1016, 724–735 (2004) 2. Doupe, A.J., Kuhl, P.K.: The Bengalese finch: a window on the behavioral neurobiology of birdsong syntax. Birdsong and human speech: common themes and mechanisms. Annu. Rev. Neurosci. 22, 567–631 (1999) 3. Margoliash, D., Fortune, E.S.: Temporal and harmonic combination-selective neurons in the zebra finch’ s HVc. J. Neurosci. 12, 4309–4326 (1992) 4. Lewicki, M.S., Arthur, B.J.: Hierarchical organization of auditory temporal context sensitivity. J. Neurosci. 16, 6987–6998 (1996) 5. Drew, P.J., Abbott, L.F.: Model of song selectivity and sequence generation in area HVc of the songbird. J. Neurophysiol. 89, 2697–2706 (2003) 6. Deneve, S., Latham, P.E., Pouget, A.: Reading population codes: a neural implementation of ideal observers. Nat. Neurosci. 2, 740–745 (2001) 7. Pouget, A., Dayan, P., Zemel, R.: Information processing with population codes. Nat. Rev. Neurosci. 1, 125–132 (2000) 8. Green, D., Swets, J.: Signal Detection Theory and Psychophysics. Wiley, New York (1966) 9. Theunissen, F.E., Doupe, A.J.: Temporal and spectral sensitivity of complex auditory neurons in the nucleus HVc of male zebra finches. J. Neurosci. 18, 3786–3802 (1998) 10. Matsumoto, N., Okada, M., Sugase-Miyamoto, Y., Yamane, S., Kawano, K.: Population dynamics of face-responsive neurons in the inferior temporal cortex. Cerebr. Cort. 15, 1103–1112 (2005) 11. Gower, J.C.: Some distance properties of latent root and vector methods used in multivariate analysis. Biometrika 53, 325–328 (1966) 12. Sugase, Y., Yamane, S., Ueno, S., Kawano, K.: Global and fine information coded by single neurons in the temporal visual cortex. Nature 400, 869–873 (1999) 13. Reich, D.S., Mechler, F., Victor, J.D.: Formal and attribute-specific information in primary visual cortex. J. Neurophysiol. 85, 305–318 (2001)

Spontaneous Voltage Transients in Mammalian Retinal Ganglion Cells Dissociated by Vibration Tamami Motomura, Yuki Hayashida, and Nobuki Murayama Graduate school of Science and Technology, Kumamoto University, 2-39-1 Kurokami, Kumamoto 860-8555, Japan [email protected], {yukih,murayama}@cs.kumamoto-u.ac.jp

Abstract. We recently developed a new method to dissociate neurons from mammalian retinae by utilizing low-Ca2+ tissue incubation and the vibrodissociation technique, but without use of enzyme. The retinal ganglion cell somata dissociated by this method showed spontaneous voltage transients (sVT) with the fast rise and slower decay. In this study, we analyzed characteristics of these sVT in the cells under perforated-patch whole-cell configuration, as well as in a single compartment cell model. The sVT varied in amplitude with quantal manner, and reversed in polarity around −80 mV in a normal physiological saline. The reversal potential of sVT shifted dependently on the K+ equilibrium potential, indicating the involvement of some K+ conductance. Based on the model, the conductance changes responsible for producing sVT were little dependent on the membrane potential below −50 mV. These results could suggest the presence of isolated, inhibitory presynaptic terminals attaching on the ganglion cell somata. Keywords: Neuronal computation, dissociated cells, retina, patch-clamp, neuron model.

1 Introduction Elucidating the functional role of single neurons in neural information processing is intricate because the neuronal computation itself is highly nonlinear and adaptive, and depends on combinations of many parameters, e.g. the ionic conductances, the intracellular signaling, their subcellular distributions, and the cell morphology. Furthermore, the interactions with surrounding neurons/glias can alter those factors, and thereby hinder us from examining some of those factors separately. This could be overcome by pharmacologically or physically isolating neurons from the circuits. One would use the pharmacological agents those can block the synaptic signal transmission in situ, although it is hard to know whether or not such agents show any unintended side-effects. Alternatively, one can dissociate neural tissue into single neurons by means of enzymatic digestion and mechanical trituration. The dissociated single neurons often lost their fine neurites and the synaptic contacts with other cells during the dissociation procedure, and thus, are useful for examining the properties of ionic conductances at known membrane potentials [3]. Unfortunately, however, M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 64–72, 2008. © Springer-Verlag Berlin Heidelberg 2008

sVT in Mammalian Retinal Ganglion Cells Dissociated by Vibration

65

several studies have demonstrated that proteolytic enzymes employed for the cell dissociations can distort the amplitude, kinetics, localization, and pharmacological properties of ionic currents, e.g. [2]. These observations lead to attempts to isolate neurons by enzyme-free, mechanical means. Recently, we developed a new protocol for dissociating single neurons from specific layers of mammalian retinae without use of any proteolytic enzymes [12], but with a combination of the low-Ca2+ tissue incubation [9] and the vibrodissociation technique [15] which has been applied to the slices of brains and spinal cords [1]. The somata of ganglion cells dissociated by our method showed spontaneous voltage transients (sVT) with fast rise and slower decay in the time course [8]. To our knowledge, such sVT have never been reported in previous studies on the retinal ganglion cells dissociated with or without enzyme [9]. Therefore, in this study, we analyzed characteristics of these sVT in the cells under perforated-patch whole-cell configuration, as well as in a single compartment cell model. The present results could suggest the presence of inhibitory presynaptic terminals attaching to the ganglion cell somata we recorded from, as demonstrated in previous studies on the vibrodissociated neurons of brains and spinal cords [1]. If this is the case, the retinal neurons dissociated by our method would be advantageous to investigate the mechanisms of transmitter release in single tiny synaptic boutons even under the isolation from the axons and neurites.

2 Methods All animal care and experimental procedures in this study were approved by the committee for animal researches of Kumamoto University. 2.1 Cell Dissociation The neural retinas were isolated from two freshly enucleated eyes of Wistar rats (P7-P25), cut into 2-4 pieces each, and briefly kept in chilled extracellular “bath” solution. This solution contained (in mM): 140 NaCl, 3.5 KCl, 1 MgCl2, 2.5 CaCl2, 10 D-glucose, 5 HEPES. The pH was adjusted to 7.3 with NaOH. A retinal piece was then placed with photoreceptor-side down in a culture dish, covered with 0.4 ml of chilled, low-Ca2+ solution and incubated for 3-5 min. This low-Ca2+ solution contained (in mM): 140 sucrose, 2.5 KCl, 70 CsOH, 20 NaOH, 1 NaH2PO4, 15 CaCl2, 20 EDTA, 11 D-glucose, 15 HEPES. The estimated free Ca2+ concentration was 100–200 nM. The pH was adjusted to 7.2 with HCl. After the incubation, the fireblunted glass pipette horizontally vibrating in amplitude of 0.2-0.5 mm at 100 Hz was applied to the flattened surface of retina throughout under visual control with the microscope, so that the cells were dissociated from the ganglion cell layer, but least from the inner and outer nuclear layers. After removing the remaining retinal tissue, the culture dish was filled with the bath solution, and left on a vibration-isolation table for allowing cells to settle down for 15-40 min. The bath solution was replaced by a fresh aliquot supplemented with 1 mg/ml bovine serum albumin and the dissociated cells were maintained at room temperature (20-25 oC) for 2-18 hrs prior to the electrophysiological recordings described below. The ganglion cells were identified

66

T. Motomura, Y. Hayashida, and N. Murayama

based on the size criteria [6]. Nearly all those cells we made recordings from in voltage-/current-clamp showed the large amplitude of voltage-gated Na+ current and/or of action potentials (see Fig. 1A-B), verifying that they were ganglion cells [4]. 2.2 Electrophysiology Since the previous studies demonstrated that the membrane conductances of retinal ganglion cells can be modulated by the intracellular messengers, e.g. Zn2+ [13] and cAMP [7], all recordings presented here were performed in perforated-patch wholecell mode [9] to maintain cytoplasmic integrity. Patch electrodes were pulled from borosilicate glass capillaries to tip resistances of approximately 4-8 MΩ. The tip of the electrodes were filled with a recording “electrode” solution that contained (in mM): 110 K-D-gluconic acid, 15 KCl, 15 NaOH, 2.6 MgCl2, 0.34 CaCl2, 1 EGTA, 10 HEPES. The pH was adjusted to 7.2 with methanesulfonic acid. The shank of the electrodes were filled with this solution after the addition of amphotericin B as the perforating agent (260 μg/ml, with 400 μg/ml Pluronic F-127). The recordings were made after the series resistance in perforated-patch configuration reached a stable value (typically 20-40 MΩ, ranging 10-100 MΩ). In the fast current-clamp mode of the amplifier (EPC-10, Heka), the voltage monitor output was analog-filtered by the built-in Bessel filters (3-pole 10–30 kHz followed by 4-pole 2-5 kHz) and digitally sampled (5–20 kHz). The voltage drop across the series resistance was compensated by the built-in circuitry. The recording bath was grounded via an agar bridge, and the bath solution was continuously superfused over each cell recorded from, at a constant flow rate (0.4 ml/min). The volume of solution in the recording chamber was kept at about 2 ml. To apply a high-K+ solution (Fig. 2B), 8 mM NaCl in the bath solution was replaced by the equimolar KCl. An enzyme solution was made by supplementing 0.25 mg/ml papain and 2.5 mM L-cystein in the bath solution. All experiments were performed at room temperature.

3 Results Perforated-patch whole-cell recordings were made from the somata of ganglion cells dissociated by our recently developed protocol (see Methods), which offered us quantitative measurements of the intrinsic membrane properties with the least distortion due to the proteolysis by enzymes [2]. Conversely, since these cells were never exposed to any enzyme, they were useful in examining the effects of the enzymes utilized for the cell dissociation in previous studies. In fact, spike firing of the ganglion cells in response to constant current injection via the patch electrode (30-pA step in the positive direction) was irreversibly altered when the enzyme solution was superfused over those cells (n=3): 1) The resting potential depolarized by 5-20 mV and the spike firing diminished during the enzyme application; 2) When the enzyme was washed out from the recording chamber, the resting potential gradually hyperpolarized near the original level and the spike firing returned in some way;

sVT in Mammalian Retinal Ganglion Cells Dissociated by Vibration

A

67

B

C

Fig. 1. Spontaneous voltage transients (sVT) observed in the dissociated retinal ganglion cells. A: Microphotograph of the cell recorded from. Note the soma being lager than 15 μm in diameter. B: Membrane potential changes in response to step-wise constant current injections. Four traces are superimposed. The injected current was 10 pA in the negative direction and 10, 20, and 30 pA in the positive directions. C: Spontaneous hyperpolarizations under the currentclamp. The recordings were made for 50 sec in three different episodes with breaks for 12 sec between first and second, and for 6 sec between second and third. A constant current (2 pA in the positive direction) was injected to hold the membrane potential at around –70 mV (dashed gray line). Inset: Examples of sVT in an expanded time scale. Five events are recognized. Three of them are similar in their amplitude and time course, and the other two have roughly half and quarter of the largest amplitude.

3) After ~20 min of the washing out of enzyme, the spike firing reached a steady state at which the interval between the first and second spike firings in response to the current step was shorter than that before the enzyme application, by 40 11 % (mean S.E.) (not shown, [12]). These results suggest that, in previous studies on isolated retinal ganglion cells, some of the ionic channels could be significantly distorted during the dissociation procedure because of the use of proteolytic enzymes. Moreover, we found the spontaneous voltage transients (sVT) with the fast rise and slower decay in the retinal ganglion cell somata dissociated by our method [8]. Fig. 1C shows an example of sVT recorded from the cell shown in Fig. 1A. As shown in the Fig., transient hyperpolarizations spontaneously appeared under a constant current injection. Most of these hyperpolarizations are similar in amplitude and time course at a certain membrane potential (−70 mV here) and in some, the peak amplitude of hyperpolarizations was roughly the half or quarter (or one-eighth, in other cells) of the largest one (Inset). Such sVT appeared in 10-20 % of the cells we made recordings from, and could be observed in particular cells as long as we kept the recordings (0.5-2 hrs). When the enzyme solution was superfused over one of those cells, the sVT disappeared completely, and then were not seen again even after 20 min of the washing out of enzyme.

±

±

68 A

T. Motomura, Y. Hayashida, and N. Murayama B

C

Fig. 2. Reversal potential of sVT. A, B: The sVT recorded in the saline containing extracellular K+ of 3.5 mM (A) and 11.5 mM (B). The basal membrane potential (indicated by arrows) was varied by injecting holding currents ranging between –8 and +8 pA in A and between –8 and +12 pA in B. C: Plots of the peak amplitude versus basal potential. Only the events having the largest amplitude (see Fig. 1C) are taken into account. Note that the amplitude of depolarizations were plotted as negative values, and vice versa. The filled circles and open circles represent the data for 3.5-mM K+ and 11.5-mM K+, respectively.

As shown in Fig. 1C, the sVT were all recorded as hyperpolarizations when the cell was held at approximately −70 mV. Thus, the reversal potential of the ionic current producing sVT should be below this voltage. In Fig. 2, the reversal potential for sVT was measured by holding the basal membrane potential at different levels under the current-clamp, c.f. [5]. As expected, the polarity of sVT reversed around −80 mV when the basal membrane potential was varied from about −100 to −40 mV (Fig. 2A). Based on the ionic compositions in the bath and electrode solutions used in this recording, the equilibrium potential of K+ (EK) was estimated to be about −90 mV, and close to the reversal potential for sVT. When the EK was shifted by +30 mV, i.e. from about −90 to −60 mV by applying the high-K+ solution (see Method), the polarity of sVT reversed between −54 and −39 mV of the basal membrane potential (Fig. 2B). Fig. 2C plots the peak amplitude of sVT versus the basal membrane potential. The linear regressions on these plots (gray lines) crossed the abscissa (dashed line) at approximately –76 mV and –48 mV for 3.5-mM K+ and 11.5-mM K+, respectively, showing the shift of reversal potential parallel to the EK shift. Similar results were obtained in other two cells. These results indicate that the ionic conductance responsible for producing sVT is permeable to, at least, K+. In the present experiments, we made recordings from the cells without neurites or with neurites no longer than 10 μm or so. Therefore, those cells can be modeled as a single compartment shown in Fig. 3A. In this model, the unknown conductance responsible for producing sVT and the reversal potential are represented by “gx” and “Ex”, respectively. The membrane properties intrinsic to the cell are represented by membrane capacitance Cm, nonlinear conductance gm, and the apparent reversal potential Em. Here, Cm was measured with the capacitance compensation circuitry of

sVT in Mammalian Retinal Ganglion Cells Dissociated by Vibration

69

B

A

C

E

D

G

H F

Fig. 3. Conductance changes during sVT. A: Single compartment model of the isolated somata. ICm, the current through Cm. Igm, the current through gm. Igx, the current through gx. B: Voltage responses to the current steps injected. The amplitude of current steps (Iinj) were varied from –10 to +10 pA in 5-pA increment and the corresponding voltage changes (Vm), from the bottom (black) to the top (light gray), were recorded. C: Plots of the membrane potential versus the amplitude of current steps. The voltage was measured at the time points indicated by the marks in B (circle, square, triangle, rhombus, and hexagon). The solid line shows the best fit to the plots with a single exponential function. D: Voltage-dependency of gm calculated from the plots in C. The derivative of current with respect to the membrane potential gave the slope conductance gm, which could be approximated by a hyperbolic function, gm = Iα / (Vα –Vm), where Vm
±

70

T. Motomura, Y. Hayashida, and N. Murayama

the patch-clamp amplifier we used and Em was measured as the resting potential under current-clamp. As shown in Fig. 3B-D, gm (a function of membrane potential) was also estimated experimentally from the voltage shifts induced by the current steps injected to the cell. Likewise, the value of Ex was estimated as shown in Fig. 2C. Since, during sVT, the membrane potential (Vrec) was recorded while the known amplitude of current (Iinj) was injected from the electrode, the time-varying gx could be calculated from the Kirchhoff’s law, by the equation:

g x(t) =

I inj − C m

d Vrec(t) − g m {Vrec(t) − E m } dt Vrec(t) − E x

(1)

Fig. 3F shows the calculated time course of gx during the sVT shown in Fig. 3E. Although the sVT were different in amplitude, time course, and polarity when measured at different basal potentials (E), the calculated gx fairly resembled each other (F). When the peak amplitude of the gx change was plotted against the basal membrane potential, little voltage dependency was found at potentials especially below –50 mV (Fig. 3G). And also, the decay time constant of the gx change are almost constant over the potential we recorded (Fig. 3H). The mean values of peak amplitude and decay time constants were in the ranges of 0.1-0.3 nS and 0.1-0.15 sec, respectively. These results were quantitatively similar in two cells, as compared the plots with different marks in G and H, implying that similar populations of ionic channels operated to produce the sVT.

4 Discussions In previous studies, spontaneous voltage/current transients have never been reported in acutely dissociated retinal ganglion cells. Here, we made recordings from the mammalian retinal ganglion cells dissociated by the recently developed method [8], and showed the characteristics of sVT observed in those cells. The present study showed that the sVT have the peak amplitude in quantal manner (Fig. 1), and may involve K+ conductance activation (Fig. 2) which has little or slight voltagedependency over the range we examined (Fig. 3). These results are less compatible with the idea that the voltage-gated K+ conductance are activated spontaneously by intracellular messengers like Ca2+, as in the dissociated retinal amacrine cells [11]. Previous studies have shown that spontaneous postsynaptic potentials or currents can be recorded in the single neurons vibrodissociated from brains and spinal cords [1]. Since the dissociation protocol we used here utilized the vibrodissociation technique [8], it is feasible for us to have the working hypothesis illustrated in Fig. 4, to infer the underlying mechanisms of sVT we observed in the retinal ganglion cells. If this hypothesis is the case, it enables one to investigate the functional machinery controlling the transmitter release in single synaptic terminals under the isolation from the axons and neurites [14].

sVT in Mammalian Retinal Ganglion Cells Dissociated by Vibration

71

Fig. 4. A working hypothesis for the underlying mechanism of sVT. It shows that the transmitter released from an isolated presynaptic terminal (the left-hand side) activates the K+ conductance on the membrane of the dissociated ganglion cell somata we recorded from (the right-hand side).

In our preliminary experiments, it was found that the sVT frequency, but not the amplitude, decreased when a low-Ca2+ solution was superfused over the cell and increased briefly upon application of caffeine. Thus, it is suggested that, at least, the sVT are elicited via an intracellular Ca2+-dependent process. Although we cannot rule out the contribution of cytoplasmic Ca2+ in the cells recorded from to the effects of extracellular low-Ca2+ and caffeine on the sVT, those experimental results are consistent with the present hypothesis (Fig. 4). The retinal ganglion cells in situ/vivo receive the inhibitory transmitter GABA released from the presynaptic amacrine cells, and the GABAb receptor, which is generally coupled with K+ conductance, is known to be expressed in rat ganglion cells [10]. However sVT were not suppressed when GABAb receptor antagonist (2hydroxysaclofen or SCH 50911) was applied. Further studies remain to be conducted to reveal the underlying mechanisms of the sVT in retinal ganglion cells somata. Acknowledgments. The authors shall be grateful to Dr. N. Akaike for his valuable suggestions on our dissociation protocol and lending most of the equipments used in the present experiments, to Dr. K. Hayashi for lending the microscope. This work was partly supported by the Japan Ministry of Education, Science, Sports and Culture, Grant–in–Aid for Young Scientists (B), 17700398, 2005 to Y.H.

References 1. Akaike, N., Moorhouse, A.J.: Techniques: applications of the nerve-bouton preparation in neuropharmacology. Trends in Pharmacological Science 24(1), 44–47 (2003) 2. Armstrong, C.E., Roberts, W.M.: Electrical properties of frog saccular hair cells: distortion by enzymatic dissociation. Journal of Neuroscience 18(8), 2962–2973 (1998)

72

T. Motomura, Y. Hayashida, and N. Murayama

3. Armstrong, C.M., Gilly, W.F.: Access resistance and space clamp problems associated with whole-cell patch clamping. Methods in enzymology 207, 100–122 (1992) 4. Barres, B.A., Silverstein, B.E., Corey, D.P., Chun, L.L.: Immunological, morphological, and electrophysiological variation among retinal ganglion cells purified by panning. Neuron 1(9), 791–803 (1988) 5. Coombs, J.S., Eccles, J.C., Fatt, P.: The specific ionic conductances and the ionic movements across the motoneuronal membrane that produce the inhibitory post-synaptic potential. Journal of Physiology 130(2), 326–374 (1955) 6. Guenther, E., Schmid, S., Grantyn, R., Zrenner, E.: In vitro identification of retinal ganglion cells in culture without the need of dye labeling. Journal of Neuroscience Methods 51(2), 177–181 (1994) 7. Hayashida, Y., Ishida, A.T.: Dopamine receptor activation can reduce voltage-gated Na+ current by modulating both entry into and recovery from inactivation. Journal of Neurophysiology 92(5), 3134–3141 (2004) 8. Hayashida, Y., Motomura, T., Murayama, N.: Vibrodissociation of rat retinal ganglion cells attached with inhibitory synaptic boutons. Investigative Ophthalmology & Visual Science 47, E-Abstract 3763 (2006) 9. Hayashida, Y., Partida, G.J., Ishida, A.T.: Dissociation of retinal ganglion cells without enzymes. Journal of Neuroscience Methods 137(1), 25–35 (2004) 10. Koulen, P., Malitschek, B., Kuhn, R., Bettler, B., Wassle, H., Brandstatter, J.H.: Presynaptic and postsynaptic localization of GABA(B) receptors in neurons of the rat retina. European Journal of Neuroscience 10(4), 1446–1456 (1998) 11. Mitra, P., Slaughter, M.M.: Mechanism of generation of spontaneous miniature outward currents (SMOCs) in retinal amacrine cells. Journal of General Physiology 119(4), 355– 372 (2002) 12. Motomura, T., Hayashida, Y., Murayama, N.: Mechanical Dissociation of Retinal Neurons with Vibration. IEEJ Transactions on Electronics, Information and Systems 127(10) (in press, 2007) 13. Tabata, T., Ishida, A.T.: A zinc-dependent Cl- current in neuronal somata. Journal of Neuroscience 19(13), 5195–5204 (1999) 14. von Gersdorff, H., Matthews, G.: Dynamics of synaptic vesicle fusion and membrane retrieval in synaptic terminals. Nature 367(6465), 735–739 (1994) 15. Vorobjev, V.S.: Vibrodissociation of sliced mammalian nervous tissue. Journal of Neuroscience Methods 38(2-3), 145–150 (1991)

Region-Based Encoding Method Using Multi-dimensional Gaussians for Networks of Spiking Neurons Lakshmi Narayana Panuku and C. Chandra Sekhar Department of Computer Science and Engineering, Indian Institute of Technology Madras, Chennai 600 036, India {panuku,chandra}@cs.iitm.ernet.in

Abstract. In this paper, we address the issues in representation of continuous valued variables by firing times of neurons in the spiking neural network used for clustering multi-variate data. The existing range-based encoding method encodes each dimension separately. This method does not make use of the correlation among the different variables, and the knowledge of the distribution of data. We propose a region-based encoding method that places multi-dimensional Gaussian receptive fields in the data-inhabited regions, and captures the correlation among the variables. Effectiveness of the proposed encoding method in clustering the complex 2-dimensional and 3-dimensional data sets is demonstrated.

1

Introduction

Artificial neural networks (ANNs) have been shown to have the ability to extract patterns from complex data [1, 2]. Based on the computational units used, these ANN models can be classified into three generations [3]. The McCulloch-Pitts neurons, which are considered as first generation neurons, can only give binary output. The computational units that output continuous values, like sigmoidal units, are considered as the second generation neurons. Biologically, the output of a sigmoidal unit can be interpreted as the firing rate of a neuron. Under the assumption that, in biological neural networks the continuously varying mean firing rate of a neuron (rate code) contains the information about the neuron’s time varying state of excitation, the sigmoidal units can model the computations in biological systems. Recently, the timing of the action potentials or spikes has been recognized as a possible means of neural information coding rather than the average firing rate of the neurons [4, 5, 6]. It is shown that coding with the timing of spikes allows powerful neuronal information processing [7]. These results have generated considerable interest in the third generation time-based neurons like spiking neurons [3]. Various models of spiking neural networks (SNNs) like leaky integrate-fire model, spike response model, and liquid state machine [6, 8] and various learning methods for these models have been reported in the literature [9, 10, 11]. The SNNs have been used in many applications such as signal M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 73–82, 2008. c Springer-Verlag Berlin Heidelberg 2008 

74

L.N. Panuku and C.C. Sekhar

coincidence detection [2], isolated word recognition [8], and implementation of temporal-RBF networks [12]. In [13], a Hebbian based learning mechanism is proposed for spiking neuron models with multi-delay connections, namely multi-delay SNNs (MDSNNs). This learning mechanism is observed to select the connections with matching delays. For clustering the data, this approach is extended in [10] by considering not only the firing or non-firing of a neuron, but also its firing time. A coding scheme to convert an analog input variable into firing times of neurons is proposed. However, this method has limitations in both clustering capacity as well as precision [14]. To overcome this limitation, Bohte, et al., [14], proposed a population coding based encoding method that encodes the values of input variables using multiple overlapping 1-dimensional (1-D) Gaussian receptive fields (GRFs). This method is shown to cluster a number of data sets at low expense in terms of neurons while enhancing clustering capacity and precision. Using this encoding method and a multi-layer MDSNN with lateral connections in the hidden layer, complex data like the interlocking cluster data is clustered. However, this encoding method does not make use of the correlation present among the variables in the multi-variate data. It uniformly places the GRFs along each dimension, leading to a large neuron count and increased computational cost. The boundaries given by the MDSNN with this encoding method are observed to be combinations of linear segments. To overcome these limitations, we propose a novel encoding method that places multi-dimensional GRFs in the data-inhabited regions and uses the correlation present in the data. We show that the proposed encoding method helps the MDSNNs in clustering a number of 2-D and 3-D nonlinearly separable data sets, while keeping a low neuron count. Moreover, the cluster boundaries given by the MDSNN with this encoding method are observed to follow the shapes of the clusters. This paper is organized as follows: The architecture of the MDSNN and the Hebbian based learning rule for clustering are described in Section 2. The existing range-based encoding method and its limitations are discussed in Section 3. Section 4 presents the proposed region-based encoding method that uses the multi-dimensional GRFs. The performance of the proposed encoding method for clustering different complex data sets is also given in this section.

2

Multi-delay Spiking Neural Networks

The architecture of the MDSNN consists of a fully connected feedforward network of spiking neurons with connections implemented as multiple delayed synaptic terminals, as shown in Fig. 1(a). A connection from pre-synaptic neuron i to post-synaptic neuron j consists of a fixed number (m) of synaptic terminals. Each terminal serves as a subconnection that is associated with a different delay and weight. The delay dl of a synaptic terminal l is the difference between the firing time of the pre-synaptic neuron, and the time when the post-synaptic potential (PSP) resulting from terminal l starts raising.

Region-Based Encoding Method Using Multi-dimensional Gaussians

75

The time varying impact of the pre-synaptic spike on a post-synaptic neuron is described by a spike response function, ε(.), also referred to as the PSP. The PSP is modeled by the α-function, as in [14]. A neuron j in the network generates a spike when the value of its internal state variable xj , the “membrane potential”, crosses a threshold ϑ. The internal state variable xj (t) is defined as follows: xj (t) =

m 

  l wij ε t − ti − dl ,

(1)

i∈Γj l=1 l where Γj is the set of neurons pre-synaptic to neuron j, wij is the weight of the th l synaptic terminal between the neurons i and j, and ti is the firing time of the pre-synaptic neuron i. The time at which xj (t) crosses the threshold ϑ with a positive slope is the firing time of the neuron j, denoted by tj .

Output Layer 1

j

J

L(Δt)

j

00 11 1111 00 0 ij 1 00

m

0011 11 0 l 1 00 wij 11 00 0 1 w 00 11 1

t

i+

d

1

t

i+

w 1 0 0 ij 1 d

t

l

i

i+

d

β

m

1

11 011 11 000 1100 0 11 i 00

11 00 11 00 11 00 11 00

I

c 0

(a) f(a)

1

Δt

b

Input Layer

(b) 2

3

4

5

6

7

8

t=0 T5 T4

t=9

T T63

coding threshold

a

Value of Variable

T(a)=[ NF, NF, 9, 1, 0, 8, NF, NF ]

(c) Fig. 1. (a) Architecture of an MDSNN. (b) Learning function L(Δt). (c) Range-based encoding of an input variable with value a into firing times, T (a).

For clustering, the weights of the terminals of the connections between the input neurons and the winning neuron, i.e., the output neuron that fires first, are modified using a time-variant of Hebbian learning. The learning rule for the l weight wij of the synaptic terminal with delay dl is as follows:     l Δwij = ηL(Δtlij ) = η (1 − b) exp −(Δtlij − c)2 /β 2 + b , (2) where Δtlij denotes the time difference between the onset of PSP at the lth synaptic terminal, tP SP li , and the firing time of the winning neuron, tj , i.e, Δtlij

76

L.N. Panuku and C.C. Sekhar

= tP SP li −tj = (ti +dl )−tj . The parameter c determines the position of the peak, b determines the negative update given to a neuron for which Δt is significantly different from c, and β sets the width of the positive part of the learning function (Fig. 1(b)). The weight of a synaptic terminal is limited to the range 0 to wmax , the maximum value that a weight can take. In our experiments, the range of the values for delays dl is set to 0 − 9 milliseconds with a resolution of 1 millisecond, i.e., m = 9. The parameters b and η are set to −0.2 and 0.01, while c and β are empirically chosen. In [14], an MDSNN with fixed threshold units is considered for the task of clustering. This model, with the above mentioned learning rule (Eqn. 2), could cluster linearly separable data. However, when applied on nonlinearly separable data like the single-ring data and the interlocking cluster data, all the data points are grouped into a single cluster. To overcome this limitation, we use a varyingthreshold method [15], in which the threshold of a spiking neuron is initialized to a small, positive value and is gradually increased (in steps of Δϑ) as the learning progresses, until it reaches a maximum value, ϑmax . Moreover, when a multi-layer MDSNN is trained to cluster complex data, the layers are trained using multi-stage learning method [15] in which the nth layer is trained before starting the training of (n + 1)th layer. With these two extensions, the MDSNNs are able to cluster the single-ring data and the interlocking cluster data [15]. In our studies, we use the varying-threshold method and the multi-stage learning method. The values of Δϑ and ϑmax are determined empirically.

3

Range-Based Encoding

When the input variables are continuous valued attributes, it is necessary to encode the value of each variable into firing times of neurons in the input layer of MDSNN. Bohte, et al., [14], proposed an encoding method that encodes the values of input variables by a population code obtained from neurons with graded and overlapping sensitivity profiles. As this method encodes each variable with 1-D GRFs uniformly placed to cover the whole range of values that the variable can take, it is called the range-based encoding method. In this method, the range of values for each input variable is determined. For the range [Imin ...Imax ] of a variable, n(> 2) GRFs are used. The center of the ith GRF is set to μi = Imin + ((2i − 3)/2)((Imax − Imin )/(n − 2)). One GRF is placed outside the range at each of the two ends. All the GRFs encoding an input variable will have the same width. The width of a GRF is set to σ = (1/γ)((Imax − Imin )/(n − 2)), where γ controls the extent of overlap between the GRFs. For an input variable with value a, the activation value of the ith GRF with center μi and width σ is given by   fi (a) = exp −(a − μi )2 /(2 σ 2 ) (3) The firing time of the neuron associated with this GRF is inversely proportional to fi (a). For a highly stimulated GRF, with value of fi (a) close to 1.0, the firing time t = 0 milliseconds is assigned. When the activation value of the

Region-Based Encoding Method Using Multi-dimensional Gaussians

77

GRF is small, the firing time t is high indicating that the neuron fires later. In our experiments, the firing time of an input neuron is chosen to be in the range 0 to 9 milliseconds. While converting the activation values of GRFs into firing times, a coding threshold is imposed on the activation value. A GRF that gives an activation value less than the coding threshold will be marked as not-firing (NF), and the corresponding input neuron will not contribute to the membrane potential of the post-synaptic neuron. The range-based encoding method is illustrated in Fig. 1(c). For multi-variate data, each variable is encoded separately, effectively not using the correlation present among the variables. In this encoding method, 1-D GRFs are uniformly placed along an input dimension, without considering the distribution of the data. Hence, when the data is sparse some GRFs, placed in the regions where data is not present, are not effectively used. This results in a high neuron count and computational cost. The widths of the GRFs are derived without using the knowledge of the data distribution, except for the range of values that the variables take. Taking one GRF along an input dimension and quantizing its activation value results in the formation of intervals within the range of values of that variable, such that one or more intervals are mapped onto a particular quantization level. When 2-D data is encoded by taking an array of 1-D GRFs along each input dimension, the input space is quantized into rectangular grids such that all the input patterns falling into a particular rectangular grid have the same vector of quantization levels, and hence the same encoded time vector. Additionally, one or more rectangular grids may have the same encoded time vector. For multivariate data, the input space is divided into hypercuboids. To demonstrate this, the single-ring data (shown in Fig. 2(a)) is encoded by placing 5 GRFs along each dimension, dividing the input space into grids as shown in Fig. 2(b). A 10-2 MDSNN, having 10 neurons in the input layer and 2 neurons in the output layer, is trained to cluster this data. The space of data points as represented by the output layer neurons is shown in Fig. 2(c). The cluster boundary is observed to be a combination of linear segments defined by the rectangular grid boundaries formed by encoding. The shape of the boundary formed by the MDSNN is significantly different from the desired circle-shaped boundary between the two clusters in the single-ring data. Increasing the number of GRFs used for encoding each dimension may give a boundary that is a combination of smaller linear segments, at the expense of high neuron count. However, this may not result in proper clustering of the data, as the choice of number of GRFs is observed to be crucial in the range-based encoding method. When the range-based encoding method is used along with the varyingthreshold method and the multi-stage learning method [15] to cluster complex data sets such as the double-ring data and the spiral data, it is observed that proper subclusters are not formed by the neurons in the hidden layer. As each dimension is encoded separately, the spatially disjoint subsets of data points, as shown by the marked regions in Fig. 3, that have similar encoding along a particular dimension are found to be represented by a single neuron in the hidden

78

L.N. Panuku and C.C. Sekhar 2

1

1.5

0.8 0.6

1

0.4

0.5

0.2 0

0

−0.2

−0.5

−0.4

−1

−0.6

−1.5

−0.8

−2 −2 −1.5 −1 −0.5

0(a)0.5

1

1.5

−1 −1 −0.8−0.6−0.4−0.2

2

0

0.2 0.4 0.6 0.8

(b)

1

Fig. 2. Clustering the single-ring data encoded using the range-based encoding method: (a) The single-ring data, (b) data space quantization due to the range-based encoding, and (c) data space representation by the output neurons

15 3

10 5

y

y

1

0

−1 −5 −3

−5 −5

−10 −15 −3

x

−1

1 (a)

3

−15

−10

−5

0 x

(b)

5

10

15

Fig. 3. Improper subclusters formed when the data is encoded with the range-based encoding method for (a) the double-ring data and (b) the spiral data

layer. This binding is observed to form during the initial iterations of learning when the firing thresholds of neurons are low. The established binding cannot be unlearnt in the subsequent iterations, leading to improper clustering at the output layer. An encoding method that overcomes the above discussed limitations is proposed in the next section.

4

Region-Based Encoding Using Multi-dimensional Gaussian Receptive Fields

Using multi-dimensional GRFs for encoding helps in capturing the correlation present in the data. One approach would be to uniformly place the multidimensional GRFs covering the whole range of the input data space. However, this results in an exponential increase in the number of neurons in the input layer with the dimensionality of the data. To circumvent this, we propose a region-based encoding method that places the multi-dimensional GRFs only in the data-inhabited regions, i.e., the regions where data is present. The mean vectors and the covariance matrices of these GRFs are computed from the data in the regions, thus capturing the correlation present in the data. To identify the data-inhabited regions in the input space, first the k-means clustering is performed on the data to be clustered, with the value of k being larger than the number of actual clusters. On each of the regions, identified using the k-means clustering method, a multi-dimensional GRF is placed by computing the mean vector and the covariance matrix from the data in that region. Response of the ith GRF for a multi-variate input pattern a is computed as,

Region-Based Encoding Method Using Multi-dimensional Gaussians

 1 t −1 fi (a) = exp − (a − μi ) Σi (a − μi ) , 2

79



(4)

where, μi and Σi are the mean vector and the covariance matrix of the ith GRF respectively, and fi (a) is the activation value of that GRF. As discussed in Section 3, these activation values are translated into firing times in the range 0 to 9 milliseconds and the non-optimally stimulated input neurons are marked as NF. By deriving the covariance for a GRF from the data, the region-based encoding method captures the correlation present in the data. The regions identified by k-means clustering and the data space quantization resulting from this encoding method, for the single-ring data used in Section 3, are shown in Fig 4(a) and 4(b) respectively. The boundary given by the MDSNN with the region-based encoding method is shown in Fig. 4(c). This boundary is more like the desired circle-shaped boundary, as against the combination of linear segments observed with the range-based encoding method (see Fig. 2(c)). 2

2.5

1.5

2 1.5 1

1 0.5 0

0.5 0

−0.5

−0.5 −1 −1.5 −2

−1 −1.5 −2 −2 −1.5 −1 −0.5

0 (a)0.5

1

1.5

2

−2.5 −2.5 −2 −1.5 −1 −0.5

0

0.5

b)

1

1.5

2

2.5

Fig. 4. (a) Regions identified by k-means clustering with k = 8, (b) data space quantization due to the region-based encoding and (c) data space representation by the neurons in the output layer

Next, we study the performance of the proposed region-based encoding method in clustering complex 2-D and 3-D data sets. For the double-ring data, the k-means clustering is performed with k = 20 and the resulting regions are shown in Fig 5(a). Over each of the regions, a 2-D GRF is placed to encode the data. A 20-8-2 MDSNN is trained using the multi-stage learning method, discussed in Section 2. It is observed that out of 8 neurons in the hidden layer 3 neurons do not win for any of the training examples and the data is represented by the remaining 5 neurons as shown in Fig 5(b). These 5 neurons provide the input in the second stage of learning to form the final clusters (Fig 5(c)). The resulting cluster boundaries are seen to follow the data distribution as shown in Fig 5(d). Similarly, the spiral data is encoded using 40, 2-D GRFs. The regions of data identified using the k-means clustering method are shown in in Fig 6(a). A 40-20-2 MDSNN is trained to cluster the spiral data. As shown in Fig 6(b), 14 subclusters are formed in the hidden layer that are combined in the next layer to form the final clusters as shown in Fig 6(c). The region-based encoding method helps in proper subcluster formation at the hidden layer (Fig 5(b) and Fig 6(b)), against the range-based encoding method (Fig 3). The proposed method is also used to cluster 3-D data sets namely, the interlocking donuts data and the 3-D ring data. The interlocking donuts data is

80

L.N. Panuku and C.C. Sekhar 4

4

4

3

3

3

2

2

2

1

1

0

0

0

−1

−1

−1

−2

−2

−2

−3 −4 −4

1

−3

−3 −3

−2

−1

0

1

2

3

(a)

4

−4 −4

−3

−2

−1

0

(b) 1

2

3

4

−4 −4

−3

−2

−1

0

1

2

3

4

(c)

Fig. 5. Clustering the double-ring data: (a) Regions identified by k-means clustering with k = 20, (b) subclusters formed at the hidden layer, (c) clusters formed at the output layer and (d) data space representation by the neurons in the output layer

15

15

15

10

10

10

5

5

5

0

0

0

−5

−5

−5

−10

−10 −15 −15

−10

−5

0

(a)

5

10

15

−15 −15

−10

−10

−5

0

5

10

15

−15 −15

−10

−5

(b)

0

(c)

5

10

15

Fig. 6. Clustering the spiral data: (a) Regions identified by k-means clustering with k = 40, (b) subclusters formed at the hidden layer, (c) clusters formed at the output layer and (d) data space representation by the neurons in the output layer 1

2 1 0 −1 −2 −2

−1

0

1

(a) 2

2 1.5 1 2 0.5 0 1 −0.5 0 −1 −1.5 −1 −2 3 3 −2

0

1 0

−1 2 −2 −1

−1 2 1.5

1 0

0 2

1

1 0

(b)−1

−2 2

−1 −2 −2

−1

0

(c)

1

2

1 0.5

2 0 −0.5

1 0 −1 −1.5

−1 −2 −2

(d)

Fig. 7. Clustering of the interlocking donuts data and the 3-D ring data: (a) Regions identified by k-means clustering on the interlocking donuts data with k = 10 and (b) clusters formed at the output layer. (c) Regions identified by k-means clustering on the 3-D ring data with k = 5 and (d) clusters formed at the output layer.

encoded with 10, 3-D GRFs and a 10-2 MDSNN is trained to cluster this data. The k-means clustering results and the final clusters formed by the MDSNN are shown in Fig 7(a) and (b) respectively. The clustering results for the 3-D ring data with the proposed encoding method are shown in Fig 7(c) and 7(d). For comparison, the performance of the range-based encoding method and the region-based encoding method, for different data sets, is presented in Table 1. It is observed that the region-based encoding method outperforms the range-based encoding method for clustering complex data sets like the double-ring data and the spiral data. For the cases, where both the methods give the same or almost the same performance, the number of neurons used in the input layer is given in the parentheses. It is observed that the region-based encoding method always maintains a low neuron count, there by reducing the computational cost. The difference between the neuron counts for the two methods may look small for these 2-D and 3-D data sets. However, as the dimensionality of the data increases, this

Region-Based Encoding Method Using Multi-dimensional Gaussians

81

Table 1. Comparison of the performance (in %) of MDSNNs using the range-based encoding method and the region-based encoding method for clustering. The numbers in parentheses give the number of neurons in the input layer. Data set

Encoding method Range-based Region-based encoding encoding

Double-ring data 74.82 Spiral data 66.18 Single-ring data 100.00 (10) Interlocking cluster data 99.30 (24) 3-D ring data 100.00 (15) Interlocking donuts data 97.13 (21)

100.00 100.00 100.00 100.00 100.00 100.00

(8) (6) (5) (10)

difference can be significant. From these results, it is evident that the proposed encoding method scales well to higher dimensional data clustering problems, while keeping a low count of neurons. Additionally, and more importantly, the nonlinear cluster boundaries given by the region-based encoding method follow the distribution of the data or shapes of the clusters.

5

Conclusions

In this paper, we have proposed a new encoding method using multi-dimensional GRFs for MDSNNs. We have demonstrated that the proposed encoding method effectively uses the correlation present in the data and positions the GRFs in the data-inhabited regions. We have also shown that the proposed method results in a low neuron count as opposed to the encoding method proposed in [14] and the simple approach of placing multi-dimensional GRFs covering the data space. This in turn results in low computational cost for clustering. With the encoding method proposed in [14], the cluster boundaries obtained for clustering nonlinearly separable data are observed to be combinations of linear segments and the MDSNN is failed to cluster the double-ring data and the spiral data. We have experimentally shown that with the proposed encoding method, the MDSNNs could cluster complex data like the double-ring data and the spiral data, while giving smooth nonlinear boundaries that follow the data distribution. In the existing range-based encoding method, when the data consists of clusters with different scales, i.e., narrow and wider clusters, then the GRFs with different widths are used. This technique is called multi-scale encoding. However, in the region-based encoding method the widths of the multi-dimensional GRFs are automatically computed from the data-inhabited regions. The widths of these GRFs can be different. In the proposed method, for clustering the 2-D and 3-D data, the value of k is decided empirically and the formation of subclusters at the hidden layer is verified visually. However, for higher dimensional data, it is necessary to ensure the formation of subclusters automatically.

82

L.N. Panuku and C.C. Sekhar

References 1. Haykin, S.: Neural Networks: A Comprehensive Foundation. Prentice Hall PTR, Englewood Cliffs (1998) 2. Kumar, S.: Neural Networks: A Classroom Approach. Tata McGraw-Hill, New Delhi (2004) 3. Maass, W.: Networks of Spiking Neurons: The Third Generation of Neural Network Models. Trans. Soc. Comput. Simul. Int. 14(4), 1659–1671 (1997) 4. Bi, Q., Poo, M.: Precise Spike Timing Determines the Direction and Extent of Synaptic Modifications in Cultured Hippocampal Neurons. Neuroscience 18, 10464–10472 (1998) 5. Maass, W., Bishop, C.M.: Pulsed Neural Networks. MIT-Press, London (1999) 6. Gerstner, W., Kistler, W.M.: Spiking Neuron Models. Cambridge University Press, Cambridge (2002) 7. Maass, W.: Fast Sigmoidal Networks via Spiking Neurons. Neural Computation 9, 279–304 (1997) 8. Verstraeten, D., Schrauwen, B., Stroobandt, D., Campenhout, J.V.: Isolated Word Recognition with the Liquid State Machine: A Case Study. Information Processing Letters 95(6), 521–528 (2005) 9. Bohte, S.M., Kok, J.N., Poutre, H.L.: Spike-Prop: Error-backpropagation in Temporally Encoded Networks of Spiking Neurons. Neural Computation 48, 17–37 (2002) 10. Natschlager, T., Ruf, B.: Spatial and Temporal Pattern Analysis via Spiking Neurons. Network: Comp. Neural Systems 9, 319–332 (1998) 11. Ruf, B., Schmitt, M.: Unsupervised Learning in Networks of Spiking Neurons using Temporal Coding. In: Gerstner, W., Hasler, M., Germond, A., Nicoud, J.-D. (eds.) ICANN 1997. LNCS, vol. 1327, pp. 361–366. Springer, Heidelberg (1997) 12. Hopfield, J.J.: Pattern Recognition Computation using Action Potential Timing for Stimulus Representations. Nature 376, 33–36 (1995) 13. Gerstner, W., Kempter, R., Van Hemmen, J.L., Wagner, H.: A Neuronal Learning Rule for Sub-millisecond Temporal Coding. Nature 383, 76–78 (1996) 14. Bohte, S.M., Poutre, H.L., Kok, J.N.: Unsupervised Clustering with Spiking Neurons by Sparse Temporal Coding and Multilayer RBF Networks. IEEE Transactions on Neural Networks 13, 426–435 (2002) 15. Panuku, L.N., Sekhar, C.C.: Clustering of Nonlinearly Separable Data using Spiking Neural Networks. In: de S´ a, J.M., Alexandre, L.A., Duch, W., Mandic, D. (eds.) ICANN 2007. LNCS, vol. 4668, Springer, Heidelberg (2007)

Firing Pattern Estimation of Biological Neuron Models by Adaptive Observer Kouichi Mitsunaga1 , Yusuke Totoki2 , and Takami Matsuo2 1

2

Control Engineering Department, Oita Institute of Technology, Oita, Japan Department of Architecture and Mechatronics, Oita University, 700 Dannoharu, Oita, 870-1192, Japan

Abstract. In this paper, we present three adaptive observers with the membrane potential measurement under the assumption that some of parameters in HR neuron are known. Using the Strictly Positive Realness and Yu’s stability criterion, we can show the asymptotic stability of the error systems. The estimators allow us to recover the internal states and to distinguish the firing patterns with early-time dynamic behaviors.

1

Introduction

In traditional artificial neural networks, the neuron behavior is described only in terms of firing rate, while most real neurons, commonly known as spiking neurons, transmit information by pulses, also called action potentials or spikes. Model studies of neuronal synchronization can be separated in those where models of the integrated-and-fire type are used and those where conductance-based spiking and bursting models are employed[1]. Bursting occurs when neuron activity alternates, on slow time scale, between a quiescent state and fast repetitive spiking. In any study of neural network dynamics, there are two crucial issues that are: 1) what model describes spiking dynamics of each neuron and 2) how the neurons are connected[3]. Izhikevich considered the first issue and compared various models of spiking neurons. He reviewed the 20 types of real (cortical) neurons response, considering the injection of simple dc pulses such as tonic spiking, phasic spiking, tonic bursting, phasic bursting. Through out his simulations, he suggested that if the goal is to study how the neuronal behavior depends on measurable physiological parameters, such as the maximal conductance, steady-state (in)activation functions and time constants, then the Hodgkin-Huxley type model is the best. However, its computational cost is the highest in all models. He also pointed out that the Hindmarsh-Rose(HR) model is computationally simple and capable of producing rich firing patterns exhibited by real biological neurons. Nevertheless the HR model is a computational one of the neuronal bursting using three coupled first order differential equations[5,6], it can generate a tonic spiking, phasic spiking, and so on, for different parameters in the model equations. Charroll simulated that the additive noise shifts the neuron model into two-frequency region (ı.e. bursting) and the slow part of the M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 83–92, 2008. c Springer-Verlag Berlin Heidelberg 2008 

84

K. Mitsunaga, Y. Totoki, and T. Matsuo

responses allows being robust to added noises using the HR model[7]. The parameters in the model equations are important to decide the dynamic behaviors in the neuron[12]. From the measurement theoretical point of view, it is important to estimate the states and parameters using measurement data, because extracellular recordings are a common practice in neuro-physiology and often represent the only way to measure the electrical activity of neurons[8]. Tokuda et al. applied an adaptive observer to estimate the parameters of HR neuron by using membrane potential data recorded from a single lateral pyloric neuron synaptically isolated from other neurons[13]. However, their observer cannot guarantee the asymptotic stability of the error system. Steur[14] pointed out that HR equations could not transformed into the adaptive observer canonical form and it is not possible to make use of the adaptive observer proposed by Marino[10]. He simplified the three dimensional HR equations and write as one-dimensional system with exogenous signal using contracting and the wandering dynamics technique. His adaptive observer with first-order differential equation cannot estimate the internal states of HR neurons. We have recently presented adaptive observers with full states measurement and with the membrane potential measurement[15]. However, the estimates of the states by the observer with output measurement are not enough to recover the immeasurable internal states. In this paper, we present three adaptive observers with the membrane potential measurement under the assumption that some of parameters in HR neuron are known. Using the Kalman-Yakubovich lemma, we can show the asymptotic stability of the error systems based on the standard adaptive control theory[11]. The estimators allow us to recover the internal states and to distinguish the firing patterns with early-time dynamic behaviors. The MATLAB simulations demonstrate the estimation performance of the proposed adaptive observers.

2

Review of Real (Cortical) Neuron Responses

There are many types of cortical neurons responses. Izhikevich reviewed 20 of the most prominent features of biological spiking neurons, considering the injection of simple dc pulses[3]. Typical responses are classified as follows[4]: – Tonic Spiking (TS): The neuron fires a spike train as long as the input current is on. This kind of behavior can be observed in the three types of cortical neurons: regular spiking excitatory neurons (RS), low-threshold spiking neurons (LTS), and first spiking inhibitory neurons (FS). – Phasic Spiking (PS): The neuron fires only a single spike at the onset of the input. – Tonic Bursting: The neuron fires periodic bursts of spikes when stimulated. This behavior may be found in chattering neurons in cat neocortex. – Phasic Bursting (PB): The neuron fires only a single burst at the onset of the input.

Firing Pattern Estimation of Biological Neuron Models

85

– Mixed Mode (Bursting Then Spiking) (MM): The neuron fires a phasic burst at the onset of stimulation and then switch to the tonic spiking mode. The intrinsically bursting excitatory neurons in mammalian neocortex may exhibit this behavior. – Spike Frequency Adaptation (SFA): The neuron fires tonic spikes with decreasing frequency. RS neurons usually exhibit adaptation of the interspike intervals, when these intervals increase until a steady state of periodic firing is reached, while FS neurons show no adaptation.

3

Single Model of HR Neuron

The Hindmarsh-Rose(HR) model is computationally simple and capable of producing rich firing patterns exhibited by real biological neurons. 3.1

Dynamical Equations

The single model of the HR neuron[1,5,6] is given by x˙ = ax2 − x3 − y − z + I y˙ = (a + α)x2 − y z˙ = μ(bx + c − z) where x represents the membrane potential y and z are associated with fast and slow currents, respectively. I is an applied current, and a, α, μ, b and c are constant parameters. We rewrite the single HR neuron as a vectorized form: ˙ = h(w) + Ξ(x, z)θ (S0 ) : w where ⎡ ⎤T ⎡ ⎤ ⎡ 2 ⎤ x −(x3 + y + z) x 1 0 00 0 ⎦, Ξ(x, z) = ⎣ 0 0 x2 0 0 0 ⎦, −y w = ⎣ y ⎦ , h(w) = ⎣ z 0 0 0 0 x 1 −z  T  T θ = θ1 , θ2 , θ3 , θ4 , θ5 , θ6 = a, I, a + α, μb, μc, μ . 3.2

Numerical Examples

The HR model shows a large variety of behaviors with respect to the parameter values in the differential equations[12]. Thus, we can characterize the dynamic behaviors with respect to different values of the parameters. We focus on the parameters a and I. The parameter a is an internal parameter in the single neuron and I is an external depolarizing current. For the fixed I = 0.05, the HR model shows a tonic bursting with a ∈ [1.8, 2.85] and a tonic spiking with a ≥ 2.9. On the other hand, for the fixed a = 2.8, the HR model shows a tonic bursting with I ∈ [0, 0.18] and a tonic spiking with a ∈ [0.2, 5].

86

K. Mitsunaga, Y. Totoki, and T. Matsuo

2 1.5 −0.3 −0.4

1

−0.5 −0.6

x

z

0.5 0

−0.7 −0.8 −0.9

−0.5

−1 8 6

−1

2 1

4 −1.5 0

0

2 200

400

600

800

1000

Fig. 1. The response of x in the tonic bursting

−1 0

y

time

−2

x

Fig. 2. 3-D surface of x, y, z in the tonic bursting

2 1.5 −0.5 1

−0.6 −0.7

x

z

0.5

−0.8 0 −0.9 −0.5

−1 8 6

−1

2 1

4 −1.5 0

0

2 200

400

600

800

1000

Fig. 3. The response of x in the tonic spiking

0

y

time

−1 −2

x

Fig. 4. 3-D surface of x, y, z in the tonic spiking

1.5

−0.2

1

−0.4

z

x

2

0.5

−0.6 −0.8

0 −1 8 −0.5

6

2 4

−1 0

1 2

200

400

600

800

1000

time

Fig. 5. The response of x1 in the intrinsic bursting neuron

y

0 0

−1

x

Fig. 6. 3-D surface of x, y, z in the intrinsic bursting neuron

The parameters of the HR model in the tonic bursting (TB) are given by a = 2.8, α = 1.6, c = 5, b = 9, μ = 0.001, I = 0.05. Figure 1 shows the response of x. Figure 2 shows the 3 dimensional surface of x, y, z. We call this neuron the intrinsic bursting neuron(IBN).

Firing Pattern Estimation of Biological Neuron Models

87

The parameters of the HR model in the tonic spiking (TS) are given by a = 3.0, α = 1.6, c = 5, b = 9, μ = 0.001, I = 0.05. The difference between the tonic bursting and the tonic spiking is only the value of the parameter a. Figure 3 shows the responses of x. Figure 4 shows the 3 dimensional surface of x, y, z. We also call this neuron the intrinsic spiking neuron(ISN). When the external current change from I = 0.05 to I = 0.2, the IBN shows a tonic spiking. Figure 5 shows the responses of x. Figure 6 shows the 3 dimensional surface of x, y, z.

4 4.1

Synaptically Coupled Model of HR Neuron Dynamical Equations

Consider the following synaptically coupled two HR neurons[1]: x˙ 1 = a1 x21 − x31 − y1 − z1 − gs (x1 − Vs1 )Γ (x2 ) y˙ 1 = (a1 + α1 )x21 − y1 , z˙1 = μ1 (b1 x1 + c1 − z1 ) x˙ 2 = a2 x22 − x32 − y2 − z2 − gs (x2 − Vs2 )Γ (x1 ) y˙ 2 = (a2 + α2 )x22 − y2 , z˙2 = μ2 (b2 x2 + c2 − z2 ) where Γ (x) is the sigmoid function given by Γ (x) =

4.2

1 . 1 + exp(−λ(x − θs ))

Numerical Examples

Consider the IBN neuron with a = 2.8 and the ISN neuron with a = 10.8 whose other parameters are as follows: αi = 1.6, ci = 5, bi = 9, μi = 0.001, Vsi = 2, θs = −0.25, λ = 10. Figures 7,8 show the responses of the membrane potentials in the coupling of IBN neuron and ISN neuron with the coupling strength gs = 0.05, respectively. Each neuron behaves as an intrinsic single neuron. As increasing the coupling strength, however, the IBN neuron shows a chaotic behavior. Figures 9,10 show the responses of the membrane potentials in the coupling of IBN neuron and ISN neuron with the coupling strength gs = 1, respectively. Figure 11 shows the response of the membrane potentials in the coupling of two same IBNs with

88

K. Mitsunaga, Y. Totoki, and T. Matsuo 2

12

1.5

10 8

1

6

x

x

0.5 4

0 2 −0.5

0

−1

−2

−1.5 0

200

400

600

800

−4 0

1000

200

400

time

600

800

1000

time

Fig. 7. The response of x1 of the IBN

Fig. 8. The response of x2 of the ISN

2

12

1.5

10 8

1

6

x

x

0.5 4

0 2 −0.5

0

−1 −1.5 0

−2

200

400

600

800

−4 0

1000

200

time

400

600

800

1000

time

Fig. 9. The response of x1 of the IBN

Fig. 10. The response of x2 of the ISN

2

2

1.5

1.5

1 1

x

x

0.5 0.5

0 0 −0.5 −0.5

−1 −1.5 0

200

400

600

time

800

1000

−1 0

200

400

600

800

1000

time

Fig. 11. The response of x1 of the IBN- Fig. 12. The response of x1 of the IBNIBN coupling with gs = 0.05 IBN coupling with gs = 1

the coupling strength gs = 0.05. In this case, two IBNs synchronize as bursting neurons. Figure 12 shows the response of the membrane potentials in the coupling of two same IBNs with the coupling strength gs = 1. Two IBNs synchronize as spiking neurons.

Firing Pattern Estimation of Biological Neuron Models

5

89

Adaptive Observer with Full States

We present the parameter estimation problem to distinguish the firing patterns by using early-time dynamic behaviors. In this section, assuming that the full states are measurable, we present an adaptive observer to estimate all parameters in the single HR neuron. 5.1

Construction of Adaptive Observer

We present an adaptive observer as ˆ ˆ˙ = W (w ˆ − w) + h(w) + Ξ θ (O0 ) : w ˆ is an estimate of the ˆ = [ˆ where w x, yˆ, zˆ] is an estimate of the states, θ unknown parameters and W is selected as a stable matrix. Using the standard adaptive control theory[11], the parameter update law is given by ˆ˙ = Γ Ξ T P (w − w). ˆ θ where P is a positive definite solution of the following Lyapunov equation for a positive definite matrix Q: W T P + P W = −Q. 5.2

Numerical Examples

We will show the simulation results of single IBN case. The parameters in the tonic bursting are given by a = 2.8, α = 1.6, c = 5, b = 9, μ = 0.001, I = 0.05. The parameters of the adaptive observers are selected as W = −10I3 , Γ = diag{100, 50, 300}. Figure 13 shows the estimation behavior of a (solid line) and I (dotted line). The estimates a ˆ and Iˆ converge to the true values of a and I.

6

Adaptive Observer with a Partial State

We assume that the membrane potential x is available, but the others are immeasurable. In this case, we consider following problems: – Estimate y and z using the available signal x; – Estimate the parameter a or I to distinguish the firing patterns by using early-time dynamic behaviors.

90

K. Mitsunaga, Y. Totoki, and T. Matsuo

6.1

Construction of Adaptive Observer

The parameters a and I are key parameters that determine the firing pattern. The HR model can be rewritten by the following three forms[9]: ˙ = Aw + h1 (x) + b1 (x2 a) (S1 ) : w ˙ = Aw + h2 (x) + b2 (I) (S2 ) : w ˙ = Aw + h3 (x) + b2 (θ T ξ) (S3 ) : w where

(1) (2) (3)

⎡

⎤ ⎡ 3 ⎤ ⎡ ⎤ ⎡ ⎤ 0 −1 −1 −x + I 1 1 A = ⎣ 0 −1 0 ⎦ , h1 = ⎣ αx2 ⎦ , b1 = ⎣ 1 ⎦ , b2 = ⎣ 0 ⎦ , μb 0 −μ μc 0 0 ⎡ 3 ⎤ ⎡ 3⎤ 2   −x + ax −x a x2 2 ⎦ 2 ⎦ ⎣ ⎣ h2 = (a + α)x , h3 = δx ,θ = ,ξ = . I 1 μc μc

In (S1 ) and (S2 ), the unknown parameters are assumed to be a and I, respectively. In (S3 ), we assume that the parameter δ = a + α is known, and a and I are unknown. Since the measurable signal is x, the output equation is given by   x = cw = 1 0 0 w We present adaptive observers that estimate the parameter for each system (Si ), i = 1, 2, 3, as follows: ˆ 1 + h1 (x) + b1 (x2 a (O1 ) : wˆ˙ 1 = Aw ˆ) + g(x − x ˆ) ˙ ˆ ˆ 2 + h2 (x) + b2 (I) + g(x − x (O2 ) : wˆ2 = Aw ˆ) T ˙ ˆ ˆ 2 + h3 (x) + b2 (θ ξ) + g(x − xˆ) (O3 ) : wˆ2 = Aw

(4) (5) (6)

where g is selected such that A − gc is a stable. Since (A, b1 , c) and (A, b2 , c) are strictly positive real, the parameter estimation laws are given as ˙ a ˆ˙ = γ1 x2 (x − x ˆ), Iˆ = γ2 (x − xˆ).

(7)

Using the Kalman-Yakubovich (KY) lemma, we can show the asymptotic stability of the error system based on the standard adaptive control theory[11]. 6.2

Numerical Examples

We will show the simulation results of single IBN case. The parameters in the tonic spiking are same as in the previous simulation. Figures 14 and 15 show the estimated parameters by the adaptive observers (O1 ) and (O2 ), respectively. Figures 16 and 17 show the responses of y (solid line) and its estimate yˆ (dotted line) the adaptive observers (O1 ) for t ≤ 500 and for t ≤ 20, respectively. Figure 18 shows the responses of z (solid line) and its estimate zˆ (dotted line) by the adaptive observers (O1 ). The simulation results of other cases are omitted. The states and parameters can be asymptotically estimated.

Firing Pattern Estimation of Biological Neuron Models

5

5

a ˆ Iˆ

4.5

91

4.5

4 4 3.5 3.5 3

a ˆ

a ˆ Iˆ

3 2.5 2

2.5

1.5 2 1 1.5

0.5 0 0

50

100

150

1 0

200

20

40

time

60

80

100

time

ˆ Fig. 13. a ˆ(solid line) and I(dotted line) in the adaptive observer (O0 ) with full states

Fig. 14. a ˆ(solid line) in the adaptive observer (O1 ) with x

0.1

7

y yˆ

6

0

5

−0.1

y yˆ

Iˆ

4

−0.2

3

−0.3 2

−0.4

−0.5 0

1

20

40

60

80

0 0

100

100

200

300

400

500

time

time

ˆ Fig. 15. I(solid line) in the adaptive observer (O2 ) with x

Fig. 16. y(solid line) and yˆ(dottedline) in the adaptive observer (O1 ) (t ≤ 500)

7

0

y yˆ

z zˆ

6 5

−0.5

y yˆ

z zˆ

4 3

−1 2 1 0 0

5

10

15

20

time

Fig. 17. y(solid line) and yˆ(dottedline) in the adaptive observer (O1 ) (t ≤ 20)

−1.5 0

5

10

15

20

time

Fig. 18. z(solid line) and zˆ(dottedline) in the adaptive observer (O1 ) (t ≤ 20)

92

7

K. Mitsunaga, Y. Totoki, and T. Matsuo

Conclusion

We presented estimators of the parameters of the HR model using the adaptive observer technique with the output measurement data such as the membrane potential. The proposed observers allow us to distinguish the firing pattern in early time and to recover the immeasurable internal states.

References 1. Belykh, I., de Lange, E., Hasler, M.: Synchronization of Bursting Neurons: What Matters in the Network Topology. Phys. Rev. Lett. 94, 101–188 (2005) 2. Izhikevich, E.M.: Simple Model of Spiking Neurons. IEEE Trans on Neural Networks 14(6), 1569–1572 (2003) 3. Izhikevich, E.M.: Which model to use for cortical spiking neurons? IEEE Trans on Neural Networks 15(5), 1063–1070 (2004) 4. Watts, L.: A Tour of NeuraLOG and Spike - Tools for Simulating Networks of Spiking Neurons (1993), http://www.lloydwatts.com/SpikeBrochure.pdf 5. Hindmarsh, J.L., Rose, R.M.: A model of the nerve impulse using two first order differential equations. Nature 296, 162–164 (1982) 6. Hindmarsh, J.L., Rose, R.M.: A model of neuronal bursting using three coupled first order differential equations. Proc. R. Soc. Lond. B. 221, 87–102 (1984) 7. Carroll, T.L.: Chaotic systems that are robust to added noise, CHAOS, 15, 013901 (2005) 8. Meunier, N., Narion-Poll, R., Lansky, P., Rospars, J.O.: Estimation of the Individual Firing Frequencies of Two Neurons Recorded with a Single Electrode. Chem. Senses 28, 671–679 (2003) 9. Yu, H., Liu, Y.: Chaotic synchronization based on stability criterion of linear systems. Physics Letters A 314, 292–298 (2003) 10. Marino, R.: Adaptive Observers for Single Output Nonlinear Systems. IEEE Trans. on Automatic Control 35(9), 1054–1058 (1990) 11. Narendra, K.S., Annaswamy, A.M.: Stable Adaptive Systems. Prentice Hall Inc., Englewood Cliffs (1989) 12. Arena, P., Fortuna, L., Frasca, M., Rosa, M.L.: Locally active Hindmarsh-Rose neurons. Chaos, Soliton and Fractals 27, 405–412 (2006) 13. Tokuda, I., Parlitz, U., Illing, L., Kennel, M., Abarbanel, H.: Parameter estimation for neuron models. In: Proc. of the 7th Experimental Chaos Conference (2002), http://www.physik3.gwdg.de/∼ ulli/pdf/TPIKA02 pre.pdf 14. Steur, E.: Parameter Estimation in Hindmarsh-Rose Neurons (2006), http://alexandria.tue.nl/repository/books/626834.pdf 15. Fujikawa, H., Mitsunaga, K., Suemitsu, H., Matsuo, T.: Parameter Estimation of Biological Neuron Models with Bursting and Spiking. In: Proc. of SICE-ICASE International Joint Conference 2006 CD-ROM, pp. 4487–4492 (2006)

Thouless-Anderson-Palmer Equation for Associative Memory Neural Network Models with Fluctuating Couplings Akihisa Ichiki and Masatoshi Shiino Department of Applied Physics, Faculty of Science, Tokyo Institute of Technology, 2-12-2 Ohokayama Meguro-ku Tokyo, Japan

Abstract. We derive Thouless-Anderson-Palmer (TAP) equations and order parameter equations for stochastic analog neural network models with fluctuating synaptic couplings. Such systems with finite number of neurons originally have no energy concept. Thus they defy the use of the replica method or the cavity method, which require the energy concept. However for some realizations of synaptic noise, the systems have the effective Hamiltonian and the cavity method becomes applicable to derive the TAP equations.

1

Introduction

The replica method [1] for random spin systems has been successfully employed in neural network models of associative memory to have the order parameters and the storage capacity [2] and the cavity method [3] has been employed to derive the Thouless-Anderson-Palmer (TAP) equations [4,5]. However these techniques require the energy concept. On the other hand, various types of neural network models which have no energy concept, such as networks with temporally fluctuating synaptic couplings, may exist. The alternative approach to the replica method to derive the order parameter equations, called the self-consistent signal-to-noise analysis (SCSNA), is closely related to the cavity concept in the case where networks have free energy [6,7]. An advantage to apply the SCSNA to neural networks is that the energy concept is not required to derive the order parameter equations once the TAP equations are obtained. The SCSNA, which was originally proposed for deriving a set of order parameter equations for deterministic analog neural networks, becomes applicable to stochastic networks by noting that the TAP equations define the deterministic networks. Furthermore, the coefficients of the Onsager reaction terms characteristic to the TAP equations which determine the form of the transfer functions in analog networks are selfconsistently obtained through the concept shared by the cavity method and the SCSNA. Thus the TAP equations as well as the order parameter equations are derived self-consistently by the hybrid use of the cavity method and the SCSNA in the case where the energy concept exists. However the networks with synaptic noise, which have no energy concept, defy the use of the cavity method to have the TAP equations. On the other hand, as in [8], the network with a specific M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 93–101, 2008. c Springer-Verlag Berlin Heidelberg 2008 

94

A. Ichiki and M. Shiino

involvement of synaptic noise can be analyzed by the cavity method to derive the TAP equation in the thermodynamic limit, since the energy concept appears as an effective Hamiltonian in this limit. It is natural to consider such neural network models with fluctuating synaptic couplings, since the synaptic couplings in real biological systems are updated by learning rules and the time-sequence of the synaptic couplings may be stochastic under the influence of noisy external stimuli. Thus the study on such networks is required to understand the retrieval process of the realistic networks. The aim of this paper is two-fold: (i) we will investigate the networks with which realization of synaptic noise have the energy concept to apply the cavity method to derive the TAP equations, (ii) we will show the TAP equations for the networks when the concept of the effective Hamiltonian appears. This paper is organized as the follows: in the next section, we will briefly review how the energy concept appears in the network with synaptic noise and derive the TAP equations and the order parameter equations by using the cavity method and the SCSNA [8]. Once the effective Hamiltonian is found, the replica method is also applicable to derive the order parameter equations. However in the present paper, to make clear the relationship between the TAP equations and the order parameter equations, we do not use the replica trick. In section 3, we will investigate the cases where the energy concept appears in the networks with synaptic noise. We will see that the TAP equations and the order parameter equations for some models can be derived in the framework mentioned in section 2. We will also mention that some difficulties to derive the TAP equations arise in some models with other involvements of synaptic noise. In the last section, we will make discussions on the structure of the TAP equations for the network with temporally fluctuating synaptic noise and conclude this paper.

2

Brief Review on Effective Hamiltonian, TAP Equations and Order Parameter Equations

In this section, we briefly review the cavity method becomes applicable to the network with fluctuating synaptic couplings [8]. Then we derive the TAP equations and the order parameter equations self-consistently in the framework of the SCSNA. In this section, we deal with the following stochastic analog neural network of N neurons with temporally fluctuating synaptic noise (multiplicative noise):  x˙ i = −φ (xi ) + Jij (t)xj + ηi (t), (1) j(=i)

ηi (t)ηj (t ) = 2Dδij δ(t − t ),

(2)

where xi (i = 1, · · · , N ) represents a state of the neuron at site i taking a continuous value, φ(xi ) is a potential of an arbitrary form which  determines the probability distribution of xi in the case without the input j(=i) Jij xj , ηi the Langevin white noise with its noise intensity 2D and Jij (t) the synaptic coupling.

TAP Equation for Associative Memory Neural Network Models

95

We note here that, in the case of associative memory neural network, the synaptic coupling Jij is usually defined by the well-known Hebb learning rule. However, in the present paper, we will deal with the coupling Jij fluctuating around the Hebb rule with a white noise: Jij (t) = J¯ij + ij (t), ˜ 2D ij (t)kl (t ) = δik δjl δ(t − t ), N

(3) (4)

p where J¯ij is defined by the usual Hebb learning rule J¯ij ≡ N1 μ=1 ξiμ ξjμ with p = αN the number of patterns embedded in the network, ξiμ = ±1 is the μth embedded pattern at neuron i, and ij (t) denotes the synaptic noise independent of ηi (t), which we assume in the present model as a white noise with its intensity ˜ . 2D/N Using Ito integral, we obtain the Fokker-Planck equation corresponding to the Langevin equation (1) as ⎧ ⎫ N   ∂ ⎬   ∂P (t, x) ∂ ⎨  ˜q =− −φ (xi ) + J¯ij xj − D + Dˆ P (t, x), (5) ∂t ∂xi ⎩ ∂xi ⎭ i=1

j(=i)

 where qˆ ≡ N1 j(=i) x2j . Since the self-averaging property holds in the thermodynamic limit N → ∞, qˆ is identified as qˆ =

N 1  2 x . N i=1 i

(6)

The order parameter qˆ is obtained self-consistently in our framework as seen below. Supposing qˆ is given, one can easily find the equilibrium probability density for the Fokker-Planck equation (5) as ⎧ ⎛ ⎞⎫ N ⎨ ⎬   PN (x) = Z −1 exp −βeff ⎝ φ(xi ) − J¯ij xi xj ⎠ , (7) ⎩ ⎭ i=1

i<j

where Z denotes the normalization constant and −1 ˜ qˆ βeff ≡D+D

(8)

plays the role of the effective temperature of the network. The temperature of the system is modified as a consequence of the multiplicative noise and it depends on the order parameter qˆ. Notice here that the equilibrium distribution of the system becomes Gibbs distribution in the thermodynamic limit N → ∞. The equilibrium solution for equation (5) in the finite N -body system differs√from the  indeed 1 2 2 probability density (7). However, since N1 x − x  = O(1/ N ), the j(=i) j N j j difference between the probability densities in the finite N -body system PN√and in the system in the thermodynamic system PN →∞ is PN →∞ − PN = O(1/ N ).

96

A. Ichiki and M. Shiino

Thus one can conclude that the equilibrium density for equation (5) converges to the probability density (7) in the thermodynamic limit N → ∞. Since we have explicitly written down the equilibrium probability density (7) as a Gibbsian form, one can define the effective Hamiltonian of (sufficiently large) N -body system as N 

HN ≡

i=1

φ(xi ) −



J¯ij xi xj .

(9)

i<j

Since we have found the effective Hamiltonian and the effective temperature, one can apply the usual cavity method [3] to this system and derive the TAP equation. According to the cavity method, we divide the Hamiltonian of N -body system (9) into that of (N −1)-body system and the part involving the state of ith neuron as HN = φ(xi ) − hi xi + HN −1 ,  where hi ≡ j(=i) J¯ij xj is the local field at site i and the Hamiltonian of (N −1)  body system HN −1 is given as HN −1 ≡ j(=i) φ(xj ) − j
˜ −1

=Z

j(=i)

exp { −βeff [φ(xi ) − hi xi ]} PN −1 (hi ),

where Z˜ is the normalization constant and PN −1 (hi ) denotes the probability density of the local field hi in the (N − 1)-body system defined as ⎡ ⎤ ⎛ ⎞    −1 ⎣ PN −1 (hi ) ≡ ZN dxj ⎦ δ ⎝hi − J¯ij xj ⎠ exp [−βeff HN −1 ] , −1 j(=i)

j(=i)

where ZN −1 denotes the normalization constant. Since the local field is given as the summation of a sufficiently large number of random variables and their cross√ correlations are expected to be O(1/ N ), one can expect that PN −1 (hi ) turns out to be a Gaussian density in the thermodynamic limit N → ∞ according to the central limit theorem:   1 (hi − hi N −1 )2 PN −1 (hi ) = √ exp − , 2σ 2 2πσ 2 where ·N −1 represents the thermal average with respect to the (N − 1)-body probability density PN −1 (x) and σ 2 is the variance of PN −1 (hi ), which is evaluated later self-consistently in the framework of the SCSNA. Then taking the

TAP Equation for Associative Memory Neural Network Models

97

average of xi with respect to the marginal probability PN (xi , hi ) straightforwardly yields xi  = F (hi N −1 ), where F is a transfer function defined as     2 dx x exp −βeff φ(x) − yx − βeff2σ x2    . F (y) ≡  2 dx exp −βeff φ(x) − yx − βeff2σ x2

(10)

(11)

Similarly hi N −1 is obtained as hi N −1 = hi  − βeff σ 2 xi . Thus we have the pre-TAP equation ⎛ ⎞  xi  = F ⎝ J¯ij xj  − ΓOns xi ⎠ ,

(12)

j(=i)

where ΓOns ≡ βeff σ 2 . Since the concrete form of the transfer function F depends on the effective temperature βeff and the variance of the local field σ 2 , it is necessary to obtain βeff and σ 2 to have the TAP equation [7,9]. Then we can apply the SCSNA to equation (12) to determine βeff and σ 2 selfconsistently. We assume that the only one condensed pattern {ξi1 } is retrieved. N Using the overlap order parameter mμ ≡ N1 i=1 ξiμ xi  and the concept of the SCSNA [7,9], we have 

hi  = ξi1 m1 + ξiμ mμ + ziμ + ΓSCSNA xi ,

(13)

where ν≥2 ξiν mν = ξiμ mμ + ziμ + γxi , ΓSCSNA ≡ γ − α and ziμ is a Gaussian random variable with zero mean. We will evaluate the overlap mμ selfconsistently and obtain ziμ and γ. Substituting equation (13) into the pre-TAP equation (12) reads   xi  = F ξi1 m1 + ξiμ mμ + ziμ + (ΓSCSNA − ΓOns )xi  and comparing this equation with equation (10) yields [7] ΓSCSNA = ΓOns , since hi N −1 is considered to be a Gaussian random variable √ which should not contain the Onsager reaction term. Noting that mμ = O(1/ N ) for μ ≥ 2, one can obtain the overlap for noncondensed patterns as  μ 1 ξ F (ξj1 m1 + zjμ ), N (1 − U ) j=1 j

(14)

N 1   1 1 F (ξj m + zjμ ), N j=1

(15)

N

mμ = U≡

98

A. Ichiki and M. Shiino

where F  denotes the derivative of the transfer function F and the order expan√ sion of F with respect to 1/ N has been applied to xi  = F (ξi1 m1 +ziμ +ξiμ mμ ). Using equation (14) and the definitions of ziμ and γ, one finds α , 1−U  1 = N (1 − U )

γ= ziμ



ξiν ξjν F (ξj1 m1 + zjν ).

ν(=1,μ) j(=i)

Thus the variance of ziμ is evaluated as σz2 =

 2 α F (ξm1 + z) 2 (1 − U )

ξ,z

,

(16)

where ·ξ,z represents the average over a random variables ξ = ±1 and the Gaussian variable z, and the self-averaging property has been used. Similarly one obtains the set of order parameter equations as  m1 = ξF (ξm1 + z) ξ,z , (17)   1 U = F (ξm + z) ξ,z , (18) ΓOns = ΓSCSNA =

αU . 1−U

(19)

For the present model, it is required to evaluate the order parameter qˆ to determine the form of the transfer function F . Since qˆ is related to the susceptibility and, by definition of F (11), the order parameter U corresponds with the susceptibility as U = βeff (x2  − x2 ), one finds qˆ =

U (1 − U )2 2 + σz . βeff α

(20)

The set of equations (16), (17), (18), (19), (20) takes a closed form and one can determine the form of F self-consistently as well as the set of order parameters. Therefore substituting into the pre-TAP equation (12) the solutions βeff and ΓOns that are self-consistently obtained within this framework yields the TAP equation.

3

Models with Effective Hamiltonian and Effective Temperature

In this section, we examine the applicability of the concept shown in the previous section for the network models with various types of involvement of synaptic noise. As is shown in the previous section, the network has the effective Hamiltonian when the equilibrium density becomes Gibbsian one in the thermodynamic limit. Since the forms of the equilibrium probability densities are estimated by investigating the forms of the Fokker-Planck equations corresponding to the

TAP Equation for Associative Memory Neural Network Models

99

original models, we will see the concrete forms of the drift and diffusion coefficients of the Fokker-Planck equations in the thermodynamic limit. Once the Fokker-Planck equation for each model is obtained, the form of the equilibrium distribution is estimated and hence we can know whether the network has energy concept in the thermodynamic limit. We deal with the following models: the state of ith neuron xi obeys the dynamics (1) and (2). The synaptic couplings Jij are given in Table 1: Table 1.

case

Jij (t)

noise

(i)

J¯ij + ij (t)

(ii)

J¯ij + i (t)j (t)

(iii)

J¯ij + i (t)˜ j (t)

(iv)

J¯ij (1 + ij (t)) J¯ij (1 + j (t))

˜ ij (t)kl (t ) = 2δik δjl δ(t − t )/βN ! ˜ δij δ 1/2 (t − t ) i (t)j (t ) = 2/βN √ ˜ γ δij δ 1/2 (t − t ) i (t)j (t ) = 2/βN √  ˆ 1−γ δij δ 1/2 (t − t ) ˜ i (t)˜ j (t ) = 2/βN  i (t)˜ j (t ) = 0 ij (t)kl (t ) = 2δik δjl δ(t − t )/β˜

i (t)j (t ) = 2δij δ(t − t )/β˜ ! ˜ ij δ 1/2 (t − t ) (vi) J¯ij (1 + i (t)j (t)) i (t)j (t ) = 2/βδ √ ˜ γ i (t)j (t ) = 2δij δ 1/2 (t − t )/βN √ ¯  1/2  ˆ (vii) Jij (1 + i (t)˜ j (t)) ˜ i (t)˜ j (t ) = 2δij δ (t − t )/βN −γ  i (t)˜ j (t ) = 0 ˜ (viii) J¯ij + j (t) i (t)j (t ) = 2δij δ(t − t )/βN ˜ (ix) J¯ij + i (t) i (t)j (t ) = 2δij δ(t − t )/βN (v)

(xi)

J¯ij (1 + i (t)) J¯ij + (t)

i (t)j (t ) = 2δij δ(t − t )/β˜ ˜ (t)(t ) = 2δ(t − t )/βN

(xii)

J¯ij (1 + (t))

(t)(t ) = 2δ(t − t )/β˜

(x)

Note that, in Table 1, · denotes the average with respect to the noise, β and β˜ relate with the noise intensity of i and ˜i respectively and γ arbitrary real number. The case (i) has been treated in the previous section. (2) Using Ito integral, we can straightforwardly find the diffusion coefficient Dij for each model as shown in Table 2. In Table √ 2, the order parameters qˆ and M  are defined as qˆ ≡ i x2i /N , M ≡ i xi / N and the local field is defined as  (1) (1) hi ≡ j(=i) J¯ij xj . The drift coefficients Di for all cases are same as Di =  −φ (xi ) + j(=i) J¯ij xj . Thus there exists the energy concept or the effective Hamiltonian for the mod(2) els (i)-(vii). In these models, the effective temperature 1/βeff = Dii also exits.

100

A. Ichiki and M. Shiino

Therefore the TAP equations and the order parameter equations for these models can be obtained in the framework described in the previous section. However, in (2) the case (viii), the diffusion coefficient Dij contains non-zero off-diagonal elements and hence the equilibrium distribution does not take the form of the Gibbs distribution. This difficulty occurs also in the cases (xi) and (xii). In the case (ix), the effective Hamiltonian and the effective temperature exist. However, in this case, evaluating the quantity M is required to determine the effective temperature. For the quantity M the self-averaging property does not hold and thus the effective temperature can not be found in the framework mentioned previously. In the case (x), the local field hi itself is the thermally fluctuating random variable, and hence the dynamics for the local fields hi ’s are also required to estimate the effective temperature. Note that the equilibrium distribution for this model may not become the Gibbs distribution since the dynamics of the local fields hi ’s should be also considered. In the case (xi), there exist the offdiagonal elements in the diffusion coefficient and it depends on M , which can not be evaluated by the self-averaging property. In the case (xii), the off-diagonal elements on the diffusion coefficient also exist and it depends on the temporally fluctuating random variables hi ’s. Therefore deriving the TAP equations and the order parameter equations for the models (viii)-(xii) is the future problem. Table 2. (2)

case

Dij

energy concept

(i)

˜ ij (1/β + qˆ/β)δ ˜ ij (1/β + qˆ/β)δ

exist

(ii)

exist

˜ ij (vi) (1/β + αˆ q /β)δ ˆ ij (vii) (1/β + αˆ q /β˜β)δ (viii) δij /β + qˆ/β˜

exist

exist exist exist not exist

˜ ij (ix) (1/β + M /β)δ 2 ˜ (x) (1/β + h /β)δij

not exist

δij /β + M 2 /β˜ (xii) δij /β + hi hj /β˜

not exist

2

i

(xi)

4

exist

ˆ ij (iii) (1/β + qˆ/β˜β)δ ˜ ij (iv) (1/β + αˆ q /β)δ ˜ ij (v) (1/β + αˆ q /β)δ

not exist not exist

Conclusion

In this paper we have derived the TAP equations and the order parameter equations for the neural network models with synaptic noise. In the models presented

TAP Equation for Associative Memory Neural Network Models

101

in this paper, the concept of energy or the Hamiltonian does not exist in the original finite N -body system. As we have seen in the previous section, the effective Hamiltonian and the effective temperature exist in the thermodynamic limit and then the TAP equations together with the order parameter equations are derived by the use of the cavity method and the SCSNA as shown in section 2 for the cases (i)-(vii). For the cases (viii)-(xii), however, some difficulties mentioned in the previous section arise. To derive the TAP equations and the order parameter equations for these cases remains as the future problem. One of the authors (A. I.) was supported by the 21st Century COE Program at Tokyo Tech ”Nanometer-Scale Quantum Physics” by the Ministry of Education, Culture, Sports, Science and Technology.

References 1. Sherrington, D., Kirkpatrick, S.: Solvable Model of a Spin-Glass. Phys. Rev. Lett. 35, 1792–1796 (1975) 2. Amit, D.J., Geutfreund, H., Sompolinsky, H.: Storing Infinite Numbers of Patterns in a Spin-Glass Model of Neural Networks. Phys. Rev. Lett. 55, 1530–1533 (1985) 3. M´ezard, M., Parisi, G., Virasoro, M.A.: Spin Glass Theory and Beyond. World Scientific, Singapore (1987) 4. Thouless, D.J., Anderson, P.W., Palmer, R.G.: Solution of ’solvable model of a spin glass’. Philos. Mag. 35, 593–601 (1977) 5. Morita, T., Horiguchi, T.: Exactly solvable model of a spin glass. Solid. State. Comm. 19, 833–835 (1976) 6. Shiino, M., Fukai, T.: Self-consistent signal-to-noise analysis and its application to analogue neural networks with asymmetric connections. J. Phys. A: Math. Gen. 25, L375-L381 (1992) 7. Shiino, M., Yamana, M.: Statistical mechanics of stochastic neural networks: Relationship between the self-consistent signal-to-noise analysis. Thouless-AndersonPalmer equation, and replica symmetric calculation approaches. Phys. Rev. E. 69, 011904-1-13 (2004) 8. Ichiki, A., Shiino, M.: Thouless-Anderson-Palmer equation for analog neural network with temporally fluctuating white synaptic noise. J. Phys. A: Math. Theor. 40, 9201– 9211 (2007) 9. Shiino, M., Fukai, T.: Self-consistent signal-to-noise analysis of the statistical behavior of analog neural networks and enhancement of the storage capacity. Phys. Rev. E. 48, 867–897 (1993)

Spike-Timing Dependent Plasticity in Recurrently Connected Networks with Fixed External Inputs Matthieu Gilson1,2,3 , David B. Grayden1,2,3 , J. Leo van Hemmen4 , Doreen A. Thomas1,3 , and Anthony N. Burkitt1,2,3 1

Department of Electrical and Electronic Engineering, The University of Melbourne 2 The Bionic Ear Institute, Melbourne, Australia 3 NICTA, The Victorian Research Lab, The University of Melbourne 4 Physik Department, die Technische Universit¨ at M¨ unchen, Germany [email protected]

Abstract. This paper investigates spike-timing dependent plasticity (STDP) for recurrently connected weights in a network with fixed external inputs (homogeneous Poisson pulse trains). We use a dynamical system to model the network activity and predict its asymptotic evolution, which turns out to qualitatively depend on the learning parameters and the correlation structure of the inputs. Our predictions are supported by numerical simulations of Poisson neuron networks in general cases as well as for certain cases when using Integrate-And-Fire (IF) neurons. Keywords: Spiking neurons, neurodynamics, learning, STDP, dynamical system, recurrent network.

1

Introduction

The hypothesis that changes in the efficacies of neuronal connections depend upon the correlations in timing of pre- and postsynaptic action potentials (or spikes) has received considerable experimental support [1]. Such a learning mechanism is related to the notion of Hebbian learning [6] and can implement functional organisation in neural networks. The understanding of the underlying mechanisms behind learning in the brain is crucial both from a physiological level and for applications (eg. neural prostheses, robotics). Spike-timing dependent plasticity (STDP, [3]) is a fruitful candidate for this mechanism, that has been analysed in previous studies for feed-forward network architectures of Poisson neurons [7,8]. This theory was recently extended to the case of recurrently connected architectures, and applied to a fully connected network with no external input [2]. This paper pursues the development of this framework [2] based on the linear Poisson neuron model and additive STDP (as described in Sec. 2). The network activity is characterised by the firing rates of and the correlations between the neurons. The evolution of the network activity (or neural dynamics) is described by a dynamical system (as discussed in Sec. 3). It is analysed in terms of M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 102–111, 2008. c Springer-Verlag Berlin Heidelberg 2008 

Spike-Timing Dependent Plasticity in Recurrently Connected Networks

103

equilibria (or fixed points) and stability in order to predict the asymptotic network behaviour when stimulated by steady external inputs (Sec. 4).

2 2.1

Models Linear Poisson Neuron

In this study, we use the linear Poisson neuron [7]. The neuronal output firing mechanism is approximated by an inhomogeneous Poisson process generated by the Poisson parameter ρ(t) (it determines the probability of spiking an action potential, considered as an instantaneous event); ρ(t) evolves according to the sum of all the synaptic contributions (cf. Fig. 1.A)     ρ(t) = ν0 + Jk (t) (t − tk,n − dk ) . (1) k

n

As in [7], the total synaptic influx is the temporal and spatial (over the synapses) summation of the post-synaptic potentials (PSP) modelled by the synaptic response kernel  weighted by the synaptic efficacies Jk (or weights; note that we only consider positive weights here).  models the current injected into the postsynaptic neuron induced by one single spike (indexed by n) at time tk,n at the k th synaptic connection, after the fixed delay dk . Finally, ν0 is the spontaneous firing rate caused by background activity (identical for all the neurons). 2.2

Additive STDP

Gerstner et al. [3] first proposed a framework to study additive STDP, where the change of synaptic weight Jk induced by a sole pair of pre- and post-synaptic pulses at times (tk , tout ) is ΔJk ∝ win + wout + W (tk − tout ).

(2)

Jk is the weight of the k th synapse. win and wout are the rate-based learning parameters (one contribution per pulse at the pulse time); W is the STDP learning window (cf. Fig. 1.B) so that a synaptic weight is potentiated for a given pulse pair if a presynaptic input precedes a postsynaptic spike (i.e. “takes part” in the output firing), and is depressed otherwise [3]. Usually, such changes are scaled by a learning rate η. 2.3

Link with the Physiology

In our abstract model, the neurons are excited by external inputs, which convey “neural information” that is assumed to be encoded in their firing rate and correlation structures. The neurons also have a spontaneous activity that can be understood as a consequence of a background activity. There are thus two sources of stochasticity, which can be interpreted as follows: one is related to the external inputs (pulse trains described by their firing rate and correlation

104

M. Gilson et al.

Fig. 1. A: Linear Poisson neuron model. The output pulse train S(t) (top) is generated using the inhomogeneous Poisson parameter ρ(t) (middle). Each pre-synaptic pulse at the kth synapse (bottom) induces a variation of ρ determined by the post-synaptic response kernel , the synaptic weight Jk (t) and the delay dk . B: STDP window function W (in arbitrary unit). Each side of the real axis is determined by a negative exponential: u → cx exp(−|u|/τx ) with the index x = D for the depression part (u > 0) and x = P for the potentiation part (u < 0), cf. [2, Sec. 5] for details and the parameter values.

structures) and it can be linked to the variability observed in physiological data; the other is due to the additional impact of the background activity upon the generation of the spike output of each neuron (intrinsic stochasticity modelled by the Poisson process).

3 3.1

Theoretical Analysis Characterisation of the Neural Activity

We consider a network of N Poisson neurons (referred to as internal neurons) stimulated with M Poisson pulse trains (or external inputs), as shown in Fig. 2. The activity of the neural network can be described using the firing rates and the pairwise correlations, plus the weights. In the terminology of the theory of dynamical systems, these variables characterise the “state” of the network activity at each time t and their evolution is referred to as neural dynamics. Similar to [7,2], we consider the time-averaged firing rates νi (t) (for the internal neuron indexed by i; T is a given time period)  1 t νi (t)  Si (t ) dt (3) T t−T where Si (t) is the spike-time series of the ith neuron and the brackets . . . denotes the ensemble averaging. Likewise for the coefficient correlations (timeaverage correlation function convoluted with the STDP window function W ): W Dik (t) between the ith internal neuron and the k th external input   t   +∞ 1 W Dik (t)  W (u) Si (t )Sˆk (t + u) dt du (4) T t−T −∞ th and QW and j th internal neurons (with Si and Sj ) [2, Sec. 3]. ij (t) between the i

Spike-Timing Dependent Plasticity in Recurrently Connected Networks

105

Fig. 2. Presentation of the network and the notation. The internal neurons (within the network) are indexed by i ∈ [1..N ], and their output pulse trains are denoted Si (t) (it can be understood as a sum of “Dirac functions” at each spiking time [2, Sec. 2]). Likewise for the external input pulse trains Sˆk (t) (k ∈ [1..M ]). The time-averaged firing rates are denoted by νi (t), cf. Eq. 3; the correlation coefficients within the network by Qij (t) (resp. Dik (t) between a neuron in the network and an external input; and ˆ kl (t) between two external inputs), cf. Eq. 4. The weight of the connection from the Q kth external input onto the ith internal neuron is denoted Kik (t) (resp. Jij (t) from the j th internal neuron onto the ith internal neuron).

The stimulation parameters (which represent the “information” carried in the external inputs) are determined by the time-averaged input spiking rates (ˆ νk (t), ˆ W (t), defined simidefined similarly to νi (t)) and their correlation coefficients (Q kl W larly to Dik (t) and QW ij (t)) [2, Sec. 3]. In this paper, we only consider stimulation parameters that are constant in time. 3.2

Learning Equations

Learning equations can be derived from Eq. 2 as in Kempter et al. [7]. This requires the assumption that the internal pulse trains are statistically independent (this can be considered valid when the number of recurrent connection is large enough) and a small learning rate η. This leads to the matrix equation Eq. 10 for the weights between internal neurons (resp. to Eq. 9 for the input weights K). 3.3

Activation Dynamics

In order to study the evolution of the weights described by Eqs. 9 and 10, we need to evaluate the neuron time-average firing rates (the vector ν(t)) and their time-average correlation coefficients (the matrices DW (t) and QW (t)). Similar to Kempter et al. [7] and Burkitt et al. [2], we approximate the instantaneous firing rate Si (t) of the ith Poisson neuron by its expected inhomogeneous Poisson parameter ρ(t) (cf. Eq. 1) and we neglect the impact of the short-time dynamics (the synaptic response kernel  and the synaptic delays dˆik and dij ) by using time averaged variables (over a “long” period T ). We require that the

106

M. Gilson et al.

learning occurs slowly compared to the activation mechanisms (cf. the neuron and synapse models) so that T is large compared to the time scale of these mechanisms but small compared to the inverse of the learning parameter η −1 . This leads to the consistency matrix equations of the firing rates (Eq. 6) and of the correlation coefficients (Eqs. 7 and 8). See Burkitt et al. [2, Sec. 3] for details of the derivation. Note that the consistency equations of the correlation coefficients as defined in [2, Sec. 3 and 4] have been reformulated to express the usual covariance using the assumption that the correlations are quasi-constant in time [5] (this implies ˆ V T [2, Sec. 3] is actually equal to Q ˆ W ). Eqs. 7 and 8 express the impact that Q of the connectivity (through the term [I − J]−1 K) on the internal firing rates and the cross covariances in terms of the input covariance Cˆ W

ˆW − W  νˆ νˆT , Cˆ W  Q

(5)

  W (u) du evaluates the balance between the potentiation and the where W depression of our STDP rule. These equations are obtained by combining the equations in [2] with the firing-rate consistency equation Eq. 6. 3.4

Network Dynamical System

Putting everything together, the network dynamics is described by

 ν = [I − J]−1 ν0 E + K νˆ (6)

  ν νˆT = [I − J]−1 K Q ˆW − W  νˆ νˆT DW − W (7)

  ν ν T = [I − J]−1 K Q ˆW − W  νˆ νˆT K T [I − J]−1T QW − W (8)

 dK ˆ T + DW = ΦK win E νˆT + wout ν E (9) dt

 dJ = ΦJ win E ν T + wout ν E T + QW , (10) dt where E is the unit vector of N elements (likewise for Eˆ with M elements); ΦJ is a projector on the space of N × N matrices to which J belongs [2, Sec. 3]. The effect of ΦJ is to nullify the coefficients corresponding to a missing connection in the network, viz. all the diagonal terms because the self-connection of a neuron onto itself is forbidden. More generally, such an projection operator can account for any network connectivity [2, Sec. 3]. Note that time has been rescaled, in order to remove η from these equations and for simplicity of notation the dependence on time t will be omitted in the rest of this paper. The matrix [I − J(t)] is assumed invertible at all time (the contrary would imply a diverging behaviour of the firing rates [2, Sec. 4]).

4

Recurrent Network with Fixed Input Weights

We now examine the case of a recurrently connected network with fixed input weights K and learning on the recurrent weights J. Only Eqs. 6, 8 and 10 remain,

Spike-Timing Dependent Plasticity in Recurrently Connected Networks

107

which simplifies the analysis. In the case of full recurrent connectivity, ΦJ in Eq. 10 only nullifies the diagonal terms of the square matrix in its argument. 4.1

Analytical Predictions

Homeostatic equilibrium. Similarly to [7,2], we derive the scalar equations of the mean firing rate νav  N −1 i νi and weight Jav  (N (N − 1))−1 i=j Jij . It consists of neglecting the inhomogeneities of the firing rates and of the weights over the network, as well as of the connectivity, and we obtain ν0 + (K νˆ)av 1 − (N − 1)Jav  

in  +C  ν2 , = w + wout νav + W av

νav = J˙av

(11)

 is defined as where (K νˆ)av denotes the mean of the matrix product K νˆ and C ˆW T   (K C K )av . C 2 [ν0 + (K νˆ)av ]

(12)

Eqs. 11 are thus the same equations as for the case with no input [2, Sec. 5], with  replaced by, resp., ν0 + (K νˆ)av and W  + C.  It qualitatively implies the ν0 and W   < 0, the network exhibits same dynamical behaviour and, provided that W + C a homeostatic equilibrium (when it is realisable, it requires in particular μ > 0), and the means of the firing rates and of the weights converge towards win + wout +C  W μ − ν0 − (K νˆ)av = , (N − 1)μ

∗ νav =μ  − ∗ Jav

(13)

If the correlation inputs are time-invariant functions and if the correlations are positive (i.e. more likely to fire in a synchronous way) and homogeneous  is of the same sign as W . among the correlated input pool, it follows that C  < 0 as in [7,2]. Therefore, the condition for stability reverts to W Structural dynamics. For the particular case of uncorrelated inputs (or more generally when K Cˆ W K T = 0), the fixed-point structure of the dynamical system is qualitatively the same as for the case of no external input [2, Sec. 5]: a homogeneous distribution for the firing rates and a continuous manifold of fixed points for the internal weights. In the case of “spatial” inhomogeneities over the input correlations, the network dynamics shows a different evolution. To illustrate and to compare this with the case of feed-forward connections, we consider the network configuration described in Fig. 3 and inspired by [7], where one input pool has correlation while the other pool has uncorrelated sources. In general, the equations that

108

M. Gilson et al.

Fig. 3. Architecture of the simulated network. The inputs are divided into two subpools, each feeding half of the internal network with means K1 and K2 for the input weights from each subpool. Similarly to the recurrent weights J and the firing rates νˆ and ν, the inhomogeneities are neglected within each subpool and they are assumed to be all identical to their mean. The weights and delays are initially set with 10% random variation around a mean.

determine the fixed points have no accurate solution. Yet, we can reduce the dimensionality by neglecting the variance within each subpool and make approximations to evaluate the asymptotic distribution of the firing rates, which turns out to be bimodal. 4.2

Simulation Protocol and Results

We simulated a network of Poisson neurons as described in Fig. 3 with random initial recurrent weights (uniformly distributed in a given interval, as well as all the synaptic delays). An issue with such simulations is to maintain positive internal weights during their homeostatic convergence because they individually diverge. Thus, all equilibria are not realisable depending on the initial distributions of K and J and the weight bounds [7,2]. See [2, Sec. 5] for details about the simulation parameters. An interesting first case to test the analytical predictions consists of two pools of uncorrelated inputs, each feeding half of the network with distinct weights (K1 νˆ1 = K2 νˆ2 and Cˆ W = 0). Each half of the internal network thus has distinct firing rates initially and, as predicted, the outcome is a convergence of these firing-rates towards a uniform value (similar to the case of no external input [2, Sec. 5]). In the case of a full connected (for both the K and the J according to Fig. 3) network stimulated by one correlated input pool (short-time correlation inspired by [4] so that Cˆ W = 0) and one uncorrelated one, both the internal firing rates and the internal weights also exhibit a homeostatic equilibrium. As shown in Fig. 4, the means over the network (thick solid lines) converge towards the predicted equilibrium values (dashed lines). Furthermore, the individual firing rates tend to stabilise and their distribution remains bimodal (the subpool #1 excited by  < 0). The recurrent correlated inputs fires at a lower rate eventually when W weights individually diverge similarly to [2, Sec. 5] and reorganise so that the outgoing weights from the subpool #1 (see the means over each weight subpool

Spike-Timing Dependent Plasticity in Recurrently Connected Networks 25

0.03

15

0.02 0.015 J11

0.01

#1

#2

J21

0.025

#2

weights

firing rates (Hz)

#1 20

10 0

109

0.005 10000

0 0

20000

10000

J22 J12 20000

time (sec)

time (sec)

Fig. 4. Evolution of the firing rates (left) and of the recurrent weights (right) for N = 30 fully-connected Poisson neurons (cf. Fig.3) with short-time correlated inputs. The outcome is a quasi bimodal distribution of the firing rates (the grey bundle, the mean is the thick solid line) around the predicted homeostatic equilibrium (dashed line). The subgroup #1 that receives correlated inputs is more excited initially but fires at a lower rate at the end of the simulation (cf. the two thin black solid lines which represent the mean over each subpool). The internal weights individually diverge, while their mean (thick line) converges towards the predicted equilibrium value (dashed line). They reorganise themselves so that the weights outgoing from the subpool #2 (that receives uncorrelated inputs) become silent, while the ones from #1 are strengthened. Note that the homeostatic equilibrium is preserved even when some weights saturate.

firing rates (Hz)

25 #1

#2

20

15 #2 0

#1 10000

20000

time (sec) Fig. 5. Evolution of the firing rates for a partially connected network of N = 75 neurons. Both the K and the J have 40% probability of connection with the same setup as the network in Fig. 4 (to preserve the total input synaptic strength). The mean firing rate (very thick line) still converges towards the predicted equilibrium value (dashed line) and the two subgroups (grey bundles, each mean represented by a thin black solid line) get separated similarly to the case of full connectivity. The internal weights (not shown) exhibit similar dynamics as for the case of full connectivity.

J11 and J21 in Fig. 4) are strengthened while the other ones almost become silent (see J12 and J22 ). In other words, the subpool that receives correlated inputs takes the upper hand in the recurrent architecture.

110

M. Gilson et al.

30

firing rates (Hz)

firing rates (Hz)

30

25

20

15 0

10000

time (sec)

25

20

15 0

10000

time (sec)

Fig. 6. Simulation of networks of IF neurons with partial random connectivity of 50%. The network qualitatively exhibits the expected behaviour in the case of uncorrelated inputs (left) and inputs with one correlated pool and one uncorrelated pool (right).

In the case of partial connectivity for both the K and the J, the behaviour of the individual firing rates (cf. Fig. 5) still follows the predictions but they are more dispersed, their convergence is slower and the bimodal distribution is not always observed as clearly as in the case of full connectivity (in Fig. 5 the means of the two internal neuron subpools clearly remain separated though). The homeostatic equilibrium of the internal weights also holds and they individually diverge. The partial connectivity needs to be rich enough for the predictions of the mean variable to be accurate enough. First results with IF neurons comply with the analytical predictions remain valid even if the activation mechanisms are more complex (here with a connectivity of 50%, cf. Fig. 6).

5

Discussion and Future Work

The analytical results presented here are preliminary, and further investigation is needed to gain better understanding of the interplay between the input correlation structure and STDP. Nevertheless, our results illustrate two points: STDP induces a stable activity in recurrent architectures similar to that for feed-forward ones (homeostatic regulation of the network activity under the con < 0); and the qualitative structure of the internal firing rates is mainly dition W determined by the input correlation structure. Namely, a “poor” correlation structure (uncorrelated or delta-correlated inputs, so that Cˆ W = 0) induces a homogenisation of the firing activity. Finally, partial connectivity impacts upon the structure of the internal firing rates, but such networks still exhibit similar behaviour to fully connected ones. ˆ W K T suggest Preliminary results involving more complex patterns of K Q more complex interplay between the input correlation structure and the equilibrium distribution of the network firing rates and weights. Such cases are under investigation and may constitute more “interesting” dynamic behaviour of the network from a cognitive modelling point of view, namely through the

Spike-Timing Dependent Plasticity in Recurrently Connected Networks

111

relationship between the attractors of the network activity and the input structure. The case of learning for both the input connections and the recurrent ones will also form part of a future study. Comparison with IF neurons suggests that the impact of the neuron activation mechanisms on the weight dynamics may not be significant. It can be linked to the separation of the time scales between them in the case of slow learning. Note that with our approximations the IF neurons are assumed to be in “linear input-output regime” (no bursting for instance).

Acknowledgments The authors thank Iven Mareels, Chris Trengove, Sean Byrnes and Hamish Meffin for useful discussions that introduced significant improvements. MG is funded by two scholarships from The University of Melbourne and NICTA. ANB and DBG acknowledge funding from the Australian Research Council (ARC Discovery Projects #DP0453205 and #DP0664271) and The Bionic Ear Institute.

References 1. Bi, G.Q., Poo, M.M.: Synaptic modification by correlated activity: Hebb’s postulate revisited. Annual Review of Neuroscience 24, 139–166 (2001) 2. Burkitt, A.N., Gilson, M., van Hemmen, J.L.: Spike-timing-dependent plasticity for neurons with recurrent connections. Biological Cybernetics 96, 533–546 (2007) 3. Gerstner, W., Kempter, R., van Hemmen, J.L., Wagner, H.: A neuronal learning rule for sub-millisecond temporal coding. Nature 383, 76–78 (1996) 4. Gutig, R., Aharonov, R., Rotter, S., Sompolinsky, H.: Learning input correlations through nonlinear temporally asymmetric hebbian plasticity. Journal of Neuroscience 23, 3697–3714 (2003) 5. Hawkes, A.G.: Point spectra of some mutually exciting point processes. Journal of the Royal Statistical Society Series B-Statistical Methodology 33, 438–443 (1971) 6. Hebb, D.O.: The organization of behavior: a neuropsychological theory. Wiley, Chichester (1949) 7. Kempter, R., Gerstner, W., van Hemmen, J.L.: Hebbian learning and spiking neurons. Physical Review E 59, 4498–4514 (1999) 8. van Rossum, M.C.W., Bi, G.Q., Turrigiano, G.G.: Stable hebbian learning from spike timing-dependent plasticity. Journal of Neuroscience 20, 8812–8821 (2000)

A Comparative Study of Synchrony Measures for the Early Detection of Alzheimer’s Disease Based on EEG Justin Dauwels, Fran¸cois Vialatte, and Andrzej Cichocki RIKEN Brain Science Institute, Saitama, Japan [email protected], {fvialatte,cia}@brain.riken.jp

Abstract. It has repeatedly been reported in the medical literature that the EEG signals of Alzheimer’s disease (AD) patients are less synchronous than in age-matched control patients. This phenomenon, however, does at present not allow to reliably predict AD at an early stage, so-called mild cognitive impairment (MCI), due to the large variability among patients. In recent years, many novel techniques to quantify EEG synchrony have been developed; some of them are believed to be more sensitive to abnormalities in EEG synchrony than traditional measures such as the cross-correlation coefficient. In this paper, a wide variety of synchrony measures is investigated in the context of AD detection, including the cross-correlation coefficient, the mean-square and phase coherence function, Granger causality, the recently proposed corr-entropy coefficient and two novel extensions, phase synchrony indices derived from the Hilbert transform and time-frequency maps, information-theoretic divergence measures in time domain and timefrequency domain, state space based measures (in particular, non-linear interdependence measures and the S-estimator), and at last, the recently proposed stochastic-event synchrony measures. For the data set at hand, only two synchrony measures are able to convincingly distinguish MCI patients from age-matched control patients (p < 0.005), i.e., Granger causality (in particular, full-frequency directed transfer function) and stochastic event synchrony (in particular, the fraction of non-coincident activity). Combining those two measures with additional features may eventually yield a reliable diagnostic tool for MCI and AD.

1

Introduction

Many studies have shown that the EEG signals of AD patients are generally less coherent than in age-matched control patients (see [1] for an in-depth review). It is noteworthy, however, that this effect is not always easily detectable: there tends to be a large variability among AD patients. This is especially the case for patients in the pre-symptomatic phase, commonly referred to as Mild Cognitive Impairment (MCI), during which neuronal degeneration is occurring prior to the clinical symptoms appearance. On the other hand, it is crucial to predict AD at M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 112–125, 2008. c Springer-Verlag Berlin Heidelberg 2008 

A Comparative Study of Synchrony Measures for the Early Detection of AD

113

an early stage: medication that aims at delaying the effects of AD (and hence intend to improve the quality of life of AD patients) are the most effective if applied in the pre-symptomatic phase. In recent years, a large variety of measures has been proposed to quantify EEG synchrony (we refer to [2]–[5] for recent reviews on EEG synchrony measures); some of those measures are believed to be more sensitive to perturbations in EEG synchrony than classical indices as for example the cross-correlation coefficient or the coherence function. In this paper, we systematically investigate the state-ofthe-art of measuring EEG synchrony with special focus on the detection of AD in its early stages. (A related study has been presented in [6,7] in the context of epilepsy.) We consider various synchrony measures, stemming from a wide spectrum of disciplines, such as physics, information theory, statistics, and signal processing. Our aim is to investigate which measures are the most suitable for detecting the effect of synchrony perturbations in MCI and AD patients; we also wish to better understand which aspects of synchrony are captured by the different measures, and how the measures are related to each other. This paper is structured as follows. In Section 2 we review the synchrony measures considered in this paper. In Section 3 those measures are applied to EEG data, in particular, for the purpose of detecting MCI; we describe the EEG data set, elaborate on various implementation issues, and present our results. At the end of the paper, we briefly relate our results to earlier work, and speculate about the neurophysiological interpretation of our results.

2

Synchrony Measures

We briefly review the various families of synchrony measures investigated in this paper: cross-correlation coefficient and analogues in frequency and time-frequency domain, Granger causality, phase synchrony, state space based synchrony, information theoretic interdependence measures, and at last, stochastic-event synchrony measures, which we developed in recent work. 2.1

Cross-Correlation Coefficient

The cross-correlation coefficient r is perhaps one of the most well-known measures for (linear) interdependence between two signals x and y. If x and y are not linearly correlated, r is close to zero; on the other hand, if both signals are identical, then r = 1 [8]. 2.2

Coherence

The coherence function quantifies linear correlations in frequency domain. One distinguishes the magnitude square coherence function c(f ) and the phase coherence function φ(f ) [8].

114

2.3

J. Dauwels, F. Vialatte, and A. Cichocki

Corr-Entropy Coefficient

The corr-entropy coefficient rE is a recently proposed [9] non-linear extension of the correlation coefficient r; it is close to zero if x and y are independent (which is stronger than being uncorrelated). 2.4

Coh-Entropy and Wav-Entropy Coefficient

One can define a non-linear magnitude square coherence function, which we will refer to as “coh-entropy” coefficient cE (f ); it is an extension of the corr-entropy coefficient to the frequency domain. The corr-entropy coefficient rE can also be extended to the time-frequency domain, by replacing the signals x and y in the definition of rE by their time-frequency (“wavelet”) transforms. In this paper, we use the complex Morlet wavelet, which is known to be well-suited for EEG signals [10]. The resulting measure is called “wav-entropy” coefficient wE (f ). (To our knowledge, both cE (f ) and wE (f ) are novel). 2.5

Granger Causality

Granger causality1 refers to a family of synchrony measures that are derived from linear stochastic models of time series; as the above linear interdependence measures, they quantify to which extent different signals are linearly interdependent. Whereas the above linear interdependence measures are bivariate, i.e., they can only be applied to pairs of signals, Granger causality measures are multivariate, they can be applied to multiple signals simultaneously. Suppose that we are given n signals X1 (k), X2 (k), . . . , Xn (k), each stemming from a different channel. We consider the multivariate autoregressive (MVAR) model: X(k) =

p 

A(j)X(k − ) + E(k),

(1)

=1 

where X(k) = (X1 (k), X2 (k), . . . , Xn (k))T , p is the model order, the model coefficients A(j) are n × n matrices, and E(k) is a zero-mean Gaussian random vector of size n. In words: Each signal Xi (k) is assumed to linearly depend on its own p past values and the p past values of the other signals Xj (k). The deviation between X(k) and this linear dependence is modeled by the noise component E(k). Model (1) can also be cast in the form: E(k) =

p 

˜ A(j)X(k − ),

(2)

=0 1

The Granger causality measures we consider here are implemented in the BioSig library, available from http://biosig.sourceforge.net/

A Comparative Study of Synchrony Measures for the Early Detection of AD

115

 ˜ ˜ where A(0) = I (identity matrix) and A(j) = −A(j) for j > 0. One can transform (2) into the frequency domain (by applying the z-transform and by  substituting z = e−2πiΔt , where 1/Δt is the sampling rate):

˜ −1 (f )E(f ) = H(f )E(f ). X(f ) = A 

(3)

The power spectrum matrix of the signal X(k) is determined as S(f ) = X(f )X(f )∗ = H(f )VH∗ (f ), 

(4)

where V stands for the covariance matrix of E(k). The Granger causality measures are defined in terms of coefficients of the matrices A, H, and S. Due to space limitations, only a short description of these methods is provided here, additional information can be found in existing literature (e.g., [4]). From these coefficients, two symmetric measures can be defined: – Granger coherence |Kij (f )| ∈ [0, 1] describes the amount of in-phase components in signals i and j at the frequency f . – Partial coherence (PC) |Cij (f )| ∈ [0, 1] describes the amount of in-phase components in signals i and j at the frequency f when the influence (i.e., linear dependence) of the other signals is statistically removed. The following asymmetric (“directed”) Granger causality measures capture causal relations: 2 – Directed transfer function (DTF) γij (f ) quantifies the fraction of inflow to channel i stemming from channel j. – Full frequency directed transfer function (ffDTF) |Hij (f )|2  Fij2 (f ) =  m ∈ [0, 1], 2 f j=1 |Hij (f )|

(5)

2 is a variation of γij (f ) with a global normalization in frequency. – Partial directed coherence (PDC) |Pij (f )| ∈ [0, 1] represents the fraction of outflow from channel j to channel i  2 – Direct directed transfer function (dDTF) χ2ij (f ) = Fij2 (f )Cij (f ) is non-zero if the connection between channel i and j is causal (non-zero Fij2 (f )) and 2 direct (non-zero Cij (f )).

2.6

Phase Synchrony

Phase synchrony refers to the interdependence between the instantaneous phases φx and φy of two signals x and y; the instantaneous phases may be strongly synchronized even when the amplitudes of x and y are statistically independent. The instantaneous phase φx of a signal x may be extracted as [11]: 

φH x(k)] , x (k) = arg [x(k) + i˜

(6)

116

J. Dauwels, F. Vialatte, and A. Cichocki

where x ˜ is the Hilbert transform of x. Alternatively, one can derive the instantaneous phase from the time-frequency transform X(k, f ) of x: 

φW x (k, f ) = arg[X(k, f )].

(7)

The phase φW x (k, f ) depends on the center frequency f of the applied wavelet. By appropriately scaling the wavelet, the instantaneous phase may be computed in the frequency range of interest. The phase synchrony index γ for two instantaneous phases φx and φy is defined as [11]:   γ =  ei(nφx −mφy )  ∈ [0, 1], (8) where n and m are integers (usually n = 1 = m). We will use the notation γH and γW to indicate whether the instantaneous phases are computed by the Hilbert transform or time-frequency transform respectively. In this paper, we will consider two additional phase synchrony indices, i.e., the evolution map approach (EMA) and the instantaneous period approach (IPA) [12]. Due to space constraints, we will not describe those measures here, instead we refer the reader to [12]2 ; additional information about phase synchrony can be found in [6]. 2.7

State Space Based Synchrony

State space based synchrony (or “generalized synchronization”) evaluates synchrony by analyzing the interdependence between the signals in a state space reconstructed domain (see e.g., [7]). The central hypothesis behind this approach is that the signals at hand are generated by some (unknown) deterministic, potentially high-dimensional, non-linear dynamical system. In order to reconstruct such system from a signal x, one considers delay vectors X(k) = (x(k), x(k − τ ), . . . , x(k − (m − 1) τ ))T , where m is the embedding dimension and τ denotes the time lag. If τ and m are appropriately chosen, and the signals are indeed generated by a deterministic dynamical system (to a good approximation), the delay vectors lie on a smooth manifold (“mapping”) in Rm , apart from small stochastic fluctuations. The S-estimator [13], here denoted by Sest , is a state space based measure obtained by applying principal component analysis (PCA) to delay vectors3 . We also considered three measures of nonlinear interdependence, S k , H k , and N k (see [6] for details4 ). 2.8

Information-Theoretic Measures

Several interdependence measures have been proposed that have their roots in information theory. Mutual Information I is perhaps the most well-known 2 3

4

Program code is available at www.agnld.uni-potsdam.de/%7Emros/dircnew.m We used the S Toolbox downloadable from http://aperest.epfl.ch/docs/ software.htm Software is available from http://www.vis.caltech.edu/~rodri/software.htm

A Comparative Study of Synchrony Measures for the Early Detection of AD

117

information-theoretic interdependence measure; it quantifies the amount of information the random variable Y contains about random variable X (and vice versa); it is always positive, and it vanishes when X and Y are statistically independent. Recently, a sophisticated and effective technique to compute mutual information between time series was proposed [14]; we will use that method in this paper5 . The method of [14] computes mutual information in time-domain; alternatively, this quantity may also be determined in time-frequency domain (denoted by IW ), more specifically, from normalized spectrograms [15,16] (see also [17,18]). We will also consider several information-theoretic measures that quantify the dissimilarity (or “distance”) between two random variables (or signals). In contrast to the previously mentioned measures, those divergence measures vanish if the random variables (or signals) are identical ; moreover, they are not necessarily symmetric, and therefore, they can not be considered as distance measures in the strict sense. Divergences may be computed in time domain and timefrequency domain; in this paper, we will only compute the divergence measures in time-frequency domain, since the computation in time domain is far more involved. We consider the Kullback-Leibler divergence K, the R´enyi divergence Dα , the Jensen-Shannon divergence J, and the Jensen-R´enyi divergence Jα . Due to space constraints, we will not review those divergence measures here; we refer the interested reader to [15,16]. 2.9

Stochastic Event Synchrony (SES)

Stochastic event synchrony, an interdependence measure we developed in earlier work [19], describes the similarity between the time-frequency transforms of two signals x and y. As a first step, the time-frequency transform of each signal is approximated as a sum of (half-ellipsoid) basis functions, referred to as “bumps” (see Fig. 1 and [20]). The resulting bump models, representing the most prominent oscillatory activity, are then aligned (see Fig. 2): bumps in one timefrequency map may not be present in the other map (“non-coincident bumps”); other bumps are present in both maps (“coincident bumps”), but appear at slightly different positions on the maps. The black lines in Fig. 2 connect the centers of coincident bumps, and hence, visualize the offset in position between pairs of coincident bumps. Stochastic event synchrony consists of five parameters that quantify the alignment of two bump models: – ρ: fraction of non-coincident bumps, – δt and δf : average time and frequency offset respectively between coincident bumps, – st and sf : variance of the time and frequency offset respectively between coincident bumps. The alignment of the two bump models (cf. Fig. 2 (right)) is obtained by iterative max-product message passing on a graphical model; the five SES parameters are determined from the resulting alignment by maximum a posteriori (MAP) 5

The program code (in C) is available at www.klab.caltech.edu/~kraskov/MILCA/

118

J. Dauwels, F. Vialatte, and A. Cichocki

Fig. 1. Bump modeling Coincident bumps (ρ = 27%) 30

25

25

20

20

15

15

f

f

Bump models of two EEG signals 30

10

10

5

5

00

200

400

t

600

800

00

200

400

t

600

800

Fig. 2. Coincident and non-coincident activity (“bumps”); (left) bump models of two signals; (right) coincident bumps; the black lines connect the centers of coincident bumps

estimation [19]. The parameters ρ and st are the most relevant for the present study, since they quantify the synchrony between bump models (and hence, the original time-frequency maps); low ρ and st implies that the two time-frequency maps at hand are well synchronized.

3

Detection of EEG Synchrony Abnormalities in MCI Patients

In the following section, we describe the EEG data we analyzed. In Section 3.2 we address certain technical issues related to the synchrony measures, and in Section 3.3, we present and discuss our results. 3.1

EEG Data

The EEG data6 analyzed here have been analyzed in previous studies concerning early detection of AD [21]–[25]. They consist of rest eyes-closed EEG data 6

We are grateful to Prof. T. Musha for providing us the EEG data.

A Comparative Study of Synchrony Measures for the Early Detection of AD

119

recorded from 21 sites on the scalp based on the 10–20 system. The sampling frequency was 200 Hz, and the signals were band pass filtered between 4 and 30Hz. The subjects comprised two study groups. The first consisted of a group of 25 patients who had complained of memory problems. These subjects were then diagnosed as suffering from MCI and subsequently developed mild AD. The criteria for inclusion into the MCI group were a mini mental state exam (MMSE) score above 24, the average score in the MCI group was 26 (SD of 1.8). The other group was a control set consisting of 56 age-matched, healthy subjects who had no memory or other cognitive impairments. The average MMSE of this control group was 28.5 (SD of 1.6). The ages of the two groups were 71.9 ± 10.2 and 71.7 ± 8.3, respectively. Pre-selection was conducted to ensure that the data were of a high quality, as determined by the presence of at least 20 sec. of artifact free data. Based on this requirement, the number of subjects in the two groups described above was reduced to 22 MCI patients and 38 control subjects. 3.2

Methods

In order to reduce the computational complexity, we aggregated the EEG signals into 5 zones (see Fig. 3); we computed the synchrony measures (except the S-estimator) from the averages of each zone. For all those measures except SES, we used the arithmetic average; in the case of SES, the bump models obtained from the 21 electrodes were clustered into 5 zones by means of the aggregation algorithm described in [20]. We evaluated the S-estimator between each pair of zones by applying PCA to the state space embedded EEG signals of both zones. We divided the EEG signals in segments of equal length L, and computed the synchrony measures by averaging over those segments. Since spontaneous EEG is usually highly non-stationary, and most synchrony measures are strictly speaking only applicable to stationary signals, the length L should be sufficiently small; on the other hand, in order to obtain reliable measures for synchrony, the length should be chosen sufficiently large. Consequently, it is not a priori clear how to choose the length L, and therefore, we decided to test several values, i.e., L = 1s, 5s, and 20s. In the case of Granger causality measures, one needs to specify the model order p. Similarly, for mutual information (in time domain) and the state space based measures, the embedding dimension m and the time lag τ needs to be chosen; the phase synchrony indices IPA and EMA involve a time delay τ . Since it is not obvious which parameter values amount to the best performance for detecting AD, we have tried a range of parameter settings, i.e., p = 1, 2,. . . , 10, and m = 1, 2,. . . , 10; the time delay was in each case set to τ = 1/30s, which is the period of the fastest oscillations in the EEG signals at hand. 3.3

Results and Discussion

Our main results are summarized in Table 1, which shows the sensitivity of the synchrony measures for detecting MCI. Due to space constraints, the table only shows results for global synchrony, i.e., the synchrony measures were averaged

120

J. Dauwels, F. Vialatte, and A. Cichocki

Fp1

Fpz

Fp2

1 F7

F3

T3

3 C3

T5

F4

Fz

2

T4

C4

Cz

Pz

P3

F8

4

P4

T6

5 O1

Oz

O2

Fig. 3. The 21 electrodes used for EEG recording, distributed according to the 10– 20 international placement system [8]. The clustering into 5 zones is indicated by the colors and dashed lines (1 = frontal, 2 = left temporal, 3 = central, 4 = right temporal and 5 = occipital).

over all pairs of zones. (Results for local synchrony and individual frequency bands will be presented in a longer report, including a detailed description of the influence of various parameters such as model order and embedding dimension on the sensitivity.) The p-values, obtained by the Mann-Whitney test, need strictly speaking to be Bonferroni corrected; since we consider many different measures simultaneously, it is likely that a few of those measures have small p-values merely due to stochastic fluctuations (and not due to systematic difference between MCI and control patients). In the most conservative Bonferroni post-correction, the p-values need to be divided by the number of synchrony measures. From the table, it can be seen that only a few measures evince significant differences in EEG synchrony between MCI and control patients: full-frequency DTF and ρ are the most sensitive (for the data set at hand), their p-values remain significant (pcorr < 0.05) after Bonferroni correction. In other words, the effect of MCI and AD on EEG synchrony can be detected, as was reported earlier in the literature; we will expand on this issue in the following section. In other to gain more insight in the relation between the different measures, we calculated the correlation between them (see Fig. 5; red and blue indicate strong correlation and anti-correlation respectively). From this figure, it becomes strikingly clear that the majority of measures are strongly correlated (or anticorrelated) with each other; in other words, the measures can easily be classified in different families. In addition, many measures are strongly (anti-)correlated with the classical cross-correlation coefficient r, the most basic measure; as a result, they do not provide much additional information regarding EEG synchrony. Measures that are only weakly correlated with the cross-correlation coefficient include the phase synchrony indices, Granger causality measures, and stochastic-event synchrony measures; interestingly, those three families of synchrony measures are mutually uncorrelated, and as a consequence, they each seem to capture a specific kind of interdependence.

A Comparative Study of Synchrony Measures for the Early Detection of AD

121

Table 1. Sensitivity of synchrony measures for early prediction of AD (p-values for Mann-Whitney test; * and ** indicate p < 0.05 and p < 0.005 respectively) Measure

Cross-correlation

Coherence

Phase Coherence

Corr-entropy

Wave-entropy

p-value

0.028∗

0.060

0.72

0.27

0.012∗

References Measure

[8]

[9] PDC

DTF

ffDTF

dDTF

0.15

0.16

0.60

0.34

0.0012∗∗

0.030∗

Measure

Kullback-Leibler

R´enyi

Jensen-Shannon

Jensen-R´enyi

IW

I

p-value

0.072

0.076

0.084

0.12

0.060

0.080

Measure

Nk

Sk

Hk

S-estimator

p-value

0.032∗

0.29

0.090

0.33

p-value

Granger coherence Partial Coherence

References

[4]

References

[15]

References

[14]

[6]

Measure

Hilbert Phase

p-value

0.15

[13]

Wavelet Phase

Evolution Map Instantaneous Period

0.082

References

0.020∗

0.072

[6]

[12]

Measure

st

ρ

p-value

0.92

0.00029∗∗

In Fig. 4, we combine the two most sensitive synchrony measures (for the data set at hand), i.e., full-frequency DTF and ρ. In this figure, the MCI patients are fairly well distinguishable from the control patients. As such, the separation is not sufficiently strong to yield reliable early prediction of AD. For this purpose, the two features need to be combined with complementary features, for example, derived from the slowing effect of AD on EEG, or perhaps from different modalities such as PET, MRI, DTI, or biochemical indicators. On the other hand, we remind the reader of the fact that in the data set at hand, patients did not carry out any specific task; moreover, the recordings were short (only 20s). It is plausible that the sensitivity of EEG synchrony could be further improved by increasing the length of the recordings and by recording the EEG before, while, and after patients carry out specific tasks, e.g., working memory tasks.

0.5 MCI CTR

0.45 0.4

ρ

0.35 0.3 0.25 0.2 0.15 0.045

0.05

Fij2

0.055

Fig. 4. ρ vs. ffDTF

0.06

122

J. Dauwels, F. Vialatte, and A. Cichocki

state space

corr/coh mut inf

phase

divergence

Granger

SES

N k (X|Y ) N k (Y |X) S k (X|Y ) S k (Y |X) H k (X|Y ) 5 H k (Y |X) Sest r c rE 10 wE IW I γH γW 15 φ EMA IPA K(Y |X) K(X|Y )20 K Dα J Jα Kij 25 Cij Pij γij Fij χij st ρ

0.8

0.6

0.4

0.2

0

−0.2

−0.4

−0.6

30

−0.8 5

10

15

20

25

30

Fig. 5. Correlation between the synchrony measures

4

Conclusions

In previous studies, brain dynamics in AD and MCI patients were mainly investigated using coherence (cf. Section 2.2) or state space based measures of synchrony (cf. Section 2.7). During working memory tasks, coherence shows significant effects in AD and MCI groups [26] [27]; in resting condition, however, coherence does not show such differences in low frequencies (below 30Hz), neither between AD and controls [28] nor between MCI and controls [27]. These results are consistent with our observations. In the gamma range, coherence seems to decrease with AD [29]; we did not investigate this frequency range, however, since the EEG signals analyzed here were band pass filtered between 4 and 30Hz. Synchronization likelihood, a state space based synchronization measure similar to the non-linear interdependence measures S k , H k , and N k (cf. Section 2.7), is believed to be more sensitive than coherence to detect changes in AD patients [28]. Using state space based synchrony methods, significant differences were found between AD and control in rest conditions [28] [30] [32] [33]. State space based synchrony failed to retrieve significant differences between MCI patient and control subjects on a global level [32] [33], but significant effects were observed locally: fronto-parietal electrode synchronization likelihood progressively decreased through MCI and mild AD groups [30]. We report here a lower p-value for the state space based synchrony measure N k (p = 0.032) than for coherence (p = 0.06); those low p-values, however, would not be statistically significant after Bonferroni correction.

A Comparative Study of Synchrony Measures for the Early Detection of AD

123

By means of Global Field Synchronization, a phase synchrony measure similar to the ones we considered in this paper, Koenig et al. [31] observed a general decrease of synchronization in correlation with cognitive decline and AD. In our study, we analyzed five different phase synchrony measures: Hilbert and wavelet based phase synchrony, phase coherence, evolution map approach (EMA), and instantaneous period approach (IPA). The p-value of the latter is low (p=0.020), in agreement with the results of [31], but it would be non-significant after Bonferroni correction. The strongest observed effect is a significantly higher degree of local asynchronous activity (ρ) in MCI patients, more specifically, a high number of noncoincident, asynchronous oscillatory events (p = 0.00029). Interestingly, we did not observe a significant effect on the timing jitter st of the coincident events (p = 0.92). In other words, our results seem to indicate that there is significantly more non-coincident background activity, while the coincident activity remains well synchronized. On the one hand, this observation is in agreement with previous studies that report a general decrease of neural synchrony in MCI and AD patients; on the other hand, it goes beyond previous results, since it yields a more subtle description of EEG synchrony in MCI and AD patients: it suggests that the loss of coherence is mostly due to an increase of (local) non-coincident background activity, whereas the locked (coincident) activity remains equally well synchronized. In future work, we will verify this conjecture by means of other data sets.

References 1. Jong, J.: EEG Dynamics in Patients with Alzheimer’s Disease. Clinical Neurophysiology 115, 1490–1505 (2004) 2. Pereda, E., Quiroga, R.Q., Bhattacharya, J.: Nonlinear Multivariate Analysis of Neurophsyiological Signals. Progress in Neurobiology 77, 1–37 (2005) 3. Breakspear, M.: Dynamic Connectivity in Neural Systems: Theoretical and Empirical Considerations. Neuroinformatics 2(2) (2004) 4. Kami´ nski, M., Liang, H.: Causal Influence: Advances in Neurosignal Analysis. Critical Review in Biomedical Engineering 33(4), 347–430 (2005) 5. Stam, C.J.: Nonlinear Dynamical Analysis of EEG and MEG: Review of an Emerging Field. Clinical Neurophysiology 116, 2266–2301 (2005) 6. Quiroga, R.Q., Kraskov, A., Kreuz, T., Grassberger, P.: Performance of Different Synchronization Measures in Real Data: A Case Study on EEG Signals. Physical Review E 65 (2002) 7. Sakkalis, V., Giurc˘ aneacu, C.D., Xanthopoulos, P., Zervakis, M., Tsiaras, V.: Assessment of Linear and Non-Linear EEG Synchronization Measures for Evaluating Mild Epileptic Signal Patterns. In: Proc. of ITAB 2006, Ioannina-Epirus, Greece, October 26–28 (2006) 8. Nunez, P., Srinivasan, R.: Electric Fields of the Brain: The Neurophysics of EEG. Oxford University Press, Oxford (2006) 9. Xu, J.-W., Bakardjian, H., Cichocki, A., Principe, J.C.: EEG Synchronization Measure: a Reproducing Kernel Hilbert Space Approach. IEEE Transactions on Biomedical Engineering Letters (submitted to, September 2006)

124

J. Dauwels, F. Vialatte, and A. Cichocki

10. Herrmann, C.S., Grigutsch, M., Busch, N.A.: EEG Oscillations and Wavelet Analysis. In: Handy, T. (ed.) Event-Related Potentials: a Methods Handbook, pp. 229– 259. MIT Press, Cambridge (2005) 11. Lachaux, J.-P., Rodriguez, E., Martinerie, J., Varela, F.J.: Measuring Phase Synchrony in Brain Signals. Human Brain Mapping 8, 194–208 (1999) 12. Rosenblum, M.G., Cimponeriu, L., Bezerianos, A., Patzak, A., Mrowka, R.: Identification of Coupling Direction: Application to Cardiorespiratory Interaction. Physical Review E, 65 041909 (2002) 13. Carmeli, C., Knyazeva, M.G., Innocenti, G.M., De Feo, O.: Assessment of EEG Synchronization Based on State-Space Analysis. Neuroimage 25, 339–354 (2005) 14. Kraskov, A., St¨ ogbauer, H., Grassberger, P.: Estimating Mutual Information. Phys. Rev. E 69(6), 66138 (2004) 15. Aviyente, S.: A Measure of Mutual Information on the Time-Frequency Plane. In: Proc. of ICASSP 2005, Philadelphia, PA, USA, March 18–23, vol. 4, pp. 481–484 (2005) 16. Aviyente, S.: Information-Theoretic Signal Processing on the Time-Frequency Plane and Applications. In: Proc. of EUSIPCO 2005, Antalya, Turkey, September 4–8 (2005) 17. Quiroga, Q.R., Rosso, O., Basar, E.: Wavelet-Entropy: A Measure of Order in Evoked Potentials. Electr. Clin. Neurophysiol (Suppl.) 49, 298–302 (1999) 18. Blanco, S., Quiroga, R.Q., Rosso, O., Kochen, S.: Time-Frequency Analysis of EEG Series. Physical Review E 51, 2624 (1995) 19. Dauwels, J., Vialatte, F., Cichocki, A.: A Novel Measure for Synchrony and Its Application to Neural Signals. In: Honolulu, H.U. (ed.) Proc. IEEE Int. Conf. on Acoustics and Signal Processing (ICASSP), Honolulu, Hawai’i, April 15–20 (2007) 20. Vialatte, F., Martin, C., Dubois, R., Haddad, J., Quenet, B., Gervais, R., Dreyfus, G.: A Machine Learning Approach to the Analysis of Time-Frequency Maps, and Its Application to Neural Dynamics. Neural Networks 20, 194–209 (2007) 21. Chapman, R., et al.: Brain Event-Related Potentials: Diagnosing Early-Stage Alzheimer’s Disease. Neurobiol. Aging 28, 194–201 (2007) 22. Cichocki, A., et al.: EEG Filtering Based on Blind Source Separation (BSS) for Early Detection of Alzheimer’s Disease. Clin. Neurophys. 116, 729–737 (2005) 23. Hogan, M., et al.: Memory-Related EEG Power and Coherence Reductions in Mild Alzheimer’s Disease. Int. J. Psychophysiol. 49 (2003) 24. Musha, T., et al.: A New EEG Method for Estimating Cortical Neuronal Impairment that is Sensitive to Early Stage Alzheimer’s Disease. Clin. Neurophys. 113, 1052–1058 (2002) 25. Vialatte, F., et al.: Blind Source Separation and Sparse Bump Modelling of TimeFrequency Representation of EEG Signals: New Tools for Early Detection of Alzheimer’s Disease. In: IEEE Workshop on Machine Learning for Signal Processing, pp. 27–32 (2005) 26. Hogan, M.J., Swanwick, G.R., Kaiser, J., Rowan, M., Lawlor, B.: Memory-Related EEG Power and Coherence Reductions in Mild Alzheimer’s Disease. Int. J. Psychophysiol. 49(2), 147–163 (2003) 27. Jiang, Z.Y.: Study on EEG Power and Coherence in Patients with Mild Cognitive Impairment During Working Memory Task. J. Zhejiang Univ. Sci. B 6(12), 1213– 1219 (2005) 28. Stam, C.J., van Cappellen van Walsum, A.M., Pijnenburg, Y.A., Berendse, H.W., de Munck, J.C., Scheltens, P., van Dijk, B.W.: Generalized Synchronization of MEG Recordings in Alzheimer’s Disease: Evidence for Involvement of the Gamma Band. J. Clin. Neurophysiol. 19(6), 562–574 (2002)

A Comparative Study of Synchrony Measures for the Early Detection of AD

125

29. Herrmann, C.S., Demiralp, T.: Human EEG Gamma Oscillations in Neuropsychiatric Disorders. Clinical Neurophysiology 116, 2719–2733 (2005) 30. Babiloni, C., Ferri, R., Binetti, G., Cassarino, A., Forno, G.D., Ercolani, M., Ferreri, F., Frisoni, G.B., Lanuzza, B., Miniussi, C., Nobili, F., Rodriguez, G., Rundo, F., Stam, C.J., Musha, T., Vecchio, F., Rossini, P.M.: Fronto-Parietal Coupling of Brain Rhythms in Mild Cognitive Impairment: A Multicentric EEG Study. Brain Res. Bull. 69(1), 63–73 (2006) 31. Koenig, T., Prichep, L., Dierks, T., Hubl, D., Wahlund, L.O., John, E.R., Jelic, V.: Decreased EEG Synchronization in Alzheimer’s Disease and Mild Cognitive Impairment. Neurobiol. Aging 26(2), 165–171 (2005) 32. Pijnenburg, Y.A., Made, Y.v., van Cappellen, A.M., van Walsum, Knol, D.L., Scheltens, P., Stam, C.J.: EEG Synchronization Likelihood in Mild Cognitive Impairment and Alzheimer’s Disease During a Working Memory Task. Clin. Neurophysiol. 115(6), 1332–1339 (2004) 33. Yagyu, T., Wackermann, J., Shigeta, M., Jelic, V., Kinoshita, T., Kochi, K., Julin, P., Almkvist, O., Wahlund, L.O., Kondakor, I., Lehmann, D.: Global dimensional complexity of multichannel EEG in mild Alzheimer’s disease and age-matched cohorts. Dement Geriatr Cogn Disord 8(6), 343–347 (1997)

Reproducibility Analysis of Event-Related fMRI Experiments Using Laguerre Polynomials Hong-Ren Su1,2, Michelle Liou2,*, Philip E. Cheng2, John A.D. Aston2, and Shang-Hong Lai1 1

Dept. of Computer Science, National Tsing Hua University, Hsinchu, Taiwan 2 Institute of Statistical Science, Academia Sinica, Taipei, Taiwan [email protected]

Abstract. In this study, we introduce the use of orthogonal causal Laguerre polynomials for analyzing data collected in event-related functional magnetic resonance imaging (fMRI) experiments. This particular family of polynomials has been widely used in the system identification literature and recommended for modeling impulse functions in BOLD-based fMRI experiments. In empirical studies, we applied Laguerre polynomials to analyze data collected in an eventrelated fMRI study conducted by Scott et al. (2001). The experimental study investigated neural mechanisms of visual attention in a change-detection task. By specifying a few meaningful Laguerre polynomials in the design matrix of a random effect model, we clearly found brain regions associated with trial onset and visual search. The results are consistent with the original findings in Scott et al. (2001). In addition, we found the brain regions related to the mask presence in the parahippocampal, superior frontal gyrus and inferior parietal lobule. Both positive and negative responses were also found in the lingual gyrus, cuneus and precuneus. Keywords: Reproducibility analysis, Event-related fMRI.

1 Introduction We previously proposed a methodology for assessing reproducibility evidence in fMRI studies using an on-and-off paradigm without necessarily conducting replicated experiments, and suggested interpreting SPMs in conjunction with reproducibility evidence (Liou et al., 2003; 2006). Empirical studies have shown that the method is robust to the specification of hemodynamic response functions (HRFs). Recently, BOLD-based event-related fMRI experiments have been widely used as an advanced alternative to the on-and-off design for studies on human brain functions. In eventrelated fMRI experiments, the duration of stimulus presentation is generally longer and there are no obvious contrasts between the experimental and control conditions to be used in data analyses. In order to detect possible brain activations during stimulus presentation and task performance, there have been a variety of event-related HRFs proposed in the literature. In this study, we introduce the use of orthogonal causal * Corresponding author. M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 126–134, 2008. © Springer-Verlag Berlin Heidelberg 2008

Reproducibility Analysis of Event-Related fMRI Experiments

127

Laguerre polynomials for modeling response functions. This particular family of polynomials has been widely used in the system identification literature and was recommended for modeling impulse functions in fMRI experiments (Saha et al., 2004). In the empirical study, we applied Laguerre polynomials to analyze data in the study by Scott et al. (2001). The dataset was published by the US fMRI Data Center and is available for public access. The original experiment involved 10 human subjects and investigated brain functions associated with a change-detection task. In the experimental task, subjects look attentively at two versions of the same picture in alternation, separated by a brief mask interval. The experiment additionally analyzed behavioral responses that subjects detected something changing between pictures and pressed a button with hands. In our reproducibility analysis, a few meaningful Laguerre polynomials matching the experimental design were inserted into a random effect model and reproducibility analyses were conducted based on the selected polynomials. In the analyses, we successfully located brain regions associated with the visual change-detection task similar to those found in Scott et al.. Additionally, we found other interesting brain regions that were not included in the previous study.

2 Method In this section, we will briefly describe the method for investigating the reproducibility evidence in fMRI experiments, and outline the family of Laguerre polynomials including those used in our empirical study. 2.1 Reproducibility Analysis In the SPM generalized linear model, the fMRI responses in the ith run can be expressed as

yi = X i β i + ei ,

(1)

where yi is the vector of image intensity after pre-whitening, Xi is the design matrix, and β i is the vector containing the unknown regression parameters. In the random effect model, the regression parameters

βi

are additionally assumed to be random

from a multivariate Gaussian distribution with common mean μ and variance Ω . The empirical Bayes estimate of

βi

in the random effect model would shrink all

estimates toward the mean μ , with greater shrinkage at noisy runs. In fMRI studies, the true status of each voxel is unknown, but can be estimated using the t-values (i.e., standardized β estimates) within individual runs derived from the random effect model along with the maximum likehood estimation method. By specifying a mixed multinomial model, the receiver-operation characteristic (ROC) curve can be estimated using the maximum likelihood estimation method and t-values of all image voxels. The curve is simply a bivariate plot of sensitivity versus the false alarm rate. The threshold (or the operational point) on the ROC curve for classifying voxels into the active/inactive status was found by maximizing the kappa value. We follow the

128

H.-R. Su et al.

same definition in Liou et al. (2006) to categorize voxels according to reproducibility, that is, a voxel is strongly reproducible if its active status remains the same in at least 90% of the runs, moderately reproducible in 70-90% of the runs, weakly reproducible in 50-70% of the runs, and otherwise not reproducible. The brain activation maps are constructed on the basis of strongly reproducible voxels, but include voxels that are moderately reproducible and spatially proximal to those strongly reproducible voxels. 2.2 Laguerre Polynomials The Laguerre polynomials can be used for detecting experimental responses. This family of polynomials can be specified as follows: L

h (t ) = ∑ f i g ia (t ) ,

(2)

i =1

where h(t) is the design coefficients to be input into Xi in (1); L is the order of Laguerre polynomial; ƒi is the coefficient of the basis function, and gia(t) is the inverse Z transform of the i-th Laguerre polynomial given by ~ a ⎡ z −1 z −1 − a i −1 ⎤ −1 g ia (t ) = Z −1 ⎢ ( ) = Z [ g i ( z )] ⎥ −1 −1 ⎣1 − az 1 − az ⎦

(3)

where a is a time constant. As an illustration, Fig. 1 gives the response coefficients corresponding to L=2 and L =3.

h(t), L=2

h(t), L=3

(a)

(b)

Fig. 1. The boxcar functions of experimental conditions in the Scott et al. study are depicted in (a), and the Laguerre polynomials h(t) with L=2 and L =3 are depicted in (b)

3 Event-Related fMRI Experiments We here introduce the experimental design behind the fMRI dataset used in the empirical study, and select the design matrix suitable for the study.

Reproducibility Analysis of Event-Related fMRI Experiments

129

3.1 Experimental Design In our empirical study, the dataset contains functional MR images of 10 subjects who went through 10-12 experimental runs, with 10 stimulus trials in each run. Experimental runs involved the change-detection task in which two images within a pair differed in either the presence/absence of the position of a single object or the color of the object. The two images were presented alternatively for 40 sec. In the first 30 sec, each image was presented for 300 msec followed by a 100 msec mask. However, the mask was removed in the last 10 sec. Subjects pressed a button when detecting something changing between the pair of images. The experimental images and stimulus duration are shown in Fig. 2.

Fig. 2. The experimental images and stimulus duration in the Scott et al. study

3.2 Experimental Design Matrix We used the Laguerre polynomials in Fig. 1 for specifying the design matrix in (1) instead of the theoretical HRFs. According to the original experimental design, there are two contrasts of interest in the Scott et al. study. The first is the response after the task onset within the 40 secs trial, and the second is the difference between stimulus presentations with and without the mask, that is, responses during the image presentation with the mask (0 ~ 30 sec.) and without the mask (30 ~ 40 sec.) in Fig. 2. The boxcar functions in Fig. 1 can also be specified in the design matrix as was suggested in the study by Liou et al. (2006) for the on-and-off design. However, the two boxcar functions are not orthogonal to each other and carry redundant information on experimental effects. In the event-related fMRI experiment, the duration of stimulus presentation is always longer than that in the on-and-off design. The theoretical HRFs vanish during the stimulus presentation. There might be brain regions continuously responding to the stimulus. The Laguerre polynomials are orthogonal and offer possibilities for examining all kinds of experimental effects. We

130

H.-R. Su et al.

might also consider Laguerre polynomials in Fig. 1 as a smoothed version of the boxcar functions.

4 Results In the Scott et al. study, a response-contingent event-related analysis technique was used in the data analyses, and the original results showed brain regions associated with different processing components in the visual change-detection task. For instance, the lingual gyrus, cuneus, precentral gyrus, and medial frontal gyrus showed activations associated with the task onset. And the pattern of activation in dorsal and ventral visual pathways was temporally associated with the duration of visual search. Finally parietal and frontal regions showed systematic deactivations during task performance. In the reproducibility analysis with Laguerre polynomials, we found the similar activation regions associated with the task onset, visual search and deactivations. In addition, we found activation regions in the parahippocampal, superior frontal gyrus, supramarginal gyrus and inferior parietal lobule. Both positive and negative responses were also found in the lingual gyrus, cuneus and precuneus which are also reproducible across all subjects; this finding is consistent with our previous data analyses of fMRI studies involving object recognition and word/pseudoword reading (Liou et al., 2006). Table 1 lists a few activation regions in the change-detection task for the 10 subjects. Table 1. The activation regions in the change-detection task. The plus sign indicates the positive response and minus sign indicates the negative response.

Subjects Lingual gyrus Precuneus Cuneus Posterior cingulate Medial frontal gyrus Parahippocampal gyrus Superior frontal gyrus Supramarginal gyrus

1 +/+/+/+/+/-

2 +/+/+/-

+ + +

3 +/+/+/+/+/+/+ +

4 +/+/+/+/-

5 +/+/+/+/+/+ +

6 +/+/+/+ +/-

7 +/+/+/+/+/+/-

8 +/+/+/+/+/+/-

9 +/+/+/-

10 +/+/+/+/+/-

+ +

In the table, there are 4 subjects showing activations in the superior frontal gyrus and supramarginal gyrus in the change-detection task. The two regions have been referred to in fMRI studies on language process (e.g., the study on word and pseudoword reading). The 4 subjects, on average, had longer reaction time in the change-detection task, that is, a delay of pressing the button until the image presentation without the mask (30-40 sec.). Fig. 3 shows the brain activation regions for Subjects 5 and 7 in the Scott et al. study. Subject 5 involved the superior frontal gyrus and supramarginal gyrus and had the longest reaction time compared with other subjects in the experiment. On the other hand, Subject 7 had relatively shorter reaction time and showed no activations in the two regions.

Reproducibility Analysis of Event-Related fMRI Experiments

Fig. 3. Brain activation regions for Subjects 5 and 7 in the Scott et al. study

131

132

H.-R. Su et al.

Subject 5

Fig. 3. (continued)

Reproducibility Analysis of Event-Related fMRI Experiments

Subject 7

Fig. 3. (continued)

133

134

H.-R. Su et al.

5 Discussion The reproducibility evidence suggests that the 10 subjects consistently show a pattern of increased/decreased responses in the lingual gyrus, cuneus, and precuneus. Similar observations were also found in our empirical studies on other datasets published by the fMRIDC. In the fMRI literature, the precuneus, posterior cingulate and medial prefrontal cortex are known to be the default network in a resting state and show decreased activities in a variety of cognitive tasks. The physiological mechanisms behind the decreased responses are still under investigation. However, discussions on the network have given a focus on the decreased activities. We would suggest to consider both positive and negative responses when interpreting the default network. By the method of reproducibility analyses, we can clearly classify brain regions that show consistent responses across subjects and those that show patterns and inconsistencies across subjects (see results in Table 1). Higher mental functions are individual and their localization in specific brain regions can be made only with some probabilities. Accordingly, the higher mental functions are connected with speech, that is, external or internal speech organizing personal behavior. Subjects differ from each other as a result of using different speech designs when making decisions in performing experimental tasks. Change of functional localization is an additional characteristic of a subject’s psychological traits. The proposed methodology would assist researchers in identifying those brain regions that are specific to individual speech designs and those that are consistent across subjects. Acknowledgments. The authors are indebted to the fMRIDC at Dartmouth College for supporting the datasets analyzed in this study. This research was supported by the grant NSC 94-2413-H-001-001 from the National Science Council (Taiwan).

References 1. Liou, M., Su, H.-R., Lee, J.-D., Aston, J.A.D., Tsai, A.C., Cheng, P.E.: A method for generating reproducible evidence in fMRI studies. NeuroImage 29, 383–395 (2006) 2. Huettel, S.A., Guzeldere, G., McCarthy, G.: Dissociating neural mechanisms of visual attention in change detection using functional MRI. Journal of Cognitive Neuroscience 13(7), 1006–1018 (2001) 3. Liou, M., Su, H.-R., Lee, J.-D., Cheng, P.E., Huang, C.-C., Tsai, C.-H.: Bridging Functional MR Images and Scientific Inference: Reproducibility Maps. Journal of cognitive Neuroscience 15(7), 935–945 (2003) 4. Saha, S., Long, C.J., Brown, E., Aminoff, E., Bar, M., Solo, V.: Hemodynamic transfer function estimation with Laguerre polynomials and confidence intervals construction from functional magnetic resonance imaging (FMRI) data. IEEE ICASSP 3, 109–112 (2004) 5. Andrews, G.E., Askey, R., Roy, R.: Laguerre Polynomials. In: §6.2 in Special Functions, pp. 282–293. Cambridge University Press, Cambridge (1999) 6. Arfken, G.: Laguerre Functions. In: §13.2 in Mathematical Methods for Physicists, 3rd ed., Orlando, FL, pp. 721–731. Academic Press, London (1985)

The Effects of Theta Burst Transcranial Magnetic Stimulation over the Human Primary Motor and Sensory Cortices on Cortico-Muscular Coherence Murat Saglam1, Kaoru Matsunaga2, Yuki Hayashida1, Nobuki Murayama1, and Ryoji Nakanishi2 1

Graduate School of Science and Technology, Kumamoto University, Japan 2 Department of Neurology, Kumamoto Kinoh Hospital, Japan [email protected], {yukih,murayama}@cs.kumamoto-u.ac.jp

Abstract. Recent studies proposed a new paradigm of repetitive transcranial magnetic stimulation (rTMS), “theta burst stimulation” (TBS); to primary motor cortex (M1) or sensory cortex (S1) can influence cortical excitability in humans. Particularly it has been shown that TBS can induce the long-lasting effects with the stimulation duration shorter than those of conventional rTMSs. However, in those studies, effects of TBS over M1 or S1 were assessed only by means of motor- and/or somatosensory-evoked-potentials. Here we asked how the coherence between electromyographic (EMG) and electroencephalographic (EEG) signals during isometric contraction of the first dorsal interosseous muscle is modified by TBS. The coherence magnitude localizing for the C3 scalp site, and at 13-30Hz band, significantly decreased 30-60 minutes after the TBS on M1, but not that on S1, and recovered to the original level in 90-120 minutes. These findings indicate that TBS over M1 can suppress the corticomuscular synchronization. Keywords: Theta Burst Transcranial Magnetic Stimulation, Coherence Electroencephalogram, Electromyogram, Motor Cortex.

1 Introduction Previous studies have demonstrated dense functional and anatomical projections among motor cortex building a global network which realizes the communication between the brain and peripheral muscles via the motor pathway [1, 2]. The quality of the communication is thought to highly depend on the efficacy of the synaptic transmission between cortical units. In the past few decades, repetitive transcranial magnetic stimulation (rTMS) was considered to be a promising method to modify cortical circuitry by leading the phenomena of long-term potentiation (LTP) and depression (LTD) of synaptic connections in human subjects [3]. Furthermore, a recently developed rTMS paradigm, called “theta burst stimulation” (TBS) requires less number of the stimulation pulses and even offers the longer aftereffects than conventional rTMS protocols do [4]. Previously, efficiency of TBS has M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 135–141, 2008. © Springer-Verlag Berlin Heidelberg 2008

136

M. Saglam et al.

been assessed by means of signal transmission from cortex to muscle or from muscle to cortex, by measuring motor-evoked-potential (MEP) or somatosensory-evoked-potential (SEP), respectively. It was shown that TBS applied over the surface of sensory cortex (S1) as well as primary motor cortex (M1) could modify the amplitude of SEP (recorded from the S1 scalp site) lasting for tens of minutes after the TBS [5]. On the other hand, the amplitude of MEP was not modified by the TBS applied over S1, while the MEP amplitude was significantly decreased by the TBS applied over M1 [4, 5]. In the present study, we examined the effects of TBS applied over either M1 or S1 on the functional coupling between cortex and muscle by measuring the coherence between electroencephalographic (EEG) and electromyographic (EMG) signals during voluntary isometric contraction of the first dorsal interosseous (FDI) muscle.

2 Methods 2.1 Subjects Seven subjects among whole set of recruited participants (approximately twenty) showed significant coherence and only those subjects participated to TBS experiments. Experiments on M1 and S1 performed on different days and subjects did not report any side effects during or after the experiments. 2.2 Determination of M1 and S1 Location The optimal location of the stimulating coil was determined by searching the largest MEP response(from the contralateral FDI-muscle) elicited by single pulse TMS while

Fig. 1. EEG-EMG signals recorded before and after the application of TBS as depicted in the experiment time line. Subjects were asked to contract four times at each recording set. Location and intensity for actual location were determined after pre30 session. Pre0 recording was done to confirm searching does not make any conditioning. TBS paradigm is illustrated in the rightabove inset.

The Effects of Theta Burst Transcranial Magnetic Stimulation

137

moving the TMS coil in 1cm steps on the presumed position of M1. Stimulation was applied by a with a High Power Magstim 200 machine and a figure-of-8 coil with mean loop diameter of 70 mm (Magstim Co., Whitland, Dyfed, UK). The coil was placed tangentially to the scalp with the handle pointing backwards and laterally at a 45˚ angle away from the midline. Based on previous reports S1 is assumed to be 2cm posterior from M1 site [5]. 2.3 Theta Burst Stimulation Continuous TBS (cTBS) paradigm of 600 pulses was applied to the M1 and S1 location. cTBS consists of 50Hz triplets of pulses that are repeating themselves at every 0.2s (5Hz) for 40s[4]. Intensity of each pulse was set to 80% active motor threshold (AMT) which is defined as the minimum stimulation intensity that could evoke an MEP of no less than 200μV during slight tonic contraction. 2.4 EEG and EMG Recording EEG signals were recorded based on the international 10-20 scalp electrode placement method (19 electrodes) with earlobe reference. EMG signal, during isometric hand contraction at 15% level from the maximum, was recorded from the FDI muscle of the right hand with reference to the metacarpal bone of the index finger. EEG and EMG signals were recorded with 1000 Hz sampling frequency and passbands of 0.5-200Hz and 5-300 Hz, respectively. Each recording set consists of 4 one-minute-long recordings with 30s-rest time intervals. To assess TBS effect with respect to time, each set was performed 30 minutes before (pre30), just before (pre0), 0,30,60,90 and 120 minutes after the delivery of TBS. Stimulation location and intensity were determined between pre30 and pre0 recordings (Fig. 1). 2.5 Data Analysis Coherence function is the squared magnitude of the cross-spectra of the signal pair divided by the product their power spectra. Therefore cross- and power-spectra between EMG and 19 EEG channels were calculated. Fast Fourier transform, with an epoch size of 1024, resulting in frequency resolution of 0.98 Hz was used to convert the signals into frequency domain. The current source density (CSD) reference method was utilized in order to achieve spatially sharpened EEG signals [6]. Coherency between EEG and EMG signals was obtained using the expression below:

0≤κ (f) = 2 xy

S xy ( f )

2

S xx ( f ) S yy ( f )

≤1

(1)

where Sxy(f) represents the cross-spectral density function. Sxx(f) and Syy(f) stand for the auto-spectral density of the signals x and y, respectively. Since coherence is a normalized measure of the correlation between signal pairs, κ2xy (f) =1 represents a perfect linear dependence and κ2xy (f) =0 indicates a lack of linear dependence within those signal pairs. Coherence values for κ2xy (f) > 0 are assumed to be statistically significant only if they are above 99% confidence limit that is estimated by:

138

M. Saglam et al. 1

α ⎞ ( n−1) ⎛ CL(α %) := 1 − ⎜1 − ⎟ ⎝ 100 ⎠ where n is the number of epochs used for cross- and power- spectra calculations.

(2)

3 Results First we have confirmed that EEG-EMG coherence values for all (n=7) subjects lie above 99% significance level (coh ~= 0.02) and within beta (13-30 Hz) frequency Table 1. Mean ± standard error of mean (SEM) of beta band EEG (C3)-EMG coherence values and peak frequencies for TBS-over-M1 and S1 experiments (n=7)

Coherence Magnitude

M1 S1

Peak M1 Frequency(Hz) S1

Pre30 (min)

Pre0 (min)

0 (min)

30 (min)

60 (min)

90 (min)

120 (min)

0.061±0.008

0.051±0.014

0.053±0.014

0.03±0.007

0.031±0.009

0.057±0.015

0.067±0.016

0.059±0.021

0.068±0.026

0.058±0.027 0.061±0.023

0.052±0.014

0.034±0.006

0.040±0.016

20.50±1.92

21.80±1.98

22.48±2.04

23.73±1.84

18.57±1.2

20.75±1.62

18.23±1.82

21.17±1.79

21.80±1.97

23.08±1.82

23.10±2.15

18.22±2.13

21.17±1.08

19.52±.3.21

Fig. 2. Coherence spectra between EEG and EMG signals at pre30 session. A, Coherence spectra between 19 EEG channels and EMG (FDI) for all subjects (n=7) are superimposed and topographically according to the approximate locations of the electrodes on the scalp. Each electrode labeled with respect to its location (Fp: Frontal pole F: Frontal T: Temporal C: Central P: Parietal O: Occipital). B, Expanded view of the coherence spectra between EEG (C3 scalp site) and EMG (FDI) for all subjects (n=7). Each line style specifies different subject’s coherence spectra. Coherence values only above 99% significance level (indicated by the solid horizontal line) are highlighted.

The Effects of Theta Burst Transcranial Magnetic Stimulation

139

band at C3 scalp site. Maximum coherence levels were observed at C3 (n=6) and at F3 (n=1) scalp sites whereas no significant coherence was observed at other locations (Figure 2). These results are in well agreement with the previous studies on coherence between EEG-EMG during isometric contraction [7, 8]. Table 1 shows the average absolute coherence values and peak frequencies at all trials before and after the application of TBS. Figure 3 demonstrates the normalized EEG (C3)-EMG coherence values as average of all subjects. Coherence values obtained before the TBS were taken as control and set to 100%. Average beta band coherence suppressed to 56.2% after 30 minutes and 54.5% after 60 minutes with statistical significance(p<0.05, Bonferonni multiple comparison test) and recovered to the original control level 90 and 120 minutes when TBS applied over M1. However there is no statistically significant coherence change with respect to time when TBS is delivered to S1 (coherence values are 103.8% after 30 minutes, 99.8% after 60 minutes).

Fig. 3. EEG (C3)-EMG Coherence at beta band (n=7). Pre-stimulation coherence values were taken as control level (100%). Coherence was suppressed (* p<0.05, Bonferonni multiple comparison test) 30 and 60 minutes after TBS-over-M1 and recovered back to the control level. Error bars indicate standard error of mean (±SEM).

4 Discussion The present results show that TBS over human motor cortex can change the corticomuscular functional coupling. TBS-over-M1 inhibited EEG (C3)-EMG (FDI) coherence for about 60 minutes, this inhibition has similar time course with MEP

140

M. Saglam et al.

amplitude suppression after TBS-over-M1 [4, 5]. The parallel responses of corticomuscular coherence and MEP to TBS may suggest that TBS-driven LTP/LTD of synaptic connections, involved in the circuits for MEP generation (i.e. I wave circuits), not only produces MEP but also plays an important role in rhythmic activities in the motor system. However it is difficult to assess the exact LTP/LTD scenario after TBS in conscious human experiments. On the other hand, we know that effects of cTBS count on pre- and postsynaptic N-methyl-D-aspartate receptors (NMDARs) which are highly associated with LTP/LTD [10, 11]. Blocking NMDARs with pharmacological agents (i.e. memantine) eliminates the inhibitory effects of cTBS [11]. Thus we may conclude that NMDAR-related LTP/LTD of synaptic connections within motor circuit is involved in corticomuscular coherence phenomenon and it can be temporarily modified by TBS. Among conventional rTMS studies, there are few reports investigating the effects of rTMS on cortico-muscular coherence. Similar changes (suppression) in corticomuscular coherence were observed using longer rTMS conditioning (15min with 0.9Hz rTMS vs. 40s with TBS-over-M1) although it was applied over premotor cortex in that study [9]. Absence of evidence in the literature hinders us to make a complete comparison between rTMS-over-M1 and TBS-over-M1. However it is evident that TBS has prolonged suppressive effect (15min effect with 0.9Hz rTMS vs. 60min effect with TBS). Therefore present data confirms that TBS could be a more prominent technique than rTMS by means of cortico-muscular coherence. Since rTMS has a therapeutic potential in clinical use [12, 13], a shorter yet stronger stimulation technique should be considered. On the other hand, TBS-over-S1 showed no significant coherence change although stimulation site is just 2 cm posterior to M1. The lack of suppressive effect after TBS-over-S1 suggests that there is no conditioning on M1 due to a current spread from S1 to M1 so that TBS can be regarded as a focal stimulation tool. The coherence was suppressed after of TBS-over-M1 rather than TBS-over-S1 condition. This result agrees with the fact that MEP amplitude is suppressed by TBS-over-M1 but not by TBS-over-S1. Therefore present findings could indicate that similar mechanism is evident for the MEP generation and coupling between cortical and muscular sites.

References 1. Brown, P., Marsden, J.F.: Cortical network resonance and motor activity in humans. Neuroscientist 7(6), 518–527 (2001) 2. Chouinard, P.A., Paus, T.: The primary motor and premotor areas of the human cerebral cortex. Neuroscientist 12(2), 143–152 (2006) 3. Filipovic, S.R., Siebner, H.R., Rowe, J.B., Cordivari, C., Gerschlager, W., Rothwell, J.C., Frackowiak, R.S., Bhatia, K.P.: Modulation of cortical activity by repetitive transcranial magnetic stimulation (rTMS): a review of functional imaging studies and the potential use in dystonia. Adv. Neurol. 94, 45–52 (2004) 4. Huang, Y.Z., Edwards, M.J., Rounis, E., Bhatia, K.P., Rothwell, J.C.: Theta burst stimulation of the human motor cortex. Neuron 45, 201–206 (2005) 5. Ishikawa, S., Matsunaga, K., Nakanishi, R., Kawahira, K., Murayama, N., Tsuji, S., Huang, Y.Z., Rothwell, J.C.: Effect of theta burst stimulation over the human sensorimotor cortex on motor and somatosensory evoked potentials. Clinical Neurophysiology (in press, 2007)

The Effects of Theta Burst Transcranial Magnetic Stimulation

141

6. Mima, T., Hallett, M.: Electroencephalographic analysis of cortico-muscular coherence: reference effect, volume conduction and generator mechanism. Clinical Neurophysiology 110, 1892–1899 (1999) 7. Murayama, N., Lin, Y.Y., Salenius, S., Hari, R.: Oscillatory interaction between human motor cortex and trunk muscles during isometric contraction. Neuroimage 14, 1206–1213 (2001) 8. Conway, B.A., Halliday, D.M., Farmer, S.F., Shahani, U., Maas, P., Weir, A.I., Rosenberg, J.R.: Synchronization between motor cortex and spinal motoneuronal pool during the performance of a maintained motor task in man. J. Physiol. 489(3), 917–924 (1995) 9. Chen, W.H., Mima, T., Siebner, T., Oga, H.R., Hara, T., Satow, H., Begum, T., Shibasaki, H.: Low-frequency rTMS over lateral premotor cortex induces lasting changes in regional activation and functional coupling of cortical motor areas. Clinical Neurophysiology 114(9), 1628–1637 (2003) 10. Huang, Y.Z., Chen, R.S., Rothwell, J.C., Wen, H.Y.: The after-effect of human theta burst stimulation is NMDA receptor dependent. Clin. Neurophysiol. 118(5), 1028–1032 (2007) 11. MacDermott, A.B., Mayer, M.L., Westbrook, G.L., Smith, S.J., Barker, J.L.: NMDAreceptor activation increases cytoplasmic calcium concentration in cultured spinal cord neurones. Nature 321(6069), 519–522 (1986) 12. Lefaucheur, J.P.: Repetitive transcranial magnetic stimulation (rTMS): insights into the treatment of Parkinson’s disease by cortical stimulation. Neurophysiol. Clin. 36(3), 125– 133 (2006) 13. Joo, E.Y., Han, S.J., Chung, S.H., Cho, J.W., Seo, D.W., Hong, S.B.: Antiepileptic effects of low-frequency repetitive transcranial magnetic stimulation by different stimulation durations and locations. Clin Neurophysiol 118(3), 702–708 (2007)

Interactions between Spike-Timing-Dependent Plasticity and Phase Response Curve Lead to Wireless Clustering Hideyuki Cˆ ateau1 , Katsunori Kitano2 , and Tomoki Fukai1 2

1 RIKEN BSI, Hirosawa, Wako, 3510198 Saitama, Japan Department of Human and Computer Intelligence, Ritsumeikan University, 1-1-1 Nojihigashi, Kusatsu, Shiga 525-8577, Japan

Abstract. A phase response curve characterizes the signal transduction between neurons in a minimal manner,whereas spike-timing-dependent plasiticity (STDP) characterizes the way to rewire networks in an activitydependent manner. The present paper demonstrates that these two key properties both related to spikes work synergetically to carve functionally useful circuits in the brain. STDP working on a population of neurons that prefer asynchrony turns out to convert the initial asynchronous firing to clustered firing with synchrony within a cluster. They get synchronized within a cluster despite their preference to asynchrony because STDP selectively disrupts intra-cluster connections, which we call wireless eclustering.

1

Introduction

Synchrony tendency of coupled oscillators or neurons is predicted by the phase response curve (PRC) of a neuron that describes an amount of advance or delay by synaptic input given at a specific phase in an interspike interval[1,2]. It is intriguing to know how this useful theory based on the fixed coupling strength between neurons generalizes to the cases where synaptic strength varies as observed in the real brain. A number of experiments[3] have established that synaptic strength changes depending on pre- and postsynaptic spike times and theoretical implications of such spike-timing-dependent plasticity (STDP) have been extensively studied [4,5,6,7,8,9]. Since the PRC and STDP both refer to the timings of spikes, a natural question is how these two properties of a neuronal network interact each other to carve a functional network in the brain. To answer this question, we first use a neuron model whose PRC can be systematically controlled [10] unlike the simpler leaky integrate-and-fire (LIF) model. The model neurons favors either asynchronous (Model A) or synchronous (Model B) firing depending on the values of the model parameters. Our simulations show that STDP working on the network of Model A neurons converts an asynchronously firing neurons into three or more cyclically activated clusters of neurons. Model A neurons can synchronize within a cluster despite their preference to asynchrony because, as we see later, STDP selectively disrupts M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 142–150, 2008. c Springer-Verlag Berlin Heidelberg 2008 

Interactions between STDP and PRC Lead to Wireless Clustering

143

intra-cluster connections, nullifying the asynchrony preference. If STDP works on the network of Model B neurons, however, the neurons simply get synchronized globally, analogous to what was observed in [11], and nothing interesting happens. Next we show that the self-organized cyclic activity appears also under biologically realistic settings using a Hodgkin-Huxley type neuron model, suggesting the generality of the concept. In the self-organization, PRC specifies the timing preference and influences the way how STDP works. Importantly, STDP in turn influences the way how PRC is readout. Before the STDP learning begins, the initial slope of an effective PRC (defined later) determines the stability of the global synchrony. After the STDP learning forms the cyclic activity consisting of n clusters, the slope of the effective PRC at θ = 2π(1 − 1/n) determines its stability. Thus, the two key features of spiking neurons, PRC and STDP, work synergetically to organize functional networks in the brain. Previously studies have shown [12,13,14] that STDP helps the pacemaker neuron entrain an innervated neuron(s), which was called ”frequency synchrony” meaning that neurons start firing at the same frequency but with different phases as opposed to the ”phase synchrony” studied here. Building upon the firm theoretical analysis of frequency synchrony [1,12,13,14], it is now important to ask when and how the frequency synchrony specializes to the phase synchrony because the phase synchrony can effectively enlarge otherwise tiny EPSC to the size capable of reliably eliciting spikes in innervated neurons.

2

Self-organization of Izhikevich Neurons

We consider a population of neurons firing quasi-periodically. We use a spiking neuron model proposed by Izhikevich[10]. Depending on the values of four parameters, a, b, c and d, this model can produce many different voltage trajectories similar to what are found in real neurons. Fifty Model A neurons that favor asynchrony (a = 0.02, b = 0.2, c = −50, d = 1.26) are connected in an all-to-all manner with uniform synaptic strength and with a range of synaptic delays of 2±0.2ms. The neurons fire quasi-periodically driven by supra-threshold stochastic inputs, I = I0 + σζ(t) with I0 = 30mV /ms and σ = 1.5mV /ms1/2 , where ζ(t) is the Gaussian white noise. With no synaptic plasticity at work, initial uniform distribution of firing phases (Fig.1a) remain asynchronous because the neurons favor synchrony. However, the effects of the standard additive STDP rule with hard boundaries (0 < w < 1 ) defined [4] by  Δw =

A+ exp(−Δt/τ+ ) for Δt ≥ 0, −A− exp(−|Δt|/τ− ) for Δt < 0,

(1)

with A+ = 0.05, A− = 0.0525 and τ+ = τ− = 20ms converts the asynchronous firing into clusters of synchronous firing (Fig. 1b).

144

H. Cˆ ateau, K. Kitano, and T. Fukai

membrane potential [mV]

C

D 40 20 0

−20 −40 −60 −80

4900

100 110 120 130 140 150 160 170 180 190 200

4920

4940

F

E

4960

4980

5000

time [ms]

time [ms]

presynaptic neurons 5

10

15

20

25

30

35

40

45

50

50 5

postsynaptic neurons

45

neuron index

40 35 30 25 20 15 10 5

10 15 20 25 30 35 40 45

9890

G

9910

9930 9950 time [ms]

9970

9990

H

presynaptic neurons 5

10

15

20

25

30

35

40

50

45

50

5

postsynaptic neurons

10 15 20 25 30 35 40 45 50

J

I

1

θ

θ

2

Fig. 1. Clustering of Model A neurons by STDP. Fifty Model A neurons fire in complete asynchrony before the STDP learning starts. (a) Voltage trajectories of all the neurons drawn in different colors are overlaid. (b) The neurons start firing in three different clusters with intra-cluster synchrony due to STDP. (c) A gray scale representation of the connection strength between neurons with black being the strongest. (d) A raster plot corresponding to (b). Neurons are aligned according to spike times. (e) The connection matrix when the higher-order rule of STDP is applied. A synaptic change is discounted by 1 − exp(−(tpost spike 2 − tEPSP by pre )/τremove ) with when a triplet of event, tpost spike 1 < tEPSP by pre < tpost spike 2 , happened. (f) A schematic drawing showing the network topology corresponding to (d) and (e). (g), (h) Positive feedback mechanism leading to the wireless clustering. In (g), the vertical lines indicate spike times of two neurons; the small wedges indiate EPSPs elicited by spikes. In (h), distributed firing phases of neurons are represented by the filled circules in different colors.

The network topology underlying this clustered synchronous is picturized by a matrix of synaptic strengths (Fig.1c), where neurons are indexed according to firing times after the learning (t > 9.89sec). The three divisions apparent in Fig.1c correspond to three synchronously firing clusters of neurons (Fig.1d).

Interactions between STDP and PRC Lead to Wireless Clustering

145

Interestingly, STDP appears to have removed the intra-cluster connections almost completely (Fig.1d). Such clustering without connections is observed commonly under various simulation conditions and we will call it wireless clustering. It turns out the wireles clustering has happened because we reflected experimental procedures of STDP faithfully. The standard STDP rule implies potentiation for positive timing difference, Δt = tpost − tpre > 0, and depression for a negative timing difference, Δt < 0. Many have argued that the asymmetry of this rule produces a one-way coupling (see e.g. [4] ). Such arguments would be valid if Δt represented the time difference between post- and presynaptic spikes. However, actually most experimental literatures [3] define Δt to be the time difference between a postsynaptic spike and the onset or peak of the somatic excitatory postsynaptic potential (EPSP) induced by a presynaptic spike: Δt = tpost spike − tEPSP by pre . Hence the above argument does not apply. A somatic EPSP should lag behind a presynaptic spike for a few msec. Therefore, if two neurons fire in exact synchrony (Fig.1g), Δt < 0 negative [12] for both directions, thereby weakens connections bidirectionally. Now, how does this mechanism convert initial asynchronous firing to clustered synchronous firing (Fig.1b)? Initial asynchronous firing (Fig.1a) is represented as firing phases evenly spread around the circle (Fig.1h, left). The firing remains asynchronous without STDP . However, with the phases of many neurons squeezed into the circle, any single neuron must have neighboring neurons that unwillingly fire synchronously with it(Fig.1h). Among these neurons, the abovementioned mechanism weakens the connections bidirectionally. As their synaptic connections weaken, mutual repulsion is also weakened. This then further synchronizes their firing. This positive feedback mechanism develops wireless clusters (Figs.1h). Although this mechanism qualitatively explains how the clustering happens, a quantitative question how many clusters are formed requires further consideration. We will later see that a stability analysis tells the possible number of clusters. In contrast to the vanishing intra-cluster connections, the inter-cluster connections survive and can be unidirectional (Fig.1d), which defines the cyclic network topology such as shown in Fig.1f, upper. Let us ask how we can change this 3-cycle topology. We find that one of the recently observed higher-order rules of STDP [15,16] increases the number of clusters (Fig.1e). The higher-order rule shown in [16] implies the gross reduction in the LTD effect because LTP override the immediately preceding LTD, while LTP simply cancels partly the immediately preceding LTD. The weakened LTD effect is likely to increase the total number of potentiated synapses, which is in consistent with the increased ratio of black areas in Fig.1e compared to Fig.1d. In contrast to such cluster-wise synchrony observed with Model A neurons, Model B neurons that favor synchrony ( a = 0.02, b = 0.2, c = −50, d = 40) selforganize into the globally synchronous state with or without synchrony (Fig.2). Due to the global synchrony, mutual synaptic connections are largely lost, and each neuron ends up being driven by the external input individually, having little sense of being present as a population. The global synchrony gives too strong

146

H. Cˆ ateau, K. Kitano, and T. Fukai

D

50 45 40 35 30 25 20 15 10 5 0

9750

postsynaptic neurons

neuron index

C

9800

9850 9900 time [ms]

9950

5

presynaptic neurons 10 15 20 25 30 35 40 45 50

5 10 15 20 25 30 35 40 45 50

10000

Fig. 2. Global synchrony observed with Model B neurons that favor synchrony. A raster plot (a) and connection matrix (b) of fifty Model B neurons. The neurons were aligned with the connection-based method: neurons are defined to belong to the same cluster whenever there mutual connections are small enough.

an impact and also has minimal coding capacity because all the neurons behave identically, and it appears to bear more similarity to the pathological activity such as seizure in the brain than to meaningful information processing. By contrast, the clustered synchrony arising in the network of Model A neurons appears functionally useful. Generally in the brain, the unitary EPSP amplitude (∼ 0.5mV ) is designed to be much smaller than the voltage rise needed to elicit firing (∼ 15mV ). Therefore, single-neuron activity alone cannot cause other neurons to respond. Hence, it is difficult to regard the single-neuron activity as a carrier of information transferred back and forth in the brain. In contrast, the self-organized assembly of tens of Model A neurons (Figs. 1d) looks an ideal candidate for a carrier of information in the brain because their impact on other neurons are strong enough to elicit responses. Additionally, a cluster can reliably code the timing information. The PRC, Z(2πt/T ), representing the amount of advance/delay of the next firing time in response to the input at t in the firing interval [0, T ] has been mostly used to decide whether a coupled pair of neurons or oscillators tend to synchronize or desynchronize under the assumption that the connection strengths between the neurons are equal and unchanged. Specifically, suppose that a pair of neurons are mutually connected and a spike of one neuron introduces a current with the waveform of EP SC(t) in an innervated neuron after a transmission T delay of τd . The effective PRC defined as Γ− (θ) = T1 0 Z(2πt /T )EP SC(t − θ τd − T 2π )dt is known to decide their synchrony tendency. If Γ− (θ) < 0 at θ = 0 is positive (negative), the two neurons are desynchronized (synchronized). This synchrony condition is inherited to a population of neurons coupled in an allto-all or random manner as far as the connection strengths remain unchanged. Theoretically calculated Γ− (θ)s for Model A and B (Fig.3a,b) explain that the all-to-all netwok of Model A (B) neurons exhibit global asynchrony (synchrony). Note that both Model A and B neurons belong to type II[19] so that both model neurons favor synchrony if they are delta-coupled with no synaptic delay. After STDP is switched on, the network consisting of Model A neurons, is selforganized into the 3-cycle circuit (Fig.1d) with a successive phase difference of the clusterd activity being Δsuc θ = 2π/3. Stability analysis shows that the slope

Interactions between STDP and PRC Lead to Wireless Clustering

147

(b)

(a) 0.035

0.25

2pi/3

0.03

0.2

2pi/4 0.15

0.025 0.02 0.015

0.1

0.01

0.05

0.005 0 -0.05

0

0

pi/2

pi

3pi/2

2pi

-0.005

0

pi/2

pi

3pi/2

2pi

Fig. 3. Effective PRCs and schema of triad mechanism. The effective PRC of Model A and B were calculated with the adjoint method [19] and shown shown in (a) and (b). The slope at θ = 0 is positive for (a) but negative for (b) although it hardly recognizable with this resolution. The slope at θ = 2π/3 is negative for (a) but positive for (b). The dashed lines represent θ = 2π/3 and θ = 2π/4.

of Γ− (θ) not at the origin but at θ = 2π − Δsuc θ now determines the stability of the 3-cycle activity: the 3-cycle activity is stable if Γ− (2π − Δsuc θ) < 0. Fig.3a tells that the 3-cycle activity shown in Fig.1c is stable. The stable cyclic activity is achieved through the following synergetic process: (1) PRC determines the preferred network activity (e.g. asynchronous or synchronous), (2) the network activity determines how STDP works, STDP modifies the network structure (e.g. from all-to-all to cyclic ), and (3) the network structure determines how the PRC is readout (e.g. θ = 0 or θ = 2π −Δsuc θ) ), closing the loop. Generally, we can show that the n-cylce activity whose successive phase difference equals Δsuc θ = 2π/n is stable if Γ− (2π − Δsuc θ) < 0. PRCs of biologically plausible neuron models or real neurons [20] tend to have a negative slope in a later phase of the firing interval and converge to zero at θ = 2π because the membrane potential starts the regenerative depolarization and becomes insensitive to any synaptic input. The corresponding effective PRCs inherit this negative slope in the later phase and tends to stabilize the n-cycle activity for some n.

3

Self-organization of Hodgkin-Huxley Type Neurons

Next we see that the self-organized cyclic activity with the wireless clustering is also observed in biologically realistic setting. Our simulations as described in [18] with 200 excitatory and 50 inhibitory neurons modeled with the HodgkinHuxley (HH) formalism exhibits the 3-cyclic activity with the wireless clustering (Fig.4a,b). The setup here is biologically realistic in that (1) HH type neurons are used, (2) physiologically known percentage of inhibitory neurons with nonplastic synapses are included, (3) neurons fire with high irregularity due to large noise in the background input unlike the well-regulated firing as shown in Fig.1c. Interestingly, the effective PRC (Fig.4c) of the HH type neuron shares important features with that of Model A: the positive initial slope implying the preference to asynchrony and a negative later slope stabilizing the 3-cycle activity.

148

H. Cˆ ateau, K. Kitano, and T. Fukai (a)

presynaptic neurons

(b)

20

200

postsynaptic neurons

160 140

neuron index

60

80

100 120 140 160 180 200

20

180

120 100 80 60 40 20 0 5000

40

40 60 80 100 120 140 160 180

5040

5080

5120

5160

200

5200

time(msec)

(c) 3.5

2pi/3 2pi/4

3 2.5 2 1.5 1 0.5 0

0

pi/2

pi

3pi/2

2pi

Fig. 4. Conductance-based model also develops the wireless clustering. (a) A raster plot of 200 HH-type excitatory neurons showing 3-cycle activity. (b) The corresponding connection matrix showing the wirelessness. (c) Effective PRC or Γ− (θ) of the conductance-based model.

Generally, technical difficulty in the HH simulations is their massive computational demands due to the complexity of the system. That difficulty has hidered theoretical analysis, and has left the studies largely experimental. In particular, previously we tried hard to understand why we never observed 4-cycle or longer in vain. However, the analytic argument we developed here with the simplified model gives a clear insight into the biologically plausible but complex system. Comparison of Fig.3a and Fig.4c reveals that the negative slope of Γ− (θ) of the HH model is located at more left than that of Model A, indicating less stability of long cycles in the HH simulations. With the larger amount of noise in the HH simulations in mind, it is now understood that 4-cycle and longer can be easily destabilized in the HH simulations. Thus, our analysis developed with the simplefied system serves as a useful tool to understand a biologically realistic but complex systems. There is, howevr, an interesting difference between the Model A and HH simulations. Although the intra-cluster wirelessness is a fairly good first approximation in the HH model simulations (Fig.4b), it is not as exact as in the Model A simulations (Fig.1d,e). Interestingly, an elimination of the residual intracluster connections destroys the cyclic activity, suggesting the supportive role of the tiny residual intra-cluster connections.

4

Discussion

In the previous simulation study[17] using the LIF model, cyclic activity was observed to propagate only at the theoretical speed limit: it takes only τd from one cluster to the next, requiring the zero membrane integration time. To understand

Interactions between STDP and PRC Lead to Wireless Clustering

149

why it was the case, we first remind that the effective PRC needs a negative slope at 2π − Δsuc θ to stabilize the cylic activity. However, the slope of the PRC of an θ LIF model, Z(θ) = c exp( τTm 2π ), is always positive except at the the end point, T where Z(2π − 0) = c exp( τm ) and Z(2π + 0) = c, implying Z  (2π) = −∞. This infinitely sharp negative slope of the PRC at θ = 2π is rounded and displaced to 2π − 2πτd /T in Γ− (θ) (see its definition). Since this is the only place where Γ− (θ) has a negative slope, The cyclic activity is stable only if Δsuc θ = 2πτd /T , implying the propagation at the theoretical speed limit. We demonstrated an intimate interplay between PRC and STDP using the Izhikevich neuron model as well as the HH type model. The present study complements previous studies using the phase oscillator [11,14], where its mathematical tractability was exploited to analytically investigate the stability of the global phase/frequency synchrony. The self-organization or unsupervised learning by STDP studied here complements the supervised learning studied in [22]. The propagation of synchronous firing and temporal evolution of synaptic strength under STDP is know to be analyzed semi-analytically with the Fokker-Planck equation STDP [5,6,8,9,21]. It is interesting future direction to see how the Fokker-Planck equation can be used to understand the interplay between PRC and STDP.

Acknowledgement The present authors thank Dr. T. Takewaka at RIKEN BSI for offering the code to calculate the PRC.

References 1. Kuramoto, Y.: Chemical oscillations,waves,and turbulence. Springer, Berlin (1984) 2. Ermentrout, G., Kopell, N.: SIAM J. Math.Anal. 15, 215 (1984) 3. Markram, H., et al.: Science 275, 213 (1997); Bell, C.C., et al.: Nature 387, 278 (1997); Magee, J.C., Johnston, D.: Science 275, 209 (1997); Bi, G.-Q., Poo, M.-M.: J. Neurosci. 18, 10464 (1998); Feldman, D. E., Neuron 27, 45 (2000); Nishiyama, M., et al.: Nature 408, 584 (2000) 4. Song, S., et al.: Nat. Neurosci. 3, 919 (2000) 5. van Rossum, M.C., Turrigiano, G.G., Nelson, S.B.: J. Neurosci. 22,1956 (2000) 6. Rubin, J., et al.: Phys. Rev. Lett. 86, 364 (2001) 7. Abbott, L.F., Nelson, S.B.: Nat. Neurosci. 3, 1178 (2000) 8. Gerstner, W., Kistler, W.M.: Spiking neuron model. Cambridge University Press, Cambridge (2002) 9. Cˆ ateau, H., Fukai, T.: Neural Comput., 15, 597 (2003) 10. Izhikevich, E.M.: IEEE Trans. Neural Netw. 15, 1063 (2004) 11. Karbowski, J.J., Ermentrout, G.B.: Phys. Rev. E. 65, 031902 (2002) 12. Nowotny, T., et al.: J. Neurosci. 23, 9776 (2003) 13. Zhigulin, V.P., et al.: Phys. Rev. E, 67, 021901 (2003) 14. Masuda, N., Kori, H.: J. Comp. Neurosci, 22, 327 (2007) 15. Froemke, R.C., Dan, Y.: Nature, 416, 433 (2002)

150 16. 17. 18. 19. 20.

H. Cˆ ateau, K. Kitano, and T. Fukai

Wang, H.-X., et al.: Nat. Neurosci, 8, 187 (2005) Levy, N., et al.: Neural Netw. 14, 815 (2001) Kitano, K., Cˆ ateau, H., Fukai, T.: Neuroreport, 13, 795 (2002) Ermentrout, G.B.: Neural Comput. 8, 979 (1996) Reyes, A.D., Fetz, E.E.: J. Neurophysiol. 69, 1673 (1993); Reyes, A.D., Fetz, E.E.: J. Neurophysiol. 69, 1661 (1993); Oprisan, S.A., Prinz, A.A., Canavier, C.C.: Biophys. J., 87, 2283 (2004); Netoff, T.I., et al.: J Neurophysiol. 93, 1197 (2005); Lengyel, M., et al.: Nat. Neurosci. 8, 1667 (2005); Galan, R.F., Ermentrout, G.B., Urban, N.N.: Phys. Rev. Lett. 94, 158101 (2005); Preyer, A.J., Butera, R.J.: Phys. Rev. Lett. 95, 13810 (2005); Goldberg, J.A., Deister, C. A., Wilson, C.J.: J. Neurophysiol., 97, 208 (2007); Tateno, T., Robinson, H.P.: Biophys. J., 92, 683 (2007); Mancilla, J.G., et al.: J. Neurosci. 27, 2058 (2007); Tsubo, Y., et al.: Eur J. Neurosci, 25, 3429 (2007) 21. Cˆ ateau, H., Reyes, A.D.: Phys. Rev. Lett. 96, 058101, and references therein (2006) 22. Lengyel, M., et al.: Nat. Neurosci. 8, 1677 (2005)

A Computational Model of Formation of Grid Field and Theta Phase Precession in the Entorhinal Cells Yoko Yamaguchi1, Colin Molter1, Wu Zhihua1,2, Harshavardhan A. Agashe1, and Hiroaki Wagatsuma1 1

Lab For Dynamic of Emergent Intelligence, RIKEN Brain Science Institute, Wako, Saitama, Japan 2 Institute of Biophysics, Chinese Academy of Sciences, Beijing, China [email protected]

Abstract. This paper proposes a computational model of spatio-temporal property formation in the entorhinal neurons recently known as “grid cells”. The model consists of module structures for local path integration, multiple sensory integration and for theta phase coding of grid fields. Theta phase precession naturally encodes the spatial information in theta phase. The proposed module structures have good agreement with head direction cells and grid cells in the entorhinal cortex. The functional role of theta phase coding in the entorhinal cortex for cognitive map formation in the hippocampus is discussed. Keywords: Cognitive map, hippocampus, temporal coding, theta rhythm, grid cell.

1 Introduction In rodents, it is well known that a hippocampal neuron increases its firing rate in some specific position in an environment [1]. These neurons are called place cells and considered to provide neural representation of a cognitive map. Recently it was found that the entorhinal neurons, giving major inputs to the hippocampus fire at positions distributing in a form of a triangular-grid-like patterns in the environment [2]. They are called “grid cells” and their spatial firing preference is termed “grid fields”. Interestingly, temporal coding of space information, “theta phase precession” initially found in hippocampal place cells were also observed in grid cells in the superficial layer of the entorhinal cortex [3], as shown in Figs.1 – 3. A sequence of neural firing is locked to theta rhythm (4~12 HZ) of local field potential (LFP) during spatial exploration. As a step to understand cognitive map formation in the rat hippocampus, the mechanism to form the grid field and also the mechanism of phase precession formation in grid cells must be clarified. Here we propose a model of neural computation to create grid cells based on known property of entorhinal neurons including “head direction cells” which fires M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 151–159, 2008. © Springer-Verlag Berlin Heidelberg 2008

152

Y. Yamaguchi et al.

when the animal’s head has some specific direction in the environment. We demonstrate that theta phase precession in the entorhinal cortex naturally emerge as a consequence of grid cell formation mechanism.

Fig. 1. Theta phase precession observed in rat hippocampal place cells. When the rat traverses in a place field, spike timing of the place cell gradually advances relative to local field potential (LFP) theta rhythm. In a running sequence through place filed A-B-C, the spike sequence in order of A-B-C emerge in each theta cycle. The spike sequence repeatedly encoded in theta phase is considered to lead robust on-line memory formation of the running experience through asymmetric synaptic plasticity in the hippocampus.

Fig. 2. Network structure of the hippocampal formation (DG, CA3, CA1, the entorhinal cortex (EC deeper layer and EC superficial payer) and cortical areas giving multimodal input. Theta phase precession was initially found in the hippocampus, and also found in EC superficial layer. EC superficial layer can be considered as an origin of theta phase precession.

A Computational Model of Formation of Grid Field and Theta Phase Precession

153

HC place cell EC grid cell EC LFP theta

Time Fig. 3. Top) A grid field in an entorhinal grid cell (left) and a place field (right) in a hippocampal place cell. Bottom) Theta phase precession observed in the EC grid cell and in the hippocampus place cell.

2 Model Firing rate of the ith grid cell at a location (x, y) in a given environment increases in the condition given by the relation:

x = α i + nAi cos φ i + mAi cos(φ i + π / 3), y = β i + nAi sinφ i + mAi sin(φ i + π / 3), with

n, m = integer + r ,

(1)

where φ i , Ai and ( α i , β i ) denote one of angles characterizing the grid orientation, a distance of nearby vertices, and a spatial phase of the grid field in an environment. The parameter r is less than 1.0 giving the relative size of a field with high firing rate.

154

Y. Yamaguchi et al.

Fig. 4. Illustration of a hypercolumn structure for grid field computation in the hypothesized entorhinal cortex. The bottom layer consists of local path integration module with a hexagonal direction system. The middle layer associates output of local path integration and visual cue in a given environment. The top layer consists of triplet of grid cells whose grid fields have a common orientation, a common spatial scale and complementary spatial phases. Phase precession is generated at the grid cell at each grid field.

The computational goal to create a grid field is to find the region with n, m = integer + r . We hypothesize that the deeper layer of the entorhinal cortex works as local path integration systems by using head direction and running velocity. The local path integration results in a variable with slow gradual change forming a grid field. This change can cause the gradual phase shift of theta phase precession in accordance with the phenomenological model of theta phase precession by Yamaguchi et al. [4]. The schematic structure of the hypothesized entorhinal cortex and multimodal sensory system is illustrated in Fig. 4. The entorhinal layer includes head direction cells in the deeper layer and grid cells in the superficial layer. Cells with theta phase precession can be considered as stellate cells. The set of modules along vertical direction form a kind of functional column with a direction preference. These columns form a hypercolumnar structure with a set of directions. Mechanisms in individual modules are explained below. 2.1 A Module of Local Path Integrator

The local path integration module consists of six units. During animal’s locomotion with a given head direction and velocity, each unit integrates running distance in each direction with an angle dependent coefficient. Units have preferred vector directions distributing with π/3 intervals as shown in Fig. 5. Computation of animal displacement in given directions in this module is illustrated in Fig, 6. The maximum integration length of the distance in each direction is assumed to be common in a module, corresponding to the distance between nearby vertices of the subsequently formed grid field. This computation gives (n.m) in eq. (1). These systems distribute in the deeper layer of the entorhinal cortex in agreement with observation of head direction cells. Different modules have different vector

A Computational Model of Formation of Grid Field and Theta Phase Precession

155

directions and form a hypercolumn set covering the entire running directions or entire orientation of resultant grid field. The entorhinal cortex is considered to include multiple hypercolumns with different spatial scales. They are considered to work in parallel possibly to give stability in a global space by compensating accumulation of local errors.

Fig. 5. Left) A module of local path integrator with hexagonal direction vectors and a common vector size. Right) Activity coefficient of each vector unit. A set of these vector units to give a displacement distance measure computes an animal motion in a give head direction.

Fig. 6. Illustration of computation of local path integration in a module. Animal locomotion in a give head direction is computed by a pair of vector units among six vectors to give a position measure.

2.2 Grid Field Formation with Visual Cues

Computational results of local path integration are projected to next module in the superficial layer of the entorhinal cortex, which has multiple sensory inputs in a given environment. The association of path integration and visual cues results in the relative location of path integration measure ( α i , β i ) in eq. (1) in the module. Further interaction in a set of three cells as shown in Fig. 7 can give robustness of the parameter ( α i , β i ). Possible interaction among these cells is mutual inhibition to give supplementary distribution of three grid fields.

156

Y. Yamaguchi et al.

2.3 Theta Phase Precession in the Grid Field

The input of the parameter (n, m) and ( α i , β i ), to a cell at next module, at the top part of the module, can cause theta phase precession. It is obtained by the fundamental mechanism of theta phase generation proposed by Yamaguchi et al. [4] [5]. The mechanism needs the presence of a gradual increase of natural frequency in a cell with oscillation activity. Here we find that the top module consists of stellate cells with intrinsic theta oscillation. The natural increase in frequency is expected to emerge by the input of path integration at each vertex of a grid field.

Fig. 7. A triplet of grid fields with the same local path integration system and different spatial phases can generate mostly uniform spatial representation where a single grid cell fires at every location. The uniformity can help robust assignment of environmental spatial phases under the help of environmental sensory cues. The association is processed in the middle part of each column in the entorinal cortex. Projection of each cell output to the module at the top generates a grid field wit theta phase precession as explained in text.

3 Mathematical Formulation Simple mathematical formulation of the above model is phenomenologically given below. The locomotion of animal is represented by current displacement (R, φc ) computed with head direction φ , and running velocity. An elementary vector at a H column of in local path integration system has a vector angle φ and its length A. The i output of the ith vector system I is given by

⎧⎪1 if I (φi ) = ⎨ ⎪⎩ 0

φi − φ H < π / 2 and − r < S (φi ) < r , otherwise, (2)

with S (φi ) = R cos(φi − φ c ) ( S mod A ).

A Computational Model of Formation of Grid Field and Theta Phase Precession

157

where r and A respectively represent the field radius and the distance between neighboring grid vertices. The output of path integration module Di to the middle layer is given by

Di = ∏ I (φi + kπ / 3). k

(3)

Through association with visual cues, spatial phase of the grid is determined. (Details are not shown here.) The term Eqs. (2-3) from the middle layer to the top layer gives on-off regulation and also a parameter with gradual increase in a grid field. Dynamics of the membrane potential G i of the cell at the top layer is given by d dt

G i = f (G i ,t) + aDi S(φi ) + I theta ,

(4)

where f is a function of time-dependent ionic currents and a is constant. The last term I theta denotes a sinusoidal current representing theta oscillation of inhibitory neurons. In a proper dynamics of f, the second term in the right had side gives activation of the grid cell oscillation and gradual increase in its natural frequency. According to our former results by using a phenomenological model [5], the last term of theta currents leads phase locking of grid cells with gradual phase shift. This realizes a cell with grid field and theta phase precession. One can test Eq. (4) by applying several types of equations including a simple reduced model and biophysical model of the hippocampus or entorhinal cells. An example of computer experiments is given in the following section.

4 Computer Simulation of Theta Phase Precession The mechanism of theta phase precession was phenomenologically proposed by Yamaguchi et al. [4][5] as coupling of two oscillations. One is LFP theta oscillation with a constant frequency of theta rhythm. The other is a sustained oscillation with gradual increase in natural frequency. The sustained oscillation in the presence of LFP theta exhibits gradual phase shift as quasi steady states of phase locking. The simulation by using a hippocamal pyramidal cell [6] is shown in Fig. 8. It is obviously seen that LFP theta instantaneously captures the oscillation with gradual increase in natural frequency into a quasi-stable phase at each theta cycle to give gradual phase shift. This phase shift is robust against any perturbation as a consequence of phase locking in nonlinear oscillations. The simulation with a model of an entorhinal stellate cell [7] was also elucidated. We obtained similar phase precession with stellate cell model. One important property of stellate cell is the presence of sub threshold oscillations, while synchronization of this oscillation can be reduced to a simple behavior of the phase model. Thus, the mechanism of phenomenological model [5] is found to endow comprehensive description of phase locking of complex biophysical neuron models.

158

Y. Yamaguchi et al.

(a)

(b) Fig. 8. Computer experiment of theta phase precession by using a hippocampal pyramidal neuron model [6]. (a)Bottom: Input current with gradual increase. Top: Resultant sustained oscillation with gradual increase in natural frequency of the membrane potential. (b) In the presence of LFP theta (middle), the neuronal activity exhibit theta phase precession.

5 Discussions and Conclusion We elucidated a computational model of grid cells in the entorhnal cortex to investigate how temporal coding works for spatial representation in the brain, A computational model of formation of grid field was proposed based on local path integration. This assumption was found to give theta phase precession within the grid field. This computational mechanism does not need an assumption of learning in repeated trials in a novel environment but enables instantaneous spatial representation. Furthermore, this model has good agreements with experimental observations of head direction cells and grid cells. The networks proposed in the model predict local interaction networks in the entorhinal cortex and also head direction systems distributed in many areas. Although computation of place cells based on grid cells is beyond this paper, emergence of theta phase precession in the entorhinal cortex can be used for place cell formation and also instantaneous memory formation in the hippocampus [8]. These computational model studies with space-time structure for environmental space representation enlightens the temporal coding over distributed areas used in real-time operation of spatial information in ever changing environment.

A Computational Model of Formation of Grid Field and Theta Phase Precession

159

References 1. O’Keefe, J., Nadel, L.: The hippocampus as a cognitive map. Clarendon Press, Oxford (1978) 2. Fyhn, M., Molden, S., Witter, M., Moser, E.I., Moser, M.B.: Spatial representation in the entorhinal cortex. Sience 305, 1258–1264 (2004) 3. Hafting, T., Fyhn, M., Moser, M.B., Moser, E.I.: Phase precession and phase locking in entorhinal grid cells. Program No. 68.8, Neuroscience Meeting Planner. Atlanta, GA: Society for Neuroscience (2006.) Online (2006) 4. Yamaguchi, Y., Sato, N., Wagatsuma, H., Wu, Z., Molter, C., Aota, Y.: A unified view of theta-phase coding in the entorhinal-hippocampal system. Current Opinion in Neurobiology 17, 197–204 (2007) 5. Yamaguchi, Y., McNaughton, B.L.: Nonlinear dynamics generating theta phase precession in hippocampal closed circuit and generation of episodic memory. In: Usui, S., Omori, T. (eds.) The Fifth International Conference on Neural Information Processing (ICONIP 1998) and The 1998 Annual Conference of the Japanese Neural Network Society (JNNS 1998), Kitakyushu, Japan. Burke, VA, vol. 2, pp. 781–784. IOS Press, Amsterdam (1998) 6. Pinsky, P.F., Rinzel, J.: Intrinsic and network rhythmogenesis in a reduced traub model for CA3 neurons. Journal of Computational Neuroscience 1, 39–60 (1994) 7. Fransén, E., Alonso, A.A., Dickson, C.T., Magistretti, J., Hasselmo, M.E.: Ionic mechanisms in the generation of subthreshold oscillations and action potential clustering in entorhinal layer II stellate neurons 14(3), 368–384 (2004) 8. Molter, C., Yamaguchi, Y.: Theta phase precession for spatial representation and memory formation. In: The 1st International Conference on Cognitive Neurodynamics (ICCN 2007), Shanghai, 2-09-0002 (2007)

Working Memory Dynamics in a Flip-Flop Oscillations Network Model with Milnor Attractor David Colliaux1,2 , Yoko Yamaguchi1 , Colin Molter1 , and Hiroaki Wagatsuma1 1

Lab for Dynamics of Emergent Intelligence, RIKEN BSI, Wako, Saitama, Japan 2 Ecole Polytechnique (CREA), 75005 Paris, France [email protected]

Abstract. A phenomenological model is developed where complex dynamics are the correlate of spatio-temporal memories. If resting is not a classical fixed point attractor but a Milnor attractor, multiple oscillations appear in the dynamics of a coupled system. This model can be helpful for describing brain activity in terms of well classified dynamics and for implementing human-like real-time computation.

1

Introduction

Neuronal collective activities of the brain are widely characterized by oscillations in human and animals [1][2]. Among various frequency bands, distant synchronization in theta rhythms (4-8 Hz oscillation defined in human EEG) is recently known to relate with working memory, a short-term memory for central execution in human scalp EEG [3][4] and in neural firing in monkeys [5][6]. For long-term memory, information coding is mediated by synaptic plasticity whereas short-term memory is stored in neural activities [7]. Recent neuroscience reported various types of persistent activities of a single neuron and a population of neurons as possible mechanisms of working memory. Among those, bistable states, up- and down-states, of the membrane potential and its flip-flop transitions were measured in a number of cortical and subcortical neurons. The up-state, characterized by frequent firing, shows stability for seconds or more due to network interactions [8]. However it is little known whether flip-flop transition and distant synchronization work together or what kind of processings are enabled by the flip-flop oscillation network. Associative memory network with flip-flop change was proposed for working memory with classical rate coding view [9], while further consideration on dynamical linking property based on firing oscillation, such as synchronization of theta rhythms referred above, is likely essential for elucidation of multiple attractor systems. Besides, Milnor extended the concept of attractors to invariant sets with Lyapunov unstability, which has been of interest in physical, chemical and biological systems. It might allow high freedom in spontaneous switching among semi-stable states [12]. In this paper, we propose a model of oscillation associative memory with flip-flop change for working memory. We found that M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 160–169, 2008. c Springer-Verlag Berlin Heidelberg 2008 

Working Memory Dynamics in a Flip-Flop Oscillations Network Model

161

the Milnor attractor condition is satisfied in the resting state of the model. We will first study how the Milnor attractor appears and will then show possible behaviors of coupled units in the Milnor attractor condition.

2 2.1

A Network Model Structure

In order to realize up- and down-states where up-state is associated with oscillation, phenomenological models are joined. Traditionally, associative memory networks are described by state variables representing the membrane potential {Si } [9]. Oscillation is assumed to appear in the up-state as an internal process within each variable φi for the ith unit. Oscillation dynamics is simply given by a phase model with a resting state and periodic motion [10,11]. cos(φi ) stands for an oscillation current in the dynamics of the membrane potential. 2.2

Mathematical Formulation of the Model

The flip-flop oscillations network of N units is described by the set of state variables {Si , φi } ∈ N × [0, 2π[N (i ∈ [1, N ]). Dynamic of Si and φi is given by the following equations:  dSi Wij R(Sj ) + σ(cos(φi ) − cos(φ0 )) + I± dt = −Si + dφi (1) = ω + (β − ρS i )sin(φi ) dt with R(x) = 12 (tanh(10(x − 0.5)) + 1), φ0 = arcsin( −ω β ) and cos(φ0 ) < 0. R is the spike density of units and input I± will be taken as positive (I+ ) or negative (I− ) pulses (50 time steps), so that we can focus on the persistent activity of units after a phasic input. ω and β are respectively the frequency and the stabilization coefficient of the internal oscillation. ρ and σ represent mutual feedback between internal oscillation and membrane potential. Wij are the connection weights describing the strength of coupling between units i and j. φ0 is known to be a stable fixed point of the equation for φ, and 0 to be a fixed point for the S equation.

3 3.1

An Isolated Unit Resting State

The resting state is the stable equilibrium when I = 0 for a single unit. We assume ω < β so that M0 = (0, φ0 ) is the fixed point of the system. To study the linear stability of this fixed point, we write the stability matrix around M0 :   −1 −σsin(φ0 ) DF |M0 = (2) −ρsin(φ0 ) βcos(φ0 )

162

D. Colliaux et al.

The sign of the eigenvalues of DF |M0 and thus the stability of M0 depends only on μ = ρσ. With our choice of ω = 1 and β = 1.2, μc ≈ 0.96. If μ < μc , M0 is a stable fixed point and there is another fixed point M1 = (S1 , φ1 ) with φ1 < φ0 which is unstable. If μ > μc , M0 is unstable and M1 is stable with φ1 > φ0 . Fixed points exchange stability as the bifurcation parameter μ increases (transcritical bifurcation). The simplified system according to eigenvectors (X1 , X2 ) of the matrix DF |M0 gives a clear illustration of the bifurcation as dx1 dt dx2 dt

= ax21 + λ1 x1 = λ2 x2

(3)

Here a = 0 is equivalent to μ = μc and in this condition there is a positive measure basin of attraction but some directions are unstable. The resting state M0 is not a classical fixed point attractor because it does not attract all trajectories from an open neighborhood, but it is still an attractor if we consider Milnor’s extended definition of attractors. Phase plane (S, φ) Fig. 1 shows that for μ close to the critical value, nullclines cross twice staying close to each other in between. That narrow channel makes the configuration indistinguishable from a Milnor attractor in computer experiments.

Fig. 1. Top: Phase space (S, φ) with vector field and nullclines of the system. The dashed domain in B shows that M0 have positive measure basin of attraction when μ = μc . Bottom: Fixed points with their stable and unstable directions for the equivalent simplified system. A: μ < μc . B: μ = μc . C:μ > μc .

Since we showed μ is the crucial parameter for the stability of the resting state, we can now consider ρ = 1 and study the dynamics according to σ with a close look near the critical regime (σ = μc ).

Working Memory Dynamics in a Flip-Flop Oscillations Network Model

3.2

163

Constant Input Can Give Oscillations

Under constant input there are two possible dynamics: fixed point and limit cycle. If    ω    (4) β − S  < 1 there is a stable fixed point (S1 , φ1 ) with φ1 solution of ω + (β − σ(cos(φ1 ) − cos(φ0 )) − I)sin(φ1 ) = 0 S1 = σ(cosφ1 − cosφ0 ) + I

(5)

If condition 4 is not satisfied, the φ equation in 1 will give rise to oscillatory dynamics. Identifying S with its temporal average, dφ dt = ω + Γ sin(φ) with  2π dφ Γ = β − S will be periodic with period 0 ω+(β−S)sin(φ) . This approximation   gives an oscillation at frequency ω = ω 2 − (β − S)2 , which is qualitatively in good agreement with computer experiments Fig. 2. σ=μc 6

1.2 S minimum S maximum Frequency (theoretical) Frequency

5

1

4

0.8

3 f

S

0.6 2 0.4 1 0.2 0 0 -1 -0.4

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

1.4

I

Fig. 2. For each value of constant current I, maximum and minimum values of S1 are plotted. Dominant frequency of S1 obtained by FFT is compared to the theoretical value when S is identified with its temporal average: Frequency VS Frequency (theoretical).

If we inject an oscillatory input into the system, S oscillates at the same frequency provided the input frequency is low. For higher frequencies, S cannot follow the input and shows complex oscillatory dynamics with multiple frequencies.

164

4

D. Colliaux et al.

Two Coupled Units

For two coupled units, flip-flop of oscillations is observed under various conditions. We will analyze the case μ = 0 and flip-flop properties under various strengths of connection weights, assuming symmetrical connections (W12 = W2,1 = W ). 4.1

Influence of the Feedback Loop

In equation 1, ρ and σ implement a feedback loop representing mutual influence of φ and S for each unit. The Case µ = 0. In the case σ = 0 or ρ = 0, φ remains constant φ = φ0 : the system is then a classical recurrent network. This model was used to provide associative memory network storing patterns in fixed point attractors [9]. For small coupling strength, the resting state is a fixed point. For strong coupling strength, two more fixed points appear, one unstable, corresponding to threshold, and one stable, providing memory storage. After a transient positive input I+ above threshold, the coupled system will be in up-state. A transient negative input I− can bring it back to resting state. For a small perturbation (σ  1 and ρ = 1), the active state is a small up-state oscillation but associative memory properties (storage, completion) are preserved. Growing Oscillations. The up-state oscillation in the membrane potential dynamics triggered by giving an I+ pulse to unit 1 grows when σ increases and saturates to an up-state fixed point for strong feedback. Interestingly, for a range of feedback strength values near μc , S returns transiently near the Milnor attractor resting state. Projection of the trajectories of the 4-dimensional system on a 2-dimensional plane section P illustrates these complex dynamics Fig. 3. A cycle would intersect this plane in two points. For each σ value, we consider S1 for these intersection points. For a range between 0.91 and 1.05 with our choice of parameters, there are much more than two intersection points M*, suggesting chaotic dynamics. 4.2

Influence of the Coupling Strength

The dynamics of two coupled units can be a fixed point attractor, as in the resting state (I = 0), or down-state or up-state oscillation (depending on the coupling strength), after a transient input. Near critical value of the feedback loop, in addition to these, more complex dynamics occur for intermediate coupling strength.

Working Memory Dynamics in a Flip-Flop Oscillations Network Model

165

Fig. 3. A: Influence of the feedback loop- Bifurcation diagram according to σ (Top). S1 coordinates of the intersecting points of the trajectory with a plane section P according to σ(Bottom). B: Influence of the coupling strengh - S1 maximum and minimum values and average phase difference (φ1 − φ2 ) according to W (Top). S1 coordinates of the intersecting points of the trajectory with a plane section P according to W (Bottom).

Down-state Oscillation. For small coupling strength, the system periodically visits the resting state for a long time and goes briefly to up-state. The frequency of this oscillation increases with coupling strength. The two units are anti-phase (when Si takes maximum value, Sj takes minimum value) Fig. 4 (Bottom). Up-state Oscillation. For strong coupling strength, a transient input to unit 1 leads to an up-state oscillation Fig. 4 (Top). The two units are perfectly in-phase at W = 0.75 and phase difference stays small for stronger coupling strength. Chaotic Dynamics. For intermediate coupling strength, an intermediate cycle is observed and more complex dynamics occur for a small range (0.58 < W < 0.78

166

D. Colliaux et al.

Fig. 4. Si temporal evolution, (S1 , S2 ) phase plane and (Si , φi ) cylinder space. Top: Up-state oscillation for strong coupling. Middle: Multiple frequency oscillation for intermediate coupling. Bottom: Down-state oscillation for weak coupling.

with our parameters) before full synchronization characterized by φ1 − φ2 = 0. The trajectory can have many intersection points with P and S ∗ in Fig. 3 shows multiple roads to chaos through period doubling.

Working Memory Dynamics in a Flip-Flop Oscillations Network Model

5

167

Application to Slow Selection of a Memorized Pattern

5.1

A Small Network

The network is a set N of five units consisting in a subset N1 of three units A,B and C and another N2 of two units D and E. In the set N, units have symmetrical all-to-all weak connections (WN = 0.01) and in each subset units have symmetrical all-to-all strong connections (WNi = 0.1 ∗ M ) with M a global parameter slowly varying in time between 1 and 10. These subsets could represent two objects stored in the weight matrix. 5.2

Memory Retrieval and Response Selection

S

We consider a transient structured input into the network. For constant M, a partial or complete stimulation of a subset Ni can elicit retrieval and completion of the subset in an up-state as would do a classical auto-associative memory network.

2 1.5 1 0.5 0

A 0

20000

40000

60000

80000

100000

120000

140000

80000

100000

120000

140000

80000

100000

120000

140000

80000

100000

120000

140000

80000

100000

120000

140000

S

t 2 1.5 1 0.5 0

B 0

20000

40000

60000

S

t 2 1.5 1 0.5 0

C 0

20000

40000

60000

S

t 2 1.5 1 0.5 0

D 0

20000

40000

60000

S

t 2 1.5 1 0.5 0

E 0

20000

40000

60000 t

Fig. 5. Slow activation of a robust synchronous up-state in N1 during slow increase of M

In the Milnor attractor condition more complex retrieval can be achieved when M is slowly increased. As an illustration, we consider transient stimulation of units A and B from N1 and unit E from N2 Fig. 5. N2 units show anti-phase

168

D. Colliaux et al.

oscillations with increasing frequency. N1 units first show synchronous downstate oscillations with long stays near the Milnor attractor and gradually go toward sustained up-state oscillations. In this example, the selection of N1 in up-state is very slow and synchrony between units plays an important role.

6

Conclusion

We demonstrated that, in cylinder space, a Milnor attractor appears at a critical condition through forward and reverse saddle-node bifurcations. Near the critical condition, the pair of saddle and node constructs a pseudo-attractor, which can serves for observation of Milnor attractor-like properties in computer experiments. Semi-stability of the Milnor attractor in this model seems to be associated with the variety of oscillations and chaotic dynamics through period doubling roads. We demonstrated that an oscillations network provides a variety of working memory encoding in dynamical states under the presence of a Milnor attractor. Applications of oscillatory dynamics have been compared to classical autoassociative memory models. The importance of Milnor attractors was proposed in the analysis of coupled map lattices in high dimension [11] and for chaotic itinerancy in the brain [13]. The functional significance of flip-flop oscillations networks with the above dynamical complexity is of interest for further analysis of integrative brain dynamics.

References 1. Varela, F., Lachaux, J.-P., Rodriguez, E., Martinerie, J.: The brainweb: Phase synchronization and large-scale integration. Nature Reviews Neuroscience (2001) 2. Buzsaki, G., Draguhn, A.: Neuronal oscillations in cortical networks. Science (2004) 3. Onton, J., Delorme, A., Makeig, S.: Frontal midline EEG dynamics during working memory. NeuroImage (2005) 4. Mizuhara, H., Yamaguchi, Y.: Human cortical circuits for central executive function emerge by theta phase synchronization. NeuroImage (2004) 5. Rainer, G., Lee, H., Simpson, G.V., Logothetis, N.K.: Working-memory related theta (4-7Hz) frequency oscillations observed in monkey extrastriate visual cortex. Neurocomputing (2004) 6. Tsujimoto, T., Shimazu, H., Isomura, Y., Sasaki, K.: Prefrontal theta oscillations associated with hand movements triggered by warning and imperative stimuli in the monkey. Neuroscience Letters (2003) 7. Goldman-Rakic, P.S.: Cellular basis of working memory. Neuron (1995) 8. McCormick, D.A.: Neuronal Networks: Flip-Flops in the Brain. Current Biology (2005) 9. Durstewitz, D., Seamans, J.K., Sejnowski, T.J.: Neurocomputational models of working memory. Nature Neuroscience (2000) 10. Yamaguchi, Y.: A Theory of hippocampal memory based on theta phase precession. Biological Cybernetics (2003)

Working Memory Dynamics in a Flip-Flop Oscillations Network Model

169

11. Kaneko, K.: Dominance of Minlnor attractors in Globally Coupled Dynamical Systems with more than 7 +- 2 degrees of freedom (retitled from ‘Magic Number 7 +- 2 in Globally Coupled Dynamical Systems’) Physical Review Letters (2002) 12. Fujii, H., Tsuda, I.: Interneurons: their cognitive roles - A perspective from dynamical systems view. Development and Learning (2005) 13. Tsuda, I.: Towards an interpretation of dynamic neural activity in terms of chaotic dynamical systems. Behavioural and Brain Sciences (2001)

Corticopetal Acetylcholine: Possible Scenarios on the Role for Dynamic Organization of Quasi-Attractors Hiroshi Fujii1,2, Kazuyuki Aihara2,3, and Ichiro Tsuda4,5 1

Department of Information and Communication Sciences, Kyoto Sangyo University, Kyoto 603-8555, Japan [email protected] 2 Institute of Industrial Science, the University of Tokyo, Tokyo 153-8505 [email protected] 3 ERATO, Japan Science and Technology Agency, Tokyo 151-0065, Japan 4 Research Institute for Electronic Science, Hokkaido University, Sapporo 060-0812, Japan [email protected] 5 Center of Excellence COE in Mathematics, Department of Mathematics, Hokkaido University, Sapporo 060-0810, Japan

（ ）

Abstract. A new hypothesis on a possible role for the corticopetal acetylcholine (ACh) is provided from a dynamical systems standpoint. The corticopetal ACh helps to transiently organize a global (inter- and intra-cortical) quasi-attractors via gamma range synchrony when it is behaviorally needed as top-down attentions and expectation.

1 Introduction 1.1 Corticopetal Acetylcholine Achetylcholine (ACh) is the first substance identified as a neurotransmitter by Otto Loewi [19]. Although it is increasingly recognized that ACh plays a critical role, not only in arousal and sleep, but in higher cognitive functions as attention, conscious flow, and so on, the question on the way in which ACh works in those cognitive processes remains a mystery [11]. The corticopetal ACh, originated in the nucleus basalis of Meinert (NBM), a part of the basal forebrain (BF), is the primary source of cortical ACh, and the major target of BF projections is the cortex [21]. Behavioral studies and those using immunotoxin as well provide consistent evidence of the role of ACh in top-down attentions. A blockage of NBM ACh, either by deasease-related or drug-induced, causes a severe loss of attentions: selective attention, sustained attention, and divided attention together with a shift of attention. ACh concerns conscious flow (Perry & Perry [24]). Continual death of cholinergic neurons in NBM causes Lewy Body Dementia (LBD), one of the most salient symptoms of which is the complex visual hallucination (CVH) [1].1 1

Perry and Perry [24] noted those hallucinatory LBD patients who see: “integrated images of people or animals which appear real at the time”, “insects on walls”, or “bicycles outside the fourth storey window”. Images are generally vivid and colored, continue for a few minutes (neither seconds nor hours). It is to be noted that “many of those experiences are enhanced during eye closed and relieved by visual input”, and “nicotinic anatagonists, such as mecamylamine, are not reported to induce hallucinations”.

M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 170–178, 2008. © Springer-Verlag Berlin Heidelberg 2008

Corticopetal Acetylcholine: Possible Scenarios on the Role for Dynamic Organization

171

1.2 Attentions, Cortical State Transitions and Cholinergic Control System from NBM Top-down flow of signals which accompanies attentions, expectation and so on may cause state transitions in the “down stream” cortices. Fries et al. [7] reported an increase of synchrony in high-gamma range in accordance of selective attention. (See, also Jones [15], Buschman et al. [2].) Metherate et al. [22] stimulated NBM in in vivo preparations of auditory cortex. Of particular interest in their observations is that NBM ACh produced a change in subthreshold membrane potential fluctuations from large-amplitude, slow (1-5 Hz) oscillations to low-amplitude, fast (20-40 Hz) (i.e., gamma) oscillations. A shift of spike discharge pattern from phasic to tonic was also observed.2 They pointed out that in view of the wide spread projections of NBM neurons, larger intercortical networks could also be modified. Together with Fries et al. data, it is suggested that NBM cholinergic projections may induce a state transition as a shift of frequency, and change of discharge pattern in neocortical neurons. This may be consistent with the observation made by Kay et al. [16]. During perceptual processing in the olfactory-limbic axis, a cascade of brain events at the successive stages of the task as “expectation” and/or “attention” was observed. ‘Local’ transitions of the olfactory structures indicated by modulations of EEG signals as gamma amplitude, periodicity, and coherence were reported to exist. Kay et al. also observed that the local dynamics transiently falls into attractor-like states. Such ‘local’ transitions of states are generally postulated to be triggered by ‘topdown’ glutamatergic spike volleys from “upper stream” organizations. However, such brain events of state transitions with a change in synchrony could be a result of collaboration of descending glutamatergic spike volleys and ascending ACh afferents from NBM. (See, also [26]).3

2 Neural Correlate of Conscious Percepts and the Role of the Corticopetal ACh 2.1 Neural Correlates of Conscious Percepts and Transient Synchrony The corticopetal ACh pathway might be the critical control system which may trigger various kinds of attentions receiving convergent inputs from other sensory and association areas as Sarter et al. [25], [26] argued. In order to discuss the role of the 2

NBM contains cholinergic neurons and non-cholinergic neurons. GABAergic neurons are at least twice more numerous than cholinergic neurons [15] The Metherate et al. observations (above) may be the result of collective functioning of both the cholinergic and GABAergic projections. Wenk [31] argued another possibility that the NBM ACh projections on the reticular thalamic nucleus might cause the cortical state change. 3 Triangular Attentional Pathway: The above arguments may better be complemented by the triangular amplification circuitry, a pathway consisting of parietal cortex → prefrontal cortex → NBM → sensory cortex [10]. This may constitute the cholinergic control system from NBM, i.e., the top-down attentional pathway for cholinergic modulations of responses in cortical sensory areas.

172

H. Fujii, K. Aihara, and I. Tsuda

corticopetal ACh related to attentions, we first begin with the question: What is the neural correlate of conscious percepts? The recent experiments by Kenet et al. [17] show the possibility that in a background (or spontaneous) state where no external stimuli exist, the visual cortex fluctuates between specific internal states. This might mean that cortical circuitry has a number of pre-existing and intrinsic internal states which represent features, and the cortex fluctuates between such multiple intrinsic states even when no external inputs exist. (See, also Treisman et al. [27].) Attention and Dynamical Binding Through Synchrony In order that perception of an object makes sense, its features, or fragmentary subassemblies, must be bound as a unity. How does the “binding” of those local fragmentary dynamics into a global dynamics is done? A widely accepted view is that top-down signals as expectation, attentions, and so on may play the role of such an integration, which is mediated by (possibly gamma) synchronization among the concerned assemblies representing stimuli or events. We postulate that such a process is a basis of global intra- and inter-cortical conjunctions of brain activities. See, Varela et al. [30],Womelsdorf et al. [32]. See, also Dehaene and Changeux [5].) The neural correlate of conscious percepts is a globally integrated state of related brain networks mediated by synchrony over gamma and other frequency bands. In mathematical terms, such transient processes of synchrony of global, but between selected groups of neurons may be described as a transitory state of approaching a global attractor. We note that such a “transitory” state may be conceptualized as an attractor ruin (Tsuda [28], Fujii et al. [8]), in which there are orbits approaching and stay there for a while, but may simultaneously possess repelling orbits from itself. The proper Milnor attractor and its perturbed structures can be a specific representation of attractor ruins [8]. However, since the concept of attractor ruins may include a wider class of non-classical attractors than the Milnor attractor, we may use the term “attractor ruins” in this paper to include possible but unknown classes of ruins. 2.2 Role of the Corticopetal ACh: A Working Hypothesis How can top-down attentions, expectation, etc. contribute to conscious perception with the aid of ACh? Assuming the arguments in the preceding section, this question could be translated into: “How do the corticopetal ACh projections work for the emergence of globally organized attractor ruins via transient synchrony?” We summarize our tentative proposition as a Working Hypothesis in the following. Working Hypothesis: The role of the corticopetal ACh accompanied with top-down contextual signals as attentions and so on is the mediator for dynamically organizing quasi-attractors, which are required in conscious perception, or in executing actions. ACh “integrates” multiple “floating subnetworks” into “a transient synchrony group” in the gamma frequency range. Such a transiently emerging synchrony group can be regarded as an attractor ruin in the dynamical systems-theoretic sense.

Corticopetal Acetylcholine: Possible Scenarios on the Role for Dynamic Organization

173

3 Do the Existing Experimental Data Support the Working Hypothesis? 3.1 Introductory Remarks: Transient Synchronization by Virtue of Pre- and Post-synaptic ACh Modulations ACh may have both pre-synaptic and post-synaptic effects on individual neurons in the cortex. First, top-down glutamatergic spike volleys flow into cortical layers, which might convey contextual information on stimuli. The corticopetal ACh arrives concomitantly with the glutamatergic volleys. If ACh release modulates synaptic connectivity between cortical neurons even in an effective sense by virtue of “pre-synaptic modulations”, metamorphosis of the attractor landscape4 should be inevitable. Post-synaptic influences of the corticopetal ACh on individual neurons – either inhibitory or excitatory, might cause deep effects on their firing behavior, and might induce a state transition with a collective gamma oscillation. A consequence of the three effects together might trigger a specific group of networks to oscillate in gamma frequency with phase synchrony. These are all speculative stories based on experimental evidence. We need at least to examine the present status on the experimental data concerning the cortical ACh influence on individual neurons. This may be the place to add a comment on the specificity of the corticopetal ACh projections on the cortex, which may be a point of arguments. It is reported that the cholinergic afferents specialized synaptic connections with post synaptic targets, rather than releasing ACh non-specifically (Turrini et al..[29].) 3.2 Controversy on Experimental Data Let us review quickly the existing experimental data. As noted before, “there exist little consensus among researchers for more than a half century” [11]. The following is not intended to give a complete review, but to give a preliminary knowledge which may be of help to understand the succeeding discussions. ACh have two groups of receptors, one is the muscarinic receptors, mAChRs with 5 subtypes, and the other is nicotinic receptors, nAChRs with 17 subtypes. The nAChR is a relatively simple cationic (Na+ and Ca2+) channel, the opening of which leads to a rapid depolarization followed by desensitization. Most of mAChRs activation exhibits the slower onset and longer lasting G-protein coupled second messenger generation. Here the primary interest is in mAChRs. 5 The functions of mAChRs are reported to be two-fold: one is pre-synaptic, and the other is postsynaptic modulations. 4

“Attractor landscape” is usually used for potential systems. Here, we use it to mean the landscape of “basins” (absorbing regions), of classical and non-classical attractors as attractor ruins. 5 The nicotinic receptors, nAChRs may work as a disinhibition system to layer 2/3 pyramidal neurons [4]), the exact function of which is not known yet.

174

H. Fujii, K. Aihara, and I. Tsuda

Post-synapticModulations The results of traditional studies may be divided into two opposing data. The majority view is that mAChRs function as excitatory transmitter for post-synaptic neurons (see, e.g., McCormick [20]), while there are minority data that claim inhibitory functioning. The latter, however, has been considered to be a consequence of ACh excitation of interneurons, which in turn may inhibit post-synaptic pyramidal neurons (PYR). Recently, Gulledge et al. [11], [12] stated that transient mAChR activation generates strong and direct transient inhibition of neocortical PYR. The underlying ionic process is the induction of calcium release from internal stores, and subsequent activation of small-conductance (SK–type) calcium-activated potassium channels. The authors claim that the traditional data do not describe the actions of transient mAChR activation, as is likely to happen during synaptic release of ACh by the following reasons. 1.

2.

In vivo ACh concentration: Previous studies typically used high concentrations of muscarinic agonists (1–100 mM). Extracellular concentrations of ACh in the cortex are at least one order of magnitude lower than those required to depolarize PYR in vitro. Phasic (transient) application vs. bath application: Most data depended on experiments with bath applications, which may correspond to prolonged, tonic mAChR stimulation. The ACh release accompanied with attentions, etc., would better correspond to a transient puff application as the authors’ experiment.

The specificity of ACh afferents on postsynaptic targets was already noted. [29]. Pre-synaptic Modulations Experimental works on pre-synaptic modulations are mostly based on ACh bath applications, and the modulation data were measured in terms of local field potentials (LFP). Most results claimed the pathway specificity of modulations. Typically, it was concluded that muscarinic modulation can strongly suppress intracortical(IC) synaptic activity while exerting less suppression, or actually enhancing, thalamocortical (TC) inputs. [14]. Gil et al. [9] reported that 5 μM muscarine decreases both IC and TC pathway transmission, and that those data were presynaptic effects, since membrane potential and input resistance were unchanged. Recently, Kuczewski et al. [18] studied the same problem, and obtained different results from the previous ones. Low ACh (less than 100 μM) shows facilitation, and as ACh concentration goes high, the result is depression. This is true for the both IC and TC pathways, i.e., for the layer 2/3 and layer 4. 3.3 Possible Scenarios The lack of consistent experimental data makes our job complicated. The situation might be compared to playing a jigsaw puzzle with, many pieces missing and some pieces mingled from other jigsaw pictures. What we can do at this moment may be to propose possible alternatives of scenarios for the role of the corticopetal ACh.

Corticopetal Acetylcholine: Possible Scenarios on the Role for Dynamic Organization

175

The following are the list of prerequisites and evidences on which our arguments should be based. 1.

2.

3.

Two modulations may occur simultaneously inside the 6 layers of the cortex. The firing characteristics of individual neurons, and the strength of synaptic connections may change dynamically either as post-synaptic or pre-synaptic modulations. Virtually no models, to our knowledge, have been proposed, which took the net effects of the two modulations into account. The interaction of ACh with top-down glutamatergic spike volleys should be considered. The majority of neurons alter in response to combined exposure to both acetylcholine and glutamate concomitantly. (Perry & Perry [24].) As a post-synaptic influence, ACh release may change the firing regime of neurons, and induce gamma oscillation [6], [23], [31].

As to pre-synaptic modulations, the details of synaptic processes appear to be largely unknown. The significance of experimental studies can not be overemphasized. Now let us try to draw a dessin for possible scenarios on the role of corticopetal ACh. Here we may put three corner stones for the models: 1. 2. 3.

Who triggers the gamma oscillation? Who (and how to) modulates the effective connectivity? What is the mechanism of phase synchrony and what is the role of it?

Scenario I The basic idea is that in the default, low level state of ACh, globally organized attractors do not virtually exist, and may take the form of floating fragmentary dynamics. Then, ACh release may help to strengthen the synaptic connections presynaptically. One of the roles of post-synaptic modulation is to start up the gamma oscillation. (Here the influence of GABAergic projections from NBM might play a role.) Another, but important role will be stated later. Scenario II The effective modulation of synaptic connectivity might be carried by, rather than the pre-synaptic modulation, the phase synchrony of the gamma oscillation itself which is triggered by the post-synaptic modulation. Such a mechanism for the change of synaptic connectivity, and resulting binding of fragmentary groups of neurons was proposed by Womelsdorf et al. [32]. They claimed that the mutual influence among neuronal groups depends on the phase relation between rhythmic activities within the groups. Phase relations supporting interactions among the groups preceded those interactions by a few milliseconds, consistent with a mechanistic role. See, also Buzsaki [3]. For the case of Scenario II, the role of the post-synaptic modulation is to start up the gamma oscillation, and the reset of its phase, as Gulledge and Stuart [11] suggested. The transient hyper-polarization may play the role of referee to start up the oscillation among the related groups in unison. The Scenario I makes the postsynaptic modulation carry the two roles of starting up the gamma oscillation, and

176

H. Fujii, K. Aihara, and I. Tsuda

resetting its phase. For the pre-synaptic modulation a bigger role of realization of attractors by virtue of synaptic strength modulation is assigned.

4 Concluding Discussions The critical role of the corticopetal ACh in cognitive functions, together with its relation to some disease-related symptoms as complex visual hallucinations in DLB, and its apparent involvements in the neocortical state change have motivated us to the study of the functional role(s) of the corticopetal ACh from dynamical systems standpoints. Cognitive functions are phenomena carried by the brain dynamics. We hope that understanding the cognitive dynamics with the dynamical systems language would open new theoretical horizons. It is of some help to consider the conceptual difference of the two “forces” which flow into the 6 layers of the neocortex. Glutamate spike volleys could be, if viewed as an event in a dynamical system, an external force, which may kick the orbit to another orbit, and may sometimes to out of the “basin” of the present attractor beyond the border – the separatrix. In contrary to this situation, ACh projections – though transient, could be regarded as a slow parameter working as a bifurcation parameter that modifies the landscape itself. What we are looking at in the preceding arguments is that the two phenomena happen concomitantly at the 6 layers of the cortex. Hasselmo and McGaughy [13] emphasized the ACh role in memory as: “high acetylcholine sets circuit dynamics for attention and encoding; low acetylcholine sets dynamics for consolidation”, which is based on some experimental data on selective pre-synaptic depression and facilitation. However, in view of the potential role of attentions in local bindings or global integrations, we may pose alternative (but not necessarily exclusive,) scenarios on the ACh function as temporarily modifying the quasi-attractor landscape, in collaboration with glutamatergic spike volleys. Rather, we speculate that the process of memorization itself would realized through such a dynamic formation of attractor ruins, for which mAChR may play a role.

Acknowledgements The first author (HF) was supported by a Grant-in-Aid for Scientific Research (C), No. 19500259, from the Ministry of Education, Culture, Sports, Science and Technology of the Japanese Government. The second author (KA) was partially supported by Grant-in-Aid for Scientific Research on Priority Areas 17022012 from the Ministry of Education, Culture, Sports, Science, and Technology, the Japanese Government. The third author (IT) was partially supported by a Grant-in-Aid for Scientific Research on Priority Areas, No. 18019002 and No. 18047001, a Grant-inAid for Scientific Research (B), No. 18340021, Grant-in-Aid for Exploratory Research, No. 17650056, a Grant-in-Aid for Scientific Research (C), No. 16500188, and the 21st Century COE Program, Mathematics of Nonlinear Structures via Singularities.

Corticopetal Acetylcholine: Possible Scenarios on the Role for Dynamic Organization

177

References 1. Behrendt, R.-P., Young, C.: Hallucinations in schizophrenia, sensory impairment, and brain disease: A unifying model. Behav. Brain Sci. 27, 771–787 (2004) 2. Buschman, T.J., Miller, E.K.: Top-down Versus Bottom-Up Control of Attention in the Prefrontal and Posterior Parietal Cortices. Science 315, 1860–1862 (2007) 3. Buzsaki, G.: Rhythms of the Brain. Oxford University Press, Oxford (2006) 4. Christophe, E., Roebuck, A., Staiger, J.F., Lavery, D.J., Charpak, S., Audinat, E.: Two Types of Nicotinic Receptors Mediate an Excitation of Neocortical Layer I Interneurons. J. Neurophysiol. 88, 1318–1327 (2002) 5. Dehaene, S., Changeux, J.-P.: Ongoing Spontaneous Activity Controls Access to Consciousness: A Neuronal Model for Inattentional Blindness. PLoS Biology 3, 910–927 (2005) 6. Detari, L.: Tonic and phasic influence of basal forebrain unit activity on the cortical EEG. Behav. Brain. Res. 115, 159–170 (2000) 7. Fries, P., Reynolds, J.H., Rorie, A.E., Desimone, R.: Modulation of Oscillatory Neuronal Synchronization by Selective Visual Attention. Science 291, 1560–1563 (2001) 8. Fujii, H., Aihara, K., Tsuda, I.: Functional Relevance of ‘Excitatory’ GABA Actionsin Cortical Interneurons: A Dynamical Systems Approach. J. Integrative Neurosci. 3, 183– 205 (2004) 9. Gil, Z., Connors, B.W., Yael Amitai, Y.: Differential Regulation of Neocortical Synapses by Neuromodulators and Activity. Neuron 19, 679–686 (1997) 10. Golmayo, L., Nunez, A., Zaborsky, L.: Electrophysiological Evidence for the Existence of a Posterior Cortical-Prefrontal-Basal Forebrain Circuitry in Modulating Sensory Responses in Visual and Somatyosensory Rat Cortical Areas. Neuroscience 119, 597–609 (2003) 11. Gulledge, A.T., Stuart, G.J.: Cholinergic Inhibition of Neocortical Pyramidal Neurons. J. Neurosci 25, 10308–10320 (2005) 12. Gulledge, A.T., Susanna, S.B., Kawaguchi, Y., Stuart, G.J.: Heterogeneity of phasic signaling in neocortical neurons. J. Neurophysiol. 97, 2215–2229 (2007) 13. Hasselmo, M.E., McGaughy, J.: High acetylcholine sets circuit dynamics for attention and encoding; Low acetylcholine sets dynamics for consolidation. Prog. Brain Res. 145, 207– 231 (2004) 14. Hsieh, C.Y., Cruikshank, S.J., Metherate, R.: Differential modulation of auditory thalamocortical and intracortical synaptic transmission by cholinergic agonist. Brain Res 880, 51–64 (2000) 15. Jones, B.E., Muhlethaler, M.: Cholinergic and GABAergic neurons of the basal forebrain: role in cortical activation. In: Lydic, R., Baghdoyan, H.A. (eds.) Handbook of Behavioral State Control, pp. 213–233. CRC Press, London (1999) 16. Kay, L.M., Lancaster, L.R., Freeman, W.J.: Reafference and attractors in the olfactory system during odor recognition. Int. J. Neural Systems 4, 489–495 (1996) 17. Kenet, T., Bibitchkov, D., Tsodyks, M., Grinvald, A., Arieli, A.: Nerve cell activity when eyes are shut reveals internal views of the world. Nature 425, 954–956 (2003) 18. Kuczewski, N., Aztiria, E., Gautam, D., Wess, J., Domenici, L.: Acetylcholine modulates cortical synaptic transmission via different muscarinic receptors, as studied with receptor knockout mice. J. Physiol. 566.3, 907–919 (2005) 19. Loewi, O.: Ueber humorale Uebertragbarkeit der Herznervenwirkung. Pflueger’s Archiven Gesamte Physiologie 189, 239–242 (1921) 20. McCormick, D.A., Prince, D.A.: Mechanisms of action of acetylcholine in the guinea-pig cerebral cortex in vitro. J. Physiol. 375, 169–194 (1986)

178

H. Fujii, K. Aihara, and I. Tsuda

21. Mesulam, M.M., Mufson, E.J., Levey, A.I., Wainer, B.H.: Cholinergic innervation of cortex by the basal forebrain: cytochemistry and cortical connections of the septal area, diagonal band nuclei, nucleus basalis (substantia innominata), and hypothalamus in the rhesus monkey. J. Comp. Neurol. 214, 170–197 (1983) 22. Metherate, R., Charles, L., Cox, C.L., Ashe, J.H.: Cellular Bases of Neocortical Activation: Modulation of Neural Oscillations by the Nucleus Basalis and Endogenous Acetylcholine. J. Neurosci. 72, 4701–4711 (1992) 23. Niebur, E., Hsiao, S.S., Johnson, K.O.: Synchrony: a neuronal mechanism for attentional selection? Curr. Opin. Neurobiol. 12, 190–194 (2002) 24. Perry, E.K., Perry, R.H.: Acetylcholine and Hallucinations: Disease-Related Compared to Drug-Induced Alterations in Human Consciousness. Brain Cognit. 28, 240–258 (1995) 25. Sarter, M., Gehring, W.J., Kozak, R.: More attention should be paid: The neurobiology of attentional effort. Brain Res. Rev. 51, 145–160 (2006) 26. Sarter, M., Parikh, V.: Choline Transporters, Cholinergic Transmission and Cognition. Nature Reviews Neurosci. 6, 48–56 (2005) 27. Treisman, A.M., Gelade, G.: A feature-integration theory of attention. Cognit. Psychol. 12, 97–136 (1980) 28. Tsuda, I.: Chaotic Itinerancy as a Dynamical Basis of Hermeneutics of Brain and Mind. World Future 32, 167–185 (1991) 29. Turrini, P., Casu, M.A., Wong, T.P., De Koninck, Y., Ribeiro-da-Silva, A., And Cuello, A.C.: Cholinergic nerve terminals establish classical synapses in the rat cerebral cortex: synaptic pattern and age—related atrophy. Neiroscience 105, 277–285 (2001) 30. Varela, F., Lachaux, J.-P., Rodriguez, E., Martinerie, J.: The Brainweb: Phase synchronization and large-scale integration. Nature Rev. Neurosci. 2, 229–239 (2001) 31. Wenk, G.L.: The Nucleus Basalis Magnocellularis Cholinergic System: One Hundred Years of Progress. Neurobiol. Learn. Mem. 67, 85–95 (1997) 32. Womelsdorf, T., Schoffelen, J.M., Oostenveld, R., Singer, W., Desimone, R., Engel, A.K., Fries, P.: Modulation of neuronal interactions through neuronal synchronization. Science 316, 1578–1579 (2007)

Tracking a Moving Target Using Chaotic Dynamics in a Recurrent Neural Network Model Yongtao Li and Shigetoshi Nara Graduate School of Natural Science and Technology, Okayama University, 3-1-1 Tsushima-naka, Okayama 700-8530, Japan [email protected]

Abstract. Chaotic dynamics introduced in a recurrent neural network model is applied to controlling an tracker to track a moving target in two-dimensional space, which is set as an ill-posed problem. The motion increments of the tracker are determined by a group of motion functions calculated in real time with firing states of the neurons in the network. Several groups of cyclic memory attractors that correspond to several simple motions of the tracker in two-dimensional space are embedded. Chaotic dynamics enables the tracker to perform various motions. Adaptively real-time switching of control parameter causes chaotic itinerancy and enables the tracker to track a moving target successfully. The performance of tracking is evaluated by calculating the success rate over 100 trials. Simulation results show that chaotic dynamics is useful to track a moving target. To understand them further, dynamical structure of chaotic dynamics is investigated from dynamical viewpoint. Keywords: Chaotic dynamics, tracking, moving target, neural network.

1 Introduction Biological systems have became a hot research around the world because of their excellent functions not only in information processing, but also in well-regulated functioning and controlling, which work quite adaptively in various environments. However, we are yet poor of understanding the mechanisms of biological systems including brains despite many efforts of researchers because enormous complexity originating from dynamics in systems is very difficult to be understood and described using the conventional methodologies based on reductionism, that is, decomposing a system into parts or elements. The conventional reductionism more or less falls into two difficulties: one is “combinatorial explosion” and the other is “divergence of algorithmic complexity”. These difficulties are not yet solved. On the other hand, dynamical viewpoint to understand the mechanism seems to be a plausible method. In particular, chaotic dynamics experimentally observed in biological systems including brains[1,2] has suggested a viewpoint that chaotic dynamics would play important roles in complex functioning and controlling of biological systems including brains. From this viewpoint, many dynamical models have been constructed for approaching the mechanisms by means of large-scale simulation or heuristic methods. Artificial neural networks in which chaotic dynamics can be introduced has been attracting great interests, and the relation between chaos and functions has been discussed M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 179–188, 2008. c Springer-Verlag Berlin Heidelberg 2008 

180

Y. Li and S. Nara

[9,10,11,12]. As one of those works, Nara and Davis found that chaotic dynamics can occur in a recurrent neural network model(RNNM) consisting of binary neurons [3], and they investigated the functional aspects of chaos by applying it to solving a memory search task with an ill-posed context[7]. To show the potential of chaos in controlling, chaotic dynamics was applied to solving two-dimensional mazes, which are set as ill-posed problems[8]. Two important points were proposed. One is a simple coding method translating the neural states into motion increments , the other is a simple control algorithm, switching a system parameter adaptively to produce constrained chaos. The conclusions show that constrained chaos behaviours can give better performance to solving a two-dimensional maze than that of random walk. In this paper, we develop the idea and apply chaotic dynamics to tracking a moving target, which is set as another ill-posed problem. Let us state about a model of tracking a moving target. An tracker is assumed to move in two-dimensional space and track a moving target along a certain trajectory by employing chaotic dynamics. the tracker is assumed to move with discrete time steps. The state pattern is transform into the tracker’s motion by the coding of motion functions, which will be given in a later section. In addition, several limit cycle attractors, which are regarded as the prototypical simple motions, are embedded in the network. By the coding of motion function, each cycle corresponds to a monotonic motion in twodimensional space. If the state pattern converges into a prototypical attractor, the tracker moves in a monotonic direction. Introducing chaotic dynamics into the network generated non-period state pattern, which is transformed into chaotic motion of the tracker by motion functions. Adaptive switching of a system parameter by a simple evaluation between chaotic dynamics and attractor’s dynamics in the network results in complex motions of the tracker in various environments. Considering this point, a simple control algorithm is proposed for tracking a moving target. In actual simulation, the present method using chaotic dynamics gives novel performance. To understand the mechanism of better performance, dynamical structure of chaotic dynamics is investigated from statistical data.

2 Memory Attractors and Motion Functions Our study works with a fully interconnected recurrent neural network consisting of N binary neurons. Its updating rule is defined by    Si (t + 1) = sgn Wi j S j (t) (1) j∈Gi (r)

 sgn(u) = • • • •

+1 u ≥ 0; −1 u < 0.

S i (t) = ±1(i = 1 ∼ N): the firing state of a neuron specified by index i at time t. Wi j : connection weight from the neuron S j to the neuron S i (Wii is taken to be 0) r: fan-in number for the neuron S i , named as connectivity,(0 < r < N). Gi (r): a spatial configuration set of connectivity r.

Tracking a Moving Target Using Chaotic Dynamics

181

At a certain time t, the state of neurons in the network can be represented as a N-dimensional state vector S(t), called as state pattern. Time development of state pattern S(t) depends on the connection weight matrix {Wi j } and connectivity r, therefore, in our study, Wi j are determined in the case of full connectivity r = N − 1, by a kind of orthogonalized learning method[7]and taken as follows. Wi j =

L  K  μ=1 λ=1

λ † (ξλ+1 μ )i · (ξ μ ) j

(2)

where {ξ λμ |λ = 1 . . . K, μ = 1 . . . L} is an attractor pattern set, K is the number of memory patterns included in a cycle and L is the number of memory cycles. ξλ† μ is the conjugate λ λ† λ   vector of ξμ which satisfies ξμ · ξμ = δμμ · δλλ ,where δ is Kronecker’s delta. This method was confirmed to be effective to avoid spurious attractors[3,4,5,6,7,8]. Biological data show that neurons in brain causes various motions of muscles in body with a quite large redundancy. Therefore, the network consisting of N neurons is used to realize two-dimensional motion control of an tracker. We confirmed that chaotic dynamics introduced in the network does not so sensitively depend on the size of the neuron number[7].In our actual computer simulation,N = 400. Suppose that an tracker moves from the position (p x (t), py (t)) to (p x (t + 1), py (t + 1)) with a set of motion increments ( f x (t),  fy (t)). The state pattern S(t) at time t is a 400-dimensional vector, and we transform it to two-dimensional motion increments by the coding of motion functions ( f x (S(t)), fy (S(t))). In 2-dimensional space, the actual motion of the tracker is given by           4 A·C p x (t + 1) p (t) f (S(t)) p (t) = x + x = x + (3) py (t + 1) py (t) fy (S(t)) py (t) N B·D where A, B, C, D are four independent N/4 dimensional sub-space vectors of state pattern S(t). Therefore, after the inner product between two independent sub-space vectors is normalized by 4/N, motion functions range from -1 to +1. In our actual simulations, two-dimensional space is digitized with a resolution 0.02 due to the binary neuron state ±1 and N = 400. Now, let us consider the construction of memory attractors corresponding to prototypical simple motions. We take 24 attractor patterns consisting of (L=4 cycles) × (K=6 patterns per cycle). Each cycle corresponds to one prototypical simple motion. We take four types of motion that one tracker moves toward (+1, +1), (-1, +1), (-1, -1), (+1, -1) in two-dimensional space. Each attractor pattern consists of four random subspace vectors A, B, C and D, where C = A or −A, and D = B or −B. So only A and B are independent random patterns. From the law of large number, memory patterns are almost orthogonal each other. Furthermore, in determining {Wi j }, the orthogonalized learning method was employed. Therefore, memory patterns are orthogonalized each other. The corresponding relations between memory attractors and prototypical simple motions are shown as follows. ( f x (ξλ1 ), fy (ξλ1 )) = (+1, +1) ( f x (ξλ3 ), fy (ξλ3 ))

= (−1, −1)

( f x (ξλ2 ), fy (ξλ2 )) = (−1, +1) ( f x (ξλ4 ), fy (ξλ4 )) = (+1, −1)

182

Y. Li and S. Nara

3 Introducing Chaotic Dynamics in RNNM Now let us state the effects of connectivity r. In the case of full connectivity r = N − 1, the network can function as a conventional associative memory. If the state pattern S(t) is one or near one of the memory patterns ξλμ , finally the output sequence S(t + kK)(k = 1, 2, 3 . . .) will converge to the memory pattern ξλμ . In other words, for each memory pattern, there is a set of the state patterns, called as memory basin Bλμ . If S(t) is in the memory basin Bλμ , then the output sequence S(t + kK)(k = 1, 2, 3 . . .) will converge to the memory pattern ξλμ . It is quite difficult to estimate basin volume accurately because of enormous amounts of calculation for the whole state patterns in N-dimensional state space. Therefore, a statistical method is applied to estimating the approximate basin volume. First, a sufficiently large amount of state patterns are sampled in the state space. Second, each sample is taken as initial pattern and updated with full connectivity. Third, it is taken statistic which memory attractor limk→∞ S(kK) of each sample would converge into. The distribution of statistic data over the whole samples is regarded as the approximate basin volume for each memory attractor(see Fig.1). The basin volume shows that almost all initial state patterns converge into one of the memory attractors averagely and there are seldom spurious attractors. 0.06

Basin volume

0.05 0.04 0.03 0.02 0.01 0 0

5

10 15 20 25 Memory pattern number

30

Fig. 1. Basin volume: The horizontal axis represents memory pattern number(1-24). Basin 25 corresponds to samples that converged into cyclic outputs with a period of six steps but not any one memory attractor. Basin 26 corresponds to samples excluded from any other case(1-25). The vertical axis represents the ratios between the corresponding samples and the whole samples.

Next, we continue to decrease connectivity r. When r is large enough, r N, memory attractors are stable. When r becomes smaller and smaller, more and more state patterns gradually do not converge into a certain memory pattern despite the network is updated for a long time, that is, attractors become unstable. Finally, when r becomes quite small, state pattern becomes non-period output, that is, non-period dynamics occurs in the state space. In our previous papers, we confirmed that the non-period dynamics in the network is chaotic wandering. In order to investigate the dynamical structure, we calculated basin visiting measures and it suggests that the trajectory can pass the

Tracking a Moving Target Using Chaotic Dynamics

183

whole N-dimensional state space, that is, cyclic memory attractors ruin due to a quite small connectivity [3,4,5,6,7].

4 Motion Control and Tracking Algorithm When connectivity r is sufficiently large, one random initial pattern converges into one of four limit cycle attractors as time evolves. By the coding transformation of motion functions, the corresponding motion of the tracker in 2-dimensional space becomes monotonic(see Fig.2). On the other hand, when connectivity r is quite small, chaotic dynamics occurs in the state space, correspondingly, the tracker moves chaotically (see Fig.3). If the updating of state pattern in chaotic regime is replaced by random 400-bitpattern generator, the tracker shows random walk(see Fig.4). Obviously, chaotic motion is different from random walk, and has a certain dynamical structure.

6 4 5

4 r=30

r=399

Random walk

2

4

2

0

3 2 1

START

START

-2

-2

-4

-4

-6

0 -1 0

1

2

3

4

5

6

Fig. 2. Monotonic motion

START

-6

-8 -1

0

(500 steps)

-10 -10

-8

-6

-4

-2

-8 0

2

Fig. 3. Chaotic walk

4

(500 steps)

-10 -10

-8

-6

-4

-2

0

2

4

Fig. 4. Random walk

Therefore, when the network evolves, monotonic motion and chaotic motion can be switched by switching the connectivity r. Based on this idea, we proposed a simple algorithm to track a moving target, shown in Fig.5. First, an tracker is assumed to be tracking a target that is moving along a certain trajectory in two-dimensional space, and the tracker can obtain the rough directional information D1 (t) of the moving target, which is called as global target direction. At a certain time t, the present position of the tracker is assumed at the point (p x (t), py (t)). This point is taken as the origin point and two-dimensional space can be divided into four quadrants. If the target is moving in the nth quadrant, D1 (t) = n (n = 1, 2, 3, 4). Next, we also suppose that the tracker can know another directional information D2 (t) = m (m = 1, 2, 3, 4), which is called global motion direction. It means that the tracker has moved toward the mth quadrant from time t − 1 to t, that is, in the previous step. Global target direction D1 (t) and global motion direction D2 (t) are time-dependent variables. If these information are taken as feedback to the network in real time, the connectivity r also becomes a time-dependent variable r(t) and is determined by D1 (t) and D2 (t). In Fig.5, RL is a sufficiently large connectivity and RS is a quite small connectivity that can lead to chaotic dynamics in the neural network. Adaptive switching of connectivity is the core idea of the algorithm. When the synaptic connectivity r(t) is determined by comparing two directions,D1(t−1) and D2 (t−1), the motion increments of the tracker are calculated from the state pattern of the network updated with r(t). The new motion

184

Y. Li and S. Nara

Fig. 5. Control algorithm of tracking a moving target: By judging whether global target direction D1 (t) coincides with global motion direction D2 (t) or not, adaptive switching of connectivity r between RS and RL results in chaotic dynamics or attractor’s dynamics in state space. Correspondingly, the tracker is adaptively tracking a moving target in two-dimensional space.

causes the next D1 (t) and D2 (t), and produces the next connectivity r(t + 1). By repeating this process, the synaptic connectivity r(t) is adaptively switching between RL and RS , the tracker is alternatively implementing monotonic motion and chaotic motion in two-dimensional space.

5 Simulation Results In order to confirm that this control algorithm is useful to tracking a moving target, the moving target should be set. Firstly, we have taken nine kinds of trajectories which the target moves along, which are shown in Fig.6 and include one circular trajectory and eight linear trajectories. Suppose that the initial position of the tracker is the origin(0,0) of two-dimensional space. The distance L between initial position of the tracker and that of the target is a constant value. Therefore, at the beginning of tracking, the tracker is at the circular center of the circular trajectory and the other eight linear trajectories are tangential to the circular trajectory along a certain angle α, where the angle is defined by the x axis. The tangential angle α = nπ/4 (n = 1, 2, . . . , 8), so we number the eight linear trajectories as LTn , and the circular trajectory as LT0 . 20 15 10 5

Object

Target

0 -5 -10

Capture

-15 -20 -20

Fig. 6. Trajectories of moving target: Arrow represents the moving direction of the target. Solid point means the position at time t=0.

-15

-10

-5

0

5

10

15

20

Fig. 7. An example of tracking a target that is moving along a circular trajectory with the simple algorithm. the tracker captured the moving target at the intersection point.

Tracking a Moving Target Using Chaotic Dynamics

185

Next, let us consider the velocity of the target. In computer simulation, the tracker moves one step per discrete time step, at the same time, the target also moves one step with a certain step length S L that represents the velocity of the target. The motion increments of the tracker ranges from -1 to 1, so the step length S L is taken with an interval 0.01 from 0.01 to 1 up to 100 different velocities. Because velocity is a relative quantity, so S L = 0.01 is a slower target velocity and S L = 1 is a faster target velocity relative to the tracker. Now, let us look at a simulation of tracking a moving target using the algorithm proposed above, shown in Fig.7. When an target is moving along a circular trajectory at a certain velocity, the tracker captured the target at a certain point of the circular trajectory, which is a successful capture to a circular trajectory.

6 Performance Evaluation

(a) circular target trajectory

-2

et

10 20 30 40 50 60 0 Connectivity

Ve l

oc

ity

x10 100 80 60 40 20 rg

Success Rate

1 0.8 0.6 0.4 0.2 0 0

Ta

Ve

et

10 20 30 40 50 60 0 Connectivity

lo

ci

ty

x10-2 100 80 60 40 20 rg

1 0.8 0.6 0.4 0.2 0 0

Ta

Success Rate

To show the performance of tracking a moving target, we have evaluated the success rate of tracking a moving target that moves along one of nine trajectories over 100 initial state patterns. In tracking process, the tracker sufficiently approaching the target within a certain tolerance during 600 steps is regarded as a successful trial. The rate of successful trials is called as the success rate. However, even though tracking a same target trajectory, the performance of tracking depends not only on synaptic connectivity r, but also on target velocity or target step length S L. Therefore, when we evaluate the success rate of tracking, a pair of parameters, that is, one of connectivity r(1 ≤ r ≤ 60) and one of target velocity S L(0.01 ≤ T ≤ 1.0), is taken. Because we take 100 different 100 target velocity with a same interval 0.01, we have C60 pairs of parameters. We have evaluated the success rate of tracking a moving target along different trajectories. Two examples are shown as Fig.8(a) and (b). By comparing Fig.8(a) and (b), we are sure that tracking a moving target of circular trajectory has better performance than that of linear trajectory. However, to some linear trajectories, quite excellent performance was observed. On the other hand, the success rate highly depends on connectivity r and the target velocity S L even if the same target trajectory is set. In order to observe the performance clearly, we have taken the data

(b) linear target trajectory

Fig. 8. Success rate of tracking a moving target along (a)a circle trajectory;(b)a linear trajectory: The positive orientation obeys the right-hand rule. The vertical axis represents success rate, and two axes in the horizontal plane represents connectivity r and target velocity S L, respectively.

Y. Li and S. Nara

1

1

0.8

0.8 Success rate

Success rate

186

0.6 0.4 0.2

0.6 0.4 0.2

0

0 0

20

40

60

80

100

0

-2

Target Velocity ( x10 )

(a) r = 16: downward tendency

20

40

60

80

100

Target Velocity ( x10-2)

(b) r = 51: upward tendency

Fig. 9. Success rates drawn from Fig.8(a): We take the data of a certain connectivity and show them in two dimension diagram. The horizontal axis represents target velocity from 0.01 to 1.0, and the vertical axis represents success rate.

of certain connectivities from Fig.8(a), and plot them in two-dimensional coordinates, shown as Fig.9. Comparing these figures, we can see a novel performance, when the target velocity becomes faster, the success rate has a upward tendency, such as r = 51. In other words, when the chaotic dynamics is not too strong, it seems useful to tracking a faster target.

7 Discussion In order to show the relations between the above cases and chaotic dynamics, from dynamical viewpoint, we have investigated dynamical structure of chaotic dynamics. To a quite small connectivity from 1 to 60, the network performs chaotic wandering for long time from a random initial state pattern. During this hysteresis, we have taken a statistics of continuously staying time in a certain basin [8] and evaluated the distribution p(l, μ) which is defined by p(l, μ) = {the number of l | S(t) ∈ βμ in τ ≤ t ≤ τ + l and S(τ − 1)  βμ and S(τ + l + 1)  βμ , μ| μ ∈ [1, L]} K  βμ = Bλμ

(4)

(5)

λ=1

T =



lp(l, μ)

(6)

l

where l is the length of continuously staying time steps in each attractor basin, and p(l, μ) represents a distribution of continuously staying l steps in attractor basin L = μ within T steps. In our actual simulation, T = 105 . To different connectivity r=15 and r=50, the distribution p(l, μ) are shown in Fig.10(a) and Fig.10(b). In these figures, different basins are marked with different symbols. From the results, we can know that continuously staying time l becomes longer and longer with increase of the connectivity r. Referring to those novel performances talked in previous section, let us try to consider the reason.

100000

Frequency distribution of staying

Frequency distribution of staying

Tracking a Moving Target Using Chaotic Dynamics

Basin 1 Basin 2 Basin 3 Basin 4

10000 1000 100 10 1 2

4

6

8

10

12

14

Continuously staying time steps

(a) r = 15: shorter

16

187

100000 Basin 1 Basin 2 Basin 3 Basin 4

10000 1000 100 10 1 10

20

30

40

50

60

Continuously staying time steps

(b) r = 50: longer

Fig. 10. The log plot of the frequency distribution of continuously staying time l: The horizontal axis represents continuously staying time steps l in a certain basin μ during long time chaotic wandering, and the vertical axis represents the accumulative number p(l, μ) of the same staying time steps l in a certain basin μ. continuously staying time steps l becomes long with the increase of connectivity r.

First, in the case of slower target velocity, a decreasing success rate with the increase of connectivity r is observed from both circular target trajectory and linear ones. This point shows that chaotic dynamics localized in a certain basin for too much time is not good to track a slower target. Second, in the case of faster target velocity, it seems useful to track a faster target when chaotic dynamics is not too strong. Computer simulations shows that, when the target moves quickly, the action of the tracker is always chaotic so as to track the target. In past experiments, we know that motion increments of chaotic motion is very short. Therefore, shorter motion increments and faster target velocity result in bad tracking performance. However, when continuously staying time l in a certain basin becomes longer, the tracker can move toward a certain direction for l steps. This would be useful for the tracker to track the faster target. Therefore, when connectivity becomes a little large (r=50 or so), success rate arises following the increase of target velocity, such as the case shown in Fig.9. As an issue for future study, a functional aspect of chaotic dynamics still has context dependence.

8 Summary We proposed a simple method to tracking a moving target using chaotic dynamics in a recurrent neural network model. Although chaotic dynamics could not always solve all complex problems with better performance, better results often were often observed on using chaotic dynamics to solve certain ill-posed problems, such as tracking a moving target and solving mazes [8]. From results of the computer simulation, we can state the following several points. • A simple method to tracking a moving target was proposed • Chaotic dynamics is quite efficient to track a target that is moving along a circular trajectory.

188

Y. Li and S. Nara

• Performance of tracking a moving target of a linear trajectory is not better than that of a circular trajectory, however, to some linear trajectories, excellent performance was observed. • The length of continuously staying time steps becomes long with the increase of synaptic connectivity r that can lead chaotic dynamics in the network. • Continuously longer staying time in a certain basin seems useful to track a faster target.

References 1. Babloyantz, A., Destexhe, A.: Low-dimensional chaos in an instance of epilepsy. Proc. Natl. Acad. Sci. USA. 83, 3513–3517 (1986) 2. Skarda, C.A., Freeman, W.J.: How brains make chaos in order to make sense of the world. Behav. Brain. Sci. 10, 161–195 (1987) 3. Nara, S., Davis, P.: Chaotic wandering and search in a cycle memory neural network. Prog. Theor. Phys. 88, 845–855 (1992) 4. Nara, S., Davis, P., Kawachi, M., Totuji, H.: Memory search using complex dynamics in a recurrent neural network model. Neural Networks 6, 963–973 (1993) 5. Nara, S., Davis, P., Kawachi, M., Totuji, H.: Chaotic memory dynamics in a recurrent neural network with cycle memories embedded by pseudo-inverse method. Int. J. Bifurcation and Chaos Appl. Sci. Eng. 5, 1205–1212 (1995) 6. Nara, S., Davis, P.: Learning feature constraints in a chaotic neural memory. Phys. Rev. E 55, 826–830 (1997) 7. Nara, S.: Can potentially useful dynamics to solve complex problems emerge from constrained chaos and/or chaotic itinerancy? Chaos. 13(3), 1110–1121 (2003) 8. Suemitsu, Y., Nara, S.: A solution for two-dimensional mazes with use of chaotic dynamics in a recurrent neural network model. Neural Comput. 16(9), 1943–1957 (2004) 9. Tsuda, I.: Chaotic itinerancy as a dynamical basis of Hermeneutics in brain and mind. World Futures 32, 167–184 (1991) 10. Tsuda, I.: Toward an interpretation of dynamic neural activity in terms of chaotic dynamical systems. Behav Brain Sci. 24(5), 793–847 (2001) 11. Kaneko, K., Tsuda, I.: Chaotic Itinerancy. Chaos 13(3), 926–936 (2003) 12. Aihara, K., Takabe, T., Toyoda, M.: Chaotic Neural Networks. Phys. Lett. A 114, 333–340 (1990)

A Generalised Entropy Based Associative Model Masahiro Nakagawa Nagaoka University of Technology, Kamitomioka 1603-1, Nagaoka, Niigata 940-2188, Japan [email protected]

Abstract. In this paper, a generalised entropy based associative memory model will be proposed and applied to memory retrievals with analogue embedded vectors instead of the binary ones in order to compare with the conventional autoassociative model with a quadratic Lyapunov functionals. In the present approach, the updating dynamics will be constructed on the basis of the entropy minimization strategy which may be reduced asymptotically to the autocorrelation dynamics as a special case. From numerical results, it will be found that the presently proposed novel approach realizes the larger memory capacity even for the analogue memory retrievals in comparison with the autocorrelation model based on dynamics such as associatron according to the higher-order correlation involved in the proposed dynamics. Keywords: Entropy, Associative Memory, Analogue Memory Retrieval.

1 Introduction During the past quarter century, the numerous autoassociative models have been extensively investigated on the basis of the autocorrelation dynamics. Since the proposals of the retrieval models by Anderson, [1] Kohonen, [2] and Nakano, [3] some works related to such an autoassociation model of the inter-connected neurons through an autocorrelation matrix were theoretically analyzed by Amari, [4] Amit et al . [5] and Gardner [6] . So far it has been well appreciated that the storage capacity of the autocorrelation model , or the number of stored pattern vectors, L , to be completely associated vs the number of neurons N, which is called the relative storage capacity or loading rate and denoted as
c = L / N , is estimated as
c ~0.14 at most for the autocorrelation learning model with the activation function as the signum one ( sgn (x) for the abbreviation) [7,8] . In contrast to the abovementioned models with monotonous activation functions, the neuro-dynamics with a nonmonotonous mapping was recently proposed by Morita, [9] Yanai and Amari, [10] Shiino and Fukai [11]. They reported that the nonmonotonous mapping in a neurodynamics possesses a remarkable advantage in the storage capacity,
c ~0.27, superior than the conventional association models with monotonous mappings, e.g. the signum or sigmoidal function. In the present paper, we shall propose a novel approach based on the entropy defined in terms of the overlaps, which are defined by the innerproducts between the M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 189–198, 2008. © Springer-Verlag Berlin Heidelberg 2008

190

M. Nakagawa

state vector and the analogue embedded vectors instead of the previously investigated binary ones [1-16,25].

2 Theory Let us consider an associative model with the embedded analogue vector (r ) ei (1
i
N,1
r
L) , where N and L are the number of neurons and the number of embedded vectors. The states of the neural network are characterized in terms of the output vector si (1
i
N ) and the internal states
i (1
i
N ) which are related each other in terms of

s i =f


(1
i
N),

i

(1)

where f ( • ) is the activation function of the neuron. Then we introduce the following entropy I which is to be related to the overlaps as L

I=- 1 2 (r)

where the overlaps m

(r) 2

m

log m

(r) 2

,

(2)

r=1

(r=1,2,...,L) are defined by N (r)

m

†(r)

= e

(3)

i s i;

i=1

here the covariant vector e relation,

†(r )

is defined in terms of

i

the following orthogonal

N



e

†(r)

ie

(s)

a

(r') rr' e i ,

(4a)

,

(4b)

i =
rs

(1
r,s
L) ,

(4)

i=1

e

L †(r) i = r'=1

a

rr' =(




-1

)

rr'

e

(r) (r') i e i

N




rr' =



.

(4c)

i=1

The entropy defined by eq.(2) can be minimized by the following condition

m and

(r)

=


rs

(1
r,s
L),

(5)

A Generalised Entropy Based Associative Model L



m

(r) 2

191

(6)

=1

r=1

That is, regarding m (1
r
L) as the probability distribution in eq.(2), a target (r) pattern may be retrieved by minimizing the entropy I with respect to m or the state vector si to achieve the retrieval of a target pattern in which the eqs.(5a) and (5b) are to be satisfied. Therefore the entropy function may be considered to be a functional to be minimized during the retrieval process of the auto-association model instead of the conventional quadratic energy functional, E, i.e. (r) 2

E=- 1 2

N N

  w ij s †

is j

,

(6a)

js j ,

(6b)

i=1 j=1

†

where s i is the covariant vector defined by L N

s

†

i=



e

†(r)

ie

†(r)

r=1 j=1

and the connection matrix w ij is defined in terms of L (r)

w ij =  e

ie

†(r)

j.

(6c)

r=1

According to the steepest descent approach in the discrete time model, the updating rule of the internal states
i (1
i
N ) may be defined by

 i (t+1) =-


I s

(1
i
N) ,

†

(7)

i

where
(> 0 ) is a coefficient. Substituting eqs.(2) and (3) into eq.(7) and noting the following relation with aid of eq.(6b), N

m

(r)

N

= e

†(r)

(r)

i s i = e

i=1

is

† i

,

(8)

i=1

one may readily derive the following relation.

 i (t+1)=-
I† =+ 1
† 2 s i s i = 1
† 2 s i L

m

(r) 2

2

log

=
 e =
 e

(r)

i

r=1 L

 

N

e

(r)

†

js

j

t

r=1 j=1



(r)

(r) 2

r=1



e

(r)

js

js

†

†

j

2

t

j=1 N

e

(r)

js

†

j

t

1+log

j=1 im

log m

N

N (r)

r=1

L

L



e

(r)

j

t

2

j=1

1+log m

(r) 2

.

(9)

192

M. Nakagawa

Generalizing somewhat the above dynamics in order to combine the quadratic approach (
 0) and the present entropy one (
 1) , we propose the following dynamic rule, in a somewhat ad-hoc manner, for the internal states L

N

 i (t+1)=  e

(r)

i

r=1

N



†(r)

e

js j

t

1+log 1-
+


j=1



e

†(r)

js j

t

2

j=1

L

=  e

(r)

i

m

(r)

t

1+log 1-
+
m

(r)

2

t

. (10)

r=1

In practice, in the limit of
 0 , the above dynamics will be reduced to the autocorrelation dynamics. L

 e (r) i

 i (t+1)=lim L

=  e

m


0 r=1

(r)

(r)

im

r=1

(r)

t

L

1+log 1-
+
m N

t =-  e

(r)

r=1

i



(r)

2

t

N

e

† (r)

j=1

j s j (t) =  w ij s j (t) . j=1

(11)

On the other hand, eq.(10) results in eq.(9) in the case of
 1 . Therefore one may control the dynamics between the autocorrelation (
 0) and the entropy based approach (
 1 ).

3 Numerical Results The embedded vectors are set to the binary random vectors as follows.

e (r) i =z

(r) i

(1
i
N,1
r
L)

(12)

where z i (1
i
N ,1
r
L) are the zero-mean pseudo-random numbers between -1 and +1. For simplicity, the activation function , eq.(1), is assumed to be a piecewise linear function instead of the previous signum form for the binary embedded vectors[25] and set to (r )

s i =f (
i )=


1+sgn 1-
i

2

i

+sgn


1-sgn 1-
i

where denotes the signum function sgn ( •) defined by

-1 (x<0) sgn x = 0 (x=0) . +1 (x>0)

2

i

,

(13)

(14)

A Generalised Entropy Based Associative Model

193

The initial vector si (0) (1
i
N ) is set to

s i (0)=

where e

(r ) i

-e (s) i (1
i
H d ) , +e (s) i (H d +1
i
N)

(15)

is a target pattern to be retrieved and H d is the Hamming distance

between the initial vector si (0) and a target vector e

(s) i

. The retrieval is succeeded if

N

m

(s)

(t ) =  e

†(s) i s

i (t

)

(16)

i=1

results in ±1 for t
1, in which the system may be in a steady state such that

s i (t+1)=s i (t) ,

(17a)


i (t+1)=
i (t) .

(17b)

To see the retrieval ability of the present model, the success rate Sr is defined as the rate of the success for 1000 trials with the different embedded vector sets e (r )i (1
i
N ,1
r
L) . To control from the autocorrelation dynamics after the initial state (t~1) to the entropy based dynamics (t~ Tmax ) , the parameter
in eq.(10) was simply controlled by


= t
T max

max

(0
t
T

max ) ,

(18)

where Tmax and
max are the maximum values of the iterations of the updating according to eq.(10) and
, respectively. Choosing N =200, η = 1, Tmax = 25, L/N=0.5 and α max = 1, we first present an example of the dynamics of the overlaps in Figs.1(a) and (b) (Entropy based approach). Therein the cross symbols( × ) and the open circles(o) represent the success of retrievals, in which eqs.(5a) and (5b) are satisfied, and the entropy defined by eq.(2), respectively, for a retrieval process. In addition the time dependence of the parameter α / α max defined by eq.(18) are depicted as dots ( i ). In Fig. 1 after a transient state, it is confirmed that the complete association corresponding to eqs.(5a) and (5b) can be achieved. Then we shall present the dependence of the success rate Sr on the loading rate
= L / N are depicted in Figs.2 (a) and (b) for H d / N = 0.3 , N =100 for the entropy approach and the associatron, respectively. From these results, one may confirm the larger memory capacity of the presently proposed model defined by eq.(10) in

194

M. Nakagawa <EAM2A> N=100 Ns=100 Tmax=50 k=0 Hd=10 idia=1 iana=1 ictl=1 iotg=1 ient=1 izero=0 alpmax=1 1

0.8

0.6

Overlaps < o(n) >

0.4

0.2 0

-0.2 -0.4 -0.6 -0.8

-1 0

5

10

15

(a)

25 n

20

30

35

40

45

50

H d / N = 0.1

<EAM2A> N=100 Ns=100 Tmax=50 k=0 Hd=30 idia=1 iana=1 ictl=1 iotg=1 ient=1 izero=0 alpmax=1

1

0.8 0.6

Overlaps < o(n) >

0.4 0.2 0 -0.2 -0.4 -0.6 -0.8 -1 0

5

10

15

(b)

20

25 n

30

35

40

45

50

H d / N = 0.3 (r)

Fig. 1. The time dependence of overlaps m eq.(10)

of the present entropy based model defined by

A Generalised Entropy Based Associative Model

195

<EAM2A> N=100 Ns=100 Tmax=50 k=0 Hd=30 idia=1 iana=1 ictl=1 iotg=1 ient=1 izero=0 alpmax=1

1 0.9

Success Rate Sr(L/N)

0.8 0.7

Success Rate MemCap= 0.9999 Hd/N= 0.3

0.6 0.5 0.4 0.3 0.2 0.1 0

0

0.1

0.2

0.3

0.4

0.5 L/N

0.6

0.7

0.8

0.9

1

(a) Entropy based Model defined by eq.(10)

<EAM2A> N=100 Ns=100 Tmax=50 k=0 Hd=30 idia=1 iana=1 ictl=1 iotg=1 ient=0 izero=0 alpmax=1 1

Success Rate MemCap= 0.0134 Hd/N= 0.3

0.9

Success Rate Sr(L/N)

0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

0

0.1

0.2

0.3

0.4

0.5 L/N

0.6

0.7

0.8

0.9

1

(b) Conventional Associatron Model defined by eq.(11) Fig. 2. The dependence of the success rate on the loading rate α = L / N of the present entropy based model defined by eqs.(10) and (11). Here the Hamming distance is set to H d / N = 0.3.

196

M. Nakagawa

comparison with the conventional autoassociation model defined by eq.(11). In practice, it is found that the present approach may achieve the high memory capacity beyond the conventional autocorrelation strategy even for the analogue embedded vectors as well as the previously concerned binary case [15,16,25].

4 Concluding Remarks In the present paper, we have proposed an entropy based association model instead of the conventional autocorrelation dynamics. From numerical results, it was found that the large memory capacity may be achieved on the basis of the entropy approach. This advantage of the association property of the present model is considered to result from the fact such that the present dynamics to update the internal state eq.(10) assures that the entropy, eq.(2) is minimized under the conditions, eqs.(5a) and (5b), which corresponding to the succeeded retrieval of a target pattern. In other words, the higher-order correlations in the presently proposed dynamics, eq.(10), which was ignored in the conventional approaches, [1-11] was found to play an important role to improve memory capacity, or the retrieval ability. To conclude this work, we shall show the dependence of the storage capacity, which is defined as the area covered in terms of the success rate curves as shown in Fig.3 , on the Hamming distance in Fig.3 for the analogue embedded vectors (Ana) as well as the previous binary ones (Bin). In addition OL and CL imply the orthogonal learning model and the autocorrelation learning model, respectively. Therein one may see again the great advantage of the present model based on the entropy functional to be minimized beyond the conventional quadratic form [12,13] even for the analogue embedded vectors. In fact one may realize the considerably larger storage capacity in the present model in comparison with the associatron over H d / N
0.5 . The memory retrievals for the associatron based on the quadratic 1

n a m

n a m

n a m

a n m

a n m

0.9 Memory Capacity

0.8

a m n

a m n

t

a m n

0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

t s s

t s

t s

t s

t s

t s

a m

t s

n

a m n

a

Entropy based Model (OL:Ana)

m

Entropy based Model (OL:Bin)

n

Entropy based Model (CL:Bin)

s

Associatron(OL:Bin)

t

Associatron(OL:Wii=0:Bin) t s

t s

0.010.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 Hd/N

Fig. 3. The dependence of the storage capacity on the Hamming distance. Here symbols a, m and n are for the entropy based approach with eq. (10) as well as the orthogonal learning (OL) and the autocorrelation learning (CL) [16,17], in which Ana and Bin imply the analogue embedded vectors and the binary ones, respectively. In addition we presented the associatron in symbols s with the orthogonal learning [13], and the associatron in symbols t with orthogonal learning under the condition wii = 0 [12], respectively.

A Generalised Entropy Based Associative Model

197

Lyapunov functionals to be minim ized become troublesome near H d / N = 0 .5 as seen in Fig.3 since the directional cosine between the initial vector and a target pattern eventually vanishes therein. Remarkably, even in such a case, the present model attains a remarkably large memory capacity because of the higher-order correlations involved in eq.(10) as expected from Figs. 1 and 2 for the analogue vectors as well as the binary ones previously investigated [15,16,25]. As a future problem, it seems to be worthwhile to involve a chaotic dynamics in the present model introducing a periodic activation function such as sinusoidal one as a nonmonotonic activation function [14]. The entropy based approach [15] with chaos dynamics [14] is now in progress and will be reported elsewhere together with the synergetic models [17-24] in the near future.

References 1. Anderson, J.A.: A Simple Neural Network Generating Interactive Memory. Mathematical Biosciences 14, 197–220 (1972) 2. Kohonen, T.: Correlation Matrix Memories. IEEE Transaction on Computers C-21, 353– 359 (1972) 3. Nakano, K.: Associatron-a Model of Associative Memory. IEEE Trans. SMC-2, 381–388 (1972) 4. Amari, S.: Neural Theory of Association and Concept Formation. Biological Cybernetics 26, 175–185 (1977) 5. Amit, D.J., Gutfreund, H., Sompolinsky, H.: Storing Infinite Numbers of Patternsin a Spinglass Model of Neural Networks. Physical Review Letters 55, 1530–1533 (1985) 6. Gardner, E.: Structure of Metastable States in the Hopfield Model. Journal of Physics A19, L1047–L1052 (1986) 7. Kohonen, T., Ruohonen, M.M.: Representation of Associated Pairs by Matrix Operators. IEEE Transaction C-22, 701–702 (1973) 8. Amari, S., Maginu, K.: Statistical Neurodynamics of Associative Memory. Neural Networks 1, 63–73 (1988) 9. Morita, M.: Neural Networks. Associative Memory with Nonmonotone Dynamics 6, 115– 126 (1993) 10. Yanai, H.-F., Amari, S.: Auto-associative Memory with Two-stage Dynamics of nonmonotonic neurons. IEEE Transactions on Neural Networks 7, 803–815 (1996) 11. Shiino, M., Fukai, T.: Self-consistent Signal-to-noise Analysis of the Statistical Behaviour of Analogu Neural Networks and Enhancement of the Storage Capacity. Phys. Rev. E48, 867 (1993) 12. Kanter, I., Sompolinski, H.: Associative Recall of Memory without Errors. Phys. Rev. A 35, 380–392 (1987) 13. Personnaz, L., Guyon, I., Dreyfus, D.: Information Storage and Retrieval in Spin-Glass like Neural Networks. J. Phys(Paris) Lett. 46, L-359 (1985) 14. Nakagawa, M.: Chaos and Fractals in Engineering, p. 944. World Scientific Inc., Singapore (1999) 15. Nakagawa, M.: Autoassociation Model based on Entropy Functionals. In: Proc. of NOLTA 2006, pp. 627–630 (2006) 16. Nakagawa, M.: Entropy based Associative Model. IEICE Trans. Fundamentals EA-89(4), 895–901 (2006)

198

M. Nakagawa

17. Fuchs, A., Haken, H.: Pattern Recognition and Associative Memory as Dynamical Processes in a Synergetic System I. Biological Cybernetics 60, 17–22 (1988) 18. Fuchs, A., Haken, H.: Pattern Recognition and Associative Memory as Dynamical Processes in a Synergetic System II. Biological Cybernetics 60, 107–109 (1988) 19. Fuchs, A., Haken, H.: Dynamic Patterns in Complex Systems. In: Kelso, J.A.S., Mandell, A.J., Shlesinger, M.F. (eds.), World Scientific, Singapore (1988) 20. Haken, H.: Synergetic Computers and Cognition. Springer, Heidelberg (1991) 21. Nakagawa, M.: A study of Association Model based on Synergetics. In: Proceedings of International Joint Conference on Neural Networks 1993 NAGOYA, JAPAN, pp. 2367– 2370 (1993) 22. Nakagawa, M.: A Synergetic Neural Network. IEICE Fundamentals E78-A, 412–423 (1995) 23. Nakagawa, M.: A Synergetic Neural Network with Crosscorrelation Dynamics. IEICE Fundamentals E80-A, 881–893 (1997) 24. Nakagawa, M.: A Circularly Connected Synergetic Neural Networks. IEICE Fundamentals E83-A, 881–893 (2000) 25. Nakagawa, M.: Entropy based Associative Model. In: Proceedings of ICONIP 2006, pp. 397–406. Springer, Heidelberg (2006)

The Detection of an Approaching Sound Source Using Pulsed Neural Network Kaname Iwasa1, Takeshi Fujisumi1 , Mauricio Kugler1 , Susumu Kuroyanagi1, Akira Iwata1 , Mikio Danno2 , and Masahiro Miyaji3 1

Nagoya Institute of Technology, Gokiso-cho, Showa-ku, Nagoya, 466-8555, Japan [email protected] 2 Toyota InfoTechnology Center, Co., Ltd, 6-6-20 Akasaka, Minato-ku, Tokyo, 107-0052, Japan 3 Toyota Motor Corporation, 1 Toyota-cho, Toyota, Aichi, 471-8572, Japan

Abstract. Current automobiles’ safety systems based on video cameras and movement sensors fail when objects are out of the line of sight. This paper proposes a system based on pulsed neural networks able to detect if a sound source is approaching a microphone or moving away from it. The system, based on PN models, compares the sound level difference between consecutive instants of time in order to determine its relative movement. Moreover, the combined level difference information of all frequency channels permits to identify the type of the sound source. Experimental results show that, for three different vehicles sounds, the relative movement and the sound source type could be successfully identified.

1

Introduction

Driving safety is one of the major concerns of the automotive industry nowadays. Video cameras and movement sensors are used in order to improve the driver’s perception of the environment surrounding the automobile [1][2]. These methods present good performance when detecting objects (e.g., cars, bicycles, and people) which are in line of sight of the sensor, but fail in case of obstruction or dead angles. Moreover, the use of multiple cameras or sensors for handling dead angles increases the size and cost of the safety system. The human being, in contrast, is able to perceive people and vehicles around itself by the information provided by the auditory system [3]. If this ability could be reproduced by artificial devices, complementary safety systems for automobiles would emerge. Cause of diffraction, sound waves can contour objects and be detected even when the source is not in direct line of sight. A possible approach for processing temporal data is the use of Pulsed Neuron (PN) models [4]. This type of neuron deals with input signals on the form of pulse trains, using an internal membrane potential as a reference for generating pulses on its output. PN models can directly deal with temporal data and can be efficiently implemented in hardware, due to its simple structure. Furthermore, M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 199–208, 2008. c Springer-Verlag Berlin Heidelberg 2008 

200

K. Iwasa et al.

high processing speeds can be achieved, as PN model based methods are usually highly parallelizable. A sound localization system based on pulsed neural networks has already being proposed in [5] and a sound source identification system, with a corresponding implementation on FPGA, was introduced in [6]. This paper focuses specifically on the relative moving direction of a sound emitting object, and proposes a method to detect if a sound source is approaching or moving away from it using a microphone. The system, based on PN models, compares the sound level difference between consecutive instants of time in order to determine its relative movement. Moreover, the proposed method also identifies the type of the sound source by the use of PN model based competitive learning pulsed neural network for processing the spectral information.

2

Pulsed Neuron Model

When processing time series data (e.g., sound), it is important to consider the time relation and to have computationally inexpensive calculation procedures to enable real-time processing. For these reasons, a PN model is used in this research.

Input Pulses

A Local Membrane Potential p1(t)

IN 1(t) The Inner Potential of the Neuron IN 2 (t)

w1 w2 p2(t) wk pk(t)

IN k (t)

I(t)

θ

Output Pulses o(t)

w n pn(t)

IN n (t)

Fig. 1. Pulsed neuron model

Figure 1 shows the structure of the PN model. When an input pulse INk (t) reaches the k th synapse, the local membrane potential pk (t) is increased by the value of the weight wk . The local membrane potentials decay exponentially with a time constant τk across time. The neuron’s output o(t) is given by o(t) = H(I(t) − θ) n  I(t) = pk (t)

(1) (2)

k=1 t

pk (t) = wk INk (t) + pk (t − 1)e− τ

(3)

The Detection of an Approaching Sound Source

201

where n is the total number of inputs, I(t) is the inner potential, θ is the threshold and H(·) is the unit step function. The PN model also has a refractory period tndti , during which the neuron is unable to fire, independently of the membrane potential.

3

The Proposed System

The basic structure of the proposed system is shown in Fig.2. This system consists of three main blocks, the frequency-pulse converter, the level difference extractor and the sound source classifier, from which the last two are based on PN models. The relative movement (approaching or moving away) of the sound source is determined by the sound level variation. The system compares a signal level x(t) from a microphone with the level in a previous time x(t−Δt). If x(t) > x(t−Δt), the sound source is getting closer to a microphone, if x(t) < x(t−Δt), it is moving away. After the level difference having been extracted, the outputs of the level difference extractors contain the spectral pattern of the input sound, which is then used for recognizing the type of the source. 3.1

Filtering and Frequency-Pulse Converter

Initially, the input signal must be pre-processed and converted to a train of pulses. A bank of 4th order band-pass filters decomposes the signal in 13 frequency channels equally spaced in a logarithm scale from 500 Hz to 2 kHz. Each frequency channel is modified by the non-linear function shown in Eq.(4), and the resulting signal’s envelope is extracted by a 400 Hz low-pass filter. Finally, Input Signal Filter Bank & Frequency - Pulse Converter f1

f2

fN

Time Delay

x(t)

Time Delay

x(t- D t)

Level Difference Extractor

x(t)

x(t- D t)

Level Difference Extractor

Time Delay

x(t)

x(t- D t)

Level Difference Extractor

Sound Source Classifier Approaching Detection & Sound Classification

Fig. 2. The structure of the recognition system

202

K. Iwasa et al.

each output signal is independently converted to a pulse train, whose rate is proportional to the amplitude of the signal.  1 x(t) 3 x(t) ≥ 0 F (t) = 1 (4) 1 3 x(t) x(t) < 0 4 3.2

Level Difference Extractor

Each pulse trains generated by the Frequency-Pulse converter is inputted in a Level Difference Extractor (LDE) independently. The LDE, shown in Fig. 3, is composed by two parts, the Lateral Superior Olive (LSO) model and the Level Mapping Two (LM2) model [7]. In LSO model and LM2 model, each neurons work as Eq.(3). The LSO is responsible for the time difference extraction itself, while the LM2 extracts the envelope of the complex firing pattern. Each pulse train correspondent to each frequency channel is inputted in a LSO LSO model. The PN potential of f th channel, ith LSO neuron Ii,f (t) is calculated as follows: LSO B Ii,f (t) = pN i,f (t) + pi,f (t)

pN i,f (t)

=

N wi,f xf (t)

+

pN i,f (t

(5) − 1)e

t LSO

−τ

B B pB i,f (t) = wi,f xf (t − Δt) + pi,f (t − 1)e

−τ t LSO

(6) (7)

N where τLSO is the time constant of the LSO neuron and the weights wi,f and B wi,f are defined as: ⎧ ⎧ 0.0 i=0 0.0 i=0 ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨1.0 ⎨ i>0 1.0 i<0 N B wi,f = wi,f = (8) i i γ γ ⎪ ⎪ −10 −b < i < 0 −10 0
where α, γ are parameters for adjustment of the weights K is the index of the last neuron of each side of the LSO (totalizing 2K + 1 neurons, including the central neuron) and b is the index of the last inner neuron of each side of the LSO. The inner neurons have current input weights smaller than delayed input weights. They are used to make a feature of the input level difference clear when the input level difference is small. As larger the signal becomes, more neurons fire on the LSO model. The LM2 stage then generates a clearer output, extracting the envelope of the firing pattern generated by the LSO. The potentials in the LM2 are calculated as follows: LM2 S Il,f (t) = pD l,f (t) + pl,f (t)

(9)

D pD l,f (t) = mi,f (t) + pl,f (t − 1)e

pSl,f (t)

= −mi,f (t) +

pSl,f (t

t LM 2

−τ

− 1)e

t LM 2

−τ

(10) (11)

where τLM2 is the time constant of the LM2 neuron and mi,f (t) is the output of the ith LSO neuron in f th frequency channel.

The Detection of an Approaching Sound Source

203

Fig. 3. Level difference extractor

3.3

Sound Source Classifier

The sound source classifier is based on the Competitive Learning Network using Pulsed Neurons (CONP) proposed in [5]. The basic structure of CONP is shown in Fig.4. The CONP is constructed on PN models. In the learning process of CONP, the neuron with the most similar weights to the input (winner neuron) should be chosen for learning in order to obtain a topological relation between inputs and outputs. However, in the case of two or more neurons firing, it is difficult to decide which one is the winner, as their outputs are only pulses, and not real values. In order to this, CONP has extra external units called control neurons. Based on the output of the Competitive Learning (CL) neurons, the control neurons’ outputs increase or decrease the inner potential of all CL neurons, keeping the number of firing neurons equal to one. Controlling the inner potential is equivalent to controlling the threshold. Two types of control neurons are used in this work. The No-Firing Detection (NFD) neuron fires when no CL neuron fires, increasing their inner potential. Complementarily, the Multi-Firing Detection (MFD) neuron fires when two or more CL neurons fire at the same time, decreasing their inner potential [5]. The CL neurons are also controlled by another potential, named the input potential pin (t), and a gate threshold θgate . The input potential is calculated as the sum of the inputs (with unitary weights), representing the rate of the input pulse train. When pin (t) < θgate , the CL neurons are not updated by the control neurons and become unable to fire, as the input train has a too small potential for being responsible for an output firing. Furthermore, the input potential of each CL neuron is decreased along time by a factor β, to follow rapid changes on the inner potential and improving its adjustment.

204

K. Iwasa et al.

Fig. 4. Competitive Learning Network using Pulsed Neurons (CONP)

Considering all the described adjustments on the inner potential of CONP neurons, the output equation (3) of each CL neurons becomes:  n  o(t) = H pk (t) − θ k=1



+ pnf d (t) − pmf d (t) − β · pin (t)

(12)

where pnf d (t) and pmf d (t) corresponds respectively to the potential generated by NFD and MFD neurons’ outputs, pin (t) is the input potential and β (0 ≤ β ≤ 1) is a parameter.

4

Experimental Results

Three different sound sources were used on the experiments: “police car”, “ambulance” and “scooter”. The first two correspond to the alarm sounds of the vehicles, while the last corresponds to the engine sound of a scooter. All the signals were recorded from a static sound source. The moving sound source signals were generated by computer, with the sound intensity at each instant of time calculated as: S(t) = 20Sb log10

d(t) db

(13)

where Ib is a sound intensity in the center position, db and d(t) are, respectively, the distance between the sensor and the sound source at center position and the distance at time t. All signal have 4.0 s of duration and the sound source is normal to the sensor at 2.0 s, as shown in Fig. 5.

The Detection of an Approaching Sound Source

205

Microphone

d(t)

d b = 1m Sound Source 0.0s

2.0s

S(t)

4.0s

Sb Fig. 5. Sound source movement on experiments Table 1. Parameters of each module used on the experiments Input Sound Sampling frequency 48 kHz Quantization bits 16 bits Number of frequency channels 13 channels Delay time Δt 0.4 s Level Difference Extractor Number of total LSO neurons 2K + 1 51 units Number of inner LSO neurons 2b + 1 11 units Number of output neurons L 48 units Threshold θLSO /θLM 2 0.001 / 0.001 Time constant τLSO /τLM 2 0.1 s / 35.0 μs Parameter α/β 60 / 60

4.1

Level Difference Information Extraction

The level difference information was extracted as described in section 3.2. The used parameters for the signal acquisition, preprocessing and level difference extraction are shown in Table 1. Figure 6 shows the output of the LDE model for the “police car” signal in four distinct intervals of time. The x-axis corresponds to the index of the neurons in the LM2, representing the level difference information, and the y-axis corresponds to the frequency channels. The gray level intensity represents the rate of the output pulse train. The firing pattern differs significantly from each interval of time, especially when comparing the graphics of opposite relative movements. Although the LM2 could not successfully extract the envelope from the firing pattern of the signals corresponding to a sound source moving away from the sensor, the result is enough clear for distinguishing it from an approaching sound source signal. Figure 7 shows the firing patterns of each kind of sound for the approaching (interval of 0.0 ∼ 2.0 s) and moving away (2.0 ∼ 4.0 s) cases. As different frequency components present different firing information, it is possible to classify the sound source, as described in the next section.

206

K. Iwasa et al.

Fig. 6. Level Difference Extractor output of the “police car” dataset

Fig. 7. Comparing the output of level difference information for each dataset

The Detection of an Approaching Sound Source

207

Table 2. Parameters of CONP used on the experiments Competitive learning Neuron Number of Inputs of CL neurons 637 units Number of CL neurons 30 units Threshold θ 1.0×10−4 Gating threshold θgate 150.0 Rate for input pulse frequency β 0.0629 Time constant τp 0.1 s Refractory period tndti 10 ms Learning coefficient α 2.0×10−8 Learning iterations 1000 Control Neurons(NFD/MFD) Time constant τNF D /τM F D 0.5 ms / 1.0 ms Threshold θNF D /θM F D -1.0×10−3 / 2.0 Connection weight to each CL neurons 16.0 / -16.0

Table 3. Results of sound recognition (A = approaching, M = moving away)

Input Sound police A M ambulance A M scooter A M

4.2

police car A M 70.6 6.8 6.8 88.3 1.1 4.2 3.8 0.2 0.0 0.0 0.0 1.9

Recognition Rate[%] ambulance scooter A M A M 2.4 7.3 12.9 0.0 0.0 4.9 0.0 0.0 82.8 9.9 2.0 0.0 7.3 86.3 0.0 2.4 5.7 0.0 94.3 0.0 0.3 5.4 0.0 92.4

Sound Source Classification

The firing information patterns provided by all the level difference extractors are recognized by the CONP model described in section 3.3. The CONP model was trained according to the parameters shown in Table 2. Table 3 shows the accuracy of the CONP model for each dataset. The recognition rate is defined as the ratio between the number of neuron’s firing corresponding to the correct vehicle and relative movement and the total number of firings. The correct sound source and relative direction could be recognized with an average accuracy of 85.8%. The results of the “scooter” dataset present a better recognition rate than the “police car” and “ambulance” datasets. The reason for this is that the sound signal of the “scooter” dataset is constant over time, in opposite to the alarm sounds of the other two vehicles, which actually correspond to two different and alternated sounds. Thus, the CONP model can be more efficiently trained with

208

K. Iwasa et al.

the “scooter” data than the others, which would require more data in order to obtain a comparable accuracy.

5

Conclusions

This paper proposes a system for detecting the approaching and classifying a sound source using pulsed neural networks. The system extracts the level difference information from pulse trains corresponding to several frequency bands. The firing pattern is then classified by a CONP model, which identifies the type and recognizes the relative movement of the sound source. The experimental results confirmed that the PN model based level difference extractor can successfully detect the relative movement (approaching or moving away) of a sound source. By using the firing pattern provided by the LDE, the sound source type and relative movement could be correctly classified with a average accuracy of 85.8%. Future works include the detection of a sound source position (its distance from the sensor) and the combination of the proposed system with a sound localization method. The hardware implementation of the proposed systems using an FPGA device is also in progress.

Acknowledgment This research is supported in part by a grant from the Hori Information Science Promotion Foundation.

References 1. Surendra, G., Osama, M., Robert, F.K.M., Nikolaos, P.P.: Detection and Classification of Vehicles. IEEE Trans. ITS 3(1), 37–47 (2002) 2. Chieh-Chi, W., Thorpe, C., Thrun, S.: Online simultaneous localization and mapping with detection and tracking of moving objects: theory and results from a ground vehicle in crowded urban areas. In: Proceedings of ICRA 2003, pp. 842–849 (2003) 3. Pickles, J.O.: An Introduction to the Physiology of Hearing. Academic Press, London (1988) 4. Maass, W., Bishop, C.M.: Pulsed Neural Networks. MIT Press, Cambridge (1998) 5. Kuroyanagi, S., Iwata, A.: A Competitive Learning Pulsed Neural Network for Temporal Signals. In: Proceedings of ICONIP 2002, pp. 348–352 (2002) 6. Iwasa, K., Kuroyanagi, S., Iwata, A.: A Sound Localization and Recognition System using Pulsed Neural Networks on FPGA. In: Proceeding of International Joint Conference of Neural Networks 2007 (to appear, August 2007) 7. Kuroyanagi, S., Iwata, A.: Auditory Pulse Neural Network Model to Extract the Inter-Aural Time and Level Difference for Sound Localization. IEICE Trans. Information and Systems E77-D(4), 466–474 (1994) 8. Kuroyanagi, S., Iwata, A.: Auditory Pulse Neural Network Model for Sound Localization -Mapping of the ITD and ILD-. IEICE J78-D2(2), 267–276 (1996) (in Japanese)

Sensitivity and Uniformity in Detecting Motion Artifacts Wen-Chuang Chou1, Michelle Liou1, and Hong-Ren Su1,2 1

Institute of Statistical Science, Academia Sinica, 128, Academia Rd. Sec.2, Taipei 115, Taiwan 2 Department of Computer Science, National Tsing Hwa University, 101, Sec.2, Kuang-Fu Rd., Hsinchu, 300 Taiwan {wcchou,mliou,stevensu}@stat.sinica.edu.tw

Abstract. Removing artifacts due to head motion is a preprocessing procedure necessary for any fMRI analysis. In fMRI tool boxes, there have been standard algorithms for correcting motion artifacts. However, those tool boxes fail to indicate the extent to which the correction has been successfully done. Without knowing motion contamination especially after correction, the subsequent analysis using averaged fMRI data across subjects could be misleading. In this study, we proposed seven summary indices for measuring motion artifacts. The indices can be applied after motion correction by the image registration algorithms. In the simulation studies, we analyzed a real fMRI data set using a statistical method and estimated the brain activation maps. The real image data were then randomly shifted or rotated to simulate different degrees of head motion. The data contaminated by random motion were then corrected using the SPM image coregistration algorithms. The indices of motion contamination were computed using the corrected images. The corrected images were then analyzed again using the same statistical method. The consistency between the brain activation maps based on real data and those based on simulated data was used as a standard to evaluate the usefulness of the proposed seven indices. The results show that some indices are informative with regards to the degree of motion contamination in preprocessed fMRI data.

1 Introduction Implementing image registration (or motion correction) in fMRI tool boxes has become a routine procedure before statistical analyses. A major purpose of image registration is to make a distinction between the change of signal intensity due to head motion and that due to brain activities. There are situations in which fMRI images are seriously contaminated by head motion and cannot be completely recovered by image registration methods. It would be informative to have indices indicating motion contamination in the preprocessed data. Without knowing motion contamination, the subsequent analysis using averaged fMRI data across subjects could be misleading. Rather than focusing on motion correction algorithms [1], M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 209–218, 2008. © Springer-Verlag Berlin Heidelberg 2008

210

W.-C. Chou, M. Liou, and H.-R. Su

[2], [3], we propose seven indices for detecting motion contamination in preprocessed fMRI data. The indices can be applied after motion correction by the image registration algorithms. In the simulation studies, we analyzed a real fMRI data set using a statistical method and estimated the brain activation maps. The image data were then randomly shifted or rotated to simulate different degrees of head motion. The data contaminated by random motion were corrected using the SPM image coregistration algorithms. The motion indices were computed using the corrected images. The corrected images were then analyzed again using the same statistical method. The consistency between the activation maps of real data and those of simulated data was used as a standard to evaluate the usefulness of the proposed indices. In the next section, the seven indices are introduced in details. In Section 3, the performance of indices are evaluated by comparing their uniformity in and sensitivity to detecting motion artifacts in real fMRI data. We finally discuss the use of different indices.

2 Measuring Motion Artifacts The indices we propose here provide a summary measure for motion contamination especially in the preprocessed fMRI data without necessarily knowing real motion artifacts. In real applications, it is nearly impossible to define the true errors caused by head movement or brain warping. In this study, we give a focus on finding adequate indices based on the notion that similarity between image volumes adjacent in the time domain would be a good indicator of any dislocation due to head movement in fMRI time series. The ensuing indices are therefore completely decided by the innate nature of image signals. In this section, we give seven indices and compare between them according to their sensitivity to small motion artifacts and uniformity in measuring different degrees of contamination. 2.1 Ratio Image Uniformity We use the ratio image uniformity adopted in AIR 3.0 [4] as the first index Iriu. The ratio image is created by means of computing the ratio of intensity of two image volumes on a voxel-by-voxel basis, and then the uniformity of this ratio is represented by its standard deviation σ. Uniformity guarantees the similarity of volumes, and gives smaller standard deviations. In other words, we would expect small motion contamination in fMRI data if the ratio has a smaller standard deviation across image volumes. Here we denote the intensity value of the focal image as M, the intensity value of the reference image as N, and the voxel coordinate by r. The first index Iriu can be simply expressed as

I riu = σ (

M (r ) ). N (r )

(1)

Sensitivity and Uniformity in Detecting Motion Artifacts

211

2.2 Scaled Least-Squared Difference The second index Isls , referred to as scaled least-squared difference, is to describe the global intensity difference by using a modified least-squared approach [5]. In this index, global intensity of one volume is rescaled to that of another volume. Given the total voxel number v, this normalized index can be shown as

I sls = with M and

⎞ 1 ⎛ M ⎜⎜ M (r ) − N (r ) ⎟⎟ ∑ v r ⎝ N ⎠

σ

2 M (r )

+σ

2 N (r )

2

,

(2)

N being the mean intensity of these two volumes, respectively.

2.3 Correlation Coefficient The information of how two images correlate with each other can refer to their correlation coefficient, here denoted as the third index Icc [6]. The high correlation indicates that the image time-series are under a stable status. Let Cov(M,N) be the covariance between M and N and the correlation coefficient is defined as

I cc =

Cov( M (r ), N (r ))

σ M ( r )σ N ( r )

.

(3)

2.4 Joint Entropy The voxel similarity based on information theory has been broadly applied in image registration because of their accuracy and robustness [1], [7], [8]. The commonlyused joint entropy, as adopted here, measures the dispersion of a distribution computed from a joint intensity histogram. The joint distribution pM,N can be estimated by each entry in the joint histogram of two image volumes divided by the total number of voxels. The entropy is low when the images are so similar that two anatomical structures overlap each other and their joint distribution shows certain concentrated clusters. The joint entropy of two images M and N can be calculated by the following equation:

I je = − ∑ p M ,N (i , j ) log p M ,N (i , j )

(4)

i, j

where i and j indicate the intensity value of M and N, respectively. 2.5 Relative Entropy The relative entropy, also called Kullback–Leibler divergence, measures the distance between two probability distributions. Given the probability of the focal image PM and that of the reference image PN , we can compute the relative entropy using the following formula:

212

W.-C. Chou, M. Liou, and H.-R. Su

I re =

P ( j) ⎤ 1⎡ PM ( j ) + ∑ PN ( j ) N ⎢ ∑ PM ( j ) ⎥. 2⎣ j PN ( j ) PM ( j ) ⎦ j

(5)

where j denotes the jth intensity value. 2.6 Weighted Kappa The weighted kappa index Ikw proposed by Cohen [13] is a measure of agreement between two ratings of the same image. The calculation procedure follows the following steps: Step 1: The observed weighted proportion of agreement is given by

Po = ∑ w(i, j ) pM , N (i, j ) ,

(6)

i, j

where the summation is over all possible intensity values, and the chance-expected proportion of agreement by

Pc = ∑ w(i, j ) pM (i) pN ( j ) .

(7)

i, j

In the equations above, the weights are computed by

w(i, j ) = 1 −

i− j , G −1

(8)

where G is the total number of distinct intensity values. Equation (8) assigns higher weights to intensity values closer to each other. Step 2: The weighted kappa value can be calculated by

I kw =

Po − Pc 1 − pc

(9)

Cicchetti and Fleiss [14] gave the sample standard error of a kappa estimate, and the sample estimate in (9) can be tested for significance in applications. 2.7 Mean Distance to the Principal Component The final index is defined as the average distance to the principal component. Here we use the principal components analysis [10] to find the principal axis in the joint histogram of two image volumes. The distance to the principal axis is defined as the orthogonal projection of each voxel to the axis. The average of the distances can be used as an indicator of the degree of image contamination.

Sensitivity and Uniformity in Detecting Motion Artifacts

213

3 Experimental Results In the empirical study, we used a real fMRI dataset collected in the Mechelli et al. study [11], which is a part of the general collection of the US fMRI Data Center. The data of the third subject in the Mechelli et al. study with 360 image volumes were selected for our empirical study. Based on the preprocessed images provided in the Table 1. The means and standard deviations of motion indices fordifferent simulation datasets Datasets Degree of rotation - X axis - Y axis - Z axis

1

2

3

4

5

6

15 2 2

2 15 2

2 2 15

15 15 2

2 15 15

15 2 15

Motion Indices

Ratio Image Uniformity

mean variance

47.95 42.92 19.25 67.40 50.56 54.47 2288.2 1374.8 128.64 2190.8 1350.0 1870.3

Least-squared

mean variance

0.193 0.143 0.155 0.244 0.219 0.252 0.0157 0.0085 0.0081 0.0112 0.0071 0.0126

Correlation Coefficient

mean variance

0.903 0.927 0.922 0.877 0.889 0.873 0.0040 0.0021 0.0020 0.0028 0.0018 0.0032

Joint Entropy

mean variance

7.565 7.457 7.493 7.638 7.601 7.623 0.109 0.113 0.098 0.066 0.045 0.064

Relative Entropy

mean variance

0.0921 0.0756 0.0745 0.0805 0.0897 0.0731 0.0154 0.0125 0.0120 0.0136 0.0205 0.0119

Weighted Kappa

mean variance

0.773 0.802 0.789 0.741 0.747 0.706 0.0066 0.0084 0.0045 0.0044 0.0033 0.0045

mean variance

8.501 7.417 8.126 10.121 9.863 10.449 11.586 8.333 8.571 8.113 5.710 8.518

Mean Distance to the Principal Component

214

W.-C. Chou, M. Liou, and H.-R. Su

Reproducibility Maps (Original data)

Reproducibility Maps (Dataset No. 1)

Reproducibility Maps (Dataset No. 2)

Reproducibility Maps (Dataset No. 3)

Reproducibility Maps (Dataset No. 4)

Reproducibility Maps (Dataset No. 5)

Reproducibility Maps (Dataset No. 6)

Fig. 1. The activation maps for empirical and simulated datasets estimated using the method of reproducibility analysis for a few slices in the Mechelli et al. study. The increased and decreased responses for Subject 3 are indicated by the red and green colors respectively. The superior frontal gyrus, and supramarginal gyrus are shown in the slices located in the upper yellow block, and the noise mainly appears in images located in the lower block.

Sensitivity and Uniformity in Detecting Motion Artifacts

215

dataset, we conducted the reproducibility analysis to estimate the brain activation maps [12]. The activation maps are used as the standard to evaluate the performance of the proposed motion indices. In the simulation study, we rotated the preprocessed

Fig. 2. Plots of motion indices for the preprocessed images provided by the US fMRI Data Center and the corrected fMRI images in the datasets No. 2 and 6. The contaminated datasets were corrected for motion by maximizing mutual information based on the Powell algorithm. In the six plots, the gray line refers to the motion indices of the preprocessed images; the red line dotted with solid triangles and yellow line with empty reversed triangles respectively refer to the indices of corrected images in the datasets No. 2 and 6. The vertical and horizontal axes represent the index values and image volumes along the time scale, respectively.

216

W.-C. Chou, M. Liou, and H.-R. Su

data along the X-, Y-, and Z-directions randomly within the range shown in Table 1 to simulate different degrees of motion contamination. The contaminated images were then corrected using the maximization of normalized mutual information based on the Powell algorithm [7], [8]. The motion indices were also applied to the corrected images. Finally, we conducted the reproducibility analysis again to estimate the brain activation maps based on the corrected images. Table 1 gives the means and standard deviations of motion indices for different simulated datasets. Datasets 1, 2 and 3 are less contaminated by motion as compared with datasets 4, 5 and 6. Therefore, a sensitive index should give larger values (or smaller correlations and kappa values) for datasets 4, 5 and 6. In general, the proposed motion indices more or less show the contrast between modest and serious motion contamination. The mean distance to the principal component tends to give greater contrast compared with other indices. Because the Mechelli et al. study was conducted for investigating the words and pseudo-words reading, the activation regions found by the reproducibility analysis correspond to the superior frontal gyrus and supramarginal gyrus. The activation maps based on corrected images are shown in Fig. 1. Among the three datasets with modest contamination, Dataset No. 2 after coregistration using the Powell algorithm gives almost the same activation maps as the original results. The mean distance to the principal component also outperforms other motion indices by giving the smallest value to Dataset No. 2. In terms of uniformity, the scaled leastsquared, correlation coefficient, joint entropy, weight kappa, and mean distance to the principal component indices yield reasonable results. The five indices are also sensitive to motion contamination. Freire et al. reported similar unfavorable results for the ratio image uniformity index [1]. The change in marginal intensity probabilities between adjacent image volumes are not sensitive to serious head motion. Therefore the relative entropy index fails to detect motion contamination for the simulated datasets. In Fig. 2, we plot the motion indices (except for the relative entropy index) across image volumes based on the original preprocessed images, and the corrected images for the simulated datasets No. 2 and No. 6. The sharp peaks in the plots for the original fMRI data suggests the dislocations (head motion) between two adjacent image volumes. The plots suggest that the scaled least-squared, correlation coefficient, and mean distance to the principal component indices serve as good criteria by comparing between the ensuring reproducibility maps of original data and those of simulated data. However, the plots for joint entropy and weighted kappa suggest that the two indices do not differentiate the degree of contamination in datasets No. 2 and No. 6. The algorithm used for correcting motion contamination was based on maximizing the normalized mutual information defined as (HM(i)+HN(i))/HM,N(i,j). If the maximized normalized mutual information and individual marginal entropy of corrected images in datasets No. 2 and No. 6 are similar, we can always obtain similar joint entropy.

Sensitivity and Uniformity in Detecting Motion Artifacts

217

4 Discussion In this study, we have proposed the use of seven indices for detecting motion contamination in fMRI data. Based on the empirical results, the scaled least-squared, correlation coefficient, joint entropy, weighted kappa and mean distance to the principal component perform uniformly in measuring dislocations between two image volumes. In applications, the plotting of motion indices across image volumes before and after image registration would be informative for detecting possible effects due to motion contamination on the subsequent statistical analyses. Among the five indices, the mean distance to the principal component is the best choice for informing the usefulness of preprocessed fMRI data. Although we have some conclusions in this study, much quantitative work has to be done to further verify the performance of these indices. Acknowledgments. This research was supported by the grant NSC 95-2413-H-001001-MY3 from the National Science Council (Taiwan).

References 1. Freire, L., Mangin, J.F.: Motion correction algorithms create spurious brain activations in the absence of subject motion. Neuroimage 14, 709–722 (2001) 2. Hellier, P., Barillot, C., Corouge, I., Gibaud, B., Le Goualher, G., Collins, D.L., Evans, A., Malandaln, G., Ayache, N., Christensen, G.E., Johnson, H.J.: Retrospective evaluation of intersubject brain registration. IEEE Transactions on Medical Imaging 22, 1120–1130 (2003) 3. Oakes, T.R., Johnstone, T., Walsh, K.S.O., Greischar, L.L., Alexander, A.L., Fox, A.S., Davidson, R.J.: Comparison of fMRI motion correction software tools. Neuroimage 28, 529–543 (2005) 4. Woods, R.P., Grafton, S.T., Holmes, C.J., Cherry, S.R., Mazziotta, J.C.: Automated image registration: I. General methods and intrasubject, intramodality validation. Journal of Computer Assisted Tomography 22, 139–152 (1998) 5. Alpert, N.M., Berdichevsky, D., Levin, Z., Morris, E.D., Fischman, A.J.: Improved methods for image registration. Neuroimage 3, 10–18 (1996) 6. Ghahramani, S.: Fundamentals of Probability, 2nd edn (2000) 7. Maes, F., Collignon, A., Vandermeulen, D., Marchal, G., Suetens, P.: Multimodality image registration by maximization of mutual information. IEEE Transactions on Medical Imaging 16, 187–198 (1997) 8. Studholme, C., Hill, D.L.G., Hawkes, D.J.: An overlap invariant entropy measure of 3D medical image alignment. Pattern Recognition 32, 71–86 (1999) 9. Cohen, J.: Weighted Kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychological Bulletin 70, 213–220 (1968) 10. Cicchetti, D.V., Fleiss, J.L.: Comparison of the null distributions of weighted kappa and the C ordinal statistic. Applied Psychological Measurement 1, 195–201 (1977)

218

W.-C. Chou, M. Liou, and H.-R. Su

11. Jolliffe, I.T.: Principal Component Analysis. Springer, Heidelberg (2002) 12. Mechelli, A., Friston, K.J., Price, C.J.: The effects of presentation rate during word and pseudoword reading: A comparison of PET and fMRI. Journal of Cognitive Neuroscience 12, 145–156 (2000) 13. Liou, M., Su, H.R., Lee, J.D., Aston, J.A.D., Tsai, A.C., Cheng, P.E.: A method for generating reproducible evidence in fMRI studies. Neuroimage 29, 383–395 (2006)

A Ring Model for the Development of Simple Cells in the Visual Cortex Takashi Hamada1 and Kazuhiro Okada2 1

National Institute of Advanced Industrial Science and Technology (AIST), Kansai center 1-8-31, Midorigaoka, Ikeda, Osaka 586-8577, Japan [email protected] 2 Murata Machinery, Ltd., Communication Equipment Division 136, Takeda-Mukaishiro-cho, Fushimi, Kyoto 612-8686, Japan [email protected]

Abstract. A model was proposed for the development of simple receptive fields in the cat visual cortex, based on the empirical evidence that the development is due to dark or spontaneous activities in the lateral geniculate nucleus (LGN). We assumed that several cortical cells are arranged in a ring, with mutual excitatory and inhibitory connections of fixed weights; The cells also receive excitatory synapses from LGN cells, whose synaptic weights are initially set to be random and then modified according to the Hebb rule as well as to positive correlations among nearby LGN cells. Computer simulation showed that the cortical cells finally acquire two-dimensional simple receptive fields with their phases gradually varied along the ring. Keywords: Simple cell, development, spontaneous activity, correlation, Hebb rule, model.

1 Introduction Cells in the layer 4 of the cat visual cortex have so-called simple receptive fields in which several elongated On and Off sub-areas are alternately arranged [1]. If a slit of light is positioned on the On sub-area with axes of their elongations aligned each other and turned on, or if a slit of light positioned and oriented on the Off sub-area is turned off, the cells yield firings. If firings to the stimulations of the On sub-areas are plotted positively and those to the Off sub-areas negatively with respect to the positions of the slit, the profile of the firings is described with a Gabor function [2]: A sinusoid with a phase parameter between 0 and 2 π multiplied by a Gaussian. Interestingly, if many simple cells are sampled and each of them is best described by a Gabor function, their phases are distributed not merely around 0 and π /2, i.e. as those of cosine and sine Gabor functions, but uniformly between 0 and 2 π [3]. Based on this evidence, we have already proposed a neural model for simple receptive fields, where several simple cells are arranged in a ring with mutual excitatory or inhibitory connections of fixed weights [4]; Phases of their receptive fields thereby varied gradually with rotation along the ring. M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 219–227, 2008. © Springer-Verlag Berlin Heidelberg 2008

220

T. Hamada and K. Okada

Meanwhile, such simple receptive fields are known to be basically organized before the eyes are naturally opened in kittens [5]. This suggests that the organizing process does not require any visual experiences. As a matter of fact, experimental studies suggested that the process could be due to some correlations in dark or spontaneous firings in the lateral geniculate nucleus (LGN) [6, 7]. A pioneering theoretical study [8] assumed the correlation to have the shape of "Mexican hat", i.e. the firings are correlated positively among nearby cells and negatively among farther cells. Assuming correlations of the "Mexican hat" as well as the ring structure, we have theoretically shown that cortical cells in the ring, whose synaptic weights from LGN are initially set to be random, develop simple receptive fields with their phases gradually varied between 0 and 2 π along the ring [9]. This study, however, concerned only with one-dimensional profiles of receptive fields perpendicular to their preferred orientations. We here propose a model, again with the ring structure, for the development of two-dimensional profiles of simple receptive fields. Computer simulation revealed that the cortical cells develop simple receptive fields whose phases gradually vary with rotation along the ring, only if the correlation was assumed to have a shape of Gaussian as recently shown experimentally in the developing LGN of ferrets [10], but fail to develop such receptive fields if the shape of “Mexican hat” is assumed for the correlation.

2 Methods 2.1 Structure of the Model For the sake of simplicity, we consider only a set of cortical cells whose receptive fields will be the same as to orientations, ocular dominances and retinal positions, but different as to phases. We assume that several cortical cells Cp, p=1~n, are arranged in a ring (the upper part in Fig. 1a) with mutual intra-cortical connection Ip,q (p, q=1~n):

⎛ 2π × p − q ⎜ = cos I p ,q ⎜ n ⎝ =0.1

⎞ ⎟ ⎟ ⎠

for p ≠ q

(1)

for p = q .

The interaction is excitatory between nearby cells and inhibitory between diagonally opposed cells on the ring (see 2.4 for detail). The ring structure is meant to be merely functional. We consider two two-dimensional arrays of LGN cells, one for On center cells and the other for Off center cells, each with the size m x m (the lower part in Fig. 1a). Each of the On LGN cells yields an excitatory synapse to each of the cortical cells: The synapses from the On array to a cortical cell Cp (p=1,…, or n) thereby compose a two-dimensional m x m list of weights

W

on p

. Similarly

W

off p

for Off LGN cells.

A Ring Model for the Development of Simple Cells in the Visual Cortex

221

Fig. 1. The model. a: Cortical cells in a ring with interactions Ip,q and arrays of On and Off LGN cells. b: Correlated spontaneous activities among LGN cells of the same type (upper) and of the different types (lower) in one of the trials with correlations in (2) and (3).

2.2 Correlation of Spontaneous Activities in LGN Although correlations of spontaneous activities have been physiologically studied in adult retina of cats [11] and in developing LGN of ferrets [10], those in the developing LGN of cats are not yet studied physiologically. We thereby tested two types of functions for the correlations. The function firstly tested for correlations between cells of the same type, i.e. among On center cells or among Off center cells, is a Gaussian in (2) where r is the separation between the centers of their receptive fields and

σ

c

the

standard deviation of the Gaussian. Thus, spontaneous activities of two LGN cell of the same type are assumed to be more positively correlated if their receptive fields are closer to each other.

C

same

2 ⎛ ⎞ ⎜ ⎟ r (r ) = exp⎜ − . 2⎟ ⎜ 2× ⎟ σc ⎠ ⎝

(2)

Correlation between spontaneous activities of LGN cells of different types, i.e. one On center cell and the other Off center cell, is assumed to follow the function (2) multiplied by -0.3:

C

diff

( r ) = − 0 .3 × C

same

(r ) .

(3)

Thus, LGN cells of different types are negatively and weakly correlated only if their receptive fields are close to each other.

222

T. Hamada and K. Okada

The function secondly tested for correlations between LGN cells of the same type has the shape of "Mexican hat":

C

same

2 ⎛ ⎞ ⎛ ⎜ ⎟ ⎜ r (r ) = exp⎜ − − 0.1 × exp⎜ − 2⎟ ⎜ 2× ⎜ 2× σ c ⎟⎠ ⎝ ⎝

⎞

2

⎟ r . (3σ c) ⎟⎟⎠

(4)

2

Thus, spontaneous activities of two cell of the same type are positively correlated if r is small, but negatively at larger values of r. Correlation between cells of different types is assumed to be the function of (4) multiplied by -0.3. The developmental process is assumed to be an iteration of many (usually more than 5000) trials, in which a set of xy coordinates for a point as the center of correlations is randomly selected in either of the two LGN arrays and correlated activities spread around the point on the array according to the same-type function. Additionally, correlated activities spread around a point with the same coordinates on the other array according to the different-type function. Fig. 1b explains such correlated spontaneous activities in one of the trials, where correlations in (2) and (3) are used. The point is incidentally selected at {x, y}={7, 6} on the On array and correlated activities is thereby positive on the On array and negative on the Off array. In another trial, the point would be differently positioned, correlation on one of the arrays would be incidentally positive and that on the other array would be negative. Correlated spontaneous activities on the On and Off arrays in the i-th trial are named

spont on

and i

spont off

, respectively. i

2.3 Initial Synaptic Weights At the earlier stage of development, LGN axons are attracted toward a cortical neuron presumably according to a gradient of diffusible molecules from the cortical cell [12]. Hence, we suppose that the closer to the center of the array a LGN cell is positioned, the stronger tends to be the synaptic weight from the LGN cell to one of the cortical cells. For describing this tendency, an arbor function A(r) is defined: 2 ⎛ ⎞ ⎜ ⎟ r A(r ) = exp⎜ − . 2⎟ ⎜ 2 ×σ ⎟ a ⎠ ⎝

(5)

It is a Gaussian with respect to distance r of a LGN cell from the center of the array, with the standard deviation

σ

a

. The list of initial synaptic weights

on

Wq

i =0

(q=1~n) is thereby assumed to be a list (m x m) of random numbers between 0 and 1, Rand, multiplied by A(r):

A Ring Model for the Development of Simple Cells in the Visual Cortex

on

Wq Similarly for

off

Wq

i =0

223

= Rand × A(r ) .

. Fig. 2a shows the i =0

(6)

on

Wq

for q=1~n in the case n=5. i =0

2.4 Modification of the Synaptic Weights Activity of a cortical cell in a trial is determined by inputs from the LGN cells as well as from the other cortical cells in the ring. Inputs from LGN cells to a cortical cell Cq (q=1,…, or n) in the i-th trial, On LGN array multiplied by i-th trial multiplied by

pre q

, is described by spontaneous activities on the i

on plus spontaneous activity on the Off array at the

Wq

i

off :

Wq

i

on off + W off pre q = W on q × spont q × spont i

i

i

i

The final activity of Cp in the i-th trial,

post p

.

(7)

i

, is a sum of the inputs from the i

LGN cells plus inputs from the other cortical cells in the ring, which is described as: n

post p Note that

I

p ,q

i

= ∑ I p ,q × q =1

pre q

.

(8)

i

(p, q=1~n) is defined so that inputs from LGN are only weakly

weighted by 0.1, while inputs from the cortical cells are weighted by 1*Cosine function (see the equation (1)). Such a design of p ,q is based on the anatomical evidence that

I

the number of synaptic buttons that a layer 4 cell receives from LGN cells occupies only a few percent among the number of synapses on its cell body and dendrites [13], which means that synaptic influences from the other cortical cells are much stronger than those from LGN cells. Synapses in the visual cortex are known to be modifiable during development [12]. We thereby assume that the synapses are modified due to the Hebb rule: Synaptic weight is strengthened when the pre-synaptic activity is correlated with the postsynaptic activity and weakened otherwise. Changes in trials are then hypothesized:

W

on p

between i-th and i+1-th

224

T. Hamada and K. Okada

on

Wp λ is off W p i +1 . where

i +1

= W on p + λ × A(r ) × i

pre p × post p i

.

(9)

i

a constant which determines the rate of development. Similarly for

We assume that the total synaptic weights over each cortical cell are conserved:

on

Wp

+ i +1

off

Wp

= const .

(10)

i +1

Note that On and Off synaptic weights are conserved jointly [14], but not separately as in [10]. The formula in (9) is computed under the constraint of (10) in a trial, and the trials are iterated until the values converge. 2.5 Visual Responses After the synaptic weights are converged, visual responses of the cortical cells are computed. We assume the receptive field of a LGN cell positioned at 0 on the array

σ

has a Gaussian form, where

r

r

is the standard deviation of the distribution.

⎛ ⎜ Rr 0 = exp⎜⎜ − ⎝ A light spot at a position

r

(r −r 0) ⎞⎟ . 2

⎟ 2×σ r ⎟⎠

(11)

2

q

0

yields inputs from LGN to Cq,

previsr , as: 0

q

previsr = Rr ×W 0

0

on q

− R ×W q . r0 off

(12) p

The final visual activity in Cp due to the spot at

I

p ,q

r postvisr , 0

, results from 0

:

postvisr = ∑ previsr × I p

q =1

0

The visual receptive field of cell respect of

r

0

.

q

n

c

p ,q

.

(13)

0

p

p

is a spatial summation of

postvisr

with 0

A Ring Model for the Development of Simple Cells in the Visual Cortex

225

2.6 Parameters The parameters were set as follows, if not noted otherwise: n=5, m=15,

σ

a

=3,

σ

c

σ

=0.5,

r

=1.5,

λ =0.1.

3 Results 3.1 Development of Simple Receptive Fields Each of the subfigures in Fig. 2a is the initial synaptic weight

W

on p

for p=1,…or n

which are random numbers multiplied by a Gaussian envelop. Note that although these subfigures, as well as those in the other columns in the figure, are vertically arranged in order of p, they actually constitute a ring structure. Fig. 2b shows the synaptic weights,

W

on p

in the left column and

W

off p

in the right, after 10000 iterations when the

Gaussians in (2) and (3) are used as the correlation functions. Note firstly that each of

W

on p

and

W

off p

(p=1, …, or n) is segregated, i.e. strengthened in some sub-areas

and weakened in the other sub-areas. Secondly, the boundaries between the sub-areas

Fig. 2. a: Initial

W

on

. b:

W

on

(left) and

W

off

(right) after trials with Gaussian correla-

tions. c: Visual receptive fields, resulted from the synaptic weights in b and Ip, q. Bars are added for clarifying translations of the receptive fields along the column. d: Visual receptive fields, after trials with the Mexican hat correlations. White arrows are added for clarifying rotations of the fields along the column. The subfigures in each of the columns are actually arranged in a ring.

226

T. Hamada and K. Okada

are roughly straight. Thirdly, configuration of the sub-areas is similar between two adjacent cells on the ring, but mostly reversed between diagonally opposed cells, so that the boundaries gradually translate with rotations along the ring: In the case of Fig. 2b, the boundaries gradually translate toward the lower-right along the downward direction in the column. Besides,

W

on p

and

W

off

for the same cell

p

C

p

(p=1,…, or

n), i.e. two subfigures in one of the rows of Fig. 2b, are reversed images each other: Sub-areas which are strongly innervated by synapses from On LGN cells are weakly innervated by synapses from Off LGN cells, and vice versa. Fig. 2c shows visual receptive fields, i.e. distribution of visual responses due to the synaptic weights in Fig. 2b as well as due to the intra-cortical interaction p,q , which

I

again show the properties described above for

W

on p

and

W

off p

. As an example, the

receptive fields are composed of On and Off sub-areas, each of which corresponds to the sub-areas in

W

on p

and

W

off p

. Besides, the configurations of the sub-areas

gradually translate with rotation along the ring. 3.2 Development of Labyrinthine Receptive Fields If the correlations have the shape of "Mexican hat" in (4) and (5), boundaries between the sub-areas after a sufficient number of trials become labyrinthine. Thus, the model fails to develop simple receptive fields. Interestingly, if the value of λ is additionally increased to 1.0, the labyrinth almost always rotates along the ring. In the case of Fig. 2d, the visual receptive fields rotate clockwise along the downward direction in the column. Note that the third property described above is again held, i.e. the configuration of the sub-areas is similar between two adjacent cells on the ring and reversed between diagonally opposed cells. The rotation along the ring is a compromise between the labyrinthine configurations and the third property.

4 Discussion If correlations of spontaneous activities among the same-type LGN cells were assumed to have the shape of "Mexican hat", labyrinthine visual receptive fields emerged, presumably due to the inhibitory peripheries in the hat. However, if the correlations were assumed to have the shape of Gaussian, simple receptive fields often emerged. As a matter of fact, correlations with the shape of Gaussian were experimentally observed in the developing LGN of ferrets [10]. The same authors also theoretically showed that the Gaussian correlations result in simple receptive fields only if the On and Off synaptic weights were constrained separately [10]. This separate constraints is biologically less plausible, however, than the joint constraint [14]. The present study showed that the Gaussian correlation plus the joint constraint result in simple receptive fields. We assumed an intra-cortical interaction, where nearby cells on the ring were mutually excited and cells diagonally opposed on the ring mutually inhibited. These interactions could be due to excitatory and inhibitory synaptic connections, but otherwise

A Ring Model for the Development of Simple Cells in the Visual Cortex

227

due to near diffusion of some excitatory substances and far diffusion of some inhibitory ones. The latter possibility reminds a chemical explanation for the Turing's diffusion-reaction equation [15], which also emerges stripes and labyrinths [16]. Configurations of the receptive fields after a sufficient number of trials were either linearly translated as shown in Fig.2c or rotated as in Fig. 2d, along the ring. These configurations could be interpreted as neural representations on the ring of translational and rotational motions, respectively.

References 1. Hubel, D.H., Wiesel, T.N.: Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J. Physiol. 160, 106–154 (1962) 2. Marcelja, S.: Mathematical description of the responses of simple cortical cells. J. Opt. Soc. Am. 70, 1297–1300 (1980) 3. DeAngelis, G.C., Ohzawa, I., Freeman, R.D.: Spatiotemporal organization of simple-cell receptive field in the cat’s striate cortex. 1. General characteristics and postnatal development. J. Neurophysiol. 69, 1091–1117 (1993) 4. Hamada, T., Yamashima, M., Kato, K.: A ring model for spatiotemporal properties of simple cells in the visual cortex. Biol. Cyb. 77, 225–233 (1997) 5. Blakemore, C., Sluyters, R.C.V.: Innate and environmental factors in the development of the kitten’s visual cortex. J. Physiol. 248, 663–716 (1975) 6. Chapman, B., Godecke, I.: Cortical cell orientation selectivity fails to develop in the absence of On-center retinal ganglion cell activity. J. Neurosci. 20, 1922–1930 (2000) 7. Sengpiel, R., Kind, P.C.: The role of activity in development of the visual system. Curr. Bio. 12, 818–826 (2002) 8. Miller, K.D.: A model for the development of simple cell receptive fields and the ordered arrangement of orientation columns through activity-dependent competition between onand off-center inputs. J. Neurosci. 14, 409–441 (1994) 9. Hamada, T., Kato, K., Okada, K.: A model for development of Gabor-receptive fields in simple cortical cells. NeuroReport 7, 745–748 (1996) 10. Ohshiro, T., Meliky, M.: Simple fall-off pattern of correlated neural activity in the developing lateral geniculate nucleus. Nature Neurosci. 9, 1541–1548 (2006) 11. Mastronarde, D.N.: Correlated firing of retinal ganglion cells. Trends Neurosci. 12, 75–80 (1989) 12. Goodman, C.S., Shatz, C.J.: Developmental mechanisms that generate precise patterns of neuronal connectivity. Cell 72/Neuron. 10(suppl.), 77–98 (1993) 13. Ahmed, B., Anderson, J.C., Douglas, R.J., Martin, K.A.C., Nelson, J.C.: Polyneuronal innervation of spiny stellate neurons in cat visual cortex. J. Comp. Neurol. 341, 39–49 (1994) 14. Willshaw, D.J., von der Malsburg, C.: How pattered neural connections can be set up by self-organization. Biol. Cyb. 58, 63–70 (1988) 15. Turing, A.M.: The chemical basis of morphogenesis. Phil. Trans. Royal Soc. London B237, 37-72 (1952) 16. Shoji, H., Iwasa, Y.: Labyrinthine versus straight-striped patterns generated by two-dimensional Turing systems. J. Theor. Biol. 237, 104–116 (2005)

Practical Recurrent Learning (PRL) in the Discrete Time Domain Mohamad Faizal Bin Samsudin, Takeshi Hirose, and Katsunari Shibata Department of Electrical and Electronic Engineering, Oita University, 700 Dannoharu, Oita 870-1192 Japan [email protected]

Abstract. One of the authors has proposed a simple learning algorithm for recurrent neural networks, which requires computational cost and memory capacity in practical order O(n2)[1]. The algorithm was formulated in the continuous time domain, and it was shown that a sequential NAND problem was successfully learned by the algorithm. In this paper, the authors name the learning “Practical Recurrent Learning (PRL)”, and the learning algorithm is simplified and converted in the discrete time domain for easy analysis. It is shown that sequential EXOR problem and 3-bit parity problem as non linearlyseparable problems can be learned by PRL even though the learning performance is often quite inferior to BPTT that is one of the most popular learning algorithms for recurrent neural networks. Furthermore, the learning process is observed and the character of PRL is shown. Keywords: Recurrent Neural Network (RNN), Supervised Learning, Practical Recurrent Learning (PRL), BPTT, Short-Term Memory.

1 Introduction When we think of the higher functions in humans, such as logical thinking, conversation, and so on, it is easily noticed that memory plays an important role in the functions. Accordingly, it is expected that the need for the RNN is going to grow drastically in the near future as the increase of the desire to the higher functions. Conventionally, there are two popular learning algorithms for recurrent neural networks that have been proposed. One is BPTT (Back Propagation Through Time)[2] and the other one is RTRL[3] (Real Time Recurrent Learning). In BPTT, all the past states of the network are stored using O(nT) of memory where n is the number of neurons and T is the present time step, and the learning is done by tracing back to the past using the memory. The order of the computational cost is O(n2T). The traced-back time step is often truncated at a constant number when T becomes large, but it is difficult to know the sufficient number of steps. On the other hand, in RTRL, the influence of each connection weight to the output of each neuron is kept in O(n3) of memory, and the order of the computation of the influence is as large as O(n4). BPTT is not practical in the meaning that the learning should be done with tracing back to the past. Even though the special hardware is developed, iteration of learning for the traceback is necessary. RTRL is not practical in the meaning that the required M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 228–237, 2008. © Springer-Verlag Berlin Heidelberg 2008

Practical Recurrent Learning (PRL) in the Discrete Time Domain

229

order O(n3) in the memory capacity and O(n4) in the computational cost are larger than O(n2) that is the order of the number of connections in a neural network. Even though each connection has some memory, a memory on the connection should have O(n) size, that means that the size of each memory should be larger according to the size of the neural network. S. Hochreiter and J. Schmidhuber have proposed a special network architecture that has some memory cells. In each memory cell, there is a linear unit with a fixed weight self-connection that enables constant, non-vanishing error flow within the memory cell[4]. They used a variant RTRL and only O(n2) of computational cost is required. However, special structure is necessary and it cannot be applied to the general recurrent neural networks. Therefore, a practical learning algorithm for the general recurrent neural networks that need O(n2) or less memory and O(n2) or less computational cost is strongly required. Then Practical Recurrent Learning (PRL) was proposed in the continuous time domain. In this paper, PRL is simplified and converted in the discrete time domain for easy analysis, and the learning performance is compared to BPTT.

2 Practical Recurrent Learning (PRL) Here, PRL is explained using an Elman-type recurrent neural network as shown in Fig. 1. Output layer (3rd layer) Hidden layer (2nd layer)

...........

Input layer (1st layer)

...........

Fig. 1. An Elman-type recurrent neural network

2.1 PRL in the Continuous Time Domain[1] This section describes roughly about PRL in the continuous time domain proposed in [4]. The forward calculation is the same as the conventional neural network that means that each hidden or output neurons calculate the weighted sum of the inputs and then non-linear function f is applied to get the output. Here, the sigmoid function whose value range is from -0.5 to 0.5 is used. In the output layer, the error signal is calculated by δ j (3) = Tr j (t ) − x j (3) (t )

(1)

where Tr : training signal, xj(3):output of the output unit. Differing from the regular BP, the derivative of the output function f’j(3) is not included. As well as the regular

230

M.F.B. Samsudin, T. Hirose, and K. Shibata

BP, the error signal in the hidden layer δi(2) is calculated from the δj(3) in the upper layer as described by the following equations. δ i (2) =

∑v

ji δ j

( 3)

(2)

(t )

j

(

(

) )

d d (3) v ji = w ji (t ) (3) f ' S (j3) (t ) − v ji x j (t ) dt dt

(3)

where wji(3): connection weight (ith hidden unit - jth output unit) , Sj(3) : the net value of the jth neuron in the output layer. f’ is included in this equation on behalf that f’ disappears in Eq. (1) in order to use f’ when the output changed. Then, in order to modify the value of weight without tracing back to the past, it is considered that the following information should be held. (a) the latest outputs of pre-synaptic neurons, (b) the outputs of pre-synaptic neuron that changes recently among all the inputs to the post-synaptic neuron, (c) the outputs of the pre-synaptic neuron that caused the change of the postsynaptic neuron’s output. Corresponding to the (a),(b),(c), three variables p(t), q(t), r(t) that hold the past information in various ways are introduced and they are always modified according to the following differential equations. τj

(

d p ji (t ) = − p ji (t ) + xi (t ) f ' S j (t ) dt

(

(

)

d q ji (t ) = xi (t ) f ' S j (t ) − q ji (t ) dt

(

( )

)

)∑ dtd xi (t )

(4)

(5)

i

)

d d r ji (t ) = xi (t ) f ' S j (t ) − r ji (t ) x j (t ) dt dt

(6)

Using the three variables, each connection weight is modified. The following equation is an example but the details can be seen in [1].

(

)

dw ji (t ) = p ji (t ) + q ji (t ) + r ji (t ) δ j (t )

(7)

Among the three variables, rji(t) is considered to be a particularly important variable with respect to the learning of a problem that needs the past information before a long time lag. Fig. 2 shows an example of the temporal change of the variable rji(t) according to the input signal xi(t) and the output signal xj(t). As shown in Fig. 2, it is the important character that rji(t) holds the information about the output of the pre-synaptic neuron that caused the change of the post-synaptic neuron’s output. This variable ignored the inputs while the output did not change. Accordingly the variable is expected to keep past and important information without tracing back to the past.

Practical Recurrent Learning (PRL) in the Discrete Time Domain

231

1.0 Input xi(t) Input 0.5 value 0 1.0 Output 0.5 value

50

Times 100

50

Times 100

50

Times 100

Output xj(t)

0 0.2 Value of 0.1 Variable rji(t) variable rji(t) 0

Fig. 2. An example of the variable rji(t) transition. From equation (11), variable rji(t) integrates the value of input xi(t) when the output xj(t) changes, and holds the information of the previous state when the output does not change.

2.2 PRL in the Discrete Time Domain In order to make the analysis of PRL learning easy, PRL learning method in the discrete time domain is introduced here. The method of learning is similar to the conventional Back Propagation method in the meaning that each connection weight are modified according to the product of the propagated error signal δ of the postsynaptic(upper) neuron and the signal that represents the output xi of the presynaptic(lower) neuron. Furthermore, to make the learning process become simple, conventional BP method is used for the learning of the connection weights between the hidden layer and the output layer and PRL learning method is used only between the input layer and the hidden layer. In the output layer, the error signal δj(3) is calculated as δ j (3) =

∂E (t ) ∂S j (3) (t )

=

∂E (t )

⋅

∂x j (3)

∂x j (3) (t ) S j (3)

(

) (

)

= Tr j (t ) − x j (3) (t ) f ' S j (3) (t ) .

(8)

Same as the conventional Back Propagation method, the modification of connection weights are calculated by Δw ji (3) = ηδ j (3) xi ( 2) .

(9)

Each neuron in the hidden layer is trained by PRL and signal δj(2) is calculated as

δ j (2) =

∑δ

( 3) k

⋅ wkj (3) (t ) .

(10)

k

From the equation above, f’(t) is not multiplied as the conventional BP method because f’(t) is included in the variable rji(t) as shown in equation (11). Considering

232

M.F.B. Samsudin, T. Hirose, and K. Shibata

that variable rji(t) does not changed when the output does not changed, and integrates the input’s value when the output changes, it is calculated as

(

)

r ji (t ) = r ji (t − 1)⎛⎜1 − Δx j ( 2) (t ) ⎞⎟ + xi ( 2) (t ) f ' S j ( 2) (t ) Δx j ( 2) (t ) ⎝ ⎠

(11)

where Δx j (t ) = x j (t ) − x j (t − 1) . Then, the modification of each connection weight in the hidden layer is calculated using only the variable rji by Δw ji (t ) ( 2) = ηδ j ( 2) r ji (t ) .

(12)

3 Simulation of EXOR and 3-Bit Parity Problems 3.1 Simulation of EXOR Problem From the previous work[1], it was shown that a sequential NAND problem could be learned by PRL, but a sequential EXOR problem could not be learned. Here, sequential EXOR problem in a fix pattern order was tried to be learned by PRL in the discrete time domain. At first, the sequential EXOR logic function is explained. EXOR problem is a logical operation on two operands that results in a logical value of 1 if and only if exactly one of the operands has a value of 1 and the other has a value of 0. The network architecture used in this paper is the same as shown in Fig. 1 besides it contains 1 output, 20 hidden units and 3 input signals. The input 1 is considered as a signal to distinguish the starting time of a pattern presentation and it is always 1 at t=0. As shown in Table 1, the value of 0 or 1 is entered to the input 2 at t=5 and the input 3 at t=15. At the other times, the signal is always 0. The training signal is given when t=time_lag (from the starting time to the time when the training signal was given) and the time_lag is set to 20 unless mentioned particularly. Parameter setup is shown in Table 2. As shown in Table 2, we used value 4.0 for initial connection weight for self-feedback connection to prevent the propagated error value from diverging or vanishing in BPTT method considering that the maximum derivative of output function is 0.25. All the valuables r are reset to 0 at t=0. Table 1. The timing of inputs and training signal in the learning of one pattern

Time, t Input 1 Input 2 Input 3

0 1 0 0

1~4 0 0 0

5 0 0 or 1 0

6~14 0 0 0

15 0 0 0 or 1

16~time_lag 0 0 0

time_lag Training signal was given

Table 2. Parameter setup

Initial weight value for self-feedback Initial weight value for the other feedback Initial weight value (input layer-hidden layer) Initial weight value (hidden layer-output layer) Termination condition

4.0 0.0 Random number (1.0~1.0) 0.0 30000 iteration(1 pattern for 1 iteration)

Practical Recurrent Learning (PRL) in the Discrete Time Domain

233

3.1.1 Simulation Result Table 3 shows the simulation result when EXOR problems was tried. Successful learning is defined as the state that the difference from the difference from the training signal is less than 0.1 for the last 4 iterations before the end of the learning. From the simulation results, it is shown that sequential EXOR problem as a non linearly-separable problem can be learned by PRL successfully as well as the case of BPTT. Moreover, we recognized that the learning performance for both learning methods has been improved when the learning rate for the feedback connections are smaller than the learning rate for the other connections in the network. Table 3. Simulation result when the learning rates on the network were varied Learning rate 1.0

0.5

0.1

Learning rate for feedback connections 1.0 0.5 0.1 0.05 0.01 0.5 0.1 0.05 0.01 0.1 0.05 0.01

Success Rate PRL (/100times) 1 12 96 100 100 42 94 100 100 94 94 60

Success Rate BPTT (/100times) 6 25 95 100 100 27 92 98 100 84 82 75

In addition, Table 4 summarizes the result of comparison for both methods when we exceeded time_lag to 100, but the timing of inputs is the same as shown in Table.1. In terms of learning ability, the conventional BPTT performs better than PRL even though the time for training by PRL is far smaller than BPTT. Table 4. Simulation result when time lag is exceeded to 100 Time Lag 20 40 60 80 100

Success Rate PRL (/100times) 100 90 79 70 68

Success Rate BPTT (/100times) 100 100 100 100 100

Training time PRL (sec) 4 9 13 17 22

Training time BPTT(sec) 8 19 33 49 69

The sequential EXOR problem as a non linearly-separable problem can be learned successfully to some extent by the PRL in the discrete time domain rather than continuous counterpart. The reason of failure for the learning in the continuous time domain is not clear, but maybe the difficulty of setting the training signal. The value of the training signal was not given at a moment, but a shape of training signal for some duration was given in [1].

234

M.F.B. Samsudin, T. Hirose, and K. Shibata

3.2 Simulation of 3-Bit Parity Problems This section presents the learning performances of the PRL in comparison to the BPTT in a sequential 3-bit parity problem in the random pattern order. In the 3-bit parity problem, the training signal is -0.4 when the number of signals whose value is 1 in 3 given inputs except for the input 1 is even, and the training signal is 0.4 when the number is odd. It is considered that the task is more difficult than EXOR because the number of inputs is larger and it might be difficult for the variable rji to keep the past information. We used the same network architecture (refer to Fig.1), but there are 4 input signals that are 1 input as a starter signal and 3 inputs that is used to calculate the parity signal. The input 1 is entered when t=0 and the value is always 1. Time_lag is set to 20 and the input 2, 3 and 4 are set to enter at t=5, 10 and 15 respectively and the value is chosen randomly from 0 and 1. Parameter setup was the same as in the previous section, but the termination condition is that the state with the squared error is less than 10-3 continues for 100 pattern of learning. Furthermore, the random pattern order is employed here to eliminate the possibility of memorizing the pattern order during the learning 3.2.1 Simulation Results The result of simulation for the 3-bit parity problem in random pattern order is shown in Table 5. Even though no traceback is done in PRL learning, this 3-bit parity problem is learned by PRL to some extent. However, the BPTT outperforms PRL for success rates and average number of iterations. Table 5. The comparison result of learning success rate and average number of iterations

Learning rate

1

Learning rate for feedback connections 0.001 0.003 0.01 0.03 0.1

Random pattern order Learning success rate (/100times) PRL BPTT 75 100 81 100 62 100 26 96 30 1

Average number of iterations PRL 133003 119909 107270 119020 108248

BPTT 17494 11393 7929 7710 6033

Here, we focused on the big difference on the average number of iterations to find the reason of inferiority. Firstly, the transition of the output neuron’s output just after the learning process begins is observed as shown in Fig. 3 for both methods. In order to make a comparison, the initial values of connection weights and pattern order are same for the both methods. Fig.3 shows that there is a big difference in the transition of output when the value of input 2 is 1 between both methods. The output seemed to increase drastically due to the presenting of the input 2 in the case of BPTT, but in the case of PRL, little change of the output is seen despite the presence of input 2. For example, the transition of the output in the case of pattern P5 and pattern P4 where a circle is put in Fig. 3 show the difference between both methods.

Practical Recurrent Learning (PRL) in the Discrete Time Domain

Time

0

21

42

63

84

105

126

147

168

189

235

210

0.1

Output

0.05

(BPTT)

0

(PRL) -0.05

-0.1

Pattern

P6 (1,1,0)

P3 P6 (0,1,1) (1,1,0)

P5 P4 (1,0,1) (1,0,0)

P5 P5 P1 P6 P3 (1,0,1) (1,0,1) (0,0,1) (1,1,0) (0,1,1)

Fig. 3. The transition of output in PRL and BPTT methods. The horizontal axis indicates time and the vertical axis indicates output’s value. P3 indicates Pattern 3 for example.

Fig. 4 shows the connection weights from hidden neurons to the output neuron and the output of each hidden neuron when input 2 is 1 at t=5 in the case of pattern P5. The initial value of connection weights from hidden neuron to output is set to 0. As shown in Fig.4, the sign of the connection weight between the hidden 20 and the output neuron is positive in the case of PRL while it is negative in the case of BPTT. As a result, the output is almost the same in the case of PRL while increases in BPTT. 0.4

0.4 PRL

Hidden neuron’s output

PRL 0.3

Connection Weight

BPTT 0.2 0.1 0 -0.1 -0.2 -0.3 -0.4

0.3

BPTT

0.2 0.1 0 -0.1 -0.2 -0.3 -0.4

0

5

10

15

20

Number of hidden neurons

0

5

10

15

20

Number of hidden neurons

Fig. 4. Connection weight from the hidden layer to the output layer and the output of hidden neurons when t=5

Then, the change of the connection weight from input 1, 2, 3 and 4 to hidden 20 is observed. As shown in Fig. 5, it is noticed that the transition of connection weights from input 1, 2, 3 and 4 to hidden 20 in the case of PRL is far smaller compared to the case of BPTT. Considering that the sign of the connection weight from hidden 20 to output are opposite between both methods, it is not a problem that the change of connection weights from the input 1 to the hidden 20 in PRL is also going to the opposite direction of the case in BPTT.

236

M.F.B. Samsudin, T. Hirose, and K. Shibata

Then, the transition of variable r in one cycle for the pattern P5 is shown in Fig. 6. The values are so small, but as expected, they changed at the time when the corresponding input is 1. Even though the values decreased a little bit when another input exists, they keep the information until the end phase of the learning process. In order to compensate the small variable r and to promote the change of the connection weights from input 1, 2, 3, and 4 to hidden neurons, the learning rate for the connection is raised up. The result of simulation is shown in Table 6. -0.4

-0.5

-0.45

Connection Weight

-0.55

Connection Weight

Input2 - Hidden20

Input1-Hidden20

-0.6

BPTT

BPTT

-0.5

-0.55

-0.65

-0.7

PRL

-0.75

PRL

-0.6

-0.65

-0.7

-0.8 0

100

200

300

400

500

0

600

100

200

300

400

500

Iteration 1.1

1.1

Input3 – Hidden20

1.05

Input4 – Hidden20

1.05

1

Connection Weight

Connection Weight

600

Iteration

BPTT

0.95

0.9

PRL

0.85

1

0.95

BPTT 0.9

0.85

PRL 0.8

0.8

0

100

200

300

400

500

600

0

100

200

300

Iteration

400

500

600

Iteration

Fig. 5. The change of the connection weight from input 1, 2, 3, and 4 to hidden20 for both methods at the early phase of learning process

0.06

r20,4

Value of variable r

0.05 0.04

r20,1

0.03

r20,2

0.02 0.01

r20,3 0

Time

-0.01 0

5

10

15

20

Fig. 6. The value of variable r from input 1, 2, 3, and 4 to hidden 20 in the case of P5

Practical Recurrent Learning (PRL) in the Discrete Time Domain

237

Table 6. The comparison result of learning success rate and average number of iteration Learning rate

1

Learning rate between input 1, 2, 3, 4 to hidden neurons 3

Learning rate for feedback connections

0.001 0.003 0.01 0.03 0.1

Learning success rate (/100times) PRL BPTT 53 59 58 47 50

100 100 100 100 6

Average number of iterations PRL BPTT 110107 96847 83112 86625 87693

22530 12943 7360 5720 4670

As shown in Table 6, even though the learning rate of input 1, 2, 3, and 4 to hidden neurons is set to be higher, BPTT still outperforms PRL in the viewpoints of both success rates and average number of iterations. Table 6 shows the characteristic of BPTT where the learning will become more successful when the learning rate is set to be small. However, the PRL does not have the same characteristic as BPTT because the learning success rate for PRL does not depend on the learning rate for the feedback connections. More experiments and analysis is required to examine whether the learning performance of the practical recurrent learning can be improved or not.

4 Conclusion By formulating PRL learning method in the discrete time domain, it could be shown that sequential EXOR problem and 3-bit parity problem as non linearly-separable problem could be learned by PRL even thought PRL is practical as opposed to BPTT and RTRL with respect O(n2) of memory and O(n2) of computational cost. However the learning performance of PRL is inferior to BPTT. A big difference is seen in the weight transition between PRL and BPTT even though the variable r keeps the past information as expected. More additional analysis and experiment is needed to develop and improve the performance of this learning method.

Acknowledgment A part of this research was supported by JSPS Grant in-Aid for Scientific Research #15300064 and #19300070.

References [1] Shibata, K., Okabe, Y., Ito, K.: Simple Learning Algorithm for Recurrent Networks to Realize Short-Term Memories. In: Proc. of IJCNN(Int’l Joint Conf. on Neural Network) 1998, pp. 2367–2372 (1998) [2] Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning internal representations by errorpropagating. Parallel Distributed Processing 1, 318–362 (1986) [3] Williams, R.J., Zipser, D.: A learning algorithm for continually running fully recurrent neural network. Neural Computation 1, 270–280 (1989) [4] Hochreiter, S., Schimidhuber, J.: Long Short Term Memory. Neural Computation 9, 1735– 1780 (1997)

Learning of Bayesian Discriminant Functions by a Layered Neural Network Yoshifusa Ito1 , Cidambi Srinivasan2 , and Hiroyuki Izumi1 1

2

Department of Policy Science, Aichi-Gakuin University Nisshin, Aichi-ken, 470-0195 Japan {ito,hizumi}@psis.aichi-gakuin.ac.jp Department of Statistics, University of Kentucky Lexington, Kentucky 40506, USA [email protected]

Abstract. Learning of Bayesian discriminant functions is a difficult task for ordinary one-hidden-layer neural networks, because the teacher signals are dichotomic random samples. When the neural network is trained, the parameters, the weights and thresholds, are usually all supposed to be optimized. However, those included in the activation functions of the hidden-layer units are optimized at the second step of the BP learning. We often experience difficulty in training such ’inner’ parameters when teacher signals are dichotomic. To overcome this difficulty, we construct one-hidden-layer neural networks with a smaller number of the inner parameters to be optimized, fixing some components of the parameters. This inevitably causes increment of the hidden-layer units, but the network learns the Bayesian discriminant function better than ordinary neural networks. Keywords: Bayesian, layered neural network, learning, quadratic form.

1

Introduction

We construct one-hidden-layer neural networks individually having a smaller number of inner parameters to be optimized. The goal of this paper is to show that the neural networks can learn the Bayesian discriminant function better than ordinary neural networks. The inner parameters are tentatively those included in the activation functions of the hidden-layer units, the connection weights between the input and hidden units and the thresholds. We use an approximation theorem in [3] to construct the network. Generally, learning of a conditional expectation from dichotomic random samples is a difficult task for neural networks. The posterior probability, which is used as a Bayesian discriminant function, is an example of such conditional expectations. Our neural network is to overcome the difficulties in learning with dichotomic random samples. In the case of approximation of a deterministic function, say f , the teacher signals are pairs (x, f (x)). Hence, the network can approximate the target function by straightly minimizing the deviations of its respective outputs F (x) from the sample signals f (x). However, in the case of learning the conditional expectation, the teacher signals are (x, ξ(x, θ)) and the target function is the conditional M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 238–247, 2008. c Springer-Verlag Berlin Heidelberg 2008 

Learning of Bayesian Discriminant Functions by a Layered Neural Network

239

expectation E[ξ(x, ·)|x], where ξ is a dichotomic random variable and θ stands for the category. Accordingly, the network has to calculate the local expectations E[ξ(x, ·)|x] and hence the learning is difficult. If the network tries to reduce the deviation of its output F from ξ each time it receives a pair (x, ξ(x, θ)), the output does not converge. There have been many theoretical considerations on learning of the conditional expectation [2], [4], [5], [8-10]. But works which have treated the simulations are rare. We have been trying to construct neural networks which can learn the Bayesian discriminant functions [4-7]. However, only when the state-conditional probability distributions were simple, the learning was successful. We have finally learned that it is almost impossible for the ordinary one-hidden-layer neural networks to learn higher dimensional Bayesian discriminant functions. Hence, we now think that it is unavoidable to simplify the parameter space of the network in order that the network can overcome the difficulties in learning with dichotomic random teacher signals. In this paper, we simplify the parameter space by decreasing the number of the inner parameters to be optimized, though this causes increment of hiddenlayer units. It is a general belief that the smaller the number of hidden layer units is, the better training of a neural network proceeds. Our method contradicts to this, because the problems in learning are untangled by increasing the number of the hidden-layer units. We illustrate a few simulation results.

2

Preliminaries

In this paper we treat the two-category, normal-distribution case similarly to [2]. The categories are denoted by θ1 and θ2 respectively and we set Θ = {θ1 , θ2 }. Let Rd be the d-dimensional Euclidean space (R = R1 ) and let x ∈ Rd be the patterns to be classified. Denote by N (μi , Σi ), i = 1, 2, the normal distributions, where μi and Σi are the mean vectors and the covariance matrices. They are the distributions of the patterns x from the respective categories. The stateconditional probability density functions are t −1 1 1 √ e− 2 (x−μi ) Σi (x−μi ) , 2πΣi

i = 1, 2.

(1)

We suppose that the covariance matrices are not degenerate and, hence, Σi as well as Σi−1 are positive definite. Let P (θ1 ) and P (θ2 ) be the prior probabilities and let P (θ1 |x) and P (θ2 |x) be the posterior probabilities of the patterns from the respective categories. Then, by the Bayesian relation, we have P (θ1 |x) P (θ1 )p(x|θ1 ) = . P (θ2 |x) P (θ2 )p(x|θ2 )

(2)

This ratio can be used as a Bayesian discriminant function [1]. The logistic function σ is defined by σ(t) = (1 + e−t )−1 . Since this is a strictly monotone increasing function, the logistic transform of the logarithm of (2):

240

Y. Ito, C. Srinivasan, and H. Izumi

σ(log

P (θ1 |x) ) = P (θ1 |x) P (θ2 |x)

(3)

is also a Bayesian discriminant function [1]. In the case where the state-conditional probability distributions are normal, the log ratio of the posterior probabilities is a quadratic form: g1 (x) = log

P (θ1 |x) 1 = − {(x − μ1 )t Σ1−1 (x − μ1 ) − (x − μ2 )t Σ2−1 (x − μ2 )} P (θ2 |x) 2 + log

P (θ1 ) 1 |Σ1 | − log . P (θ2 ) 2 |Σ2 |

(4)

The activation function of the output unit of our neural network is the logistic function and the network is trained so as to approximate (3). Hence, the inner potential of the output unit approximates the quadratic form (4). The Mahalanobis distance with respect to the normal distribution N (μi , Σi ) is defined by di (x − y) = (x − y)t Σi−1 (x − y), i = 1, 2. Hence, 1 g2 (x) = − {(x − μ1 )t Σ1−1 (x − μ1 ) − (x − μ2 )t Σ2−1 (x − μ2 )} 2

(5)

can be a Mahalanobis discriminant function. This can be obtained by removing a constant P (θ1 ) 1 |Σ1 | log − log (6) P (θ2 ) 2 |Σ2 | from (4) as stated in [7].

3

Difficulty of Learning Posterior Probabilities

A one-hidden-layer neural network outputs a linear sum of the form n 

ai (wi · x + ti ) + a0 ,

i=1

where a0 , ai and ti are constants and wi = (wi1 , · · · , wid ) are vectors. Since these constants are to be modified while training, we call them parameters. Among them we call wi1 , · · · , wid and ti inner parameters and ai and a0 outer parameters. We sometimes identify an approximation formula with the neural network based on this approximation formula. The network above has n(d+2)+1 constants to be optimized. Among them n(d + 1) are inner parameters, and n + 1 are outer parameters. Figure 1 is to show the difficulty in learning with dichotomic random sample. The space of patterns is one-dimensional.

Learning of Bayesian Discriminant Functions by a Layered Neural Network

241

The dots in Figure 1 are random samples from {(x, ξ(x, θ))}, x ∈ R, and θ ∈ Θ = {θ1 , θ2 }, where ξ(x, θ1 ) = 1 and ξ(x, θ2 ) = 0. The probability of the event, ξ(x, θ) = 1, is P (θ1 |x) = E[ξ(x, θ1 )|x]. For simple figurative representation, it is based on one-dimensional normal distributions. The smooth curve is the posterior probability, which the neural network has Fig. 1. Dichotomic samples and the to approximate. posterior probability. The abscissa is In the higher dimensional case, the the x-axis. network has to approximate a (hyper) surface. If the teacher signals are {(x, P (θ1 |x))}, P (θ1 |x) = E[ξ(x, θ)|x], then the learning is an ordinary function approximation by a neural network. However, the teacher signals are {(x, ξ(x, θ))} for our neural network. Comparing the two sets of teacher signals, one may understand that approximation of P (θ1 |x), using the random samples from {(x, ξ(x, θ))}, is more difficult. For approximation of a quadratic formula (4), Funahashi [2] has proposed an approximation formula 2d 

ai g(wi · x + ti ) + a0 .

(7)

i=1

Since d is the dimension of the space of the pattern x, this approximation formula has 2d(d + 2) + 1 constants to be optimized. Among them, 2d(d + 1) constants are inner constants. Actually an approximation formula d 

ai g(wi · x) + a0

(8)

i=0

having a smaller number of the activation functions can approximate the quadratic form [3], [5]. In this formula, the number of the constants to be optimized is (d + 1)2 + 1 and that of the inner constants is d(d + 1). Since one of the activation function is to approximate a linear form in x, the approximation formula (8) can be replaced by d 

ai g(wi · x) + w0 · x + a0 .

(9)

i=1

The values of the constants are of course distinct in (8) and (9), though the same notation is used. In (9), the numbers of the activation functions and the parameters are respectively decreased to d and (d + 1)2 . Hence, we used (9) in [6] and [7], expecting that the parameter space was simplified remarkably. Nevertheless, the network based on (9) was successful only in the case where the probability distributions are simple. In particular, training of the inner parameters was very difficult. This convinced us that to overcome the difficulties there is no other way other than decreasing the inner parameters.

242

4

Y. Ito, C. Srinivasan, and H. Izumi

Modified One-Hidden-Layer Neural Network

In this section we construct a neural network having more hidden-layer units but less parameters to be optimized. There are unit vectors uk , k = 1, · · ·, 1 d 2 d(d + 1), in R such that (uk · x), k = 1, · · · , d, are linearly independent and 2 (uk · x) , k = 1, · · · , 12 d(d + 1), are also linearly independent. Here, we illustrate an example. Suppose that uk , k = 1, · · ·, 12 d(d + 1), are distinct unit vectors in Rd such that, for k = 1, · · · , d, the d-th element is 1 and others are 0, and, for k = d + 1, · · ·, 12 d(d + 1), two elements are equal to √12 and others are 0. Then, they satisfy the condition [3]. Hence, any quadratic form Q(x) in Rd can be expressed as 1 2 d(d+1)

Q(x) =



a2k (uk · x)2 +

k=1

d 

a1k (uk · x) + a0 .

k=1

Though this expression can be generalized to polynomials of higher degree [3], this is enough in this paper. Let u be a unit vector in Rd . For a probability measure μ in Rd , we denote by μu its projection onto a line {tu : −∞ < t < ∞}. Then, we have a theorem below. Theorem 1. Let μ be a probability measure on Rd and let g ∈ C N (R) (N ≥ 2) such that g (i) (0) = 0 (1 ≤ i ≤ 2). Suppose that there exists a function g0 on R such that , for any δ (|δ| < 1), |g (i) (δt)ti | < g0 (t),

0 ≤ i ≤ 2,

and, for any unit vector u in Rd , g0 ∈ Lp (R, μu )(1 ≤ p ≤ ∞). Then, for any quadratic form Q and any ε > 0, there exist coefficients aik and constants δi (0 ≤ i ≤ 2, 1 ≤ k ≤ 12 d(d + 1)) for which 

∂α ∂α ¯ Q − Q p d < ε, ∂xα ∂xα L (R ,μ)

|α| ≤ N,

where 1 2 d(d+1)

¯ Q(x) =



k=1

a2k g(δ2 uk · x) +

d 

a1k g(δ1 uk · x) + a0 .

(10)

k=1

and uk are unit vectors satisfying the condition above. This theorem is a special case of a theorem in [3]. Though we have not enough space to prove this, it may be imaginable that (10) comes from the expression above of the quadratic form Q. The condition on g and μ is to suppress the Lp -norm in a neighborhood of infinity. If the activation function and its derivatives are bounded and the probability measure is rapidly decreasing, this condition is always satisfied.

Learning of Bayesian Discriminant Functions by a Layered Neural Network

243

Since the second sum in (10) is to approximate a linear form, the approximation formula can be modified to 

d(d+1)/2

¯ Q(x) =

a2k g(δuk · x) +

k=1

d 

a1k xk + a0 .

(11)

k=1

When δ is infinitesimal, (11) can coincide with the quadratic form Q. This approximation formula can be realized by a one-hidden-layer neural network having direct connections between the input and output layers. Figure 2 illustrates such a neural network. Since the vectors uk are fixed beforehand, the number of the parameters to be optimized in (11) is 12 (d2 + 3d + 4). Moreover, only one is an inner parameter among them and others are outer parameters. This must make the parameter space very simple. We performed simulations to confirm usefulness of the network. The network has 12 d(d+1) hidden-layer d units. The linear sum k=1 a1k xk is input to the output unit via the direct connections. Hence, the inner potential of the output unit of the neural network illustrated in Figure 2 can approximate (10). Fig. 2. A one-hidden-layer neural net- If the activation function of the output work having direct connections be- unit is the logistic function, then, by (3) tween the input layer and output unit. and (4), it can approximate the posterior probability P (θ1 |x) [2]. D = 12 d(d + 1).

5

Training of the Neural Network

Let F (x, w) be the output of the neural network. Training of the network is based on the proposition below by Ruck et al. [9]. As we have repeatedly used this proposition [4-7], we omit details here. Proposition 2. Set  E(w) =

2 

(F (x, w) − ξ(x, θi ))2 P (θi )p(x|θi )dx.

(12)

Rd i=1

Then,  E(w) =

 (F (x, w) − E[ξ(x, ·)|x]) p(x)dx +

V [ξ(x, ·)|x]p(x)dx.

2

Rd

(13)

Rd

If ξ(x, θ1 ) = 1 and ξ(x, θ2 ) = 0, then E[ξ(x, ·)|x] = P (θ1 |x). The integrand of the second integral is the conditional variance of ξ and independent of the weight

244

Y. Ito, C. Srinivasan, and H. Izumi

vector w. Hence, when E(w) is minimized, the output F (x, w) is expected to approximate P (θ1 |x) if the network has a capability of approximating the posterior probability. Accordingly, learning of the network is carried out by minimizing 1 En (w) = (F (x(k) , w) − ξ(x(k) , θ(k) ))2 , n n

(14)

k=1

where {(x(k) , ξ(x(k) , θ(k) ))}nk=1 , (x(k) , θ(k) ) ⊂ Rd × Θ, is the training set. This method of training has been treated by many authors [2,4-10]. In the case of the ordinary approximation of functions, to be minimized is 1 (F (x(k) , w) − f (x(k) ))2 . n n

En (w) =

(15)

k=1

It is obvious that minimization of (14) is more difficult than that of (15).

6

Simulations

Unlike ordinary one-hidden-layer neural networks, the network based on (11) can learn two-dimensional Bayesian discriminant functions. We have confirmed this by simulation. Fig. 3a and 3b illustrate the probability density functions of the normal distributions N (μ1 , Σ1 ) and N (μ in one of the simulations,  2 , Σ2 ) used  21 10 where μ1 = (1, 0), μ2 = (0, 0) and Σ1 = , Σ2 = respectively. These 11 01 are the state-conditional probability density functions of the categories θ1 and θ2 respectively. Fig. 3c illustrates the posterior probability of the category θ1 in the case of the prior probabilities P (θ1 ) = 0.7 and P (θ2 ) = 0.3. As stated in Section 2, this can be used as a Bayesian discriminant function. The neural network was trained with 1000 teacher signals.

a

b

c

Fig. 3. a and b are the state-conditional probability density functions of Categories θ1 and θ2 respectively, and c is the posterior probability

Fig 4 illustrates an example of the learning process. Fig 4a is the output of the network with the initial values of the parameters. It converged to Fig 4c via Fig 4b. The unit vectors are u1 = (1, 0), u2 = (0, 1) and u3 = ( √12 , √12 ). While learning, the parameters a0 , a11 , a12 , a21 , a22 , a23 were optimized, but the value

Learning of Bayesian Discriminant Functions by a Layered Neural Network

a

b

245

c

Fig. 4. Learning process. The target is the surface illustrated in Fig. 3c.

of the constant δ did not change much. This probably implies difficulty of training of the inner parameter. Even this neural network is not always successful in learning: its success depends on the choice of the initial values of the parameters. In particular, if the initial output of the network is a flat surface, learning rarely proceeds. One may recognize slight differences between the Bayesian discriminant function theoretically obtained (Fig. 3c) and that obtained by simulation (Fig. 4c). These are probably because the teacher signals are random variables and the inner parameter δ is not infinitesimal. Classification capabilities of the two discriminant functions (Fig. 3c and Fig. 4c) are compared in Table 1. By the method stated in [7], we can easily convert the Bayesian discriminant function (4) to the Mahalanobis discriminant function (5), because both are quadratic forms distinct from each other by a constant. Hence, in the table the capabilities of the Mahalanobis discriminant function theoretically obtained and that obtained by simulation are also compared. The test signals are distinct from those used for training. In this table, the numbers in the row of “Alloc. to θ1 (resp. θ2 )” are those of patterns allocated to the category θ1 (resp. category θ2 ), and in the row of “Correc. Alloc.” is the total number of correctly allocated patterns. The columns TS, BT, BS, MT and MS stand for the test signals (TS), the Bayesian discriminant functions theoretically obtained (BT, Fig. 3c) and obtained by simulation (BS, Fig. 4c), and the Mahalanobis discriminant functions theoretically obtained (MT) and obtained by simulation (MS) respectively. Only in the column TS, all signals are correctly allocated. The BT allocated 843 (resp. 157) patterns to Category θ1 (resp. θ2 ), among which 175 (resp. 37) patters were to be allocated to

Table 1. Classification results in the first example (See Text) TS

BT

BS

MT

MS

Alloc. to θ1

705

843(175)

861(187)

614 (87)

741(109)

Alloc. to θ2

295

157 (37)

139 (31)

386(178)

259(150)

Correc. Alloc.

1000

788

782

735

741

246

Y. Ito, C. Srinivasan, and H. Izumi Table 2. Classification results in the second example (See Text) TS

BT

BS

MT

MS

Alloc. to θ1

708

717 (32)

706 (26)

687 (17)

664 (11)

Alloc. to θ2

292

283 (23)

294 (28)

313 (38)

336 (55)

Correct Alloc.

1000

945

946

945

934

Category θ2 (resp. θ1 ). Since the distributions of the two categories are overlapping, even the BT can allocate correctly only 1000 − (175 + 37) = 788 patterns among 1000. However, the allocations by the BT and BS coincide at 982 patterns among 1000 (not shown in Table 2), and those by MT and MS at 924. These implies that the learning is successful. Another isillustrated  example   in Table 2, where μ1 = (1, 1), μ2 = (−1, −2) and 21 1 0.5 Σ1 = , Σ2 = . In this case the two probability distributions 11 0.5 1 are rather separated. Hence, the classifications by the respective discriminant functions are more successful. Moreover, the allocations by BT and BS coincide at 989 patterns among 1000, and those by MT and MS at 977.

7

Conclusion and Discussions

We have often experienced that the ordinary layered neural network based on the approximation formula (9) fails in approximating the posterior probability, unless the probability distribution is very simple. In the case where the dimension of the patterns x is higher than or equal to 2, it looks impossible for the network to learn the posterior probability. However, the neural network based on (11) performs the task better even in the two dimensional case. We think that this is because the latter has a smaller number of parameters to be optimized and, more importantly, they are mostly outer parameters. Our network has more hidden layer units than the minimum requirement. This contradicts to the general belief that a neural network having a smaller number of units works better. However, the number of the parameters to be trained and whether they are inner or outer may be more directly related to the complexity in learning. Since our network has less parameters to be trained and most of them are outer parameters, it is understandable that the network works better. When the network is based on (11) the error function (12) has no global minimum usually, because the output F (x, w) converges to the posterior probability P (θ1 |x) as the scaling parameter δ tends to infinitesimal. Hence, we expected that δ converges toward zero while training, but it usually stays in the neighborhood of the initial value. The network often realizes the approximation using the initially given value of δ with a slight modification. This probably implies difficulty of training of inner constants with random dichotomic teacher signals. This may in turn justify our policy of decreasing the inner parameters to be

Learning of Bayesian Discriminant Functions by a Layered Neural Network

247

trained, even if it causes increment of the number of the hidden-layer units. In short, the inner parameters are converted to the outer parameters in our neural network. The flexibility of the basis functions may characterize the neural network. It makes neural networks useful powerful tools for multiple purposes. One of the standard methods of approximating functions is to use the basis functions of a complete orthonormal system. In this case only the coefficients of the basis functions are optimized. Approximation by our neural network is an intermediate method between the two methods because the basis functions are flexible only to some extent. Since the space of target functions is restricted to quadratic forms, the flexibility of basis functions can be restricted in this paper. However, usefulness of the neural network is not limited to this special case. The approximation based on Theorem 1 is on the whole space in the Lp sense and the original theorem in [3] is on approximation of polynomials of arbitrary degree. Hence, by the polynomial approximation theorem, our neural network may be useful in approximating general continuous functions when the probability measure is rapidly decreasing. We think that the ideas of making easier the learning by converting inner parameters to outer parameters can be widely applied.

References 1. Duda, R.O., Hart, P.E.: Pattern classification and scene analysis. John Wiley & Sons, New York (1973) 2. Funahashi, K.: Multilayer neural networks and Bayes decision theory. Neural Networks 11, 209–213 (1998) 3. Ito, Y.: Simultaneous approximations of polynomials and derivatives and their applications to neural networks (submitted) 4. Ito, Y., Srinivasan, C.: Multicategory Bayesian decision using a three-layer neural network. In: Kaynak, O., Alpaydın, E., Oja, E., Xu, L. (eds.) ICANN 2003 and ICONIP 2003. LNCS, vol. 2714, pp. 253–261. Springer, Heidelberg (2003) 5. Ito, Y., Srinivasan, C.: Bayesian decision theory on three-layer neural networks. Neurocomputing 63, 209–228 (2005) 6. Ito, Y., Srinivasan, C., Izumi, H.: Bayesian learning of neural networks adapted to changes of prior probabilities. In: Duch, W., Kacprzyk, J., Oja, E., Zadro˙zny, S. (eds.) ICANN 2005. LNCS, vol. 3697, pp. 253–259. Springer, Heidelberg (2005) 7. Ito, Y., Srinivasan, C., Izumi, H.: Discriminant analysis by a neural network with Mahalanobis distance. In: Kollias, S., Stafylopatis, A., Duch, W., Oja, E. (eds.) ICANN 2006. LNCS, vol. 4132, pp. 350–360. Springer, Heidelberg (2006) 8. Richard, M.D., Lipmann, R.P.: Neural network classifiers estimate Bayesian a posteriori probabilities. Neural Computation 3, 461–483 (1991) 9. Ruck, M.D., Rogers, S., Kabrisky, M., Oxley, H., Sutter, B.: The multilayer perceptron as approximator to a Bayes optimal discriminant function. IEEE Transactions on Neural Networks 1, 296–298 (1990) 10. White, H.: Learning in artificial neural networks: A statistical perspective. Neural Computation 1, 425–464 (1989)

RNN with a Recurrent Output Layer for Learning of Naturalness J´ an Dolinsk´ y and Hideyuki Takagi Kyushu University, 4-9-1 Shiobaru, Minami-ku, Fukuoka, 815-8540 Japan [email protected], [email protected]

Abstract. The behavior of recurrent neural networks with a recurrent output layer (ROL) is described mathematically and it is shown that using ROL is not only advantageous, but is in fact crucial to obtaining satisfactory performance for the proposed naturalness learning. Conventional belief holds that employing ROL often substantially decreases the performance of a network or renders the network unstable, and ROL is consequently rarely used. The objective of this paper is to demonstrate that there are cases where it is necessary to use ROL. The concrete example shown models naturalness in handwritten letters.

1

Introduction

In engineering, recurrent neural networks (RNN) have not been often proposed as a promising solution. The difficulties with training a RNN have been overcome, and recent theoretical advances in the field have made training a RNN quicker and easier [4]. Recurrent connections have not been found to increase the approximational capabilities of the network [7]. Nevertheless, we may obtain decreased complexity, network size, etc. while solving the same problem. In some applications - such as speech recognition or object detection or prediction - our classification / prediction at time t should be more accurate if we can account for what we saw at earlier times. The most common approach to model such systems is to use a suitably powerful feed-forward network and feed it with a finite history of the inputs and optionally the outputs through a sliding window. Early attempts to improve this, often tedious, technique resulted in various network architectures based on the feed-forward topology with the recurrent connections being set to fixed values to ensure that the backpropagation rule can be used [1][6]. Many works have been done on autonomous Hopfield networks as well as on training algorithms that can be applied to a RNN in a feed-forward fashion (i.e. BPTT). Recent theoretical advances in RNN research such as Echo State Networks (ESN) afford the modeling of fully general topologies, which were difficult to train directly with the former techniques. An interesting example is a topology where output units have connections from not only the internal units but also the input units and output units, yielding a recurrent output layer - ROL. Although connections from the input units are often used, connections from the output layer are rare. In the following M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 248–257, 2008. c Springer-Verlag Berlin Heidelberg 2008 

RNN with a Recurrent Output Layer for Learning of Naturalness

249

chapters, we explain what behavior ROL implies, introduce the concept of our proposed naturalness learning, and show that using ROL not only increases the performance but is actually an intrinsic part of modeling with the proposed naturalness learning. One of the earliest RNN, where the output activation values from the previous step were used to compute the output activation values in the next step, was the Jordan network [6][5]. In the Jordan network, the activation values of the output units are fed back into the input layer through a set of extra input units called the state units. This type of network is called output-recurrent network. Various modifications to output-recurrent networks have been proposed and successfully used for modeling difficult non-linear tasks [8]. RNN with ROL, in contrast to output-recurrent network, uses the output activation values of the previous step directly to compute the output in the next step. The output activation values of the previous step can be thought of as a extra hidden units. We are aware of no papers discussing applications of RNN with ROL.

2

Dynamics of RNN with a Recurrent Output Layer

Adopting a standard perspective of system theory, we view a deterministic and discrete-time dynamical system as a function G which yields the next system output, given an input and the output history: y(n + 1) = G(..., u(n), u(n + 1); ...y(n − 1), y(n))

(1)

where u(n) is the input vector and y(n) is the output vector for the time step n. The echo-state approach enables us to approximate systems represented by G directly, without the necessity to convert a time series into static input patterns by the sliding window technique [2]. Consider a discrete-time ESN [4] consisting of K input units with an activation vector u(n) = (u1 (n), ..., uK (n))t , N internal units with an activation vector x(n) = (x1 (n), ..., xN (n))t , and L output units with an activation vector y(n) = (y1 (n), ..., yL (n))t , where t denotes the transpose. The corresponding input, internal and output connection weights are collected in the N × K, N × N , L×(K +N +L) weight matrices Win , W, Wout respectively. Optionally, a N ×L matrix Wback may be used to project the output units back to the internal units. The internal units’ activation is computed according to x(n + 1) = f(Win u(n + 1) + Wx(n) + Wback y(n))

(2)

where f denotes the component-wise application of the transfer (activation) function to each internal unit. The output is computed as y(n + 1) = fout (Wout (u(n + 1), x(n + 1), y(n))

(3)

where (u(n + 1), x(n + 1), y(n)) represents the concatenated vector consisting of input, internal and output activation vectors. The concatenated vector often

250

J. Dolinsk´ y and H. Takagi

Fig. 1. Echo-state network: the dotted lines plot connections which can be trained, the gray lines plot connections which are optional

consits only of input and internal activations or internal activation only. Fig. 1 shows the architecture of an ESN. See [4] for further details concerning the training of ESN. A closer look at Eq. (3) reveals that a system output y(n + 1) is constructed from the given input and output history via two distinct mechanisms: from the activation vectors of the internal units x(n) indirectly (by computing x(n + 1) via Eq. (2)) and optionally from the activation vectors of the input units u(n+1) and/or output units y(n) directly. The internal units’ activation x(n+1) is computed using the input and output activation u(n + 1), y(n) and the activation x(n) of the internal units from the previous step which recursively reflects the influence of input and output activations from previous steps. We can therefore rewrite Eq. (2) as x(n + 1) = E(..., u(n), u(n + 1); ..., y(n − 1), y(n))

(4)

where E depends on the history of input signal u and the history of desired output signal y itself, thus in each particular task, E shares certain properties with the desired output and/or given input. How strongly the internal activation x(n+1) is influenced by the activations u(n+1), y(n) and x(n) (which recursively consist of previous input/output activations) is controlled by the size of the weights in matrices Win , Wback and W respectively. Algebraic properties of the matrix W are particularly important for the short-term memory property of an ESN [3]. Besides using the activations of internal units, sometimes it is advantageous to use also the activations of input and output units directly. Although the activation vector x(n) reflects the history of the desired output and/or given input, the activation vectors u(n + 1), y(n) in Eq. 3 are used merely as another form of input. This usage corresponds to connecting the input units to the output units and output units to output units themselves directly. Direct connection of input units to output units is often used whereas direct connection of output

RNN with a Recurrent Output Layer for Learning of Naturalness

251

units to output units is rare. It is the connecting of output units to each other what makes the output layer recurrent. ROL implies a substantial influence of the previously generated output y(n) on the successive output y(n+1). The activation y(n) is only an approximation of a system output at the step n and, thus, it is always generated with a certain error. This error is included in the computation of the successive output activation y(n + 1) and can easily accumulate with each update step. It is for this reason that computation using ROL has been rare.

3

RNN with a Recurrent Output Layer for Naturalness Learning

In this section, we will demonstrate the principles of naturalness learning by showing how to express and model the unique quality of hand-written letters. We also explain why ROL works well with the naturalness learning. The style of writing of any given person is very much individual and can be distinguished easily from mechanically or electronically printed text. Moreover, we can distinguish between the writing of different people. Everybody learns their alphabet in a school, and while writing in one’s own individual way, a person is trying to approximate the shapes of the letters as they learned them in school. We can therefore understand handwriting as turning the basic shape of a letter as learned in a school into the writer’s particular, unique, handwritten form. A human then can be seen as a ‘filter’ which adds his or her own characteristics to the shapes of those basic letters. To explain the term of naturalness, first we must define our terminology. Let us refer to the letters used in textbooks (either printed or cursive) as the font letters (fontL). We view handwriting as the process of turning a fontL into its handwritten form. The unique quality of the handwritten letter (handL) can be then understood as the difference between the handL and the fontL and expressed as a 2-D displacement vector field of evenly spaced points along the strokes of the font and its respective handwritten version (see Fig. 2) 1 . We refer to this difference as naturalness. In other words, adding the naturalness to the fontL will result in a handL of a unique form. We can thus, assume a relation between the fontL and the naturalness. This assumption enables us to model naturalness by a system which employs certain characteristics of the fontL as its input. The task of naturalness learning is to find and learn the relation between font letters and naturalness. Speaking in the terms of naturalness learning, the target system (handwritten letters) resembles the basic system (font letters) with its behavior (shape of handwritten letters) deviating from the basic system to a certain extent. Learning and modeling naturalness using a RNN with ROL produced interesting results. As mentioned in section 2., computing with ROL is prone to 1

A displacement vector is not the only mean of expressing naturalness, it can be expressed using an arbitrary mechanism.

252

J. Dolinsk´ y and H. Takagi

Fig. 2. Hiragana letter /ka/. Naturalness expressed by 2-D displacement vector field. Font letter strokes shown in black, handwritten strokes shown in gray.

the accumulation of error from previous steps. This phenomena has been found harmful in many tasks. With naturalness learning, however, ROL performs well. In the handwriting task, we found that introducing an activation y(n) into play via update Eq. (3) always helped network to generate y(n+1) with a greater accuracy. An intuitive explanation is as follows. The way a person writes the first half of a handwritten stroke influences, to a certain extent, how the second part is going to look, i.e. distortion of a certain part of a stroke usually implies some other distortion to a successive part of the stroke. It is therefore reasonable to assume that a short sequence of points on a handwritten stroke influences where the next point will appear, with the very last point of a sequence having the greatest influence. The same holds for the naturalness extracted as a difference between handL and fontL. On the other hand, it is not only the very last point which influences the position of the next point of a stroke, but a short sequence of the previous points. Thus, a backprojection matrix Wback was used so as to ensure recent short sequence of generated output ..., y(n − 1), y(n) is also reflected in y(n + 1) via the activation x(n + 1) (see Eq. (2) and Eq. (3)). The same principle holds for the input activations u(n + 1) extracted from fontL strokes. Update Eq. (2) ensures that a short sequence ..., u(n), u(n + 1) is reflected in the activation x(n + 1) which is in turn used to compute y(n + 1) via update Eq. (3). The activation u(n + 1) is also used directly in Eq. (3) to ensure that the very last point of the input sequence has significant influence in the computation of y(n + 1). The fact that naturalness is being modeled, instead of the target system, allows us to control the amount of the naturalness being added to the font letters. A weight of value 1 will render generated letters as close to a person’s handwriting as possible. The value of, say, 0.6 will reduce the naturalness to 60%, providing us with a neat handwritten letters. Values close to 0 will render generated letters very close to the font letters. It is also possible to combine several individual’s naturalness (e.g. 40% of person A’s naturalness with 60% of person B’s naturalness).

RNN with a Recurrent Output Layer for Learning of Naturalness

253

The naturalness learning approach is not limited to the handwriting task only. We believe, one can generate natural looking movements for parts of the human body with the inverse kinematics algorithm being employed as the basic system. Generating emotional human speech with synthesized speech being used as the basic system might be another interesting application of the naturalness learning approach.

4

Experiments

In this chapter, we demonstrate how the naturalness learning approach copes with the handwriting task. The letters used in the experiments are symbols of Japanese syllabary - hiragana. The fontL were extracted from the Bitstream Vera Sans font onto 250x250 pixel canvas. Every stroke of a given letter was turned into a set of bezier curves - a path. Every path was then evenly spaced into a set of points. Let each Pfijl be the set of points represented by nij × 2 matrix where i denotes the index of the letter, j the index of the stroke within the letter and nij is the number of points. Pfijl (k) = (xk , yk ) therefore represents the k-th point of the j-th stroke of the i-th letter (k-th row of the matrix Pfijl ). The handL were first scanned and then appropriately scaled so as to ensure they fit the canvas. Every handL stroke was then aligned to its fontL stroke counterpart. This alignment ensures that the displacement vector field expresses only a shape transformation between a pair of fontL and handL strokes. In order to split strokes of handL, the same procedure was applied as with the fontL, saving the points for each stroke into Phl ij . The spacing interval along each stroke (path) of a handL was, however, also adjusted so that number of points matched those in the corresponding fontL stroke. 4.1

Data Specification

The input signals were extracted from the points of fontL’s strokes as follows. Let Dfijl (k) be the difference vector Pfijl (k + 1) − Pfijl (k). Then the inertia for the k-th point of the j-th stroke of the i-th letter is given by inertiaij (k) = Dfijl (k)

(5)

with each inertiaij (k) being the k-th row of the (nij − 1) × 2 matrix inertiaij . The inertia can be thought of as a representation of the movement of an imaginary pen which ‘wrote’ the font letter. The curvature for the k-th point of the j-th stroke of the i-th letter is given by   0 fl Dij (k) 1 curvij (k) =  (6) Dfijl (k)Dfijl (k)t

254

J. Dolinsk´ y and H. Takagi

with each curvij (k) being the k-th row of the (nij − 1) × 1 matrix curvij . Figure 3 illustrates the geometrical meaning of (5) and (6). Each matrix curvij and inertiaij was merged into a single (nij − 1) × 3 matrix Uij with each column being normalized into the interval (−1, 1). In order to partially erase the transient dynamics from the previous stroke, a zero sequence of size ngap × 3 was inserted before every Uij , resulting in the final input matrix U. Naturalness, which serves as the (2 dimensional) output signal, is represented by 2-D displacement vector field. The 2-D displacement vector field for the j-th stroke of the i-th letter is given by fl Yij = Phl ij − Pij

(7)

fl with Yij , Phl ij and Pij each being nij × 2 matrices. The last row of every Yij was dropped to ensure each pair of Yij andUij have the same length of nij − 1. Each column of Yij was scaled down to the interval (−1, 1). A zero sequence of size ngap × 2 was inserted before every Yij , resulting in the final output matrix Y.

4.2

Network Parameters

A 300 unit network was used with activation function f = RBF . Internal weights in the matrix W were randomly assigned values of 0, 0.2073, -0.2073 with probabilities 0.95, 0.025, 0.025. For a 300 × 300 matrix W, this implies a spectral radius of ≈ 0.85, providing for a relatively long short-term memory [4]. 3 input and 2 output units were attached. Input weights were randomly drawn from a uniform distribution over (−1, 1) with probability 0.9 or set to 0. With such an input matrix, the network is strongly driven by the input activations because many elements of the matrix Win are non-zero values. The network had output feedback connections, which were randomly set to one of the three values of 0, 0.1, -0.1 with probabilities 0.9, 0.05, 0.05. Such a setup of feedback connections makes the network excited only marginally by previous output activations. The activation function for the output units was identity fout (x) = x.

Fig. 3. Geometrical meaning of the input data. inertiaij (k) represents difference vector between two successive points Pijf l (k + 1), Pijf l (k). curvij (k) is the sine of angle φ.

RNN with a Recurrent Output Layer for Learning of Naturalness

255

Fig. 4. Testing with the training data: Letters produced by RNN with/out ROL with teacher-forcing switched off from n = 301

Fig. 5. Testing with the testing data: Letters produced by RNN with/out ROL

4.3

Training and Testing

The training data was made from the letters shown in Fig. 4, resulting in a 3027×3 input matrix for Utrain and a 3027×2 output matrix for Ytrain , which were prepared according to the data specification with ngap = 16 being used. Eq. (2) was used for updating with u(n), y(n) being the transposed n-th rows of matrices Utrain and Ytrain respectively. The first 300 steps were discarded and the network internal states with input unit states x(n), u(n) were collected from n = 301 through n = 3355. The output weights Wout were computed using the internal states and input unit states only. The training errors of first and second output units were msetrain,1 ≈ 9.5 × 10−4 and msetrain,2 ≈ 2.8 × 10−3 respectively. Making the output layer recurrent (ROL) and computing Wout using not only internal/input states x(n), u(n) but also output activation values

256

J. Dolinsk´ y and H. Takagi

y(n−1) reduced the training errors msetrain,1 and msetrain,2 down to ≈ 6×10−4 and ≈ 2.0 × 10−3 respectively. A visual comparison is shown in Fig. (4). The testing data was made from letters shown in Fig. 5, resulting in a 6045×3 input matrix Utest and a 6045×2 output matrix Ytest with ngap = 16 being used. For non-ROL topology the test errors were found to be msetest,1 ≈ 1.2 × 10−2 , msetest,2 ≈ 3.5 × 10−2 . Using ROL reduced the test errors to msetest,1 ≈ 0.9 × 10−3 , msetest,2 ≈ 3.0 × 10−2 . The errors msetest,i provide only a rough indication of network performance. A visual comparison between these two trials is shown in Fig. 5. We can observe that the network also produces appropriate naturalness for letters on which it had not been trained.

5 5.1

Discussion Network

Here we try to provide an insight into why the setup from section 4. works the best. The network has information concerning the shape of the strokes in fontL and handL via its history of input and output activations. The matrices Win and W from the setup in sec. 4. make the network strongly driven by the (finite) history of input. Moreover, using the most recent input activation u(n + 1) in Eq. (3) directly makes the impact even stronger. It is most likely the case that the last several points of the path from where a fontL stroke is coming has a substantial impact on the naturalness and thus also on where the handL stroke is going to continue. An intuitive explanation is that a human, while writing in one’s own individual way, is trying to ‘approximate’ a font letter shape as memorized in a school. This finding is in line with the basic idea of the naturalness learning: to model a target system (handL) by means of a basic system (fontL) and its difference (naturalness) with the target system. The feedback weights Wback from the setup in section 4. only slightly excite the network with the output history. Surprisingly, using the most recent output activation y(n) in Eq. (3) directly (= ROL) always improved the performance. A plausible explanation is that the already written part of a handL stroke influences how the going to be written part will look like with the naturalness of the the previous step having highest relevance to the naturalness generated in the next step. (i.e. distortion of a certain part of a stroke usually implies some other distortion to a successive part of the stroke). Feedback connections with larger weights (up to +1) and no ROL were also tested. With this setup the network training error was about the same as in sec. 4 but driving the network with the testing data rendered worse performance or made the network unstable. To check the robustness of the advantage of ROL, we tested the setup in sec. 4 with a range of different starting internal weights W. In all cases ROL enabled better performance.

RNN with a Recurrent Output Layer for Learning of Naturalness

5.2

257

Data Structure

The naturalness in this paper is represented as a 2-D displacement vector field (Fig. 2). We are, however, not strictly bound to this representation. The naturalness can be also represented as a set of parameters in a system which represents a difference between the target (handL) and the basic (fontL) system. The input data characterizing the basic system (handL) was made position independent so as to ensure the same stroke will be represented by the same data regardless of its starting position. This reduces significantly the complexity of the handwriting task because while separate strokes are substantially different in shape, some of their small parts are often similar. The short-term memory of a RNN makes distinction between a identical/similar short stroke sequences possible because the RNN accounts also for points before the identical/similar stroke part.

6

Conclusion

In the handwriting task we showed that by modeling the target system by means of a basic system and its difference from the target system, a substantial relevance is revealed in the difference produced in step n and step n + 1. In such a case the usage of a ROL turned out to be advantageous. We would like to confirm these findings by applying naturalness learning to other tasks as well. Modeling the unique individualistic quality of human motion is the next step in confirming the feasibility of both naturalness learning and the usability of a ROL for tasks formulated in terms of naturalness learning.

References 1. Elman, J.L.: Finding structure in time. Cognitive Science: A Multidisciplinary Journal 14(2), 179–211 (1990) 2. Jaeger, H.: The “echo state” approach to analysing and training recurrent neural networks. Sankt Augustin: GMD-Forschungszentrum Informationstechnik, GMDReport 148 (December 2001) 3. Jaeger, H.: Short term memory in echo state networks. Sankt Augustin: GMDForschungszentrum Informationstechnik, GMD-Report 152 (March 2002) 4. Jaeger, H.: Supervised training of recurrent neural networks, especially with ESN approach. Sankt Augustin: GMD-Forschungszentrum Informationstechnik, GMDReport 159 (October 2002) 5. Jordan, M.I.: Serial Order: a parallel distributed processing approach. Tech. Rep. 8604, Univ. of California at San Diego, Inst. for Cognitive Science (May 1986) 6. Jordan, M.I.: Attractor dynamics and parallelism in a connectionist sequential machine. In: Eighth Annual Conf. of Cognitive Science Society, Amherst, MA, USA, pp. 531–546 (August 1986) 7. Krose, B., van der Smagt, P.: Recurrent networks. In: Ch. 5, An introduction to neural networks, Eighth Edition, Univ. of Amsterdam (November 1996) 8. Wang, Y.-C., Chien, C.-J., Teng, C.-C.: Direct adaptive iterative learning control of nonlinear systems using an output-recurrent fuzzy neural network. IEEE Trans. on SMC-B 34(3), 1348–1359 (2004)

Using Generalization Error Bounds to Train the Set Covering Machine Zakria Hussain and John Shawe-Taylor Centre for Computational Statistics and Machine Learning Department of Computer Science University College, London {z.hussain,j.shawe-taylor}@cs.ucl.ac.uk

Abstract. In this paper we eliminate the need for parameter estimation associated with the set covering machine (SCM) by directly minimizing generalization error bounds. Firstly, we consider a sub-optimal greedy heuristic algorithm termed the bound set covering machine (BSCM). Next, we propose the branch and bound set covering machine (BBSCM) and prove that it finds a classifier producing the smallest generalization error bound. We further justify empirically the BBSCM algorithm with a heuristic relaxation, called BBSCM(τ ), which guarantees a solution whose bound is within a factor τ of the optimal. Experiments comparing against the support vector machine (SVM) and SCM algorithms demonstrate that the approaches proposed can lead to some or all of the following: 1) faster running times, 2) sparser classifiers and 3) competitive generalization error, all while avoiding the need for parameter estimation.

1

Motivation

Two algorithms that use very different mechanisms in order to build their classifiers are the support vector machine (SVM) [2,6,3] and the set covering machine (SCM) [4]. Taking the binary classification task as an example, the first more familiar algorithm finds a hyperplane in feature space that maximizes the margin between the two classes of examples. The second looks for a classifier constructed from a small subset of data derived decision functions consistent with the set of positive examples. Fundamentally these two algorithms differ in their approaches to solving a classification problem. However, as was argued in [4] it may be more effective (for some learning tasks) to build a classifier from a small subset of data derived decision functions as opposed to a maximum margin separating hyperplane. In this paper, we demonstrate a method of model selection for the SCM without the need for parameter estimation. Instead, we apply the generalization error bound directly to the algorithm in order to determine its output classifier. First we apply the bound directly to the SCM using a greedy approach and call M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 258–268, 2008. c Springer-Verlag Berlin Heidelberg 2008 

Using Generalization Error Bounds to Train the Set Covering Machine

259

the heuristic the bound set covering machine (BSCM). Next we look to globally optimize the bound by proposing the branch and bound set covering machine (BBSCM) and prove that the algorithm will indeed find the hypothesis with the smallest generalization error bound. This algorithm turns out to be too slow experimentally and so we introduce a relaxation called BBSCM(τ ) and prove that its solutions are a factor τ from the optimal. The structure of this paper will be as follows: In the following section we describe the set covering machine (SCM) and state two generalization error bounds for the SCM. Section 3 describes the bound set covering machine (BSCM). In part 4 we discuss in detail the branch and bound set covering machine (BBSCM). Section 5 describes a heuristic derived from the BBSCM algorithm called BBSCM(τ ). Section 6 proves the optimality of the algorithms proposed. The experimental section 7 discusses results for several UCI repository data sets with section 8 concluding the paper.

2

The Set Covering Machine

Definition 1. Let S = {(x1 , y1 ) . . . (xm , ym )} be a sample where xi ∈ X = P ∪ N is a training example and yi ∈ {−1, +1} its classification. We use the convention x = (x1 , . . . , xn ) for each training example and refer to X as the training set. Let P be the set of positive (+1) and N the set of negative (−1) training examples for the conjunction case and let P be the set of negative (−1) and N the set of positive (+1) training examples in the disjunction case. The set covering machine tries to build a conjunction or disjunction of datadependent features (data derived decision functions) in order to classify future test examples. The set of data derived decision functions we will use are the following set of data-dependent balls. Definition 2. For a training example xi ∈ X with label yi ∈ {−1, 1} and (realvalued) radius ρ, let fi,ρ be the following data-dependent ball centered around xi :  yi if d(x, xi ) ≤ ρ fi,ρ (x) = y¯i otherwise where y¯i is the complement of yi and d(x, xi ) is the distance between x and xi . Furthermore, we define a center xi ∈ X and restrict the definition of a border xj ∈ P. Also the radius ρ is defined as:  d(xi , xj ) + α if xi ∈ P ρ= d(xi , xj ) − α if xi ∈ N where α is a small positive real number. For any ball h ∈ H, let ν(h) be the set of N examples correctly classified by h and let π(h) be the set of P examples misclassified by h. Given these definitions we have the following description for the usefulness of a data-dependent ball.

260

Z. Hussain and J. Shawe-Taylor

Definition 3. The usefulness (or utility) U of h is expressed as: U (h) = |ν(h)| − p|π(h)|

(1)

where p is a small positive real number. Finally, when we discuss a P example we will mean an example from the set of P. Similarly, an N example will denote an example from the set of N examples. Using these definitions we can now describe how to train the SCM with datadependent balls. 2.1

Training

The SCM algorithm uses a greedy approach to try and completely classify the set of N examples whilst misclassifying a small number of P examples. Let N be the set of N examples to be covered and let P be the set of P examples that have been misclassified. Therefore, initially hypothesis B ← ∅, N ← N and P ← ∅. At the first iteration the SCM algorithm looks for ball hi that maximizes the utility value U (hi ). After the ball hi with the highest usefulness is added, the hypothesis becomes B ← B ∪ {hi }, N ← N  {ν(hi )} and P ← P ∪ {π(hi )}. This is repeated until no more N examples are left to cover (N = ∅) or until the early stopping criterion |B| ≤ s is satisfied. Clearly the algorithm is greedy as it only adds ball h to hypothesis B if it maximizes the utility U (h). Once the SCM has output its hypothesis B, it can be used to make predictions. 2.2

Testing

Given hypothesis B containing a conjunction or disjunction of data-dependent balls hi , we can classify example x like so:  y if hi (x) = y for i = 1, . . . , |B| B(x) = (2) y¯ otherwise where y¯ is the complement of y and, from Definition 1, y = 1 for a conjunction and y = −1 for a disjunction. 2.3

Generalization Error Bounds

We now state a loss bound for the SCM, that will be used for the theoretical and experimental section of the paper. We also use a second bound but only reference it in order to save space. Theorem 1. [4] Suppose a SCM finds a solution given by a set B of features with R = R(B) = |B| features, R+ = R+ (B) of which are centered around P examples, with kp = kp (B) the number of P training errors and kn = kn (B) the number of N training errors on a sample of size m > 2R + kp + kn . Then with

Using Generalization Error Bounds to Train the Set Covering Machine

261

probability 1 − δ over random draws of training sets, the generalization error  of the resulting classifier can be bounded by       −1 m 2R + (m, R, R , kp , kn ) ≤ 1 − exp ln + ln m − 2R − kp − kn 2R R+    m − 2R 2m2 R + ln + ln (3) kp + kn δ The second bound we will use is the Equation 10 of [5] and is slightly tighter than the above bound. These bounds are tight and non-trivial (i.e., always less than 1). Exploiting this verity we will apply the generalization error bound directly to obtain classifiers for the SCM and, with it, remove the need for parameter estimation in the SCM. Because of its greedy application of the bound, we call this first heuristic the bound set covering machine (BSCM).

3

The Bound Set Covering Machine

In this variant of the SCM we allow the algorithm to be driven by one of the generalization error bounds given by equation (3) or Equation 10 of [5] . For simplicity and to save space we only describe the remaining work with Theorem 1. Although the bound of [5] can also be applied with the same reasoning. Given any hypothesis B and ball h, we can calculate the risk bound of adding ball h to B as (B ∪ {h}) using (m, R, R+ , kp , kn ). This is true for adding any ball h. Therefore, similarly to the SCM, the bound set covering machine (BSCM) algorithm can be described as follows: Initially hypothesis B ← ∅, N ← N , P ← ∅ and best bound ∗ ← 1. At the first iteration, the BSCM algorithm looks for ball hi that minimizes the generalization error bound (B ∪ {hi }). Ball hi that minimizes (B ∪ {hi }) is added to the hypothesis B ← B ∪ {hi }, N ← N  {ν(hi )}, P ← P ∪ {π(hi )} and best bound ∗ ← (B ∪ {hi }). This is repeated until no new loss bound (B ∪ {h}), for any new ball h, can be found such that (B ∪ {h}) < ∗ . Finally the resulting hypothesis B can be used to classify new test examples using (2). Note that the BSCM has eliminated the need for penalty parameter p and soft-stopping parameter s that was present in the SCM. It is clear that the BSCM heuristic is greedy in its solution in much the same way as the SCM, with the difference that it looks to minimize the generalization error bound as opposed to maximizing the utility function for each ball. However, the nature of a greedy approach implies that we are not always guaranteed a globally optimal classifier. Therefore, in the following section, we present a branch and bound approach to solving the set covering machine that ensures a much stronger result, that of global optimality.

4

The Branch and Bound Set Covering Machine

In this section we use a branch and bound approach to enumerate all possible hypotheses that can be generated from the set of data-dependent balls. This is

262

Z. Hussain and J. Shawe-Taylor

done by evaluating the bound every time a new ball can be added to the current hypothesis. If a hypothesis cannot be extended with a ball such that the new bound is smaller than the best bound currently found then there is no need to include this ball in the current hypothesis. Note that the choice of balls can also be thought of in terms of a branch and bound search tree. Where any ball at a particular depth d is found from its parent ball at depth d − 1 and can produce child balls at level d + 1. The motivation for such a strategy for solving the set covering machine is that if the function to be minimized is the generalization error bound then we are assured of finding a hypothesis with the smallest generalization error bound. 4.1

Algorithm

The algorithm relies on functions addball and createtable. We have detailed below each function and also included the pseudo code. Before we describe the functions in detail we will now give some notation. Let S be the sample containing training set X and label set Y , H the set of data-dependent balls and T the machine type which can either be a conjunction or disjunction of balls. As earlier, let π(h) be the set of P examples misclassified by h and ν(h) the set of N examples correctly classified by h. For any hypothesis B and any ball hi ∈ B let Bi = B ∪{hi }. The generalization error bound of B is given by (B) where, (B) = (m, R, R+ , kp , kn ), also for the same hypothesis B the best potential (bp) generalization error bound η(B) is given by, η(B) = (m, R + 1, R+ , kp , 0). So the bp generalization error bound η(B) is the bound (Bi ) for Bi = B ∪{hi } if a single ball hi can be added to hypothesis B such that all of the remaining N examples are covered and none of the remaining P examples misclassified. Contrast this to the generalization error bound (B) which is simply the bound given for hypothesis B. Algorithm BBSCM: The input of the algorithm is sample S that contains training set X = P ∪ N , machine type T that is either a ‘conjunction’ or ‘disjunction’ and H that is the set of data-dependent balls computed from training set X. The algorithm contains local variables B,N ,P ,∗ and global variable . Initially B ← ∅ is the empty hypothesis that we start with, N contains all the N training examples that are still to be covered, P is empty because no P examples are initially misclassified, ∗ is the best generalization error bound found so far for any hypothesis B and initially set to 1. Global variable is set to the

Using Generalization Error Bounds to Train the Set Covering Machine

263

Input : S,T ,H B ← ∅, N ← N , P ← ∅, ∗ ← 1,  ← |H|; Call : addball(B,∗ ,N ,P ) Output : A T =‘conjunction’ or T =‘disjunction’ of data-dependent balls B∗ ⊆ H

Algorithm 1. BBSCM(S, T, H) Function addball(B,∗,N ,P )

1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: 16: 17: 18: 19: 20:

Data: Consider all h ∈ H  B ; Order according to ({h} ∪ B) → (h1 , 1 , η1 ), . . . , (h ,  , η ) ; for i = 1, . . . ,  do Btemp ← B, N temp ← N , P temp ← P if ηi < ∗ then B ← B ∪ {hi }, N ← N  {ν(hi )}, P ← P ∪ {π(hi )} if i < ∗ then B∗ ← B, ∗ ← i call createtable(∗ ,m) end if found ← false, R ← |B|, A ← −|N | while ¬found do R ← R + 1, A ← A + |N temp| − |N |, kn ← table(R, |P |) if kn = −1 or A ≥ −kn then found ← true end if end while if kn = −1 then call addball(B,∗ ,N ,P ) end if end if end for

number of balls possible for current B. Using the above inputs and variables, algorithm BBSCM calls recursively function addball(B,∗,N ,P ) (see below). Finally the output of algorithm BBSCM is a conjunction or disjunction of balls (classifier/hypothesis) B ∗ that can be used for classification using equation (2). Function addball: This function adds each possible ball hi to the current hypothesis B and checks to see if its generalization error bound i is smaller than the best generalization error bound ∗ found so far. If so, then the value of B ∗ is replaced with B and the best risk bound ∗ is replaced with i (line 6). Also at this stage function createtable is called (line 7) to get table (see description of function createtable below). On line 11 if table(R,|P |) returns kn = −1 then this indicates that there is no bound for R and |P | that is smaller than ∗ . If table(R,|P |) is a positive integer kn, then there is a possibility of finding a ball to add to the current

264

Z. Hussain and J. Shawe-Taylor

hypothesis that will give a smaller risk bound than ∗ provided there exists a set of R additional balls that leave no more than kn N examples uncovered and no additional P examples misclassified. If kn ≥ 0, then line 12 checks whether a larger number of N examples can be covered using R balls (see Lemma 2). If so, then the procedure calls itself recursively (line 17) until all balls in H have been enumerated. Function createtable: Local variable array table is an m × m matrix where m = |X| is the number of training examples. Initially all values in the table are set to −1. Function createtable calculates for R balls and kp misclassifications (on the P examples) the number of N examples that can be covered without creating a bound lower than the best bound ∗ found so far (line 5). This function returns table. Function createtable(∗,m) 1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13:

5

Initialize: table ← −1, R+ ← 0, kpfound ← true, kp ← −1; while kpfound do kpfound ← false, kp ← kp + 1, Rfound ← false, R ← 0 while Rfound do Rfound ← false, R ← R + 1, kn ← 0 while (m, R, R+ , kp, kn) < ∗ do kn ← kn + 1 end while if kn > 0 then kpfound ← true, Rfound ← true end if table(R, kp) ← kn − 1 end while end while

BBSCM(τ )

BBSCM(τ ) allows a trade-off between the accuracy of the classifier and speed. In function createtable(∗, m) the while condition computes (m, R, R+ , kp, kn) <∗ . In the BBSCM(τ ) heuristic this while condition becomes (m, R, R+ , kp, kn) <τ · ∗ . Allowing the BBSCM algorithm to ignore solutions whose generalization error bounds are not a factor τ from the optimal found so far. Clearly setting τ = 1 gives the BBSCM algorithm – however as mentioned above this algorithm is too slow for data sets with m > 100 training examples. Therefore we would like to set τ < 1. Setting τ close to 1 may cause the heuristic to be too slow but create hypotheses that have small generalization error bounds similar to those for τ = 1. Setting τ close to 0 will speed up the solution but may not create a large enough search space in order for BBSCM(τ ) to find hypotheses with relatively small generalization error bounds. Hence, setting τ

Using Generalization Error Bounds to Train the Set Covering Machine

265

is unlike setting a regularization parameter since it is trading accuracy against time - bigger τ is always better in terms of generalization error bounds, but costs more in terms of computational time.

6

Theory

Before looking at the first Theorem we would like to present the following lemma that allows the theorem to be proved. Note that because of space constraints we only use Theorem 1 for this section, although, the results also hold for other SCM bounds. Also, we only give a sketch proof of the main theorem because the full proofs of all the results in this section can be found in the longer version of the paper. Lemma 1. The generalization error bound given by equation (3) (Theorem 1) is monotonically increasing in the second parameter. Theorem 2. Let ∗ be the smallest generalization error bound currently found for hypothesis B ∗ by the BBSCM algorithm. For any hypothesis Bi that gives η(Bi ) > ∗ there exists no extension Bˆ ⊇ Bi such that the generalization error ˆ ≤ ∗ . bound for (B) This theorem states that for any hypothesis Bi if the bp generalization error bound ηi is worse than the best bound ∗ then there is no need to try and cover any more N examples from this ball as there will never be a smaller bound than the best ∗ found so far. Lemma 2. Let U be a set covered by A1 , . . . , Ak . For any V ⊆ U ∃ j such that |Aj ∩V | ≥ k1 . |V | Lemma 3. Suppose createtable(∗ , m) has been executed and kn table(R, kp) ≥ 0. It follows that (m, R, R+ , kp, kn + 1) > ∗ for R+ ≥ 0.

=

Using Lemma 2 and 3 we can now prove that the BBSCM(τ ) algorithm will only disregard a ball for inclusion if it cannot lead to a hypothesis with generalization error bound smaller than the one currently found by BBSCM(τ ). Hence, giving solutions that are a τ factor from the optimal. Theorem 3 (main theorem). If algorithm BBSCM(τ ) outputs hypothesis B ∗ with generalization error bound ∗ then there exists no hypothesis Bˆ such that ˆ < τ · ∗ . generalization error bound (B) Proof (sketch). We prove this result by contradiction. We consider a hypothesis whose generalization error bound is smaller than the one found by the BBSCM(τ ). Next we claim and prove, using Lemma 2 and 3, that if this hypothesis does indeed have a smaller loss bound than the BBSCM(τ ) solution then it should have been chosen by the BBSCM(τ ). However, this hypothesis was not chosen by the BBSCM(τ ) algorithm. Hence, a contradiction.



266

Z. Hussain and J. Shawe-Taylor

Theorem 4. If algorithm BBSCM outputs a hypothesis B ∗ then its generalization error bound ∗ will be globally optimal (i.e. the smallest bound possible). Proof. Apply Theorem 3 with τ = 1.



From these theoretical results we have shown that the BBSCM is guaranteed to find the hypothesis B ∗ with the smallest generalization error bound ∗ .

7

Experiments

Experiments were conducted on seven standard UCI repository data sets [1] and are described in [4]. All examples with contradictory labels and whose attributes contained unknown values were removed (this reduced considerably the Votes data set). We compared the results against the SVM and SCM. The SVM was equipped with a Gaussian kernel and the SCM used the L2 metric to construct its set of data-dependent balls. All training sets were evaluated using 10-fold cross validation and both the SVM and SCM were trained with an extra 10-fold cross validation split in order to evaluate the regularization parameters for each algorithm. Note, the running times include this parameter tuning phase. In contrast, the BSCM and BBSCM(τ ) experiments were only conducted using one 10-fold cross validation split as there were no parameters to tune. For the SVM, we report in Table 1 the number of Support Vectors (SVs), the number of seconds (time) spent training and tuning parameters and the error (err) of the classifiers averaged over all 10 sets for various values of Gaussian width parameter γ and regularization parameter C. For the SCM we report the machine type (T ), where “c” denotes a conjunction and “d” denotes a disjunction of balls that gave the smallest CV error. To compare against the number of support vectors in the SVM we also report the average number of balls (b) for each classifier over the 10 folds. For the BSCM and BBSCM(τ ) we also give the generalization error bound (thm) used for each heuristic where “(1)” denotes Theorem 1 and “(2)” denotes the Theorem of [5] given by their Equation 10. Finally, we chose “c” and “d” for those machines that gave the smallest generalization error bound for each CV split. Furthermore, for the BBSCM(τ ) we only used τ = 0.1 as the search space grew too large for anything larger than this value. From Table 1 we observe that the BSCM and BBSCM(τ ) heuristics are competitive with the SVM and SCM in terms of generalization error for the BreastW, Votes, Pima, Haberman and Glass data sets. Also, both heuristics are considerably sparser than the SVM. Finally, the BSCM exhibits faster running times when compared with the SVM and SCM.

8

Conclusion

We have proposed the branch and bound set covering machine (BBSCM) algorithm and two heuristics BSCM and BBSCM(τ ) along with theoretical results

Using Generalization Error Bounds to Train the Set Covering Machine

267

Table 1. SVM and SCM model-selection results using 10-fold cross validation Data Set

SVM SCM BSCM BBSCM(τ ) SVs time err T b time err thmT b timeerr thmT berr BreastW 79.2 506.1 .035 c 2 658.4 .0395 (2) c 1.546.7 .0335 (2) c 1.0351 Votes 18.5 8.8 .1033 c 1 5.4 .1154 (2) d 1.50.1 .14 (2) c 1.1538 Pima 398.111071.9.246 c 5.9 1091.9.2721 (2) c 2.899.4 .2579 (2) c 1.2891 Haberman 136.45129.2 .2413 d13 272.5 .2789 (1) c 1 25.5 .2585 (2) c 1.2585 Bupa 199.43083.1 .2694 d32.6128 .3478 (2) d 4.143.2 .3769 (2) d 1.3913 Glass 92.4 183.8 .2383 c 3.8 57.9 .2331 (2) c 1 1.2 .2379 (2) c 1.2393 Credit 325.97415.1 .2545 d3.7 831.2 .317 (1) c 1 53.3 .3247 (2) c 1.3247

and experimental evidence to show that the algorithm is competitive with the SCM and SVM. The main novelty of this paper is that we have eradicated the need for the parameters in the SCM by using tight generalization error bounds to train the algorithm. We have shown theoretical results that guarantee the BBSCM algorithm will find a classifier with the smallest generalization error bound. Furthermore, we have also shown that heuristics that try to minimize directly the loss bound are competitive with traditional cross-validation model selection techniques. We believe this helps bring together the ideas of learning theory and practical machine learning – using tight bounds directly to carry out model selection. This, we hope, is the first step towards learning machines that need little human intervention in terms of tuning regularization parameters with (hopefully) no degradation in generalization ability. Future work involves pruning further the search space of the BBSCM algorithm to allow larger data sets to be tractable. Also, the development and use of tighter loss bounds in the BBSCM, BSCM and BBSCM(τ ) algorithms is a future research direction.

Acknowledgements This work was supported in part by the IST programme of the European Community, under the PASCAL network of excellence.

References 1. Blake, C., Merz, C.: UCI Repository of machine learning databases. Department of Information and Computer Science. University of California, Irvine, CA (1998), http://www.ics.uci.edu/∼ mlearn/MLRepository.html 2. Boser, B.E., Guyon, I.M., Vapnik, V.N.: A training algorithm for optimal margin classifiers. In: Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory, pp. 144–152. ACM Press, New York (1992) 3. Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods. Cambridge University Press, Cambridge (2000)

268

Z. Hussain and J. Shawe-Taylor

4. Marchand, M., Shawe-Taylor, J.: Learning with the set covering machine. In: Proceedings of the Eighteenth International Conference on Machine Learning (ICML 2001), pp. 345–352 (2001) 5. Marchand, M., Sokolova, M.: Learning with decision lists of data-dependent features. Journal of Machine Learning Reasearch 6, 427–451 (2005) 6. Vapnik, V.N.: Statistical Learning Theory. Wiley, New York (1998)

Model of Cue Extraction from Distractors by Active Recall Adam Ponzi Laboratory for Dynamics of Emergent Intelligence, RIKEN BSI, Saitama, Japan [email protected]

Abstract. Cues are informative signals animals must use to make decisions in order to obtain rewards, usually after intervening temporal delays, typified in the cue-action-reward task. In behavioural experiments the cue is often clearly distinguished from other stimuli, by a salience such as brightness for example, however in the real world animals face the problem of recognizing real cues from among other environmental distracting stimuli. Furthermore once the cue is recognized it must cause the animal to make a certain action to obtain reward. Therefore the animal faces a compound chicken-and-egg problem to obtain reward. First it must recognize the cue and then it must learn that the cue must initiate a certain action. But how can the animal recognize the cue before it has learned the action to obtain reward, since in this initial learning stage the cue is only partially predictive of reward? Here we present a simple neural network model of how animals extract cues from background distractor stimulus, all presented with equal salience, based on successive testing of different stimulus-action allocations over several trials. A stimulus is selected and gated into working memory to drive an action and then reactivated at the end period to be reinforced if correct. If the stimulus is not reinforced over several trials it is suppressed and a different stimulus is selected. If the stimulus is a real cue but it drives the incorrect action, its cue-action allocation is suppressed. This mechanism is enhanced by the property of cue mutual exclusion in trials which also provides a simple model of bottom-up attention and pop-out. The model is based on the cortical and hippocampal projections to the dopamine system through the striatum including a model of salience gated working memory and a reinforcement and punishment system based on dopamine feedback balance. We illustrate the model by numerical simulations of a rat learning to navigate a T-maze and show how it deterministically discovers the correct cue-action allocations.

1 Introduction The Basal Ganglia are well known to be involved in reward learning mechanisms [1,2] and cortico-basal ganglia loops are critical for the learning of rewarded cued procedures and in cued working memory tasks [3]. Dopamine is thought to play the role of reward prediction error [1,4], where the burst firing of dopamine cells is increased by unexpected rewards and reduced if an expected reward is omitted. The striatum, which is the main input structure of the Basal Ganglia, recieves a strong input from the midbrain dopaminergic system and the prominent striatal projection neurons, the medium spiny neurons, are also known to reflect reward expectation themselves [5,6]. Dopamine firing M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 269–278, 2008. c Springer-Verlag Berlin Heidelberg 2008 

270

A. Ponzi

can also be triggered by novel stimuli that do not involve reward [7]. Rapid detection of a cue change by striatal neurons has been studied by Pasupathy [3]. The basal-ganglia, the prefrontal cortex and the midbrain dopamine nuclei projecting to them are also strongly implicated in working memory [8]. Persistent neural activity in recurrent circuits plays a central role in the maintenance of information in working memory [9]. Some theories suggest a gating role for phasic dopamine release, such that dopamine release is required for read-in to working memory [11,10,12]. Seemingly paradoxically dopamine can suppress or enhance striatal activity [13], and can extend the duration of enhanced activity. In spatial working memory tasks some hippocampal neurons (‘splitter cells’ or ‘episodic cells’) fire selectively dependent on the context of a specific recent response or future goal [14,15,18]. For example, during performance of a spatial alternation task, many hippocampal pyramidal neurons fire selectively after right turn or left turn trials, even though the rat is running in the same direction through the same location on the stem of the maze on both types of trials. Ferbinteaneu et al. [17] describe retrospective and prospective coding in hippocampal neural assemblies in spatial working memory tasks. The origin of journeys influenced firing even when rats made detours showing that recent memory modulated neuronal activity more than spatial trajectory. Diminshed retrospective and prospective coding was observed in error trials suggesting this signal was important for task performance. Mulder et al. [19] recorded hippocampal output structures associated with the motor system (nucleus accumbens and ventromedial caudate nucleus) in rats solving a plus-maze. They found a variety of responses including neurons that fired continuously from the moment the rat left one location until it arrived at a goal site, or at an intermediated place, such as the maze centre. They suggest their results support the view that the ventral striatum provides an interface between limbic and motor systems, permitting contextual representations to trigger movements and have an impact on action sequences in goal directed behaviour. Barnes et al. [20] have also described task and expert neurons in dorsal striatum.

2 Model This model is composed of two parts, the cue detection part and the action selection and reinforcement part. The model is an extension of the model presented in [21,22]. In those papers we describe a cue response task typical of primate studies which consists of three stages, stage 1 is the cue presentation stage, stage 2, the action selection stage and stage 3, the reward stage, where each stage is separated by delay periods. In the work presented here, this model is re-motivated as a spatial working memory temporal credit assignment model typical for rat maze tasks and described in Fig.1(a). In stage 1 a cue is presented together with some distractors. Here we suggest this corresponds to the spatial view of the animal from the initial location. The spatial views from each of the two initial locations have some parts in common and also some differences. For example a visual cue may be visible from one location but not the other, or the same cue may be visible, but in a different location with respect to the animal’s head direction, or other background cues. Here we simplify these possibilities and represent the view simply as an activation of several units of the input ‘P ’ layer. In initial position ‘A’

Model of Cue Extraction from Distractors by Active Recall

271

a fixed set of background distractors is activated simultaneously to a cue. In initial position ‘B’ the same fixed set of background distractors is activated, but the cue from location ‘A’ is not activated and instead a different unit is activated. In stage 2 an action must be selected by the animal from some possible alternatives. The correct action to choose depends on the cue presented in stage 1, i.e. the initial location. Here the animal must turn right at the junction if it started in one location and left if it started in the other location. In stage 3 reward is given to the animal if the action made in stage 2 is the correct one. The basic idea of the model is that the cue presented in a layer P in stage 1 is gated into working memory in a recurrent layer ‘Q’, see Fig.1(b). P refers to primary since this layer simply reflects the external environment, while Q is the layer after P . The working memory of the cue is reactivated in layer Q during the action selection stage 2. The reactivation is itself driven by an external trigger signal which is also presented in layer P and given by the external environment, see Fig.1(a). During this stage 2 the reactivated cue drives a winner-take-all (WTA) action selection system in layer ‘M ’ which activates the action which is driven strongest by the cue. Layer Q is connected to layer M in an all-to-all fashion so the action j activated in layer M when QM the cell i is activated in layer Q depends on the synaptic weights Jij between layers QM Q and M . The action j selected is given by the strongest weight maxj {Ji=cue,j }. As explained in [21,22] the cue in working memory in layer Q is again reactivated in stage 3 and reward is given simultaneously if the action made in stage 2 is correct. The reactivation is itself driven by another end signal in the P layer, see Fig.1(a). Since the second reactivation activates the same Q layer cell i as stage 2, this also reactivates the

Fig. 1. (a) Task structure described in the text. The animal runs around the track and the task is simplified to three stages shown as solid boxes on the track. The cue presentation stage 1 is at the initial location, the action selection stage 2 (Tjunction ) is when the junction is sensed and the end stage 3 (Tend ) is when the end of the track is sensed. Reactivations of stimuli presented in stage 1 occur in stages 2 and 3 together with activation of the action selecting M (MSN) cells (G(t) = 1) in the latter halves of these stages, depicted by hatched boxes. Here the task is depicted as an alternation task, but in this paper we study the more difficult task where the animal is removed from the end location and randomly placed at one of the two initial locations each trial. (b) Anatomy of the model system described in the text. The primary P and recurrent Q layers are suggested to be part of cortex or hippocampus which projects to striatum, while the M layer where the cells coding for actions are located may be striatum medium spiny neurons (MSN). The N layer cell is not shown but is also suggested to be striatum projecting to thalamus.

272

A. Ponzi

QM same action j. If reward is present the synapse Jij is strengthened, while if reward is absent the same synapse is weakened. This process therefore targets a particular synapse for reinforcement or punishment. Regardless of whether reward is received or not, after a longer delay the next trial starts with one of the cues chosen at random presented together with some distractors. In [21,22] we show that the system can easily find the correct cue to action allocation by an exploratory process using the punishment signal to depress selected but unwanted actions. When the correct cue-action allocation is discovered it is stabilized by dopamine negative feedback which limits the growth in QM the weights Jij . A drawback of the model presented in [21,22] was that the distractors and cue presented in stage 1 had to be artificially distinguished by an incentive salience level which was set to be high for the cue and low for the distractors, which were presented simultaneously. In fact the working memory was a winner-take-all system so that the unit activated with maximum salience during stage 1 was the unit gated into working memory to be reactivated in stages 2 and 3. Only if the cue had sufficiently high relative salience was it gated into working memory. If its salience was too low, a distractor could be gated in instead. Here we address this defect and therefore consider the problem of cue extraction from among distractors, where all cues are presented externally with the same salience. I.e. we allow the system to discover the cue and give it a high salience, while the distractors are given a low salience. The cue extraction model described here is comprised of the P and Q layers shown in Fig.1(b) together with the thalamus T and the dopamine system. The Q layer activities qi (t) are given by, ⎛ ⎞  dqi = −k1 qi + k2 g ⎝ wij qj ⎠f (T (t) − TK ) + k3 xi Pi f (TK − T (t)). (1) dt j=i

In this equation the qi (t) can be considered firing rates or membrane potentials. The first term on the RHS represents the exponential decay of qi (t) back to zero with rate k1 when there is no activation. The second term models the effect of the all-to-all modifyable recurrent collaterals with weights wij (t), where g(x) is the sigmoidal function which provides the non-linearity and limits the activation of this term. The weights wij (t) are given by, dwij = k5 qi − k4 qi qj − k6 wij . (2) dt The combination of Eq.1 and Eq.2 describes a winner-take-all system as described in more detail in [21,22]. The k6 term is an exponential decay which reverts the wij (t) to zero between trials. The third term in Eq.1 is the one-to-one input from primary P layer. In this third term Pi (t) = 1, 0 are the activities of P layer cells which respond directly to the external environment and are unity when active and zero at other times. As described above during the cue presentation period one of the two cue cells is set to unity while the other is set to zero, e.g. PcueA = 1, PcueB = 0 and all the distractors are all set to unity. Here we have five distractors and two cues so that the total number of P cells and Q cells is N P = N Q = 7. The factor xi (t) in the third term of Eq.1 is the modifyable

Model of Cue Extraction from Distractors by Active Recall

273

salience of input Pi . The input with the maximum salience from among those presented is the input gated into working memory. We wish xi to be high for the cues and low for the distractors, and it is this variable which was artificially and externally fixed in the previous modeling [22], we will describe it below. The factors f (T (t) − TK ) and f (TK − T (t)) in the second and third terms of Eq.1 model the reactivation and P layer input down-gating respectively. T (t) is considered to be the activity of the thalamus while TK is its baseline activity and f (x) = x when x > 0 and f (x) = 0 otherwise is a positivity function. When the thalamus activity is above baseline T (t) > TK the recurrent collateral term is activated which causes the reactivation of the cue presented in stage 1. At these times the input from the P layer is down-gated so that any input active at reactivation times does not interfere with the reactivation of the previously presented cue. On the other hand when T (t) <= TK there is no reactivation and the activity of the Q layer cells is driven and determined by their topographic inputs Pi . The thalamus activity is given by, dT = −T (t) + TK + Tjunction + Tend . dt

(3)

This describes activation of the thalamus only when the animal is at the junction stage 2 (Tjunction = 1) and the end stage 3 (Tend = 1), see Fig.1(b), where reward is located. In fact we suggest the animal has already learned to make these reactivations and they are represented in striatal cells projecting to thalamus. In the discussion section we describe how the Tend signal can be learned by the animal. The important variables are the saliences xi . Here we suggest that the xi can be appropriately modeled by, dxi = τx qi (D(t) − DK ). dt

(4)

Here τx is a slow timescale generating learning over many trials and the term (D(t) − DK ) is the excess dopamine level D(t) over its baseline DK , and it is given by,  dD = −D(t) + DK − k7 qi (t) + kR R(t) + otherterms dt i

(5)

This equation includes an inhibitory term as the sum over the Q layer activities and an excitatory term R(t) with factor kR which describes the primary reward, activated in stage 3 during the end period where Tend = 1, when the animal makes a correct action in stage 2. It is easy to understand how the pair of equations Eq.4 and Eq.5 produce the desired behaviour. Suppose the cue has a higher salience xcue than the distractors it is presented together with. Then it will drive the corresponding qcue during the cue presentation period stage 1 more than the other qi are driven and will therefore be gated into working memory and reactivated at the action selection stage 2 and the end stage 3. Suppose this cue also drives the correct action, then dopamine will be positively activated in stage 3 by the R(t) term during the second reactivation of the qcue . Eq.4 will be positive for the reactivated qcue and that particular xcue will be increased by LTP. Therefore next trial it occurs the cue qcue will be even more likely to be gated into working memory. Indeed the salience xcue for this cue has a positive feedback

274

A. Ponzi

and would grow without bound except for the fact that during the cue presentation  period LTD is generated by the inhibition of dopamine by the term −k7 i qi (t) in Eq.5, which depends on the magnitude xcue and will therefore limit the growth of the salience xcue to a fixed value. Therefore we see that there is a stable attractor state for the cue. However the situation is different for the distractors. Suppose one of the distractors has the maximum salience xdist1 from among the presented stimuli. This qdist1 will be gated into working memory and drive a given action in stage 2 and be reactivated at the end period 3. However since this is not a cue there is only a fifty-fifty chance that it is driving the correct action for the cue it happens to be presented with and primary reward will therefore only be found half the time. During trials where primary reward is not found the reactivation at the end period 3 will actually inhibit dopamine Eq.5 and cause a suppression of xdist1 , Eq.4, during the reactivation. xdist1 will wander around depending on chance sequences of trials without a stable fixed value, but with a long time average value well below what can be attained for xcue . After some time xdist1 will drop down sufficiently for another distractor, xdist2 to become maximal and be gated into working memory and then tested over several trials, eventually being suppressed. Once xcue has been selected however, if it drives the correct action, it will quickly attain its maximum stable attractor value. We now also describe the action selection system presented previously in [21,22]. The system is described by the M layer which takes an all-to-all projection from the Q layer and forms a winner take all system, ⎛ ⎞ M N  QM  dMi = −Mi + f ⎝ Jij (t)qj − k2 Mj + k3 Mi ⎠ G(t) (6) dt j j=1 The M units represent the actions and here we have two of them N M = 2 corresponding to right and left turns. This equation describes a standard winner-take-all system. The term G(t) is set to unity when actions are allowed to be selected and zero otherwise, see Fig.1(a). We suggest it is represented in striatum in a similar way to Tjunction and Tend in Eq.3 and the reason it is different from Tjunction and Tend will be described in more detail below. The actual action selected is given by the sign of F (t) fixed by integration of the M units over the junction period Tjunction = 1, M

N /2  dF (t) = Tjunction ( Mi (t) − dt i=1

M N 

Mi (t)) − (1 − Tjunction )F (t)

(7)

i=N M /2

QM As described the synaptic weights Jij from the Q to the M layer are updated by three way Hebbian learning [23,24] which is reinforced in the presence of dopamine and depressed without dopamine, QM dJij QM QM = −Jij (t) + f ((D(t) − DK )Mi qj + Jij (t)). dt

(8)

The dopamine system  Eq.5 is extended to include a negative feedback projection from the M cells, −k8 j Mj (t). As described in [21,22] this system can find and stabilize

Model of Cue Extraction from Distractors by Active Recall

275

the correct cue-action allocation providing the cues are known. Together with the cue extraction model described above, the cue is successfully extracted and bound to the correct action. We now illustrate this model by numerical simulations integrated by fourth order Runge-Kutta.

3 Results The behaviour of the model is best illustrated by studying examples of its time series. Fig.2 describes a time series of the Q layer activities qi (t) for all seven cells, two cues and five distractors over a 100 trial experiment. This figure illustrates successive cue testing where the stimulus with maximum salience xmax is gated into working memory and drives an action and is then reactivated at the end period and reinforced if it drove the correct action and punished if not. Fig.3 describes in more detail time series of individual trials late in learning after the correct cues have been found. This system is able to extract the correct cues and find the correct cue-action allocations. If a cue is extracted once and happens to drive the correct action its salience is easily strengthened as described above and the action it is driving is reinforced as described in more detail in [21,22]. Should a cue be extracted but not drive the correct action the situation is different however. If there is no positive dopamine signal at the end period, not only will the correct action not be reinforced but also the salience will not be strengthened, even though it is in fact the correct cue, but is simply not driving the correct action. A parsimonious point about this model is that this situation is easily

Q layer activities qi(t), Dopamine Modulator D(t)

Q layer activities qi(t), Dopamine Modulator D(t)

70 100

80

60

40

20

0

60 50 40 30 20 10 0

20

40

Trial Number

60

80

100

6

8

10

12 Trial Number

14

16

18

Fig. 2. (a) Time series of 100 trials of the Q layer activites, qi (t), overlaid on top of each other, each consisting of three stages separated by intertrial delay periods. In this figure the three trial stages of cue presentation, action selection reactivation, and end period reactivation are difficult to see each trial. Both cues are found by around trial number 80. Also shown is the dopamine modulator. (b) Magnification from panel (a) showing trials 6 to 18 illustrating successive cue testing. During trials 6 to 12 one particular distractor has maximum salience and is gated into working memory. Some trials it drives the correct action in stage 2 and some not, dependent on which particular cue it is presented with, and therefore is not consistently reinforced as can be seen from the dopamine time series. By trial 12 this distractor is punished sufficiently for its salience to become non-maximal and another new distractor becomes the maximally salient one, until trial 18 when it too is punished sufficiently. In trial 18 one of the correct cues becomes the dominant stimulus for the first time.

A. Ponzi Q layer activites, M layer activites, Dopamine modulator

276

A layer activites, Dopamine modulator

30

20

10

M1

1

10

0

10

D

-1

10

-2

10

M2 -3

10

0 5000

5020

5040 5060 Time

5080

5240

5260

5280 5300 Time

5320

5340

2260

2280

2300 Time

2320

2340

Fig. 3. (a) Time series of Q layer activities qi (t) for two consecutive trials after the cues have been recognised in two different panels one for each cue type. (b) A similar figure in log scale for clarity. The selective activation of a single cue at the presentation stage 1, the action selection stage 2 and end period stage 3 is clear. The other qi (t) are only weakly activated and invisible near zero. Also shown is the dopamine D(t) time series which is depressed below baseline during the cue presentation stage 1 and reactivation stage 2. It is excited above baseline during the end stage 3 by primary reward because these cues are driving the correct actions. Also shown are the two M layer action selection cells, M1 (t) and M2 (t). Each of the two cues activates a different M cell as required, while the other is suppressed. The M cells are activated in the end stage 3 for reinforcement as well as in the action selection stage 2. Notice that the M cells are activated late in the two reactivation periods to allow the cue reactivation to reach its fixed point level. This is modeled by setting the term G(t) to unity in Eq.6 late in the action selection periods.

dealt with. In fact supposing the correct cue is reactivated at stage 2 but drives the wrong action, in this model the cue is still reactivated by the end signal stage 3 but there is no primary reward and in fact the reactivation suppresses dopamine via the inhibitory projection in Eq.5. This negative dopamine signal targets the action selected by the cue for punishment. Since the cue in fact selects the incorrect action this is an appropriate behaviour of the model. Although the cue will not be given salience by this action, next time the cue is extracted from the distractors by successive testing it will more likely drive the correct action. Once it has driven the correct action its salience will be forever strengthened and the correct action reinforced and stabilized. Indeed this situation is shown in Fig.2. One of the two cues is found for the first time at trial 18, however as can be seen in the below baseline dopamine trace during the end stage 3 of trial 18 in Fig.2(b) it happens to be driving the incorrect action. However the negative dopamine is sufficient to depress the incorrect cue-action allocation so that at a later trial when this correct cue is again tested it drives the correct action. The other cue is similarly extracted, tested and its cue-action allocation reversed and by trial 80 they form a stable attractor state and remain at high salience for the rest of the experiment, while the distractors are suppressed as can be seen in Fig.2(a). Another parsimonious point of this model is that the cues is are preferentially picked out over the distractors, or pop-out due to the fact that they exclude each other in trials. Exclusion means that cues are not presented during the stimulus presentation stage 1 as often as the distractors are. As described above the combination of Eqs. 4 and 5 means that qi (t) cells active during the cue presentation stage cause inhibition of dopamine

Model of Cue Extraction from Distractors by Active Recall

277

D(t) in Eq.5 which causes LTD in their corresponding xi (t). Cues which are not active do not receive this LTD punishment. This exclusion property of the cues means the excluded cue is favoured over the distractors and is picked out more readily as a form of novelty response.

4 Discussion We have presented a model of cue extraction in a general approach to the problem of temporal credit assignment and shown that though simple the model behaves as desired with several parsimonious aspects. We have described cue extraction and action binding by successive testing enhanced by cue mutual exclusion. We describe constant distractors but note extraction will also occur if distractors are sampled randomly each trial, providing distractors have a non-zero probability of co-occurrence with each other in any given trial. Even in the case the distractors have exclusive sets, since they are not predictive of reward, the system will still work. One drawback of the model presented above is the necessity for an end signal to drive final stage 3 reactivation. At times when primary reward is found the stage 3 reactivation could in principle be driven by primary reward, however the problem with this idea would be that no reactivation would occur when primary reward was not found, in contradiction to the requirement described above for discovery of the correct action from the punishment signal. To deal with this difficulty we extend the model to include another set of cells N . These cells learn to fire at the reward locations even when there is no reward and are used to drive the thalamus reactivations. Here we only have one and it is given by, dN/dt = −N + j JjP N (t)Pj , where the weights JjP N are given by dJjP N /dt = (D(t) − DK )N Pj . It is activated by projection from the primary P layer representing the external environment and LTP occurs only in the presence of excess dopamine. Accordingly the dopamine signal Eq.5 is modified further to include an inhibitory projection −k9 N (t) from this cell. Therefore when the N cell has been bound to a given location represented by the Pj by positive R(t) and the animal returns to this location when primary reward is absent (since the cue was wrong) the N unit acts to generate a negative dopamine signal. This enhances the action punishment behaviour described above when reward is not found. Furthermore we modify the thalamus signal T (t) Eq.3 by replacing the end signal by this N cell activation, dT /dt = −T (t)+TK + Tjunction + N (t), so that the locations where reward has been experienced activate replays even if reward is not found on subsequent visits. We do not implememt this model here, but as shown in [25] it works well without the necessity for an end signal. In [25] we also include working memory maintenance of actions as well as cues to allow rapid reversal of cue-action allocations without any noise, accounting for some of the known properties of the dopamine signal [1]. In future work we will show how to produce the junction thalamus signal in an internally defined way. We have not specified the location of the P and Q layers, but anywhere in cortex or hippocampus is possible. At a certain level the Q layer may even represent the hippocampus (CA3) while the P layer may correspond to cortex projecting to hippocampus, the M and N region the nucleus accumbens while the dopamine system the VTA. Our ‘gating’ of PFC layer input into hippocampus would agree with Lisman and Grace [7] suggestions concerning control of behaviourally significant information flow.

278

A. Ponzi

The reactivation of the cue in the Q layer at the junction stage 2 for action selection means that after learning the cell that is firing at the junction depends on where the animal is coming from and also after learning of the task, where the animal is going to. This may therefore correspond to the hippocampal splitter cells [14,15] described in the introduction. Since the plus maze in both allocentric and egocentric conditions will obey similar principles, where however the cue presentation stage corresponds to the spatial view of the animal from the cross junction, which depends on the direction the animal approaches, encoded in the place cells, this work may also explain the plus maze retrospective and prospective coding [17]. Furthermore the striatal MSN neurons show response characteristics of context or task neurons, such as are found in the dorsal striatum [20] and ventral striatum [19], described in the introduction. This can be both the M action selecting cells, and the N reactivation generating cells. N cells may also generate the specific spatial response characteristic of expert neurons at the T-maze junction seen in Barnes et al. [20].

References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26.

Schultz, W.: J. Neurophysiol. 80(1), 1–27 (1998) Wickens, J.: 8, R77–R109 (1997) Pasupathy, A., Miller, E.K.: Nature 433, 873 (2005) Kawagoe, R., Takikawa, Y., Hikosaka, O.: J. Neurophysiol. 91(2), 1013–1024 (2004) Hollerman, J.R., Tremblay, L., Schultz, W.: J. Neurophysiol. 80, 947–963 (1998) Nicola, S.M., Surmeier, J., Malenka, R.C.: Ann. Rev. Neuroscience. 23, 185 (2000) Lisman, J.E., Grace, A.A.: Neuron 46, 703 (2005) Goldman-Rakic, P.S.: Neuron 14(3), 477–485 (1995) Gruber, A.J., Dayan, P., Gutkin, B.S., Solla, S.A.: J. Comput. Neurosci. 20, 153–166 (2006) Cohen, J.D., Braver, T.S., Brown, J.W.: Curr. Opin. Neurobiol. 12(2), 223–229 (2002) Dreher, J.C., Guigon, E., Burnod, Y.: J. Cog. Neurosci. 14(6), 853–865 (2002) Frank, M.J., Loughry, B., O’Reilly, R.C.: Cog. A. & B. Neurosci. 1(2), 137–160 (2001) Kiyatkin, E.A., Rebec, G.V.: J. Neurophysiol. 75(1), 142–153 (1996) Hasselmo, M.E., Eichenbaum, H.: Neural Networks, 18, 1172 (2005) Wood, E.R., Dudchenko, P.A., Robitsek, R.J., Eichenbaum, H.: Neuron 27, 623–633 (2000) Kunec, S., Hasselmo, M.E., Kopell, N.: J.Neurophysiol. 94 (2005) Ferbinteanu, J., Shapiro, M.L.: Neuron 40, 1227–1239 (2003) Anderson, M.I., Jeffery, K.J.: J. Neurosci. 23, 8827–8835 (2003) Mulder, A.B., Tabuchi, E., Wiener, S.I.: European J.Neuroscience 19, 1923–1932 (2001) Barnes, T.D., Kubota, Y., Hu, D., Jin, D.Z., Graybiel, A.M.: Nature 437, 1158 (2005) Ponzi, A.: Proceedings of IJCNN 2007 (2007) Ponzi, A.: Neural Networks (to appear, 2008) Reynolds, J.N.J., Hyland, B.I., Wickens, J.R.: Nature 413, 67 (2001) Nakahara, H., Amari, S., Hikosaka, O.: Neural Computation 14, 819–844 (2002) Ponzi, A.: Proceedings of ICCN 2007 (2007) Ponzi, A.: IEICE Technical Report 103(163), 19–24 (2006)

PLS Mixture Model for Online Dimension Reduction Jiro Hayami and Koichiro Yamauchi Graduate School of Information Science and Technology, Hokkaido University, Sapporo Hokkaido 060-0814, Japan

Abstract. This article presents an online learning method for modeling high dimensional input data. This method approximates a nonlinear function by summing up several local linear functions. Each linear function is represented as the weighted sum of a small number of dominant variables, which are extracted by the partial least squares (PLS) regression method. Moreover, a radial function, which represents the respective input area of each linear function, is also redefined using the dominant variables. This article also presents an online deterministic annealing expectation maximization (DAEM) algorithm which includes a temperature control mechanism for acquireing the most suitable system parameters. Experimental results show the effective learning behavior of the new method. Keywords: Partial Least Squares (PLS) Method, PLS Mixture Model, Dimension Reduction, Online DAEM Algorithm.

1

Introduction

In real-world applications, learning systems usually have to perform online learning of high-dimensional data. In such learning tasks, the learning machines may have to overcome the following problems. – The learning machine must achieve a quick response to newly presented samples. – High-dimensional data usually have multicollinearity or include irrelevant dimensions. If the learning machines learn them, they normally waste a huge amount of resources, so somehow they need to reduce the amount of resources for learning such datasets. To solve these problems, we propose a variation of the mixture of experts(ME). The ME used here is a mixture of the local linear models. Each linear model is tuned by the learning samples distributed in a local narrow area so that it shows relatively quick responses to newly presented learning samples. Moreover, in the proposed method, each local linear model and corresponding kernel reduces the number of useless dimensions by using both the partial least squares method (PLS) (e.g. [1]) and the online DAEM algorithm proposed later M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 279–288, 2008. c Springer-Verlag Berlin Heidelberg 2008 

280

J. Hayami and K. Yamauchi

in sec. 4.4. PLS is specialized for selecting dimensions from multi-dimensional patterns, which have multicollinearity. Using PLS, the proposed system achieves online dimension selection not only for each linear model but also for the corresponding Gaussian kernel.

2

Related Work

Schaal et al. have proposed Receptive Field Weighted Regression (RFWR) [2] This has a similar architecture to our model, but it does not reduce useless dimensions. Later, RFWR was improved to Locally Weighted Projection Regression(LWPR) [3]. The improved model reduces useless dimensions of each linear model by using the PLS method but does not reduce the dimensions of each kernel. Ishii et al. have constructed an online EM (expectation maximization) algorithm on a normalized Gaussian network, which has a similar architecture to that of RFWR[4]. However, their model may not reduce the input dimensions if the data have multicollinearity.

3

PLS Regression

The PLS regression method extracts latent variables, which correlate with both inputs and outputs, and approximates the output function. Let x, D be the input vector x = (x1 · · · xD ) and let D be its dimensions D = dim(x). In this paper, the output of the learning machine is a one-dimensional output value y. Now, let N and r be the number of learning samples and the dimensions of the latent variables, respectively. Then, a sample matrix can be written as X = (x1 · · · xN )T , y = (y1 · · · yN )T . We need to standardize the learning samples before PLS regression. In the standardization process, the averaged vector is subtracted from each learning sample. The input and output vectors are represented using latent vectors ti and the two loading matrices P and q as follows ¯ = t1 pT + t2 pT + · · · + tr pT + E = TPT + E X−X (1) 1

2

r

¯ = t1 q1 + t2 q2 + · · · + tr qr + f = Tq + f , y−y (2) ¯ and y ¯ denote the average input where E and f denote residual errors and, X vector and the output value. In the initial state, the number of latent variables is zero. The latent variables are increased one by one until the residual errors are small enough to ignore. For example, let’s consider the case where there is one latent variable. First ˜ = X−X ¯ of all, we have to generate the standard input and output matrices: X ˜ = y−y ¯ . We assume that the latent variable t1 is generated by a linear y transformation: ˜ t1 = w1 X. (3) ˜ T t1 for w1  = 1 using the Now, we derive the matrix w1 so as to maximize y Lagrangian technique.

PLS Mixture Model for Online Dimension Reduction

w1 =

˜ Ty ˜ X ˜ Ty ˜ X

281

(4)

The loading matrices p1 and q1 can be derived by least squares estimation. ˜ T t1 ˜ T t1 X y p1 = T , q1 = T (5) t1 t1 t1 t1 The residual errors E and f are ¯ − t1 pT , E = X−X

¯ − t1 q1 . f =y−y

1

(6)

˜ and y ˜ by E and f , we can derive the next latent variable by If we replace X executing Eqs.(3)-(6) again. Now we can describe the output value y using the latent variables as follows[5]. ¯ )W(PT W)−1 q, f (x) = y¯ + (x − x

(7)

T

where P = (p1 · · · pr ), q = (q1 · · · qr ) and W = (w1 · · · wr ). 3.1

Online PLS Method

So far, we have explained the PLS method using all the learning samples: T T X = (x1 · · · xN ) y = (y1 · · · yN ) . To execute the PLS method in an online manner, we need a buffer for storing all the learning samples. However, this approach not only wastes a huge amount of storage space but also has a high computational cost. To solve this problem, the modified PLS method [6][7] derives the latent variables from two covariance matrices: ΣXy and Σ. Here, ΣXy is the covariance ˜Ty ˜ /N , while Σ denotes the covariance matrix of x and y, given by ΣXy = X T ˜ X/N ˜ ˜ and y ˜ are the standardized matrices matrix of x give by Σ = X , where X ˜ = X−X ¯ and y ˜ =y−y ¯. X Therefore, wr , pr and qr are sequentially derived as below. wr =

ΣXy r ΣXy r 

,

pr = Σr wr /cr ,

 T qr = ΣXy wr /cr , r

(8)

where cr = wr T Σr wr . To get the next (the r + 1-th) new latent variable, we just repeat the above equations using the updated covariance matrices. Xy ΣXy − cr pr qr , r+1 ← Σr

Σr+1 ← Σr − cr pr pr T

(9)

ΣXy 1

and Σ1 can be derived in an online manner, so the latent variables can also be derived in an online manner. Therefore, ΣXy = xy T  /  1  −  x  y T /  1 2 1 Σ1 = xxT  /  1  −  x  x T /  1 2 ,

(10)

where the notation  x  denotes an approximated average of x, which will be defined in Eq(21). Equations (8) and (9) are repeated until ΣXy r  < ε or Σr  < ε where ε is a small threshold. In this study, we executed the online PLS method combined with a new online learning method namely the ‘online-DAEM’ algorithm described in 4.4.

282

4

J. Hayami and K. Yamauchi

Proposed System

The proposed system architecture is similar to that of normalized Gaussian networks (NGnet)[8]. The PLS regression method is embedded in the learning method for reducing dimensions. In particular, the PLS regression method is applied not only to the learning method for the linear regression model but also to that for the kernel. In this paper, we call the kernel the ‘PLS kernel.’ Section 4.1 explains the whole system structure. 4.3 and 4.2 explain the linear function and PLS kernel, respectively. 4.4 explains the ‘online DAEM’ algorithm, which determines the PLS parameters. 4.5 explains the unit manipulation procedure. 4.1

Outline

Output

The proposed system consists of a linear function and a corresponding Gaussian kernel function, as shown in Fig.1. The Gaussian kernel function determines the respective input area for the corresponding linear function. Let fc (x) be the output of each model unit and g(x|c) be the kernel function. Then, the ultimate output is  fc (x)g(x|c) yˆ = c , (11) c g(x|c)

PLS regression PLS kernel

Dimension reduction

Dimension reduction

Dimension reduction

Inputs

Fig. 1. PLS mixture model where g(x|c) denotes the kernel function. It is expressed by   ¯ c )Σ−1 ¯ c )T (x − x c (x − x g(x|c) = exp − . 2

(12)

Note that the kernel function above does not depend on the scaling of data. By this modification, we cannot deal with the network as a statistical model. 4.2

Linear Function

In this model, the PLS regression method explained in section 3 is applied to construct the linear function. As a result, the linear function overcomes the problem of multicollinearity. The output of the c-th unit is −1 fc (x) = yc + (x − xc )Wc (PT qc , c Wc )

where qc denotes the loading matrix of the c-th unit.

(13)

PLS Mixture Model for Online Dimension Reduction

4.3

283

PLS Kernel

To define the kernel function, we have to calculate the inverse of the covariance matrix Σc . However, if there are too few samples, Σc cannot be a regular matrix, so Σ−1 cannot be derived. To overcome this problem, the proposed method c derives a pseudo inverse matrix of Σc . Let Σ∗P LSc be the pseudo inverse matrix.  −1 T Σ∗P LSc = Wc WcT Σc Wc Wc (14) ¯ c |P LS = (x − x ¯ c )Σ∗P LSc (x − x ¯ c )T |x − x   ¯ c |P LS |x − x g(x|c) = exp − 2

(15) (16)

where Wc is the c-th unit’s matrix defined in Eq(7). ¯ c |P LS . Then we can obtain the Mahalanobis distance in the input space: |x − x In this paper, we call the kernel defined in Eq(16) the ‘PLS kernel.’ Figures. 2, 3 and 4 show examples of the PLS kernel where the numbers of latent variables are 0, 1, and 2, respectively. If the number of latent variable is zero, the PLS kernel is a uniform distribution. In the case of one latent variable, the PLS kernel has variance along the 1st PLS direction but has a uniform distribution along the other directions. In the case of two latent variables, the PLS kernel has two variances along the two PLS directions. Therefore, the respective area of the PLS kernel is larger when there are fewer latent variables. Note that this causes the problem that the units allocated in early steps of the learning become dominant in the network. This causes the degrades the generalization ability of the network. To overcome this problem, the system restricts the width of the PLS kernel according to the number of learning iterations of the unit. Therefore, in the early steps of the learning, the shape of the PLS kernel is a hyper sphere, but it gradually changes to an oval-shaped hyper ellipse reflecting the PLS directions. Therefore, Σ∗c = λ(t)c kc ID + (1 − λ (t)c ) Σ∗P LSc where λ(t)c =

1 

1c −t˜ 2

1 + exp

 (17)

where  1 c denotes an approximated average of the c-th unit’s respective ratio, which reflects the number of parameter-updates of the unit approximately. y

y

y

1

1

1

0.5

0.5

0.5 1

0-1

0

-0.5 x0

0

0.5

-0.5 1 -1

x1

1

1

0.5 0-1

0.5 0

-0.5 x0

0

0.5

-0.5 1 -1

x1

0-1

0.5 0

-0.5 x0

0

0.5

-0.5

x1

1 -1

Fig. 2. PLS kernel without Fig. 3. PLS kernel with Fig. 4. PLS kernel with two latent variables any latent variable one latent variable

284

J. Hayami and K. Yamauchi

 1 c is calculated by using Eq(21). Thus, the shape of the PLS kernel is the weighted sum of the hyper sphere and hyper ellipse reflecting the latent variables. The weight for the hyper sphere λ(t) is large in the early steps of the learning, but it gradually decreases to zero, whereas the weight for the hyper ellipse 1 − λ(t) gradually increases to 1. The kc determines the initial variance of the hyper sphere, t˜ denotes the averaged fitness value required for the weights for the 1st and 2nd terms to become 0.5, and ID denotes D dimensional unit matrix. Thus, the modified PLS kernel is   ¯ c )Σ∗c (x − x ¯ c )T (x − x g(x|c) = exp − (18) 2 4.4

Online DAEM Algorithm

The PLS mixture model is a variation of the mixture of experts(ME). There are various learning methods suitable for learning the ME. In this method, we selected a variation of the EM algorithm as the learning method because its convergence speed is fast. Although the PLS mixture model is not a statistical model, we can apply an online EM algorithm for learning the PLS mixture model. An online EM algorithm for NGnet parameter estimation, which has a similar structure to our model, has been proposed by Ishii et al. [4]. Generally speaking, however, the quality of the resultant parameters is affected by their initial parameter values. Ueda et al. proposed the Deterministic Annealing EM (DAEM) algorithm to overcome this problem[9]. The original DAEM algorithm, however, was not an online learning algorithm. To apply DAEM to our model, we must extend it to an online learning algorithm. The following text explains the online DAEM learning algorithm for a mixture of PLS models. The covariance matrix of x and y, ΣXy c , the covariance matrix ¯ c , and the average of y, y¯c are modified by the of x, Σc , the average vector of x, x online DAEM algorithm. The online DAEM algorithm executes E- and M-steps alternately to updating parameters every time it receives one instance. E-step In the E-step, the respective ratio of eachunit to the i-th instance (current instance) is calculated. Let p c|xi , yi , θ(i−1) be the respective ratio of the c-th unit to the i-th instance.   g (xi |c)τ yc (xi , yi ) p c|xi , yi , θ(t) = C , (19)  τ g (xi |m) ym (xi , yi ) m=1

where τ denotes inverted temperature and is gradually increased as τ ← τ · β (β > 1) every time it receives one instance1 . yc (xi , yi ) denotes a suitability of the c-th linear unit for the current input (xi , yi ): 1

Although PLS kernel function is not probability density function and Eq(19) is also not precise probability density function, they have similar properties as that of probability density function.

PLS Mixture Model for Online Dimension Reduction



yc (xi , yi ) = exp −(fc (x) − yi )2 /(2σ 2 )

285

(20)

M-step In the M-step, the parameters are updated using the value calculated in the E  (t) (t) step. Let pc be the respective value of module c at time t pc = p c|xt , yt , θ(t) (t)

and let  x c be the approximated average of x for the module c. If pc > θlearning , the c-th unit executes below procedure. (t−1)  x (t) +p(t) c = ρ  x c c xt ,

(21)

where ρ denotes the forgetting coefficient and 0  ρ ≤ 1. Then, the parameters that maximize the likelihood for incomplete data are Σc =  xxT c /  1 c −  x c  x Tc /  1 2c ΣXy c =  xy T c /  1 c −  x c  y Tc /  1 2c xc =  x c /  1 c ,

y c = y c /  1 c

(22) (23) (24)

Note that the notation ‘t’ is omitted in the above equations. Thus, xc , yc , Σc and ΣXy c are derived. From Σc and ΣXy c , latent variables are derived by repeating Eqs.(8) and (9). 4.5

Unit Manipulation

Each unit is allocated adaptively. If the suitabilities of all units for a new learning sample i are less than a threshold, a new unit is allocated. Therefore, if the condition (∀c) (p (xi , yi , c|θ) < )

(25)

is satisfied, a new unit N + 1 is allocated: xN +1 = xi ,

y N +1 = yi ,

Σ∗N +1 = kN +1 ID ,

(26)

where kN +1 is the parameter used in Eq.(17) and denotes a small positive threshold. Note that the initial output function of the new allocated linear model is f (x) = y¯. Although unit pruning or merging strategies can also be applied to the PLS mixture model, we do not use such strategies in this paper to show the effect of the PLS clearly.

5

Experiments

To examine the performances of the proposed system, we tested it using two datasets: ones including irrelevant and correlated dimensions.

286

5.1

J. Hayami and K. Yamauchi

Performances of Dataset Including Irrelevant Dimensions

The proposed PLS mixture model is good at reducing variables in the case of there are several correlated dimensions. However, it could also reduce irrelevant dimensions if its parameters were optimized. To verify this property, we tested our model using the following synthetic dataset. y = cos (2(x0 + x1 )) ,

(27)

where x0 and x1 are independent from each other. We set both x0 and x1 to uniform random values in the intervals 0 ≤ x0 ≤ 1, 0 ≤ x1 ≤ 1, respectively.

0.2

Fig. 6. PLS kernels after 5000 observations

RMSE(NG-OEM) RMSE(PLS-ODAEM) Number of Models(NG-OEM) Number of Models(PLS-ODAEM)

0.18 0.16

RMSE

0.14

80 70 60 50

0.12 40 0.1 30

0.08

Number of Models

Fig. 5. PLS kernels after 1000 observations

20

0.06

10

0.04 0.02 0

5000

0 10000

step

Fig. 7. RMSE, number of units versus number of iterations

PLS Mixture Model for Online Dimension Reduction

287

Note that x0 and x1 can be converted to 1 1 t0 = √ x0 + √ x1 , 2 2

1 1 t1 = √ x0 − √ x1 . 2 2

(28)

From these equations, we can see that t0 is related to y but t1 is not. Therefore, y is one dimensional function in nature. PLS-kernel distributions after 1000 and 5000 observations are shown in Figs. 5 and 6. We can see that, in the early steps of the learning, the PLS kernels was shaped like a sphere like shape, but they became ellipse shaped in the latter steps of the learning. In particular, the widths along t0 became narrow, while those along t1 became wide. This means that the kernels are forced to ignore irrelevant t1 . Using this strategy, the model achieves a high generalization ability. 5.2

Performances for a Multicollinearity Dataset

To verify the effect of PLS kernels, we tested the proposed model using multicollinearity dataset. This is a 10 dimensional dataset including two essential dimensions x0 and x1 , which have following relation to y.       

y = max exp −10x0 2 , exp −50x1 2 , 1.25 exp −5 x0 2 + x1 2 , (29) where x0 and x1 are independent of each other. We set them to uniform random values in the intervals −1 ≤ x0 ≤ 1, −1 ≤ x1 ≤ 1, respectively. The other dimensions correlate with x0 and x1 as follows. 1 x2 = x0 , x4 = √ (x0 − x1 ), x6 = 2 1 x3 = x1 , x5 = √ (x0 + x1 ), x7 = 2

1 1 √ (x0 − 2x1 ), x8 = √ (2x0 − x1 ), 5 5 1 1 √ (x0 + 2x1 ), x9 = √ (2x0 + x1 ) 5 5

We also compared the performance of our model with that of an NGnet using online EM algorithm (NG-OEM)[4] without pruning, merging, or dividing strategies2 . For these two models, we performed five trials of 10000 iterations, and their root mean square errors (RMSEs) were averaged over the five trials. The number of units was also examined in the same manner as the RMSE. The performances of PLS-ODAEM (our model) and NG-OEM are shown in Fig. 7. We can see that the number of units was smaller for PLS-ODAEM than for NG-OEM. The convergence speed of the RMSE of PLS-ODAEM was slower than that of NG-OEM, but the ultimate RMSE of PLS-ODAEM was less than that of NG-OEM. 2

There are cases that the pruning and merging strategies also bring the dimension reduction effect. To investigate PLS effect clearly, we examined the performances without the pruning and merging strategies.

288

6

J. Hayami and K. Yamauchi

Conclusion

In this paper, we proposed a PLS mixture model using an online DAEM algorithm. The new model reduces the number of input dimensions using the partial least squares method (PLS). The PLS was applied to both the linear model and the Gaussian kernel and the parameters were optimized by the online DAEM algorithm. Actually, this model does not satisfy the condition for the statistical model so far. However, if we restrict λ(t)c in Eq(17) larger than 0, we will be able to use p(x|c) = g(x|c)/ g(x|c)dx instead of g(x|c) to make our method satisfy the condition. This model can be applied not only to a multicollinearity dataset but also to datasets that include irrelevant variables. We plan to improve the model by adding pruning, merging and dividing strategies.

References 1. Wold, H.: Soft modeling by latent variables; the nonlinear iterative partial least squares approach. In: Gani, J. (ed.) Perspectives in Probability and Statistics, Papers in Honour of M. S Bartlett, pp. 520–540. Academic Press, Inc., London (1975) 2. Schaal, S., Atkeson, C.G.: Constructive incremental learning from only local information. Neural Computation 10(8), 2047–2084 (1998) 3. Vijayakumar, S., Souza, A.D., Schaal, S.: Incremental online learning in high dimensions. Neural Computation 17, 2602–2634 (December) 4. Sato, M.-a., Ishii, S.: On-line em algorithm for the normalized gaussian network. Neural Computation 12, 407–432 (2000) 5. Manne, R.: Analysis of two partial-least-squares algorithms for multivariate calibration. Chemometrics and Intelligent Laboratory Systems 22, 187–197 (1987) 6. Lindgren, S.W.F., Geladi, P.: The kernel algorithm for pls. Journal of Chemometrics 7(1), 45–59 (1993) 7. Ter Braak, C.J.F., De Jong, S.: Comments on the pls kernel algorithm. Journal of Chemometrics 8(2), 169–174 (1994) 8. Xu, L., Jordan, M.I., Hinton, G.E.: An alternative model for mixtures of experts. In: Tesauro, G., Touretzky, D., Leen, T. (eds.) Advances in Neural Information Processing Systems, vol. 7, pp. 633–640. The MIT Press, Cambridge (1995) 9. Ueda, N., Nakano, R.: Deterministic annealing variant of the em algorithm. Advances in Neural Information Processing System 7, 545–552 (1995)

Analysis on Bidirectional Associative Memories with Multiplicative Weight Noise Chi Sing Leung1 , Pui Fai Sum2 , and Tien-Tsin Wong3 1

2

Department of Electronic Engineering, City University of Hong Kong [email protected] Institute of Electronic Commerce, National Chung Hsing University Taiwan 3 Department of Computer Science and Engineering, The Chinese University of Hong Kong

Abstract. In neural networks, network faults can be exhibited in different forms, such as node fault and weight fault. One kind of weight faults is due to the hardware or software precision. This kind of weight faults can be modelled as multiplicative weight noise. This paper analyzes the capacity of a bidirectional associative memory (BAM) affected by multiplicative weight noise. Assuming that weights are corrupted by multiplicative noise, we study how many number of pattern pairs can be stored as fixed points. Since capacity is not meaningful without considering the error correction capability, we also present the capacity of a BAM with multiplicative noise when there are some errors in the input pattern. Simulation results have been carried out to confirm our derivations.

1

Introduction

Associative memories have a wide range of applications including content addressable memory and pattern recognition [1,2]. An important feature of associative memories is the ability to recall the stored patterns based on partial or noisy inputs. One form of associative memories is the bivalent additive bidirectional associative memory (BAM) [3] model. There are two layers, FX and FY , of neurons in a BAM. Layer FX has n neurons and layer FY has p neurons. A BAM is used to store pairs of bipolar patterns, (X h , Y h )’s, where h = 1, 2, · · · , m; X h ∈ {+1, −1}n; Y h ∈ {+1, −1}p; and m is the number of patterns stored. We shall refer to these patterns pairs as library pairs. The recall process is an iterative one starting with a stimulus pair (X (0) , Y (0) ) in FX . After a number of iterations, the patterns in FX and FY should converge to a fixed point which is desired to be one of the library pairs. BAM has three important features [3]. Firstly, BAM can perform both heteroassociative and autoassociative data recalls: the final state in layer FX represents the autoassociative recall, while the final state in layer FY represents the heteroassociative recall. Secondly, the initial input can be presented in one of the two layers. Lastly, BAM is stable during recall. In other words, for any connection matrix, a BAM always converges to a stable state. Several methods have been proposed to improve its capacity [4,5,6,7,8]. M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 289–298, 2008. c Springer-Verlag Berlin Heidelberg 2008 

290

C.S. Leung, P.F. Sum, and T.-T. Wong

Although the capacity of BAM has been intensively studied with a perfect laboratory environment consideration[9,10,11,12,13], practical realization of BAM may encounter the problem of inaccuracy in the stored weights. All the previous studies assume that the stored weights matrix is noiseless. However, this is not always the case when training a BAM for some real applications. One kind of weight faults is due to the hardware or software precision [14,15]. For example, in the digital implementation, when we use a low precision floating point format, such as 16-bit half-float[16], to represent trained weights, truncation errors will be introduced. The magnitude of truncation errors is proportional to that of the trained weights. Hence, truncation errors can be modelled as multiplicative weight noise[17,18]. This paper focuses on the quantitative impact of multiplicative weight noise to the BAM capacity. We will study how many number of pattern pairs can be stored as fixed points when multiplicative weight noise presents. Since capacity is not meaningful without considering the error correction capability, we also present the capacity of BAM with multiplicative noise when there are some errors in the input pattern. The rest of this paper is organized as follows. Section 2 introduces the BAM model and multiplicative weight noise. In section 3, the capacity analysis on BAM with multiplicative weight noise is used. Simulation examples are given in Section 4. Then, we conclude our work in Section 5.

2 2.1

BAM with Multiplicative Weight Noise BAM

The BAM, as proposed by Kosko [3], is a two-layer nonlinear feedback heteroassociative memory in which m library pairs (X 1 , Y 1 ), · · · , (X m , Y m ) are stored, where X h ∈ {−1, 1}n and Y h ∈ {−1, 1}p. There are two layers of neurons in BAM; layer FX has n neurons and layer FY has p neurons. The connection matrix between the two layers is denoted as W . The encoding equation, as proposed by Kosko, is given by W =

m 

Y h X hT

(1)

h=1

which can be rewritten as wji =

m 

xih yjh ,

(2)

h=1

where Xh = (x1h , x2h , · · · , xnh )T and Yh = (y1h , y2h , · · · , yph )T . The recall process employs interlayer feedback. An initial pattern X (0) presented to FX is passed through W and is thresholded, and a new state Y (1) in FY is obtained which is then passed back through W T and is thresholded again, leading to a new state X (1) in FX . The process repeats until the state of BAM converges. Mathematically, the recall process is:     Y (t+1) = sgn W X (t) , and X (t+1) = sgn W T Y (t+1) , (3)

Analysis on BAM with Multiplicative Weight Noise

291

where sgn(·) is the sign operator: ⎧ x>0 ⎨ +1 x<0 . sgn(x) = −1 ⎩ state unchanged x = 0 Using an element-by-element notation, the recall process can be written as: ⎛ ⎞  n p   (t+1) (t) (t+1) (t+1) ⎠ yi = sgn wji xi , and xj = sgn ⎝ wji yi , (4) i=1 (t)

j=1 (t)

where xi is the state of the ith FX neuron and yj is the state of the jth FY neuron. The above bidirectional process produces a sequence of pattern pairs (X (t) , Y (t) ): (X (1) , Y (1) ), (X (2) , Y (2) ), · · ·. This sequence converges to one of the fixed points (X f , Y f ) and this fixed point ideally should be one of the library pairs or nearly so. A fixed point (X f , Y f ) has the following properties: Y f = sgn(W X f ) and X f = sgn(W T Y f ).

(5)

Hence, a library pair can be retrieved only if it is a fixed point. One of the advantages of Kosko’s encoding method is the ability of incremental learning, i.e., the ability of encoding new library pairs to the model based on the current connection matrix only. With the Kosko’s encoding method, BAM can only min(n,p) correctly store up to 2 log min(n,p) library pairs. When the number of library pairs exceeds that value, a library pair may not be stored as a fixed point. 2.2

Multiplicative Weight Noise

In some electrical circuits, inaccuracy occurs in the implementation of trained weights. Errors in the weights may be caused by quantization error due to limited bits used to store the trained weights, or percentage error due to voltage perturbation. In the digital implementation, such as DSP and FPGA, the trained weights are usually stored as floating-point format. Floating-point representation of real numbers is more desirable than integer representation because floatingpoint provides large dynamic range. When we use a low precision floating point format, such as 16-bit half-float[16], to represent trained weights, truncation errors will be introduced. The magnitude of the truncation errors is proportional to that of the trained weights[19]. Hence, the truncation errors can be modelled as multiplicative weight noise. An implementation of a weight wji is denoted by w ˜ji . In multiplicative weight noise, each implemented weight deviates from its nominal value by a random percent, i.e., w ˜ji = wji + bji wji (6) where bji ’s are identical independent mean zero random variables with variance σb2 . The density function of bji ’s is symmetrical.

292

3 3.1

C.S. Leung, P.F. Sum, and T.-T. Wong

Analysis on BAM with Multiplicative Weight Noise Capacity

We will investigate the BAM’s memory capacity when the multiplicative weight noise presented. The following assumptions and notations are used. – The dimensions, n and p, are large. Also, p = rn, where r is a positive constant. – Each component of library pairs (X h , Y h )’s is a ±1 equiprobable independent random variable.  n – EUj,h is the event that i w ˜ji xih is equal to yjh (the j-th component of the library pattern Y h ). Also, EU j,h is the complement event of EUj,h . p – EVi,h is the event that j w ˜ji yjh is equal to xih (the i-th component of the library pattern X h ). Also, EV i,h is the complement event of EVi,h . With the above assumptions and the multiplicative weight noise, we will introduce Lemma 1 and Lemma 2. They will assist us to derive the capacity of BAM with multiplicative weight noise. Lemma 1: The probability Prob(EU j,h ) is approximately equal to   n Q (1 + σb2 )m for j = 1, · · · , n and h = 1, · · · , m, where Q(z) =

√1 2π

∞ z

exp( −z2 )dz. 2

 Proof: Event EU j,h means that sgn ( ni=1 w ˜ji xih )  = yjh . From (2) and (6), we have n 

w ˜ji xih =

i=1

n 

wji (1 + bj i)xih =

n  m  ( yjh xih )(1 + bj i)xih

i=1

i=1

h

n  m n  m   = nyjh + ( yjh xih )xih + ( yjh xih )bji xih i=1 h =h

(7)

i=1 h =1

Without loss of generality, we consider the library pair (X h , Y h ) having all components positive: X h = (1, . . . , 1)T and Y h = (1, . . . , 1)T . This consideration is usually used [13] and does not affect our results. We can easily verify this by use of conditional probability. Now,(7) becomes n 

w ˜ji xih = n +

i=1

=n+

n m n  m    ( yjh xih ) + ( yjh xih )bji i=1 h =h n 

n 

i=1

i=1

αji +

(8)

i=1 h =1

βji ,

(9)

Analysis on BAM with Multiplicative Weight Noise

293

where αji ’s are independent identical zero mean random variables (i.e., E[αji ] = 0, and E[αji αji ] = 0 for i  = i ) and the variance of αji ’s, denoted as Var[αji ], is equal to (m−1). Since bji ’s are independent identical zero mean random variables  and they are independent of ( m h =1 yjh xih )’s. Hence, βji ’s are independent identical zero mean random variables, where E[βji ] = 0, E[βji βji ] = 0 for i = i ), Var[βji ] = σb2 m. n n For large n, the summations, i=1 αji and i=1 βji , tend to normal with variances equal to (m − 1)n and σb2 mn,respectively. Besides, the sum of the two normal random variables is still a normal random variable. Hence, (9) becomes n 

w ˜ji xih = n +

i=1

n  i=1

αji +

n 

βji = n + α + β .

(10)

i=1

 Note that βji ’s are independent of ( m h =h yjh xih )’s. We have E[αβ] = 0, E[α+ 2 β] = 0 and Var[α + β] = (m − 1)n + σ that α + β <−n. b mn.  Event EUj,h means  n n Hence, Prob(EUj,h ) ≈ Q . For large m, Q ≈ (m−1)+σb2 m (m−1)+σb2 m   n Q . (Proof completed). (1+σ2 )m b

Using the similar way, we can have Lemma 2. Lemma 2: The probability Prob(EVi,h ) is approximately equal to   p Q (1 + σb2 )m for i = 1, · · · , p and h = 1, · · · , m. Now, we start to estimate the capacity. Let the probability that a library pair (X h , Y h ) is fixed point be P∗ : P∗ = Prob (EU1h ∩ · · · ∩ EUnh ∩ EV1h ∩ · · · ∩ EUph )   = 1 − Prob EU 1h ∪ · · · ∪ EU nh ∪ EV 1h ∪ · · · ∪ EV ph ≥ 1 − pProb(EU jh ) − nProb(EV ih ) .

(11)

From Lemmas 1 and 2, (11) becomes     n p P∗ ≥ 1 − pQ − nQ . (1 + σb2 )m (1 + σb2 )m Letting PB = pQ



 n (1+σb2 )m

and PA = nQ



P∗ ≥ 1 − PB − PA .

p (1+σb2 )m

(12)

 , we get (13)

294

C.S. Leung, P.F. Sum, and T.-T. Wong

If z is large,

 2  z 1 Q(z) ≈ exp − − log z − log 2π , 2 2

(14)

which is quite accurate for z > 3. Using the approximation (14),   n 1 n 1 PA = exp log p − − log − log 2π 2(1 + σb2 )m 2 (1 + σb2 )m 2   n 1 n 1 = exp log r + log n − − log − log 2π 2(1 + σb2 )m 2 (1 + σb2 )m 2   n 1 n = exp log n − − log + constant (15) 2(1 + σb2 )m 2 (1 + σb2 )m Clearly, if m < 2(1+σn2 ) log n , PA tends zero as n tends infinity. Similarly, we can b get that as p → ∞ and m < 2(1+σp2 ) log p , PB → 0. To sum up, for large n and b p, If min(n, p) m< (16) 2(1 + σb2 ) log min(n, p) then P∗ → 1. That means if the number m of library pairs is less than min(n,p) , a library pair is with a very high chance to be a fixed point. 2(1+σb2 ) log min(n,p) So the capacity of BAM with multiplicative weight noise is equal to min(n,p) . 2(1+σ2 ) log min(n,p) b

3.2

Error Correction

In this section, we will investigate the capacity of BAM with weight noise when the initial input is a noise version Xhnoise of a library pattern X h . Let Xhnoise contains ρn bit errors, where ρ is the input noise level. If ˜ X noise Y h = sgn(W ) h T ˜ X h = sgn(W Y h ) ˜ X h) , Y h = sgn(W

(17) (18) (19)

then a noise version X noise can successfully recall the correct the desire library h pair (X h , Y h ). Similarly, we hope that a noise version Y noise of Y h can such cessfully recall the correct the desire library pair (X h , Y h ). We will study under what condition of m, the probability of successful recall tends to one. noise Define EUj,h be the event that ni w ˜ji xnoise is equal to yjh (the j-th comih noise

ponent of the library pattern Y h ). Also, EU j,h is the complement event of  noise noise noise EUj,h . Also, define EVi,h be the event that pj w ˜ji yjh is equal to xih noise

(the i-th component of the library pattern X h ). Also, EV i,h is the complenoise ment event of EVi,h . With the above definition, we can following the proof of Lemma 1 to get the following two lemmas.

Analysis on BAM with Multiplicative Weight Noise

295

noise

Lemma 3: The probability Prob(EU i,h ) is approximately equal to  Q

(1 − 2ρ)n (1 + σb2 )m



for i = 1, · · · , p and h = 1, · · · , m. noise

Lemma 4: The probability Prob(EV i,h ) is approximately equal to  Q

(1 − 2ρ)p (1 + σb2 )m



for i = 1, · · · , p and h = 1, · · · , m. Define P∗∗ be the probability that a noise version with ρ fraction of errors can recall the desired library pair. It is not difficult to show that     noise noise P∗∗ ≥ 1−p Prob(EU jh ) + Prob(EU jh ) −n Prob(EV ih + Prob(EV ih )) . (20) From Lemmas 1-4, To sum up, for large n and p, if m<

(1 − 2ρ) min(n, p) 2(1 + σb2 ) log min(n, p)

(21)

then P∗∗ → 1. That means, when there are ρn (or ρp) bit errors in the initial input, the capacity of BAM with multiplicative weight noise is equal to (1−2ρ) min(n,p) . 2(1+σ2 ) log min(n,p) b

4 4.1

Simulation Capacity

We consider two dimensions, 512 and 1024. For each m, we randomly generate 1000 sets of library pairs. The Kosko’s rule is then used to encode the matrices. Afterwards, we add the multiplicative weight noise to the matrices. The variances σb2 of weight noise are set to 0, 0.2, 0.4. Figure 1 shows the percentage of a library pair being successfully stored. From the figure, as the variance of weight noise increases, the successful rate decreases. This phenomena agrees with our expectation. From our analysis, i.e., (16), for n = p = 512, a BAM can store up to 41, 34, and 29 pairs for σb2 equal to 0, 0.2 and 0.4, respectively. From Figure 1(a), all the corresponding successful rates are very large. Also, there are a sharply decreasing changes in successful for {m > 41, σb2 = 0}, {m > 34, σb2 = 0.2}, and {m > 29, σb2 = 0.4}.

296

C.S. Leung, P.F. Sum, and T.-T. Wong 1

1 σ2b=0 σ2b=0.2 σ2b=0.4

2

σb =0 2 σb =0.2 2 σb =0.4

0.9 0.8

0.9 0.8 0.7 Successful Rate

Successful Rate

0.7 0.6 0.5 0.4

0.6 0.5 0.4

0.3

0.3

0.2

0.2

0.1 0 0

0.1 10

20

30 40 50 60 m : Number of library pairs

70

0 0

80

20

(a) n = p = 512

40

60 80 100 120 m: Number of library pairs

140

160

(b) n = p = 1024

Fig. 1. Successful rate of a library pair being a fixed point. (a) The dimension is 512. (b) The dimension is 1024. For each value of m, we generate 1000 sets of library pairs.

1

0.8

0.7 0.6 0.5 0.4 0.3

0.7 0.6 0.5 0.4 0.3

0.2

0.2

0.1

0.1

0 0

10

20

30 40 50 60 m : Number of library pairs

(a) weight noise level

σb2

70

= 0.2

ρ=0.03125 ρ=0.0625 ρ=0.125

0.9

Successful Recall Rate

0.8 Sucessful Recall Rate

1

ρ=0.03125 ρ=0.0625 ρ=0.125

0.9

80

0 0

10

20

30 40 50 60 m : Number of library pairs

(b) weight noise level

σb2

70

80

= 0.4

Fig. 2. Successful recall rate from a noise input. The dimension is 512. For each value of m, we generate 1000 sets of library pairs. For each library pattern, we generate 10 noise versions.

Similarly, from (16), for n = p = 1024, a BAM can store up to 73, 61, and 52 pairs for σb2 equal to 0, 0.2 and 0.4, respectively. From Figure 1(b), all the corresponding successful rates are also large. Also, there are a sharply decreasing changes in successful for {m > 73, σb2 = 0}, {m > 61, σb2 = 0.2}, and {m > 52, σb2 = 0.4}. To sum up, the simulation result is consistent with our analysis (16). 4.2

Error Correction

The dimension is 512. We consider two weight noise levels, σb2 = 0.2, 0.4 ,and three input error levels, ρ = 0.003125, 0.0625, 0.125. For each m, we randomly generate 1000 sets of library pairs. The Kosko’s rule is then used to encode the matrices. Afterwards, we add the multiplicative weight noise to the matrices. For each library pair, we generate ten noise versions. We then feed the noise versions as initial input and check whether the desire library can be recalled

Analysis on BAM with Multiplicative Weight Noise 1

0.8

0.7 0.6 0.5 0.4 0.3 0.2

0.7 0.6 0.5 0.4 0.3 0.2

0.1 0 0

ρ=0.015625 ρ=0.03125 ρ=0.0625

0.9

Sucessful Recall Rate

0.8 Sucessful Recall Rate

1

ρ=0.015625 ρ=0.03125 ρ=0.0625

0.9

297

0.1 20

40 60 m: Number of library pairs

80

(a) weight noise level σb2 = 0.2

100

0 0

20

40 60 m: Number of library pairs

80

100

(b) weight noise level σb2 = 0.4

Fig. 3. Successful recall rate from a noise input. The dimension is 1024. For each value of m, we generate 1000 sets of library pairs. For each library pattern, we generate 10 noise versions.

or not. Figures 2 and 3 shows the successful recall rate. From the figures, as the input error level ρ increases, the successful rate decreases. This phenomena agrees with our expectation. From our analysis, i.e., (21), for the dimension n = p = 512 and weight noise level σb2 = 0.2, a BAM can store up to 32, 30, and 26 pairs for the input error level ρ equal to 0.03125, 0.0625 and 0.125, respectively. From Figure 2(a), all the corresponding successful rates are large. Also, there are a sharply decreasing changes in successful recall rates for {m > 32, ρ = 0.03125}, {m > 30, ρ = 0.0625}, and {m > 26, ρ = 0.125}. For other weight noise levels and dimension, we obtained similar phenomena.

5

Conclusion

We have examined the statistical storage behavior of BAM with multiplicative weight noise. The capacity of a BAM is m < 2(1+σmin(n,p) . When the num2 b ) log min(n,p) ber of library pairs is less that value, the chance of it being a fixed point is very high. Since we expect BAM has certain error correction ability, we have investigated the capacity of BAM with weight noise when the initial input is (1−2ρ) min(n,p) a noise version of a library pattern. We show that if m < 2(1+σ ,a 2 b ) log min(n,p) noise version with ρn (or ρp) errors has a high chance to recall the desire library pairs. Computer simulations have been carried out The results presented here can be extended to Hopfield network. By adopting the approach set above, we can easily obtain the result in Hopfield network by replacing min(n, p) with n in the above equations.

Acknowledgement The work is supported by the Hong Kong Special Administrative Region RGC Earmarked Grants (Project No. CityU 115606) and (Project No. CUHK 416806).

298

C.S. Leung, P.F. Sum, and T.-T. Wong

References 1. Kohonen, T.: Correlation matrix memories. IEEE Transaction Computer 21, 353– 359 (1972) 2. Palm, G.: On associative memory. Biolog. Cybern. 36, 19–31 (1980) 3. Kosko, B.: Bidirectional associative memories. IEEE Trans. Syst. Man, and Cybern. 18, 49–60 (1988) 4. Leung, C.S.: Encoding method for bidirectional associative memory using projection on convex sets. IEEE Trans. Neural Networks 4, 879–991 (1993) 5. Leung, C.S.: Optimum learning for bidirectional associative memory in the sense of capacity. IEEE Trans. Syst. Man, and Cybern. 24, 791–796 (1994) 6. Wang, Y.F., Cruz, J.B., Mulligan, J.H.: Two coding strategies for bidirectional associative memory. IEEE Trans. Neural Networks 1, 81–92 (1990) 7. Lenze, B.: Improving leung’s bidirectional learning rule for associative memories. IEEE Trans. Neural Networks 12, 1222–1226 (2001) 8. Shen, D., Cruz, J.B.: Encoding strategy for maximum noise tolerance bidirectional associative memory. IEEE Trans. Neural Networks 16, 293–300 (2005) 9. Leung, C.S., Chan, L.W.: The behavior of forgetting learning in bidirectional associative memory. Neural Computation 9, 385–401 (1997) 10. Leung, C.S., Chan, L.W., Lai, E.: Stability and statistical properties of secondorder bidirectional associative memory. IEEE Transactions on Neural Networks 8, 267–277 (1997) 11. Wang, B.H., Vachtsevanos, G.: Storage capacity of bidirectional associative memories. In: Proc. IJCNN 1991, Singapore, pp. 1831–1836 (1991) 12. Haines, K., Hecht-Nielsen, R.: A bam with increased information storage capacity. In: Proc. of the 1988 IEEE Int. Conf. on Neural Networks, pp. 181–190 (1988) 13. Amari, S.: Statistical neurodynamics of various versions of correlation associative memory. In: Proc. of the 1988 IEEE Int. Conf. on Neural Networks, pp. 181–190 (1988) 14. Burr, J.: Digital neural network implementations. In: Neural Networks, Concepts, Applications, and Implementations, vol. III, Prentice Hall, Englewood Cliffs, New Jersey (1991) 15. Holt, J., Hwang, J.-N.: Finite precision error analysis of neural network hardware implementations. IEEE Transactions on Computers 42(3), 281–290 (1993) 16. Lam, P.M., Leung, C.S., Wong, T.T.: Noise-resistant fitting for spherical harmonics. IEEE Transactions on Visualization and Computer Graphics 12(2), 254–265 (2006) 17. Bernier, J.L., Ortega, J., Rodriguez, M.M., Rojas, I., Prieto, A.: An accurate measure for multilayer perceptron tolerance to weight deviations. Neural Processing Letters 10(2), 121–130 (1999) 18. Bernier, J.L., Diaz, A.F., Fernandez, F.J., Canas, A., Gonzalez, J., Martin-Smith, P., Ortega, J.: Assessing the noise immunity and generalization of radial basis function networks. Neural Processing Letters 18(1), 35–48 (2003) 19. Sripad, A., Snyder, D.: Quantization errors in floating-point arithmetic. IEEE Transactions on Speech, and Signal Processing 26, 456–463 (1978)

Fuzzy ARTMAP with Explicit and Implicit Weights Takeshi Kamio1 , Kenji Mori1 , Kunihiko Mitsubori2 , Chang-Jun Ahn1 , Hisato Fujisaka1 , and Kazuhisa Haeiwa1 1

Dept. of Systems Engineering, Hiroshima City University, 3-4-1, Ozuka-higashi, Asaminami-ku, Hiroshima-shi, Hiroshima, 731-3194, Japan 2 Dept. of Electronics and Computer Systems, Takushoku University, 815-1, Tatemachi, Hachioji-shi, Tokyo, 193-0985, Japan [email protected]

Abstract. ARTMAP is one of the famous supervised learning systems. Many learning methods for ARTMAP have been proposed since it was devised as a system to solve Stability-Plasticity Dilemma. AL-SLMAP was implemented by slightly modifying FCSR which was the original learning method for fuzzy ARTMAP (FAM). Although AL-SLMAP can solve pattern recognition problems in a noisy environment more effectively than FCSR, AL-SLMAP is less suitable for region classification problems than FCSR. This means that AL-SLMAP has some problems which do not exist in FCSR. In this paper, we propose a learning method for FAM with explicit and implicit weights to overcome the problems. Keywords: ARTMAP, FAM, Explicit weight, Implicit weight.

1

Introduction

Adaptive resonance theory neural network (ART) is an unsupervised learning system which can generate and grow categories by comparing the similarity between an input and memories with the fineness of classification called “vigilance parameter”. In contrast, ARTMAP is a supervised learning system which consists of a learning ART (ARTa ), a supervising ART (ARTb ) and a map field (MF). ARTa and ARTb classify sample data and recognition codes respectively. MF maps each category in ARTa to the corresponding one in ARTb . An erroneous mapping is corrected by the process called “match tracking (MT)”. Many learning methods for ART and ARTMAP have been proposed since they were devised as systems to solve the Stability-Plasticity Dilemma. Here, we discuss learning methods for fuzzy ARTMAP (FAM) [1] which can classify not only binary but also analog inputs. Fast commit slow recode (FCSR) [1] is the original learning method for FAM. Since FCSR has little ability to reduce the influence of the input noise, it may generate many categories unnecessarily and decrease the recognition performance. To solve this category proliferation problem, J. S. Lee et al. proposed AL-SLMAP [2] which is the combination of average learning (AL) [2] and slow M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 299–308, 2008. c Springer-Verlag Berlin Heidelberg 2008 

300

T. Kamio et al.

learning of map field weights vector (SLMAP) [3]. Also, they reported that AL-SLMAP could solve the character recognition problem in a noisy environment more effectively than FCSR [2]. We have investigated the ability of AL-SLMAP in detail since we were very interested in it from the viewpoint of simplicity and performance. As a result, we have confirmed that AL-SLMAP cannot inhibit the category proliferation for the character recognition problem in a highly noisy environment [4] and that AL-SLMAP is less suitable for the region classification problem than FCSR. These facts made us realize that AL-SLMAP had problems in category selection, weight update, and MT. In this paper, we propose a novel FAM model and its learning method. Each node of ARTa has a weight set for each individual recognition code. When a new node is created in ARTa , all its weight sets define its category. That is to say, they work as explicit weights. As the learning proceeds, each node of ARTa has weight sets which do not define its category. Such weight sets are called implicit weights. Our learning method uses both explicit and implicit weights to solve the problems in AL-SLMAP. Finally, it has been shown by simulations that our learning method is better than FCSR and AL-SLMAP.

2 2.1

Fuzzy ARTMAP Learned by AL-SLMAP System

Here, we review fuzzy ARTMAP (FAM) [1] and AL-SLMAP [2]. First, we explain the behavior of fuzzy ART. As shown in Fig.1, it has an attentional subsystem (AS) and an orienting subsystem (OS). AS consists of an input layer (F0 ), a matching layer (F1 ) and a category layer (F2 ). If a normalized original input a ∈ [0, 1]n is given, F0 provides F1 with A ∈ [0, 1]2n . A is the complement code of a (i.e., A ≡ [a,ac ] = [a1 , · · · , an , 1 − a1 , · · · , 1 − an ]). After A goes through F1 and reaches F2 , F2 node j calculates the choice strength Tj : Tj = |A ∧ U j | /(α + |U j |), (j = 1, · · · , m), (1)  where (p ∧ q)i = min(pi , qi ), |p| = |pi |, U j ∈ 2n is the bottom-up weight vector, α > 0 is the choice parameter and m is the number of F2 nodes. The node J with the maximal choice strength is activated in F2 . If several nodes have the maximal choice strength, the node with the minimal index is selected from them. F2 activity y ∈ m satisfies yJ = 1 and yj =J = 0. The activated F2 node J provides F1 with D J ∈ 2n which is the same as the top-down weight vector, and then F1 activity x becomes A ∧ DJ . Then, OS calculates the matching degree SJ from A and x: SJ = |A ∧ D J | / |A| .

(2)

In addition, OS compares SJ with the vigilance parameter ρ ∈ [0, 1]. If SJ ≥ ρ, the node J has the category for the present input a. The category is defined by U J and D J . If SJ < ρ, the node J is reset. The reset node J remains inactivated

Fuzzy ARTMAP with Explicit and Implicit Weights

j

1

Tj

J

m

U j , Dj

F1

DJ

i

1

㪼 㫅㪺 㫅㪸 㫊㫆 㪩㪼 㫋㪆 㫊㪼 㪩㪼

F2 y

301

ρ

2n x

A F0

i

1

2n A

AS

OS

(a,a c)

Fig. 1. Fuzzy ART

Fab MF

mb

WJ

ab

yb

K k 1

Wj ab

F2 b

ab

x F2a

1

j

J

r ab

ma

1

k

K

mb

yb T ja F1a

Uj a, D ja 1

ia

D Ja

ρa

2na

x

A F0a

MT

1

ARTa

ia

F 1b

U kb, Dkb 1

ib

DKb

xb

B F 0b

A

ρb

2nb

a

2na

(a,a c)

Tkb

1

ARTb

ib

2nb

B (b,b c)

Fig. 2. Fuzzy ARTMAP (FAM)

until the input a changes. The above processes are iterated until ART finds an F2 node with SJ ≥ ρ. If all F2 nodes are reset in a learning period, a new node is added to F2 . Therefore, ρ can be regarded as the fineness of classification. Next, we explain the behavior of FAM. As shown in Fig.2, FAM consists of a learning ART (ARTa ), a supervising ART (ARTb ) and a map field (MF). In the learning period, ARTa receives a sample a ∈ [0, 1]na which may contain noise and ARTb receives the corresponding recognition code b ∈ [0, 1]nb . The vigilance parameter of ARTa (i.e., ρa ) is set to the baseline value ρa0 , whenever ARTa receives a new sample. If the category of ARTa is designated by Fa2 node mb J, ARTa provides Fab in MF with W ab which is the same as the map J ∈  field weight vector. If the category of ARTb is designated by Fb2 node K, ARTb

302

T. Kamio et al.

b b provides Fab with y b ∈ mb which satisfies yK = 1 and yk =K = 0. After ab b ab receiving W J and y , F checks the mapping from ARTa to ARTb by   ab   b  b  x  ≡ y ∧ W ab   (3) J  ≥ ρab y ,

where xab is Fab activity and ρab ∈ [0, 1] is the vigilance parameter of MF. If Eq.(3) is true, MF judges that the mapping is correct. In the case of an erroneous mapping, MF executes the match tracking (MT). MT resets Fa2 node J by increasing ρa as follows: ρa = SJa + ,

(4)

where SJa is the matching degree of Fa2 node J and  is an arbitrary small positive value. Therefore, ARTa selects another node or generates a new node after MT. The above processes are iterated until Eq.(3) is satisfied. This means that MT can correct an erroneous mapping. When the mapping is deemed correct, AL-SLMAP [2] updates weight vectors as follows: DJ = (1 + cJ )−1 A + (1 − (1 + cJ )−1 )D J a(new) a(new) a(old) UJ = βa (A ∧ D J ) + (1 − βa )U J , a(new)

a(old)

b(new)

b(old)

,

(5)

b(old)

DK = βb (B ∧ D K ) + (1 − βb )DK , b(new) b(old) b(old) UK = βb (B ∧ U K ) + (1 − βb )U K , ab(new)

WJ

ab(old)

= βab (y b ∧ W J

ab(old)

) + (1 − βab )W J

(6) ,

(7)

Fa2

where cJ is the count of selections of node J in correct mappings. βa , βb and βab ∈ (0, 1] are learning rates. Note that βb must be set to 1, whenever Fb2 node K learns for the first time. All the components of weight vectors are set to 1 before the first update. Eqs.(5), (6), and (7) are based on AL [2], FCSR [1] and SLMAP[3] respectively. The reason why FCSR updates DbK and U bK is that FCSR can optimally learn recognition codes, which are noiseless data. After the supervised learning is finished, ARTa can be used as a recognition system. This is because W ab j gives a mapping from ARTa to ARTb . However, if ab ab W j has more than one component which satisfies Wjk ≥ ρab , Fa2 node j must be deleted before using ARTa as a recognition system. 2.2

Characteristics

First, we discuss the characteristics of weight vectors created by AL-SLMAP (i.e., Eqs.(5)-(7)). D aj converges on the average of samples which have been classified into Fa2 node j. U aj approximates the distribution of samples. It is expressed by a hyper rectangle in the input space. D bk and U bk correspond to the immediate k-th kind of recognition code bk . Recognition codes bk denote noiseless data. ab a b Although the initialized W ab j relates F2 node j to all the F2 nodes, W j relates a b F2 node j to only one F2 node finally.

Fuzzy ARTMAP with Explicit and Implicit Weights

303

Next, let us consider the result of Ref.[2]. Ref.[2] shows that AL-SLMAP can inhibit the category proliferation for the character recognition problem which consists of noisy samples and noiseless recognition codes. This result must be obtained by satisfying the following conditions: (a) Since Eq.(7) makes W ab j learn slowly, the occurrence of MT is reduced. As a result, Fa2 node j can get a lot of samples. a b (b) When W ab j has just related F2 node j to only F2 node k, most of the a samples learned by F2 node j correspond to the recognition code bk . (c) Since Eq.(5) averages samples, the influence of noise is eliminated from D aj and U aj . (d) Since D aj and U aj become a good category, unnecessary MT can hardly occur. However, we have found two disappointing facts about AL-SLMAP. One is a fact that AL-SLMAP cannot inhibit the category proliferation for the character recognition problem in a highly noisy environment [4]. The other is a fact that AL-SLMAP is less suitable for the region classification problem than FCSR. From these facts, we have noticed three problems. The first problem is as follows. When Fa2 node j learns a certain set of samples corresponding to the identical recognition code, the presentation order of the samples influences U aj . The second problem is that the elimination of noise from Daj and U aj is incomplete because the condition (b) is not always satisfied. The third problem is a potential drawback of MT [4]. To solve these problems we change the choice strength for ARTa , propose FAM with explicit and implicit weights, and modify MT by using implicit weights.

3 3.1

FAM with Explicit and Implicit Weights Choice Strength for ARTa

As the distance between a sample and a category becomes smaller and the size of the category grows larger, the choice strength by Eq.(1) becomes larger. However, we think that Eq.(1) is unsuitable for the choice strength for ARTa . To verify our opinion we made Fa2 node j learn a certain set of samples corresponding to the identical recognition code. In each experiment, the samples were given in different order. Simulation results show the following. Daj was always  a  the same a vector. Although U j became a different vector, the fluctuation of U j  was small. From these results and the characteristics of Eq.(1) mentioned above, we have judged that U aj may define an improper category. To solve this problem the  A ∧ U aj  distance between a sample and a category should be estimated by not   but A ∧ Daj . In this case the bottom-up should be defined by not a  weight  a a a  vector U j but a scalar Uj because only U j is needed. Therefore, we change the definition of the choice strength for ARTa as follows:   Tja = A ∧ D aj  /(α + Uja ), (j = 1, · · · , ma ). (8)

304

T. Kamio et al. Fab Wabjk

WabjK

Wabj1

Wab jmb gj1

gjk

gjK

gjmb

u aj1

u ajk

u ajK

uajmb

d aj1

d ajk

d ajK

dajmb

Uaj, D aj

F1 a

Fig. 3. Fa2 node j of our proposed FAM

3.2

Explicit and Implicit Weights

a a b a When W ab j has just related F2 node j only to F2 node k, D j and Uj should be created by the samples corresponding only to bk . To achieve this request, we propose FAM with explicit and implicit weights. As shown in Fig.3, Fa2 node j has a weight set (dajk , uajk ) for each Fb2 node k. D aj and Uja is calculated by

D aj

m mb b     a = gjk njk djk (gjk njk ) , k=0

(9)

k=0

m mb b     Uja = gjk njk uajk (gjk njk ) , k=0

(10)

k=0

where njk is the count of updates of (dajk , uajk ). All the components of dajk are set to 1 and uajk is set to 2na before the first update. Also, gjk is given by  gjk =

ab 1, if Wjk ≥ ρab . 0, otherwise

(11)

Eqs.(9)-(11) show that (D aj , Uja ) is defined only by weight sets (dajk , uajk ) with gjk = 1. Therefore, we call a weight set (dajk , uajk ) with gjk = 1 an explicit weight and a weight set (dajk , uajk ) with gjk = 0 an implicit weight. If MF judges that the mapping from Fa2 node j to Fb2 node k is correct, (dajk , uajk ) is updated by −1 = n−1 , jk A + (1 − njk )djk    a(new)  a(old) = n−1 ∧ A . d  + (1 − n−1 jk jk jk )ujk

a(new)

djk a(new)

ujk

a(old)

(12) (13)

Fuzzy ARTMAP with Explicit and Implicit Weights

305

Eqs.(12) and (13) mean that (dajk , uajk ) is updated by the samples corresponding a b only to bk . That is to say, when W ab j has just related F2 node j only to F2 node a a k, D j and Uj are created by the samples corresponding only to bk . When ARTa is used as a recognition system, Fa2 node j must be deleted if ab W ab j has more than one component which satisfies Wjk ≥ ρab . Furthermore, all the implicit weights should be deleted from the viewpoint of resource costs. 3.3

Match Tracking

After the original MT resets the activated Fa2 nodes by increasing ρa , the erroneous mapping from ARTa to ARTb is corrected. As a result, even if there are Fa2 nodes besides the reset ones, a new node may be forcibly generated. However, there is the possibility that the erroneous mapping is corrected by the restricted MT which resets activated Fa2 nodes without increasing ρa . These facts illustrate that the original MT may needlessly generate Fa2 nodes. To solve this problem we propose the modified MT using implicit weights. From now on, we call the original MT “MTup ” and the restricted MT “MTfix ”. The modified MT is the combination of MTfix and the forcible node generation. The former is used to inhibit the increment of Fa2 nodes. The latter is used to correct erroneous mappings which cannot be solved by MTfix . MTfix is the same as MTup except using ρa = ρa0 instead of Eq.(4). The forcible node generation is executed as follows. It is assumed that Fa2 node j and Fb2 node k are activated when the first erroneous mapping happens for t-th input set (a, b). At this point, an implicit weight (dajk , uajk ) is updated by Eqs.(12) and (13), and then this update increases zjk by 1. That is to say, zjk counts the updates of the weight set (dajk , uajk ) after it becomes an implicit weight. Next, the occurrence rate of erroneous mappings L(t) is calculated by  r/PR , if t ≥ PR L(t) = , (14) 1, otherwise where r is the number of input sets which give rise to erroneous mappings in the period [t − PR + 1, t]. Moreover, the change of L(t) is evaluated every PL input sets. If L(t) − L(t − PL ) > 0, the modified MT judges that there are erroneous mappings which cannot be solved by MTfix . In this case, the following equation is checked for Fa2 nodes with only one explicit weight: Zj /τ ≥ χ,

(15)

where Zj is the maximal zjk of implicit weights in Fa2 node j, τ is the summation of zjk of all the implicit weights in Fa2 layer, and χ ∈ (0, 1] is the standard for the forcible node generation. If Fa2 node J has the implicit weight (daJK , uaJK ) satisfying Eq.(15), then a new Fa2 node J  is generated as follows:  a (dJK , uaJK , 1, 1, 0), if k = K a a ab (dJ  k , uJ  k , nJ  k , WJ  k , zJ  k ) = . (16) (1, 2na , 0, 1, 0), otherwise

306

T. Kamio et al.

However, if the same Fa2 node J has satisfied Eq.(15) at the last check of Eq.(15), the node J is modified instead of generating a new Fa2 node:  a (dJk , na , 1, 1, 0), if gJk = 1 a a ab (dJk , uJk , nJk , WJk , zJk ) = . (17) (1, 2na , 0, 1, 0), otherwise This is because such Fa2 node J may provide the categories around it with bad influences. After completing these processes, the modified MT executes MTfix . Even if erroneous mappings happen again for t-th input set, the modified MT executes only MTfix .

4

Simulation Results

Simulations have been carried out to demonstrate the effectiveness of our proposed method (PM). For the alphabet character recognition problem, PM is compared with FCSR [1] and AL-SLMAP [2]. The main difference of FCSR and AL-SLMAP is the weight update method for D aJ , U aJ , and W ab J . In the case of FCSR, D aJ and U aJ are updated by Eq.(6). However, (A, D aJ , U aJ , βa ) must be given to Eq.(6) instead of (B, DbK , U bK , βb ). Also, W ab J is updated by Eq.(7) with βab = 1.

Fig. 4. Alphabet characters

The original patterns of the alphabet characters are shown in Fig.4. Each pattern is illustrated by a (7×7)-pixel image. The pixel values are set to 0 for white pixels and 1 for black ones. In the learning period, ARTa receives noisy patterns (i.e., sample data) a ∈ 7×7 and ARTb receives the corresponding recognition codes b ∈ 26 . A noisy pattern a is constructed by inverting some pixels in an original pattern selected randomly. The number of inverted pixels depends on Hamming distance (HD). In a recognition code b, one element is set to 1 and the others are set to 0. For instance, the code b corresponding to the character “A” is [1, 0, · · · , 0]. The quantity of learning data NL is 20000. In the test period, ARTa receives noisy patterns (i.e., test data) a. The quantity of test data NT is 50000. We estimate each learning method by the learning time TL (sec.), the number of generated Fa2 nodes ma , and the recognition rate for test data RT . The parameters of each learning method are as follows. In the case of FCSR, αa = 0.1, βa = 0.2, ρa0 = 0.5, αb = 1, βb = 1, ρb = 1, βab = 1, and ρab = 1. In the case of AL-SLMAP, αa = 0.1, βa = 0.2, ρa0 = 0.5, αb = 1, βb = 1, ρb = 1, βab = 0.02, and ρab = 0.75. They are the same as Ref.[2]. In the

Fuzzy ARTMAP with Explicit and Implicit Weights

TL(sec.)

PM

100

AL-SLMAP FCSR

0 0

20 HD

10 (a) The learning time.

ma

2000

PM AL-SLMAP FCSR

1000

0

0

20 HD

10 a

(b) The number of F2 nodes.

RT

1 0.8 0.6

PM AL-SLMAP

0.4

FCSR

0

10

20 HD

(c) The recognition rate for test data. Fig. 5. Simulation results for the alphabet character recognition problem

307

308

T. Kamio et al.

case of PM, αa = 0.1, ρa0 = 0.5, αb = 1, βb = 1, ρb = 1, βab = 0.02, ρab = 0.75, PR = 1000, PL = 800, and χ = 0.08. Fig.5 illustrates TL , ma , and RT . Fig.5(a) and Fig.5(b) show that TL of PM is the largest of the three methods and each method has the same ma when HD = 0. This is a result predicted before executing simulations because PM needs the highest calculation cost per Fa2 node. However, even if HD becomes large, PM keeps the constant ma . But ma in other learning methods increases rapidly. As a result, PM can finish the learning much faster than the others when HD is large. Moreover, Fig.5(b) and Fig.5(c) show that PM has better ma and RT . As HD becomes larger, this tendency becomes stronger. Therefore, we have concluded that PM can inhibit the category proliferation and keep high recognition performance in a highly noisy environment.

5

Conclusions

AL-SLMAP is one of useful learning methods for fuzzy ARTMAP (FAM) from the viewpoint of simplicity and performance. However, AL-SLMAP has problems in category selection, weight update, and match tracking. To solve these problems, we have proposed FAM with explicit and implicit weights. Simulation results have shown that our proposed method (PM) is better than FCSR and AL-SLMAP in terms of category proliferation and recognition performance when the sample data contains a large amount of noise. In the future, we will try to reduce the learning calculation cost of our FAM. Moreover, we have to compare PM with other learning methods and apply PM to more practical problems.

References 1. Carpenter, G.A., Grossberg, S., Markuzon, N., Reynolds, J.H., Rosen, D.B.: Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps. IEEE Trans. Neural Networks 3(5), 698–713 (1992) 2. Lee, J.S., Yoon, C.G., Lee, C.W.: Improvement of recognition performance for the fuzzy ARTMAP using average learning and slow learning. IEICE Trans. Fundamentals E81-A(3), 514–516 (1998) 3. Carpenter, G.A., Grossberg, S., Reynolds, J.H.: A fuzzy ARTMAP nonparametric probability estimator for nonstationary pattern recognition problems. IEEE Trans. Neural Networks 6(6), 1330–1336 (1995) 4. Kamio, T., Nomura, K., Mori, K., Fujisaka, H., Haeiwa, K.: Improvement of Fuzzy ARTMAP by Controlling Match Tracking. In: Proc. International Symposium on Nonlinear Theory and its Applications, pp. 791–794 (2006)

Neural Network Model of Forward Shift of CA1 Place Fields Towards Reward Location Adam Ponzi Laboratory for Dynamics of Emergent Intelligence, RIKEN BSI, Saitama, Japan [email protected]

Abstract. In a recent experimental paper I. Lee et al. [1] showed that the firing patterns of CA1 complex-spike neurons gradually shifted forward across trials toward prospective goal locations within a recording session over multiple trials. Here we propose a simple model of this result based on the phenomenon of awake sequence reverse replay [2] which occurs when the animal pauses at the reward location. The model is based on the CA3-CA1 anatomy with modulation of CA3-CA1 synaptic plasticity by feedback from CA3 projecting CA1 interneurons. Sequence replays, which are generated in CA3 by removal of septal inhibition on CA1 interneurons, are recoded into the synaptic weights of individual CA1 cells. This produces spatially extended CA1 firing fields, whose response provides a value function on experienced paths towards goal locations. Simulations show that the CA1 firing fields show positive movement in center of mass towards reward locations over many trials with negative shift in first few trials, and development of positive skew.

1 Introduction It is well known that the hippocampus is involved in remembering episodic events in particular environments, [3, 4, 5, 1]. The spatial information of events is represented by place cells which show strong firing correlations with particular locations. However, place coding in the hippocampus also interacts with nonspatial factors such as task demands and/or the reinforcement schedule, especially in complex memory tasks. It has been demonstrated that the place cells in the hippocampus are influenced by changes in the physical environmental context [6, 7, 8, 9, 5] but other studies have also shown that changes in the physical context are not necessary to alter the spatial firing pattern. One important factor is the reward or goal postion, and place cells change their firing characteristics when the pattern of reinforcement is altered in the same environment [10, 11, 12, 13]. In addition, task demands modulate the place cell responses in the absence of physical changes in the environment [14,15,16]. For example, [16] have shown that as a rat traverses the stem of a modified T-maze continuously while alternating between different goal locations, CA1 place cells fire more strongly in association with a particular trial type (i.e., left-to-right or right-to-left trials) of the alternation task. A recent work [1] which studied firing patterns over multiple trials revealed that the spatial firing correlates of CA1 place cells gradually shifted forward across trials, via the stem, toward prospective goal locations within a recording session. The results M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 309–316, 2008. c Springer-Verlag Berlin Heidelberg 2008 

310

A. Ponzi

suggest that the shift in a reference frame bound to physical objects [7] is not necessary to produce a systematic shift in firing locations of hippocampal neurons. Instead, the results imply that a goaloriented, cognitive reference frame can significantly influence the place cell characteristics of the hippocampal neurons, especially when animals need to parse a given physical space into multiple cognitive maps according to the mnemonic task demands [1]. Here we describe a simple neural network model of this forward shift phenonemon. The model is an extension of a previous [26, 27] modeling of CA3 generated reverse sequence replay [2], to include a projection to CA1 to describe the forward shift phenomenon. As suggested in discussion in [1], the forward shift is hypothesized to result from the reverse replay which has been observed to occur at the reward location [2]. In the present model, replays generated in CA3 at the reward location are recoded into the CA3-CA1 synaptic weights of a CA1 cell which therefore represents the trajectory of the animal preceeding the replay. Moreover the replay strength of replayed place cells depends on their distance back along the animal path from the reward location so that the CA1 cell can represent a value function, whose firing rate represents the distance to the reward location. In order to ensure that the CA1 weights reflect the replay strength we suggest that LTP/LTD in the CA3-CA1 connections is modulated by an intereuron so that LTP occurs simultaneously to CA3 replay while LTD occurs at other times. This is regulation is hypothesised to be controlled by the CA3 projecting CA1 interneurons [17, 18, 19] which are in turn regulated by septum GABAergic input. With the exception of the CA3 projection, these cells cross-field projecting cells are remarkably similar to the trilaminar neurons, particularly in regard to their dendritic morphology, laminar distribution of local (CA1) axons, and a projection through the fimbria. According to the laminar specificity of their dendritic tree, they are likely to be driven primarily by the local collaterals of CA1 pyramidal cells in a feedback manner. They exert their inhibitory effects on the dendritic tree of pyramidal cells in the CA1 region and also most notably in area CA3. The inhibition mediated by backprojection cells, therefore, is in a direction opposite to the excitatory dentate gyrusCA3-CAl axis. In fact it is suggested in [17] that a cross-regional timing of action potentials by these interneurons may be important to secure population synchrony of principal cells in distributed networks and may allow a coordinated induction of synaptic plasticity [17, 18, 19].

2 Model The model system is depicted in Fig.1(a). The CA3 system consists of a set of N pyramidal cells. In general each pyramidal cell receives inhibition from the population of various types of interneuron [17], proximal excitation from the dentate gyrus [22], mid-dendritic excitation from other pyramidal cells via the recurrent collaterals [22], and excitation from the enthorhinal cortex. Here we omit all these connections except for one-to-one excitation which carries signals generated either from dentate-gyrus or entorhinal cortex. These signals carry external environmental information such as

Neural Network Model of Forward Shift

DG/EC

311

1 to 1 CA3 Pyramid All to All Modifyable

Basket Cell

CA1 Pyramid All to All

SEPTUM

Interneuron H

Fig. 1. (a) Anatomy of model system described in text. The solid lines denote excitatory connections, the dashed lines are inhibitory and the dot-dashed lines are modifyable excitatory connections. (b) Circular track task, animal runs clockwise. The rectangles represent the topographic place specific inputs Ii = 0, 1 from the dentate gyrus or EC to CA3 pyramidal cells. They are overlapping in this example for generality. Replays are generated by activating the septal inhibition everytime the animal runs throught the location marked R for reward.

landmarks etc, or internally generated position specific signals. The pyramidal cells are then described by activities pi (t), simply given by, dpi = −k1 pi + k2 g(wi )f (HX − H(t)) + k3 Ii (x(t))| cos(Θx(t))|. dt

(1)

Here the activities pi (t) may represent membrane potentials or firing rates and we do not model spiking explicitly. The Ii (x(t)) is one-to-one topographic input which can as explained above be either place specific input which enters the CA3 pyramidal cell via the dentate gyrus, assumed to be formed from path integration of internal motion signals on the entorhinal grid [23]. In this model each pyramidal cell i is activated by a single place specific input for simplicity. Alternatively Ii (x(t)) can model sensory type information from the enthorhinal cortex on the perforant path. x(t) is the position of the animal which fixes Ii . For completeness this signal is considered modulated by the theta rhythm described by the factor | cos(Θx(t))| where Θ is a constant, x(t) is the position and |x| denote the absolute value of x. The wi (t) in Eq.1 are internal ‘excitability’ variables they are simply given by, dwi = k5 pi − k6 wi . dt

(2)

These are the variables which generate the reverseness of the replay [2], as has been described in [26] and will be explained below. They are driven by the firing pi and must decay slowly across the spatial environment, this decay is controlled by the parameter k6 . Both the pi (t) and the wi (t) are necessary because the cell has localized firing given by pi while the wi (t) reflect internal memory across the whole environment. The function g(x) in Eq.1 is the sigmoidal function which is used to limit the activity of the k2 term in Eq.1. It is only necessary to avoid fine tuning of parameters and is set

312

A. Ponzi

to be approximately linear in the region of wi (t) used in these simulations. Notice that although we suggest a simpler model with  topographic inputs here, it would not be problematic to replace the Ii in Eq.1 with j Iij for a set of inputs j to cell i. The term f (HX − H(t)) models the modulation by the interneuron H cells, (see Fig.1(a)), whose firing rate is given by H(t), which will be described below. Here f (x) = x is the positivity function f (x) = x, (x > 0), f (x) = 0, (x < 0). This term means that when the H cell firing rate is above its baseline HX the cell soma is inhibited, but when the interneuron H activity drops below its baseline activity, HX , the soma can become activated. The strength of this replay reactivation depends on the the excitabiltity wi (t) at the time of the reactivation. Therefore the cell firing Eq.1 can be driven by two different factors, the input Ii (x(t)) and a reactivation by removal of H interneuron inhibition. Notice we do not include the CA3 recurrent collaterals. It is hypothesized that these are used to produce forward replay theta sequences [20,21] encoded by time assymetric Hebbian learning. Since we here we only address the reverse replay here we do not require them. Unlike the CA3 pyramidal cells the CA1 cells do not have strong recurrent collaterals and we model them as a winner-take-all (WTA) system. They are given by,  dqi = −qi + f ( uij pj − B(t) + qi ) dt j

(3)

where qi are the activities of the CA1 pyramidal cells and uij (t) are modifyable weights for the all-to-all projection from CA3 to CA1. In the CA1 system we also include an inhibition from the CA1 basket cell activity which produces a WTA in the absence  of strong recurrent collaterals. The basket cell is simply modeled as B(t) = i qi according to the activation from the pyramidal cell population. We describe the projection weight uij (t) update by, duij = −uij (t) + f ((HX − H(t))pi qj + uij (t)). dt

(4)

In this Eq.4 we suggest the uij (t) weights are modified by the postsynaptic qj and presynaptic pi firing rates and whether LTP or LTD occurs is modifyable by the H(t) cell firing. This can occur by modification of backpropagating EPSPs in the dendritic tree of the CA1 pyramidal cells. The reason for this is to ensure that LTP occurs at the same time as replay activity in CA3, while LTD occurs outside replay activity. It is this LTP modulation that allows the CA1 cells to inherit the CA3 replayed activity and produces their broad firing fields which develop over trials. Furthermore the LTD outside the replay activity produces a stabilization of the firing over many trials and a normalization over the environment. Such a modulation of learning has been included in [25, 27, 26]. CA3 projecting CA1 interneuron H cells are known within the hippocampal region [17]. Their cell bodies and axons rest primarily in the stratum oriens, while their dendrites may extend across the strata to stratum radiatum and stratum lacunosummoleculare and to CA3. In addition the H cell population receives excitatory input from the active pyramidal cells and is regulated by an inhibitory GABAergic septal

Neural Network Model of Forward Shift

313

input [24, 22]. The medial septum diagonal band of broca (MSBD) projects to the H cells and related O-LM cells in stratum radiatum and oriens of CA1 and CA3 [22]. The H cells are simply modeled by a single unit,  dH = −H(t) + HX + k6 qi (t) − kS S(t). dt i

(5)

They are activated by projection from the the CA1 pyramidal cells qi and their firing is inhibited by activity from the septum S(t). When the pyramidal cell input and the septal input are zero then basket cell activity decays exponentially to its baseline level HX . The septal activity generates the replays as explained in [26]. The septal signal itself is considered to be possibly generated by reward signals from the hypothalamus or from the thalamus.

3 Results The model is best understood by studying examples of its time series. In this paper we only consider the circular environment depicted in Fig.1(b).

1.2

5

4

0.8

CA1 cell activities

CA1 cell activities

1

0.6

0.4

0.2

3

2

1

0

0 2400

2600

2800

Time

3000

3200

2500

3000

3500

4000

Time

Fig. 2. (a) Activity of a CA1 pyramidal cell on an early trials. (b) Activity of same CA1 pyramidal cell as (a) on a successive pair of trials late in learning.

The activation of a CA1 cell around the track for a successive pair of trials early in learning is shown in Fig.2(a) for repeated traversals around the track shown in Fig.1(b). The firing rate is quite flat throughout a whole trial. The periodic modulations are due to the periodic theta driving of inputs Ii (t) in Eq.1 while the periodic peaks are due to the different contributions of difference CA3 layer cells each CA3 cell activated topographically as depicted in Fig.1(b). The large activation at the end of each trial is caused by the replay reactivation event in CA3. The same cell for a pair of later trials is shown in Fig.2(b). Here the firing rate clearly increases across the trial and this produces the forward shift towards reward location in the center of mass. Further the increase in skew in the direction of motion towards reward is clear.

A. Ponzi

CA3 activities, H cell

314

3

2

1

0 5530

5532

5534

5536

5538

Time

5540

Fig. 3. Time series of activity of CA3 cells at replay event. Each CA3 cell is shown as solid line with different symbol. Also shown is the H cell time series as dashed line.

How this occurs is very easy to understand. As explained elsewhere [26] and shown in Fig.3 the replay reactivates the most recently experienced place cells which strongest magnitude. This creates a kind of reverseness. According to the model Eqns.1,5 the replay occurs when the H interneuron is inhibited by the activation of the septum S(t). The septum is suggested to be activated at reward locations or locations where the animal pauses. This H cell inhibition below its baseline HX = 1 is also is shown in Fig.3. This is also when LTP occurs at the at CA3-CA1 synapses according to Eq.4. Therefore the CA3 replay is recoded into a CA1 cell, whose synaptic weights uij now represent the replay strength. This CA1 cell can be said to provide a value function along the trajectory leading to the reward location since its firing rate depends on the distance to the reward location. How the center of mass (COM ) of this CA1 cell varies across trials is shown in Fig.4(a). The center of mass for each trial is calculated acording to, N i

Total firing rate Q for a CA1 cell each trial

Center of Mass (COM) of CA1 cells each trial

COM =

0.0062

0.006

0.0058

0.0056

0.0054

0.0052

q(ti )x(ti ) Q

(6)

1.5

1

0.5

0 0

10

20 Trial Number

30

0

5

10

15

20

25

30

Trial Number

Fig. 4. (a) Variation of center of mass of CA1 cells versus trial number. (b) Variation of total firing rate of CA1 cell versus trial number.

Neural Network Model of Forward Shift

315

where, Q=

N 

q(ti ).

(7)

i

In these equations ti is the time of the i − th iteration of the runge kutta integrator, and there are N iterations each trial. Therefore Q is the total firing rate for each trial while COM is the total firing rate weighted by the position x(ti ) of the animal on the track at time ti each trial. As shown in Fig.4(a) the COM initially decreases rapidly and then slowly increases. We also show the change in total firing rate Q for each trial in Fig.4(b) which increases across trials and then slowly stabilizes later. In fact on the first replay event all the synaptic weights of CA1 cells have weights given by previous experience which has no relevance to the track in the simulation. The first replay event causes positive learning at these synapses and then they are slowly adjusted over trials to reflect the replayed strengths measuring the distance to the reward location in the current context and specific environment. This is what causes the drop in the COM on the first few trials and then increase after. Indeed this is as observed in [1] when tasks are switched. Fig.4(a) also reveals a small 6 trial backwards and forwards periodic variation in the center of mass. This is a quasiperiodicity induced by the non-rational ratio of track length, which determines the reward replay location, and the imposed theta oscillation period in Eq.1. However it may be relevant to real animal behaviour by counting laps around the track.

4 Discussion We have presented a simple model of the forward shift in COM of CA1 place fields based on sequence replay at the reward location. The model relies on a coordination between replays and LTP at the CA3-CA1 synapses which is proposed to be provided by the CA3 projecting CA1 interneurons [17, 18] regulated by the septum. The model reproduces the gradual positive shift in COM. In future work we will address how the model behaves under task switching, for example from the circular track to the alternation task or T-Maze. A more detailed explanation of the reverse replay modeling is given in [26].

References 1. Lee, I., Griffin, A.L., Zilli, E.A., Eichenbaum, H., Hasselmo, M.E.: Gradual Translocation of Spatial Correlates of Neuronal Firing in the Hippocampus toward Prospective Reward Locations. Neuron 51, 639–650 (2006) 2. Foster, D.A., Wilson, M.A.: Reverse replay of behavioural sequences in hippocampal place cells during the awake state. Nature 440(7084), 615–617 (2006) 3. Eichenbaum, H.: A cortical-hippocampal system for declarative memory. Nat. Rev. Neurosci. 1, 41–50 (2000) 4. McClelland, J.L.: Complementary learning systems in the brain. A connectionist approach to explicit and implicit cognition and memory. Ann. N Y Acad. Sci. 843, 153–169 (1998)

316

A. Ponzi

5. O’Keefe, J., Nadel, L.: The hippocampus as a cognitive map. Oxford University Press, Oxford (1978) 6. Anderson, M.I., Jeffery, K.J.: Heterogeneous modulation of place cell firing by changes in context. J. Neurosci. 23, 8827–8835 (2003) 7. Gothard, K.M., Skaggs, W.E., Moore, K.M., McNaughton, B.L.: Binding of hippocampal CA1 neural activity to multiple reference frames in a landmark-based navigation task. J. Neurosci. 16, 823–835 (1996) 8. Lee, I., Yoganarasimha, D., Rao, G., Knierim, J.J.: Comparison of population coherence of place cells in hippocampal subfields CA1 and CA3. Nature 430, 456–459 (2004) 9. Leutgeb, S., Leutgeb, J.K., Treves, A., Moser, M.B., Moser, E.I.: Distinct ensemble codes in hippocampal areas CA3 and CA1. Science 305, 1295–1298 (2004) 10. Breese, C.R., Hampson, R.E., Deadwyler, S.A.: Hippocampal place cells: stereotypy and plasticity. J. Neurosci. 9, 1097–1111 (1989) 11. Fyhn, M., Molden, S., Hollup, S., Moser, M.B., Moser, E.: Hippocampal neurons responding to first-time dislocation of a target object. Neuron 35, 555–566 (2002) 12. Kobayashi, T., Nishijo, H., Fukuda, M., Bures, J., Ono, T.: J.Neurophysiol. 78, 597–613 (1997) 13. Markus, E.J., Qin, Y.L., Leonard, B., Skaggs, W.E., McNaughton, B.L., Barnes, C.A.: Interactions between location and task affect the spatial and directional firing of hippocampal neurons. J.Neurosci. 15, 7079–7094 (1995) 14. Bower, M.R., Euston, D.R., McNaughton, B.L.: Sequential- context-dependent hippocampal activity is not necessary to learn sequences with repeated elements. J. Neurosci. 25, 1313– 1323 (2005) 15. Ferbinteanu, J., Shapiro, M.L.: Prospective and retrospective memory coding in the hippocampus. Neuron 40(2003), 1227–1239 (2003) 16. Wood, E.R., Dudchenko, P.A., Robitsek, R.J., Eichenbaum, H.: Hippocampal neurons encode information about different types of memory episodes occurring in the same location. Neuron 27, 623–633 (2000) 17. Freund, T., Buzsaki, G.: Interneurons of the Hippocampus. Hippocampus 6, 347–470 (1996) 18. Sik, A., Minen, A., Penttonen, M., Buzsaki, G.: Inhibitory CA1-CA3- hilar region feedback in the hippocampus. Science 265, 1722–1724 (1994) 19. Somoyogi, P., Klausberger, T.: Defined types of cortical interneurone structure space and time in the Hippocampus. J.Physiol. 562.1, 9–26 (2005) 20. Foster, D.J., Wilson, M.A.: Hippocampal theta sequences Hippocampus (2007), DOI 10.1002/hipo.20345 21. Lisman, J.E.: Relating Hippocampal Circuitry to Function: Recall of Memory Sequences by Reciprocal Dentate-CA3 Interactions. Neuron 22, 233–242 (1999) 22. Johnston, D., Amaral, D.G.: In: Shepard, G.M. (ed.) The Synaptic Organization of the Brain, Oxford University Press, Oxford (1998) 23. Hafting, T., Fyhn, M., Molden, S., Moser, M.-B., Moser, E.I.: Microstructure of a spatial map in the entorhinal cortex. Nature 436, 801 (2005) 24. Freund, T., Antal, M.: GABA-containing neurons in the septum control inhibitory interneurons in the hippocampus. Nature 336, 170–173 (1998) 25. Ponzi, A.: Dynamical System Model of Spatial Reward Learning. IEICE Technical Report 103, 163, 19–24 (2006) 26. Ponzi, A.: Model of Balance of Excitation and Inhibition in Hippocampal Sharp Wave Replays and Application to Spatial Remapping. In: Proceedings of IJCNN 2007 (2007) 27. Ponzi, A.: Simple Model of Hippocampal Splitter Cells. Japan Neural Network Society (JNNS) abstract (2006)

A New Constructive Algorithm for Designing and Training Artificial Neural Networks Md. Abdus Sattar1 , Md. Monirul Islam1,2, , and Kazuyuki Murase2,3 1

Department of Computer Science and Engineering, Bangladesh University of Engineering and Technology, Dhaka 1000, Bangladesh 2 Department of Human and Artificial Intelligence Systems, Graduate School of Engineering, University of Fukui, 3-9-1 Bunkyo, Fukui 910-8507, Japan [email protected] 3 Research and Education Program for Life Science, University of Fukui, 3-9-1 Bunkyo, Fukui 910-8507, Japan

Abstract. This paper presents a new constructive algorithm, called problem dependent constructive algorithm (PDCA), for designing and training artificial neural networks (ANNs). Unlike most previous studies, PDCA puts emphasis on architectural adaptation as well as function level adaptation. The architectural adaptation is done by determining automatically the number of hidden layers in an ANN and of neurons in hidden layers. The function level adaptation, is done by training each hidden neuron with a different training set. PDCA uses a constructive approach to achieve both the architectural as well as function level adaptation. It has been tested on a number of benchmark classification problems in machine learning and ANNs. The experimental results show that PDCA can produce ANNs with good generalization ability in comparison with other algorithms. Keywords: Artificial neural networks (ANNs), architectural adaptation, function level adaptation, constructive approach and generalization ability.

1

Introduction

Artificial neural networks (ANNs) have been widely used in many application areas. Many issues and problems, such as selection of training data, training algorithm and architectures, have to be addressed and resolved when using ANNs [8]. Among them the proper selection of an ANN architecture is of great interest because the performance of the ANN is greatly dependent on its architecture. There have been many attempts in designing and training ANNs, such as various constructive, pruning and evolutionary approaches (see review papers [4], [12] and [20]). The main problem of most existing approaches is that they can design either single hidden layered ANNs or multiple hidden layers ANNs with one neuron in each hidden layer [1], [5]-[7]. It, however, is quite difficult to decide 

Corresponding author.

M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 317–327, 2008. c Springer-Verlag Berlin Heidelberg 2008 

318

M.A. Sattar, M.M. Islam, and K. Murase

in advance whether a problem can be solved efficiently by using single hidden layered or multiple hidden layered ANNs. It is therefore necessary to devise an algorithm that is able to design both single and multiple hidden layered ANNs depending on problems complexity. This paper proposes a new constructive algorithm, called problem dependent constructive algorithm (PDCA), for designing and training feedforward ANNs. PDCA determines automatically not only the number of hidden layers in an ANN, but also the number of neurons in hidden layers. It uses a constructive approach with a layer stopping criterion for determining them. PDCA’s emphasis on training different hidden neurons with different training sets can increase the efficiency of determining ANN’s architecture automatically. PDCA differs from previous works for designing and training ANNs on a number of aspects. First, it is an algorithm that can design both single and multiple hidden layered ANNs depending on the complexity of a given problem. This approach is quite different from most existing algorithms (e.g.,[1] and [17]), which try to solve problems either by using only single or multiple hidden layers but not the both. Although single hidden layered ANNs are universal aproximators [3], multiple hidden layered ANNs are superior over single hidden layered ANNs for some problems [18]. Second, all the existing algorithms train hidden neurons in an ANN by the same training set. But PDCA creates a new training set based on the performance of the existing ANN architecture when a new neuron is added. Although this approach is used by boosting algorithm [15] for designing ANN ensembles, it is the first attempt to our best knowledge to use this concept in designing single ANNs. Third, most existing algorithms (e.g., [1], [5]-[7] and [9]) do not have any effective mechanism for stopping the addition of neurons in hidden layers. Consequently, they use only one neuron in each hidden layer resulting very deep architectures and long propagation delay, which is also not suitable for VLSI implementation. Or they use a predefined and fixed number of neurons for all hidden layers [10]. The problem of using a fixed number of neurons lies in the difficulty of selecting an appropriate number that is suitable for a given problem. To address these problems, PDCA uses a layer stopping criterion that determines automatically the number of neurons in each hidden layer. The rest of the paper is organized as follows. Section 2 describes PDCA in details. Section 3 presents results of our experimental study. Finally, Section 4 concludes the paper with a brief summary and few remarks.

2

PDCA

In order to determine automatically the number of hidden layers in ANNs and of neurons in hidden layers, PDCA uses incremental training in association with layer stopping criterion in designing ANNs. In our incremental training, hidden layers and hidden neurons are added to the ANN architecture one by one in a constructive fashion during training. The layer stopping criterion is used to decide when to add a new hidden layer by stopping the growth i.e., neurons addition to a hidden layer. To obtain efficient solution, PDCA trains each hidden

A New Constructive Algorithm for Designing and Training ANNs

319

neuron in an ANN with a different training set and stops automatically the ANN construction process. Though any kinds of ANNs and activation functions can be used for PDCA, for this work, we used PDCA to design feedforward ANNs with sigmoid activation function. The feedforward ANNs considered here are generalized multilayer perceptrons. In such architecture, the first hidden layer receives only network inputs (I) while other hidden layer(s) receives I plus the outputs of the preceding hidden layer(s). The output layer receives signals only from all hidden layers. The major steps of PDCA are summarized in Fig. 1, which are explained further as follows. Step 1. Create an initial ANN architecture consisting of three layers i.e., an input layer, a hidden layer and an output layer. The number of neurons in the input and output layers is the same as the number of inputs and outputs

Initial partial training

Create an initial ANN architecture Stop ANN construction?

Yes

Create a training set No Train the ANN

No

Stop initial training ? Final ANN

Stop layer construction ?

Yes

No

Final partial training Yes Add one hidden layer Stop ANN construction ?

Add one neuron

No

No

Stop final training ?

Yes

Fig. 1. Flowchart of PDCA

Yes

320

M.A. Sattar, M.M. Islam, and K. Murase

of a given problem, respectively. Initially, the hidden layer contains only one neuron. Randomly initialize the connection weights of the ANN within a certain range and label the hidden layer with I. Step 2. Create a new training set for the newly added hidden neuron based on the performance of the existing ANN architecture. PDCA uses adaboost.M2 algorithm [16], which is a variant of boosting algorithm [15], in creating training sets. It is here important to note that the original training set is used for training the initial architecture. Step 3. Partially train the ANN by backpropagation learning algorithm for a certain number of training epochs. This training phase is known as the initial training of the existing ANN architecture. The number of epochs, τ , is specified by the user. Partial training means that an ANN is trained for a fixed number of epochs regardless whether it has converged or not. Step 4. Check the termination criterion for stopping the ANN construction process. If the criterion is satisfied, go to the Step 12. Otherwise continue. Step 5. Compute the ANN error, i.e., E, on the training set. If E reduces by a threshold  after the training epochs τ , go to the Step 3 for further training of the existing architecture. It is here assumed that the training process is progressing well and it is necessary to train the existing architecture further. Otherwise continue for the final training of the existing architecture. Step 6. Add a small amount of noise to input and output connection weights of a previously added neuron in the I-labeled hidden layer. Partially train the ANN by backpropagation learning algorithm for τ epochs. This training phase is known as the final training of the existing ANN architecture. Step 7. Check the termination criterion for stopping the ANN construction process. If the criterion is satisfied, go the Step 12. Otherwise continue. Step 8. Compute E on the training set. If E reduces by a threshold  after the training epochs τ , go to the Step 6 for further training of the existing architecture. It is here assumed that the final training phase is progressing well and it is necessary to train the existing ANN further. Otherwise continue for modifying the existing architecture by adding hidden neurons or layers. Step 9. Check the criterion for stopping the growth of I-labeled hidden layer construction process. If the criterion is satisfied, stop the construction process of the I-labeled hidden layer by freezing the input and output connectivities of a previously added neuron in the hidden layer and continue. Otherwise go to the Step 11 for adding a neuron to the hidden layer. Freezing, which was first introduced in [1], means that the frozen connection weights will not be trained i.e., changed when the ANN will be trained in future. Step 10. Replace the labeling of the I-labeled hidden layer by label F . Add a new hidden layer above the existing hidden layer(s) of the ANN. Initially the new hidden layer contains one neuron and it is labeled with I. The connection weights of the neuron is initialized in the same way as described in the Step 1 and go to the Step 2. Step 11. Add one neuron to the I-labeled hidden layer and freeze the input and output connectivities of a previously added neuron in this layer. The

A New Constructive Algorithm for Designing and Training ANNs

321

connection weights of the newly added neuron is initialized in the same way as described in the Step 1 and go to the Step 2. Step 12. The existing ANN architecture is the final architecture for a given problem. As PDCA trains only one, i.e., the newly added neuron at a time, hence other nonlinear optimization methods, such as BFS or quasi-Newton [17] methods that are computationally expensive but have faster convergence, can easily be used in PDCA for training ANNs. Although the design of ANNs could be formulated as a multi objective optimization problem, PDCA uses a very simple cost function, the ANN error. The processes and criteria, incorporated in PDCA at different stages, are described briefly in the following subsections. 2.1

Termination Criterion

PDCA uses a criterion based on both training and validation errors to decide when the training process of an ANN is to be stopped. To formally describe the criteria, let Eva (τ ) and Eopt (τ ) are the validation error at training epoch τ and lowest validation error obtained in epochs up to τ , respectively. The generalization loss, GL, at epoch τ can be defined by the following equation [11].   Eva (τ ) GL(τ ) = −1 (1) Eopt (τ ) A high generalization loss can be one obvious reason to stop training, because it directly indicates overfitting. However, it is desirable not to stop the training process if the training error Etr is still progressing very rapidly. To formalize this notion, let a training strip of length k to be a sequence of k epochs numbered n + 1...n + k, where n is divisible by k. The training progress in a training strip (Pk ) can be used to measure how much the average training error of the strip is larger than the minimum training error during the strip. It can be defined by the following equation [11].  τ   τ  =τ −k+1 Etr (τ ) Pk (τ ) = −1 (2) k. minττ  =τ −k+1 Etr (τ  ) PDCA terminates the training process when GL(τ )/Pk (τ ) > α, where α is a user specified positive number. The reason for using training and validation data in the termination condition is to anticipate the behavior of test data better. 2.2

Layer Stopping Criterion

PDCA uses a simple criterion for deciding when to stop the growth of a I-labeled hidden layer. The criteria is based on the contribution of neurons in a hidden layer. The contribution, Ck , of a neuron k at any training epoch is   1 1 Ck = 100 − (3) E Ek

322

M.A. Sattar, M.M. Islam, and K. Murase

Where E is the network error and Ek is the network error excluding neuron k. The layer stopping criterion stops the growth of a I-labeled hidden layer when its contribution to an ANN, measured after the addition of each hidden neuron, failed to improve after the addition of a certain number of neurons, indicated by the parameter mh below. In other words, the growth of a hidden layer stops when the following is true: Ck (m + mh ) ≤ Ck (m), m = 1, 2, . . . ,

(4)

where mh (mh > 0) is a user specified positive integer number. If mh = 0 then all hidden layers of an ANN can consist of one hidden neuron only like CCA [1]. In PDCA, each hidden layer can consist of several neurons because mh greater than zero is used. It is here worth mentioning that no neurons are added to a hidden layer after its growth process has been stopped. Furthermore, the neurons in a hidden layer whose growth process has been stopped will not be trained anymore. 2.3

Creation of New Training Sets

Adaboost.M2 algorithm [16], which is proposed for training ANN ensembles, is used in PDCA to create different training sets for different hidden neurons in ANNs. It maintains a probability distribution D over the original training set T . Initially, D = 1/M where M is number of examples in T . The algorithm trains the first ANN in the ensemble by using the original training set T . After training the first ANN, D is updated in which the probability of incorrectly and correctly classified examples is increased and decreased, respectively. A new training set T  is created based on updated D by sampling M examples at random with replacement from T . The second ANN of the ensemble is then trained by T  . This process is repeated for other ANNs in the ensemble. The strategy used in adaboost for training ANNs in an ensemble can easily be incorporated in PDCA for training hidden neurons in an ANN. This is because PDCA trains hidden neurons in an ANN one by one, which is similar to train ANNs in an ensemble one after another by adaboost. In addition, PDCA trains only one, i.e., a newly added hidden neuron, at a time by freezing the input and output connectivities of a previously added neuron. This is also similar to training one ANN in the ensemble by adaboost algorithm. The use of different training sets at different stages in the training process of an ANN will facilitate to achieve functional adaptation in PDCA.

3

Experimental Studies

This section evaluates the performance of PDCA on several benchmark classification problems. Table 1 shows the summary characteristics of the problems displaying considerable diversity in the number of examples, attributes and classes. The detail description of all these problems can be obtained from [13] except iris and letter which can be obtained from UCI Machine Learning Repository.

A New Constructive Algorithm for Designing and Training ANNs

323

Table 1. Characteristics of experimental data sets Data set

Cancer Card Diabetes Glass Gene Iris Letter Thyroid

Number of input output training validation testing attributes classes examples examples examples 9 2 350 175 174 14 2 345 173 172 8 2 384 192 192 9 6 107 54 53 120 3 1588 794 793 9 2 107 54 53 16 26 10000 5000 5000 21 3 3600 1800 1800

A. Experimental Setup Table 1 shows the partitioning and the number of examples in each partition for the datasets of different problems. The name of the partitions is training set, validation set, and testing set. In all data sets, the first M examples were used for the training set, the following N examples for the validation set, and the final P examples for the testing set. It should be kept in mind that such partitions do not represent the optimal ones in practice. While the training set and testing set are used to train and to evaluate the generalization ability of trained ANNs, respectively, the union of training and validation sets is used to determine whether to add hidden layers or neurons or to stop the training altogether. In all experiments, one bias neuron with a fixed input +1 is connected to all hidden layers and to the output layer. The logistic sigmoid function is used for neurons in hidden and output layers. 3.1

Experimental Results

Table 2 shows the results of PDCA for 50 independent runs on several classification problems. The generalization performance of produced ANNs is measured by testing error rate (TER), which refers to the percentage of wrong classification produced by trained ANNs. The number of hidden layers (HL) and hidden neurons (HN), respectively, refer to the total number of hidden layers and hidden neurons in produced ANNs. The SD refers to the standard deviation of different results produced by PDCA in 50 independent runs. It is quite natural that complex architectures will be needed for solving difficult problems and simple architectures for solving easy problems. PDCA shows this characteristics which is seen from Table 2. For example, ANNs produced by PDCA had on average 2.84 hidden layers for the glass problem, while 1.06 hidden layers for the cancer problem. It is well known that the glass and cancer are one of the most difficult and easy problems, respectively, in the realm of machine learning [13]. It is also apparent from their error rates achieved by PDCA; for the glass problem it is 0.323, while for the cancer problem it is 0.009.

324

M.A. Sattar, M.M. Islam, and K. Murase

Table 2. Performance of PDCA based on number of hidden layers (HL), number of hidden neurons (HN), number of epochs and testing error rate (TER) for eight classification problems. The results were averaged over 50 independent runs. Problem Cancer Card Diabetes Glass Gene Iris Letter Thyroid

HL Mean SD 1.06 0.7 1.54 1.1 2.04 1.2 2.84 1.5 1.00 1.1 1.46 0.8 2.76 1.3 2.04 0.9

HN Epochs Mean SD Mean SD 3.36 1.2 289.34 20 2.14 1.3 195.82 18 4.20 1.5 220.76 27 4.54 1.6 311.50 30 1.28 0.9 135.90 17 2.46 1.0 125.30 11 24.32 3.8 8037.84 280 4.28 1.7 260.34 16

TER Mean SD 0.009 0.003 0.116 0.021 0.183 0.032 0.323 0.035 0.092 0.021 0.035 0.007 0.128 0.010 0.011 0.001

Table 3. Performance of PDCAWL based on number of hidden neurons (HN), number of epochs and testing error rate (TER) for four classification problems. The results were averaged over 50 independent runs. Problem

HN Epochs Mean SD Mean SD Diabetes 5.64 1.3 261.32 25 Glass 6.58 1.6 370.90 30 Letter 30.64 4.2 8590.18 291 Thyroid 6.86 1.4 301.42 16

TER Mean SD 0.214 0.035 0.364 0.029 0.167 0.021 0.015 0.001

Table 4. Performance of PDCAST based on number of hidden layers (HL), number of hidden neurons (HN), number of epochs and testing error rate (TER) for five classification problems. The results were averaged over 50 independent runs. Problem

HL HN Mean SD Mean SD Cancer 1.20 0.9 4.13 1.2 Glass 2.78 1.5 4.17 1.4 Gene 1.61 1.3 3.13 1.6 Iris 1.45 0.7 2.53 1.0 Letter 2.82 1.8 36.93 4.1

Epochs Mean SD 315.77 23 301.33 23 290.17 19 130.84 9 9220.02 311

TER Mean SD 0.013 0.004 0.298 0.030 0.128 0.023 0.038 0.007 0.176 0.035

In order to observe the effect of layer stopping criterion, a new set of experiments was carried out where PDCA without the layer stopping criterion (PDCAWL) was used and the result is depicted in Table 3. Since PDCAWL does not use the layer stopping criterion, it can only able to design single hidden layered ANNs. The comparison of the Table 3 with the Table 2 indicates the positive effect of using the layer stopping criterion in PDCA. To observe the effect of using different training sets in PDCA, another new set of experiments was carried out where all the newly added neurons were trained

A New Constructive Algorithm for Designing and Training ANNs

325

with the same training set. We call this version of PDCA as PDCAST. The result of the experiment is depicted in Table 4. The effect of using different training sets can easily be understood when the results of PDCAST (Table 4) are compared with those of PDCA (Table 2). It is observed that PDCA produces more compact ANN architectures and takes less number of epochs in comparison with its counterpart PDCAST. The t-test based on number of hidden neurons, number of epochs and testing error rate indicated that PDCA was significantly better than its counterpart PDCAWL and PDCAST at 95% confidence level.

4

Comparison

This section compares the performance of PDCA with CCA [1] and aCasper [19]. The reason for selecting these algorithms is that they are able to design multiple hidden layered ANNs like PDCA. CCA and aCasper use quickprop [2] and RPROP [14] algorithms, respectively, for training ANNs. It is known that quickprop and RPROP algorithms are faster than backpropagation algorithm used in PDCA. In this section, we therefore compare PDCA with CCA and aCasper in terms of average TER and HN of 50 independent runs. Table 5 compares PDCAs results with those produced by CCA and aCasper on six classification problems. It can be seen from the table that PDCA achieved the smallest TER for five out of six problems and the second smallest next to aCasper [19] for one problem. The improved performance of PDCA could be attributed to a couple of factors such as determining the number of hidden layers in ANNs and of neurons in hidden layers automatically and using different data for training different hidden neurons.

Table 5. Comparison among PDCA, CCA [1], and aCasper [19], in terms of average testing error rate (TER) and number of hidden neurons (HN) Algorithm PDCA HN TER CCA HN TER aCasper HN TER

5

Cancer 3.36 0.009 5.18 0.019 4.86 0.018

Card Diabetes Glass Gene Thyroid 2.14 4.20 4.54 1.28 4.28 0.116 0.183 0.323 0.092 0.011 1.07 9.78 8.07 2.73 25.04 0.137 0.245 0.347 0.133 0.030 0.12 3.02 4.18 0.00 4.64 0.135 0.231 0.306 0.117 0.016

Conclusions

This paper proposes a new constructive algorithm to design as well as train ANNs. Neither the number of hidden layers in an ANN nor the number of neurons in hidden layers needs to be predefined and fixed. They are determined automatically during the training of an ANN. Experiments have been carried out on several benchmark classification problems to evaluate how well PDCA

326

M.A. Sattar, M.M. Islam, and K. Murase

performed on different problems in comparison with other ANN design algorithms. In almost all cases, PDCA outperformed the others. However, our experimental study appeared to have revealed one weakness of PDCA in dealing with the glass problem, which has a small number of training examples. It would be interesting in the future to analyze PDCA more rigorously in order to gain more insights into when PDCA are most likely to perform well and for what kind of problems. The use of different activation functions like the one used in [8] in association with different training sets would also be an interesting future research topic. Acknowledgement. MMI is currently a Visiting Associate Professor at University of Fukui supported by the Fellowship from Japanese Society for Promotion of Science (JSPS). This work was in part supported by grants to KM from JSPS, Yazaki Memorial Foundation for Science and Technology, and University of Fukui.

References 1. Fahlman, S.E., Lebiere, C.: The cascade-correlation learning architecture. In: Touretzky, D.S. (ed.) Advances in Neural Information Processing System, vol. 2, pp. 524–532. Morgan Kaufmann, San Mateo, CA (1990) 2. Fahlman, S.E.: An empirical study of learning speed in back-propagation networks. Tech Report CMU-CS-88-162, Carnegie Mellon University (1988) 3. Hornik, K., Stinchcombe, M., White, H.: Multilayer feedforward networks are universal approximators. Neural Networks 2, 359–366 (1989) 4. Kwok, T.Y., Yeung, D.Y.: Constructive algorithms for structure learning in feedforward neural networks for regression problems. IEEE Transactions on Neural Networks 8, 630–645 (1997) 5. Lehtokangas, M.: Modified cascade-correlation learning for classification. IEEE Transactions on Neural Networks 11, 795–798 (2000) 6. Lehtokangas, M.: Fast initialization for cascade-correlation learning. IEEE Transactions on Neural Networks 10, 410–413 (1999) 7. Lehtokangas, M.: Modeling with constructive backpropagation. Neural Networks 12, 707–716 (1999) 8. Ma, L., Khorasani, K.: Constructive feedforward neural networks using hermite polynomial activation functions. IEEE Transactions on Neural Networks 16, 821– 833 (2005) 9. Monirul Islam, Md., Murase, K.: A new algorithm to design compact two-hiddenlayer artificial neural networks. Neural Networks 14, 1265–1278 (2001) 10. Phatak, D.S., Koren, I.: Connectivity and performance tradeoffs in the cascade correlation learning architecture. IEEE Transactions on Neural Networks 5, 930– 935 (1994) 11. Prechelt, L.: Automatic early stopping using cross validation: quantifying the criteria. Neural Networks 11, 761–767 (1998) 12. Reed, R.: Pruning algorithms - a survey. IEEE Transactions on Neural Networks 4, 740–747 (1993) 13. Prechelt, L.: PROBEN1- A set of neural network benchmark problems and benchmarking rules. Technical Report 21/94, Faculty of Informatics, University of Karlsruhe, Germany (1994)

A New Constructive Algorithm for Designing and Training ANNs

327

14. Riedmiller, M., Braun, H.: RPROP - A Fast Adaptive Learning Algorithm. In: Proc. of the 1992 International Symposium on Computer and Information Sciences, Turkey, November 1992, pp. 279–285 (1992) 15. Schapire, R.E.: The strength of weak learnability. Machine Learning 5, 197–227 (1990) 16. Schwenk, H., Bengio, Y.: Boosting Neural Networks. Neural Computation 12, 1869– 1887 (2000) 17. Setiono, R., Hui, L.C.K.: Use of quasi-Newton method in a feed forward neural network construction algorithm. IEEE Trans. Neural Networks 6, 273–277 (1995) 18. Tamura, S., Tateishi, M.: Capabilities of a four-layered feedforward neural network: four layers versus three. IEEE Transactions on Neural Networks 8, 251–255 (1997) 19. Treadgold, N.K., Gedeon, T.D.: Exploring constructive cascade networks. IEEE Transactions on Neural Networks 10, 1335–1350 (1999) 20. Yao, X.: A review of evolutionary artificial neural networks. International Journal of Intelligent Systems 8, 529–567 (1993)

Effective Learning with Heterogeneous Neural Networks Llu´ıs A. Belanche-Mu˜ noz Dept. de Llenguatges i Sistemes Inform` atics, Universitat Polit`ecnica de Catalunya, Barcelona, Spain [email protected]

Abstract. This paper introduces a class of neuron models accepting heterogeneous inputs and weights. The neuron model computes a userdefined similarity function between inputs and weights. The neuron transfer function is formed by composition of an adapted logistic function with the power mean of the partial input-weight similarities. The resulting neuron model is capable of dealing directly with mixtures of continuous quantities (crisp or fuzzy) and discrete quantities (ordinal, integer, binary or nominal). There is also provision for missing values. An artificial neural network using these neuron models is trained using a breeder genetic algorithm until convergence. A number of experiments are carried out using several real-world benchmarking problems. The network is compared to a standard radial basis function network and to a multi-layer perceptron and shown to learn from non-trivial data sets with superior generalization ability in most cases, at a comparable computational cost. A further advantage is the interpretability of the learned weights.

1

Introduction

In many important domains from the real world, objects are described by a mixture of continuous and discrete variables, usually containing missing information and characterized by an underlying uncertainty or imprecision. For example, in the well-known UCI repository [1] over half of the problems contain explicitly declared nominal attributes, let alone other data types, usually unreported. In the case of artificial neural networks (ANN), this heterogeneous information has to be encoded in the form of real-valued quantities, although in most cases there is enough domain knowledge to characterize the nature of the variables. We present a general framework for dealing with data heterogeneity in ANNs under the conceptual cover of similarity, in which a class of neuron models accepting heterogeneous inputs and weights computes a user-defined similarity function between these inputs and weights. The similarity function is defined by composition of a specific power mean of the partial input-weight similarities plus a logistic function. The resulting neuron model then accepts mixtures of continuous and discrete quantities, with explicit provision for missing information. Other data types are possible by extension of the model. M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 328–337, 2008. c Springer-Verlag Berlin Heidelberg 2008 

Effective Learning with Heterogeneous Neural Networks

329

An artificial neural network using these neuron models is trained using a breeder genetic algorithm until convergence. A number of experiments are carried out that illustrate the validity of the approach, using several benchmarking problems (classification and regression), selected as representatives because of the diversity in the richness and kind of data heterogeneity. The network is compared to a standard radial basis function neural network and to a multi-layer perceptron, and shown to learn from non-trivial data sets with a generalization ability that is superior in most cases to both networks. A further advantage of the approach is the interpretability of the learned weights. The paper is organized as follows. Section 2 further motivates the basis of the approach and reviews some widespread ways of coping with data heterogeneity in ANNs; section 3 introduces material on similarity measures; section 4 builds the heterogeneous neural network and proposes specific similarity measures. Finally, section 5 presents experimental work.

2

Motivation

The action of a feed-forward ANN may be viewed as a mapping through which points in the input space are transformed into corresponding points in an output space. The task is to find a structure in the data, to transform it from an input space to a new hidden space (the space spanned by the hidden units) in such a way that the problem is simpler when projected to this space. This processing is repeated for all the subsequent hidden layers. It seems clear that adequate transformations will require less units and therefore a hidden space of less dimensions. This in turn leads to simpler mappings which are less likely to overfit. The integration of heterogeneous data sources in information processing systems has been advocated elsewhere [2]. In this sense, a shortcoming of the existent neuron models is the difficulty of adding prior knowledge to the model. ANN current practice assumes that an input may be faithfully represented as a point in Rn , and the geometry of this space is meant to capture the meaningful relations in input space. There is no particular reason why this should be the case. Knowledge representation influences the practical success of a learning task [3], but has received little attention in the ANN community. The form in which data are presented to the network is of primary relevance. In essence, the task of the hidden layer(s) is to find a new, more convenient representation for the problem given the data representation chosen, a crucial factor for a successful learning process that can have a great impact on generalization ability [4]. As Anderson and Rosenfeld put it, “prior information and invariances should be built into the design of a ANN, thereby simplifying the network design by not having to learn them” [5]. Further, the activity of the units should have a well defined meaning in terms of the input patterns [6]. Without the aim of being exhaustive, commonly used methods (see, e.g. [7,4]) are the following: Ordinal variables coded as real-valued or using a thermometer. Nominal variables coded using a 1-out-of-c or c − 1 representation.

330

L.A. Belanche-Mu˜ noz

Missing information is difficult to handle, specially when the lost parts are of significant size. The entire examples can be removed, or the values “filled in” with the mean, median, nearest neighbour or by adding another input equal to one only if the value is absent. Statistical approaches need to model the input distribution itself [4], or are computationally intensive[8]. Vagueness and uncertainty are considerations usually put aside. These encodings being intuitive, their precise effect on network performance is not clear, because of the change in input distribution, the increase (sometimes acute) in input dimension and other subtler mathematical effects derived from imposing an order or a continuum where there was none. This issue can dramatically affect generalization for the worse, due to the curse of dimensionality. In this scenario, the use of hidden units which can adequately capture the desired mapping and do not add input dimension is thus of great practical importance.

3

Similarity Measures

Let us represent patterns belonging to a space X  = ∅ as a vector xi of n components, where each component xik represents the value of a particular feature k. A similarity measure is a unique number expressing how “like” two patterns are, given these features. A similarity measure may be defined as a function s : X × X −→ Is ⊂ R. Assume that s is upper bounded, exhaustive and total. This implies that Is is upper bounded and also that sup Is exists. Define smax ≡ sup Is and define smin ≡ inf Is if it exists. Let us denote by sij the similarity between xi and xj , that is, sij = s(xi , xj ). A similarity measure may be required to satisfy the following axioms, for any xi , xj ∈ X: Axiom S1. (Reflexivity) s(xi , xi ) = smax . This implies sup Is ∈ Is . Axiom S2. (Consistency) s(xi , xj ) = smax =⇒ xi = xj . Axiom S3. (Symmetry) s(xj , xi ) = s(xi , xj ). Axiom S4. (Boundedness) s is lower bounded when ∃a ∈ R such that s(xi , xj ) ≥ a, for all xi , xj ∈ X. This is equivalent to ask that inf Is exists. Axiom S5. (Closedness) a lower bounded function s is closed if there exist xi , xj ∈ X such that s(xi , xj ) = smin . This is equivalent to ask that inf Is ∈ Is . The similarity axioms enumerated above ensure a consistent definition of such functions, but should be taken as desiderata. Some similarity relations may fulfill part or all of them [9]. Other properties (like transitivity) may be of great interest in some contexts, but are not relevant for this work. The only requirement expressed so far about the set X is the existence of an equivalence relation. However, elements in this set can be simple or complex (i.e. composed by one or more variables). For atomic elements the similarity can be trivially computed, but for complex elements, we need to define a way to combine the partial similarities sk for each variable k to get a meaningful value sijk = sk (xik , xjk ). This combination has an important semantic role and it is not a trivial choice. We use in this work the concept of an A-average [10], as

Effective Learning with Heterogeneous Neural Networks

331

follows. Let [a, b] be a non-empty real interval. Call A(x1 , . . . , xn ) the A-average of x1 , ..., xn ∈ [a, b] to every n-place real function A fulfilling: Axiom A1. A is continuous, symmetric and strictly increasing in each xi . Axiom A2. A(x, . . . , x) = x. Axiom A3. For any positive integer k ≤ n: A(x1 , . . . , xn ) = A(yk , . . . , yk , xik+1 , . . . , xin )    k times where yk = A(xi1 , , . . . , xik ) and (i1 , ..., in ) is a permutation of (1, . . . , n). The means defined by these axioms fulfill Cauchy’s property of means, namely, that min xi ≤ A(x1 , . . . , xn ) ≤ max xi with equality only if x1 = x2 = . . . = xn . The proof is straightforward using axioms A1 and A2. This result is even stronger than idempotency; thus, the previous bounds actually impose idempotency. A further interesting property of these means is that we can add averaged elements to an A-average without changing the overall result. Formally, A(x1 , ..., xn ) = A(z1 , ..., zm , x1 , ..., xn ) if and only if A(z1 , ..., zm ) = A(x1 , ..., xn ). As a consequence, if y = A(x1 , ..., xn ), then A(x1 , ..., xn , A(x1 , ..., xn )) = y. Theorem. Let f : [a, b] −→ R be a continuous, strictly monotone mapping. Let g be the inverse function of f . Then,  n  1 A(x1 , ..., xn ) ≡ g f (xi ) n i=1 is a well-defined A-average for all n ∈ N and xi ∈ [a, b] [10]. An important class of A-averages is formed by choosing f (z) = z q , q ∈ R, to obtain:  Mq (x1 , ..., xn ) =

1 (xi )q n i=1 n

 q1

Several well-known means are found by choosing particular values of q. Specifically, the arithmetic mean for q = 1, the geometric mean for q = 0, the quadratic mean for q = 2 and the harmonic mean for q = −1. A property of the means Mq is that Mq (x1 , ..., xn ) ≥ Mq (x1 , ..., xn ) if and only if q > q  , with equality only if x1 = x2 = . . . = xn . We shall refer to these as the power means.

4 4.1

Heterogeneous Neural Networks The Heterogeneous Neuron Model

We develop in this section neuron models allowing for heterogeneous and imˆ n → Rout ⊆ R. Here R denotes precise inputs, defined as a mapping h : H n ˆ the reals and H is a cartesian product of an arbitrary number n of source sets ˆ (k) , k = 1 . . . , n. These source sets may include extended reals R ˆ k = Rk ∪ {X } H

332

L.A. Belanche-Mu˜ noz

(where each Rk is a real interval), extended families of fuzzy sets Fˆk = Fk ∪{X }, ˆk = Ok ∪ {X }, M ˆ k = Mk ∪ {X }, where and extended finite sets of the form O the Ok have a full order relation, while the Mk have not. The special symbol X extends the source sets and denotes the unknown element (missing information), behaving as an incomparable element w.r.t. any ordering relation. According to this definition, neuron inputs are vectors composed of n elements among which there might be reals, fuzzy sets, ordinals, categorical and missing data. ˆn × H ˆ n → Is a similarity function in H ˆ n , where Consider now a function s : H we take Is ⊆ [0, 1] for simplicity. This function is formed by combination of ˆ (k) × H ˆ (k) → Is . The sk measures are normalized n partial similarities sk : H to a common real interval ([0, 1] in this case) and computed according to different formulas for different variables (possibly but not necessaryly determined by variable type alone). The desired neuron model can be devised that is both similarity-based and handles data heterogeneity and missing values, as follows. ˆ n having a Let Γi (x) the function computed by the i-th neuron, where x ∈ H n ˆ weight vector μi ∈ H and smoothing parameter γi . Then Γi (x) = sˇ(γi Mq ([sk (xk , μi,k )]nk=1 )), q ∈ R, γi ∈ (0, 1] The activation function sˇ is any sigmoid-like automorphism (a monotonic bijection) in [0, 1]. The eventually missing similarities (because one or both of their arguments is missing) are regarded as ignorance and must not contribute in favour nor against the overall measure. In order to do this, let C = {k1 , . . . , km } be the set of indexes of the similarities that could not be computed. Then define sX (x, μi ) = Mq ([sk (xk , μi,k )]nk=1,k∈C / ), that is the power means of the successfully computed similarities. Thanks to the previous properties of A-averages, Mq ([sk (xk , μi,k )]nk=1 ) = Mq ([sk (xk , μi,k )]nk=1,k∈C / , sX (x, μi ), . . . , sX (x, μi )).    m times 4.2

Heterogeneous Neural Networks

Heterogeneous neural networks are neural architectures built out of the previously defined neuron models, thus allowing for heterogeneous or missing inputs. A feed-forward architecture, with a hidden layer composed of heterogeneous neurons and a linear output layer is a straightforward choice, thus conforming a hybrid structure. The heterogeneous neural network computes the function: fhnn (x) =

h 

αi Γi (x)

(1)

i=1

where h > 0 is the number of (heterogeneous) hidden neurons and {αi } is the set of mixing coefficients. The HNN thus keeps linearity in the output layer and interpretability in the hidden layer. It can be naturally seen as a generalization of the RBF. This is so because the response of hidden neurons is localized: centered at a given object (the neuron weight, where response is maximum), falling down

Effective Learning with Heterogeneous Neural Networks

333

as the input is less and less similar to that center. The general training procedure for the heterogeneous neural network (HNN for short) is based on evolutionary algorithms, due to the missing information and the likely non-differentiability of the similarity measure. Specifically, the training procedure for the HNN used in this work is based on a Breeder Genetic Algorithm (BGA) [11], a method in mid position between Genetic Algorithms (GA) and Evolution Strategies. While in GA selection is stochastic and meant to mimic Darwinian evolution, BGA selection is driven by a deterministic breeding mechanism, an artificial method in which only the best individuals —usually a fixed percentage τ of total population size μ— are selected to be recombined and mutated, as the basis to form a new generation. The used BGA does not need any coding scheme, being able to cope with heterogeneous or missing genes [12]. 4.3

Heterogeneous Similarity Measures

We present in this section specific measures defined to belong to the interval [0, 1] for the sake of illustration, but there are many variations that may be superior by making better use of available expert knowledge. Ordinal variables. It is assumed that the values of the variable form a linearly ordered space (O, ). Let xik , xjk ∈ O, xik  xjk , Plk the fraction of values of variable k that take on the value xlk and the summation run through all the ordinal values xlk such that xik  xlk and xlk  xjk [13]. sijk =

2log(Pik + . . . + Pjk ) logPik + . . . + logPjk

(2)

Nominal variables. It is assumed that no partial order exists among these values and the only possible comparison is equality. The basic similarity measure for these variables is the overlap. Let N be a nominal space and xik , xjk ∈ N . Then sijk = 1 if xik = xjk and 0 otherwise. Continuous variables. Let x, y ∈ Γ = [r− , r+ ] ⊂ R, r+ > r− . The standard metric in R is a metric in Γ . Therefore, for any two values xik , xjk ∈ R: ⎛ ⎞ ⎜ |xik − xjk | ⎟ sijk = sˆR ⎝ ⎠ sup |x − y|

(3)

x,y∈Γ

where sˆR : [0, 1] −→ [0, 1] is a decreasing continuous function. The family sˆR (z) = (1 − z β )α , 0 < β ≤ 1, α ≥ 1 is used here, with α = 2, β = 1. Integer variables. Given that N ⊂ R, any distance-based similarity in R is also valid in N. A flexible choice does not limit the range of integer values (assumed positive for convenience). In this case, a self-normalizing distance-based similarity sijk = sˆN (|xik − xjk |) is indicated, where sˆN : [0, ∞) −→ (0, 1] is a 1 decreasing continuous function. In particular, the function sˆN (z) = 1+z can be used.

334

L.A. Belanche-Mu˜ noz

Binary variables. In the data analysis literature there are many similarity measures defined on collections of binary variables (see e.g. [14]). This is mostly due to the uncertainty over how to accommodate negative (i.e. false-false) matches. The present situation is that of comparison of a single binary variable rather than a whole vector. In general, one should know which of the two matches is the really relevant (true-true or false-false). For these reasons, treating the variable as purely nominal can result in a loss of relevant information. Since this meta-knowledge is usually not handy, we use in this work a frequencybased approach, as follows. Let xik , xjk be two binary values and let Plk be again the fraction of values of variable k that take on the value xlk . We define: 2xy sijk = h(1 − Pik , 1 − Pjk ), where h(x, y) = x+y is the harmonic mean between x and y. This measure compares the agreement on the rarity of each value: the similarity is higher the less frequent the values are. Fuzzy variables. For variables representing fuzzy sets, similarity relations from the point of view of fuzzy theory have been defined elsewhere [15], and different choices are possible. In possibility theory, the possibility expresses the likeliness of co-occurrence of two vague propositions, with a value of one standing for ˜ B ˜ of a reference set X, it is defined as: absolute certainty. For two fuzzy sets A, ˜ Π(A) ˜ (B) = sup (μA ˜ ∩B ˜ (u)) = sup (min (μA ˜ (u), μB ˜ (u))) u∈X

u∈X

In our case, if Fk is an arbitrary family of fuzzy sets, and xik , xjk ∈ Fk , the following similarity relation can be used sijk = Π(xik ) (xjk ). Notice that this measure indeed fulfills axioms S1, S2 and S3.

5

An Experimental Comparison

A number of experiments are carried out to illustrate the validity of the approach, using nine benchmarking problems, selected as representatives because Table 1. Basic features of the data sets. ‘Missing’ refers to the percentage of missing values, p to the number of examples and n to that of variables. Types are: C (continuous), N (nominal), I (integer), D (ordinal) and F (fuzzy). ‘Output’ is R for regression or a number indicating the number of classes. Name Annealing Audiology Boston Housing Credit Screening Heart Disease Horse Colic South African Heart Servo Data Solar Flares

p 898 226 506 690 920 364 462 167 1066

n Data Types Output Missing 28 6C,19N,3I 6 none 69 8D,61N 4 2.3% 13 11C,1I,1B R none 15 6C,6N,3B 2 0.6% 13 3C,5N,3I,2B 2 16.2% 20 6D,5N,2I,5C,2B 3 26.1% 9 7C,1I,1N 2 none 4 2I,2N R none 9 4N,3D,2B 2 none

Effective Learning with Heterogeneous Neural Networks

335

of the richness in data heterogeneity, taken from [7] and [1]. Their main characteristics are displayed in Table 1. For every data set, the available documentation is analysed for an assessment on the more appropriate treatment. Missing information is also properly identified. The HNN is compared to a standard radial basis function network (RBF) and to a multi-layer perceptron (MLP). The three networks are trained using 72

1 RBF MLP

70

RBF MLP 0.9

HNN

68

HNN

0.8

66 0.7 64 0.6 62 0.5 60 0.4

58

0.3

56

54

0.2 0

3

6

9

12

15

18

21

24

27

30

0

3

6

(a) Audiology

9

12

15

18

21

24

27

30

24

27

30

24

27

30

24

27

30

(b) Boston Housing

90

80

85

RBF MLP

RBF MLP

HNN

HNN 75

80

70 75

70 65

65 60 60

55

55 0

3

6

9

12

15

18

21

24

27

30

0

3

6

(c) Credit Screening

9

12

15

18

21

(d) Heart Disease

70

94 RBF MLP

RBF MLP

92

HNN 65

HNN

90 88

60

86 84

55

82 80

50

78 76

45

74 0

3

6

9

12

15

18

21

24

27

30

0

3

6

9

(e) Horse Colic

12

15

18

21

(f) Annealing

78

81.5 RBF MLP

76

RBF MLP

81

HNN

HNN 80.5

74 80 72 79.5 70

79

68

78.5 78

66 77.5 64 77 62

76.5

60

76 0

3

6

9

12

15

18

21

24

(g) South African Heart

27

30

0

3

6

9

12

15

18

21

(h) Solar Flares

Fig. 1. Generalization results for the networks. See text for an explanation of axis.

336

L.A. Belanche-Mu˜ noz

exactly the same procedure to exclude this source of variation from the analysis. The RBF and MLP networks need a pre-processing of the data, following the recommendations in [7]. The RBF neuron model has provision for a smoothing parameter, different for every hidden neuron. The network architecture is fixed to one single layer of hidden neurons ranging from 1 to 30 plus as many output neurons as required by the task, logistic for classification problems and linear otherwise. The input variables for the RBF and MLP network are standardized to zero mean, unit standard deviation. This is not needed by the heterogeneous neurons because they compute a normalized measure, but is beneficial for the other networks. The output is presented for all networks in 1-out-of-c mode (that is, c outputs) for classification problems with c classes. The error measure in this case is generalized cross-entropy (GCE) [4]. For regression problems, the mean is subtracted to the continuous output and the normalized root mean square (NRMS) error is reported. Each data set is split in three parts, for training (TR), validation (VA) and test (TE), as 50%-25%-25%. The BGA task is to minimize error (either NRMS or GCE) on the TR part, until 500 generations or convergence. Then the network having the lowest error on the VA part is returned. The reported performance is the NRMS or GCE of that network on the TE part. This process is repeated ten times and the average is taken. We present performance results in Fig. 1(a) to (h). In each figure, the abscissa represents the number of neurons in the hidden layer, and the ordinate the average NRMS or GCE of that network on the TE part, as explained above. For classification problems, the abscissa value 0 is used to indicate the percentage of examples in the majority class (that is, the minimum acceptable performance, as a reference). For regression problems, this value is set to 1.0. Note that for classification problems, higher values are better, whereas for regression problems, lower values are better. The HNN shows enhanced performance for most of the problems, specially for those displaying higher heterogeneity or missingness, and less so when this is not the case. This is reasonable since the HNN behaves as a RBF network whene there is no heterogeneity or missingness. The interpretability of the learned weights is also enhanced, since the weights are of the same type as their matching input. We additionally present performance results for the Servo Data (Fig. 2), which are specially interesting. This is a small data set with only 4 variables (none of them continuous) and less than 200 examples. For 16

1.1 RBF

RBF

MLP 14

HNN

HNN

1

12 0.9 10 0.8 8 0.7 6 0.6 4

0.5

2

0

0.4 0

3

6

9

12

15

(a)

18

21

24

27

30

0

3

6

9

12

15

18

21

24

27

30

(b)

Fig. 2. Generalization performance results for the Servo Data. Left: heavy overfit by the MLP network. Right: only RBF and HNN.

Effective Learning with Heterogeneous Neural Networks

337

this data set the MLP overfits heavily, due to the amplification in the number of variables introduced by the coding scheme of the nominal ones, which are many-valued.

6

Conclusions

An heterogeneous neuron that computes a similarity index in [0, 1], followed by a logistic function has been proposed. Neural architectures making use of such neurons are termed heterogeneous neural networks. A body of experimental evidence has shown very promising results in several difficult learning tasks. These results also illustrate the importance of data preparation and the use of a more adequate similarity measure that captures relations in the data. As further work, we are currently working on an clustering-based alternative training scheme.

References 1. Murphy, P.M., Aha, D.: UCI Repository of machine learning databases. UCI Dept. of Information and Computer Science (1991) 2. Kaburlassos, V., Petridis, V.: Learning and decision-making in the framework of fuzzy lattices. In: New learning paradigms in soft computing, Physica (2002) 3. Natarajan, B.: Machine learning: A theoretical approach. Morgan Kaufmann, San Francisco (1991) 4. Bishop, C.: Neural Networks for Pattern Recognition, Oxford (1995) 5. Anderson, J.A., Rosenfeld, E. (eds.): Neurocomputing: foundations of research. MIT Press, Cambridge (1988) 6. Omohundro, S.M.: Geometric Learning Algorithms. Technical Report 89-041. Univ. of Berkeley, CA (1989) 7. Prechelt, L.: Proben1-A set of Neural Network Benchmark Problems and Benchmarking Rules. Fac. f¨ ur Informatik. U. Karlsruhe T.R. 21/94 8. Tresp, V., Ahmad, S., Neuneier, R.: Training Neural Networks with Deficient Data. In: NIPS, vol. 6, Morgan Kaufmann, San Francisco (1994) 9. Chandon, J.L., Pinson, S.: Analyse Typologique. Masson (1981) 10. Hille, E.: Methods in Classical and Functional Analysis. Addison-Wesley, Reading (1971) 11. M¨ uhlenbein, H., Schlierkamp-Voosen, D.: Predictive Models for the Breeder Genetic Algorithm. Evolutionary Computation 1(1) (1993) 12. Belanche, L.: Evolutionary Optimization of Heterogeneous Problems. In: Guerv´ os, J.J.M., Adamidis, P.A., Beyer, H.-G., Fern´ andez-Villaca˜ nas, J.-L., Schwefel, H.-P. (eds.) PPSN 2002. LNCS, vol. 2439, pp. 475–484. Springer, Heidelberg (2002) 13. Lin, D.: An Information-Theoretic Definition of Similarity. In: Proceedings of International Conference on Machine Learning (1998) 14. Everitt, B.: Cluster Analysis. Heinemann Books (1977) 15. Zadeh, L.: Fuzzy Sets as a basis for a theory of possibility. Fuzzy Sets and Systems 1, 3–28 (1978)

Pattern-Based Reasoning System Using Self-incremental Neural Network for Propositional Logic Akihito Sudo, Manabu Tsuboyama, Chenli Zhang, Akihiro Sato, and Osamu Hasegawa Dept. of Computer Intelligence and Systems Science, Tokyo Institute of Technology Imaging Science and Engineering Lab., Tokyo Institute of Technology 4259 Nagatsuta-cho, Midori-ku, Yokohama, Japan {sudo,hasegawa}@isl.titech.ac.jp http://www.isl.titech.ac.jp/∼hasegawalab/

Abstract. We propose an architecture for reasoning with pattern-based if-then rules that is effective for intelligent systems like robots solving varying tasks autonomously in a real environment. The proposed system can store pattern-based if-then rules of propositional logic, including conjunctions, disjunctions, negations, and implications. The naive pattern-based reasoning can store pattern-based if-then rules and make inferences using them. However, it remains insufficient for intelligent systems operating in a real environment. The proposed system uses an algorithm that is inspired by self-incremental neural networks such as SONIN and SOINN-AM in order to achieve incremental learning, generalization, avoidance of duplicate results, and robustness to noise, which are important properties for intelligent systems. Keywords: Reasoning, neural network, intelligent agent, intelligent robot.

1

Introduction

Reasoning is an essential process of human intelligence. Although many reasoning systems have been proposed, they remain insufficient for intelligent systems that must crawl about in the real world, such as robots. We consider that one reason is that most of them address only symbol-based if-then rules. Several reasoning systems have been recognized for their effectiveness and have been commercialized, but they perform only in a specific domain. Although production systems are a much sought conventional reasoning system, they require human experts to input their knowledge in advance. Consequently, such systems perform well only in a specific domain, the knowledge of which an expert has provided. In contrast, nobody can predict correctly what the complete knowledge list for the intelligent systems operating in a varying environment; it must therefore learn knowledge that is provided sequentially from environments in an incremental manner. If the intelligent system uses a conventional symbol-based reasoning M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 338–347, 2008. c Springer-Verlag Berlin Heidelberg 2008 

Pattern-Based Reasoning System Using Self-incremental Neural Network

339

system to make inferences using learned knowledge from environments, it must convert learned patterns to symbols before reasoning because what it obtains from environments through its sensors are patterns, not symbols. This strategy seems to be impractical currently and might be impossible even in the future. Converting patterns to symbols remains a difficult task so far in spite of many studies having been done as a task of classification. Discriminating cats from dogs is not an easy problem for current discriminators. Problems will remain even after one invents a classifier as powerful as a human. Intelligent systems must generate new symbols when they encounter what should be addressed as another symbol than those they already know. Humans must think or act with or about the object to assign a symbol to it. Then, humans must form inferences using patterns obtained from the object because it has not been symbolized yet. Intelligent systems also must make inferences using patterns to yield new symbols. Several approaches were proposed to realize pattern-based reasoning systems. Tsukimoto proposed pattern-based reasoning where perceptrons, which have learned patterns, are employed as atomic propositions of if-then rules[4]. In [3], a pattern-based reasoning system was proposed where binary vectors are employed as atomic propositions using a non-monotone neural network[3]. The method proposed in [4] can deal with both conjunction, disjunction and negation although one proposed by Yamane et al. can’t. On the other hand, patterns should be represented with vectors rather than perceptrons for robots because it is difficult to determine when a new perceptron is generated as a new proposition. Moreover, real-valued data should be dealt with because sensor information take real value although [3] deal with only binary vectors. There has been no study which realizes a pattern-based reasoning system which can deal with both conjunction, disjunction and negation and employs real-valued vector as a representation of a pattern. In this paper, we propose a novel reasoning system that can deal with both conjunction, disjunction and negation and employs real-valued vector as a representation of a pattern. The proposed method can learn if-then rules, atomic propositions of which are patterns represented as real-valued vectors. For instance, the proposed method can learn “((A∧B)∨C)→(D∧E∧F)”, where A–F are patterns represented as real-valued vectors1 . When the proposed system having learned if-then rules obtains a fact, it produces an inference from the fact, picking up and connecting learned if-then rules. It outputs C∨D and (E∧¬F)∨D if the proposed system which has learned (A∧B)→(C∨D) and C→(E∧¬F) obtains “A∧B” as a fact. In addition to basic function for pattern-based reasoning discribed above, some additional functions are realized by the proposed method for application to an intelligent system like a robot operating in dynamic environments: incremental learning, generalization ability, avoidance of duplication of reasoning results, and robustness to noise. The proposed system achieves them through 1

We use ∧, ∨, ¬, →, respectively, for conjunction, disjunction, negation, and implication.

340

A. Sudo et al.

Fig. 1. Two types of noisy data

the use of a self-incremental neural network inspired by SOINN[1] and SOINNAM[2]. Incremental learning of if-then rules is required because a user cannot provide a system that has complete knowledge in advance. The proposed system can learn if-then rules that are provided incrementally without collapsing previously learned knowledge. In [3], about 20% of reasoning is incorrect when it has incrementally learned new if-then-rules which is the same amount of ifthen rules that had been learned previously in a batch manner. The proposed system does not store the same knowledge as (or very similar knowledge to) a previously stored one. This enables the proposed system to avoid suffering redundancy of memory size when it learns much knowledge incrementally. In a real environment, robots rarely obtain identical data to previously learned data. The proposed system can find out similarly learned if-then rules when different data are given. The proposed system categorizes learned if-then rules in online manner to avoid duplication of reasoning results. If the if-then rules are not categorized, similar if-then rules are used completely as other rules, which engenders the duplication of results. For example, if the system which does not categorize knowledge stores “A → B” and “A’ →B’” where (A, A’) and (B, B’) are very similar pattern pairs, both B and B’ are outputs even when they can be interpreted as the same result. Note that our approach is categorizing if-then rules, not each pattern. We consider that our approach engenders generating symbols by intelligent systems as a result of their own reasoning. Two types of noise exist in real environments, as shown in Fig. 1. Noise shown in Fig. 1(a) taints the original data. Noise shown in Fig. 1(b) is a pattern that is independent from the original patterns. The proposed system is robust to both types of noise.

2

Proposed Method

The proposed system consists of long-term memory, short-term memory, a learning machine, and a reasoning machine, as shown in Fig. 2. The long-term memory stores if-then rules learned from environments. The short-term memory temporarily stores input data. The learning machine executes a learning algorithm when if-then rules are provided. Unnecessary if-then rules are eliminated and ifthen rules are categorized into several clusters in an online manner when learning. The reasoning machine executes a reasoning algorithm and outputs an OR tree as a reasoning result when a fact like A ∧ B is given. Not only the minimum function, but also incremental learning, generalization, avoidance of duplication of reasoning results, and robustness to noise are achieved through the use of

Pattern-Based Reasoning System Using Self-incremental Neural Network Input

Long Term Memory

Short Term Memory Learning Phase

If-Then Rule Reasoning Phase Fact Reference

Learning Machine

Find winners in LTM New knowledge? YES

NO

Add Inputted If-then Rule Make an edge between the first winner and the second winner

341

Cluster 1 Cluster 2 Cluster 3 ･･･

Reference Reference update Reasoning Machine Generate a tree which has Add conclusion parts single node holding input Eliminate old edges corresponding to the similar Find conditioning parts conditioning parts The number of input data is similar to input as child nodes greater than threshold? YES Remove knowledge belonging to Similar conditioning part exists? Find conditioning parts similar to leafs of the tree cluster which has less than two YES members. Reference

Output

Fig. 2. Architecture of the proposed system

algorithms that are inspired by a self-incremental neural network that is based on SOINN[1] and SOINN-AM[2]. 2.1

Learning Algorithm

The learning machine executes the learning algorithm when the system obtains an if-then rule. First, the input if-then rule is resolved into several if-then rules, both the conditional part and the sequential part of which are conjunctive of literals, which is accomplished as follows: both the conditional part and the sequential part are transformed to disjunctive normal forms; then every combination of clauses is picked up. For example, if “(A∧B)∨(C∧¬D)→(¬E∧F)” is input, it is resolved to “(A∧B)→(¬E∧F)” and “(C∧¬D)→(¬E∧F)”. Note that a conditional part and sequential part of any if-then rule can be transformed to a disjunctive normal form because any form of a logical formula can be transformed to a disjunctive normal form. The resolved if-then rules are stored temporarily in the short-term memory. The system would suffer a high computational load and duplication of reasoning results if all the resolved if-then rules were simply stored in the long-term memory. The steps described in the paragraph below let the learning machine determine which if-then rule should be stored in the long-term memory and categorize if-then rules in the long-term memory into several clusters using edges, which are carried out in online manner. Those steps are executed repeatedly for every resolved if-then rule. Note that a “learning datum” below denotes a single resolved if-then rule. First, the most similar if-then rule (first winner) and the second-most similar if-then rule (second winner) to a learning datum are sought among the if-then rules in the long-term memory which are the same form of the learning datum. Here, the same form means that the number of positive and negative propositions of the conditional part and the sequential part are identical to each other. For example, “A∧¬B→C” is regarded as having the same form of “D∧¬E→F”, but

342

A. Sudo et al.

is not regarded as the same form of “¬D∧¬E→F”. The similarity between the same-form if-then rules are calculated using the following distance measure d:  M N   1  d(k, k ) = √ min Pi − Pk i  + min Qi − Qki  + {k1 ,···,kN }∈NN D {k1 ,···,kM }∈NM i=1 i=1  ˜ ˜ M N   ˜i − Q ˜ k , min P˜i − P˜ ki  + min Q {k1 ,···,kM ˜ }∈NM

{k1 ,···,kN ˜ }∈NN

i=1

i

i=1

where D represents the dimension of the patterns, NM = {1, 2, · · · , M }, NN = {1, 2, · · · , N }, and the learning data k and an if-then rule k  stored in the longterm memory are denoted respectively as follows. ⎞ ⎛ ⎞ ⎛ ⎞ M  ⎛ N ˜ ˜ M N     ˜j ⎠ Pi ∧ ⎝ ¬Qj ⎠ → ⎝ P˜i ⎠ ∧ ⎝ ¬Q i=1

M  i=1

 Pi

⎛ ∧⎝

j=1 N 

j=1

⎞

⎛

¬Qj ⎠ → ⎝

i=1 ˜ M 

i=1

⎞

⎛

P˜ i ⎠ ∧ ⎝

j=1 ˜ N 

⎞ ˜ j ⎠ ¬Q

j=1

Then, whether a learning datum is a member of the same cluster of both the first winner and the second winner is determined. The method of the determination is described in the end of this subsection. The learning datum is added to the longterm memory as new knowledge and the learning process for the learning data finishes if the learning datum is not a member of the same cluster of either the first winner or the second winner. If the learning datum is a member of the same cluster of both the first winner and the second winner, then the learning datum is not added to the long-term memory and the previously stored knowledge in the long-term memory is updated as follows. An edge between the first winner and the second winner is generated if the edge has not been generated yet, and the age of the edge is set to zero. That edge is used to generate clusters of ifthen rules in the long-term memory. If-then rules connected with an edge are regarded as members of the same cluster. Subsequently, the age of every edge emanating from the first winner is increased by 1. Every edge with age greater than a parameter Λedge , is removed. If-then rules in the long-term memory which have less than two edges are removed if a parameter λ is an integer multiple of the number of the previously provided learning data. The noise depicted in Fig. 1(b) is expected to be eliminated by this step. Here, we describe the method to determine whether a learning datum is a member of the same cluster of both the first and the second winners. The similarity threshold of the first (or second) winner is calculated as follows: maxk∈N d(k, kw ) (if N = ∅) s= , maxk∈A d(k, kw ) (if N = ∅)

Pattern-Based Reasoning System Using Self-incremental Neural Network

343

A∧B D∧￢E

C F

G

H∧￢I

￢J

Fig. 3. Example of an OR tree as a result of reasoning

where N is the set of all the if-then rules connected to the first (or second) winner by the edge, A is the set of all the if-then rules in the long-term memory, and kw denotes the first (or second) winner. The learning datum is regarded as a member of the same cluster of both the first and second winners if the distance between the first winner and the learning datum is less than the similarity threshold of the first winner and the distance between the second winner and the learning datum is less than the similarity threshold of the second winner. 2.2

The Reasoning Algorithm

When a fact such as A∧B is input, the system generates the tree, the root of which is the input fact; no node exists without the root. The distance between the fact and the conditional part of each if-then rule in the long term memory is calculated. If there exist conditional parts with distance of less than δr , their sequential parts are added to the tree as children of the root. Here we describe details of the reasoning algorithm. When a fact is input, the tree exists for which the root is the fact and no node without the root exists. The distance between the fact and the conditional part of each if-then rule in the long-term memory is calculated. If there exist conditional parts with distance less than δr , their sequential parts are added to the tree as children of the root, where the only nearest sequential part is added from the same cluster if several sufficiently close sequential parts exist in the same cluster. The reasoning process terminates if no child is added to the tree. The above steps are applied to the children instead of the fact if a new child exists. The above steps are executed repeatedly as long as a new child is generated. Figure 3 shows an example of an output tree. The reasoning machine outputs that tree when A’∧B’ is obtained as a fact if the longterm memory holds “A∧B→C∨(D∧¬E)” and “C’→F∨G∨(H∧I), D’∧¬E’→¬J” where (A, A’) - (E, E’) are pairs of sufficiently similar patterns. An output tree should be addressed as an OR tree; then a deduced proposition is a disjunctive normal form generated by combining several propositions in nodes. If the reasoning machine outputs the tree shown in Fig. 3, then “C∨(D∧¬E)”, “C∨¬J”, “F∨G∨(H∧I) ∨(D∧¬E)”, and “F∨G∨(H∧I) ∨¬J” are correct reasoning results. A reasoning result that is represented as a disjunctive normal form can be interpreted as the possibility of an environment. The system discovers the possibility of places that the system can reach from the present location if the present location is input to the system as a fact and “the supermarket or the station or the drug store” is the reasoning result.

344

A. Sudo et al.

Conditional Rule

Sequential Rule

Input Layer

・・・

・・・

D

￢BD

￢C

￢￢BC

A

A

Competitive Layer

Fig. 4. Neural Network Model of the Proposed Method

2.3

Neural Network Model

The architecture of the proposed system is as shown in Fig. 2 in terms of knowledge engineering; the proposed system can be interpreted as a self-incremental neural network like SOINN[1], SOINN-AM[2], and Growing Neural Gas[5]. The proposed neural network model has an input layer and a competitive layer, as shown in Fig. 4. The input layer obtains an if-then rule or a fact. New nodes are generated adaptively when the input layer obtains an if-then rule. Each if-then rule is represented with several nodes connected by edges, although other selfincremental neural networks represent learned data with a single node. Those edges are different-type edges from those used to generate clusters. The proposed model requires m × n neurons to represent an if-then rule, the conditional part of which has m literals and the sequential part has n literals. Each neuron holds a pair of literals picked up from the conditional part and the sequential part one-by-one, which means that each neuron holds two vectors and flags of negation. Neurons representing A∧¬B→¬C∧D are shown in Fig. 4.

3

Simulation Results

In the experiment, the proposed system learned the sequentially provided ifthen rules, the atomic propositions of which were the images taken in the real environment. It then produced inferences using the learned knowledge. We used 280 images of the 14 objects shown in Fig. 5 taken from 20 different angles. Each image has 56 × 46 pixels. The if-then rules the systems learned were “A→B”,

A. Close door B. Open door

C. Lab.

D. Wall

E. Hallway

F. Elevator

G. Desk-1

H. Desk-2

I. Desk drawer-1 J. Desk drawer-2 K. Desk drawer-1 L. Desk drawer-2 M. Down stairs N. Room plate O. White Noise (close) (close) (open) (open)

Fig. 5. Images for the experiment

Pattern-Based Reasoning System Using Self-incremental Neural Network

345

Table 1. Number of members in the clusters and compression rate Cluster No. If-then rule Number of members Compression rate (%) 1 AB 87 78.25 2 BD 145 63.75 3 BE 50 87.5 4 E(CN) 64 84 5 EF 31 92.25 6 EM 146 63.5 7 (CN)(GI) 84 79 8 (CN)(HJ) 111 72.25 9 (GI)K 140 65 10 (HJ)L 122 69.5 Total — 980 75.5

∧ ∧

∧

Fig. 6. Reasoning results g n i n o s a e r t c e r r o c

)

100

% ( o i t a r

系列 δ系列2 δ系列1

1 δr = 0.3

50 0 10

15

20

25

SN ratio

30

r

= 0.2

r

= 0.1

35

Fig. 7. Ratio of correct reasoning when noisy data are given

“B→D∨E”, “E→(C∧N)∨F∨M”, “(C∧N)→(G∧I) ∨ (H∧J)”, “(G∧I)→K”, and “(H∧J)→L”. They mean “the closed door → the open door”, “the open door → the wall ∨ the hallway”, “the hallway → (the laboratory ∧ the room plate) ∨ the elevator ∨ the down stairs”, “the laboratory ∧ the room plate→(the desk 1 ∧ the closed drawer 1)∨(the desk 2 ∧ the closed drawer 2)”, “(the desk 1 ∧ the

346

A. Sudo et al.

closed drawer 1)→ the open drawer 1”, “(the desk 2 ∧ the closed drawer 2)→ the open drawer 2”. The system obtained if-then rules, each atomic proposition of which was selected randomly from 20 different-angled images. Random selection from among them provided learning data for 4400 times for each lf-then rule. The parameters were set to Λedge = 100, λ = 50. The number of if-then rules in the long-term memory after learning was 980; these rules are categorized into 10 different clusters. Each cluster had members corresponding respectively only to “A→B”, “B→D”, “B→E”, “E→(C∧N)”, “E→F”, “E→M”, “(C∧N)→(G∧I)”, “(C∧N)→(H∧J)”, “(G∧I)→K”, or “(H∧J)→L”; one example is the members of clusters that consisted only of if-then rules corresponding to “A→B”. Consequently, the system was able to categorize if-then rules appropriately, which were the same rules as those in a symbol-based manner, into the same cluster. Table 1 shows the number of members of each cluster and data compression rates. We estimated the compression rate as (1 − Nm/Nl ), where Nm is the number of members of the cluster and Nl is the number of if-then rules provided to the system as learning data. Although the learning data consisted of 4000 if-then rules, the atomic propositions of which were different from one another, only 980 rules were stored in the long-term memory in all. The system eliminated 3020 rules as similar rules of previously stored rules. Note that the system obtained the learning data sequentially. The result of the learning indicate that the system learns new data without collapsing previously learned data and avoids consuming memory through eliminating similar data. We provided one learned image as a fact to test whether the system can produce inferences correctly. The parameter was set to δr = 0.3. The reasoning result is shown in Fig. 6. This tree has neither omissions nor duplication. The system was able to avoid duplication of results because it generated clusters appropriately. In all, 500,376 nodes were created in the output tree when we made the system regard every if-then rule in the long-term memory as belonging to different clusters. The reasoning result can be interpreted thusly: the system discovered where it can reach from the front of the closed door. One can find the importance of storing an if-then rule which has a conjunction in the reasoning about the contents of the closed drawers. The system would not make a correct inference if it were unable to deal with the conjunction because the closed drawers appear to be almost identical. To examine whether the system can reason correctly from the type of noisy data shown in Fig. 1(b), we provided ten white noise samples, one example of which is shown in Fig. 5O. Then, the system made no child node of the white noises because it was able to judge that all the white noise consisted of unknown patterns. The system proposed in [3] cannot judge whether an input fact has been learned. It outputs a wrong pattern that is entirely unrelated to learned patterns when it obtains an unknown pattern. We generated noisy data by adding Gaussian noise to 20 images of Fig. 5A to examine that the system can reason correctly from the type of noisy data shown in Fig. 1(a). Actually, 10 noisy images were generated from the same image in each noise level, which means that the system obtained 200 noisy images of the same noisy level. We estimated the correct reasoning ratio as “the number of

Pattern-Based Reasoning System Using Self-incremental Neural Network

347

correct reasoning / the number of reasoning with the noisy image”. Figure 7 shows that a boundary SN ratio exists. The system outputs the same result as that of the system that obtained the original image if the SN ratio is greater than the boundary. The system regards the fact as an unknown pattern and generates no child node of the fact if the SN ratio of a fact is less than the boundary. The boundary can be configured through setting. The system proposed in [3] also has the boundary of noise level to produce correct inferences, but a user cannot configure it.

4

Conclusion and Future Work

We proposed a novel reasoning system that can learn a pattern-based if-then rule including conjunction, disjunction, and negation. The proposed system is suitable for systems operating in the real world, such as intelligent robots, because the proposed system achieves incremental learning, generalization, avoidance of duplication of results, and robustness to noise. We will develop this study in two directions. The first direction is applying results of this study to development of a robot. Using the pattern-based reasoning system, a robot is expected to learn knowledge autonomously from an environment and solve tasks by reasoning using the learned knowledge. The second direction is an extension of the proposed system for a robot to solve complicated problems. Acknowledgments. This study was supported by the Industrial Technology Research Grant Program in 2004 from the New Energy and Industrial Technology Development Organization (NEDO) of Japan.

References 1. Shen, F., Hasegawa, O.: An Incremental Network for On-line Unsupervised Classification and Topology Learning. Neural Networks 19, 90–106 (2006) 2. Sudo, A., Sato, A., Hasegawa, O.: Associative Memory for Online Incremental Learning in Noisy Environments. In: Proc. of the 2005 International Joint Conference on Neural Networks (IJCNN 2005) (accepted, 2007) 3. Yamane, K., Hasuo, T., Suemitsu, A., Morita, M.: Pattern-based reasoning using trajectory attractors. IEICE Trans. Information and System J90-D, 933–944 (2007) 4. Tsukimoto, H.: Pattern Reasoning: Logical Reasoning of Neural Networks. IEICE Trans. Information and System J83-D-II, 744–753 (2000) 5. Fritzke, B.: A growing neural gas network learns topologies. In: Proc. of Advances in Neural Information Processing Systems (NIPS), pp. 625–632 (1995)

Effect of Spatial Attention in Early Vision for the Modulation of the Perception of Border-Ownership Nobuhiko Wagatsuma, Ryohei Shimizu, and Ko Sakai Graduate School of Systems and Information Engineering, University of Tsukuba, 1-1-1 Ten-nodai, Tsukuba, Ibaraki, 305-8577, Japan [email protected], [email protected], [email protected] http://www.cvs.cs.tsukuba.ac.jp/

Abstract. We propose a computational model consisting of mutually linked V1, V2, and PP modules. The model reproduces the effect of attention in the determination of border-ownership (BO) that tells which side of the contour owns the border. The V2 module determines BO based on surrounding contrast extracted by the V1 module that could be influenced by top-down spatial attention from the PP module. We carried out the simulations of the model with random-block ambiguous figures to test whether spatial attention alters BO for these meaningless stimuli. To compare quantitatively these results with human perception, we carried out psychophysical experiments corresponding to the simulations. The results of these two showed good agreement in that the perception of BO was flipped when altering the location of spatial attention. These results suggest that spatial attention is a crucial factor for the modulation of figure direction in meaningless figures, and that the effects of spatial attention in early visual area are crucial for the modulation of figure direction. Keywords: Attention, border ownership, figure, early vision, model, psychophysics.

1 Introduction We have a function that focuses on the most important and salient object or location at the moment, which is known as visual attention. Visual attention not only boosts our perception [1] but also alters the perception of an object, or figures, which is apparent in ambiguous figures such as the Rubin’s vase [2]. We propose that attention alters the contrast gain in early vision, and the modified contrast then alters the border-ownership (BO) signals that are essential for the determination of the figure direction [3, 4]. If attention is significant, perception of figure direction is flipped because of the modulation of the activities of BO-selective neurons. As a result, perception of the object is changed. In the case of Rubin’s vase, for example, we can perceive two objects, the vase and two facing faces, depending on which side we pay attention. Visual attention has two distinct modes: spatial attention and object-based attention. Both types of attention have been shown to facilitate human perception M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 348–357, 2008. © Springer-Verlag Berlin Heidelberg 2008

Effect of Spatial Attention in Early Vision for the Modulation of the Perception of BO

349

from a number of aspects [5]. In particular, a recent study has reported that spatial attention alters contrast gain in early visual areas [6], the mechanism for which have been reported by several modeling works [e.g.7]. These models focus on interaction between the visual attention and lower visual functions such as contrast sensitivity, however they cannot explain more complex perception like a figure/ground segregation. It has been reported that majority of neurons in monkey’s V2 and V4 showed BO selectivity: their responses change depending on which side of a border owns the contour [8]. Computational works have suggested that the BO coding is determined based on the surrounding suppression/facilitation observed in early visual areas, thus luminance contrast around the classical receptive field is crucial for the determination of BO [3,4]. These models, however, don’t reproduce the perception of BO for ambiguous figures in which BO flips alternatively. These previous studies led us to propose following hypothesis: spatial attention alters contrast gain in early vision then the increased contrast modifies the activities of BO selective neurons. Based on this hypothesis, we propose a network model consisting of mutually connected V1, V2, and PP module. Top-down spatial attention from PP alters contrast gain in V1. The change in the contrast signal then modifies activities of BO selective neurons in V2 because BO is determined solely from surrounding contrast. We carried out the simulations of the model and the corresponding psychophysical experiments to investigate the effect of attention in the BO determination. Results of the simulations and the psychophysics show good agreement: the direction of figure was flipped by spatial attention in ambiguous stimuli. In addition, the activities of BO model cells are modified depending on the location of the attention when Rubin’s vase is provided. These results suggest that perception of the figure direction is altered when spatial attention functions in early visual area. A number of previous studies have reported significant effects of attention in the visual area V2 [9] and V4 [9, 10], however we focus on the effects in early visual

Fig. 1. An illustration of the model architecture. This model consists of three modules, V1, V2 and PP, with mutual connections, except for PP to V2 pathway to avoid the direct influence of attention from PP to V2.

350

N. Wagatsuma, R. Shimizu, and K. Sakai

area, V1, to investigate bottom-up attention that biases BO-selective neurons. The bottom-up attention seems to be crucial for the determination of BO, specifically for meaningless figures, because the latency of BO signal is short [8], and the switch of figure is achieved automatically [2]. Needless to say, BO-selective neurons in V2 and V4 might be affected directly by attention, however, it is not straight forward to explain how they alter the direction of BO. Here, we focus on the effects of attention in early visual area that modulate afferent BO neurons automatically and rapidly.

2 The Model In our model, spatial attention alters contrast gain in early vision, and the increased contrast modifies the activities of BO selective neurons, which may underlie the switch of figure ground. The model consists of three modules: V1, V2 and Posterior Parietal (PP) modules, as illustrated in Fig.1. Top-down and bottom-up pathways link mutually these modules, except for PP to V2. We excluded this PP to V2 connection to avoid direct influence of the attention to BO model cells. Each module consists of 100x100 model cells distributed retinotopically. In the absence of external input, the activities of a cell at time t, A(t), is given by τ

∂A(t ) = − A(t ) + μF ( A(t )) , ∂t

(1)

where the first term on the right side is a decay, and the second term shows the excitatory, recurrent signals among the excitatory model cells. Non-linear function, F(x), is given by F ( x (t )) =

1 , Tr − τ log (1 − (1 τx(t )) )

(2)

where τ is a membrane time-constant, and Tr is absolute refractory period. The dynamics of this equation as well as appropriate values for constants has been widely studied [11]. 2.1 V1 Module The V1 module models the primary visual cortex, in which local contrast is extracted from an input stimulus, and spatial attention modulates the contrast gain. The input image Input is a 124x124 pixel, gray scale image with intensity values ranging between zero and one. The local contrast, Cθω (x, y,t), is extracted by the convolution of the image with a gabor filter, Gθω , Cθω (x, y,t) = Input(x, y) ∗ Gθω (x, y) ,

(3)

where indices x and y are spatial positions, and ω represents spatial frequency. Orientation, θ , was selected from 0, π 2 , π and 3π 2 . The extracted contrast is modulated by spatial attention, thus the contrast at the attended location is enhanced. V1 The activity of a model cell in V1 module, Aθω xy , is given by

Effect of Spatial Attention in Early Vision for the Modulation of the Perception of BO

τ

V1 ∂Aθω xy (t)

∂t

V1 V1 V1−V 2 V1,E = −Aθω (t) + Iθω xy (t) + μF(Aθωxy (t)) + I xy xy (t) + I o ,

351

(4)

where I Vxy1−V 2 shows the feedback from V2 to V1, I o is random noise, and μ represents a scaling contrast. The local contrast, Cθω , is modulated by the feedback from PP to V1, I Vxy1− PP , as given by the following equation [7]: (Cθω ( x, y , t ))

V 1, E Iθω xy (t ) =

S

δ I Vxy1 − PP ( t )

⎛ 1 + ∑⎜ ⎜ θω ⎝ ( 2 I + 1)( 2 J + 1)

γ I Vxy1− PP ( t )

⎞ ∑ ∑ Cθω ( x + j, y + i, t ) ⎟⎟ j=− J i=−I ⎠ J

I

δ I Vxy1 − PP ( t )

, (5)

where S in eq(5) prevents the denominator to become zero. γ and δ are constants. In the V1 module, spatial attention influences contrast gain, therefore the contrast at the attended location is enhanced. 2.2 V2 Module

The V2 module models BO-selective cells reported in V2 that determine the direction of BO. Activities of the BO model cells is determined based on the surrounding contrast signal extracted by the V1 module, as illustrated in Fig.2[3, 4]. Each BO model cell has single excitatory and inhibitory regions. The activity of a BO model cell is modulated based on the location and shape of these surrounding regions. To reproduce a wide variety of BO selectivity, we implemented ten types of BO-left and BO-right model cells with distinct surrounding regions.

Fig. 2. A mechanism of the BO determination [3, 4]. In the case BO-right cell, contrast signal in the excitatory surrounding region enhances the activity of the cell. In contrast, if contrast exists in the inhibitory region, the activity of BO-left cell is suppressed. A dominant model cell owns this border. In this case, BO-right cell owns the border.

352

N. Wagatsuma, R. Shimizu, and K. Sakai

The activity of a BO-selective model cell is given by τ

2, BO ∂AVxyN (t )

∂t

2, BO 2, BO 2 −V 1, BO = − AVxyN (t ) + μF ( AVxyN (t )) − γF ( AV 2,inh (t )) + I VxyN (t ) + I o ,

(6)

2−V 1, BO where I VxyN represents afferent input from V1. An index BO shows left- or right-

BO selectivity, and N represents the type of BO model cells that is distinguished by their surround region. If BO-left model cells are more active than BO-right model cells, a figure is judged as located on the left side. The third term of the equation represents the input from inhibitory cells that gathers signals from all model cells in the layer. The activity of an inhibitory V2 model-cell is given by τ

∂AV 2,inh (t ) ∂t

2 , BO = − AV 2, inh (t ) + μF ( AV 2, inh (t )) + κ ∑ F ( AVxyN (t )) ,

(7)

Nxy

where κ is a constant. This inhibitory cell receives inputs from excitatory neurons in V2, and inhibits these neurons. 2.3 Posterior Parietal (PP) Module

The PP module encodes spatial location, with the aim of facilitating the processing of the attended location. The location of spatial attention is given explicitly in this module, which will boost the contrast gain of the location in V1 module. PP module receives afferent inputs from V1 and V2 modules. The activity of an excitatory model-cell in the PP module is given by τ

PP ∂A xy (t )

∂t

PP PP PP , BT PP , A = − A xy (t ) + μ F ( A xy (t )) − γF ( A PP , inh (t )) + I xy (t ) + I xy (t ) + I o ,

(8)

PP , A I xy represents the strength of attention with a Gaussian shape, which mimics topPP , BT down attention, the details of which is out of the focus of this model. I xy represents

afferent inputs from V1 and V2 modules to PP module, this process could be considered as saliency map based on luminance contrast. When there is no top-down attention, the PP module will be activated by afferent signals from V1 and V2. The third term shows input from an inhibitory PP model-cell. The activity of the inhibitory cell is determined from the activities of all excitatory PP cells as in the case of eq(7). The PP module encodes spatial location, and facilitates the processing in and around the attended location in V1. Note that spatial attention does not directly affect BO-selective model-cells in V2 module, because we focus on the effect of spatial attention in early vision V1.

3 Simulation Results We carried out the simulations of the model with a variety of stimuli, in order to test the characteristics of the model in various situations. Specifically, we investigated whether human perception of the direction of figure is reproduced in ambiguous

Effect of Spatial Attention in Early Vision for the Modulation of the Perception of BO

353

figures. First, we compare the simulation results with that of corresponding psychophysical experiments for ambiguous, random-block figures (Fig.3(a)). Second, we present an example of the simulation results for well-known ambiguous figures, specifically Rubin’s vase (Fig.3(b)). 3.1 Simulations and Psychophysics for Ambiguous Block Stimuli

First, we carried out the simulations of the model with the block objects as illustrated in Fig.3(a). These block objects are ambiguous figures; we can perceive a right- and left-hand object as figure. Fig.4 shows the simulation results for these ambiguous figures. Black and white bars represent the proportion that the black or white object is perceived as figure, respectively. We carried out the simulation of the model with three conditions: no attention, attending to the left- or right- object. By changing the location of the spatial attention, the dominant populations of BO model cells, either right or left, are switched. This result suggests that the perception of the figure direction is altered according to the attended location. To estimate whether the model reproduces the human perception, we carried out psychophysical experiments with similar settings to the simulations, as its procedure illustrated in Fig.5. We presented to human subjects the figures identical to those used in the simulations, and measured how they perceive the BO direction. Fig.6 shows the results of the psychophysical experiment. Subjects showed a tendency that the attended object is perceived as figure. This result suggests that spatial attention has influence for the determination of the figure direction in these meaningless block figures. Because this tendency was apparent similarly in the simulation results, we tested statistically whether there is a difference in the magnitude of the attention modulation between the model and psychophysics. Here, we define the magnitude of the attention modulation as below, m=

black ( attn (black )) white ( attn ( white )) + , black ( attn ( without )) white ( attn ( without ))

(9)

where m represents the magnitude of modulation, black(x) shows the proportion that the black object is perceived as figure and white(x) does the white object. attn(y) is the conditions of the attention. There was no significant difference between the modulation magnitude between the model and psychophysics (ANOVA : p=0.6798).

(a)

(b)

Fig. 3. Examples of stimuli. (a) Ambiguous, meaningless, random-block figures. Either black or white random block object may be perceived as figure. (b) the Rubin’s vase. A grey circle indicates the location and extent of the receptive field of BO model cells.

354

N. Wagatsuma, R. Shimizu, and K. Sakai

The magnitude of the attention modulation of the model agrees with that of human perception. This result suggests that covert spatial attention could be a crucial factor for the modulation of figure direction. 3.2 Simulation Results for Ambiguous Figures – A Case of Rubin’s Vase

㻾㼍㼠㼕㼛㻌㼛㼒㻌㼠㼔㼑㻌㼍㼏㼠㼕㼢㼕㼠㼕㼑㼟㻌㼛㼒㻌㻮㻻㻌㼏㼑㼘㼘㼟㻌

We carried out the simulations of the model with other ambiguous figures including variations of the Necker cube, the Rubin’s vase (Fig.3(b)), and others. As an example,

①

Fig. 4. The simulation results for ambiguous block figures. Bottom figures are the stimuli. and show attending locations. Black and white bars exhibit the activities of BO-left and BO-right model cells, respectively. A set of three bars corresponds to three attention conditions; no attention (left), attendance to the area (middle) and to the area (right). Arrows show the magnitude of the attention modulation.

②

①

②

Fig. 5. The procedure of the psychophysical experiment. The bottom panel presents the without-attention condition. The top panel shows the with-attention task. Three subjects were asked to report which object (left or right) is perceived as figure with 2AFC paradigm for 80 trials/ condition.

355

㻾㼍㼠㼕㼛㻌㼛㼒㻌㼠㼔㼑㻌㼍㼏㼠㼕㼢㼕㼠㼕㼑㼟㻌㼛㼒㻌㻮㻻㻌㼏㼑㼘㼘㼟㻌

Effect of Spatial Attention in Early Vision for the Modulation of the Perception of BO

Fig. 6. The measured human responses with the conventions same as Fig.5. Error bars indicate standard error.

Fig. 7. The simulation result for the Rubin’s vase. These bars show the proportion of BOselective neurons. “X” icons indicate the location of the attention. If BO-left neurons are more active than BO-right neurons at the location, the vase is considered as figure. The activities of BO model cells are altered when attended location is altered accordingly.

we show the results for the Rubin’s vase in this section. When we view the Rubin’s vase, we can perceive two objects alternatively, a vase and facing two faces, but we cannot perceive these two objects simultaneously [2]. If the direction of figure is the left with response to the border (indicated by a small circle in Fig.3(b)), we will perceive a vase. If the right, we will perceive a face. The simulation results for the Rubin’s vase are shown in Fig.7. The results show that BO-left model cells are dominant regardless of the location of spatial attention. However, more than 10% of the BO model cells altered its sign when the attended location is altered accordingly: when attention is applied to the center of the image

356

N. Wagatsuma, R. Shimizu, and K. Sakai

(Fig.7 (b)), the activities of BO-left cells are increased. In contrast, when attended to the right on the face (Fig.7(c)), the activities of BO-right cells are facilitated, as compared to the without-attention case (Fig.7 (a)). The early-level processing does not seem to evoke a full switch of figure in meaningful figures including the Rubin’s vase, because higher-level vision might influence the processing of human face, as described further in Discussion. The results show that the spatial attention changes the activities of BO-selective cells in the direction consistent with human perception, suggesting an importance of the modulation in early vision.

4 Discussion Our proposed hypothesis is that spatial attention alters contrast gain in early vision then the increased contrast modifies the activities of BO selective neurons, which may lead to the switch of figure/ground. We constructed the network model that consists of three modules each corresponding to V1, V2 and PP cortical area, together with the mutual connections between them including both bottom-up and top-down pathways, but excluding PP to V2. We tested the model with ambiguous, random-block figures to examine whether the model reproduces the attention modulation in human perception. The model shows good agreement with the human psychophysics in which subjects tended to perceive attended objects as figure. Our model reproduced this tendency of human perception: the activities of BO model cells are flipped according to the location of spatial attention. This result suggests that spatial attention is a crucial factor for the modulation of figure direction, and that gain control in early vision plays an important role for the modulation. Next, we tested the model with the Rubin’s vase, one of the most famous ambiguous figures. Although our model showed a modulation in the activities of BO model cells depending on the attention, the model did not exhibit the flip of BO to the face side. It has been suggested that the processing of human face is carried out by the specialized neurons in higher visual areas such as IT and TEO. We suppose that the feedback from such higher visual areas might modulate significantly BO neurons to flip the perception to the face. It should be noted that the some degree of modulation from covert spatial attention still works with this particular example, perhaps because this pathway through early vision is automatic and independent of object familiarity or meaning. Our model predicts that spatial attention alters figure direction through the change in contrast sensitivity. However, not only spatial attention but also feature-based attention or object-based attention should have considerable effect for the human perception in the determination of figure direction [12, 13]. It is expected to take into account feature-based attention to further understand the perception of figure direction. Specifically, application of the model to a number of ambiguous stimuli proposed by psychophysicists may be valued to test quantitatively the relation of feature and BO determination. Our results provide essential and testable predictions for the fundamental problems of figure/ground segregation and attention.

Effect of Spatial Attention in Early Vision for the Modulation of the Perception of BO

357

References 1. Posner, M.I.: Orientating attention. The Quarterly Journal of Experimental Psychology 32 (1980) 2. Nicholas, W.: The art and science of visual illusions (1982) 3. Carrasco, M., Ling, S., Read, S.: Attention alters appearance. Nature Neuroscience 7, 308– 313 (2004) 4. Lee, D.K., Itti, L., Koch, C., Braun, J.: Attention activates winner-take-all competition among visual filters. Nature Neuroscience 2, 375–381 (1999) 5. Sakai, K., Nishimura, H.: Surrounding suppression and facilitation in the determination of border ownership. Journal of Cognitive Neuroscience 18, 562–579 (2006) 6. Nishimura, H., Sakai, K.: The computational model for border-ownership determination consisting of surrounding suppression and facilitation in early vision. Neurocomputing 65– 66, 77–83 (2005) 7. Deco, G., Lee, T.S.: The role of early visual cortex in visual integration: a neural model of recurrent interaction. European Journal of Neuroscience 20, 1089–1100 (2004) 8. Zhou, H., Friedma, H.S., von der Heydt, R.: Coding of border ownership in monkey visual cortex. Journal of Neuroscience 20, 6594–6611 (2000) 9. Reynolds, J.H., Chelazzi, L., Desimone, R.: Competitive Mechanism Subserve Attention in Macaque Areas V2 and V4. The Journal of Neuroscience 19, 1736–1753 (1999) 10. Reynolds, J.H., Pasternak, T., Desimone, R.: Attention Increases Sensitive of V4 Neurons. Neuron 26, 703–714 (2000) 11. Gerstner., W.: Population dynamics spiking of neuron: Fast transients, asynchronous states, and locking. Neural Computation 12, 43–89 (2000) 12. Serences, J.T., Sxhwarzbach, J., Courtney, S., Golay, X., Yantis, S.: Control of Objectbased Attention in Human Cortex. Cerebral Cortex 14, 1346–1357 (2004) 13. Mitchwll, J.F., Stoner, G.R., Reynolds, J.H.: Object-based attention determines dominance in binocular rivalry. Nature 429, 410–413 (2004)

Effectiveness of Scale Free Network to the Performance Improvement of a Morphological Associative Memory without a Kernel Image Takashi Saeki and Tsutomu Miki Graduate School of Life Science and Systems Engineering, Kyushu Institute of Technology, Kitakyushu 808-0196, Japan [email protected], [email protected] http://www.lsse.kyutech.ac.jp

Abstract. In this paper, we present a new approach of the morphological associative memory (MAM) without a kernel image to reduce the network size by using the scale free network. The MAM is one of the powerful associative memories compared to ordinary associative memories. Weak point of the MAM is to need the kernel image which is susceptibility to noise and hard to design. We have already presented the MAM without a kernel image as a practical model. However the model has a drawback that the perfect recall rate is degraded. On the other hand, it has been reported that an introduction of the scale free network to associative memories is effective in the improvement of the recall rate and the reduction of the network size. We try to reduce the network size and improve the recall rate by introducing the scale free network. Keywords: Morphological associative memory, scale free, recall rate.

1 Introduction Nowadays, complex networks are studied in various fields. Complex networks are real huge networks such as internet, human relationship and joint of brain cells. Complex network exhibits small world behavior, scale free behavior, and cluster behavior in common. Watts et al. defined small world network in 1998[4]. Recently, feature of complex network has triggered lots of researchers in different field [1], [2], [3]. On the other hand, neural network model have been studied for implementation of human associative memory. An ordinary associative memory is used McCulloch-Pitts model. As unused McCulloch-Pitts model, the morphological neural networks were derived from the theory of image algebra developed by Ritter (1990) [5]. After that, several researchers have applied morphological associative memory networks for many applications [7], [8], [9], [10]. Ritter also proposed an associative memory network using the concept of morphological neural networks (MAM) (1998) [6]. The model was superior to ordinary associative memory models such as Hopfield network in terms of calculation amount, memory capacity, and perfect recall rate. Unfortunately, the model uses a kernel image so that the model has a disadvantage that it was difficult to calculate kernel image. The MAM has two type memory M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 358–364, 2008. © Springer-Verlag Berlin Heidelberg 2008

Effectiveness of Scale Free Network to the Performance Improvement of a MAM

359

matrices so that the MAM is necessary two times size of memory matrix compared with Hopfield network. The MAM without a kernel image is inferior to the noise tolerance compared with Ritter’s model and needs two times the size of memory matrices compared with ordinary associative memories such as Hopfield network. In this paper, we proposed new type MAM which has scale free behavior in order to solve these problems. The proposed model shows superior to performance compared with an ordinary fully connected MAM in terms of the noise tolerance and the size of memory matrices.

2 Scale Free Network A scale free network consists of nodes and links. The scale free network is satisfied with power of law distribution. Power of law distribution is that some of node connects many other nodes with a link so that the network has large order coefficient, however, the most of all nodes connect few nodes so that the network has small order. BA (Barabasi-Albert) model is popular as a scale free network model [2]. The complex network which has scale free behavior is designed easily by using the BA model. The BA model has a characteristic which is the growth of network, in truth, many non-growing networks are exist. Masuda et al. proposed threshold graph which is non-growing scale free model [1]. In this model, each node has a weight and connections to other nodes with the link when sum of weight of two nodes is greater than threshold value. Many scale free network designed by the threshold graph have following features; 1. Degree distribution follows the power of law. 2. However network is growing in scale, average of cluster coefficient can not decay to zero. 3. Cluster coefficient follows the power of law. These features are effect to reduce the size of memory matrix.

3 Morphological Associative Memory: MAM The MAM has two-stage recall process as illustrated in Fig.1. If it regards two-stage recall process as one step, the MAM is an associative memory as same as other associative memories. In Fig.1, ‘X’ and ‘Y’ is a pair of the sth stored patterns. ‘Y’ is ~ an output pattern for ‘X’ or ‘ X ’ which is an input pattern.

~ X

~ Y 1st Pattern Recall

Y 2nd Pattern Recall

Fig. 1. Recall process of MAM

360

T. Saeki and T. Miki

Normal associative memories use the learning in their memory process. On the other hand, the MAM uses calculations employing two different types of memory matrices in order to store the patterns. Therefore memory process of the MAM is faster than ordinary associative memories. In this paper, the first stage is called “the first recall” and the later is called “the second recall”. The recall pattern is given by:

{

n

(

)

(

)

r ~y r = j ∨ w ji + ~x j i =1 m

yir = ∧ mij + ~ y jr j =1

(1)

or (2)

{

n

(

)

(

)

r ~y r = j ∧ m ji + ~x j i =1 m

yir = ∨ wij + ~ y rj , j =1

(3)

(4)

where a pair of Eq.(1) and Eq.(2) is equal to that of Eq.(3) and Eq.(4). n and m represent the number of total units of an input pattern and an output pattern, respectively.

~ x jr is jth unit of the corrupted pattern x r and ~ y jr is an output of the first r

recall (It is also an input of the second recall). yi represents ith unit of the output pattern y . r is the number of stored patterns. ∨ , ∧ are maximum, minimum operator, respectively. Here memory matrices wij and mij can be calculated from the following equations (5), (6). r

(

)

1 j

= y −x

) ∧ (y

(

)

= y −x

) ∨ (y

S

wij = ∧ yir − x rj

(

r =1

1 i

S

2 i

mij = ∨ yir − x rj

(

r =1

1 i

1 j

2 i

)

(

s i

)

(

s i

−x ∧ " ∧ y −x 2 j

s j

−x ∨ " ∨ y −x 2 j

s j

) )

(5)

(6)

4 Scale Free Network Type MAM Masuda et al. proposed a new design method of scale free network by using a threshold graph. The scale free network employing a threshold graph is designed based on fully connected MAM. Geometry of scale free network used threshold graph is shown Fig.2. Fig.2 is recited from Reference [1]. Threshold graph processed as follows; Step 1. n numbers nodes are given. Step 2. A weight wi is assigned to each node. The weight wi is selected randomly from probabilistic distributed function f(w).Probability distribution function is given by;

f ( x) = e − x

(7)

Effectiveness of Scale Free Network to the Performance Improvement of a MAM

361

Step 3. For node i and j , if it fill the demand of Eq.(8), node i connects j. Step 3 is carried out for all nodes. wi+wj ≥ θ

(8)

where threshold value θ is real value. Here, in this paper, MAM needs a connection to itself for the perfect recall. Connection to itself is held certainly.

Fig. 2. Geometry of scale free network [1]

5 Experimental Results In this paper, we investigate the perfect recall rate of the MAM which employs a scale free network designed by using the threshold graph with a suitable threshold value. Fig.3 shows the stored patterns. Each pattern consists of 10 × 10=100 binary units. The unit takes ‘1’ or ‘0’. The ‘1’ represents black and ‘0’ white. The one of the stored patterns with a random noise is used as the input pattern. Here, the noise is defined as to change ‘1’ to ‘0’ or ‘1’ or ‘0’, and is not over 10% of total units of a pattern.

Fig. 3. Stored Patterns

362

T. Saeki and T. Miki

10000 trials are performed in the simulations. The performance of the noise tolerance is estimated for an average of the perfect recall rate and the size of memory matrices is estimated for the average of the size of the memory matrices. Here, the perfect recall is defined as to recall the stored pattern which corresponds to noise less input one. The size of memory matrices is defined as a total size of M and W matrices. 5.1 Decision of Insert Position of a Scale Free Network The MAM has a two-stage recall process. Introducing a scale free network can be inserted to three locations (first stage, second stage, and both stages). Effective insert position of the scale free network is determined by investigating their noise tolerance. Fig.4 shows noise tolerance of the MAM introducing the scale free network in the first stage, the second stage, and both stages. Here, threshold value is ‘1’. 100

Normal

] %[ 80 te a R lla60 c e R40 t c ef r20 e P

First Second Both

0 0

2

4

6

8

10

Noise Rate[%] Fig. 4. Difference of Noise Tolerance of the MAM with Different Inserting Position of a Scale Free Network: “Normal “means a noise tolerance of fully connected MAM, “First” means introducing the scale free network in the first stage, “Second” in the second stage, and “Both” in both stages. Table 1. Noise Tolerance and Size of Memory Matrices

Normal Average of Recall Rate[%] * Noise Tolerance Size of Memory Matrices

30.7

First Stage 34.9

Second Stage 34.5

Both Stage 34.8

―

4%↑

4%↑

4%↑

20000

10291

10291

578

* As the base, Noise tolerance is defined in case of under 10% one.

Effectiveness of Scale Free Network to the Performance Improvement of a MAM

363

Next, the effect on the size of the memory matrices is investigated. Table 1. shows improvement of noise tolerance and size reduction of the memory matrices of the MAM with different inserting position of the scale free network. From results, there is a little difference in improvement on noise tolerance. But noise tolerance is superior to the normal MAM. The ordinary MAM has two fully connected networks. However, the MAM introducing scale free network in the both stages has two low connected networks. Size of memory matrices is drastically reduced by introducing a scale free network in both stages. 5.2 Evaluation of MAM with Scale Free Network in Both Stages A construction of the scale free network is controlled by threshold value. Therefore we examine the effect that a difference of a threshold value gives improvements for the MAM employing the scale free network in both stages. Table 2. shows the noise tolerance and the size of the memory matrices of the MAM which is introduced the scale free network in both stages for different threshold values. Table 2. Noise Tolerance and Size of Memory Matrices for different threshold values

Threshold θ Average of Recall Rate[%] Size of Memory Matrices

2 34.2 202

1 34.8 578

0.1 33.7 1334

0.01 36.5 2128

* As the base, Noise tolerance is defined in case of under 10% one.

As table 2. shows, noise tolerance of each threshold value is not difference. As a result, threshold value has no effect on the noise tolerance.

6 Conclusion In this paper, we proposed a new MAM model using scale free network. The performance of the proposed model was confirmed by the autoassociation experiments for alphabet patterns. Inserting position and threshold value are determined by simulations. As the results, in the proposed model the perfect recall rate is improved about 4% and the size of the memory matrices is drastically reduced 97% with compared to the ordinary fully connected MAM. To design distribution of weight for improvement of noise tolerance and to use more experimental evidence in the various conditions are future works. Acknowledgments. This work was supported in part by a 21st Century Center of Excellence Program,"World of Brain Computing Interwoven out of Animals and Robots (PI:T.Yamakawa)" granted in 2003 to Department of Brain Science and Engineering, (Graduate School of Life Science and Systems Engineering), Kyushu Institute of Technology by Japan Ministry of Education, Culture, Sports, Science and Technology.

364

T. Saeki and T. Miki

References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10.

Masuda, N., Miwa, H., Konno, N.: Phys.Rev. E 70(036124) (2004) Barabási, A.-L., Albert, R.: Science 286, 509 (1999) Albert, R., Barab´asi, A.-L.: Rev. Mod. Phys. 74, 47 (2002) Watts, D., Strogatz, S.: Nature 393 (1998) Ritter, G.X., Wilson, J.N., Davidson, J.L.: Comput, Vision Graphics Image Processing 49(3) (1990) Ritter, G.X., Sussner, P., Diaz-de-Leon, J.L.: IEEE Trans.Neural Networks 9(2) (1998) Won, Y., Gader, P.D.: Proceedings of the 1995 IEEE International Conference on Neural Networks, Perth, Australia, November 1995, vol. 4 (1995) Won, Y., Gader, P.D., Coffield, P.: IEEE Trans. Neural Networks 8(5) (1997) Davidson, J.L.: Image Algebra and Morphological Image Processing III. In: Proceedings of SPIE, San Diego, CA, July 1992, vol. 1769 (1992) Davidson, J.L., Hummer, F.: IEEE Systems Signal Processing 12(2) (1993)

Intensity Gradient Self-organizing Map for Cerebral Cortex Reconstruction Cheng-Hung Chuang1, Jiun-Wei Liou2,3,*, Philip E. Cheng2, Michelle Liou2, and Cheng-Yuan Liou3 1

Dept. of Computer Science and Information Eng., Asia University, Taichung, Taiwan 2 Institute of Statistical Science, Academia Sinica, Taipei, Taiwan 3 Dept. of Computer Science and Information Eng., National Taiwan Univ., Taipei, Taiwan [email protected]

Abstract. This paper presents an application of a self-organizing map (SOM) model based on the image intensity gradient for the reconstruction of cerebral cortex from MR images. The cerebral cortex reconstruction is important for many brain science or medicine related researches. However, it is difficult to extract deep cortical folds. In our method, we apply the SOM model based on the image intensity gradient to deform the easily extracted white matter surface and extract the cortical surface. The intensity gradient vectors are calculated according to the intensities of image data. Thus the proper cortical surface can be extracted from the image information itself but not artificial features. The simulations on T1-weighted MR images show that the proposed method is robust to reconstruct the cerebral cortex. Keywords: Self-organizing map, brain science, cortical surface reconstruction, deformable surface models, active surface models, gradient vector field (GVF).

1 Introduction Recently, due to advanced magnetic resonance imaging (MRI) techniques, MR images, which reveal high spatial resolution and soft-tissue contrast, have great potential to be used in the analysis of cognitive neuroscience, diseases (e.g., epilepsy, schizophrenia, Alzheimer's disease, etc.), and researches into anatomical structures of human brains in vivo. Modern anatomical MRI studies on human brains have been concentrated on the cerebral cortex, which is a thin and folded layer of gray matter on the cerebral surface and contains dense neurons to control high cortical functions like language and information processing. Many studies have shown that the cortical thickness decreases or changes in association with neurodegenerative diseases and psychiatric disorders, e.g., schizophrenia, Alzheimer’s disease [1], and autism [2]. The cerebral cortex reconstruction nowadays is an on-going research field that can help other researches like brain mapping to explore human brain. Generally speaking, there are three major brain tissues which can be approximately partitioned and segmented in human brains, i.e., gray matter (GM), white matter (WM), * Corresponding author. M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 365–373, 2008. © Springer-Verlag Berlin Heidelberg 2008

366

C.-H. Chuang et al.

and cerebral spinal fluid (CSF). The cerebral cortex is the thin and folded layer between GM/WM and GM/CSF interfaces [3, 4]. The cortex reconstruction means to extract GM/WM and GM/CSF boundaries and rebuild its surface. A lot of methods in the literature have been proposed to solve this problem [1–5]. These methods can be roughly categorized into stochastic and morphological types. The methods using a stochastic model [1, 4] employ labeled cortical mantle distance maps or intensity distance histograms related to the GM/WM interface so that the extraction of GM/CSF interface is needless. On the other hand, the morphological methods [2, 3, 5] apply the dilation of GM/WM interface to extract the accurate GM/CSF boundary surface since the obvious GM/WM interface is easily extracted. In the latter, a cortex is usually regarded as a double surface structure [3]. The exterior surface following the interior surface is deformed with some constraints to find out the GM/CSF interface. The deformable surface model is a powerful method to reconstruct the cortical surface. However, it is difficult to deform the surface with correct topology inside the highly folded and buried cortex with image noise. In [5], a deformable surface model based on the gradient vector flow (GVF) [6] is proposed to overcome this problem. The model using an energy minimizing function, a weighted combination of internal and external energy, is basically a three-dimensional (3-D) snake model. The internal energy controls the continuity force of surface itself, while the external energy governs the attraction force like image gradients that leads the surface. Nevertheless, the computational cost of minimizing the energy function becomes larger as number of points on the surface increases. In [7], the self-organizing map (SOM) model [810] plus a layered distance map (LDM) is applied to deform the GM/WM interface to find out the GM/CSF interface in segmented MR images. The layered distance map is calculated according to the extracted WM surface and segmented GM. Unfortunately, the distance values inside the sulci are usually symmetric that does not match the real cortical thickness. In this paper, we propose an advanced method in which the SOM model based on the image intensity gradient is applied to deform the GM/WM interface to find out the cortical surface in segmented MR images. In the simulation and experiment, a twodimensional (2-D) T1-weighted MR image and 3-D T1-weighted MRI data are used for test. In contrast with the results of previous method [7], our newly studies on T1weighted MRI data show that the proposed method gains more precise results to reconstruct cerebral cortex.

2 The Problem In the cerebral cortex, there are many narrow and deep fissures called sulci. These concave parts sometimes contain invisible or unrecognizable CSF, which makes the reconstruction of cerebral cortex laborious. To easily understand the problem, a T1-weighted MR image shown in Fig. 1(a) is illustrated. This image shows a sulcus structure, i.e. the region of interest (ROI). Its segmented image is shown in Fig. 1(b), where the white color region represents WM, the gray color region indicates GM, and the black color region is CSF, other tissues, and background. It is easy to extract the boundaries of the segmented GM, as shown in Fig. 1(c). Now, the ROI image is enlarged and shown in Fig. 1(d). The ROI image is equalized to clearly display the

Intensity Gradient Self-organizing Map for Cerebral Cortex Reconstruction

367

ideal boundary, as shown in Fig. 1(e). The extracted and ideal boundaries between Fig. 1(c) and Fig. 1(f) are obvious different and there should exist an interval within the sulcus. If the inner (WM) boundary line is deformed outward to extract the outer (GM) boundary according to the segmentation results, it will probably fail to catch the interval due to fewer extractable features inside the sulcus.

(a)

(d)

(b)

(e)

(c)

(f)

Fig. 1. Illustration of a missing interval within a sulcus in a T1-weighted MR image: (a) the raw image and the ROI, (b) its segmented image and the ROI, (c) extracted boundary of ROI, (d) magnified ROI, (e) equalized ROI image with ideal boundary, (f) ideal boundary of ROI.

3 Methods Since it is difficult to partition tissues inside sulci, the ideal GM surface is also hard to be extracted. One popular way is to dilate the GM/WM interface to extract the GM/CSF boundaries because the GM/WM boundaries are obvious and can be easily extracted. In our former method, this kind of dilation strategy is assisted by a layered distance map (LDM) [7]. The LDM is formed by calculating the connected voxels layer by layer from inner to outer surfaces within the segmented GM. Therefore, if the sulci are full of GM, distance values of LDM inside the sulci are symmetric and the detected interval is always in the center. To overcome this problem, the image intensity is employed in the new method. In a T1-weighted MR image, the intensity of WM, GM, and CSF is high, middle, and low, respectively. The GVF [6] is applied to obtain the intensity gradient flow between these three tissues. However, the intensity inhomogeneity should be removed prior to GVF processing. Fortunately,

368

C.-H. Chuang et al.

Su et al. [11] proposed a wavelet-based bias correction method which is also independent of tissue classes and computationally simple. 3.1 The Image Intensity Gradient In Fig. 2, the principle of boundary extraction inside the sulci by using GVF is demonstrated. The figures show two types of sulcus profile, where curves represent the dynamic intensity, i.e., the horizontal axis means positions and the vertical one indicates intensity values. GM, WM, and CSF are segmented and arrows denote gradient vectors. In Fig. 2(b), an interval should exist but it cannot be detected by the segmentation. The cortical thickness a is not equal to b, i.e. the structure is asymmetric and the LDM cannot find the ideal boundary. However, from the direction of gradient vectors, the most possible boundary, the long dash line in Fig. 2(b), can be easily extracted.

(a)

(b)

Fig. 2. Illustration of boundary extraction inside the sulci by using GVF: (a) the sulcus profile with a clear interval, (b) the sulcus profile with an obscure interval

The intensity GVF is computed from the bias-corrected MR images defined as I(x, y, z). The gradient of 3-D negative intensity map is defined first. Then the intensity GVF field is defined as the vector field V(x, y, z)=[u(x, y, z), v(x, y, z), w(x, y, z)] that minimizes the energy functional [6]

ε = ∫∫∫ μ ∇V + ∇f 2

2

2

V − ∇f dxdydz ,

(1)

where f represents the 3-D negative intensity map, i.e. f =–I, μ is a weighting parameter, and is the gradient operator, i.e.

▽

∇ = ( ∂∂x , ∂∂y , ∂∂z ) .

(2)

In Eq. (1), the first term controls the smoothing function while the second term governs the maintaining capability. The formulation of computing V by iteratively minimizing ε is given in [6]. The final iterative solution is

⎧ u n +1 − u n = μ∇ 2 u n − (u n − f x )( f x2 + f y2 + f z2 ) ⎪ n +1 n 2 n n 2 2 2 ⎨ v − v = μ∇ v − (v − f y )( f x + f y + f z ) , ⎪ wn +1 − wn = μ∇ 2 wn − ( wn − f )( f 2 + f 2 + f 2 ) z x y z ⎩

(3)

Intensity Gradient Self-organizing Map for Cerebral Cortex Reconstruction

▽

369

2

where is the Laplacian operator, n is the iteration number, and fx, fy, and fz are partial derivatives with respect to x, y, z, respectively. The initial conditions are set with

V 0 = (u 0 , v 0 , w0 ) = ∇f = ( f x , f y , f z )

(4)

Basically, the main goal of this iterative process is to diffuse gradient properties to reduce image noise and form the vector flow all over the volume. 3.2 The Intensity Gradient Self-organizing Map (IGSOM) Model

The SOM model, which is a nonlinear, ordered, and smooth function, is an effective algorithm for the mapping between the neuron model and input data sets. In our applications, it is desired to deform the GM/WM interface to find out the GM/CSF interface by the SOM model. The boundary voxels of GM/WM are defined as the neuron data set. The SOM model driven by intensity gradient, called IGSOM, is applied to move the neuron data within the segmented GM. Finally, the GM/CSF interface is reconstructed by the converged neuron data set. The proposed IGSOM model is described in the following. First, it is needed to initialize deformable mesh network M. Here we use the isosurface of GM/WM interface. The mesh network M is a deformable surface inside the segmented GM. The segmented GM is defined as a volume set G. The best matching function of SOM is modified and rewritten as a point function P(.), i.e. P(m j ) = m j + V (m j ), m j ∈ G and m j ∈ M ,

(5)

where mj is the randomly selected mesh network node, j is the index of mesh node, and V(mj) is the GVF on the node mj. The update function of SOM denotes m j ′ (k + 1) = m j ′ (k ) + α (k ) H ( D, k )[ P (m j ) − m j ′ (k )], where

(6)

m j′ including m j represents all neighbors around m j , k = 0, 1, 2…is the

iteration number, α ( k ) ∈ [0,1) is the learning rate, and H is the neighborhood function which decreases when the distance metric D and iteration k increase. The update function is iteratively proceeded until the average variation of input data is less than a threshold value. The Gaussian function is usually applied to be the smoothing kernel, i.e.

⎛ D ⎞ H ( D, k ) = exp ⎜ − 2 ⎟ ⎝ 2σ (k ) ⎠

(7)

where σ (k ) is the standard deviation, i.e. the width of the smoothing kernel. The distance metric D ≡ D( j ′, j ) defines the distance from m j ′ along the surface to m j . When j ′ is equal to j, i.e. m j ′ = m j , D is zero and the neighborhood function H has the maximal value 1.

370

C.-H. Chuang et al.

4 Simulation and Experiment The first experiment is a 2-D 181x217-pixel T1-weighted MR image with its expert segmentation result, as shown in Figs. 3(a) and 3(b). In our method, the parameter μ is set to 0.2 and the iteration number n is only set to 2. This is because the MR image is clear and less iteration can maintain the original gradient information. The update iteration in the IGSOM model is about 100 in this case. The learning rate and the standard deviation of the Gaussian function are fixed and set to 0.1 and 0.6, respectively. From the expert partition result, boundary pixels of GM/WM (inner) and GM/CSF (outer) interfaces can be extracted as shown in Figs. 3(c) and 3(d), respectively. In Fig. 3(d), there are many sulci lose their intervals. Figs. 3(e) and 3(f) show the experimental results by using SOM+LDM model [7] and IGSOM model, respectively. Roughly, the results are similar to each other. However, some intervals inside the sulci are over-detected in the result of earlier method. Some of them are pointed out with arrow symbols in Fig. 3(e). These defects will cause extra and deeper sulci spread on cortex.

(a)

(d)

(b)

(e)

(c)

(f)

Fig. 3. Experiments on a 2-D 181x217-pixel T1-weighted MR image: (a) the raw image, (b) the expert segmentation image, (c) the inner boundary image, (d) the outer boundary image, (e) the result by the SOM+LDM model, (f) the result by the IGSOM model

Intensity Gradient Self-organizing Map for Cerebral Cortex Reconstruction

371

Then the experiment is extended to 3-D 181x217x181-voxel T1-weighted MRI data. The cerebral cortex is separately reconstructed using SOM+LDM model [7] and IGSOM model as well. The GVF parameter μ and the iteration number n are the same as the former experiment. The update iteration in the IGSOM model is about 200 in this experiment. The learning rate and the standard deviation of the Gaussian function are also fixed and set to 0.1 and 1.0, respectively. The number of mesh network nodes is 35,851. The six views, i.e. top, bottom, left, right, front, and back, of reconstructed 3-D cortical surface are shown in Fig. 4, where figures on the left column show the result of SOM+LDM model while those on the right column display that of IGSOM model. The over-detection of sulci is revealed in the result of SOM+LDM model, i.e. the left column of Fig. 4. The measurement of error is performed by calculating the average nearest distance between mesh vertices and points of segmented GM outer surface. The maximal, minimal, and mean distances are 2.35, 0.0013, and 0.42 mm in the result of SOM+LDM model, while they are 2.32, 0.0011, and 0.42 mm in that of IGSOM model. Although both the mean distances are about the same, the maximal and minimal distances in the result of SOM+LDM model are larger than those in that of IGSOM model.

(a)

(b)

(c)

(d)

Fig. 4. Experiments of cortex reconstruction with 3-D T1-weighted MRI data: (a)(c)(e)(g)(i)(k) results by the SOM+LDM model, (b)(d)(f)(h)(j)(l) results by the IGSOM model, (a)(b) top view, (c)(d) bottom view, (e)(f) left view, (g)(h) right view, (i)(j) front view, (k)(l) back view

372

C.-H. Chuang et al.

(e)

(f)

(g)

(h)

(i)

(j)

(k)

(l) Fig. 4. (continued)

5 Conclusions In this paper, the SOM model based on image intensity gradient is applied to reconstruct the human cerebral cortex from MRI data. The method provides a good

Intensity Gradient Self-organizing Map for Cerebral Cortex Reconstruction

373

capability to deform the GM/WM surface outward to find out the GM/CSF surface. Thus the asymmetric intervals inside sulci can be automatically detected. The only requirement is the intensity bias of the MRI data should be corrected previously. The gradient vectors computed from the image intensity are used to help the SOM model to deform the GM/WM surface to a proper position of GM/CSF surface. Based on this method, even the 3-D highly folded GM/CSF surface can be reconstructed. However, in the 3-D case, one of the disadvantages is the heavy computational cost if the image size is huge or the number of mesh vertices is large. Therefore, to improve the computational efficiency is the future work. Also, from the reconstructed cerebral cortex, how to systematically evaluate the accuracy of cortical surface is the urgent study. Acknowledgments. The 3-D MRI data presented in this paper come from the McConnell Brain Imaging Centre at McGill University.

References 1. Miller, M.I., Hosakere, M., Barker, A.R., Priebe, C.E., Lee, N., Ratnanather, J.T., Wang, L., Gado, M., Morris, J.C., Csernansky, J.G.: Labeled Cortical Mantle Distance Maps of the Cingulate Quantify Differences between Dementia of the Alzheimer Type and Healthy Aging. Proc. Natl. Acad. Sci. 100, 15172–15177 (2003) 2. Chung, M.K., Robbins, S.M., Dalton, K.M., Davidson, R.J., Alexander, A.L., Evans, A.C.: Cortical Thickness Analysis in Autism with Heat Kernel Smoothing. Neuroimage 25, 1256–1265 (2005) 3. MacDonald, D., Kabani, N., Avis, D., Evans, A.C.: Automated 3-D Extraction of Inner and Outer Surfaces of Cerebral Cortex from MRI. NeuroImage 12, 340–356 (2000) 4. Barta, P., Miller, M.I., Qiu, A.: A Stochastic Model for Studying the Laminar Structure of Cortex from MRI. IEEE Trans. Med. Imaging 24, 728–742 (2005) 5. Xu, C., Pham, D.L., Rettmann, M.E., Yu, D.N., Prince, J.L.: Reconstruction of the Human Cerebral Cortex from Magnetic Resonance Images. IEEE Trans. Med. Imaging 18, 467– 480 (1999) 6. Xu, C., Prince, J.L.: Snakes, Shapes, and Gradient Vector Flow. IEEE Trans. Image processing 7, 359–369 (1998) 7. Chuang, C.H., Cheng, P.E., Liou, M., Liou, C.Y., Kuo, Y.T.: Application of SelfOrganizing Map (SOM) for Cerebral Cortex Reconstruction. International Journal of Computational Intelligence Research 3, 26–30 (2007) 8. Kohonen, T.: Self-Organizing Maps, 3rd edn. Springer, Berlin (2001) 9. Liou, C.Y., Tai, W.P.: Conformal Self-Organization for Continuity on a Feature Map. Neural Networks 12, 893–905 (1999) 10. Liou, C.Y., Tai, W.P.: Conformality in the Self-Organization Network. Artificial Intelligence 116, 265–286 (2000) 11. Su, H.-R., Liou, M., Cheng, P.E., Aston, J.A.D., Chuang, C.H.: MR Image Segmentation Using Wavelet Analysis Techniques. Neuroimage 26(1) (2005), Human Brain Mapping 2005 Abstract

Feature Subset Selection Using Constructive Neural Nets with Minimal Computation by Measuring Contribution Md. Monirul Kabir1, Md. Shahjahan3, and Kazuyuki Murase1,2 1

Department of Human and Artificial Intelligence Systems, Graduate School of Engineering [email protected] 2 Research and Education Program for Life Science University of Fukui, Bunkyo 3-9-1, Fukui 910-8507, Japan [email protected] 3 Department of Electrical and Electronic Engineering Khulna University of Engineering and Technology, Khulna 9203, Bangladesh [email protected]

Abstract. In this paper we propose a new approach to select feature subset based on contribution of input attributes in a three-layered feedforward neural network (NN). Three techniques: constructive, contribution, and backward elimination are integrated together in this method. Initially, to determine the minimal NN architecture, the number of hidden neurons is determined by a constructive approach. After that, one-by-one removal of input attributes is performed to compute their contribution. Finally, a sequential backward elimination is used to generate relevant feature subset from the original input space. The elimination process is continued depending on a criterion. To evaluate the proposed method, we applied it to four real-world benchmark problems. Experimental results confirmed that, the proposed method significantly reduces the irrelevant features without degrading the network performance and generates the feature subset with good generalization ability. Keywords: Feature subset selection, neural network, classification, contribution.

1 Introduction Feature subset selection (FSS) among the huge number of dataset is a crucial task for the success of neural network (NN). The selection of effective model and then obtaining the more descriptive data is essential to achieve good performance in classification with NN. As wrapper model always outperforms filter model [1][2], a numerous attempts have been done to solve the FSS problem in wrapper approach based on different criteria such as [3]-[9]. An extensive discussion regarding wrapper model has been introduced in Kohavi’s method [3]. This approach is capable of producing a minimal set of input variables, but the cost grows exponentially in the face of many irrelevant variables. A better solution has also been reported for FSS by Setiono [4] using NN M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 374–384, 2008. © Springer-Verlag Berlin Heidelberg 2008

Feature Subset Selection Using Constructive Neural Nets

375

but it failed to reduce the computational cost due to using a large fixed number of hidden neuron (HN). The concept of contribution is implemented for FSS by Linda [5] and S. Guan [6]. Linda uses a procedure to compute the contribution of each attribute (feature) using weights. In contrast, S. Guan considers separate training of each input attribute using a separate constructive NN model to do so, though it failed to reduce the computational cost. Two other solutions for FSS can be found in [7][8]. According to the above discussions, none of them except [5] and [6], has emphasized for FSS by measuring contribution either in backward search model or constructive approach. Since backward search always shows better performance over forward search [2], in this paper, we introduce a method called contribution based feature subset selection method (CFSS) that is comprised by three techniques: constructive, contribution, and backward elimination. It is known that the constructive approach can achieve both the compact architecture and better generalization [10]. We therefore initially use a constructive algorithm, starting with a minimal network and then adding new hidden layer neurons during training. After that, in order to find out the contribution of input individuals, removal of one-by-one input attributes is implemented. In the latter part, heuristic search namely sequential backward elimination (BE) is used by utilizing four new criteria: a) double elimination, b) backtracking, c) single elimination, and d) subset validation. Subset generation process is quickened and accomplished accurately by incorporating the above four steps in BE. In order to evaluate CFSS, we tested it on four real world benchmark problems such as breast cancer, diabetes, glass, and iris problems. The experimental result exhibited that CFSS works well in the real world classification problems and makes a compact NN architecture. The remainder of this paper is organized as follows. Section 2 describes CFSS in details. Section 3 presents the results of experimental analysis. A short discussion and conclusions are given in Section 4 and Section 5 respectively.

2 The Proposed Method In feature selection, the wrapper approaches always suffer from two major problems: high computational cost [1], and instability due to changing weights randomly [9]. In order to remedy mainly the first one, we emphasize on the compact network architecture. As we know, if the size of the network becomes larger, then expensive computation takes place during the process and the performance of the network is degraded. Therefore, we implemented a constructive algorithm in CFSS and the combination of thrice, constructive, contribution, and backward elimination (BE), exhibits the completeness of CFSS efficiently. Before describing the actual algorithm, we define some parameters used in the algorithm such as stopping criteria, measurement of contribution, and so on. 2.1 Stopping Criteria (SC) We used in this method a NN that is trained by back propagation learning (BPL) algorithm [11]. New criteria are adopted for stopping the training process early in order to improve the generalization ability and to reduce the training cost. Three

376

M.M. Kabir, M. Shahjahan, and K. Murase

stopping criteria used in CFSS are summarized in this section. The details of training process and SC are available in [12]. 2.1.1 Stopping Criterion for Adding Hidden Neurons (HNs) During the course of adding HN, the SC is adopted as Emin > E p −min . That is, when the last HN is added, the minimum training error Emin should be larger than the previously achieved minimum training error Ep-min. 2.1.2 Stopping Criterion for Backward Elimination (BE) 2.1.2.1 Calculation of Error Reduction Rate (ERR) During BE process, we need to calculate the ERR as follows,

ERR = − Here,

′ − Emin Emin Emin

′ is the minimum training error during backward elimination. Emin

2.1.2.2 Stopping Criterion in double feature deletion During the course of training, the irrelevant features are sequentially deleted double at a time in BE: the SC is adopted as ERR <= th AND CA′ => CA . Here, th refers to threshold value equals to 0.05, CA′ and CA refer to the classification accuracies during BE and before BE, respectively.

2.1.2.3 Stopping Criterion in single feature deletion During the course of training, the irrelevant features are sequentially deleted single at a time in BE: the SC is adopted as ERR <= th AND CA′ => CA . Here, th refers to threshold value equals to 0.08. 2.2 Measurement of Contribution Finding the contribution of input attributes accurately ensures the effectiveness of CFSS. Therefore, during the one-by-one removal training, we measure the minimum i

training error, Emin for removing each ith feature. Then, calculate the contribution of each feature as follows. i i Training error difference for removing the ith features is, Ediff = Emin − Emin where Emin is the minimum training error using all features. Now, we need to calculate the average training error difference for removing ith feature Eavg = 1

N

∑E N i =1

i diff

where

N is the number of features. Now, the percentage contribution of ith feature is,

Coni = 100.

i Eavg − Ediff

Eavg

Now, we can decide the rank of features according to Conmax = max(Coni ) . The worst ranking features are treated as the irrelevant features for the network as these are providing comparatively less contribution for the network.

Feature Subset Selection Using Constructive Neural Nets

377

2.3 Calculation of Network Connection The number of connections (C) of the final architecture of the network can be calculated in terms of the number of existing input attributes (x), number of existing hidden units (h), and number of outputs (o) by C = ( x × h) + ( h × o) + h + o . 2.3.1 Reduction of Network Connection (RNC) In this context we here estimate how much connection has been reduced due to feature selection. In this regard, we initially calculate the total number of connections of the achieved network before and after BE, Cbefore and Cafter , respectively. After that, we estimate RNC as, RNC = 100.

Cbefore − C after

.

Cbefore

2.3.2 Increment of Network Accuracy (INA) Due to the reduction of network connection, we estimate INA that shows how much network accuracy is improved. We measure the classification accuracy before and after BE, CAbefore and CAafter , respectively. After that, we estimate INA as, INA = 100.

CAafter − CAbefore Cbefore

.

2.4 The Algorithm

In this framework, CFSS comprises into three main steps that are summarized in Fig. 1: (a) architecture determination of NN in constructive fashion (step 1-2), (b) measurement of contribution of each feature (step 3), and (c) subset generation (step 4-8). These steps are dependent each other and the entire process has been accomplished one after another depending on particular criteria. The detail of each step is as follows. Step 1) Create a minimal NN architecture. Initially it has three layers, i.e. an input layer, an output layer, and a hidden layer with only one neuron. Number of input and output neurons is equal to the number of inputs and outputs of the problem. Randomly initialize connection weights between input layer to hidden layer and hidden layer to output layer within a range between [+1.0, -1.0]. Step 2) Train the network by using BPL algorithm and try to achieve a minimum training error, Emin. Then, add a HN, retrain the network from the beginning, and check the SC according to subsection 2.1.1. When it is satisfied, then validate the NN to test set to calculate the classification accuracy, CA and go to step 3. Otherwise, continue the HN selection process. Step 3) Now to find out the contribution of features, we perform one-by-one removal training of the network. Delete successively the ith feature and save the i individual minimum training error, E min . Then, the rank of all features can be reflected in accordance with the subsection 2.2. Then go to Step 4.

378

M.M. Kabir, M. Shahjahan, and K. Murase

1

Create an NN with minimal architecture Training

2

SC satisfied ?

yes

Validate NN, calculate accuracy

no Add HN

One-by-one training, compute contribution

3

Delete double/single feature Training 4

Validate subset, calculate accuracy

SC satisfied ?

no

yes 5

6

Backtracking no

Single deletion end ? yes

7

Further check

8

Final subset

Fig. 1. Overall Flowchart of CFSS. Here, NN, HN and SC refer to neural network, hidden neuron and stopping criterion respectively.

Step 4) This stage is the first step for BE to generate the feature subset. Initially, we attempt to delete double features of worst rank at a time to accelerate the process. Calculate the minimum training error, E′min during training. Validate the existing features using test set and calculate classification accuracy CA′ . After that, if SC mentioned in subsection 2.1.2.2 is satisfied, then continue. Otherwise, go to step 5. Step 5) We perform backtracking. That is, the lastly deleted double/single of features is restored here with associated all components. Step 6) If single deletion process has not been finished then attempt to delete feature single at a time using step 4 to filter the unnecessary ones, otherwise go to step 7. If SC mentioned in subsection 2.1.2.3 is satisfied, then continue. Otherwise, go to step 5 and then step 7.

Feature Subset Selection Using Constructive Neural Nets

379

Step 7) Before going to select the final subset, we again check through the existing features whether any irrelevant feature presents or not. The following steps are taken to accomplish this task. i) Delete the existing features from the network single in each step in accordance with worst rank, and then again retrain. ii) Validate the existing features by using test set. For the next stage, save the classification accuracy CA′′ , the responsible deleted feature, and all its components. iii) If DF<(TF-1) then continue the deletion process. Otherwise, go to the next step. Here, DF and TF refer to the number of deleted feature and total number of feature, respectively. iv) Compare the values of CA′′ according to CA′′ => CA . v) If better CA′′ are available, then identify the higher rank of feature among them. Otherwise, stop. Delete the higher ranked feature with corresponding worst ranked ones from the subset that obtained at step 7-i. After that, recall the components to the current network that was obtained at step 7-ii and stop. Step 8) Finally, we achieve the relevant feature subset with compact network.

3 Experimental Analysis This section evaluates CFSS’s performance on four well-known benchmark problems such as Breast cancer (BCR), Diabetes (DBT), Glass (GLS), and Iris (IRS) problems which are obtained from [13]. Input attributes and output classes of BCR are 9 and 2, 8 and 2 of DBT, 9 and 6 of GLS, 4 and 3 of IRS, respectively. All datasets are partitioned into three sets: a training set, a validation set, and a test set. The first 50% is used as training set for training the network, and the second 25% as validation set to check the condition of training and stop it when the overall training error goes high. The last 25% is used as test set to evaluate the generalization performance of the network and is not seen by the network during the whole training process. In all experiments, two bias nodes with a fixed input +1 are connected to the hidden layer and output layer. The learning rate η is set between [0.1,0.3] and the weights are initialized to random values between [+1.0, -1.0]. Each experiment was carried out 10 times and the presented results were the average of these 10 runs. The all experiments were done by Pentium-IV, 3 GHz desktop personal computer. 3.1 Experimental Results

Table 1 shows the average results of CFSS where the number of selected features, classification accuracies and so on are incorporated for BCR, DBT, GLS, and IRS problems in before and after feature selection process. For clarification, in this table, we measured the training error by the section 2.5 of [12], and classification accuracy CA is the ratio of classified examples to the total examples of the particular data set.

40 30 20 10 0 -10 -20 0

1

2

3

4

5

6

7

8

9

10

Classification Accuracy (%)

M.M. Kabir, M. Shahjahan, and K. Murase Attribute Contribution (%)

380

80 78 76 74 72 70 0

Attribute

2

4

6

8

10

Subset Size

Fig. 2. Contribution of attributes in breast Fig. 3. Performance of the network according to cancer problem for single run the size of subset in diabetes problem for single run Table 1. Results of four problems such as BCR, DBT, GLS, and IRS. Numbers in () are the standard deviations. Here, TERR, CA, FS, HN, ITN, and CTN refer to training error, classification accuracy, feature selection, hidden neuron, iteration, and connection respectively. Name

BCR DBT GLS IRS

Feature # 9 (0.00) 8 (0.00) 9 (0.00) 4 (0.00)

Before FS TERR (%) 3.71 (0.002) 16.69 (0.006) 26.44 (0.026) 4.69 (0.013)

CA (%) 98.34 (0.31) 75.04 (2.06) 64.71 (1.70) 97.29 (0.00)

Feature # 3.2 (0.56) 2.5 (0.52) 4.1 (1.19) 1.3 (0.48)

Table 2. Results of computational time for BCR, DBT, GLS, and IRS problems Name BCR DBT GLS IRS

Computational Time (Second) 22.10 30.07 22.30 7.90

After FS TERR (%) 3.76 (0.003) 14.37 (0.024) 26.56 (0.028) 8.97 (0.022)

CA (%) 98.57 (0.52) 76.13 (1.74) 66.62 (2.18) 97.29 (0.00)

HN#

ITN#

CTN#

2.60

249.4

18.12

2.66

271.3

16.63

3.3

1480.9

42.63

3.2

2317.1

19.96

Table 3. Results of connection reduction and accuracy increase for BCR, DBT, GLS, and IRS problems Name BCR DBT GLS IRS

Connection Decrease (%) 45.42 46.80 27.5 30.2

Accuracy Increase (%) 0.23 1.45 2.95 0.00

In each experiment, CFSS generates a subset having the minimum number of relevant features is available. Thereupon, CFSS produces better CA with a smaller number of hidden neurons HN. It is shown in Table 1 that, for example, in BCR, the network easily selects a minimum number of relevant features subset i.e. 3.2 which occurs 98.57% CA while the network used only 2.6 of hidden neurons to build the minimal network structure. In contrast, in case of remaining problems like DBT, GLS, and IRS, the results of those attributes are nearly similar as BCR except IRS. This is because, for IRS problem, the network cannot produce better accuracy any

Feature Subset Selection Using Constructive Neural Nets

381

more after feature selection rather it able to generate 1.3 relevant features subset among 4 attributes which is sufficient to exhibit the same network performance. In addition, Fig. 2 exhibits the arrangement of attributes during training according to their contribution in BCR. CFSS can easily delete the unnecessary ones and generate the subset {6,7,1}. In contrast, Fig. 3 shows the relationship between classification accuracy and subset size for DBT. We calculated the network connection of the pruned network at the final stage according to section 2.3 and the results are shown in Table 1. The computational time for completing the entire FSS process is exhibited in Table 2. After that, we estimated the connection decrement of the network corresponding to accuracy increment due to FSS as shown in Table 3 according to 2.3.1 and 2.3.2. We can see here the relation between the reduction of the network connection, and the increment of accuracy. 3.2 Comparison with other Methods

In this section, we compare the results of CFSS to those obtained by other methods NNFS and ICFS reported in [4] and [6], respectively. The results are summarized in Tables 4-7. Prudence should be considered because different technique has been involved in their methods for feature selection. Table 4. Comparison on the number of relevant features for BCR, DBT, GLS, and IRS data sets Name

CFSS

NNFS

BCR DBT GLS IRS

3.2 2.5 4.1 1.3

2.70 2.03 -

ICFS (M 1) 5 2 5 -

ICFS (M 2) 5 3 4 -

Table 6. Comparison on the average number of hidden neurons for BCR, DBT, GLS, and IRS data sets Name

CFSS

NNFS

BCR DBT GLS IRS

2.60 2.66 3.3 3.2

12 12 -

ICFS (M 1) 33.55 8.15 62.5 -

ICFS (M 2) 42.05 21.45 53.95 -

Table 5. Comparison on the average testing CA (%) for BCR, DBT, GLS, and IRS data sets Name

CFSS

NNFS

BCR DBT GLS IRS

98.57 76.13 66.62 97.29

94.10 74.30 -

ICFS (M 1) 98.25 78.88 63.77 -

ICFS (M 2) 98.25 78.70 66.61 -

Table 7. Comparison on the average number of connections for BCR, DBT, GLS, and IRS data sets Name

CFSS

NNFS

BCR DBT GLS IRS

18.12 16.63 42.63 19.96

70.4 62.36 -

ICFS (M 1) 270.4 42.75 756 -

ICFS (M 2) 338.5 130.7 599.4 -

Table 4 shows the discrimination capability of CFSS in which the dimensionality of input layer is reduced for above-mentioned four problems. In case of BCR, the result of CFSS is quite better in terms of ICFS’s two methods while it is not so with NNFS. The result of DBT is average with comparing others. But, for the case of GLS, the result is comparable or better with two methods of ICFS.

382

M.M. Kabir, M. Shahjahan, and K. Murase

The comparison on the average testing CA for all problems is shown in Table 5. It is seen that, the results of BCR and GLS are better with NNFS and ICFS while DBT with NNFS. The most important aspect however in our proposed method is, less number of connections of final network due to less number of HN in NN architecture. The results are shown in Table 6 and 7 where the number of HNs and connections of CFSS are much less in comparison to other three methods. Note that there are some missing data in Table 4-7, since IRS problem was not tested by NNFS and ICFS, and GLS problem by NNFS.

4 Discussion This paper presents a new combinatorial method for feature selection that generates subset in minimal computation due to minimal size of hidden layer as well as input layer. Constructive technique is used to achieve the compact size of hidden layer, and a straightforward contribution measurement leads to achieve reduced input layer showing better performance. Moreover, a composite combination of BE by means of double and single elimination, backtracking, and validation helps CFSS to generate subset proficiently. The results shown in Table 1 exhibit that CFSS generates subset with a small number of relevant features with producing better performance in four-benchmark problems. The results of relevant subset generation and generalization performance are better or comparable to other three methods as shown in Tables 4 and 5. From the long period, the wrapper approaches for FSS are overlooked because of the huge computation in processing. In CFSS, computational cost is much less. As seen in Table 3, due to FSS, 37.48% of computational cost is reduced in the advantage of 1.18% network accuracy for four problems in average. The computational time for different problems to complete the entire process in CFSS is shown in Table 2. We believe that these values are sufficiently low especially for clinical field. The system can give the diagnostic result of the patient to the doctor within a minute. Though the exact comparison is difficult, other methods such as NNFS and ICFS may take 4-10 times more since the numbers of hidden neurons and connections are much more as seen in Tables 6 and 7 respectively. CFSS thus provides minimal computation in feature subset selection. In addition, during subset generation, we used to meet up the generated subset for validation in each step. The reason is that, during BE we build a composite SC, which eventually find out the local minima where the network training should be stopped. Due to implementing such criterion, network produces significant performance and thus no need to validate the generated subset finally. Furthermore, further checking for irrelevant features in BE brings the completeness of CFSS. In this study we applied CFSS to the datasets with smaller number of features up to 9. To get more relevant tests for real tasks, we intend to use CFSS with other datasets having a larger number of features in future. The issue of extracting rules from NN is always demandable to interpret the knowledge how it works. For this, a NN with compactness is desirable. As CFSS can give support to fulfill the requirements, rule extraction from NN is the further task in future efficiently.

Feature Subset Selection Using Constructive Neural Nets

383

5 Conclusion This paper presents a new approach for feature subset selection based on contribution of input attributes in NN. The combination of constructive, contribution, and backward elimination carries the success of CFSS. Initially a basic constructive algorithm is used to determine a minimal and optimal structure of NN. In the latter part, one-by-one removal of input attributes is adopted that does not computationally expensive. Finally, a backward elimination with new stopping criteria are used to generate relevant feature subset efficiently. Moreover, to evaluate CFSS, we tested it on four real-world problems such as breast cancer, diabetes, glass and iris problems. Experimental results confirmed that, CFSS has a strong capability of feature selection, and it can remove the irrelevant features from the network and generates feature subset by producing compact network with minimal computational cost.

Acknowledgements Supported by grants to KM from the Japanese Society for Promotion of Sciences, the Yazaki Memorial Foundation for Science and Technology, and the University of Fukui.

References 1. Liu, H., Tu, L.: Toward Integrating Feature Selection Algorithms for Classification and Clustering. IEEE Transactions on Knowledge and Data Engineering 17(4), 491–502 (2005) 2. Dash, M., Liu, H.: Feature Selection for Classification. Intelligent Data Analysis - An International Journal 1(3), 131–156 (1997) 3. Kohavi, R., John, G.H.: Wrapper for feature subset selection. Artificial Intelligence 97, 273–324 (1997) 4. Sateino, R., Liu, H.: Neural Network Feature Selector. IEEE Transactions on Neural Networks 8 (1997) 5. Milna, L.: Feature Selection using Neural Networks with Contribution Measures. In: 8th Australian Joint Conference on Artificial Intelligence, Canberra, November 27 (1995) 6. Guan, S., Liu, J., Qi, Y.: An incremental approach to Contribution-based Feature Selection. Journal of Intelligence Systems 13(1) (2004) 7. Schuschel, D., Hsu, C.: A weight analysis-based wrapper approach to neural nets feature subset selection. In: Tools with Artificial Intelligence: Proceedings of 10th IEEE International Conference (1998) 8. Hsu, C., Huang, H., Schuschel, D.: The ANNIGMA-Wrapper Approach to Fast Feature Selection for Neural Nets. IEEE Trans. on Systems, Man, and Cybernetics-Part B: Cybernetics 32(2), 207–212 (2002) 9. Dunne, K., Cunningham, P., Azuaje, F.: Solutions to Instability Problems with Sequential Wrapper-based Approaches to Feature Selection. Journal of Machine Learning Research (2002)

384

M.M. Kabir, M. Shahjahan, and K. Murase

10. Phatak, D.S., Koren, I.: Connectivity and performance tradeoffs in the cascade correlation learning architecture, Technical Report TR-92-CSE-27, ECE Department, UMASS, Amherst (1994) 11. Rumelhart, D.E., McClelland, J.: Parallel Distributed Processing. MIT Press, Cambridge (1986) 12. Prechelt, L.: PROBEN1-A set of neural network benchmark problems and benchmarking rules, Technical Report 21/94, Faculty of Informatics, University of Karlsruhe, Germany (1994) 13. newman, D.J., Hettich, S., Blake, C.L., Merz, C.J.: UCI Repository of Machine Learning Databases, Dept. of Information and Computer Sciences, University of California, Irvine (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html

Dynamic Link Matching between Feature Columns for Different Scale and Orientation Yasuomi D. Sato1 , Christian Wolff1 , Philipp Wolfrum1 , and Christoph von der Malsburg1,2 1

2

Frankfurt Institute for Advanced Studies (FIAS), Johann Wolfgang Goethe University, Max-von-Laue-Str. 1, 60438, Frankfurt am Main, Germany Computer Science Department, University of Southern California, LA, 90089-2520, USA

Abstract. Object recognition in the presence of changing scale and orientation requires mechanisms to deal with the corresponding feature transformations. Using Gabor wavelets as example, we approach this problem in a correspondence-based setting. We present a mechanism for finding feature-to-feature matches between corresponding points in pairs of images taken at different scale and/or orientation (leaving out for the moment the problem of simultaneously finding point correspondences). The mechanism is based on a macro-columnar cortical model and dynamic links. We present tests of the ability of finding the correct feature transformation in spite of added noise.

1

Introduction

When trying to set two images of the same object or scene into correspondence with each other, so that they can be compared in terms of similarity, it is necessary to find point-to-point correspondences in the presence of changes in scale or orientation (see Fig. 1). It is also necessary to transform local features (unless one chooses to work with features that are invariant to scale and orientation, accepting the reduced information content of such features). Correspondencebased object recognition systems [1,2,3,4] have so far mainly addressed the issue of finding point-to-point correspondences, leaving local features unchanged in the process [5,6]. In this paper, we propose a system that can not only transform features for comparison purposes, but also recognize the transformation parameters that best match two sets of local features, each one taken from one point in an image. Our eventual aim will be to find point correspondences and feature correspondences simultaneously in one homogeneous dynamic link matching system, but we here take point correspondences as given for the time being. Both theoretical [7] and experimental [8] investigations are suggesting 2D-Gabor-based wavelets as features to be used in visual cortex. These are best sampled in a log-polar manner [11]. This representation has been shown to be particularly useful for face recognition [12,13], and due to its inherent symmetry it is highly appropriate for implementing a transformation system for scale and M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 385–394, 2008. c Springer-Verlag Berlin Heidelberg 2008 

386

Y.D. Sato et al.

Fig. 1. When images of an object differ in (a) scale or (b) orientation, correspondence between them can only established if also the features extracted at a given point (e.g., at the dot in the middle of the images) are transformed during comparison

orientation. The work described here is based entirely on the macro-columnar model of [14]. In computer simulations we demonstrate feature transformation and transformation parameter recognition.

2

Concept of Scale and Rotation Invariance

Let there be two images, called I and M (for image and model). The GaborL based wavelet transform J¯ (L = I or M), has the form of a vector, called a jet, whose components are defined as convolutions of the image with a family of Gabor functions:    1 1 L ¯ Jk,l (x0 , a, θ) = I(x) 2 ψ Q(θ)(x0 − x) d2 x, (1) a a R2   1 1 D2 2 ψ(x) = 2 exp(− 2 |x| ) exp(ixT · e1 ) − exp(− ) , (2) D 2D 2   cos θ sin θ Q(θ) = . (3) −sin θ cos θ Here a simultaneously represents the spatial frequency and controls the width of a Gaussian envelope window. D represents the standard deviation of the Gaussian. The individual components of the jet are indexed by n possible orientations and (m + m1 + m2 ) possible spatial frequencies, which are θ = πk/n (k ∈ {0, 1, . . . , (n−1)}) and a = al0 (0 < a0 < 1, l ∈ {0, 1, . . . , (m+m1 +m2 −1)}). Here parameters m1 and m2 fix the number of scale steps by which the image can be scaled up or down, respectively. m (> 1) is the number of scales in the jets of the model domain. L L The Gabor jet J¯ is defined as the set {J¯k,l } of NL Gabor wavelet components extracted from the position x0 of the image. In order to test the ability to find the correct feature transformation described below, we add noise of strength σ1 to L L the Gabor wavelet components, J˜k,l = J¯k,l (1 + σ1 Sran ) where Sran are random L numbers between 0 and 1. Instead of the Gabor jet J¯ itself, we employ the L sum-normalized J˜k,l . Jets can be visualized in angular-eccentricity coordinates,

Dynamic Link Matching between Feature Columns (a)

387

(b)

Fig. 2. Comparison of Gabor jets in the model domain M and the image domain I (left and right, resp., in the two part-figures). Jets are visualized in log-polar coordinates: The orientation θ of jets is arranged circularly while the spatial frequency l is set radially. The jet J M of the model domain M is shown right while the jet J I from the transformed image in the image domain I is shown left. Arrows indicate which components of J I and J M are to be compared. (a) comparison of scaled jets and (b) comparison of rotated jets.

in which orientation and spatial frequency of a jet are arranged circularly and radially (as shown in Fig.2). The components of jets that are taken from corresponding points in images I and M may be arranged according to orientation (rows) and scale (columns): ⎛ ⎞ I I 0 · · · 0 J0,m · · · J0,m 0 ··· 0 1 −1 1 −1+m ⎜ .. .. .. .. .. ⎟ , J I = ⎝ ... (4) . . . . .⎠ I I 0 · · · 0 Jn−1,m · · · Jn−1,m 0 ··· 0 1 −1 1 −1+m

⎛

JM

⎞ M M J0,m · · · J0,m 1 −1 1 −1+m ⎜ ⎟ .. .. =⎝ ⎠. . . M M Jn−1,m1 −1 · · · Jn−1,m1 −1+m

(5)

Let us assume the two images M and I to be compared have the exact same structure, apart from being transformed relative to each other. Let us assume the transformation conforms to the sample grid of jet components. Then there will be pair-wise identities between components in Eqs.(4) and (5). If the jet in I is scaled relative to the jet in M, the non-zero components of J I are shifted along the horizontal (or, in Fig.2(a), radial) coordinate. If the image I, and correspondingly the jet J I , is rotated, then jet components are circularly permuted along the vertical axis in Eq.(4) (see Fig.2(b)). When comparing scaled and rotated jets, the non-zero components of J I are shifted along both axes simultaneously. There are model jet components of m different scales, and to allow for m1 steps of scaling down the image I and m2 steps of scaling it up. The jet in Eq.(4) is padded on the left and right with m1 and m2 columns of zeros, respectively.

388

Y.D. Sato et al.

Fig. 3. A dynamic link matching network model of a pair of single macro-columns. In the I and M domains, the network consists of, respectively, NI and NM feature units (mini-columns). The units in the I and M domains represent components of the jets J I and J M . The Control domain C controls interconnections between the I and M macrocolumns, which are all-to-all at the beginning of a ν cycle. By comparing the two jets and through competition of control units in C, the initial all-to-all interconnections are dynamically reduced to a one-to-one interconnection.

3

Modelling a Columnar Network

In analogy to [14], we set up a dynamical system of variables, both for jet components and for all possible correspondences between jet components. These variables are interpreted as the activity of cortical mini-columns (here called “units”, “feature units” in this case). The set of units corresponding to a jet, either in I or in M, form a cortical macro-column (or short “column”) and inhibit each other, the strength of inhibition being controlled by a parameter ν, which cyclically ramps up and falls down. In addition, there is a column of control units, forming the control domain C. Each control unit stands for one pair of possible values of relative scale and orientation between I and M and can evaluate the similarity in the activity of feature units under the corresponding transformation. The whole network is schematically shown in Fig.3. Each domain, or column in our simple case, consists of a set of unit activities {P1L , . . . , PNLL } in the respective domain (L = I or M). NI = (m + m1 + m2 ) × n and NM = m × n represent respectively the total number of units in the I and M domains. The control unit’s activations are PC = (P1C , . . . , PNCC )T , where NC = (m1 + m2 + 1) × n. The equation system for the activations in the domains is given by  dPαL = fα (PL ) + κL CE JαL + κL (1 − CE )EαLL + ξα (t), dt (α = 1, . . . , NL and L = {M, I}\L), dPγC = fγ (PC ) + κC Fγ + ξγ (t), dt (γ = 1, . . . , NC ),

(6)

(7)

Dynamic Link Matching between Feature Columns

389

Fig. 4. Time course of all activities in the domains I, M and C over a ν cycle for two jets without relative rotation and scale, for a system with size m = 5, n = 8, m1 = 0 and m2 = 4. The activity of units in the M and I columns is shown within the bars on the right and the bottom of the panels, respectively, each of the eight subblocks corresponding to one orientation, with individual components corresponding to different scales. The indices k and l at the top label the orientation and scale of the I jet, respectively. The large matrix C contains, in the top version, as non-zero (white) entries all allowed component-to-component correspondences between the two jets (the empty, black, entries would correspond to forbidden relative scales outside the range set by m1 and m2 ). The matrix consists of 8 × 8 blocks, corresponding to the 8 × 8 possible correspondences between jet orientations, and each of these sub-matrices contains m×(m+m1 +m2 ) entries to make room for all possible scale correspondences. Each control unit controls (and evaluates) a whole jet to jet correspondence with the help of its entries in this matrix. The matrix in the lowest panel has active entries corresponding to just one control unit (for identical scale and orientation). Active units are white, silent units black. Unit activity is shown for three moments within one ν cycle. At t = 3.4 ms, the system has already singled out the correct scale, but is still totally undecided as to relative orientation. System parameters are κI = κM = 2, κC = 5, CE = 0.99, σ = 0.015 and σ1 = 0.1.

390

Y.D. Sato et al.

where ξα (t) and ξγ (t) are Gaussian noise of strength σ 2 . CE ∈ [0, 1] scales the  ratio between the Gabor jet JαL and the total input EαLL from units in the other domain, L , and κL controls the strength of the projection L to L. κC controls the strength of the influence of the similarity Fγ on the control units. Here we explain the other terms in the above equations. The function fα (·) is:   L L L L L 2 fα (P ) = a1 Pα Pα − ν(t) max {Pβ } − (Pα ) , (8) β=1,...,NL

where a1 = 25. The function ν(t) describes a cycle in time during which the feedback is periodically modulated:

k (νmax − νmin ) t−T (Tk  t < T1 + Tk ) T1 + νmin , ν(t) = . (9) νmax , (T1 + Tk  t < T1 + Tk + Trelax ) Here T1 [ms] is the duration during which ν increases with t while Tk (k = 1, 2, 3, . . .) are the periodic times at which the inhibition ν starts to increase. Trelax is a relaxation time after ν has been increased. νmin = 0.4, νmax = 1, T1 = 36.0 ms and Trelax = T1 /6 are set up in our study. Because of the increasing ν(t) and the noise in Eqs.(6) and (7), only one minicolumn in each domain remains active at the end of the cycle, while the others are deactivated, as shown in Fig.4.  If we define the mean-free column activities as P˜αL := PαL − (1/NL ) α PαL , the interaction terms in Eqs. (6) and (7) are  EαLL

=

NL NC  

β ˜ L PβC Cαα  Pα ,

(10)

β ˜I P˜γM Cγγ  Pγ  .

(11)

β=1 α =1

Fβ =

NM  NI  γ=1

γ  =1

Here NC = n×(m1 +m2 +1) is the number of permitted transformations between the two domains (n different orientations and m1 + m2 + 1 possible scales). The β expression Cαα  designates a family of matrices, rotating and scaling one jet into another. If we write β = (o, s) so that o identifies the rotation and s the scaling, then we can write the C matrices as tensor product: C β = Ao ⊗ B s , where Ao is an n × n matrix with a wrap-around shifted diagonal of entries 1 and zeros otherwise, implementing rotation o of jet components at any orientation, and B s is an m × (m + m1 + m2 ) matrix with a shifted diagonal of entries 1, and zeros otherwise, implementing an up or down shift of jet components of any scale. The C matrix implementing identity looks like the lowest panel in Fig. 4. The function E of Eq. (10), transfers feature unit activity from one domain to the other, and is a weighted superposition of all possible transformations, weighted with the activity of the control units. The function of F in Eq. (11) evaluates the similarity of the feature unit activity in one domain with that in the other under mutual transformation.

Dynamic Link Matching between Feature Columns

391

Through the columnar dynamics of Eq. (6), the relative strengths of jet components are expressed in the history of feature unit activities (the strongest jet component, for instance, letting its unit stay on longest) so that the sum of products in Eq. (11), integrated over time in Eq. (7), indeed expresses jet similarities. The time course of all unit activities in our network model during a ν cycle is demonstrated in Fig.4. In this figure, the non-zero components of the jets used in the I and M domains are identical without any rotation and scale. At the beginning of the ν cycle, all units in the I, M and C domains are active, as indicated in Fig.4 by the white jet-bars on the right and the bottom, and indicated by all admissible correspondences in the matrix being on. The activity state of the units in each domain is gradually reduced following Eqs.(6) – (11). In the intermediate state at t = 3.4 ms, all control units for the wrong scale have switched off, but the control units for all orientations are still on. At t = 12.0 ms, finally, only one control unit has survived, the one that correctly identifies the identity map. This state remains stable during the rest of the ν cycle.

4

Results

In this Section, we describe tests performed on the functionality of our macrocolumnar model by studying the robustness of matching to noise introduced into the image jets (parameter σ1 ) and the dynamics of the units, Eqs. (6) and (7) (parameter σ) and robustness to the admixture of input from the other domain, Eq. (6), due to non-vanishing (1 − CE ). A correct match is one in which the surviving control unit corresponds to the correct transformation pair which the I jet differs from the M jet. Jet noise is to model differences between jets actually extracted from natural images. Dynamic noise is to reflect signal fluctuations to be expected in units (minicolumns) composed of spiking neurons, and a nonzero admixture parameter may be relevant in a system in which transfer of jet information between the domains is desired. For both of the experiments described below, we used Gabor jets extracted from the center of real facial images. The transformed jets used for matching were extracted from images that had been scaled and rotated relative to the center position by exactly the same factors and angles that are also used in the Gabor transform. This ensures that there exists a perfect match between any two jets produced from two different rotated and scaled versions of the same face. The first experiment uses a single facial image. This experiment investigates the influence of noise in the units (σ) and of the parameter CE . Using a pair of the same jets, we obtain temporal averages of the correct matching over 10 ν cycles for each size (n = 4, 5, . . . , 9, 10, m = 4, m1 = 0 and m2 = m − 1) of the macro-column. From these temporal averages, we calculate √ the sampling average over all n. The standard error is estimated using [σd / N1 ] where σd is standard deviation of the sampling average. N1 is the sample size. Fig.5(a) shows results for κI = κM = 1 and κC = 5. For strong noise (σ = 0.015 to 0.02), the correct matching probabilities for CE = 0.98 to CE = 0.96

392

Y.D. Sato et al.

(a)

(b) CE=1.0 CE=0.98 CE=0.96 CE=0.95 CE=0.94

1

Probability Correct

Probability Correct

0.8

0.6

0.4 CE=1.0 CE=0.98 CE=0.96 CE=0.95 CE=0.94

0.2

1

0.75

0 0.5 0

0.005

0.01

0.015

0.02

0

0.005

σ

0.01

0.015

0.02

σ

Fig. 5. Probability of correct match between units in the I and M domains. (a) κI = κM = 1 and κC = 5. (b) κI = 2.3, κM = 5 and κC = 5. σ=0.000

Probability Correct

1 0.8 0.6 0.4 0.2 0 0

0.05

0.1 σ1

0.15

0.2

Fig. 6. Probability of the correct matching for comparisons of two identical faces. 6 facial images are used. Our system is set with m = 4, m1 = m2 = m − 1, n = 8, κI = κM = 6.5, κC = 5.0, CE = 0.98 and σ = 0.0.

take better values than the ones for CE = 1. Interestingly, for these low CE values, matching gets worse for weaker noise, collapsing in the case of σ = 0. This effect requires further investigation. However, for κI = 2.3 and κM = 5, we have found that the correct matching probability takes higher values than in case of Fig.5(a). In particular, our system model with higher noise σ = 0.015 even demonstrates perfect correct matching for CE = 0.98, independent of n. Next, we have investigated the robustness of the matching against the second type of noise (σ1 ), using 6 different face types. Since the original image is resized and/or rotated with m1 + m2 + 1 = 7 scales and n = 8 orientations, a total of 56 images could be employed in the I domain. Here our system is set with κI = κM = 6.5, κC = 5.0 and CE = 0.98. For each face image, we take temporal averages of the correct matching probability for each size-orientation of the image, in a similar manner as described above. Averages and standard errors of the temporal averages on 6 different facial images can be plotted, in terms of σ1 [Fig. 6]. The result of this experiment is independent of σ as long as σ  0.02. As a result of Fig. 6, as random noise in the jets is increased, the correct matching probability smoothly decreases for one image, or it abruptly shifts down to around 0 at a certain σ1 for the other image. We have also obtained

Dynamic Link Matching between Feature Columns

393

perfect matching, independent of σ1 when using the same but rotated or resized face. The most interesting point that we would like to insist is that the probability takes higher values than 87.74% as σ < 0.02 and σ1 < 0.08 (see Fig.6). Therefore, we can say that network model has a high recognition ability for scale and orientation invariance, which is not dependent of different facial types.

5

Discussion and Conclusion

The main purpose of this communication is to convey a simple concept. Much more work needs to be done on the way to practical applications, for instance, more experiments with features extracted from independently taken images of different scale and orientation to better bring out the strengths and weaknesses of the approach. In addition to comparing and transforming local packages of feature values (our jets), it will be necessary to also handle spatial maps, that is, sets of point-to-point correspondences, a task we will approach next. Real applications involve, of course, a continuous range of transformation parameters, whereas we here had admitted only transformations from the same sample grid used for defining the family of wavelets in a jet. We hope to address this problem by working with continuous superpositions of transformation matrices for the neighboring transformation parameter values that straddle the correct value. Our approach may be seen as using brute force, as it requires many separate control units to sample the space of transformation parameters with enough density. However, the same set of control units can be used in a whole region of visual space, and in addition to that, for controlling point-to-point maps between regions, as spatial maps and feature maps stand in one-to-one correspondence to each other. The number of control units can be reduced even further if transformations are performed in sequence, by consecutive, separate layers of dynamic links. Thus, the transformation from an arbitrary segment of primary visual cortex to an invariant window could be done in the sequence translation – scaling – rotation. In this case, the number of required control units would be the sum and not the product of the number of samples for each individual transformation. A disadvantage of that approach might be added difficulties in finding correspondences between the domains. Further work is required to elucidate these issues.

Acknowledgements This work was supported by the EU project Daisy, FP6-2005-015803 and by the Hertie Foundation. We would like to thank C. Weber for help with preparing the manuscript.

References 1. Anderson, C.H., Van Essen, D.C., Olshausen, B.A.: Directed visual attention and the dynamic control of information flow. In: Itti, L., Rees, G., Tsotsos, J.K. (eds.) Neurobiology of attention, pp. 11–17. Academic Press/Elservier (2005) 2. Weber, C., Wermter, S.: A self-organizing map of sigma-pi units. Neurocomputing 70, 2552–2560 (2007)

394

Y.D. Sato et al.

3. Wiskott, L., v.d. Malsburg, C.: Face Recognition by Dynamic Link Matching, ch. 4. In: Sirosh, J., Miikkulainen, R., Choe, Y. (eds.) Lateral Interactions in the Cortex: Structure and Function, vol. 4, Electronic book (1996) 4. Wolfrum, P., von der Malsburg, C.: What is the optimal architecture for visual information routing? Neural Comput. 19 (2007) 5. Lades, M.: Invariant Object Recognition with Dynamical Links, Robust to Variations in Illumination. Ph.D. Thesis, Ruhr-Univ. Bochum (1995) 6. Maurer, T., von der Malsburg, C.: Learning Feature Transformations to Recognize Faces Rotated in Depth. In: Fogelman-Souli´e, F., Rault, J.C., Gallinari, P., Dreyfus, G. (eds.) Proc. of the International Conference on Artificial Neural Networks ICANN 1995, EC2 & Cie, p. 353 (1995) 7. Daugmann, J.G.: Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. J. Opt. Soc. Am. A 2, 1160–1169 (1985) 8. Jones, J., Palmer, L.: An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex. J. Neurophysiol. 58, 1233–1258 (1987) 9. Pentland, A., Moghaddam, B., Starner, T.: View-based and modular eigenspaces for face recognition. In: Proc. of the third IEEE Conference on Computer Vision and Pattern Recognition, pp. 84–91 (1994) 10. Wiskott, L., Fellous, J.-M., Kr¨ uger, von der Malsburg, C.: Face Recognition by Elastic Bunch Graph Matching. IEEE Trans. Pattern Anal. & Machine Intelligence 19, 775–779 (1997) 11. Marcelja, S.: Mathematical description of the responses of simple cortical cells. J. Optical Soc. Am. 70, 1297–1300 (1980) 12. Yue, X., Tjan, B.C., Biederman, I.: What makes faces special? Vision Res. 46, 3802–3811 (2006) 13. Okada, K., Steffens, J., Maurer, T., Hong, H., Elagin, E., Neven, H., von der Malsburg, C.: The Bochum/USC Face Recognition System and How it Fared in the FERET Phase III Test. In: Wechsler, H., Phillips, P.J., Bruce, V., Fogelman Souli´e, F., Huang, T.S. (eds.) Face Recognition: FromTheory to Applications, pp. 186–205. Springer, Heidelberg (1998) 14. L¨ ucke, J., von der Malsburg, C.: Rapid Correspondence Finding in Networks of Cortical Columns. In: Kollias, S., Stafylopatis, A., Duch, W., Oja, E. (eds.) ICANN 2006. LNCS, vol. 4131, pp. 668–677. Springer, Heidelberg (2006)

Perturbational Neural Networks for Incremental Learning in Virtual Learning System Eiichi Inohira1 , Hiromasa Oonishi2 , and Hirokazu Yokoi1 1

Kyushu Institute of Technology, Hibikino 2-4, 808-0196 Kitakyushu, Japan {inohira,yokoi}@life.kyutech.ac.jp 2 Mitshbishi Heavy Industries, Ltd., Japan

Abstract. This paper presents a new type of neural networks, a perturbational neural network to realize incremental learning in autonomous humanoid robots. In our previous work, a virtual learning system has been provided to realize exploring plausible behavior in a robot’s brain. Neural networks can generate plausible behavior in unknown environment without time-consuming exploring. Although an autonomous robot should grow step by step, conventional neural networks forget prior learning by training with new dataset. Proposed neural networks features adding output in sub neural network to weights and thresholds in main neural network. Incremental learning and high generalization capability are realized by slightly changing a mapping of the main neural network. We showed that the proposed neural networks realize incremental learning without forgetting through numerical experiments with a twodimensional stair-climbing bipedal robot.

1

Introduction

Recently, humanoid robots such as ASIMO [1] dramatically develop in view of hardware and are prospective for working just like a human. Although many researchers have studied artificial intelligence for a long time, humanoid robots have not so high autonomy as experts are not wanted. Humanoid robots should accomplish missions by itself rather than experts give them solutions such as models, algorithms, and programs. Researchers have studied to realize a robot with learning ability through trial and error. Such studies uses so-called soft computing techniques such as reinforcement learning [2] and central pattern generator (CPG) [3]. Learning of a robot saves expert’s work to some degree but takes much time. These techniques are less efficient than humans. Humans instantly act depending on a situation by using imagination and experience. Even if a human fails, a failure will serve himself as experience. In particular, humans know characteristics of environment and their behavior and simulate trial and error to explore plausible behavior in their brains. In our previous work, Yamashita and et al. [4] have proposed a bipedal walking control system with virtual environment based on motion control mechanism of primates. In this study, exploring plausible behavior by trial and error is carried M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 395–404, 2008. c Springer-Verlag Berlin Heidelberg 2008 

396

E. Inohira, H. Oonishi, and H. Yokoi

out in a robot’s brain. However, the problem is that exploring takes much time. Kawano and et al. [5] have introduced learning of a relation of environmental data and an optimized behavior into the previous work to save time. Generating behavior becomes fast because a trained neural network (NN) immediately outputs plausible behavior due to its generalization capability, which is an ability to obtain a desired output at an unknown input. However, a problem in this previous work is that conventional neural networks cannot realize incremental learning. It means that a trained NN forgets prior learning by training with new dataset. Realizing incremental learning is indispensable to learn various behaviors step by step. We presents a new type of NNs, perturbational neural networks (PNN) to achieve incremental learning. PNN consists of main NN and sub NN. Main NN learns with representative dataset and never changes after that. Sub NN learns with new dataset as output error of main NN for the dataset is canceled. Sub NN is connected with main NN as additional terms of weights and thresholds in main NN. We guess that connections through weights and thresholds slightly changes characteristics of main NN, which means that perturbation is given to mapping of main NN, and that incremental learning does not sacrifice its generalization capability.

2

A Virtual Learning System for a Biped Robot

We assume that a robot tries to achieve a task in an unknown environment. Virtual learning is defined as learning of virtual experience in virtual environment in a robot’s brain. Virtual environment is generated from sensory information on real environment around a robot and used for exploring behavior fit for a task without real action. Exploring such behavior in virtual environment is regarded as virtual experience by trial and error or ingenuity. Virtual learning is to memorize a relation between environment and behavior under a particular task. Benefits of virtual learning are (1) to reduce risk of robot’s failure in the real world such as falling and colliding and (2) to enable a robot to immediately act and achieve learned tasks in similar environments. The former is involved in a robot’s safety and physical wastage. The latter leads to saving work and time to achieve a task. We presented virtual learning system for a biped robot [4,5]. This virtual learning system has virtual environment and NNs for learning as shown in Fig.1. We assume that central pattern generator (CPG) generates motion of a biped robot. CPG of a biped robot consists of neural oscillators corresponding to its joints. Then behavior is described as CPG parameters rather than joint angles. CPG parameters for controlling a robot are called CPG input. Optimized CPG input for a plausible behavior is given by exploring in virtual environment. Virtual learning is realized by NNs. A NN learned a mapping from environmental data to optimized CPG input, i.e, a mapping from real environment to robot’s behavior. NNs have generalization capability, which means that desired output can be generated at an unknown input. When a NN learned a mapping

PNNs for Incremental Learning in Virtual Learning System

397

Virtual Environment

Exploring optimized motion Environmental data

Does it serve a purpose ?

CPG

CPG input Robot

NN

Fig. 1. Virtual learning system for a biped robot

with representative data, it can generate desired CPG input at unknown environment and then a biped robot achieves a task. It takes much time to train NN, but NN is fast to generate output. CPG input is first generated by NN and then checked in virtual environment. When the CPG input generated by NN achieve a task, it is sent to a CPG. Otherwise, CPG input obtained by exploring, which takes time, is used for controlling a robot. And NN learns with CPG input obtained by exploring to correct its error. Combination of NN and exploring realizes autonomous learning and covers their shortcomings each other.

3

Neural Networks for Incremental Learning

A key of our virtual learning system is NN. A virtual learning system should develop with experience in various environments and tasks. However, multilayer perceptron (MLP) with back-propagation (BP) learning algorithm, which is widely used in many applications, has a problem that it forgets prior learning through training with a new dataset because its connection weights are overwritten. If all training datasets are stored in a memory, prior learning can be reflected in a MLP by training with all the datasets, but training with a large volume of dataset takes huge amount of time and memory. Of course humans can learn something new with keeping old experiences. A virtual learning system needs to accumulate virtual experiences step by step. 3.1

Perturbational Neural Networks

The basic idea of PNN is that a NN learns a new dataset by adding new parameters to constant weights and thresholds after training without overwriting them. PNN consists of main NN and sub NN as shown in Fig.2. Main NN learns with representative datasets and is constant after this learning. Sub NN learns with new datasets and generates new terms in weights and thresholds of main

398

E. Inohira, H. Oonishi, and H. Yokoi Input

Main NN

Output

Δw, Δh Sub NN

Fig. 2. A perturbational neural network

Neuron w1 Δ w1

Input

x1 −

f

z1

Output

Δh

h From Sub NN

Fig. 3. A neuron in main NN

NN, i.e., Δw and Δh. A neuron of PNN is shown in Fig.3. Input-output relation of a conventional neuron is given by the following equation.    z=f wi xi − h (1) i

where z denotes output, wi weight, xi input, and f (·) activation function. A neuron of PNN is given as follows.    z=f (wi + Δwi )xi − (h + Δh)

(2)

i

where Δwi and Δh are outputs of sub NN. Training of PNN is divided into two phases. First, main NN learns a main NN with representative dataset. For instance, it is assumed that representative datasets consists of environmental data (EA , EB , EC ) and CPG parameters

PNNs for Incremental Learning in Virtual Learning System

399

PA , PB , P C EA , EB , E C

Main NN Δw, Δh Sub NN 0

Fig. 4. Training of NN with representative dataset

PD Stop learning

ED

Main NN Δw, Δh Sub NN

Fig. 5. Training of NN with new dataset

(PA , PB , PC ) as shown in Fig.4. Training of main NN is the same as NN with BP. At this time, sub NN learn as Δw and Δh equal zero. Then, for representative dataset, sub NN has no effect on main NN. Next, PNN learns with new dataset. For instance, it is assumed that new dataset consists of environmental data ED and CPG parameters PD as shown in Fig.5. Main NN does not learn with the new dataset, but reinforcement signals are passed to sub NN through main NN. Output error of main NN for ED exists because it is unknown for main NN. Sub NN learns with new dataset as such error is canceled. 3.2

Related Work

Some authors [6,7,8] have proposed other methods in which Mixture of Experts (MoE) architecture is applied to incremental learning of neural networks. The MoE architecture is based on divide-and-conquer approach. It means that a complex mapping which a signal neural network should learn is divided into simple mappings which a neural network can learn easily in such architecture. On the other hand, a PNN learns a complex mapping by connecting sub NNs with a main NN. A PNN is expected to have global generalization capability because it does not divided a mapping to local mappings. In the MoE architecture, a expert neural network is expected to have local generalization capability. However, a system would not have global generalization capability because it is not concerned. Therefore, when generalization capability is focused on, a PNN should be used.

400

E. Inohira, H. Oonishi, and H. Yokoi

A PNN have a problem that it needs large number of connections from sub NNs to a main NN. It means that a PNN is very inefficient in resources. Although we have not yet study efficiency in a PNN, we guess that a PNN has room to reduce the connections, and it would be our future work.

4 4.1

Numerical Experiments Setup

We evaluate generalization capability of proposed NN for incremental learning through numerical experiments of a robot climbing stairs. Simplified experiments are performed because we focus on a NN for virtual learning.

θ3 θ4

θ2 θ1

θ5

Fig. 6. A two-dimensional five-link biped robot model

θ3

Neural oscillator

θ2

θ4

θ1

θ5

Fig. 7. A CPG for a biped robot

A two-dimensional five-link model of a bipedal robot is used as shown in Fig. 6. Five joint angles defined in Fig. 6 are controlled by five neural oscillators corresponding to each joint angle. Length of all link is 100 cm. A task of a bipedal robot is to climb five stairs. Height and depth of stairs are used for

PNNs for Incremental Learning in Virtual Learning System

401

Table 1. Dimension of representative stairs Stairs Height [cm] Depth [cm] A 30 60 B 10 70 C 30 100 D 10 90

Output

Input

Main NN Sub NN Sub NN for hidden layer

Δwh , Δhh

Sub NN for output layer

Δwo , Δho

Fig. 8. A PNN used for experiments

environmental data. A robot concerns only kinematics in virtual environment and ignores dynamics. A CPG shown in Fig. 7 is used for controlling of a bipedal robot. We used Matsuoka’s neural oscillator model [9] in a CPG. In this study, 16 connection weights w and five constant external inputs u0 are defined as CPG input for controlling a bipedal robot. CPG input for climbing stairs are given through exploring their parameter space by GA and are optimized for each environment. Internal parameters of CPG are also given by GA and are constant in all environments because it takes much time to explore all CPG parameters including internal parameters. Internal parameters of CPG are optimized for walking on a horizontal plane. The four pairs shown in Table 1 are defined as representative data for NN training. Then inputs and outputs of the NNs numbers 2 and 21 respectively. The following targets are compared to evaluating their generalization capability. – CPG whose parameters are optimized from each of the three representative environmental data, i.e., stairs A, B, and C – MLP trained for stairs A, B, and C (MLP-I) – MLP trained for stairs D after trained for stairs A, B, and C (MLP-II) – PNN trained in the same way as the above MLP These targets are optimized and trained for one to four kinds of stairs. Generalization capability of each targets is calculated by the number of conditions which is different from the representative stairs and where a biped robot can climb five

402

E. Inohira, H. Oonishi, and H. Yokoi

stairs successfully. Stairs’ height ranges from 4 cm to 46 cm and width from 40 cm to 110 cm. MLPs has 30 neurons in a hidden layer. Initial weights of MLPs are given by uniform random numbers ranging between ±0.3. PNN used in this paper has two sub NNs as shown in Fig.8. Main NN in PNN is the same as the above MLP. Sub NN for hidden layer has 100 neurons and 90 outputs. Sub NN for output layer has 600 neurons and 561 outputs. All initial weights in PNN are given in the same way as the MLPs. One learning cycle is defined as stairs A, B, and C are given to MLP or PNN sequentially. MLP-I is trained for the three kinds of stairs until 10000 learning cycles where sum of squared error is much less than 10−7 . MLP-II is trained for stairs D until 1000 learning cycles after trained for the three kinds of stairs. As mentioned below, although the number of learning cycles for stairs D is small, MLP-II forgets stair A, B, and C. Training condition for Main NN in PNN is the same as MLP-I. Incremental learning of PNN for stairs D means that the two sub NNs are trained for stairs D while the main NN is constant. These sub NNs are trained until 700 learning cycles. Learning rate of NNs is optimized through preliminary experiments because their performance heavily depends on it. We used learning rate minimizing sum of squared error under a certain number of learning cycles for each of NNs. Learning rate used in MLP-I and sub NN of PNN is 0.30 and 0.12 respectively. 4.2

Experimental Results

In Fig. 9, mark x denotes successful condition in climbing five stairs and a circle a given condition as representative dataset. Fig. 9 (a) to (c) show that successful conditions spreads around representative stairs to some degree. It means that CPG has a certain generalization capability by itself as already known. However, generalization capability of CPG is not as much as stairs A, B, and C are covered simultaneously. On the other hand, Fig. 9 (d) shows that MLP-I covers intermediate conditions in stairs A, B, and C. Effect of virtual learning with NNs is clear. Fig. 9 (e) and (f) are involved in incremental learning. Fig. 9 (e) shows that PNN is successful in incremental learning. Moreover, when conditions near stairs C are focused on, generalization capability of PNN is larger than MLP-I. Fig. 9 (f) shows that MLP-II forgot the precedent learning on stairs A, B, and C and fails in incremental learning. It is known that incremental learning of MLP with BP fails because connection weights are overwritten by training with new dataset. In PNN, main NN is constant after training with initial dataset. Then main NN does not forget initial dataset. Incremental learning is realized by sub NNs in PNN. The problems of sub NNs are effects on adjusting of connection weights, i.e., whether it is successful in incremental learning or not, and whether performance for initial dataset decreases or not. From the experimental results, we showed that PNN realized incremental learning and increased generalization capability by incremental learning.

PNNs for Incremental Learning in Virtual Learning System

45

45

40

40

25 20

10

80 100 Depth [cm] (d) Trained NN with data A, B, and C

45

45

40

40

35

35

25 20

A

30

C

25 20

B

10 5 40

60

80 100 Depth [cm] (b) CPG without NN for data B

D

80 100 Depth [cm] (e) Trained proposed NN with data D after the three data

45

45

40

40

35

60

25 20

Height [cm]

35 C

30

30

15 10 80 100 Depth [cm] (c) CPG without NN for data C

C

20

10 60

A

25

15

5 40

60

15 B

10 5 40

B

5 40

80 100 Depth [cm] (a) CPG without NN for data A

15

Height [cm]

20

10 60

C

25

15

30

A

30

15

5 40

Height [cm]

Height [cm]

30

35 A

Height [cm]

Height [cm]

35

403

5 40

B

D

60

80 100 Depth [cm] (f) Trained conventional NN with data D after the three data

Fig. 9. A comparison of generalization capability of CPG and MLP and PNN in stair climbing

404

5

E. Inohira, H. Oonishi, and H. Yokoi

Conclusions

We proposed a new type of NNs for incremental learning in a virtual learning system. Our proposed NNs features adjusting weights and thresholds externally to slightly change a mapping of a trained NN. This paper demonstrated numerical experiments with a two-dimensional five-link biped robot and climbing-stairs task. We showed that PNN is successful in incremental learning and has generalization capability to some degree. This study is very limited to focus on only verifying our approach. In future work, we will study PNN with as much data as an actual robot needs and compare with related work quantitatively.

References 1. Hirai, K., Hirose, M., Haikawa, Y., Takenaka, T.: The development of honda humanoid robot. In: Proc. IEEE ICRA, vol. 2, pp. 1321–1326 (1998) 2. Mahadevan, S., Connell, J.: Automatic programming of behavior-based robots using reinforcement learning. Artificial Intelligence 55, 311–365 (1992) 3. Kuniyoshi, Y., Sangawa, S.: Early motor development from partially ordered neuralbody dynamics: experiments with a cortico-spinal-musculo-skeletal model. Biological Cybernetics 95, 589–605 (2006) 4. Yamashita, I., Yokoi, H.: Control of a biped robot by using several virtual environments. In: Proceedings of the 22nd Annual Conference of the Robotics Society of Japan 1K25 (in Japanese) (2004) 5. Kawano, T., Yamashita, I., Yokoi, H.: Control of the bipedal robot generating the target by the simulation in virtual space (in Japanese). IEICE Technical Report 104, 65–69 (2004) 6. Haruno, M., Wolpert, D.M., Kawato, M.: MOSAIC model for sensorimotor learning and control. Neural Computation 13, 2201–2220 (2001) 7. Schaal, S., Atkeson, C.G.: Constructive incremental learning from only local information. Neural Computation 10, 2047–2084 (1998) 8. Yamauchi, K., Hayami, J.: Incremental learning and model selection for radial basis function network through sleep. IEICE TRANSACTIONS on Information and Systems E90-D, 722–735 (2007) 9. Matsuoka, K.: Sustained oscillations generated by mutually inhibiting neurons with adaptation. Biological Cybernetics 52, 367–376 (1985)

Bifurcations of Renormalization Dynamics in Self-organizing Neural Networks Peter Tiˇ no University of Birmingham, Birmingham, UK [email protected]

Abstract. Self-organizing neural networks (SONN) driven by softmax weight renormalization are capable of finding high quality solutions of difficult assignment optimization problems. The renormalization is shaped by a temperature parameter - as the system cools down the assignment weights become increasingly crisp. It has been reported that SONN search process can exhibit complex adaptation patterns as the system cools down. Moreover, there exists a critical temperature setting at which SONN is capable of powerful intermittent search through a multitude of high quality solutions represented as meta-stable states. To shed light on such observed phenomena, we present a detailed bifurcation study of the renormalization process. As SONN cools down, new renormalization equilibria emerge in a strong structure leading to a complex skeleton of saddle type equilibria surrounding an unstable maximum entropy point, with decision enforcing “one-hot” stable equilibria. This, in synergy with the SONN input driving process, can lead to sensitivity to annealing schedules and adaptation dynamics exhibiting signatures of complex dynamical behavior. We also show that (as hypothesized in earlier studies) the intermittent search by SONN can occur only at temperatures close to the first (symmetry breaking) bifurcation temperature.

1

Introduction

For almost three decades there has been an energetic research activity on application of neural computation techniques in solving difficult combinatorial optimization problems. Self-organizing neural network (SONN) [1] constitutes an example of a successful neural-based methodology for solving 0-1 assignment problems. SONN has been successfully applied in a wide variety of applications, from assembly line sequencing to frequency assignment in mobile communications. As in most self-organizing systems, dynamics of SONN adaptation is driven by a synergy of cooperation and competition. In the competition phase, for each item to be assigned, the best candidate for the assignment is selected and the corresponding assignment weight is increased. In the cooperation phase, the assignment weights of other candidates that were likely to be selected, but were not quite as strong as the selected one, get increased as well, albeit to a lesser M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 405–414, 2008. c Springer-Verlag Berlin Heidelberg 2008 

406

P. Tiˇ no

degree. The assignment weights need to be positive and sum to 1. Therefore, after each SONN adaptation phase, the assignment weights need to be renormalized back onto the standard simplex e.g via the softmax function [2]. When endowed with a physics-based Boltzmann distribution interpretation, the softmax function contains a temperature parameter T > 0. As the system cools down, the assignments become increasingly crisp. In the original setting SONN is annealed so that a single high quality solution to an assignment problem is found. Yet, renormalization onto the standard simplex is a double edged sword. On one hand, SONN with assignment weight renormalization have empirically shown sensitivity to annealing schedules, on the other hand, the quality of solutions could be greatly improved [3]. Interestingly enough, it has been reported recently [4] that there exists a critical temperature T∗ at which SONN is capable of powerful intermittent search through a multitude of high quality solutions represented as meta-stable states of the adaptation SONN dynamics. It is hypothesised that the critical temperature may be closely related to the symmetry breaking bifurcation of equilibria in the autonomous softmax dynamics. At present there is still no theory regarding the dynamics of SONN adaptation driven by the softmax renormalization. Consequently, the processes of crystallising a solution in an annealed version of SONN, or of sampling the solution space in the intermittent search regime are far from being understood. The first steps towards theoretical underpinning of SONN adaptation driven by softmax renormalization were taken in [5,4,6]. For example, in [5] SONN is treated as a dynamical system with a bifurcation parameter T . The cooperation phase was not included in the model, The renormalization process was empirically shown to result in complicated bifurcation patterns revealing a complex nature of the search process inside SONN as the systems gets annealed. More recently, Kwok and Smith [4] suggested to study SONN adaptation dynamics by concentrating on the autonomous renormalization process, since it is this process that underpins the search dynamics in the SONN. In [6] we initiated a rigorous study of equilibria of the autonomous renormalization process. Based on dynamical properties of the autonomous renormalization, we found analytical approximations to the critical temperature T∗ as a function of SONN size. In this paper we complement [6] by reporting a detailed bifurcation study of the renormalization process and give precise characterization and stability types of equilibria, as they emerge during the annealing process. Interesting and intricate equilibria structure emerges as the system cools down, explaining empirically observed complexity of SONN adaptation during intermediate stages of the annealing process. The analysis also clarifies why the intermittent search by SONN occurs near the first (symmetry breaking) bifurcation temperature of the renormalization step, as was experimentally verified in [4,6]. Due to space limitations, we cannot fully prove statements presented in this study. Detailed proofs can be found in [7] and will be published elsewhere.

Bifurcations of Renormalization Dynamics in SONNs

2

407

Self-organizing Neural Network and Iterative Softmax

First, we briefly introduce Self-Organizing Neural Network (SONN) endowed with weight renormalization for solving assignment optimization problems (see e.g. [4]). Consider a finite set of input elements (neurons) i ∈ I = {1, 2, ..., M } that need to be assigned to outputs (output neurons) j ∈ J = {1, 2, ..., N }, so that some global cost of an assignment A : I → J is minimized. Partial cost of assigning i ∈ I to j ∈ J is denoted by V (i, j). The “strength” of assigning i to j is represented by the “assignment weight” wi,j ∈ (0, 1). The SONN algorithm can be summarized as follows: The connection weights wi,j , i ∈ I, j ∈ J , are first initialized to small random values. Then, repeatedly, an output item jc ∈ J is chosen and the partial costs V (i, jc ) incurred by assigning all possible input elements i ∈ I to jc are calculated in order to select the “winner” input element (neuron) i(jc ) ∈ I that minimizes V (i, jc ). The “neighborhood” BL (i(jc )) of size L of the winner node i(jc ) consists of L nodes i  = i(jc ) that yield the smallest partial costs V (i, jc ). Weights from nodes i ∈ BL (i(jc )) to jc get strengthened: wi,jc ← wi,jc + η(i)(1 − wi,jc ),

i ∈ BL (i(jc )),

(1)

where η(i) is proportional to the quality of assignment i → jc , as measured by V (i, jc ). Weights1 wi = (wi,1 , wi,2 , ..., wi,N ) for each input node i ∈ I are then renormalized using softmax wi,j ←

wi,j T ) N wi,k . k=1 exp( T )

exp(

(2)

We will refer to SONN for solving an (M, N )-assignment problem as (M, N )SONN. As mentioned earlier, following [4,6] we strive to understand the search dynamics inside SONN by analyzing the autonomous dynamics of the renormalization update step (2) of the SONN algorithm. The weight vector wi of each of M neurons in an (M, N )-SONN lives in the standard (N − 1)-simplex, SN −1 = {w = (w1 , w2 , ..., wN ) ∈ RN | wi ≥ 0, i = 1, 2, ..., N, and

N 

wi = 1}.

i=1

Given a value of the temperature parameter T > 0, the softmax renormalization step in SONN adaptation transforms the weight vector of each unit as follows: w → F(w; T ) = (F1 (w; T ), F2 (w; T ), ..., FN (w; T )) , where Fi (w; T ) = 1

exp( wTi ) , i = 1, 2, ..., N, Z(w; T )

Here  denotes the transpose operator.

(3)

(4)

408

P. Tiˇ no

 wk N and Z(w; T ) = N k=1 exp( T ) is the normalization factor. Formally, F maps R 0 to SN −1 , the interior of SN −1 . 0 Linearization of F around w ∈ SN −1 is given by the Jacobian J(w; T ): J(w; T )i,j =

1 [δi,j Fi (w; T ) − Fi (w; T )Fj (w; T )], T

i, j = 1, 2, ..., N,

(5)

where δi,j = 1 iff i = j and δi,j = 0 otherwise. 0 The softmax map F induces on SN −1 a discrete time dynamics known as Iterative Softmax (ISM): w(t + 1) = F(w(t); T ).

(6)

The renormalization step in an (M, N )-SONN adaptation involves M separate renormalizations of weight vectors of all of the M SONN units. For each temperature setting T , the structure of equilibria in the i-th system, wi (t + 1) = F(wi (t); T ), gets copied in all the other M − 1 systems. Using this symmetry, it is sufficient to concentrate on a single ISM (6). Note that the weights of different units are coupled by the SONN adaptation step (1). We will study systems for N ≥ 2.

3

Equilibria of SONN Renormalization Step

We first introduce basic concepts and notation that will be used throughout the paper. An (r − 1)-simplex is the convex hull of a set of r affinely independent points in Rm , m ≥ r−1. A special case is the standard (N −1)-simplex SN −1 . The convex hull of any nonempty subset of n vertices of an (r − 1)-simplex Δ, n ≤ r,  is called an (n − 1)-face of Δ. There are nr distinct (n − 1)-faces of Δ and each (n − 1)-face is an (n − 1)-simplex. Given a set of n vertices w1 , w2 , ..., wn ∈ Rm defining an (n − 1)-simplex Δ in Rm , the central point, n

w(Δ) =

1 wi , n i=1

(7)

is called the maximum entropy point of Δ. We will denote the set of all (n − 1)-faces of the standard (N − 1)-simplex SN −1 by PN,n . The set of their maximum entropy points is denoted by QN,n , i.e. QN,n = {w(Δ)| Δ ∈ PN,n }.

(8)

The n-dimensional column vectors of 1’s and 0’s are denoted by 1n and 0n , respectively. Note that wN,n = n1 (1n , 0N −n ) ∈ QN,n . In addition, all the other elements of QN,n can be obtained by simply permuting coordinates of wN,n . Due to this symmetry, we will be able to develop most of the material using wN,n only and then transfer the results to permutations of wN,n . The maximum entropy point wN,N = N −1 1N of the standard (N − 1)-simplex SN −1 will be denoted

Bifurcations of Renormalization Dynamics in SONNs

409

simply by w. To simplify the notation we will use w to denote both the maximum entropy point of SN −1 and the vector w − 0N . We showed in [6] that w is a fixed point of ISM (6) for any temperature setting T and all the other fixed points w = (w1 , w2 , ..., wN ) of ISM have exactly two different coordinate values: wi ∈ {γ1 , γ2 }, such that N −1 < γ1 < N1−1 and 0 < γ2 < N −1 , where N1 is the number of coordinates γ1 larger than N −1 . Since 0 w ∈ SN −1 , we have 1 − N1 γ1 γ2 = . (9) N − N1 The number of coordinates γ2 smaller than N −1 is denoted by N2 . Obviously,   N2 = N − N1 . If w = (γ1 1N1 , γ2 1N2 ) is a fixed point of ISM (6), so are all NN1 distinct permutations of it. We collect w and its permutations in a set    1 − N γ 1 1 EN,N1 (γ1 ) = v| v is a permutation of γ1 1N1 , 1 . (10) N − N1 N −N1 The fixed points in EN,N1 (γ1 ) exist if and only if the temperature parameter T is set to [6]

 TN,N1 (γ1 ) = (N γ1 − 1) −(N − N1 ) · ln 1 −

N γ1 − 1 (N − N1 )γ1

−1 .

(11)

We will show that as the system cools down, increasing number of equilibria emerge in a strong structure. Let w, v ∈ SN −1 be two points on the standard simplex. The line from w to v is parametrized as (τ ; w, v) = w + τ · (v − w),

τ ∈ [0, 1].

(12)

Theorem 1. All equilibria of ISM (6) lie on lines connecting the maximum entropy point w of SN −1 with the maximum entropy points of its faces. In particular, for 0 < N1 < N and γ1 ∈ (N −1 , N1−1 ), all fixed points from EN,N1 (γ1 ) lie on lines (τ ; w, w), where w ∈ QN,N1 . Sketch of the Proof: Consider the maximum entropy point wN,N1 = N11 (1N1 , 0N2 ) of an (N1 −1)-face of SN −1 . Then w(γ1 ) = (γ1 1N1 , γ2 1N2 ) lies on the line (τ ; w, wN,N1 ) for the parameter setting τ = 1 − N γ2 . Q.E.D. The result is illustrated in figure 1. As the (M, N )-SONN cools down, the ISM equilibria emerge on lines connecting w with the maximum entropy points of faces of SN −1 of increasing dimensionality. Moreover, on each such line there can be at most two ISM equilibria. Theorem 2. For N1 < N/2, there exists a temperature TE (N, N1 ) > N −1 such that for T ∈ (0, TE (N, N1 )], ISM fixed points in EN,N1 (γ1 ) exist for some γ1 ∈

410

P. Tiˇ no

Fig. 1. Positions of equilibria of SONN renormalization illustrated for the case of 4-dimensional weight vectors w. ISM is operating on the standard 3-simplex S3 and its equilibria can only be found on the lines connecting the maximum entropy point w (filled circle) with maximum entropy points of its faces. Triangles, squares and diamonds represent maximum entropy points of 0-faces (vertices), 1-faces (edges) and 2-faces (facets), respectively.

(N −1 , N1−1 ), and no ISM fixed points in EN,N1 (γ1 ), for any γ1 ∈ (N −1 , N1−1 ), can exist at temperatures T > TE (N, N1 ). For each temperature T ∈ (N −1 , TE (N, N1 )), there are two coordinate values γ1− (T ) and γ1+ (T ), N −1 < γ1− (T ) < γ1+ (T ) < N1−1 , such that ISM fixed points in both EN,N1 (γ1− (T )) and EN,N1 (γ1+ (T )) exist at temperature T . Furthermore, as the temperature decreases, γ1− (T ) decreases towards N −1 , while γ1+ (T ) increases towards N1−1 . For temperatures T ∈ (0, N −1 ], there is exactly one γ1 (T ) ∈ (N −1 , N1−1 ) such that ISM equilibria in EN,N1 (γ1 (T )) exist at temperature T . Sketch of the Proof: The temperature function TN,N1 (γ1 ) (11) is concave and can be continuously extended to [N −1 , N1−1 ) with TN,N1 (N −1 ) = N −1 and limγ1 →N −1 TN,N1 (γ1 ) = 1 0 < N −1 . The slope of TN,N1 (γ1 ) at N −1 is positive for N1 < N/2. Q.E.D. Theorem 3. The bifurcation temperature TE (N, N1 ) is decreasing with increasing number N1 of equilibrium coordinates larger than N −1 . Sketch of the Proof: It can be shown that for any feasible value of γ1 > N −1 , if there are two fixed points w ∈ EN,N1 (γ1 ) and w ∈ EN,N1 (γ1 ) of ISM, such that N1 < N1 , then w exists at a higher temperature than w does. For a given N1 < N/2, the bifurcation temperature TE (N, N1 ) corresponds to the maximum of TN,N1 (γ1 ) on γ1 ∈ (N −1 , N1−1 ). It follows that N1 < N1 implies TE (N, N1 ) > TE (N, N1 ). Q.E.D.

Bifurcations of Renormalization Dynamics in SONNs

411

Theorem 4. If N/2 ≤ N1 < N , for each temperature T ∈ (0, N −1 ), there is exactly one coordinate value γ1 (T ) ∈ (N −1 , N1−1 ), such that ISM fixed points in EN,N1 (γ1 (T )) exist at temperature T . No ISM fixed points in EN,N1 (γ1 ), for any γ1 ∈ (N −1 , N1−1 ) can exist for temperatures T > N −1 . As the temperature decreases, γ1 (T ) increases towards N1−1 . Sketch of the Proof: Similar to the proof of theorem 2, but this time the slope of TN,N1 (γ1 ) at N −1 is not positive. Q.E.D. Let us now summarize the process of creation of new ISM equilibria, as the (M, N )-SONN cools down. For temperatures T > 1/2, the ISM has exactly one equilibrium - the maximum entropy point w of SN −1 [6]. As the temperature is lowered and hits the first bifurcation point, TE (N, 1), new fixed points of ISM w ∈ QN,1 , one on each line. The lines connect w emerge on the lines (τ ; w, w), of SN −1 . As the temperature decreases further, on each line, with the vertices w the single fixed point splits into two fixed points, one moves towards w, the other in QN,1 (vertex of SN −1 ). moves towards the corresponding high entropy point w When the temperature reaches the second bifurcation point, TE (N, 2), new fixed w ∈ QN,2 , one on each line. This points of ISM emerge on the lines (τ ; w, w), of the time the lines connect w with the maximum entropy points (midpoints) w edges of SN −1 . Again, as the temperature continues decreasing, on each line, the single fixed point splits into two fixed points, one moves towards w, the other in QN,2 (midpoint moves towards the corresponding maximum entropy point w of an edge of SN −1 ). The process continues until the last bifurcation temperature TE (N, N1 ) is reached, where N1 is the largest natural number smaller than N/2. At TE (N, N1 ), new fixed points of ISM emerge on the lines (τ ; w, w), ∈ QN,N1 , connecting w with maximum entropy points w of (N1 − 1)-faces w of SN −1 . As the temperature continues decreasing, on each line, the single fixed point splits into two fixed points, one moves towards w, the other moves towards in QN,N1 . At temperatures below the corresponding maximum entropy point w N −1 , only the fixed points moving towards the maximum entropy points of faces of SN −1 exist. In the low temperature regime, 0 < T < N −1 , a fixed point occurs on every w ∈ QN,N1 , N1 = N/2 , N/2 + 1, ..., N − 1. Here, x denotes line (τ ; w, w), the smallest integer y, such that y ≥ x. As the temperature decreases, the fixed ∈ QN,N1 points w move towards the corresponding maximum entropy points w of (N1 − 1)-faces of SN −1 . The process of creation of new fixed points and their flow as the temperature cools down is demonstrated in figure 2 for an ISM operating on 9-simplex S9 (N = 10). We plot against each temperature setting T the values of the larger coordinate γ1 > N −1 = 0.1 of the fixed points existing at T . The behavior of ISM in the neighborhood of its equilibrium w is given by the structure of stable and unstable manifolds of the linearized system at w outlined in the next section.

412

P. Tiˇ no Bifurcation structure of ISM equilibria (N=10) T (10,4)

T (10,2)

E

TE(10,1)

E

1 T (10,3) E

N =1 1

0.8

gamma

1

N =2 1

0.6

N =3 1

0.4

N =4 1

0.2

0

0

0.05

0.1

0.15

0.2

0.25

temperature T

Fig. 2. Demonstration of the process of creation of new ISM fixed points and their flow as the system temperature cools down. Here N = 10, i.e. the ISM operates on the standard 9-simplex S9 . Against each temperature setting T , the values of the larger coordinate γ1 > N −1 of the fixed points existing at T are plotted. The horizontal bold line corresponds to the maximum entropy point w = 10−1 110 .

4

Stability Analysis of Renormalization Equilibria

The maximum entropy point w is not only a fixed point of ISM (6), but also, regarded as a vector w − 0N , it is an eigenvector of the Jacobian J(w; T ) 0 at any w ∈ SN −1 , with eigenvalue λ = 0. This simply reflects the fact that ISM renormalization acts on the standard simplex SN −1 , which is a subset of a (N − 1)-dimensional hyperplane with normal vector 1N . We have already seen that w plays a special role in the ISM equilibria structure: all equilibria lie on lines going from w towards maximum entropy points of faces of SN −1 . The lines themselves are of special interest, since we will show that these lines are invariant manifolds of the ISM renormalization and their directional vectors are eigenvectors of ISM Jacobians at the fixed points located on them. Theorem 5. Consider ISM (6) and 1 ≤ N1 < N . Then for each maximum ∈ QN,N1 of an (N1 − 1)-face of SN −1 , the line (τ ; w, w), entropy point w is an invariant set τ ∈ [0, 1) connecting the maximum entropy point w with w under the ISM dynamics. Sketch of the Proof: into (3) and realizing The result follows from plugging parametrization (τ ; w, w) (after some manipulation) that for each τ ∈ [0, 1), there exists a parameter = (τ1 ; w, w). setting τ1 ∈ [0, 1) such that F((τ ; w, w)) Q.E.D.

Bifurcations of Renormalization Dynamics in SONNs

413

The proofs of the next two theorems are rather involved and we refer the interested reader to [7]. Theorem 6. Let w ∈ EN,N1 (γ1 ) be a fixed point of ISM (6). Then, w∗ = w − w is an eigenvector of the Jacobian J(w; TN,N1 (γ1 )) with the corresponding eigenvalue λ∗ , where 1. if N/2 ≤ N1 ≤ N − 1, then 0 < λ∗ < 1. 2. if 1 ≤ N1 < N/2 and N −1 < γ1 < (2N1 )−1 , then λ∗ > 1. 3. if 1 ≤ N1 < N/2 , then there exists γ¯1 ∈ ((2N1 )−1 , N1−1 ), such that for all ISM fixed points w ∈ EN,N1 (γ1 ) with γ1 ∈ (¯ γ1 , N1−1 ), 0 < λ∗ < 1. We have established that for an ISM equilibrium w, both w and w∗ = w− w are eigenvectors of the ISM Jacobian at w. Stability types of the remaining N − 2 eigendirections are characterized in the next theorem. Theorem 7. Consider an ISM fixed point w ∈ EN,N1 (γ1 ) for some 1 ≤ N1 < N and N −1 < γ1 < N1−1 . Then, there are N − N1 − 1 and N1 − 1 eigenvectors of Jacobian J(w; TN,N1 (γ1 )) of ISM at w with the same associated eigenvalue 0 < λ− < 1 and λ+ > 1, respectively.

5

Discussion – SONN Adaptation Dynamics

In the intermittent search regime by SONN [4], the search is driven by pulling promising solutions temporarily to the vicinity of the 0-1 “one-hot” assignment values - vertices of SN −1 (0-dimensional faces of the standard simplex SN −1 ). The critical temperature for intermittent search should correspond to the case where the attractive forces already exist in the form of attractive equilibria near the “one-hot” assignment suggestions (vertices of SN −1 ), but the convergence rates towards such equilibria should be sufficiently weak so that the intermittent character of the search is not destroyed. This occurs at temperatures lower than, but close to the first bifurcation temperature TE (N, 1) (for more details, see [7]). In [4] it is hypothesised that there is a strong link between the critical temperature for intermittent search by SONN and bifurcation temperatures of the autonomous ISM. In [6] we hypothesised (in accordance with [4]) that even though there are many potential ISM equilibria, the critical bifurcation points are related only to equilibria near the vertices of SN −1 , as only those could be guaranteed by the theory of [6] (stability bounds) to be stable, even though the theory did not prevent the other equilibria from being stable. In this study, we have rigorously shown that the stable equilibria can in fact exist only near the vertices of SN −1 , on the lines connecting w with the vertices. Only when N1 = 1, there are no expansive eigendirections of the local Jacobian with λ+ > 1. As the SONN system cools down, more and more ISM equilibria emerge on the lines connecting the maximum entropy point w of the standard simplex SN −1 with the maximum entropy points of its faces of increasing dimensionality. With decreasing temperature, the dimensionality of stable and unstable manifolds

414

P. Tiˇ no

of linearized ISM at emerging equilibria decreases and increases, respectively. At lower temperatures, this creates a peculiar pattern of saddle type equilibria surrounding the unstable maximum entropy point w, with decision enforcing “one-hot” stable equilibria located near vertices of SN −1 . Trajectory towards the solution as the SONN system anneals is shaped by the complex skeleton of saddle type equilibria with stable/unstable manifolds of varying dimensionalities and can therefore, in synergy with the input driving process, exhibit signatures of a very complex dynamical behavior, as reported e.g. in [5]. Once the temperature is sufficiently low, the attraction rates of stable equilibria near the vertices of SN −1 are so high that the found solution is virtually pinned down by the system. Even though the present study clarifies the prominent role of the first (symmetry breaking) bifurcation temperature TE (N, 1) in obtaining the SONN intermittent search regime and helps to understand the origin of complex SONN adaptation patterns in the annealing regime, many interesting open questions remain. For example, no theory as yet exists of the role of abstract neighborhood BL (i(jc )) of the winner node i(jc ) in the cooperative phase of SONN adaptation. We conclude by noting that it may be possible to apply the theory of ISM in other assignment optimization systems that incorporate the softmax assignment weight renormalization e.g. [8,9].

References 1. Smith, K., Palaniswami, M., Krishnamoorthy, M.: Neural techniques for combinatorial optimization with applications. IEEE Transactions on Neural Networks 9, 1301–1318 (1998) 2. Guerrero, F., Lozano, S., Smith, K., Canca, D., Kwok, T.: Manufacturing cell formation using a new self-organizing neural network. Computers & Industrial Engineering 42, 377–382 (2002) 3. Kwok, T., Smith, K.: Improving the optimisation properties of a self-organising neural network with weight normalisation. In: Proceedings of the ICSC Symposia on Intelligent Systems and Applications (ISA 2000), Paper No.1513-285 (2000) 4. Kwok, T., Smith, K.: Optimization via intermittency with a self-organizing neural network. Neural Computation 17, 2454–2481 (2005) 5. Kwok, T., Smith, K.: A noisy self-organizing neural network with bifurcation dynamics for combinatorial optimization. IEEE Transactions on Neural Networks 15, 84–88 (2004) 6. Tiˇ no, P.: Equilibria of iterative softmax and critical temperatures for intermittent search in self-organizing neural networks. Neural Computation 19, 1056–1081 (2007) 7. Tiˇ no, P.: Bifurcation structure of equilibria of adaptation dynamics in selforganizing neural networks. Technical Report CSRP-07-12, University of Birmingham, School of Computer Science (2007), http://www.cs.bham.ac.uk/∼ pxt/PAPERS/ism.bifurc.tr.pdf 8. Gold, S., Rangarajan, A.: Softmax to softassign: Neural network algorithms for combinatorial optimization. Journal of Artificial Neural Networks 2, 381–399 (1996) 9. Rangarajan, A.: Self-annealing and self-annihilation: unifying deterministic annealing and relaxation labeling. Pattern Recognition 33, 635–649 (2000)

Variable Selection for Multivariate Time Series Prediction with Neural Networks Min Han and Ru Wei School of Electronic and Information Engineering, Dalian University of Technology, Dalian 116023, China [email protected]

Abstract. This paper proposes a variable selection algorithm based on neural networks for multivariate time series prediction. Sensitivity analysis of the neural network error function with respect to the input is developed to quantify the saliency of each input variables. Then the input nodes with low sensitivity are pruned along with their connections, which represents to delete the corresponding redundant variables. The proposed algorithm is tested on both computergenerated time series and practical observations. Experiment results show that the algorithm proposed outperformed other variable selection method by achieving a more significant reduction in the training data size and higher prediction accuracy. Keywords: Variable selection, neural network pruning, sensitivity, multivariate prediction.

1 Introduction Nonlinear and chaotic time series prediction is a practical technique which can be used for studying the characteristics of complicated dynamics from measurements. Usually, multivariate variables are required since the output may depend not only on its own previous values but also on the past values of other variables. However, we can’t make sure that all of the variables are equally important. Some of them may be redundant or even irrelevant. If these unnecessary input variables are included into the prediction model, the parameter estimation process will be more difficult, and the overall results may be poorer than if only the required inputs are used [1]. Variable selection is such a problem to discard the redundant variables, which will reduce the number of input variables and the complexity of the prediction model. A number of variable selection methods based on statistical or heuristics tools have been proposed, such as Principal Component Analysis (PCA) and Discriminant Analysis. These techniques attempt to reduce the dimensionality of the data by creating new variables that are linear combinations of the original ones. The major difficulty comes from the separation of variable selection process and prediction process. Therefore, variable selection using neural network is attractive since one can globally adapt the variable selector together with the predictor. Variable selection with neural network can be seen as a special case of architecture pruning [2], where the pruning of input nodes is equivalent to removing the corresponding M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 415–425, 2008. © Springer-Verlag Berlin Heidelberg 2008

416

M. Han and R. Wei

variables from the original data set. One approach to pruning is to estimate the sensitivity of the output to the exclusion of each unit. There are several ways to perform sensitivity analysis with neural network. Most of them are weight-based [3], which is based on the idea that weights connected to important variables attain large absolute values while weights connected to unimportant variables would probably attain values somewhere near zero. However, smaller weights usually result in smaller inputs to neurons and larger sigmoid derivatives in general, which will increase the output sensitivity to the input. Mozer and Smolensky [4] have introduced a method which estimates which units are least important and can be deleted over training. Gevrey et al. [5] compute the partial derivatives of the neural network output with respect to the input neurons and compare performances of several different methods to evaluate the relative contribution of the input variables. This paper concentrates on a neural-network-based variable selection algorithm as the tool to determine which variables are to be discarded. A simple sensitivity criterion of the neural network error function with respect to each input is developed to quantify the saliency of each input variables. Then the input nodes are arrayed by a decreasing sensitivity order so that the neural network can be pruned efficiently by discarding the last items with low sensitivity. The variable selection algorithm is then applied to both computer-generated data and practical observations and is compared with the PCA variable reduction method. The rest of this paper is organized as follows. Section 2 reviews the basic concept of multivariate time series prediction and a statistical variable selection method. Section 3 explains the sensitivity analysis with neural networks in detail. Section 4 presents two simulation results. The work is finally concluded in section 5.

2 Modeling Multivariate Chaotic Time Series The basic idea of chaotic time series analysis is that, a complex system can be described by a strange attractor in its phase space. Therefore, the reconstruction of the equivalent state space is usually the first step in chaotic time series prediction. 2.1 Multivariate Phase Space Reconstruction Phase space reconstruction from observations can be accomplished by choosing a suitable embedding dimension and time delay. Given an M-dimensional time series{Xi, i=1, 2,…, M}, where Xi=[xi(1), xi(2), …, xi(N)]T, N is the length of each scalar time series. As in the case of univariate time series (where M=1), the reconstructed phase-space can be made as [6]:

X (t ) = [ x1 (t ), x1 (t − τ 1 ), " , x1 (t − (d1 − 1)τ 1 ), ," ,

(1)

xM (t ), xM (t − τ M )," , xM (t − (d M − 1)τ M ] where t = L, L + 1," , N , L = max(di − 1) ⋅τ i + 1 , τi and di ( i = 1, 2," , M ) are the time 1≤ i ≤ M

delays and embedding dimensions of each time series, respectively. The delay time τi can be calculated using mutual information method and the embedding dimension is computed with the false nearest neighbor method.

Variable Selection for Multivariate Time Series Prediction with Neural Networks

417

M

According to Takens’ embedding theorem, if D = ∑ di is large enough there exist i =1

an mapping F: X(t+1)=F{X(t)}. Then the evolvement of X(t)→X(t+1) reflects the evolvement of the original dynamics system. The problem is then to find an appropriate expression for the nonlinear mapping F. Up to the present, many chaotic time series prediction models have been developed. Neural network has been widely used because of its universal approximation capabilities. 2.2 Neural Network Model

A multilayer perceptron (MLP) with a back propagation (BP) algorithm is used as a nonlinear predictor for multivariate chaotic time series prediction. MLP is a supervised learning algorithm designed to minimize the mean square error between the computed output of the neural network and the desired output. The network usually consists of three layers: an input layer, one or more hidden layers and an output layer. Consider a three layer MLP that contains one hidden layer. The D dimensional delayed time series X(t) are used as the input of the network to generate the network output X(t+1). Then the neural network can be expressed as follows: NI

o j = f (∑ xi wij(I) )

(2)

i =0

NH

yk = ∑ w(O) jk o j

(3)

j =1

where [ x1 , x2 ," , xNI ] = X (t ) denotes the input signal, N I is number of input signal to the neural network, wij(I) is the weight connected from the ith input neuron to the jth hidden neuron, oj are the output of the jth hidden neuron, N H is the number of neurons in the hidden layer, [ y1 , y2 ," , y NO ] = X (t + 1) is the output, N O is the number of output neurons and w(O) is the weight connected from the jth hidden neuron jk and the kth output neuron. The activation function f(·) is the sigmoid function given by

f ( x) =

1 1 + exp(− x)

(4)

The error function of the net is usually defined as the sum square of the error N

No

E = ∑∑ [ yk (t ) − pk (t )]2 , t=1,2,…N t =1 k =1

where pk(t) is the desired output for unit k, N is the length of the training sample.

(5)

418

M. Han and R. Wei

2.3 Statistical Variable Selection Method

For the multivariate time series, the dimension of the reconstructed phase space is usually very high. Moreover, the increase of the input variable numbers will lead to the high complexity of the prediction model. Therefore, in many practical applications, variable selection is needed to reduce the dimensionality of the input data. The aim of variable selection in this paper is to select a subset of R inputs that retains most of the important features of the original input sets. Thus, D-R irrelevant inputs are discarded. The Principle Component Analysis (PCA) is a traditional technique for variable selection [7]. PCA attempts to reduce the dimensionality by first decomposing the normalize input vector X(t) d with singular value decomposition (SVD) method X = U ∑V T

(6)

where ∑ = diag[ s1 s2 ... s p 0 ... 0] , s1 ≥ s2 " ≥ s p are the first p eigenvalues of X ar-

rayed by a decreasing order, U and V are both orthogonal matrixes. Then the first k singular values are preserved as the principle components. The final input can be obtained as

Z = U T X

(7)

where U is the first k rows of U. PCA is an efficient method to reduce the input dimension. However, we can’t make sure that the factors we discard have no influence to the prediction output because the variable selection and prediction process are separated individually. Neural network selector is a good choice to combine the selection process and prediction process.

3 Sensitivity Analysis with Neural Networks Variable selection with neural networks can be achieved by pruning the input nodes of a neural network model based on some saliency measure aiming to remove less relevant variables. The significance of a variable can be defined as the error when the unit is removed minus the error when it is left in place:

Si = EWithoutUnit _ i − EWithUnit _ i = E ( xi = 0) − E ( xi = xi )

(8)

where E is the error function defined in Eq.(5). After the neural network has been trained, a brute-force pruning method for ever input is setting the input to zero and evaluate the change in the error. If it increases too much, the input is restored, otherwise it is removed. Theoretically, this can be done by training the network under all possible subsets of the input set. However, this exhaustive search is computational infeasible and can be very slow for large network. This paper uses the same idea with Mozer and Smolensky [4] to approximate the sensitivity by introducing a gating term α i for each unit such that

o j = f (∑ wijα i oi ) i

where o j is the activity of unit j, wij is the weight from unit i to unit j.

(9)

Variable Selection for Multivariate Time Series Prediction with Neural Networks

419

The gating term α is shown in Fig.1, where α iI , i = 1, 2," , N I is the gating term of the ith input neuron and α Hj , j = 1, 2," , N H is the gating term of the jth output neuron.

α1I

x1 xi

xN I

α1H

#

#

#

α NI

α

yk H NH

I

Fig. 1. The gating term for each unit

The gating term α is merely a notational convenience rather than a parameter that must be implied in the net. If α = 0 , the unit has no influence on the network; If α = 1 , the unit behaves normally. The importance of a unit is then approximated by the derivative Si = −

∂E ∂α i

(10) α i =1

By using a standard error back-propagation algorithm, the derivative of Eq.(9) can be expressed in term of network weights as follows NH N NO ⎡ ⎤ ∂E ∂E ∂yk (O ) = − ⋅ = ∑∑ ⎢( pk (t ) − yk (t ) ) ∑ w jk o j ⎥ H H ∂α j ∂yk ∂α j t =1 k =1 ⎣ j =1 ⎦

(11)

NH N NO ⎡ ⎤ ∂E ∂E ∂yk = − ⋅ = p ( t ) − y ( t ) w(jkO ) o j (1 − o j ) wij( I ) xi (t ) ⎥ ( ) ∑∑ ⎢ k ∑ k I I ∂α i ∂yk ∂α i t =1 k =1 ⎣ j =1 ⎦

(12)

S Hj = −

SiI = −

where SiI is the sensitivity of the ith input neuron, S Hj is the sensitivity of the jth output neuron. Thus the algorithm can prune the input nodes as well as the hidden nodes according to the sensitivity over training. However, the undulation is high when the sensitivity is calculated directly using Eq.(11) and Eq.(12) because of the engineering approximation in Eq.(10). Sometimes, it may delete the input incorrectly. In order to possibly reduce the dimensionality of input vectors, the sensitivity matrix needs to be evaluated over the entire training set. This paper develops several ways to define the overall sensitivity such as: (1) The mean square average sensitivity:

Si , avg =

1 N ∑ Si (t )2 T t =1

where T is the number of data in the training set.

(13)

420

M. Han and R. Wei

(2) The absolute value average sensitivity:

Si , abs =

1 N ∑ Si (t ) T t =1

(14)

(3) The maximum absolute sensitivity:

Si ,max = max Si (t ) 1≤ t ≤ N

(15)

Any of the sensitivity measure in Eqs.(13)~(15) can provide a useful criterion to determine which input is to be deleted. For succinctness, this paper uses the mean square average sensitivity as an example. An input with a low sensitivity has little or no influence on the prediction accuracy and can therefore be removed. In order to get a more efficient criterion for pruning inputs, the sensitivity is normalized. Define the absolute sum of the sensitivity for all the input nodes NI

S = ∑ Si

(16)

i =1

Then the normalized sensitivity of each unit can be defined as S Sˆi = i S

(17)

where the normalized value Sˆi is between [0 1]. The input variables is then arrayed by a decreasing sensitivity order: Sˆ1 ≥ Sˆ2 ≥ " ≥ SˆN I

(18)

The larger values of Sˆi (t ), i = 1, 2," , N I present the important variables. Define the sum the first k terms of the sensitivity ηk as k

η k = ∑ Sˆ j

(19)

j =1

where k=1,2,…,NI . Choosing a threshold value 0 < η 0 < 1 , if ηk > η0 , the first k values are preserved as the principal components and the last term of the inputs with low sensitivity are removed. The number of variable remained is increasing as the threshold η0 increase.

4 Simulations In this section, two simulations are carried out on both computer-generated data and practical observed data to demonstrate the performance of the variable selection method proposed in this paper. Then the simulation results are compared with the

Variable Selection for Multivariate Time Series Prediction with Neural Networks

421

PCA method. The prediction performance can be evaluated by two error evaluation criteria [8]: the Root Mean Square Error ERMSE and Prediction Accuracy EPA: 1

ERMSE

⎛ 1 N 2 ⎞2 =⎜ [ P(t ) − O(t )] ⎟ ∑ ⎝ N − 1 t =1 ⎠

(20)

N

EPA =

∑ [( P(t ) − P )(O(t ) − O )] t =1

m

m

(21)

( N − 1)σ Pσ O

where O(t) is the target value, P(t) is the predicted value, Om is the mean value of O(t), σO is the standard deviation of y(t), Pm and σP are the mean value and standard deviation of P(t), respectively. ERMSE reflects the absolute deviation between the predicted value and the observed value while EPA denotes the correlation coefficient between the observed and predicted value. In ideal situation, if there are no errors in prediction, these parameters will be ERMSE =0 and EPA =1. 4.1 Prediction of Lorenz Time Series

The first data is derived from the Lorenz system, given by three differential equations: ⎧ dx(t ) ⎪ dt = a ( − x(t ) + y (t ) ) ⎪ ⎪ d y (t ) = bx(t ) − y (t ) − x(t ) z (t ) ⎨ ⎪ dt ⎪ d z (t ) ⎪ dt = x(t ) y (t ) − c(t ) z (t ) ⎩

(22)

where the typical values for the coefficients are a=10, b=8/3, c=28 and the initial values are x(0)=12, y(0)=2, z(0)=9. 1500 points of x(t), y(t) and z(t) obtained by fourorder Runge-Kutta method are used as the training sample and 500 points as the testing sample. In order to extract the dynamics of this system to predict x(t+1), the parameters for phase-space reconstruction are chosen as τx=τy=τz=3, mx=my=mz=9. Thus a MLP neural network with 27 input nodes, one hidden layer of 20 neurons and one output node are considered and a back propagation training algorithm is used. After the training process of the MLP neural network is topped, sensitivity analysis is carried out to evaluate the contribution of each input variable to the error function of the neural network. The trajectories of the sensitivity through training for each input are shown in Fig.2. It can be seen that the sensitivity undulates through training and finally converges when the weights and error are steady. The normalized sensitivity measures in Eq.(17) are calculated. A threshold η0 = 0.98 is chosen to determine which inputs are discarded. Thus the input dimension of neural network is reduced to 11. The original input matrix is replaced by the reduced input matrix and the structure of the neural networks is simplified. The prediction performance over the testing samples with the reduced inputs is shown in Fig.4.

422

M. Han and R. Wei

8

0.7

7

0.6

6

0.5

5

0.4

4

0.3

3 0.2

2

0.1

1 0

0 0

2000

4000

6000 Epoch

8000

10000

0

5

10

15 20 Input Nodes

25

30

Fig. 2. The trajectories of the input sensitivityFig. 3. The normalized sensitivity for each input through training node

x(t)

20 10

Ovserved

Predicted

0

-10

Error

-20 1 0.5 0

-0.5 -1

0

100

200

Time

300

400

500

Fig. 4. The observed and predicted values of Lorenz x(t) time series

The solid line in Fig. 4 represents the observed values while the dashed line represents the predicted values. It can be seen from Fig.4 that the chaotic behaviors of x(t) time series are well predicted and the errors between the observed values and the predicted values are small. The prediction performance are calculated in Table 1 and compared with the PCA variable reduction method. Table 1. Prediction performance of the x(t) time series

Input Nodes ERMSE EPA

With All Variables 27 0.1278 0.9998

PCA Selection 11 0.1979 0.9997

NN Selection 11 0.0630 1.0000

The prediction performance in Table 1 are comparable for the variable selection method with neural networks and the PCA method while the algorithm proposed in this paper obtains the best prediction accuracy.

Variable Selection for Multivariate Time Series Prediction with Neural Networks

423

4.2 Prediction of the Rainfall Time Series

Rainfall is an important variable in hydrological systems. The chaotic characteristic of the rainfall time series has been proven in many papers [9]. In this section, the simulation is taken on the monthly rainfall time series in the city of Dalian, China over a period of 660 months (from 1951 to 2005). The performance of the rainfall may be influenced by many factors, so in this paper five other time series such as the temperature time series, air-pressure time series, humidity time series, wind-speed time series and sunlight time series are also considered. This method also follows the Taken’s theorem to reconstruct the embedding phase space first with the dimension and delay-time as m1=m2=m3=m4=m5=m6=9, τ1=τ2=τ3 =τ4=τ5=τ6=3. Then the input of the neural network contains L=660-(9-1)×3=636 data points. In the experiments, this data set is divided into a training set composed of the first 436 points and a testing set containing the remaining 200 points. The neural network used in this paper then contains 54 input nodes, 20 hidden notes and 1 output. The threshold is also chosen as η0 = 0.98 . The trajectory of the input sensitivity and the normalized sensitivity for ever inputs are shown in Fig.5 and Fig.6, respectively. Then 34 input nodes are remained according to the sensitivity value. 6

0.2

Normalized Sensitivity

5 4 3 2

0.12 0.08 0.04

1 0

0.16

0

2000

4000 6000 Epoch

8000

10000

Fig. 5. The trajectories of the input sensitivity through training

0

0

10

20

30 40 Input Nodes

50

60

Fig. 6. The normalized sensitivity for each input node

The observed and predicted values of rainfall time series are shown in Fig.7, which gives high prediction accuracy. It can be seen from the figures that the chaotic behaviors of the rainfall time series are well predicted and the errors between the observed values and the predicted values are small. Corresponding values of ERMSE and EPA are shown in Table 2. Both of the figures and the error evaluation criteria indicate that the result for multivariate chaotic time series using the neural network based variable selection is much better than the results with all variables and PCA method. It can be concluded from the two simulations that the variable selection algorithm using neural networks is able to capture the dynamics of both computer-generated and practical time series accurately and gives high prediction accuracy.

424

M. Han and R. Wei

rainfall(mm)

400

Observed Predicted

300 200 100

error(mm)

0 200 100 0

-100 -200

0

40

80 120 t (month)

160

200

Fig. 7. The observed and predicted values of rainfall time series Table 2. Prediction performance of the rainfall time series

Input Nodes ERMSE EPA

With All Variables 54 22.2189 0.9217

PCA Selection 43 21.0756 0.9286

NN Selection 31 18.1435 0.9529

5 Conclusions This paper studies the variable selection algorithm using the sensitivity for pruning input nodes in a neural network model. A simple and effective criterion for identifying input nodes to be removed is also derived which does not require high computational cost and proves to work well in practice. The validity of the method was examined through a multivariate prediction problem and a comparison study was made with other variable selection methods. Experimental results encourage the application of the proposed method to complex tasks that need to identify significant input variables. Acknowledgements. This research is supported by the project (60674073) of the National Nature Science Foundation of China, the project (2006CB403405) of the National Basic Research Program of China (973 Program) and the project (2006BAB14B05) of the National Key Technology R&D Program of China. All of these supports are appreciated.

References [1] Verikas, B.M.: Feature selection with neural networks. Pattern Recognition Letters 23, 1323–1335 (2002) [2] Castellano, G., Fanelli, A.M.: Variable selection using neural network models. Neuralcomputing 31, 1–13 (2000)

Variable Selection for Multivariate Time Series Prediction with Neural Networks

425

[3] Castellano, G., Fanelli, A.M., Pelillo, M.: An iterative method for pruning feed-forward neural networks. IEEE Trans. Neural Networks 8(3), 519–531 (1997) [4] Mozer, M.C., Smolensky, P.: Skeletonization: a technique for trimming the fat from a network via a relevance assessment. NIPS 1, 107–115 (1989) [5] Gevrey, M., Dimopoulos, I., Lek, S.: Review and comparison of methods to study the contribution of variables in artificial neural network models. Ecol. Model. 160, 249–264 (2003) [6] Cao, L.Y., Mees, A., Judd, K.: Dynamics from multivariate time series. Physica D 121, 75–88 (1998) [7] Han, M., Fan, M., Xi, J.: Study of Nonlinear Multivariate Time Series Prediction Based on Neural Networks. In: Wang, J., Liao, X.-F., Yi, Z. (eds.) ISNN 2005. LNCS, vol. 3497, pp. 618–623. Springer, Heidelberg (2005) [8] Chen, J.L., Islam, S., Biswas, P.: Nonlinear dynamics of hourly ozone concentrations: nonparametric short term prediction. Atmospheric environment 32(11), 1839–1848 (1998) [9] Liu, D.L., Scott, B.J.: Estimation of solar radiation in Australia from rainfall and temperature observations. Agricultural and Forest Meteorology 106(1), 41–59 (2001)

Ordering Process of Self-Organizing Maps Improved by Asymmetric Neighborhood Function Takaaki Aoki1 , Kaiichiro Ota2 , Koji Kurata3 , and Toshio Aoyagi1,2 1 CREST, JST, Kyoto 606-8501, Japan Graduate School of Informatics, Kyoto University, Kyoto 606-8501, Japan Faculty of Engineering, University of the Ryukyus, Okinawa 903-0213, Japan [email protected]

2 3

Abstract. The Self-Organizing Map (SOM) is an unsupervised learning method based on the neural computation, which has recently found wide applications. However, the learning process sometime takes multi-stable states, within which the map is trapped to a undesirable disordered state including topological defects on the map. These topological defects critically aggravate the performance of the SOM. In order to overcome this problem, we propose to introduce an asymmetric neighborhood function for the SOM algorithm. Compared with the conventional symmetric one, the asymmetric neighborhood function accelerates the ordering process even in the presence of the defect. However, this asymmetry tends to generate a distorted map. This can be suppressed by an improved method of the asymmetric neighborhood function. In the case of one-dimensional SOM, it found that the required steps for perfect ordering is numerically shown to be reduced from O(N 3 ) to O(N 2 ). Keywords: Self-Organizing Map, Asymmetric Neighborhood Function, Fast ordering.

1

Introduction

The Self-Organizing Map (SOM) is an unsupervised learning method of a type of nonlinear principal component analysis [1]. Historically, it was proposed as a simplified neural network model having some essential properties to reproduce topographic representations observed in the brain [2,3,4,5]. The SOM algorithm can be used to construct an ordered mapping from input stimulus data onto two-dimensional array of neurons according to the topological relationships between various characters of the stimulus. This implies that the SOM algorithm is capable of extracting the essential information from complicated data. From the viewpoint of applied information processing, the SOM algorithm can be regarded as a generalized, nonlinear type of principal component analysis and has proven valuable in the fields of visualization, compression and data mining. With based on the biological simple learning rule, this algorithm behaves as an unsupervised M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 426–435, 2008. c Springer-Verlag Berlin Heidelberg 2008 

A

B

1

0.5

0 0

0.5

1

Reference Vector mi

Ordering Process of SOMs Improved by Asymmetric Neighborhood Function

427

1 0.5 0 0

N Unit Number i

Fig. 1. A: An example of a topological defect in a two-dimensional array of SOM with a uniform rectangular input space. The triangle point indicates the conflicting point in the feature map. B: Another example of topological defect in a one-dimensional array with scalar input data. The triangle points also indicate the conflicting points.

learning method and provides a robust performance without a delicate tuning of learning conditions. However, there is a serious problem of multi-stability or meta-stability in the learning process [6,7,8]. When the learning process is trapped to these states, the map seems to be converged to the final state practically. However, some of theses states are undesirable for the solution of the learning procedure, in which typically the map has topological defects as shown in Fig. 1A. The map in Fig. 1A, is twisted with a topological defect at the center. In this situation, two-dimensional array of SOM should be arranged in the square space, for the input data taken uniformly from square space. But, this topological defect is a global conflicting point which is difficult to remove by local modulations of the reference vectors of units. Therefore, it will require a sheer number of learning steps to rectify the topological defect. Thus, the existence of the topological defect critically aggravates the performance of the SOM algorithm. To avoid the emergence of the topological defect, several conventional and empirical methods have been used. However, it is more favorable that the SOM algorithm works well without tuning any model parameters, even when the topological defect emerged. Thus, let us consider a simple method which enables the effective ordering procedure of SOM in the presence of the topological defect. Therefore we propose an asymmetric neighborhood function which effectively removes the topological defect [9]. In the process of removing the topological defect, the conflicting point must be moved out toward the boundary of the arrays and vanished. Therefore, the motive process of the defect is essential for the efficiency of the ordering process. With the original symmetric neighborhood, the movement of the defect is similar to a random walk stochastic process, whose efficiency is worse. By introducing the asymmetry of the neighborhood function, the movement behaves like a drift, which enables the faster ordering. For this reason, in this paper we investigate the effect of an asymmetric neighborhood function on the performance of the SOM algorithm for the case of one-dimensional and two-dimensional SOMs.

428

2 2.1

T. Aoki et al.

Methods SOM

The SOM constructs a mapping from the input data space to the array of nodes, we call the ‘feature map’. To each node i, a parametric ‘reference vector’ mi is assigned. Through SOM learning, these reference vectors are rearranged according to the following iterative procedure. An input vector x(t) is presented at each time step t, and the best matching unit whose reference vector is closest to the given input vector x(t) is chosen. The best matching unit c, called the ‘winner’ is given by c = arg mini x(t) − mi . In other words, the data x(t) in the input data space is mapped on to the node c associated with the reference vector mi closest to x(t). In SOM learning, the update rule for reference vectors is given by mi (t + 1) = mi (t) + α · h(ric )[x(t) − mi (t)],

ric ≡ ric  ≡ ri − rc 

(1)

where α, the learning rate, is some small constant. The function h(r) is called the ‘neighborhood function’, in which ric is the distance from the position rc of the winner node c to the position ri of a node i on the array of units. A widely  used  r2 neighborhood function is the Gaussian function defined by, h(ric ) = exp − 2σic2 . We expect an ordered mapping after iterating the above procedure a sufficient number of times. 2.2

Asymmetric Neighborhood Function

We now introduce a method to transform any given symmetric neighborhood function to an asymmetric one (Fig. 2A). Let us define an asymmetry parameter β (β ≥ 1), representing the degree of asymmetry and the unit vector k indicating the direction of asymmetry. If a unit i is located on the positive direction with k, then the component parallel to k of the distance from the winner to the unit is scaled by 1/β. If a unit i is located on the negative direction with k, the parallel component of the distance is scaled by β. Hence, the asymmetric function hβ (r), transformed from its symmetric counterpart h(r), is described by ⎧   ⎪  −1 r 2 ⎨ + r⊥ 2 (ric · k ≥ 0) 1 hβ (ric ) = 2 +β · h(˜ ric ), r˜ic =  β , ⎪ β ⎩ βr 2 + r 2 (r · k < 0) ⊥ ic  (2) where r˜ic is the scaled distance from the winner. r is the projected component of ric , and r⊥ are the remaining components perpendicular to k, respectively. In addition, in order to single  ∞out the effect of asymmetry, the overall area of the neighborhood function, −∞ h(r)dr, is preserved under this transformation. In the special case of the asymmetry parameter β = 1, hβ (r) is equal to the original symmetric function h(r). Figure 2B displays an example of asymmetric Gaussian neighborhood functions in the two-dimensional array of SOM.

Ordering Process of SOMs Improved by Asymmetric Neighborhood Function

A

Oriented in the positive direction c

Oriented in the negative direction β

1/β

c

k

k

B

C

Inverse the direction 1.0

hβ(r)

k

1.0

Interval: T 0.5

1 0.5

0.5

distance

0 -6 -3

429

distance

Reduce the degree of asymmetric

0

3

6-6

-3

0

3

6

1.0

1.0

1.0

0.5

0.5

0.5

b=3

b=2

b=1

Fig. 2. A: Method of generating an asymmetric neighborhood function by scaling the distance ric asymmetrically.The degree of asymmetry is parameterized by β. The distance of the node on the positive direction with asymmetric unit vector k, is scaled by 1/β. The distance on the negative direction is scaled by β. Therefore, the asymmetric 2 function is described by hβ (ric ) = β+1/β h(˜ ric ) where r˜ic is the scaled distance of node i from the winner c. B: An example of an asymmetric Gaussian function. C: An illustration of the improved algorithm for asymmetric neighborhood function.

Next, we introduce an improved algorithm for the asymmetric neighborhood function. The asymmetry of the neighborhood function causes to distort the feature map, in which the density of units does not represent the probability of the input data. Therefore, two novel procedures will be introduced. The first procedure is an inversion operation on the direction of the asymmetric neighborhood function. As illustrated in Fig. 2C, the direction of the asymmetry is turned in the opposite direction after every time interval T , which is expected to average out the distortion in the feature map. It is noted that the interval T should be set to a larger value than the typical ordering time for the asymmetric neighborhood function. The second procedure is an operation that decreases the degree of asymmetry of the neighborhood function. When β = 1, the neighborhood function equals the original symmetric function. With this operation, β is decreased to 1 with each time step, as illustrated in Fig. 2C. In our numerical simulations, we adopt a linear decreasing function. 2.3

Numerical Simulations

In the following sections, we tested learning procedures of SOM with sample data to examine the performance of the ordering process. The sample data is

430

T. Aoki et al.

generated from a random variable of a uniform distribution. In the case of onedimensional SOM, the distribution is uniform in the ranges of [0, 1]. Here we use Gaussian function for the original symmetric neighborhood function. The model parameters in SOM learning was used as follows: The total number of units N = 1000, the learning rate α = 0.05 (constant), and the neighborhood radius σ = 50. The asymmetry parameter β = 1.5 and asymmetric direction k is set to the positive direction in the array. The interval period T of flipping the asymmetric direction is 3000. In the case of two-dimensional SOM ( 2D → 2D map), the input data taken uniformly from square space, [0, 1] × [0, 1]. The model parameters are same as in one-dimensional SOM, excepted that The total number of units N = 900 (30 × 30) and σ = 5. The asymmetric direction k is taken in the direction (1, 0), which can be determined arbitrary. In the following numerical simulations, we also confirmed the result holds with other model parameters and other form of neighborhood functions. 2.4

Topological Order and Distortion of the Feature Map

For the aim to examine the ordering process of SOM, lets us consider two measures which characterize the property of the future map. One is the ‘topological order’ η for quantifying the order of reference vectors in the SOM array. The units of the SOM array should be arranged according to its reference vector mi . In the presence of the topological defect, most of the units satisfy the local ordering. However, the topological defect violates the global ordering and the feature map is divided into fragments of ordering domains within which the units satisfy the order-condition. Therefore, the topological order η can be defined as the ratio of the maximum domain size to the total number of units N , given by maxl Nl , (3) N where Nl is the size of domain l. In the case of one-dimensional SOM, the order-condition for the reference vector of units is defined as, mi−1 ≤ mi ≤ mi+1 , or mi−1 ≥ mi ≥ mi+1 . In the case of two-dimensional SOM as referred in the previous section, the order-condition is also defined explicitly with the vector product a(i,i) ≡ (m(i+1,i) −m(i,i) )×(m(i,i+1) −m(i,i) ). Within the ordering domain, the vector products a(i,i) of units have same sign, because the reference vectors are arranged in the sample space ordered by the position of the unit. The other is the ‘distortion’ χ, which measures the distortion of the feature map. The asymmetry of the neighborhood function tends to distort the distribution of reference vectors, which is quite different from the correct probability density of input vectors. For example, when the probability density of input vectors is uniform, a non-uniform distribution of reference vectors is formed with an asymmetric neighborhood function. Hence, for measuring the non-uniformity in the distribution of reference vectors, let us define the distortion χ. χ is a coefficient of variation of the size-distribution of unit Voronoi tessellation cells, and is given by  Var(Δi ) χ= , (4) E(Δi ) η≡

Ordering Process of SOMs Improved by Asymmetric Neighborhood Function

431

where Δi is the size of Voronoi cell of unit i. To eliminate the boundary effect of the SOM algorithm, the Voronoi cells on the edges of the array are excluded. When the reference vectors are distributed uniformly, the distortion χ converges to 0. It should be noted that the evaluation of Voronoi cell in two-dimensional SOM is time-consuming, and we approximate the size of Voronoi cell by the size of the vector product a(i,i) . If the feature map is uniformly formed, the approximate value also converges to 0.

3 3.1

Results One-Dimensional Case

In this section, we investigate the ordering process of SOM learning in the presence of a topological defect in symmetric, asymmetric, and improved asymmetric cases of the neighborhood function. For this purpose, we use the initial condition that a single topological defect appears at the center of the array. Because the density of input vectors is uniform, the desirable feature map is a linear arrangement of SOM nodes. Figure 3A shows a typical time development of the reference vectors mi . In the case of the symmetric neighborhood function, a single defect remains around the center of the array even after 10000 steps. In contrast, in the case of the asymmetric one, this defect moves out to the right so that the reference vectors are ordered within 3000 steps. This phenomenon can also be confirmed in Fig. 3B, which shows the time dependence of the topological order η. In the case of the asymmetric neighborhood function, η rapidly converges to 1 (completely ordered state) within 3000 steps, whereas the process of eliminating the last defect takes a large amount of time (∼18000 step) for the symmetric one. On the other hand, one problem arises in the feature map obtained with the asymmetric neighborhood function. After 10000 steps, the distribution of the reference vectors in the feature map develops an unusual bias (Fig. 3A). Figure 3C shows the time dependence of the distortion χ during learning. In the case of the symmetric neighborhood function, χ eventually converges to almost 0. This result indicates that the feature map obtained with the symmetric one has an almost uniform size distribution of Voronoi cells. In contrast, in the case of the asymmetric one, χ converges to a finite value (= 0). Although the asymmetric neighborhood function accelerates the ordering process of SOM learning, the resultant map becomes distorted which is unusable for the applications. Therefore, the improved asymmetric method will be introduced, as mentioned in Method. Using this improved algorithm, χ converges to almost 0 as same as the symmetric one (Fig. 3C). Furthermore, as shown in Fig. 3B, the improved algorithm preserves the faster order learning. Therefore, by utilizing the improved algorithm of asymmetric neighborhood function, we confer the full benefit of both the fast order learning and the undistorted feature map. To quantify the performance of the ordering process, let us define the ‘ordering time’ as the time at which η reaches to 1. Figure 4A shows the ordering

432

T. Aoki et al.

A

Symmetric 1

1

0

1

0 0

1000

1

0 0

1000

0 0

1000

0

1000

0

1000

Asymmetric 1

1

0

1

0 0

1000

1

0 0

1000

1

0 0

1000

0 0

1000

Improved asymmetric 1

1

0

0 0 1000 t=1000

t=0

1 0 0 1000 t=5000

B

Distortion χ

1

0.5

0

0 0 1000 t=10000

0 1000 t=20000

Symmetric Asymmetric Improved asym.

C

Topological order η

1

1.5 1 0.5 0

0

10000 Time

20000

0

10000 Time

20000

Fig. 3. The asymmetric neighborhood function enhances the ordering process of SOM. A: A typical time development of the reference vectors mi in cases of symmetric, asymmetric, and improved asymmetric neighborhood functions. B: The time dependence of the topological order η. The standard deviations are denoted by the error bars, which cannot be seen because they are smaller than the size of the graphed symbol. C: The time dependence of the distortion χ.

time as a function of the total number of units N for both improved asymmetric and symmetric cases of the neighborhood function. It is found that the ordering time scales roughly as N 3 and N 2 for symmetric and improved asymmetric neighborhood functions, respectively. For detailed discussion about the reduction of the ordering time, refer to the Aoki & Aoyagi (2007). Figure 4B shows the dependency of the ordering time on the width of neighborhood function, which indicates that ordering time is proportional to (N/σ)k with k = 2/3 for asymmetric/symmetric neighborhood function. This result implies that combined usage of the asymmetric method and annealing method for the width of neighborhood function is more effective. 3.2

Two-Dimensional Case

In this section, we investigate the effect of asymmetric neighborhood function for two-dimensional SOM (2D → 2D map). Figure 5 shows that a similar fast

Ordering Process of SOMs Improved by Asymmetric Neighborhood Function

Ordering time

108 10

7

10

6

10

5

N2.989±0.002

10

7

10

6

Sym. σ = 8 Sym. σ = 16 Sym. σ = 32 Sym. σ = 64 Asym.σ = 8 Asym.σ = 16 Asym.σ = 32 Asym.σ = 64

N3

105

104 10

108

Symmetric Improved asym. Fitting

104

N1.917±0.005

3

103

10

104

The number of units N

433

N2

3

102

Scaled number of units N/σ

Fig. 4. Ordering time as a function of the total number of units N . The fitting function is described by Const. · N γ .

A

Symmetric 1

1

0

1

0 0

0

1

0

1

0

1

0

1

Asymmetric 1

1

0

1

0 0

1

1

0 0

1

0 0

1

Improved asymmetric 1

1

0

0 0

t=0

1

0 0

t=1000

B

1 t=5000

1

0.5

0

0 1 t=20000

Symmetric Asymmetric Improved asym.

C Distortion χ

Topological order η

1

1

0.5

0 0

10000 Time

20000

0

10000 Time

20000

Fig. 5. A: A typical time development of reference vectors in two-dimensional array of SOM for the cases of symmetric, asymmetric and improved asymmetric neighborhood functions. B: The time dependence of the topological order η. C: The time dependence of the distortion χ.

434

T. Aoki et al.

Population

Symmetric

Improved asymmetric

0.5

0.5

0

0 0

25000

50000

Ordering time

0

25000

50000

Ordering time

Fig. 6. Distribution of ordering times when the initial reference vectors are generated randomly. The white bin at the right in the graph indicates a population of failed trails which could not converged to the perfect ordering state within 50000 steps.

ordering process can be realized with an asymmetric neighborhood function in two-dimensional SOM. The initial state has a global topological defect, in which the map is twisted at the center. In this situation, the conventional symmetric neighborhood function has trouble in correcting the twisted map. Because of the local stability, this topological defect is never corrected even with a huge learning iteration. The asymmetric neighborhood function also is effective to overcome such a topological defect, like the case of one-dimensional SOM. However, the same problem of ’distortion map’ occurs. Therefore, by using the improved asymmetric neighborhood function, the feature map converges to the completely ordered map in much less time without any distortion. In the previous simulations, we have considered a simple situation that a single defect exists around the center of the feature map as an initial condition in order to investigate the ordering process with the topological defect. However, when the initial reference vectors are set randomly, the total number of topological defects appearing in the map is not generally equal to one. Therefore, it is necessary to consider the statistical distribution of the ordering time, because the total number of the topological defects and the convergence process depend generally on the initial conditions. Figure 6 shows the distribution of the ordering time, when the initial reference vectors are randomly selected from the uniform distribution [0, 1] × [0, 1]. In the case of symmetric neighborhood function, a part of trial could not converged to the ordered state with trapped in the undesirable meta-stable states, in which the topological defects are never rectified. Therefore, although the fast ordering process is observed in some successful cases (lucky initial conditions), the formed map with symmetric one is highly depends on the initial conditions. In contrast, for the improved asymmetric neighborhood function, the distribution of the ordering time has a single sharp peak and the successive future map is constructed stably without any tuning of the initial condition.

4

Conclusion

In this paper, we discussed the learning process of the self-organized map, especially in the presence of a topological defect. Interestingly, even in the presence

Ordering Process of SOMs Improved by Asymmetric Neighborhood Function

435

of the defect, we found that the asymmetry of the neighborhood function enables the system to accelerate the learning process. Compared with the conventional symmetric one, the convergence time of the learning process can be roughly reduced from O(N 3 ) to O(N 2 ) in one-dimensional SOM(N is the total number of units). Furthermore, this acceleration with the asymmetric neighborhood function is also effective in the case of two-dimensional SOM (2D → 2D map). In contrast, the conventional symmetric one can not rectify the twisted feature map even with a sheer of iteration steps due to its local stability. These results suggest that the proposed method can be effective for more general case of SOM, which is the subject of future study.

Acknowledgments This work was supported by Grant-in-Aid for Scientific Research from the Ministry of Education, Science, Sports, and Culture of Japan: Grant number 18047014, 18019019 and 18300079.

References 1. Kohonen, T.: Self-organized formation of topologically correct feature maps. Biol. Cybern. 43(1), 59–69 (1982) 2. Hubel, D.H., Wiesel, T.N.: Receptive fields, binocular interaction and functional architecture in cats visual cortex. J. Physiol.-London 160(1), 106–154 (1962) 3. Hubel, D.H., Wiesel, T.N.: Sequence regularity and geometry of orientation columns in monkey striate cortex. J. Comp. Neurol. 158(3), 267–294 (1974) 4. von der Malsburg, C.: Self-organization of orientation sensitive cells in striate cortex. Kybernetik 14(2), 85–100 (1973) 5. Takeuchi, A., Amari, S.: Formation of topographic maps and columnar microstructures in nerve fields. Biol. Cybern. 35(2), 63–72 (1979) 6. Erwin, E., Obermayer, K., Schulten, K.: Self-organizing maps - stationary states, metastability and convergence rate. Biol. Cybern. 67(1), 35–45 (1992) 7. Geszti, T., Csabai, I., Cazok´ o, F., Szak´ acs, T., Serneels, R., Vattay, G.: Dynamics of the kohonen map. In: Statistical mechanics of neural networks: proceedings of the XIth Sitges Conference, pp. 341–349. Springer, New York (1990) 8. Der, R., Herrmann, M., Villmann, T.: Time behavior of topological ordering in self-organizing feature mapping. Biol. Cybern. 77(6), 419–427 (1997) 9. Aoki, T., Aoyagi, T.: Self-organizing maps with asymmetric neighborhood function. Neural Comput. 19(9), 2515–2535 (2007)

A Characterization of Simple Recurrent Neural Networks with Two Hidden Units as a Language Recognizer Azusa Iwata1 , Yoshihisa Shinozawa1 , and Akito Sakurai1,2 1

Keio University, Hiyoshi, Kohoku-ku, Yokohama 223-8522, Japan 2 CREST, Japan Science and Technology Agency

Abstract. We give a necessary condition that a simple recurrent neural network with two sigmoidal hidden units to implement a recognizer of the formal language {an bn |n > 0} which is generated by a set of generating rules {S → aSb, S → ab} and show that by setting parameters so as to conform to the condition we get a recognizer of the language. The condition implies instability of learning process reported in previous studies. The condition also implies, contrary to its success in implementing the recognizer, difficulty of getting a recognizer of more complicated languages.

1

Introduction

Pioneered by Elman [6], many researches have been conducted on grammar learning by recurrent neural networks. Grammar is defined by generating or rewriting rules such as S → Sa and S → b. S → Sa means that “the letter S should be rewritten to Sa,” and S → b means that “the letter S should be rewritten to b.” If we have more than one applicable rules, we have to try all the possibility. The string generated this way but with no further applicable rewriting rule is called a sentence. The set of all possible sentences is called the language generated by the grammar. Although the word “language” would be better termed formal language in contrast to natural language, we follow custom in formal language theory field(e.g. [10]). In everyday expression a sentence is a sequence of words whereas in the above example it is a string of characters. Nevertheless the essence is common. The concept that a language is defined based on a grammar this way has been a major paradigm in a wide range of formal language study and related field. A study of grammar learning focuses on restoring a grammar from finite samples of sentences of a language associated with the grammar. In contrast to ease of generation of sample sentences, grammar learning from sample sentences is a hard problem and in fact it is impossible except for some very restrictive cases e.g. the language is finite. As is well-known, humans do learn language whose grammar is very complicated. The difference between the two situations might be attributed to possible existence of some unknown restrictions on types of natural language grammars in our brain. Since neural network is a model of M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 436–445, 2008. c Springer-Verlag Berlin Heidelberg 2008 

A Characterization of Simple RNNs with Two Hidden Units

437

brain, some researchers think that neural network could learn a grammar from exemplar sentences of a language. The researches on grammar learning by neural networks are characterized by 1. focus on learning of self-embedding rules, since simpler rules that can be expressed by finite state automata are understood to be learnable, 2. simple recurrent neural networks (SRN) are used as basic mechanism, and 3. languages such as {an bn |n > 0}, {an bn cn |n > 0} which are clearly the results of, though not a representative of, self-embedding rules, were adopted for target languages. Here an is a string of n-times repetition of a character a, the language {an bn |n > 0} is generated by a grammar {S → ab, S → aSb}, and {an bn cn |n > 0} is generated by a context-sensitive grammar. The adoption of simple languages such as {an bn |n > 0} and {an bn cn |n > 0} as target languages are inevitable, since it is not easy to see whether enough generalization is achieved by the learning if realistic grammars are used. Although these languages are simple, many times when they learned, what they learned was just what they were taught, not a grammar, i.e. they did almost rote-learning. In other words, their generalization capability in learning was limited. Also their learned-results were unstable in a sense that when they were given new training sentences which were longer than the ones they learned but still in the same language, learned network changes unexpectedly. The change was more than just refinement of the learned network. Considering these situations, we may doubt that there really exists a solution or correct learned network. Bod´en et al. ([1,2,3]), Rodriguez et al. ([13,14]), Chalup et al. ([5]), and others tried to clarify the reasons and explore the possibility of network learning of the languages. But their results were not conclusive and did not give clear conditions that the learned networks satisfy in common. We will in this paper describe a condition that SRNs with two hidden units learned to recognize language {an bn |n > 0} have in common, that is, a condition that has to be met for SRN to be qualified to be successfully learned the language. The condition implies, by the way, instability of learning results. Moreover by utilizing the condition, we realize a language recognizer. By doing this we found that learning of recognizers of languages more complicated than {an bn |n > 0} are hard. RNN (recurrent neural network) is a type of networks that has recurrent connections and a feedforward network connections. The calculation is done at once for the feedforward network part and after one time-unit delay the recurrent connection part gives rises to additional inputs (i.e., additional to the external inputs) to the feedforward part. RNN is a kind of discrete time system. Starting with an initial state (initial outputs, i.e. outputs without external inputs, of feedforward part) the network proceeds to accept the next character in a string given to external inputs, reaches the final state, and obtains the final external output from the final state. SRN (simple recurrent network) is a simple type of RNN and has only one layer of hidden units in its feedforward part.

438

A. Iwata, Y. Shinozawa, and A. Sakurai

Rodriguez et al. [13] showed that SRN learns languages {an bn |n > 0} (or, more correctly, its subset {ai bi |n ≥ i > 0}) and {an bn cn |n > 0} (or, more correctly, {ai bi |n ≥ i > 0}). For {an bn |n > 0}, they found an SRN that generalized to n ≤ 16 after learned for n ≤ 11. They analyzed how the languages are processed by the SRN but the results were not conclusive so that it is still an open problem if it is really possible to realize recognizers of the languages such as {an bn |n > 0} or {an bn cn |n > 0} and if SRN could learn or implement more complicated languages. Siegelmann [16] showed that an RNN has computational ability superior to a Turing machine. But the difficulty of learning {an bn |n > 0} by SRN suggests that at least SRN might not be able to realize Turing machine. The difference might be attributed to the difference of output functions of neurons: piecewise linear function in Siegelmann’s case and sigmoidal function in standard SRN cases, since we cannot find other substantial differences. On the other hand Casey [4] and Maass [12] showed that in noisy environment RNN is equivalent to finite automaton or less powerful one. The results suggest that correct (i.e., infinite precision) computation is to be considered when we had to make research on possibility of computations by RNN or specifically SRN. Therefore in the research we conducted and will report in this paper we have adopted RNN models with infinite precision calculation and with sigmoidal function (tanh(x)) as output function of neurons. From the viewpoints above, we discuss two things in the paper: a necessary condition for an SRN with two hidden units to be a recognizer of the language {an bn |n > 0}, the condition is sufficient enough to guide us to build an SRN recognizing the language.

2

Preliminaries

SRN (simple recurrent network) is a simple type of recurrent neural networks and its function is expressed by sn+1 = σ(w s · sn + wx · xn ), Nn (sn ) = wos · sn + woc where σ is a standard sigmoid function (tanh(x) = (1 − exp(−x))/(1 + exp(−x))) and is applied component-wise. A counter is a device which keeps an integer and allows +1 or −1 operation and answers yes or no to an inquiry if the content is 0 (0-test). A stack is a device which allows operations push i (store i) and pop up (recall the laststored content, discard it and restore the device as if it were just before the corresponding push operation). Clearly a stack is more general than a counter, so that if a counter is not implementable, a stack is not, too. Elman used SRN as a predictor but Rodriguez et al. [13] and others consider it as a counter. We in this paper mainly stand on the latter viewpoint. We will first explain that a predictor is used as a recognizer or a counter. When we try to train SRN to recognize a language, we train it to predict correctly the next character to come in an input string (see e.g. [6]). As usual we adopted “one hot vector” representation for the network output. One hot vector is a vector representation in which a single element is one and the others

A Characterization of Simple RNNs with Two Hidden Units

439

are zero. The network is trained so that the sum of the squared error (difference between actual output and desired output) is minimized. By this method, if two possible outputs with the same occurrence frequency for the same input exist in a training data, the network would learn to output 0.5 for two elements in the output with others being 0, since the output vector should give the minimum of the sum of the squared error. It is easily seen that if a network correctly predicts the next character to come for a string in the language {an bn |n > 0}, we could see it behaves as a counter with limited function. Let us add a new network output whose value is positive if the original network predicts only a to come (which happens only when the input string up to the time was an bn for some n which is common practice since Elman’s work) and is negative otherwise. The modified network seems to count up for a character a and count down for b since it outputs positive value when the number of as and bs coincide and negative value otherwise. Its counting capability, though, is limited since it could output any value when a is fed before the due number of bs are fed, that is, when counting up action is required before the counter returns back to 0-state. A (discrete-time) dynamical system is represented as the iteration of a function application: si+1 = f (si ), i ∈ N , si ∈ Rn . A point s is called a fixed point of f if f (s) = s. A point s is an attracting fixed point of f if s is a fixed point and there exists a neighborhood Us around s such that limi→∞ f i (x) = s for all x ∈ Us . A point s is a repelling fixed point of f if s is an attracting fixed point of f −1 . A point s is called a periodic point of f if f n (s) = s for some n. A point s is a ω-limit point of x for f if limi→∞ f ni (x) = s for limi→∞ ni = ∞. A fixed point x of f is hyperbolic if all of the eigenvalues of Df at x have absolute values different from one, where Df = [∂fi /∂xj ] is the Jacobian matrix of first partial derivatives of the function f . A set D is invariant under f if for any s ∈ D, f (s) ∈ D. The following theorem plays an important role in the current paper. Theorem 1 (Stable Manifold Theorem for a Fixed Point [9]). Let f : Rn → Rn be a C r (r ≥ 1) diffeomorphism with a hyperbolic fixed point x. Then s,f u,f there are local stable and unstable manifolds Wloc (x), Wloc (x), tangent to the s,f s,f u,f eigenspaces Ex , Ex of Df at x and of corresponding dimension. Wloc (x) and u,f r Wloc (x) are as smooth as the map f , i.e. of class C . Local stable and unstable manifold for f are defined as follows: s,f Wloc (q) = {y ∈ Uq | limm→∞ dist(f m (y), q) = 0} u,f Wloc (q) = {y ∈ Uq | limm→∞ dist(f −m (y), q) = 0}

where Uq is a neighborhood of q and dist is a distance function. Then global  s,f stable and unstable manifold for f are defined as: W s,f (q) = i≥0 f −i (Wloc (q))  u,f u,f i and W (q) = i≥0 f (Wloc (q)). As defined, SRN is a pair of a discrete-time dynamical system sn+1 = σ(w s · sn + wx · xn ) and an external output part Nn = wos · sn + woc . We simply

440

A. Iwata, Y. Shinozawa, and A. Sakurai

write the former (dynamical system part) as sn+1 = f (sn , xn ) and the external output part as h(sn ). When RNN (or SRN) is used as a recognizer of the language {an bn |n > 0}, as described in Introduction, it is seen as a counter where the input character a is for a count-up operation (i.e., +1) and b is for a count-down operation (i.e., −1). In the following we may use x+ for a and x− for b. For abbreviation, in the following, we use f+ = f ( · , x+ ) and f− = f ( · , x− ) Please note that f− is undefined for the point outside and on the border of the square (I[−1, 1])2 , where I[−1, 1] is the closed interval [−1, 1]. In the following, though, we do not mention it for simplicity. D0 is the set {s|h(s) ≥ 0}, that is, a region where the counter value is 0. Let −i D i = f− (D0 ), that is, a region where the counter value is i. We postulate that f+ (Di ) ⊆ Di+1 . This means that any point in Di is eligible for a state where the counter content is i. This may seem to be rather demanding. An alternative would be that the point p is for counter content c if and only if p1 pi m1 mi p = f− ◦ f+ ◦ . . . ◦ f− ◦ f+ (s0 ) for a predefined s0 , some mj ≥ 0 and pj ≥ 0 i for 1 ≤ j ≤ i, and i ≥ 0 such that j=1 (pj − mj ) = c. This, unfortunately, has not resulted in a fruitful result. We also postulate that Di ’s are disjoint. Since we defined Di as a closed set, the postulate is natural. The point is therefore we have chosen Di to be closed. The postulate requires that we should keep a margin between D0 and D1 and others.

3

A Necessary Condition

We consider only SRN with two hidden units, i.e., all the vectors concerning s such as ws , sn , wos are two-dimensional.  Definition 2. Dω is the set of the accumulation points of i≥0 Di , i.e. s ∈ Dω iff s = limi→∞ ski for some ski ∈ Dki . Definition 3. Pω is the set of ω-limit points of points in D0 for f+ , i.e. s ∈ Pω ki iff s = limi→∞ f+ (s0 ) for some ki and s0 ∈ D0 . Qω is the set of ω-limit points −ki −1 of points in D0 for f− , i.e. s ∈ Qω iff s = limi→∞ f− (s0 ) for some ki and s0 ∈ D0 . Considering the results obtained in Bod´en et al. ([1,2,3]), Rodriguez et al. ([13,14]), Chalup et al. ([5]), it is natural, at least for a first consideration, to i i postulate that f+ (x) and f− (x) are not wondering so that they will converge to periodic points. Therefore Pω and Qω are postulated as finite set of hyperbolic periodic points for f+ and f− , respectively. In the following, though, for simplification of presentations, we postulate that Pω and Qω are finites set of hyperbolic fixed points for f+ and f− , respectively. Moreover, according to the same literature, the points in Qω are saddle points u,f s,f for f− , so that we further postulate that Wloc − (q) for q ∈ Qω and Wloc − (q) for q ∈ Qω are one-dimensional, where their existence is guaranteed by Theorem 1.

A Characterization of Simple RNNs with Two Hidden Units

441

Postulate 4. We postulate that f+ (Di ) ⊆ Di+1 , Di ’s are disjoint, Pω and Qω u,f are finites set of hyperbolic fixed points for f+ and f− , respectively, and Wloc − (q) s,f− for q ∈ Qω and Wloc (q) for q ∈ Qω are one-dimensional.   −1 −1 Lemma 5. f− ◦ f+ (Dω ) = Dω , f− (Dω (I(−1, 1))2 ) = Dω and Dω (I(−1, 1))2 = f− (Dω ), and f+ (Dω ) ⊆ Dω . Pω ⊆ Dω and Qω ⊆ Dω . −1 Definition 6. W u,−1 (q) is the global unstable manifold at q ∈ Qω for f− , i.e., −1 u,f− u,−1 s,f− W (q) = W (q) = W (q). i Lemma 7. For any p ∈ Dω , any accumulation point of {f− (p)|i > 0} is in Qω .

Proof. Since p is in Dω , there exist pki ∈ Dki such that p = limi→∞ pki . Suppose q in Dω is the accumulation point stated in the theorem statement, i.e., q = h limj→∞ f−j (p). We take ki large enough for any hj so that in any neighborhood of h h h −k q where f−j (p) is in, pki exists. Then q = limj→∞ f−j (pki ) = limj→∞ f−j i (ski ) ki where ki is a function of hj with ki > hj . Let ski = f− (pki ) ∈ D0 and s0 ∈ D0 −1 be an accumulation point of {ski }. Then since f− is continuous, letting nj = −n −hj + ki > 0, q = limj→∞ f− j (s0 ), i.e., q ∈ Qω .

 Lemma 8. Dω = q∈Qω W u,−1 (q). Proof. Let p be any point in Dω . Since f− (Dω ) ⊆ (I[−1, 1])2 where I[−1, 1] n is the interval [−1, 1], i.e., f− (Dω ) is bounded, and f− (Dω ) ⊆ Dω , {f− (p)} has an accumulation point q in Dω , which is, by Lemma 7, in Qω . Then q is n expressed as q = limj→∞ f−j (p). Since Qω is a finite set of hyperbolic fixed −1 n points, q = limn→∞ f− (p), i.e., p ∈ W s,f (q) = W u,f (q) = W u,−1 (q).

Since Pω ⊆ Dω , the next theorem holds. Theorem 9. A point in Pω is either a point in Qω or in W u,−1 (q) for some q ∈ Qω . Please note that since q ∈ W u,−1 (q), the theorem statement is just “If p ∈ Pω then p ∈ W u,−1 (q) for some q ∈ Qω .”

4

An Example of a Recognizer

To construct an SRN recognizer for {an bn |n > 0}, the SRN should satisfy the conditions stated in Theorem 9 and Postulate 4, which are summarized as: 1. f+ (Di ) ⊆ Di+1 , 2. Di ’s are disjoint, 3. Pω and Qω are finites set of hyperbolic fixed points for f+ and f− , respectively, u,f s,f 4. Wloc − (q) for q ∈ Qω and Wloc − (q) for q ∈ Qω are one-dimensional, and u,−1 5. If p ∈ Pω then p ∈ W (q) for some q ∈ Qω .

442

A. Iwata, Y. Shinozawa, and A. Sakurai

Let us consider as simple as possible, so that the first choice is to think about a point p ∈ Pω and q ∈ Qω , that is f+ (p) = p and f− (q) = q. Since p cannot be −1 the same as q (because f− ◦ f+ (p) = p + w−1 s · w x · (x+ − x− ) = p ), we have u,−1 to find a way to let p ∈ W (q). Since it is very hard in general to calculate stable or unstable manifolds from a function and its fixed point, we had better try to let W u,−1 (q) be a “simple” manifold. There is one more reason to do so: we have to define D0 = {x|h(x) ≥ 0} but if W u,−1 (q) is not simple, suitable h may not exist. We have decided that W u,−1 (q) be a line (if possible). Considering the function form f− (s) = σ(w s · s + wx · x− ), it is not difficult to see that the line could be one of the axes or one of the bisectors of the right angles at the origin (i.e., one of the lines y = x and y = −x). We have chosen the bisector in the first (and the third) quadrant (i.e., the line y = x). By the way q was chosen to be the origin and p was chosen arbitrarily to be (0.8, 0.8). The item 4 is satisfied by setting one of the two eigenvalues of Df− at the origin to be greater than one, and the other smaller then one. We have chosen 1/0.6 for one and 1/μ for the other which is to be set so that Item 1 and 2 are satisfied by considering eigenvalues of Df+ at p for f+ . The design consideration that we have skipped is how to design D0 = {x|h(x) ≥ 0}. A simple way is to make the boundary h(x) = 0 parallel to W u,−1 (q) for our intended q ∈ Qω . Because, if we do so, by setting the largest eigenvalue of Df− at q to be equal to the inverse of the eigenvalue of Df+ at p along the 2 2 i normal to W u,−1 , we can get the points s ∈ D0 , f− ◦ f+ (s), f− ◦ f+ (s), . . . , f− ◦ i f+ (s), . . . that belong to {an bn |n > 0}, reside at approximately equal distance from W u,−1 . Needless to say that the points belonging to, say, {an+1 bn |n > 0} have approximately equal distance from W u,−1 among them and this distance is different from that for {an bn |n > 0}. Let f− (x) = σ(Ax + B0 ), f+ (x) = σ(Ax + B1 ). We plan to put Qω = {(0, 0)}, Pω = {(0.8, 0.8)}, W u,−1 = {(x, y)|y = x}, the eigenvalues of the tangent space −1 of f− at (0, 0) are 1/λ = 1/0.6 and 1/μ (where the eigenvector on y = x is expanding), and the eigenvalues of the tangent space of f+ at (0.8, 0.8) are 1/μ and any value. Then, considering derivatives at (0, 0) and (0.8, 0.8), it is easy to see   π   1 π 1 λ 0 A=ρ ρ − , = (1 − 0.82 )μ 0 μ 2 4 4 μ where ρ(θ) is a rotation by θ. Then   λ+μ λ−μ A= λ−μ λ+μ Next from σ(B0 ) = (0, 0)T and σ((0.8λ, 0.8λ)T + B1 ) = (0.8, 0.8)T ,    −1  0 σ (0.8) − 0.8λ B0 = , B1 = . 0 σ −1 (0.8) − 0.8λ These give us μ = 5/3, λ = 0.6, B1 ≈ (1.23722, 1.23722)T .

A Characterization of Simple RNNs with Two Hidden Units

443

Fig. 1. The vector field representation of f+ (left) and f− (right) 1

0.75

0.5

0.25

-1

-0.75

-0.5

-0.25

0.25

0.5

0.75

1

-0.25

-0.5

-0.75

-1

-1

-0.75

-0.5

1

1

0.75

0.75

0.5

0.5

0.25

0.25

-0.25

0.25

0.

0.75

1

-1

-0.75

-0.5

-0.25

0.25

-0.25

-0.25

-0.5

-0.5

-0.75

-0.75

-1

-1

0.5

0.75

1

n+1 n+1 n n n Fig. 2. {f− ◦ f+ (p)| n ≥ 1} (upper), {f− ◦ f+ (p)| n ≥ 1} (lower left), and {f− ◦ n f+ (p)| n ≥ 1} (lower right) where p = (0.5, 0.95)

444

A. Iwata, Y. Shinozawa, and A. Sakurai

In Fig. 1, the left image shows the vector field of f+ where the arrows starting at x end at f+ (x) and the right image shows the vector field of f− . In Fig. 2, the upper plot shows points corresponding to strings in {an bn |n > 0}, the lower-left plot {an+1 bn |n > 0}, and the lower-right plot {an bn+1 |n > 0}. The initial point was set to p = (0.5, 0.95) in Fig. 2. All of them are for n = 1 to n = 40 and when n grows the points gather so we could say that they stay in narrow stripes, i.e. Dn , for any n.

5

Discussion

We obtained a necessary condition that SRN implements a recognizer for the language {an bn |n > 0} by analyzing its behavior from the viewpoint of discrete dynamical systems. The condition supposes that Di ’s are disjoint, f+ (Di ) ⊆ Di+1 , and Qω is finite. It suggests a possibility of the implementation and in fact we have successfully built a recognizer for the language, thereby we showed that the learning problem of the language has at least a solution. Unstableness of any solutions for learning is suggested to be (but not derived to be) due to the necessity of Pω being in an unstable manifold W u,−1 (q) for n q ∈ Qω . Since Pω is attractive in the above example, f+ (s0 ) for s0 ∈ D0 comes n exponentially close to Pω for n. By even a small fluctuation of Pω , since f+ (s0 ), u,−1 n n too, is close to W (q), f− (f+ (s0 )), which should be in D0 , is disturbed much. This means that even if we are close to a solution, by just a small fluctuation of n n Pω caused by a new training data, f− (f+ (s0 )) may easily be pushed out of D0 . Since Rodriguez et al. [14] showed that the languages that do not belong to the context-free class could be learned to some degree, we have to further study the discrepancies. Instability of grammar learning by SRN shown above might not be seen in our natural language learning, which suggests that SRN might not be appropriate for a model of language learning.

References 1. Bod´en, M., Wiles, J., Tonkes, B., Blair, A.: Learning to predict a context-free language: analysis of dynamics in recurrent hidden units. Artificial Neural Networks (1999); Proc. ICANN 1999, vol. 1, pp. 359–364 (1999) 2. Bod´en, M., Wiles, J.: Context-free and context-sensitive dynamics in recurrent neural networks. Connection Science 12(3/4), 197–210 (2000) 3. Bod´en, M., Blair, A.: Learning the dynamics of embedded clauses. Applied Intelligence: Special issue on natural language and machine learning 19(1/2), 51–63 (2003) 4. Casey, M.: Correction to proof that recurrent neural networks can robustly recognize only regular languages. Neural Computation 10, 1067–1069 (1998) 5. Chalup, S.K., Blair, A.D.: Incremental Training Of First Order Recurrent Neural Networks To Predict A Context-Sensitive Language. Neural Networks 16(7), 955– 972 (2003)

A Characterization of Simple RNNs with Two Hidden Units

445

6. Elman, J.L.: Distributed representations, simple recurrent networks and grammatical structure. Machine Learning 7, 195–225 (1991) 7. Elman, J.L.: Language as a dynamical system. In: Mind as Motion: Explorations in the Dynamics of Cognition, pp. 195–225. MIT Press, Cambridge 8. Gers, F.A., Schmidhuber, J.: LSTM recurrent networks learn simple context free and context sensitive languages. IEEE Transactions on Neural Networks 12(6), 1333–1340 (2001) 9. Guckenheimer, J., Holmes, P.: Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields. Springer, Heidelberg (Corr. 5th print, 1997) 10. Hopcroft, J.E., Ullman, J.D.: Introduction to automata theory, languages, and computation. Addison-Wesley, Reading (1979) 11. Katok, A., Hasselblatt, B.: Introduction to the Modern Theory of Dynamical Systems. Cambridge University Press, Cambridge (1996) 12. Maass, W., Orponen, P.: On the effect of analog noise in discrete-time analog computations. Neural Computation 10, 1071–1095 (1998) 13. Rodriguez, P., Wiles, J., Elman, J.L.: A recurrent neural network that learns to count. Connection Science 11, 5–40 (1999) 14. Rodriguez, P.: Simple recurrent networks learn context-free and context-sensitive languages by counting. Neural Computation 13(9), 2093–2118 (2001) 15. Schmidhuber, J., Gers, F., Eck, D.: Learning Nonregular Languages: A Comparison of Simple Recurrent Networks and LSTM. Neural Computation 14(9), 2039–2041 (2002) 16. Siegelmann, H.T.: Neural Networks and Analog Computation: beyond the Turing Limit, Birkh¨ auser (1999) 17. Wiles, J., Blair, A.D., Bod´en, M.: Representation Beyond Finite States: Alternatives to Push-Down Automata. In: A Field Guide to Dynamical Recurrent Networks

Unbiased Likelihood Backpropagation Learning Masashi Sekino and Katsumi Nitta Tokyo Institute of Technology, Japan

Abstract. The error backpropagation is one of the popular methods for training an artificial neural network. When the error backpropagation is used for training an artificial neural network, overfitting occurs in the latter half of the training. This paper provides an explanation about why overfitting occurs with the model selection framework. The explanation leads to a new method for training an aritificial neural network, Unibiased Likelihood Backpropagation Learning. Several results are shown.

1

Introduction

An artificial neural network is one of the model for function approximation. It is possible to approximate arbitrary function when the number of basis functions is large. The error backpropagation learning [1], which is a famous method for training an artificial neural network, is the gradient discent method with the squared error to learning data as a target function. Therefore, the error backpropagation learning can obtain local optimum while monotonously decreasing the error. Here, although the error to learning data is monotonously decreasing, the error to test data increases in the latter half of training. This phenomenon is called overfitting. Early stopping is one of the method for preventing the overfitting. This method stop the training when an estimator of the generalization error does not decrease any longer. For example, the technique which stop the training when the error to hold-out data does not decrease any longer is often applied. However, the early stopping basically minimize the error to learning data, therefore there is no guarantee for obtaining the optimum parameter which minimize the estimator of the generalization error. When the parameters of the basis functions (model parameter) are fixed, an artificial neural network becomes a linear regression model. If a regularization parameter is introduced to assure the regularity of this linear regression model, the artificial neural network becomes a set of regular linear regression models. The cause of why an artificial neural network tends to overfit is that the maximum likelihood estimation with respect to the model parameter is the model selection about regular linear regression models based on the empirical likelihood. In this paper, we propose the unbiased likelihood backpropagation learning which is the gradient discent method for modifying the model parameter with unbiased likelihood (information criterion) as a target function. It is expected M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 446–455, 2008. c Springer-Verlag Berlin Heidelberg 2008 

Unbiased Likelihood Backpropagation Learning

447

that the proposed method has better approximation performance because the method explicitly minimize an estimator of the generalization error. Following section, section 2 explains about statistical learning and maximum likelihood estimation, and section 3 explains about information criterion shortly. Next, in section 4, we give an explanation about an artificial neural network, regularized maximum likelihood estimation and why the error backpropagation learning cause overfitting. Then, the proposed method is explained in section 5. We show the effectiveness of the proposed method by applying the method to DELVE data set [4] in section 6. Finally we conclude this paper in section 7.

2

Statistical Learning

Statistical learning aims to construct an optimal approximation pˆ(x) of a true distribution q(x) from a set of hypotheses M ≡ {p(x|θ) | θ ∈ Θ} using learning data D ≡ {xn | n = 1, · · · , N } obtained from q(x). M is called model and approximation pˆ(x) is called estimation. When we want to clearly denote that the estimation pˆ(x) is constructed using learning data D, we use a notation pˆ(x|D). Kullback-Leibler divergence:  q(x) D(q||p) ≡ q(x) log dx (1) p(x) is used for distance from q(x) to p(x). Probability density p(x) is called likelihood, and especially the likelihood of the estimation pˆ(x) to the learning data D: N  pˆ(D|D) ≡ pˆ(xn |D) (2) n=1

is called empirical likelihood. Sample mean of the log-likelihood: N 1 1  log p(D) = log p(xn ) N N n=1

asymptotically converges in probability to the mean log-likelihood:    Eq(x) log p(x) ≡ q(x) log p(x)dx

(3)

(4)

according to the law of large numbers, where Eq(x) denotes an expectation under q(x). Because Kullback-Leibler divergence can be decomposed as     D(q||p) = Eq(x) log q(x) − Eq(x) log p(x) , (5) the maximization of the mean log-likelihood is equal to the minimization of Kullback-Leibler divergence. Therefore, statistical learning methods such as maximum likelihood estimation, maximum a posteriori estimation and Bayesian estimation are based on the likelihood.

448

M. Sekino and K. Nitta

Maximum Likelihood Estimation Maximum likelihood estimation pˆML (x) is the hypothesis p(x|θˆML ) by the maximizer θˆML of likelihood p(D|θ): pˆML (x) ≡ p(x|θˆML ), θˆML ≡ argmax p(D|θ). θ

3 3.1

(6) (7)

Information Criterion and Model Selection Information Criterion

Because sample mean of the log-likelihood asymptotically converges in probability to the mean log-likelihood, statistical learning methods are based on likelihood. However, because learning data D is a finite set in practice, the empirical likelihood pˆ(D|D) contains bias. This bias b(ˆ p) is defined as    b(ˆ p) ≡ Eq(D) log pˆ(D) −N · Eq(x) log pˆ(x|D) , (8) N where q(D) ≡ n=1 q(xn ). Because of the bias, it is known that the most overfitted model to learning data is selected when select a regular model from a candidate set of regular models based on the empirical likelihood. Addressing this problem, there have been proposed many information criteria which evaluates learning results by correcting the bias. Generally the form of the information criterion is IC(ˆ p, D) ≡ −2 log pˆ(D) + 2ˆb(ˆ p)

(9)

where ˆb(ˆ p) is the estimator of the bias b(ˆ p). Corrected AIC (cAIC) [3] estimates and corrects accurate bias of the empirical log-likelihood as N (M + 1) ˆbcAIC (ˆ pML ) = , (10) N −M −2 under the assumption that the learning model is a normal linear regression model, the true model is included in the learning model and the estimation is constructed by maximum likelihood estimation. Here, M is the number of explanatory variables and dim θ = M +1 (1 is the number of the estimators of variance.) Therefore, cAIC asymptotically equals AIC [2]: ˆbcAIC (ˆ pML ) → ˆbAIC (ˆ pML ) (N → ∞).

4

Artificial Neural Network

In a regression problem, we want to estimate the true function using learning data D = {(xn , yn ) | n = 1, · · · , N }, where xn ∈ Rd is an input and yn ∈ R is the corresponding output. An artificial neural network is defined as

Unbiased Likelihood Backpropagation Learning

f (x; θM ) =

M 

ai φ(x; ϕi )

449

(11)

i=1

where ai (i = 1, · · · , N ) are regression coefficients and φ(x; ϕi ) are basis functions which are parametalized by ϕi . Model parameter of this neural network is θM ≡ (ϕT1 , · · · , ϕTM )T (T denotes transpose.) Design matrix X, coefficient vector a and output vector y are defined as Xij ≡ φ(xi ; ϕj )

(12)

a ≡ (a1 , · · · , aM )

(13)

y ≡ (y1 , · · · , yN ) .

(14)

T

T

When the model parameter θM are fixed, the artificial neural network (11) becomes a linear regression model parametalized by a. A normal linear regression model:

 1 (y −f (x; θM))2 p(y|x; θM ) ≡ √ exp − (15) 2σ 2 2πσ 2 is usually used when the noise included in the output y is assumed to follow a normal distribution. In this paper, we call the parameter θR ≡ (aT , σ 2 )T regular parameter. 4.1

Regularized Maximum Likelihood Estimation

To assure the regularity of the normal linear regression model (15), regularized maximum likelihood estimation is usually used for estimating θR = (aT , σ 2 )T . Regularized maximum likelihood estimation maximize regularized log-likelihood: log p(D) − exp(λ)  a 2 .

(16)

λ ∈ R is called a regularization parameter. Regularized maximum likelihood estimators of the coefficient vector a and the valiance σ 2 are ˆ = Ly a (17) and σ ˆ2 = where

1 ˆ 2 ,  y − Xa N

L ≡ (X T X + exp(λ)I)−1 X T

and I is the identity matrix. Here, the effective number of the regression coefficients is

Mef f = tr XL .

(18) (19)

(20)

Therefore, Mef f is used for the number of explanatory variables M in (10)1 . 1

When the denominator of (10) is not positive value, we use ˆbcAIC (ˆ p) = ∞.

450

4.2

M. Sekino and K. Nitta

Overfitting of the Error Backpropagation Learning

The error backpropagation learning is usually used for training an artifical neural network. This method is equal to the gradient discent method based on the likelihood of the linear regression model (15), because the target function is the squared error to learning data. In what follows, we assume the noise follow a normal distribution and the normal linear regression model (15) is regular for all θM . Then, an artificial neural network becomes a set of regular linear regression models. For simplicity, let’s think about a set of regular models H ≡ {M(θM ) | θM ∈ ΘM }, where M(θM ) ≡ {p(x|θR ; θM ) | θR ∈ ΘR } is a regular model. We can define a new model MC ≡ {ˆ p(x; θM ) | θM ∈ ΘM }, where pˆ(x; θM ) is the estimation of M(θM ). Concerning model parameter θM , statistical learning methods construct the estimation of the model MC based on the empirical likelihood pˆ(D; θM ). For example, maximum likelihood estimation selects θˆMML which is the maximizer of pˆ(D; θM ): pˆML (x) = pˆ(x; θˆMML ), θˆMML ≡ argmax pˆ(D; θM ). θM

(21) (22)

Thus, the maximum likelihood estimation with respect to model parameter θM is the model selection from H based on the empirical likelihood. Because the error backpropagation is the method which realizes maximum likelihood estimation by the gradient discent method, the model M(θM ) becomes the one gradually overfitted in the latter half of the training. Therefore, we propose a learning method for model parameter θM based on unbiased likelihood which is the empirical likelihood corrected by an appropriate information criterion.

5 5.1

Unbiased Likelihood Backpropagation Learning Unbiased Likelihood

Using an information criterion IC(ˆ p, D), we define unbiased likelihood as:   1 pˆub (D) = exp − IC(ˆ p, D) . (23) 2 This unbiased likelihood satisfies     1 Eq(D) log pˆub (D) = Eq(x) log pˆ(x) N when the assumptions of the information criterion are satisfied.

(24)

Unbiased Likelihood Backpropagation Learning

5.2

451

Regular Hierarchical Model

In this paper, we consider about a certain type of hierarchical model, which we call a regular hierarchical model, defined as a set of regular models. A concise definitions of a regular hierarchical model are follows. Regular Hierarchical Model – H ≡ {M(θM ) | θM ∈ ΘM } – M(θM ) ≡ {p(x|θR ; θM ) | θR ∈ ΘR } – M(θM ) is a regular model with respect to θR . An artificial neural network is one of the regular hierarchical models. And also we define unbiased maximum likelihood estimation as follows. Unbiased Maximum Likelihood Estimation Unbiased maximum likelihood estimation pˆubML (x) is the estimation pˆ(x; θˆMubML) by the maximizer θˆMubML of the unbiased likelihood pˆub (D|θM ): pˆubML (x) ≡ pˆ(x; θˆMubML ) θˆMubML ≡ argmax pˆub (D; θM ). θM

5.3

(25) (26)

Unbiased Likelihood Backpropagation Learning

The partial differential of the unbiased likelihood is ∂ ∂ ∂ ˆ log pˆub (D; θM ) = log pˆ(D; θM ) − b(ˆ p; θM ). ∂θM ∂θM ∂θM

(27)

We define the unbiased likelihood estimation based on the gradient method with this partial differential as unbiased likelihood backpropagation learning. 5.4

Unbiased Likelihood Backpropagation Learning for an Artificial Neural Network

In this paper, we derive the unbiased likelihood backpropagation learning for an artificial neural network when the bias of the empirical likelihood is estimated by cAIC (10). The partial differential of the empirical likelihood with respect to θM which is the first term of (27), is ∂ 1 log pˆ(D; θM ) = 2 ∂θM σ ˆ ˆ ∂X a = ∂θM





ˆ ∂X a ∂θM

T

ˆ) (y − X a

 T ∂X ∂X T ∂X L+L y − 2LT X T Ly. ∂θM ∂θM ∂θM

(28)

(29)

452

M. Sekino and K. Nitta

The partial differential of cAIC (10) with respect to θM , which is the second term of (27), is ∂ ˆ N (N − 1) ∂Mef f bcAIC (ˆ p; θM ) = ∂θM (N − M − 2)2 ∂θM   ∂Mef f ∂X ∂X = 2 tr L − LT X T L . ∂θM ∂θM ∂θM

(30)

(31)

We can also obtain the partial differential of the unbiased likelihood with respect to λ as ∂ 1 ˆ ) exp(λ), log pˆ(D; θM ) = − 2 y T LT L(y − X a (32) ∂λ σ ˆ and   ∂Mef f = −tr LT L exp(λ). (33) ∂λ Now, we have already obtain the partial differential of the unbiased likelihood with respect to θM and λ, therefore it is possible to apply the unbiased likelihood backpropagation learning to an artificial neural network.

6

Application to Kernel Regression Model

Kernel regression model is one of the artificial neural networks. The kernel regression model using gaussian kernels has the model parameter of degree one, which is the size of gaussian kernels. This model is comprehensible about the behavior of learning methods. Therefore, the kernel regression model using gaussian kernels is used in the following simulations. In the implementation of the gradient discent method, we adopted quasi-Newton method with BFGS method for estimating the Hesse matrix and golden section search for determining the modification length. 6.1

Kernel Regression Model

Kernel regression model is f (x; θM ) =

N 

an K(x, xn ; θM ).

(34)

n=1

K(x, xn ; θM ) are kernel functions parametalized by model parameter θM . Gaussian kernel:    x − xn  2 K(x, xn ; c) = exp − (35) 2c2 is used in the following simulations, where c is a parameter which decides the size of a gaussian kernel. Model parameter is θM = c.

Unbiased Likelihood Backpropagation Learning

453

1 emp true cAIC

log-likelihood

0.5

0

-0.5

-1 0

2

4

6 iterations

8

10

12

(a) Empirical Likelihood Backpropagation

1 emp true cAIC

log-likelihood

0.5

0

-0.5

-1 0

2

4

6 iterations

8

10

12

(b) Unbiased Likelihood Backpropagation Fig. 1. An example of the transition of mean log empirical likelihood, mean log test likelihood and mean log unbiased likelihood (cAIC)

454

6.2

M. Sekino and K. Nitta

Simulations

For the purpose of evaluation, the empirical likelihood backpropagation learning and the unbiased likelihood backpropagation learning are applied to the 8 dimensional input to 1 dimensional output regression problems of “kin-family” and “pumadyn-family” in the DELVE data set [4]. Each data has 4 combinations of fairly linear (f) or non linear (n), and medium noise (m) or high noise (h). We use 128 samples for learning data. 50 templates are chosen randomly and kernel functions are put on the templates. Fig.1 shows an example of the transition of mean log empirical likelihood, mean log test likelihood and mean log unbiased likelihood (cAIC). It shows the test likelihood of the empirical likelihood backpropagation learning (a) decrease in the latter half of the training and overfitting occurs. On the contrary, it shows the unbiased likelihood backpropagation learning (b) keeps the test likelihood close to the unbiased likelihood (cAIC) and overfitting does not occur. Table 1 shows mean and standard deviations of the mean log test likelihood for 100 experiments. The number in bold face shows it is significantly better result by the t-test at the significance level 1%. Table 1. Mean and standard deviations of the mean log test likelihood for 100 experiments. The number in bold face shows it is significantly better result by the t-test at the significance level 1%. Data Empirical BP kin-8fm 2.323 ± 0.346 kin-8fh 1.108 ± 0.240 kin-8nm −0.394 ± 0.415 kin-8nh −0.435 ± 0.236 pumadyn-8fm −2.235 ± 0.249 pumadyn-8fh −3.168 ± 0.187 pumadyn-8nm −3.012 ± 0.238 pumadyn-8nh −3.287 ± 0.255

6.3

Unbiased BP 2.531 ± 0.140 1.599 ± 0.100 0.078 ± 0.287 0.064 ± 0.048 −1.708 ± 0.056 −2.626 ± 0.021 −2.762 ± 0.116 −2.934 ± 0.055

Discussion

The reason why the results of the unbiased likelihood backpropagation learning in Table 1 shows better is attributed to the fact that the method maximizes the true likelihood averagely, because the mean of the log unbiased likelihood is equal to the log true likelihood (see (24)). The reason why the standard deviations of the test likelihood of the unbiased likelihood backpropagation learning is smaller than that of the empirical likelihood backpropagation learning is assumed to be due to the fact that the empirical likelihood prefer the model which has bigger degree of freedom. On the contrary, the unbiased likelihood prefer the model which has appropriate degree of freedom. Therefore, the variance of the estimation of the unbiased likelihood backpropagation learning becomes smaller than that of the empirical likelihood backpropagation learning.

Unbiased Likelihood Backpropagation Learning

7

455

Conclusion

In this paper, we provide an explanation about why overfitting occurs with the model selection framework. We propose the unibiased likelihood backpropagation learning, which is the gradient discent method for modifying the model parameter with unbiased likelihood (information criterion) as a target function. And we confirm the effectiveness of the proposed method by applying the method to DELVE data set.

References 1. Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning internal representations by error propagation. In: Rumelhart, D.E., McClelland, J.L., et al. (eds.) Parallel Distributed Processing, vol. 1, pp. 318–362. MIT Press, Cambridge (1987) 2. Akaike, H.: A new look at the statistical model identification. IEEE Transactions on Automatic Control 19(6), 716–723 (1974) 3. Sugiura, N.: Further analysis of the data by Akaike’s information criterion and the finite corrections. Communications in Statistics, vol. A78, pp. 13–26 (1978) 4. Rasmussen, C.E., Neal, R.M., Hinton, G.E., van Camp, D., Revow, M., Ghahramani, Z., Kustra, R., Tibshirani, R.: The DELVE manual (1996), http://www.cs.toronto.edu/∼ delve/

The Local True Weight Decay Recursive Least Square Algorithm Chi Sing Leung, Kwok-Wo Wong, and Yong Xu Department of Electronic Engineering, City University of Hong Kong, Hong Kong [email protected]

Abstract. The true weight decay recursive least square (TWDRLS) algorithm is an efficient fast online training algorithm for feedforward neural networks. However, its computational and space complexities are very large. This paper first presents a set of more compact TWDRLS equations. Afterwards, we propose a local version of TWDRLS to reduce the computational and space complexities. The effectiveness of this local version is demonstrated by simulations. Our analysis shows that the computational and space complexities of the local TWDRLS are much smaller than those of the global TWDRLS.

1

Introduction

Training multilayered feedforward neural networks (MFNNs) using recursive least square (RLS) algorithms has aroused much attention in many literatures [1, 2, 3, 4]. This is because those RLS algorithms are efficient second-order gradient descent training methods. They lead to a faster convergence when compared with first-order methods, such as the backpropagation (BP) algorithm. Moreover, fewer parameters are required to be tuned during training. Recently, Leung et. al. found that the standard RLS algorithm has an implicit weight decay effect [2]. However, its decay effect is not substantial and so its generalization ability is not very good. A true weight decay RLS (TWDRLS) algorithm is then proposed [5]. However, the computational complexity of TWDRLS is equal to O(M 3 ) at each iteration, where M is the number of weights. Therefore, it is necessary to reduce the complexity of TWDRLS so that the TWDRLS can be used for large scale practical problems. The main goal of this paper is to reduce both the computational complexity and storage requirement. In Section 2, we derive a set of concise equations for TWDRLS and give some discussions on it. We then describe a local TWDRLS algorithm in Section 3. Simulation results are presented in Section 4. We then summarize our findings in Section 5.

2

TWDRLS Algorithm

A general MFNN is composed of L layers, indexed by 1, · · · , L from input to output. There are nl neurons in layer l. The output of the i-th neuron in the M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 456–465, 2008. c Springer-Verlag Berlin Heidelberg 2008 

The Local True Weight Decay Recursive Least Square Algorithm

457

l-th layer is denoted by yi,l . That means, the i-th neuron of the output layer is represented by yi,L while the i-th input of the network is represented by yi,1 . The connection weight from the j-th neuron of layer l − 1 to the i-th neuron of layer l is denoted by wi,j,l . Biases are implemented as weights and are specified by wi,(nl−1 +1),l , where l = 2, · · · , L. Hence, the number of weights in a MFNN is  given by M = L l=2 (nl−1 + 1)nl . In the standard RLS algorithm, we arrange all weights as a M -dimensional vector, given by  T w = w1,1,2 , · · · w1,(n1 +1),2 , · · · , wnL ,1,L , · · · , wnL ,(nL−1 +1),L . (1) The energy function up to the t-th training sample is given by E(w) =

t 

T

ˆ ˆ d(τ ) − h(w, x(τ )) + [w − w(0)] P −1 (0) [w − w(0)] 2

(2)

τ =1

where x(τ ) is a n1 -dimensional input vector, d(τ ) is the desired nL -dimensional output, and h(w0 , x(τ )) is a nonlinear function that describes the function of the network. The matrix P (0) is the error covariance matrix and is usually set to δ −1 IM×M , where IM×M is a M × M identity matrix. The minimization of (2) leads to the standard RLS equations [1, 3, 6, 7], given by  −1 K(t) = P (t − 1) H T (t) InL ×nL + H(t) P (t − 1) H T (t) (3) P (t) = P (t − 1) − K(t) H(t) P (t − 1) (4) ˆ ˆ − 1) + K(t) [d(t) − h(w(t ˆ − 1), x(t)) ] , w(t) = w(t (5)  T  ∂h(w,(t))  where H(t) = is the gradient matrix (nL × M ) of  ∂w ˆ w=w(t−1)

h(w, x(t)); and K(t) is the so-called Kalman gain matrix (M × nL ) in the classical control theory. The matrix P (t) is the so-called error covariance matrix. It is symmetric positive definite. As mentioned in [5], the standard RLS algorithm only has the limited weight decay effect, being tδo per training iteration, where to is the number of training iterations. The decay effect decreases linearly as the number of training iterations increases. Hence, the more training presentations take place, the less smoothing effect would have in the data fitting process. A true weight decay RLS algorithm, namely TDRLS, was then proposed [5], where a decay term is added to the original energy function. The new energy function is given by t

E(w) =





T ˆ ˆ d(τ ) − h(w, x(τ ))2 + αw T w + [w − w(0)] P −1 (0) [w − w(0)] (6)

τ =1

where α is a regularization parameter. The gradient of E(w) is given by t    ∂E(w) ˆ ≈ P −1 (0) [w − w(0)] + αw − H T (τ ) [d(τ ) − H(τ )w − ξ(τ )] . ∂w τ =1 (7)

458

C.S. Leung, K.-W. Wong, and Y. Xu

ˆ − 1). That means, In the above, we linearize h(w, x(τ )) around the estimate w(τ ˆ − 1), x(τ )) − H(τ )w(τ ˆ ) + ρ(τ ). h(w, x(τ )) = H(τ )w + ξ(τ ), where ξ(τ ) = h(w(τ To minimize the energy function, we set the gradient to zero. Hence, we have ˆ w(t) = P (t)r(t)

(8)

P −1 (t) = P −1 (t − 1) + H T (t) H(t) + αIM×M r(t) = r(t − 1) + H T (t) [d(t) − ξ(t)] .

(9) (10)

where

Δ

Furthermore, we define P ∗ (t) = [IM×M + αP (t − 1)]−1 P (t − 1). Hence, we have P ∗ −1 (t) = P −1 (t − 1) + αIM×M . With the matrix inversion lemma [7] in the recursive calculation of P (t), (8) becomes −1

P ∗ (t − 1) = [IM×M + αP (t − 1)] P (t − 1) (11)  −1 ∗ T ∗ T K(t) = P (t − 1) H (t) InL ×nL + H(t) P (t − 1)H (t) (12) ∗ ∗ P (t) = P (t − 1) − K(t) H(t) P (t − 1) (13) ˆ ˆ − 1)−αP (t)w(t ˆ − 1) + K(t)[d(t) − h(w(t ˆ − 1), x(t))]. (14) w(t) = w(t Equations (11)-(14) are the general global TWDRLS equations. Those are more compact than the equations presented in [5]. Also, the weight updating equation in [5] ( i.e., (14) in this paper) is more complicated. When the regularization parameter α is set to zero, the decay term αw T w vanishes and (11)-(14) reduce to the standard RLS equations. The decay effect can be easily understood by the decay term αw T w in the energy function given by (6). As mentioned in [5], the decay effect per training iterations is equal to αw T w which does not decrease with the number of training iterations. The energy function of TWDRLS is the same as that of batch model weight decay methods. Hence, existing heuristic methods [8,9] for choosing the value of α can be used for the TWDRLS’s case. We can also explain the weight decay effect based on the recursive equations (11)-(14). The main difference between the standard RLS equations and the ˆ − 1) in TWDRLS equations is the introduction of a decay term −αP (t) w(t (14). This term guarantees that the magnitude of the updating weight vector decays an amount proportional to αP (t). Since P (t) is positive definite, the magnitude of the weight vector would not be too large. So the generalization ability of the trained networks would be better [8, 9]. A drawback of TWDRLS is the requirement in computing the inverse of the M -dimensional matrix (IM×M + αP (t − 1)). This complexity is equal to O(M 3 ) which is much larger than that of the standard RLS, O(M 2 ). Hence, the TWDRLS algorithm is computationally prohibitive even for a network with moderate size. In the next Section, a local version of the TWDRLS algorithm will be proposed to solve this large complexity problem.

The Local True Weight Decay Recursive Least Square Algorithm

3

459

Localization of the TWDRLS Algorithm

To localize the TWDRLS algorithm, we first divide the weight vector into several  T small vectors, where wi,l = wi,1,l , · · · , wi,(nl−1 +1),l is denoted as the weights connecting all the neurons of layer l − 1 to the i − th neuron of layer l. We consider the estimation of each weight vector separately. When we consider the i−th neuron in layer l, we assume that other weight vectors are constant vectors. Such a technique is usually used in many numerical methods [10]. At each training iteration, we update each weight vector separately. Each neuron has its energy function. The energy function of the i − th neuron in layer l is given by E(wi,l ) =

t

 2 d(τ ) − h(wi,l , x(τ )) + αw Ti,l wi,l τ =1 T

−1 ˆ i,l (0)] Pi,l ˆ i,l (0)] . + [wi,l − w (0) [w i,l − w

(15)

Utilizing a derivation process similar to the previous analysis, we obtain the following recursive equations for the local TWDRLS algorithm. Each neuron (excepting input neurons) has its set of TWDRLS equations. For the i − th neuron in layer l, the TWDRLS equations are given by





∗ Pi,l (t − 1) = I(nl−1 +1)×(nl−1 +1) + αPi,l (t − 1)



−1

Pi,l (t − 1)

∗ T ∗ T Ki,l (t) = Pi,l (t − 1) Hi,l (t) InL ×nL + Hi,l (t) Pi,l (t − 1)Hi,l (t)

Pi,l (t) =

∗ Pi,l (t

− 1) −

∗ Ki,l (t) Hi,l (t) Pi,l (t

− 1)



−1

(16) (17) (18)

ˆ i,l (t) = w ˆ i,l (t−1)−αPi,l (t)w ˆ i,l (t−1) + Ki,l (t)[d(t)−h(w ˆ i,l (t−1), x(t))], (19) w

where Hi,l is the nL × n(nl−1 +1) local gradient matrix. In this matrix, only one row associated with the considered neuron is nonzero for output layer L. Ki,l (t) is the (nl−1 + 1) × nL local Kalman gain. Pi,l (t) is the (nl−1 + 1) × (nl−1 + 1) local error covariance matrix. The follows. There  training process of the local TWDRLS algorithm is as  L are L n neurons (excepting input neurons). Hence, there are l l=2 l=2 nl sets of TWDRLS equations. We update the local weight vectors in accordance with a descending order of l and then an ascending order of i using (16)-(19). At each training stage, only the concerned local weight vector is updated and all other local weight vectors remain unchanged. In the global TWDRLS, the complexity mainly comes from the computing the inverse of the M -dimensional matrix (IM×M + αP (t − 1)). This complexity 3 is equal to O(M complexity is equal to T CCglobal =  ). So, the computational

3   L 3 O(M ) = O . Since the size of the matrix is M × M , l=2 nl (nl−1 + 1) the space complexity (storage requirement) is equal to T CSglobal = O(M 2 ) = 

2  L O . l=2 nl (nl−1 + 1) From (16), the computational cost of local TWDRLS algorithm mainly comes from the inversion of an (nl−1 + 1) × (nl−1 + 1) matrix. In this way, the

460

C.S. Leung, K.-W. Wong, and Y. Xu

computational complexity of each set of local TWDRLS equations is equal to O((nl−1 +1)3 ) and the corresponding space complexity is equal to O((nl−1 +1)2 ). Hence, the total of the local TWDRLS is given by computational complexity

L 3 T CClocal = O n (n + 1) and the space complexity (storage requirel−1 l=2 l 

L 2 ment) is equal to T CSlocal = O n (n + 1) . They are much smaller l l−1 l=2 than the computational and space complexities of the global case.

4

Simulations

Two problems, the generalized XOR and the sunspot data prediction, are considered. We use three-layers networks. The initial weights are small zero-mean independent identically distributed Gaussian random variables. The transfer function of hidden neurons is a hyperbolic tangent. Since the generalized XOR is a classification problem, output neurons are with the hyperbolic tangent function. For the sunspot data prediction problem, output neurons are with the linear activation function. The training for each problem is performed 10 times with different random initial weights. 4.1

Generalized XOR Problem

The generalized XOR problem is formulated as d = sign(x1 x2 ) with inputs in the range [−1, 1]. The network has 2 input neurons, 10 hidden neurons, and 1 output neuron. As a result, there are 41 weights. The training set and test set, shown in Figure 1, consists of 50 and 2,000 samples, respectively. The total number of training cycles is set to 200. In each cycle, training samples from the training set are feeded to the network one by one. The decision boundaries obtained from typical networks trained with both global and local TWDRLS and standard RSL algorithms are plotted in Figure 2.

(a) Training samples

(b) Test samples

Fig. 1. Training and test samples for the generalized XOR problems

The Local True Weight Decay Recursive Least Square Algorithm

461

Table 1. Computational and space complexities of the global and local TWDRLS algorithms for solving the generalized XOR problem Algorithm Computational complexity Space complexitty Global O(6.89 × 104 ) O(1.68 × 103 ) 3 Local O(1.60 × 10 ) O(2.21 × 102 )

(a) Global TWDRLS, α = 0

(b) Local TWDRLS, α = 0

(c) Global TWDRLS, α = 0.00178

(d) Local TWDRLS, α = 0.00178

Fig. 2. Decision boundaries of various trained networks for the generalized XOR problem. Note that when α = 0, the TWDRLS is identical to RLS.

From Figures 1 and 2, the decision boundaries obtained from the trained networks with TWDRLS algorithm are closer to the ideal ones than those with the standard RLS algorithm. Also, both local and global TWDRLS algorithms produce a similar shape of decision boundaries. Figure 3 summarizes the average test set false rates in the 10 runs. The average test set false rates obtained by global and local TWDRLS algorithms are usually lower than those obtained by the standard RLS algorithm over a wide range of regularization parameters. That means, both global and local TWDRLS algorithms can improve the generalization ability. In terms of average false rate, the performance of the local TWDRLS algorithm is quite similar to that of the global ones. The computational and space complexities for global and local algorithms are listed in Table 1. From Figure 3 and Table 1, we can conclude that

462

C.S. Leung, K.-W. Wong, and Y. Xu

Fig. 3. Average test set false rate of 10 runs for the generalized XOR problem

the performance of local TWDRLS is comparable to that of the global ones, and that its complexities are much smaller. Figure 3 indicates that the average test set false rate first decreases with the regularization parameter α and then increases with it. This shows that a proper selection of α will indeed improve the generalization ability of the network. On the other hand, we observe that the test set false rate becomes very high at large values of α, especially for the networks trained with global TWDRLS algorithm. This is due to the fact that when the value of α is too large, the weight decay effect is very substantial and the trained network cannot learn the target function. In order to further illustrate this, we plot in Figure 4 the decision boundary obtained from the network trained with global TWDRLS algorithm for α = 0.0178. The figure shows that the network has already converged when the decision boundary is still quite far from the ideal one. This is because when the value of α is too large, the weight decay effect is too strong. That means, the regularization parameter α cannot be too large otherwise the network cannot learn the target function. 4.2

Sunspot Data Prediction

The sunspot data from 1700 to 1979 are normalized to the range [0,1] and taken as the training and the test sets. Following the common practice, we divide the data into a training set (1700 − 1920) and two test sets, namely, Test-set 1 (1921 − 1955) and Test-set 2 (1956 − 1979). The sunspot series is rather nonstationary and Test-set 2 is atypical for the series as a whole. In the simulation, we assume that the series is generated from the following auto-regressive model, given by d(t) = ϕ(d(t − 1), · · · , d(t − 12)) + (t)

(20)

where (t) is noise and ϕ(·, · · · , ·) is an unknown nonlinear function. A network with 12 input neurons, 8 hidden neurons (with hyperbolic tangent activation

The Local True Weight Decay Recursive Least Square Algorithm

463

Table 2. Computational and space complexities of the global and local TWDRLS algorithms for solving the sunspot data prediction Algorithm Computational complexity Space complexitty Global O(1.44 × 106 ) O(1.28 × 104 ) 4 Local O(1.83 × 10 ) O(1.43 × 103 )

Fig. 4. Decision boundaries of a trained network with local TWDRLS where α = 0.0178. In this case, the value of the regularization parameter is too large. Hence, the network cannot form a good decision boundary.

(a) Test-set 1 average RMSE

(b) Test-set 2 average RMSE

Fig. 5. RMSE of networks trained by global and local TWDRLS algorithms. Note that when α = 0, the TWDRLS is identical to RLS.

function), and one output neuron (with linear activation function) is used for approximating ϕ(·, · · · , ·). The total number of training cycles is equal to 200. As this is a time series problem, the training samples are feeded to the network sequentially in each iteration. The criterion to evaluate the model performance

464

C.S. Leung, K.-W. Wong, and Y. Xu

is the mean squared error (RMSE) of the test set. The experiment are repeated 10 times with different initial weights. Figure 5 summarizes the average RMSE in 10 runs. The computational and space complexities for global and local algorithms are listed in Table 2. We observe from Figure 5 that over a wide range of the regularization parameter α, both global and local TWDRLS algorithms have greatly improved the generalization ability of the trained networks, especially for test-set 2 that is quite different from the training set. However, the test RMSE becomes very large at large values of α. The reasons are similar to those stated in the last subsection. This is because at large value of α, the weight decay effect is too strong and so the network cannot learn the target function. In most cases, the performance of the local training is found to be comparable to that of the global ones. Also, Table 2 shows that those complexities of the local training are much smaller than those of the global one.

5

Conclusion

We have investigated the problem of training the MFNN model using the TWDRLS algorithms. We derive a set of concise equations for the local TWDRLS algorithm. The computational complexity and the storage requirement are reduced considerably when using the local approach. Computer simulations indicate that both local and global TWDRLS algorithms can improve the generation ability of MFNNs. The performance of the local TWDRLS algorithm is comparable to that of the global ones.

Acknowledgement The work is supported by the Hong Kong Special Administrative Region RGC Earmarked Grant (Project No. CityU 115606).

References 1. Shah, S., Palmieri, F., Datum, M.: Optimal filtering algorithm for fast learning in feedforward neural networks. Neural Networks 5, 779–787 (1992) 2. Leung, C.S., Wong, K.W., Sum, J., Chan, L.W.: A pruning method for recursive least square algorithm. Neural Networks 14, 147–174 (2001) 3. Scalero, R., Tepedelelenlioglu, N.: Fast new algorithm for training feedforward neural networks. IEEE Trans. Signal Processing 40, 202–210 (1992) 4. Leung, C.S., Sum, J., Young, G., Kan, W.K.: On the kalman filtering method in neural networks training and pruning. IEEE Trans. Neural Networks 10, 161–165 (1999) 5. Leung, C.S., Tsoi, A.H., Chan, L.W.: Two regularizers for recursive least squared algorithms in feedforward multilayered neural networks. IEEE Trans. Neural Networks 12, 1314–1332 (2001) 6. Mosca, E.: Optimal Predictive and adaptive control. Prentice-Hall, Englewood Cliffs, NJ (1995)

The Local True Weight Decay Recursive Least Square Algorithm

465

7. Haykin, S.: Adaptive filter theory. Prentice-Hall, Englewood Cliffs, NJ (1991) 8. Mackay, D.: Bayesian interpolation. Neural Computation 4, 415–447 (1992) 9. Mackay, D.: A practical bayesian framework for backpropagation networks. Neural Computation 4, 448–472 (1992) 10. William H, H.: Applied numerical linear algebra. Prentice-Hall, Englewood Cliffs, NJ (1989)

Experimental Bayesian Generalization Error of Non-regular Models under Covariate Shift Keisuke Yamazaki and Sumio Watanabe Precision and Intelligence Laboratory, Tokyo Institute of Technology R2-5, 4259 Nagatsuta, Midori-ku, Yokohama, 226-8503 Japan {k-yam,swatanab}@pi.titech.ac.jp

Abstract. In the standard setting of statistical learning theory, we assume that the training and test data are generated from the same distribution. However, this assumption cannot hold in many practical cases, e.g., brain-computer interfacing, bioinformatics, etc. Especially, changing input distribution in the regression problem often occurs, and is known as the covariate shift. There are a lot of studies to adapt the change, since the ordinary machine learning methods do not work properly under the shift. The asymptotic theory has also been developed in the Bayesian inference. Although many effective results are reported on statistical regular ones, the non-regular models have not been considered well. This paper focuses on behaviors of non-regular models under the covariate shift. In the former study [1], we formally revealed the factors changing the generalization error and established its upper bound. We here report that the experimental results support the theoretical findings. Moreover it is observed that the basis function in the model plays an important role in some cases.

1

Introduction

The task of regression problem is to estimate the input-output relation q(y|x) from sample data, where x, y are the input and output data, respectively. Then, we generally assume that the input distribution q(x) is generating both of the training and test data. However, this assumption cannot be satisfied in practical situations, e.g., brain computer interfacing [2], bioinformatics [3], etc. The change of the input distribution from the training q0 (x) into the test q1 (x) is referred to as the covariate shift [4]. It is known that, under the covariate shift, the standard techniques in machine learning cannot work properly, and many efficient methods to tackle this issue are proposed [4,5,6,7]. In the Bayes estimation, Shimodaira [4] revealed the generalization error improved by the importance weight on regular cases. We formally clarified the behavior of the error in non-regular cases [1]. The result shows the generalization error is determined by lower order terms, which are ignored at the situation without the covariate shift. At the same time, it appeared that the calculation of these terms is not straightforward even in a simple regular example. To cope M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 466–476, 2008. c Springer-Verlag Berlin Heidelberg 2008 

Experimental Bayesian Generalization Error of Non-regular Models

467

with this problem, we also established the upper bound using the generalization error without the covariate shift. However, it is still open how to derive the theoretical generalization error in non-regular models. In this paper, we observe experimental results calculated with a Monte Carlo method and examine the theoretical upper bound on some non-regular models. Comparing the generalization error under the covariate shift to those without the shift, we investigate an effect of a basis function in the learning model and consider the tightness of the bound. In the next section, we define non-regular models, and summarize the Bayesian analyses for the models without and with the covariate shift. We show the experimental results in Section 3 and give discussions at the last.

2

Bayesian Generalization Errors with and without Covariate Shift

In this section, we summarize the asymptotic theory on the non-regular Bayes inference. At first, we define the non-regular case. Then, mathematical properties of non-regular models without the covariate shift are introduced [8]. Finally, we state the results of the former study [1], which clarified the generalization error under the shift. 2.1

Non-regular Models

Let us define the parametric learning model by p(y|x, w), where x, y and w are the input, output and parameter, respectively. When the true distribution r(y|x) is realized by the learning model, the true parameter w∗ exists, i.e., p(y|x, w∗ ) = r(y|x). The model is regular if w∗ is one point in the parameter space. Otherwise, non-regular; the true parameter is not one point but a set of parameters such that Wt = {w∗ : p(y|x, w∗ ) = r(y|x)}. For example, three-layer perceptrons are non-regular. Let the true distribution be a zero function with Gaussian noise   1 y2 r(y|x) = √ exp − , 2 2π and the learning models be a simple three-layer perceptron   1 (y − a tanh(bx))2 p(y|x, w) = √ exp − , 2 2π where the parameter w = {a, b}. It is easy to find that the true parameters are a set {a = 0}∪{b = 0} since 0×tanh(bx) = a×tanh 0 = 0. This non-regularity (also called non-identifiability) causes that the conventional statistical method cannot be applied to such models (We will mention the detail in Section 2.2). In spite of this difficulty for the analysis, the non-regular models, such as perceptrons, Gaussian mixtures, hidden Markov models, etc., are mainly employed in many information engineering fields.

468

2.2

K. Yamazaki and S. Watanabe

Properties of the Generalization Error without Covariate Shift

As we mentioned in the previous section, the conventional statistical manner does not work in the non-regular models. To cope with the issue, a method was developed based on algebraic geometry. Here, we introduce the summary. Hereafter, we denote the cases without and with the covariate shift subscript 0 and 1, respectively. Some of the functions with a suffix 0 will be replaced by those with 1 in the next section. Let {X n , Y n } = {X1 , Y1 , . . . , Xn , Yn } be a set of training samples that are independently and identically generated by the true distribution r(y|x)q0 (x). Let p(y|x, w) be a learning machine and ϕ(w) be an a priori distribution of parameter w. Then the a posteriori distribution/posterior is defined by p(w|X n , Y n ) =

n  1 p(Yi |Xi , w)ϕ(w), Z(X n , Y n ) i=1

where Z(X n , Y n ) =

  n

p(Yi |Xi , w)ϕ(w)dw.

(1)

i=1

The Bayesian predictive distribution is given by  n n p(y|x, X , Y ) = p(y|x, w)p(w|X n , Y n )dw. When the number of sample data is sufficiently large (n → ∞), the posterior has the peak at the true parameter(s). The posterior in a regular model is a Gaussian distribution, whose mean is asymptotically the parameter w∗ . On the other hand, the shape of the posterior in a non-regular model is not Gaussian because of Wt (cf. the right panel of Fig.10 in Section 3). We evaluate the generalization error by the average Kullback divergence from the true distribution to the predictive distribution:   r(y|x) 0 G0 (n)=EX r(y|x)q0 (x) log dxdy . n ,Y n p(y|x, X n , Y n ) In the standard statistical manner, we can formally calculate the generalization error by integrating the predictive distribution. This integration is viable based on the Gaussian posterior. Therefore, this method is applicable only to regular models. The following is one of the solutions for non-regular cases. The stochastic complexity [9] is defined by F (X n , Y n ) = − log Z(X n , Y n ), (2) which can be used for selecting an appropriate model or hyper-parameters. To analyze the behavior of the stochastic complexity, the following functions play important roles: 0 n n U 0 (n) = EX )] , n ,Y n [F (X , Y 0 where EX n ,Y n [·] stands for the expectation value over r(y|x)q0 (x) and

(3)

Experimental Bayesian Generalization Error of Non-regular Models

F (X n , Y n ) = F (X n , Y n ) +

n 

469

log r(Yi |Xi ).

i=1

The generalization error and the stochastic complexity are linked by the following equation [10]: G0 (n) = U 0 (n + 1) − U 0 (n).

(4)

When the learning machine p(y|x, w) can attain the true distribution r(y|x), the asymptotic expansion of F (n) is given as follows [8]. U 0 (n) = α log n − (β − 1) log log n + O(1).

(5)

The coefficients α and β are determined by the integral transforms of U 0 (n). More precisely, the rational number −α and natural number β are the largest pole and its order of  J(z) = H0 (w)z ϕ(w)dw,  r(y|x) H0 (w) = r(y|x)q0 (x) log dxdy. (6) p(y|x, w) J(z) is obtained by applying the inverse Laplace and Mellin transformations to exp[−U 0 (n)]. Combining Eqs.(5) and (4) immediately gives

α β−1 1 0 G (n) = − +o , n n log n n log n when G0 (n) has an asymptotic form. The coefficients α and β indicate the speed of convergence of the generalization error when the number of training samples is sufficiently large. When the learning machine cannot attain the true distribution (i.e., the model is misspecified), the stochastic complexity has an upper bound of the following asymptotic expression [11]. U 0 (n) ≤ nC + α log n − (β − 1) log log n + O(1),

(7)

where C is a non-negative constant. When the generalization error has an asymptotic form, combining Eqs.(7) and (4) gives

α β−1 1 G0 (n) ≤ C + − +o , (8) n n log n n log n where C is the bias. 2.3

Properties of the Generalization Error with Covariate Shift

Now, we introduce the results in [1]. Since the test data are distributed from r(y|x)q1 (x), the generalization error with the shift is defined by

470

K. Yamazaki and S. Watanabe

 1

G

0 (n)=EX n ,Y n

 r(y|x) r(y|x)q1 (x) log dxdy . p(y|x, X n , Y n )

(9)

We need the function similar to (3) given by 1 U 1 (n) = EX EX n−1 ,Y n−1 [F (X n , Y n )]. n ,Yn

(10)

Then, the variant of (4) is obtained as G1 (n) = U 1 (n + 1) − U 0 (n).

(11)

When we assume that G1 (n) has an asymptotic expansion and converges to a constant and that U i (n) has the following asymptotic expansion 1 di U i (n) = ai n + bi log n + · · · +ci + +o , n n   i (n) TH

0

i (n) TL

1

it holds that G (n) and G (n) are expressed by 1 b0 G0 (n) = a0 + +o , n n 1 b0 + (d1 − d0 ) G1 (n) = a0 + (c1 − c0 ) + +o , (12) n n 1 0 and that TH (n) = TH (n). Note that b0 = α. The factors c1 −c0 and d1 −d0 determine the difference of the errors. We have also obtained that the generalization error G1 (n) has an upper bound G1 (n) ≤ M G0 (n), if the following condition is satisfied M ≡ max

x∼q0 (x)

3



 q1 (x) < ∞. q0 (x)

(13)

(14)

Experimental Generalization Errors in Some Toy Models

Even though we know the factors causing the difference between G1 (n) and G0 (n) according to Eq.(12), it is not straightforward to calculate the lower order terms in TLi (n). More precisely, the solvable model is restricted to find the constant and decreasing factors ci , di in the increasing function U i (n). This implies to reveal the analytic expression of G1 (n) is still open issue in non-regular models. Here, we calculate G1 (n) with experiments and observe the behavior. A non-regular model requires the sampling from the non-Gaussian posterior in the Bayes inference. We use the Markov Chain Monte Carlo (MCMC) method to execute this task [12]. In the following examples, we use the common notations: the true distribution is defined by   1 (y − g(x))2 r(y|x) = √ exp − , 2 2π

Experimental Bayesian Generalization Error of Non-regular Models

-10

-5

0

5

10

20

15

-5

0

5

10

20

15

-5

0

5

10

15

0

5

10

15

20

-10

-10

-5

0

5

10

15

20

20

Fig. 7. (μ1 , σ1 ) = (10, 1)

-10

-5

0

5

10

15

20

Fig. 8. (μ1 , σ1 )=(10, 0.5)

-5

0

5

10

15

20

Fig. 3. (μ1 , σ1 ) = (0, 2)

-10

-5

0

5

10

15

20

Fig. 6. (μ1 , σ1 ) = (2, 2)

Fig. 5. (μ1 , σ1 ) = (2, 0.5)

Fig. 4. (μ1 , σ1 ) = (2, 1)

-10

-5

Fig. 2. (μ1 , σ1 ) = (0, 0.5)

Fig. 1. (μ1 , σ1 ) = (0, 1)

-10

-10

471

-10

-5

0

5

10

15

20

Fig. 9. (μ1 , σ1 ) = (10, 2)

The training and test distributions

the learning model is given by

  1 (y − f (x, w))2 p(y|x, w) √ exp − , 2 2π

the prior ϕ(w) is a standard normal distribution, and the input distribution is of the form   1 (x − μi )2 qi (x) = √ exp − (i = 0, 1). 2σi2 2πσi The training input distribution has (μ0 , σ0 ) = (0, 1), and there are nine test distributions, where each one has the mean and variance as the combination between μ1 = {0, 2, 10} and σ1 = {1, 0.5, 2} (cf. Fig.1-9). Note that the case in Fig.1 corresponds to q0 (x). As for the experimental setting, the number of traning samples is n, the number of test samples is ntest , the number of parameter samples distributed from the posterior with the MCMC method is np , and the number of samples to have the expectation EX n ,Y n [·] is nD . In the mathematical expressions, n np 1  p(y|x, wj ) i=1 p(Yi |Xi , wj )ϕ(wj ) n n   p(y|x, X , Y )  , n p np j=1 n1p nk=1 i=1 p(Yi |Xi , wk )ϕ(wk )  nD  ntest 1  1  r(yj |xj ) G1 (n)  log , nD i=1 ntest j=1 p(yj |xj , Di )

0.4

0.49

0.31

0.22

0.13

0.04

-0.05

-0.14

-0.23

-0.32

-0.5

K. Yamazaki and S. Watanabe

-0.41

472

4 cy en qu fre

2500 2000 1500 1000 500 0

4

−4

0

b

3

2

2

−2

1

a

0

−2

0 2

-1

-2

4

-3 -3

-2

-1

0

1

2

3

−4

4

Fig. 10. The sampling from the posteriors. The left-upper panel shows the histogram of a for the first model. The left-middle one is the histogram of a3 for the third model. The left-lower one is the point diagram of (a, b) for the second model, and the right one is its histogram.

where Di = {Xi1 , Yi1 , · · · , Xin , Yin } stands for the ith set of training data, and Di and (xj , yj ) in G1 (n) are taken from q0 (x)r(y|x) and q1 (x)r(y|x), respectively. The experimental parameters were as follows: n = 500, ntest = 1000, np = 10000, nD = 100. Example 1 (Lines with Various Parameterizations) g(x) = 0, f1 (x, a) = ax, f2 (x, a, b) = abx, f3 (x, a) = a3 x, where the true is the zero function, the learning functions are lines with the gradient a, ab and a3 . In this example, all learning functions belong to the same function class though the second model is non-regular (Wt = {a = 0} ∪ {b = 0}) and the third one has the non-Gaussian posterior. The gradient parameters are taken from the posterior depicted by Fig.10. Table 1 summarizes the results. The first row indicates the pairs (μ1 , σ1 ), and the rest does the experimental average generalization errors. G1 [fi ] stands for the error of the model with fi . M G0 [fi ] is the upper bound in each case according to Eq.(13). Note that there are some blanks in the row because of the condition Eq.(14). To compare G1 [f3 ] with G1 [f1 ], the last row shows the values 3 × G1 [f3 ] of each change. Since it is regular, the first model has theoretical results:



1 1 R 1 μ2 + σ12 G0 (n) = +o , G1 (n) = +o , R = 12 . 2n n log n 2n n log n μ0 + σ02 ‘th G1 [f1 ]’ in Table 1 is this theoretical result.

Experimental Bayesian Generalization Error of Non-regular Models

473

Table 1. Average generalization errors in Example 1 (μ1 , σ1 ) 1

th G [f1 ] G1 [f1 ] G1 [f2 ] G1 [f3 ] MG0 [f1 ] MG0 [f2 ] MG0 [f3 ]

(0,1)= G0 (0,0.5)

(0,2)

(2,1)

0.001 0.00025 0.004 0.005 0.001055 0.000239 0.004356 0.006162 0.000874 0.000170 0.003466 0.004523 0.000394 0.000059 0.001475 0.002374 — — —

0.002000 0.001356 0.000667

— — —

— — —

(2,0.5)

(2,2)

(10,1)

(10,0.5)

(10,2)

0.00425 0.008 0.101 0.10025 0.104 0.005107 0.009532 0.109341 0.106619 0.108042 0.003670 0.006669 0.079280 0.078802 0.080180 0.001912 0.003736 0.040287 0.038840 0.039682 0.028784 0.019521 0.009595

— — —

— — —

1.79×1026 1.22×1026 5.98×1025

— — —

3 × G1 [f3 ] 0.001182 0.000177 0.004425 0.007122 0.005736 0.011208 0.120861 0.116520 0.119046

Table 2. Average generalization errors in Example 2 (μ1 , σ1 ) 1

G [f4 ] G1 [f5 ] G1 [f6 ] MG0 [f4 ] MG0 [f5 ] MG0 [f5 ]

(0,1)= G0 (0,0.5)

(0,2)

(2,1)

(2,0.5)

(2,2)

(10,1)

(10,0.5)

(10,2)

0.000688 0.000260 0.002312 0.002977 0.003094 0.003423 0.013640 0.012769 0.012204 0.000251 0.000103 0.000934 0.001116 0.001291 0.001318 0.004350 0.003729 0.003743 0.000146 0.000062 0.000626 0.000705 0.000875 0.000918 0.002896 0.002357 0.002489 — — —

0.001356 0.000667 0.000400

— — —

— — —

0.019521 0.009595 0.005757

— — —

— — —

1.22×1026 5.98×1025 3.59×1025

— — —

3 × G1 [f5 ] 0.000753 0.000309 0.002802 0.003348 0.003873 0.003954 0.013050 0.011187 0.011229 5 × G1 [f6 ] 0.000730 0.000310 0.003130 0.003525 0.004375 0.004590 0.014480 0.011785 0.012445

We can find that ‘th G1 [f1 ]’ are very close to G1 [f1 ] in spite of the fact that the theoretical values are established in asymptotic cases. Based on this fact, the accuracy of experiments can be evaluated to compare them. As for f2 and f3 , they do not have any comparable theoretical value except for the upper bound. We can confirm that every value in G1 [f2 , f3 ] is actually smaller than the bound. Example 2 (Simple Neural Networks). Let assume that the true is the zero function, and the learning models are three-layer perceptrons: g(x) = 0, f4 (x, a, b) = a tanh(bx), f5 (x, a, b) = a3 tanh(bx), f6 (x, a, b) = a5 tanh(bx). Table 2 shows the results. In this example, we can also confirm that the bound works. Combining the results in the previous example, the bound tends to be tight when μ1 is small. As a matter of fact, the bound holds in small sample cases, i.e., the number of training data n does not have to be sufficiently large. Though we omit it because of the lack of space, the bound is always larger than the experimental results in n = 100, 200, . . . , 400. The property of the bound will be discussed in the next section.

4

Discussions

First, let us confirm if the sampling from the posterior was successfully done by the MCMC method. Based on the algebraic geometrical method, the coefficients of G0 (n) are derived in the models (cf. Table 3). As we mentioned, f2 , f4 and

474

K. Yamazaki and S. Watanabe Table 3. The coefficients of generalization error without the covariate shift f1 f2 , f4 f3 , f5 f6 α 1/2 1/2 1/6 1/10 β 1 2 1 1

f3 , f5 have the same theoretical error. According to the examples in the previous section, we can compare the theoretical value to the experimental one, G0 (n)[f1 ] = 0.001  0.001055, G0(n)[f2 , f4 ] = 0.000678  0.000874, 0.000688 G0 (n)[f3 , f5 ] = 0.000333  0.000394, 0.000251, G0(n)[f6 ] = 0.0002  0.000146. In the sense of the generalization error, the MCMC method worked well though there is some fluctuation in the results. Note that it is still open how to evaluate the method. Here we measured by the generalization error since the theoretical value is known in G0 (n). However, this index is just a necessary condition. To develop an evaluation of the selected samples is our future study. Next, we consider the behavior of G1 (n). In the examples, the true function was commonly the zero function g(x) = 0. It is an important case to learn the zero function because we often prepare an enough rich Kmodel in practice. Then, the learning function will be set up as f (x, w) = i=k t(w1k )h(x, w2k ), where h is the basis function t is the parameterization for its weight, and w = {w11 , w21 , w12 , w22 , . . . , w1K , w2K }. Note that many practical models are included in this expression. According to the redundancy of the function, some of h(x, w2k ) learn the zero function. Our examples provided the simplest situations and highlighted the effect of non-regularity in the learning models. The errors G0 (n) and G1 (n) are generally expressed as

α β−1 1 G0 (n) = − +o , n n log n n log n

α β−1 1 G1 (n) = R1 − R2 +o , n n log n n log n where R1 , R2 depend on f, g, q0 , and q1 . R1 and R2 cause the difference between G0 and G1 in this expression. In Eq. (12), the coefficient of 1/n is given by b0 + (d1 − d0 ). So b0 + (d1 − d0 ) d1 − d0 R1 = =1+ . b0 α Let us denote A B as “A is the only factor to determine a value of B”. As mentioned above, f, g, q0 , q1 R1 , R2 . Though f, g α, β (cf. around Eqs.(5)-(6)), we should emphasize that α, β, q0 , q1 R1 , R2 .

Experimental Bayesian Generalization Error of Non-regular Models

475

This fact is easily confirmed by comparing f2 to f4 (also f3 to f5 ). It holds that G1 (n)[f2 ] = G1 (n)[f4 ] for all q1 although they have the same α and β (G0 (n)[f2 ] = G0 (n)[f4 ]). Thus α and β are not enough informative to describe R1 and R2 . Comparing the values in G1 [f2 ] to the ones in G1 [f4 ], the basis function (x in f2 and tanh(bx) in f4 ) seems to play an important role. To clarify the effect of basis functions, let us fix the function class. Examples 1 and 2 correspond to h(x, w2 ) = x and h(x, w2 ) = tanh(bx), respectively. The values of G1 [f1 ] and 3 × G1 [f3 ] (also 3 × G1 [f5 ] and 5 × G1 [f6 ]) can be regarded as the same in any covariate shift. This implies h, g, q0 , q1 R1 , i.e. the parameterization t(w1 ) will not affect R1 . Instead, it affects the nonregularity or the multiplicity and decides α and β. Though it is an unclear factor, the influence of R2 does not seem as large as R1 . Last, let us analyze properties of the upper bound M G0 (n). According to the above discussion, it holds that R  G1 /G0 ≤ M. The ratio G1 /G0 basically depends on g, h, q0 and q1 . However, M is determined by only the training and test input distributions, q0 , q1 M . Therefore this bound gives the worst case evaluation in any g and h. Considering the tightness of the bound, we can still improve it based on the relation between the true and learning functions.

5

Conclusions

In the former study, we have got the theoretical generalization error and its upper bound under the covariate shift. This paper showed that the theoretical value is supported by the experiments in spite of the fact that it is established under an asymptotic case. We observed the tightness of the bound and discussed an effect of basis functions in the learning models. In this paper, the non-regular models are simple lines and neural networks. It is an interesting issue to investigate more general models. Though we mainly considered the amount of G1 (n), the computational cost for the MCMC method strongly connects to the form of the learning function f . It is our future study to take account of the cost in the evaluation.

Acknowledgements The authors would like to thank Masashi Sugiyama, Motoaki Kawanabe, and Klaus-Robert M¨ uller for fruitful discussions. The software to calculate the MCMC method and technical comments were provided by Kenji Nagata. This research partly supported by the Alexander von Humboldt Foundation, and MEXT 18079007.

476

K. Yamazaki and S. Watanabe

References 1. Yamazaki, K., Kawanabe, M., Wanatabe, S., Sugiyama, M., M¨ uller, K.R.: Asymptotic bayesian generalization error when training and test distributions are different. In: Proceedings of the 24th International Conference on Machine Learning, pp. 1079–1086 (2007) 2. Wolpaw, J.R., Birbaumer, N., McFarland, D.J., Pfurtscheller, G., Vaughan, T.M.: Brain-computer interfaces for communication and control. Clinical Neurophysiology 113(6), 767–791 (2002) 3. Baldi, P., Brunak, S., Stolovitzky, G.A.: Bioinformatics: The Machine Learning Approach. MIT Press, Cambridge (1998) 4. Shimodaira, H.: Improving predictive inference under covariate shift by weighting the log-likelihood function. Journal of Statistical Planning and Inference 90, 227– 244 (2000) 5. Sugiyama, M., M¨ uller, K.R.: Input-dependent estimation of generalization error under covariate shift. Statistics & Decisions 23(4), 249–279 (2005) 6. Sugiyama, M., Krauledat, M., M¨ uller, K.R.: Covariate shift adaptation by importance weighted cross validation. Journal of Machine Learning Research 8 (2007) 7. Huang, J., Smola, A., Gretton, A., Borgwardt, K.M., Sch¨ olkopf, B.: Correcting sample selection bias by unlabeled data. In: Sch¨ olkopf, B., Platt, J., Hoffman, T. (eds.) Advances in Neural Information Processing Systems, vol. 19, MIT Press, Cambridge, MA (2007) 8. Watanabe, S.: Algebraic analysis for non-identifiable learning machines. Neural Computation 13(4), 899–933 (2001) 9. Rissanen, J.: Stochastic complexity and modeling. Annals of Statistics 14, 1080– 1100 (1986) 10. Watanabe, S.: Algebraic analysis for singular statistical estimation. In: Watanabe, O., Yokomori, T. (eds.) ALT 1999. LNCS (LNAI), vol. 1720, pp. 39–50. Springer, Heidelberg (1999) 11. Watanabe, S.: Algebraic information geometry for learning machines with singularities. Advances in Neural Information Processing Systems 14, 329–336 (2001) 12. Ogata, Y.: A monte carlo method for an objective bayesian procedure. Ann. Inst. Statis. Math. 42(3), 403–433 (1990)

Using Image Stimuli to Drive fMRI Analysis David R. Hardoon1 , Janaina Mour˜ ao-Miranda2, Michael Brammer2 , and John Shawe-Taylor1 1

The Centre for Computational Statistics and Machine Learning Department of Computer Science University College London Gower St., London WC1E 6BT {D.Hardoon,jst}@cs.ucl.ac.uk 2 Brain Image Analysis Unit Centre for Neuroimaging Sciences (PO 89) Institute of Psychiatry, De Crespigny Park London SE5 8AF {Janaina.Mourao-Miranda,Michael.Brammer}@iop.kcl.ac.uk

Abstract. We introduce a new unsupervised fMRI analysis method based on Kernel Canonical Correlation Analysis which differs from the class of supervised learning methods that are increasingly being employed in fMRI data analysis. Whereas SVM associates properties of the imaging data with simple specific categorical labels, KCCA replaces these simple labels with a label vector for each stimulus containing details of the features of that stimulus. We have compared KCCA and SVM analyses of an fMRI data set involving responses to emotionally salient stimuli. This involved first training the algorithm ( SVM, KCCA) on a subset of fMRI data and the corresponding labels/label vectors, then testing the algorithms on data withheld from the original training phase. The classification accuracies of SVM and KCCA proved to be very similar. However, the most important result arising from this study is that KCCA in able in part to extract many of the brain regions that SVM identifies as the most important in task discrimination blind to the categorical task labels. Keywords: Machine learning methods, Kernel canonical correlation analysis, Support vector machines, Classifiers, Functional magnetic resonance imaging data analysis.

1

Introduction

Recently, machine learning methodologies have been increasingly used to analyse the relationship between stimulus categories and fMRI responses [1,2,3,4,5,6,7,8, 9,10]. In this paper, we introduce a new unsupervised machine learning approach to fMRI analysis, in which the simple categorical description of stimulus type (e.g. type of task) is replaced by a more informative vector of stimulus features. We compare this new approach with a standard Support Vector Machine (SVM) analysis of fMRI data using a categorical description of stimulus type. M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 477–486, 2008. c Springer-Verlag Berlin Heidelberg 2008 

478

D.R. Hardoon et al.

The technology of the present study originates from earlier research carried out in the domain of image annotation [11], where an image annotation methodology learns a direct mapping from image descriptors to keywords. Previous attempts at unsupervised fMRI analysis have been based on Kohonen selforganising maps, fuzzy clustering [12] and nonparametric estimation methods of the hemodynamic response function, such as the general method described in [13]. [14] have reported an interesting study which showed that the discriminability of PCA basis representations of images of multiple object categories is significantly correlated with the discriminability of PCA basis representation of the fMRI volumes based on category labels. The current study differs from conventional unsupervised approaches in that it makes use of the stimulus characteristics as an implicit representation of a complex state label. We use kernel Canonical Correlation Analysis (KCCA) to learn the correlation between an fMRI volume and its corresponding stimulus. Canonical correlation analysis can be seen as the problem of finding basis vectors for two sets of variables such that the correlations of the projections of the variables onto corresponding basis vectors are maximised. KCCA first projects the data into a higher dimensional feature space before performing CCA in the new feature space. CCA [15, 16] and KCCA [17] have been used in previous fMRI analysis using only conventional categorical stimulus descriptions without exploring the possibility of using complex characteristics of the stimuli as the source for feature selection from the fMRI data. The fMRI data used in the following study originated from an experiment in which the responses to stimuli were designed to evoke different types of emotional responses, pleasant or unpleasant. The pleasant images consisted of women in swimsuits while the unpleasant images were a collection of images of skin diseases. Each stimulus image was represented using Scale Invariant Feature Transformation (SIFT) [18] features. Interestingly, some of the properties of the SIFT representation have been modeled on the properties of complex neurons in the visual cortex. Although not specifically exploited in the current paper, future studies may be able to utilize this property to probe aspects of brain function such as modularity. In the current study, we present a feasibility study of the possibility of generating new activity maps by using the actual stimuli that had generated the fMRI volume. We have shown that KCCA is able to extract brain regions identified by supervised methods such as SVM in task discrimination and to achieve similar levels of accuracy and discuss some of the challenges in interpreting the results given the complex input feature vectors used by KCCA in place of categorical labels. This work is an extension of the work presented in [19]. The paper is structured as follows. Section 2 gives a review of the fMRI data acquisition as well as the experimental design and the pre-processing. These are followed by a brief description of the scale invariant feature transformation in Section 2.1. The SVM is briefly described in Section 2.2 while Section 2.2 elaborates on the KCCA methodology. Our results in Section 3. We conclude with a discussion in Section 4.

Using Image Stimuli to Drive fMRI Analysis

2

479

Materials and Methods

Due to the lack of space we refer the reader to [10] for a detailed account of the subject, data acquisition and pre-processing applied to the data as well as to the experimental design. 2.1

Scale Invariant Feature Transformation

Scale Invariant Feature Transformation (SIFT) was introduced by [18] and shown to be superior to other descriptors [20]. This is due to the SIFT descriptors being designed to be invariant to small shifts in position of salient (i.e. prominent) regions. Calculation of the SIFT vector begins with a scale space search in which local minima and maxima are identified in each image (so-called key locations). The properties of the image at each key location are then expressed in terms of gradient magnitude and orientation. A canonical orientation is then assigned to each key location to maximize rotation invariance. Robustness to reorientation is introduced by representing local image regions around key voxels in a number of orientations. A reference key vector is then computed over all images and the data for each image are represented in terms of distance from this reference. Interestingly, some of the properties of the SIFT representation have been modeled on the properties of complex neurons in the visual cortex. Although not specifically exploited in the current paper, future studies may be able to utilize this property to probe aspects of brain function such as modularity. Image Processing. Let fil be the SIFT features vector for image i where l is the number of features. Each image i has a different number of SIFT features l, making it difficult to directly compare two images. To overcome this problem we apply K-means to cluster the SIFT features into a uniform frame. Using K-means clustering we find K classes and their respective centers oj where j = 1, . . . , K. The feature vector xi of an image stimuli i is K dimensional with j’th component xi,j . The feature vectors is computed as the Gaussian measure of the minimal distance between the SIFT features fil to the centre oj . This can be represented as   − minv∈f l d(v,oj )2

xi,j = exp

i

(1)

where d(., .) is the Euclidean distance. The number of centres is set to be the smallest number of SIFT features computed (found to be 300). Therefore after processing each image, we will have a 300 dimensional feature vector representing its relative distance from the cluster centres. 2.2

Methods

Support Vector Machines. Support vector machines [21] are kernel-based methods that find functions of the data that facilitate classification. They are derived from statistical learning theory [22] and have emerged as powerful tools for statistical pattern recognition [23]. In the linear formulation a SVM finds,

480

D.R. Hardoon et al.

during the training phase, the hyperplane that separates the examples in the input space according to their class labels. The SVM classifier is trained by providing examples of the form (x, y) where x represents a input and y it’s class label. Once the decision function has been learned from the training data it can be used to predict the class of a new test example. We used a linear kernel SVM that allows direct extraction of the weight vector as an image. A parameter C, that controls the trade-off between training errors and smoothness was fixed at C = 1 for all cases (default value).1 Kernel Canonical Correlation Analysis. Proposed by Hotelling in 1936, Canonical Correlation Analysis (CCA) is a technique for finding pairs of basis vectors that maximise the correlation between the projections of paired variables onto their corresponding basis vectors. Correlation is dependent on the chosen coordinate system, therefore even if there is a very strong linear relationship between two sets of multidimensional variables this relationship may not be visible as a correlation. CCA seeks a pair of linear transformations one for each of the paired variables such that when the variables are transformed the corresponding coordinates are maximally correlated. Consider the linear combination x = wa x and y = wb y. Let x and y be two random variables from a multi-dimensional distribution, with zero mean. The maximisation of the correlation between x and y corresponds to solving maxwa ,wb ρ = wa Cab wb subject to wa Caa wa = wb Cbb wb = 1. Caa and Cbb are the non-singular within-set covariance matrices and Cab is the between-sets covariance matrix. We suggest using the kernel variant of CCA [24] since due to the linearity of CCA useful descriptors may not be extracted from the data. This may occur as the correlation could exist in some non linear relationship. The kernelising of CCA offers an alternate solution by first projecting the data into a higher dimensional feature space φ : x = (x1 , . . . , xn ) → φ(x) = (φ1 (x), . . . , φN (x)) (N ≥ n) before performing CCA in the new feature space. Given the kernel functions κa and κb let Ka = Xa Xa and Kb = Xb Xb be the kernel matrices corresponding to the two representations of the data, where Xa is the matrix whose rows are the vectors φa (xi ), i = 1, . . . ,  from the first representation while Xb is the matrix with rows φb (xi ) from the second representation. The weights wa and wb can be expressed as a linear combination of the training examples wa = Xa α and wb = Xb β. Substituting into the primal CCA equation gives the optimisation maxα,β ρ = α Ka Kb β subject to α K2a α = β  K2b β = 1. This is the dual form of the primal CCA optimisation problem given above, which can be cast as a generalised eigenvalue problem and for which the first k generalised eigenvectors can be found efficiently. Both CCA and KCCA can be formulated as an eigenproblem. The theoretical analysis shown in [25,26] suggests the need to regularise kernel CCA as it shows that the quality of the generalisation of the associated pattern function is controlled by the sum of the squares of the weight vector norms. We 1

The LibSVM toolbox for Matlab was used to perform the classifications http://www.csie.ntu.edu.tw/∼cjlin/libsvm/

Using Image Stimuli to Drive fMRI Analysis

481

refer the reader to [25, 26] for a detailed analysis and the regularised form of KCCA. Although there are advantages in using kernel CCA, which have been demonstrated in various experiments across the literature. We must clarify that in this particular work, as we are using a linear kernel in both views, regularised CCA is the same as regularised linear KCCA (since the former and latter are linear). Although using KCCA with a linear kernel has advantages over CCA, the most important of which is in our case speed, together with the regularisation.2 Using linear kernels as to allow the direct extraction of the weights, KCCA performs the analysis by projecting the fMRI volumes into the found semantic space defined by the eigenvector corresponding to the largest correlation value (these are outputted from the eigenproblem). We classify a new fMRI volume as follows; Let αi be the eigenvector corresponding to the largest eigenvalue, and let φ(ˆ x) be the new volume. We project the fMRI into the semantic space w = Xa αi (these are the training weights, similar to that of the SVM) and using the weights we are able to classify the new example as w ˆ = φ(ˆ x)w where w ˆ is a weighted value (score) for the new volume. The score can be thresholded to allocate a category to each test example. To avoid the complications of finding a threshold, we zeromean the outputs and threshold the scores at zero, where w ˆ < 0 will be associated with unpleasant (a label of −1) and w ˆ ≥ 0 will be associated with pleasant (a label of 1). We hypothesis that KCCA is able to derive additional activities that may exist a-priori, but possibly previously unknown, in the experiment. By projecting the fMRI volumes into the semantic space using the remaining eigenvectors corresponding to lower correlation values. We have attempted to corroborate this hypothesis on the existing data but found that the additional semantic features that cut across pleasant and unpleasant images did not share visible attributes. We have therefore confined our discussion here to the first eigenvector.

3

Results

Experiments were run on a leave-one-out basis where in each repeat a block of positive and negative fMRI volumes was withheld for testing. Data from the 16 subjects was combined. This amounted, per run, in 1330 training and 14 testing fMRI volumes, each set evenly split into positive and negative volumes (these pos/neg splits were not known to KCCA but simply ensured equal number of images with both types of emotional salience). The analyses were repeated 96 times. Similarly, we run a further experiment of leave-subject-out basis where 15 subjects were combined for training and one left for testing. This gave a sum total of 1260 training and 84 testing fMRI volumes. The analyses was repeated 16 times. The KCCA regularisation parameter was found using 2-fold cross validation on the training data. Initially we describe the fMRI activity analysis. After training the SVM we are able to extract and display the SVM weights as a representation of the brain 2

The KCCA toolbox used was from http://homepage.mac.com/davidrh/Code.html

482

D.R. Hardoon et al.

regions important in the pleasant/unpleasant discrimination. A thorough analysis is presented in [10]. We are able to view the results in Figures 1 and 2 where in both figures the weights are not thresholded and show the contrast between viewing Pleasant vs. Unpleasant. The weight value of each voxel indicates the importance of the voxel in differentiating between the two brain states. In Figure 1 the unthresholded SVM weight maps are given. Similarly with KCCA, once learning the semantic representation we are able to project the fMRI data into the learnt semantic feature space producing the primal weights. These weights, like those generated from the SVM approach, could be considered as a representation of the fMRI activity. Figure 2 displays the KCCA weights. In Figure 3 the unthresholded weights values for the KCCA approach with the hemodynamic function applied to the image stimuli (i.e. applied to the SIFT features prior to analysis) are displayed. The hemodynamic response function is the impulse response function which is used to model the delay and dispersion of hemodynamic responses to neuronal activation [27]. The application of the hemodynamic function to the images SIFT features allows for the reweighting of the image features according to the computed delay and dispersion model. We compute the hemodynamic function with the SPM2 toolbox with default parameter settings. As the KCCA weights are not driven by simple categorical image descriptors (pleasant/unpleasant) but by complex image feature vectors it is of great interest that many regions, especially in the visual cortex, found by SVM are also highlighted by the KCCA. We interpret this similarity as indicating that many important components of the SIFT feature vector are associated with pleasant/unpleasant discrimination. Other features in the frontal cortex are much less reproducible between SVM and KCCA indicting that many brain regions detect image differences not rooted in the major emotional salience of the images. In order to validate the activity patterns found in Figure 2 we show that the learnt semantic space can be used to correctly discriminate withheld (testing) fMRI volumes. We also give the 2−norm error to provide an indication as to

Fig. 1. The unthresholded weight values for the SVM approach showing the contrast between viewing Pleasant vs. Unpleasant. We use the blue scale for negative (Unpleasant) values and the red scale for the positive values (Pleasant). The discrimination analysis on the training data was performed with labels (+1/ − 1).

Using Image Stimuli to Drive fMRI Analysis

483

Fig. 2. The unthresholded weight values for the KCCA approach showing the contrast between viewing Pleasant vs. Unpleasant. We use the blue scale for negative (Unpleasant) values and the red scale for the positive values (Pleasant). The discrimination analysis on the training data was performed without labels. The class discrimination is automatically extracted from the analysis.

Fig. 3. The unthresholded weight values for the KCCA approach with the hemodynamic function applied to the image stimuli showing the contrast between viewing Pleasant vs. Unpleasant. We use the blue scale for negative (Unpleasant) values and the red scale for the positive values (Pleasant).

the quality of the patterns found between the fMRI volumes and image stimuli from the testing set by Ka α − Kb β2 (normalised over the number of volumes and analyses repeats). The latter is especially important when the hemodynamic function has been applied to the image stimuli as straight forward discrimination is no longer possible to compare with. Table 1 shows the average and median performance of SVM and KCCA on the testing of pleasant and unpleasant fMRI blocks for the leave-two-block-out experiment. Our proposed unsupervised approach had achieved an average accuracy of 87.28%, slightly less than the 91.52% of the SVM. Although, both methods had the same median accuracy of 92.86%. The results of the leavesubject-out experiment are given in Table 2, where our KCCA has achieved an average accuracy of 79.24% roughly 5% less than the supervised SVM method. In both tables the Hemodynamic Function is abbreviated as HF. We are able to observe in both tables that the quality of the patterns are better than random. The results demonstrate that the activity analysis is meaningful. To further confirm the validity of the methodology we repeat the experiments with the

484

D.R. Hardoon et al.

Table 1. KCCA & SVM results on the leave-two-block-out experiment. Average and median performance over 96 repeats. The value represents accuracy, hence higher is better. For norm−2 error lower is better. Method Average Median Average  · 2 error Median  · 2 error KCCA 87.28 92.86 0.0048 0.0048 SVM 91.52 92.86 Random KCCA 49.78 50.00 0.0103 0.0093 Random SVM 52.68 50.00 KCCA with HF 0.0032 0.0031 Random KCCA with HF 1.1049 0.9492

Table 2. KCCA & SVM results on the leave-one-subject-out experiment. Average and median performance over 16 repeats. The value represents accuracy, hence higher is better. For norm−2 error lower is better. Method Average Median Average  · 2 error Median  · 2 error KCCA 79.24 79.76 0.0025 0.0024 SVM 84.60 86.90 Random KCCA 48.51 47.62 0.0052 0.0044 Random SVM 48.88 48.21 KCCA with HF 0.0016 0.0015 Random KCCA with HF 0.5869 0.0210

image stimuli randomised, hence breaking the relationship between fMRI volume and stimuli. Table 1 and 2 KCCA and SVM both show performance equivalent to the performance of a random classifier. It is also interesting to observe that when applying the hemodynamic function the random KCCA is substantially different, and worse than, the non random KCCA. Implying that the spurious correlations are found.

4

Discussion

In this paper we present a novel unsupervised methodology for fMRI activity analysis in which a simple categorical description of a stimulus type is replaced by a more informative vector of stimulus (SIFT) features. We use kernel canonical correlation analysis using an implicit representation of a complex state label to make use of the stimulus characteristics. The most interesting aspect of KCCA is its ability to extract visual regions very similar to those found to be important in categorical image classification using supervised SVM. KCCA “finds” areas in the brain that are correlated with the features in the SIFT vector regardless of the stimulus category. Because many features of the stimuli were associated with the pleasant/unpleasant categories we were able to use the KCCA results to classify the fMRI images between these categories. In the current study it is difficult to address the issue of modular versus distributed neural coding as the complexity of the stimuli (and consequently of the SIFT vector) is very high.

Using Image Stimuli to Drive fMRI Analysis

485

A further interesting possible application of KCCA relates to the detection of “inhomogeneities” in stimuli of a particular type (e.g happy/sad/disgusting emotional stimuli). If KCCA analysis revealed brain regions strongly associated with substructure within a single stimulus category this could be valuable in testing whether a certain type of image was being consistently processed by the brain and designing stimuli for particular experiments. There are many openended questions that have not been explored in our current research, which has primarily been focused on fMRI analysis and discrimination capacity. KCCA is a bi-directional technique and therefore are also able to compute a weight map for the stimuli from the learned semantic space. This capacity has the potential of greatly improving our understanding as to the link between fMRI analysis and stimuli by potentially telling us which image features were important. Acknowledgments. This work was supported in part by the IST Programme of the European Community, under the PASCAL Network of Excellence, IST2002-506778. David R. Hardoon is supported by the EPSRC project Le Strum, EP-D063612-1. This publication only reflects the authors views. We would like to thank Karl Friston for the constructive suggestions.

References 1. Cox, D.D., Savoy, R.L.: Functional magnetic resonance imaging (fmri) ‘brain reading’: detecting and classifying distributed patterns of fmri activity in human visual cortex. Neuroimage 19, 261–270 (2003) 2. Carlson, T.A., Schrater, P., He, S.: Patterns of activity in the categorical representations of objects. Journal of Cognitive Neuroscience 15, 704–717 (2003) 3. Wang, X., Hutchinson, R., Mitchell, T.M.: Training fmri classifiers to detect cognitive states across multiple human subjects. In: Proceedings of the 2003 Conference on Neural Information Processing Systems (2003) 4. Mitchell, T., Hutchinson, R., Niculescu, R., Pereira, F., Wang, X., Just, M., Newman, S.: Learning to decode cognitive states from brain images. Machine Learning 1-2, 145–175 (2004) 5. LaConte, S., Strother, S., Cherkassky, V., Anderson, J., Hu, X.: Support vector machines for temporal classification of block design fmri data. NeuroImage 26, 317–329 (2005) 6. Mourao-Miranda, J., Bokde, A.L.W., Born, C., Hampel, H., Stetter, S.: Classifying brain states and determining the discriminating activation patterns: support vector machine on functional mri data. NeuroImage 28, 980–995 (2005) 7. Haynes, J.D., Rees, G.: Predicting the orientation of invisible stimuli from activity in human primary visual cortex. Nature Neuroscience 8, 686–691 (2005) 8. Davatzikos, C., Ruparel, K., Fan, Y., Shen, D.G., Acharyya, M., Loughead, J.W., Gur, R.C., Langleben, D.D.: Classifying spatial patterns of brain activity with machine learning methods: Application to lie detection. NeuroImage 28, 663–668 (2005) 9. Kriegeskorte, N., Goebel, R., Bandettini, P.: Information-based functional brain mapping. PANAS 103, 3863–3868 (2006)

486

D.R. Hardoon et al.

10. Mourao-Miranda, J., Reynaud, E., McGlone, F., Calvert, G., Brammer, M.: The impact of temporal compression and space selection on svm analysis of singlesubject and multi-subject fmri data. NeuroImage (accepted, 2006) 11. Hardoon, D.R., Saunders, C., Szedmak, S., Shawe-Taylor, J.: A correlation approach for automatic image annotation. In: Li, X., Za¨ıane, O.R., Li, Z. (eds.) ADMA 2006. LNCS (LNAI), vol. 4093, pp. 681–692. Springer, Heidelberg (2006) 12. Wismuller, A., Meyer-Base, A., Lange, O., Auer, D., Reiser, M.F., Sumners, D.: Model-free functional mri analysis based on unsupervised clustering. Journal of Biomedical Informatics 37, 10–18 (2004) 13. Ciuciu, P., Poline, J., Marrelec, G., Idier, J., Pallier, C., Benali, H.: Unsupervised robust non-parametric estimation of the hemodynamic response function for any fmri experiment. IEEE TMI 22, 1235–1251 (2003) 14. O’Toole, A.J., Jiang, F., Abdi, H., Haxby, J.V.: Partially distributed representations of objects and faces in ventral temporal cortex. Journal of Cognitive Neuroscience 17(4), 580–590 (2005) 15. Friman, O., Borga, M., Lundberg, P., Knutsson, H.: Adaptive analysis of fMRI data. NeuroImage 19, 837–845 (2003) 16. Friman, O., Carlsson, J., Lundberg, P., Borga, M., Knutsson, H.: Detection of neural activity in functional MRI using canonical correlation analysis. Magnetic Resonance in Medicine 45(2), 323–330 (2001) 17. Hardoon, D.R., Shawe-Taylor, J., Friman, O.: KCCA for fMRI Analysis. In: Proceedings of Medical Image Understanding and Analysis, London, UK (2004) 18. Lowe, D.: Object recognition from local scale-invariant features. In: Proceedings of the 7th IEEE International Conference on Computer vision, Kerkyra, Greece, pp. 1150–1157 (1999) 19. Hardoon, D.R., Mourao-Miranda, J., Brammer, M., Shawe-Taylor, J.: Unsupervised analysis of fmri data using kernel canonical correlation. NeuroImag (in press, 2007) 20. Mikolajczyk, K., Schmid, C.: Indexing based on scale invariant interest points. In: International Conference on Computer Vision and Pattern Recognition, pp. 257–263 (2003) 21. Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines and other kernel-based learning methods. Cambridge University Press, Cambridge (2000) 22. Vapnik, V.: The Nature of Statistical Learning Theory. Springer, New York (1995) 23. Boser, B.E., Guyon, I., Vapnik, V.: A training algorithm for optimal margin classifiers. In: D. Proc. Fifth Ann. Workshop on Computational Learning Theory, pp. 144–152. ACM, New York (1992) 24. Fyfe, C., Lai, P.L.: Kernel and nonlinear canonical correlation analysis. International Journal of Neural Systems 10, 365–377 (2001) 25. Hardoon, D.R., Szedmak, S., Shawe-Taylor, J.: Canonical correlation analysis: an overview with application to learning methods. Neural Computation 16, 2639–2664 (2004) 26. Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge (2004) 27. Stephan, K.E., Harrison, L.M., Penny, W.D., Friston, K.J.: Biophysical models of fmri responses. Current Opinion in Neurobiology 14, 629–635 (2004)

Parallel Reinforcement Learning for Weighted Multi-criteria Model with Adaptive Margin Kazuyuki Hiraoka, Manabu Yoshida, and Taketoshi Mishima Saitama University, 255 Shimo-Okubo, Sakura-ku, Saitama-shi, Japan [email protected]

Abstract. Reinforcement learning (RL) for a linear family of tasks is studied in this paper. The key of our discussion is nonlinearity of the optimal solution even if the task family is linear; we cannot obtain the optimal policy by a naive approach. Though there exists an algorithm for calculating the equivalent result to Q-learning for each task all together, it has a problem with explosion of set sizes. We introduce adaptive margins to overcome this difficulty.

1

Introduction

Reinforcement learning (RL) for a linear family of tasks is studied in this paper. Such learning is useful for time-varying environments, multi-criteria problems, and inverse RL [5,6]. The family is defined as a weighted sum of several criteria. This family is linear in the sense that reward is linear with respect to weight parameters. For instance, criteria of network routing include end-to-end delay, loss of packets, and power level associated with a node [5]. Selecting appropriate weights beforehand is difficult in practice and we need try and errors. In addition, appropriate weights may change someday. Parallel RL for all possible weight values is desirable in such cases. The key of our discussion is nonlinearity of the optimal solution; it is not linear but piecewise-linear actually. This fact implies that we cannot obtain the best policy by the following naive approach: 1. Find the value function for each criterion. 2. Calculate weighted sum of them to obtain the total value function. 3. Construct a policy on the basis of the total value function. A typical example is presented in section 5. Piecewise-linearity of the optimal solution has been pointed out independently in [4] and [5]. The latter aims at fast adaptation under time-varying environments. The former is our previous report, and we have tried to obtain the optimal solutions for various weight values all together. Though we have developed an algorithm that gives exactly equivalent solution to Q-learning for each weight value, it has a difficulty with explosion of set size. This difficulty is not a problem of the algorithm but an intrinsic nature of Q-learning for the weighted criterion model. M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 487–496, 2008. c Springer-Verlag Berlin Heidelberg 2008


488

K. Hiraoka, M. Yoshida, and T. Mishima

We have introduced a simple approximation with a ‘margin’ into decision of convexity first [6]. Then we have improved it so that we obtain an interval estimation and we can monitor the effect of the approximation [7]. In this paper, we propose adaptive adjustment of margins. In margin-based approach, we have to manage large sets of vectors in the first stage of learning. The peak of the set size tends to be large if we set a small margin to obtain an accurate final result. The proposed method reduces worry of this trade-off. By changing margins appropriately through learning steps, we can enjoy small set size in the first stage with large margins, and an accurate result in the final stage with small margins. The weighted criterion model is defined in section 2, and parallel RL for it is described in section 3. Then the difficulty of set size is pointed out and margins are introduced in section 4. Adaptive adjustment of margins is also proposed there. Its behavior is verified with experiments in section 5. Finally, a conclusion is given in section 6.

2

Weighted Criterion Model

An “orthodox” RL setting is assumed for states and actions as follows. – – – – –

The The The The The

time step is discrete (t = 0, 1, 2, 3, . . .). state set S and the action set A are finite and known. state transition rule P is unknown. state st is observable. task is a Markov decision process (MDP).

1 M The reward rt+1 is given as a weighted sum of partial rewards rt+1 , . . . , rt+1 :

rt+1 (β) =

M


i βi rt+1 = β · r t+1 ,

(1)

i=1

weight vector β ≡ (β1 , . . . , βM ) ∈ RM , reward vector r t+1 ≡

1 M (rt+1 , . . . , rt+1 )

M

∈R .

(2) (3)

1 M We assume that the partial rewards rt+1 , . . . , rt+1 are also observable, whereas their reward rules R(1), . . . , R(M ) are unknown. Multi-criteria RL problems of this type have been introduced independently in [3] and [5]. We hope to find the optimal policy πβ∗ for each weight β that maximizes the expected cumulative reward with a given discount factor 0 < γ < 1, ∞ 
πβ∗ = argmax E π γ τ rτ +1 (β) , (4) π

τ =0

π

where E [·] denotes the expectation under a policy π. To be exact, π ∗ is defined π∗

as a policy that attains Qββ (s, a; γ) = Q∗β (s, a; γ) ≡ maxπ Qπβ (s, a; γ) for all state-action pairs (s, a), where the action-value function Qπβ is defined as

Parallel Reinforcement Learning for Weighted Multi-criteria Model

 Qπβ (s, a; γ)

≡E

π

∞


τ =0

    γ rτ +1 (β)  s0 = s, a0 = a .  τ

489

(5)

It is well known that MDP has a deterministic policy πβ∗ that satisfies the above condition; such πβ∗ is obtained from the optimal value function [2], πβ∗ : S → A : s  → argmax Q∗β (s, a; γ). a∈A

(6)

Thus we concentrate on estimation of Q∗β . Note that Q∗β is nonlinear with respect to β. A typical example is presented in section 5. Basic properties of the action-value function Q are described briefly in the rest of this section [4,5,6]. The discount factor γ is fixed through this paper, and it is omitted below. Proposition 1. Qπβ (s, a) is linear with respect to β for a fixed policy π. Proof. Neither P nor π depend on β from assumptions. Hence, joint distribution of (s0 , a0 ), (s1 , a1 ), (s2 , a2 ), . . . is independent of β. It implies linearity.   Definition 1. If f : RM → R can be written as f (β) = maxq∈Ω (q · β) with a nonempty finite set Ω ⊂ RM , we call f Finite-Max-Linear (FML) and write it as f = FMLΩ . It is trivial that f is convex and piecewise-linear if f is FML. Proposition 2. The optimal action-value function is FML as a function of the weight β. Namely, there exists a nonempty finite set Ω ∗ (s, a) ⊂ RM for each state-action pair (s, a), and Q∗β is written as Q∗β (s, a) =

max

q∈Ω ∗ (s,a)

q · β.

(7)

Proof. We have assumed MDP. It is well known that Q∗β can be written Q∗β (s, a) = maxπ∈Π Qπβ (s, a) for the set Π of all deterministic policies. Π finite, and Qπβ is linear with respect to β from proposition 1. Hence, Q∗β FML.

as is is  

Proposition 3. Assume that an estimated action-value function Qβ is FML as a function of the weight β. If we apply Q-learning, the updated   new Qβ (st , at ) = (1 − α)Qβ (st , at ) + α β · rt+1 + γ max Qβ (st+1 , a) (8) a∈A

is still FML as a function of β, where α > 0 is the learning rate. Proof. There exists a nonempty finite set Ω(s, a) ⊂ RM such that Qβ (s, a) = ˜ · β, maxq∈Ω(s,a) (q · β) for each (s, a). Then (8) implies Qnew ˜q β (st , at ) = maxq ˜ ∈Ω where    ˜ ≡ (1 − α)q + α(r t+1 + γq )  a ∈ A, q ∈ Ω(st , at ), q  ∈ Ω(st+1 , a) , (9) Ω because maxx f (x) + maxy g(y) = maxx,y (f (x) + g(y)) holds in general. The set ˜ is finite, and Qnew is FML. Ω   β These propositions imply that (1) the true Q∗β is FML, and (2) its estimation Qβ is also FML as long as the initial estimation is FML.

490

3

K. Hiraoka, M. Yoshida, and T. Mishima

Parallel Q-Learning for All Weights

A parallel Q-learning method for the weighted criterion model has been proposed in [6]. The estimation Qβ for all β ∈ RM are updated all together in parallel Q-learning. In this method, Qβ (s, a) for each (s, a) is treated in an FML expression: Qβ (s, a) =

max q · β = FMLΩ(s,a) (β)

(10)

q∈Ω(s,a)

with a certain set Ω(s, a) ⊂ RM . We store and update Ω(s, a) instead of Qβ (s, a) on the basis of propositions 2 and 3. Though a naive updating rule has been suggested in the proof of proposition 3, it is extremely redundant and inefficient. We need several definitions to describe a better algorithm. Definition 2. An element c ∈ Ω is redundant if FML(Ω−{c}) = FMLΩ . Definition 3. We use Ω † to represent non-redundant elements in Ω. Note that FMLΩ † = FMLΩ [5]. Definition 4. We define the following operations: cΩ ≡ {cq | q ∈ Ω},

c + Ω ≡ {c + q | q ∈ Ω},

K † K    † Ω  Ω ≡ (Ω ∪ Ω ) , Ωk ≡ Ωk , k=1

Ω ⊕ Ω  ≡ {q + q  | q ∈ Ω, q  ∈ Ω  },

(11) (12)

k=1 †

Ω
Ω  ≡ (Ω ⊕ Ω  ) .

With these operations, the updating rule of Ω is described as follows [6]:

Ω new (st , at ) = (1 − α)Ω(st , at )
α r t+1 + γ Ω(st+1 , a) .

(13)

(14)

a∈A

The initial value of Ω at t = 0 is Ω(s, a) = {o} ⊂ RM for all (s, a) ∈ S × A. It corresponds to a constant initial function Qβ (s, a) = 0. Proposition 4. When (10) holds for all states s ∈ S and actions a ∈ A, Qnew β (st , at ) in (8) is equal to FMLΩ new (st ,at ) (β) for (14). Namely, parallel Qlearning is equivalent to Q-learning for each β: {Qβ (s, a)} FML expression

{Ω(s, a)}

update → Qnew β (st , at )

FML expression . → Ω new (st , at ) update

(15)

Parallel Reinforcement Learning for Weighted Multi-criteria Model

491

y

Ω

O

x

Ω[+]Ω’

(2) Merge and sort edges according to their arguments

Ω’

(3) Connect edges to generate a polygon

(4) Shift the origin (max x in Ω)+(max x in Ω’) 㤘(max x in Ω[+]Ω’) (max y in Ω)+(max y in Ω’) 㤘(max y in Ω[+]Ω’)

(1) Set directions of edges

Fig. 1. Calculation of Ω
Ω
in (14) for two-dimensional convex polygons. Vertices of polygons correspond to Ω, Ω
and Ω
Ω
.

˜ in (9) to prove proposition 3. With the above Proof. We have introduced a set Ω operations, (9) is written as

 ˜ Ω = (1 − α)Ω(st , at ) ⊕ α r t+1 + Ω(st+1 , a) . a∈A †

˜ = Ω new (st , at ) is obtained and FMLΩ new (s ,a ) (β) = FML ˜ Then (Ω) t t Ω(st ,at ) (β) = new Qβ (st , at ) is implied.   It is well known that Ω † is equal to the vertices in the convex hull of Ω [6]. Efficient algorithms of convex hull have been developed in computational geometry † [8]. Using them, we can calculate the merged set (Ω  Ω  ) = (Ω ∪ Ω  ) . The sum  set (Ω
Ω ) have been also studied as Minkowski sum algorithms [9,10,11]. Its calculation is particularly easy for two-dimensional convex polygons (Fig.1). Before closing the present section, we note an FML version of Bellman equation in our notation. Theoretically, we can use successive iteration of this equation to find the optimal policy when we know P and R, though we must take care of numerical error in practice. Proposition 5. FML expression Q∗β = FMLΩ ∗ (β) satisfies †

Ω ∗ (s, a) = Ras + γ

a ∈A

a ∗   + Pss  Ω (s , a ),

(16)

s ∈S

where i Ras = (Ras (1), . . . , Ras (M )), Ras (i) = E[rt+1 | st = s, at = a], a  Pss = P (st+1 = s | st = s, at = a),

+ s ∈{s1 ,...,sk }

Xs = Xs1
Xs2
· · ·
Xsk .

(17) (18) (19)

492

K. Hiraoka, M. Yoshida, and T. Mishima

In particular, the next equation holds if state transition is deterministic: † Ω ∗ (s, a) = Ras + γ Ω ∗ (s , a ),

(20)

a ∈A

where s is the next state for the action a at the current state s. a Proof. Substituting (7) and Ras,β ≡ E[r t+1 (β) | st = s, at = a] = Rs · β into the ∗ a a ∗  Bellman equation Qβ (s, a) = Rs,β + γ s ∈S Pss maxa ∈A Qβ (s , a ), we obtain

max q  · β,   
a    ∗   Ω  (s, a) = Ras + γ Pss  q s  q s ∈ Ω (s , a )

max

q∈Ω ∗ (s,a)

q·β =

q  ∈Ω  (s,a)

a ∈A

(22)

s ∈S

in the same way as (9). Hence, Ω ∗ is equal to Ω  except for redundancy.

4

(21)

 

Interval Operations

Under regularity conditions, Q-learning has been proved to converge to Q∗ [1]. That result implies pointwise convergence of parallel Q-learning to Q∗β for each β because of proposition 3. From proposition 2, Q∗β (s, a) is expressed with a finite Ω ∗ (s, a). However, as we can see in Fig.1, the number of elements in the set Ω(s, a) increases monotonically and it never ‘converges’ to Ω ∗ (s, a). This is not a paradox; the following assertions can be true at the same time. 1. Vertices of polygons P1 , P2 , . . . monotonically increase. 2. Pt converges to a polygon P ∗ in the sense that the volume of the difference Pt  P = (Pt ∪ P ∗ ) − (Pt ∩ P ∗ ) converges to 0. 2’. The function FMLPt (·) converges pointwise to FMLP ∗ (·). In short, pointwise convergence of a piecewise-linear function does not imply convergence of the number of pieces. Note that it is not a problem of the algorithm. It is an intrinsic nature of pointwise Q-learning of the weighted criterion model for each weight β. To overcome this difficulty, we tried a simple approximation with a small ‘margin’ at first [6]. Then we have introduced interval operations to monitor approximation error [7]. A pair of sets Ω L (s, a) and Ω U (s, a) are updated instead of the original Ω(s, a) so that CH Ω L (s, a) ⊂ CH Ω(s, a) ⊂ CH Ω U (s, a) holds, where CH Z represents the convex hull of Z. This relation implies lower and upU X per bounds QL β (s, a) ≤ Qβ (s, a) ≤ Qβ (s, a), where Qβ (s, a) = FMLΩ X (s,a) (β) L U for X = L, U . When the difference between Q and Q is sufficiently small, it is guaranteed that the effect of the approximation can be ignored. Updating rules of Ω L and Ω U are same as those of Ω, except for the following approximations after every calculation of  and
. We assume M = 2 here. Lower approximation for Ω L : A vertex is removed if the change of the area of CH Ω L (s, a) is smaller than a threshold L /2 (Fig.2 left).

Parallel Reinforcement Learning for Weighted Multi-criteria Model

b

c

b

d a

a

d

e

b a

c

if the area of triangle /// is small

if the area of triangle /// is small d - remove c e

493

z a

- remove b,c - add z d

Fig. 2. Lower approximation (left) and upper approximation (right)

Upper approximation for Ω U : An edge is removed if the change of the area of CH Ω U (s, a) is smaller than a threshold U /2 (Fig.2 right). In this paper, we propose an automatic adjustment of the margins L , U . The below procedures are performed at every step t after the updating of Ω L , Ω U . The symbol X represents L or U here. ξs , ξw ≥ 1 and θQ , θΩ ≥ 0 are constants. 1. Check the changes of set sizes and interval width compared with the previous ones. Namely, check these values:  Xnew     ∆X (st , at ) − Ω X (st , at ) , (23) Ω = Ω  Unew   U  Lnew L ∆Q = Qβ¯ (st , at ) − Qβ¯ (st , at ) − Qβ¯ (st , at ) − Qβ¯ (st , at ) , (24) ¯ is selected beforehand. where |Z| is the number of elements in Z, and β 2. Increase of set size suggests a need of thinning, whereas increase of interval width suggests a need of more accurate calculation. Modify margins as   ˜X (∆Q ≤ θQ ) X (∆X Xnew X Ω ≤ θΩ )  = X , where ˜ = . (25) X ˜ /ξw (∆Q > θQ ) ξs  (∆X Ω > θΩ ) To avoid underflow, we set Xnew = min if Xnew is smaller than a constant min .

5

Experiments with a Basic Task of Weighted Criterion

We have verified behaviors of the proposed method. We set S = {S, G, A, B, X, Y}, A = {Up, Down, Left, Right}, s0 = S, and γ = 0.8 (Fig.3) [6]. Each action causes a deterministic state transition to the corresponding direction except at G, where the agent is moved to S regardless of its action. Rewards 1, 4b, b are offered at st = G, X, Y, respectively. If at is an action to ‘outside wall’ at st = G, the state is unchanged and a negative reward (−1) is added further. It is a weighted criterion model of M = 2, because it can be written as 1 2 the form rt+1 = β · rt+1 for r t+1 = (rt+1 , rt+1 ) and β = (b, 1). The optimal policy changes depending on the weight b. Hence, the optimal value function is

494

K. Hiraoka, M. Yoshida, and T. Mishima

S

X (4b)

G (1)

A

Y (b)

B

outside = wall (-1)

Fig. 3. Task for experiments. Numbers in parentheses are reward values. Table 1. Optimal state-value functions and optimal policies Range of weight Optimal Vb∗ (S) Optimal state transition b < −16/25 0 S → A → S → ··· −16/25 ≤ b < −225/1796 (2000b + 1280)/2101 S → A → Y → B → G → S → · · · −225/1796 ≤ b < 15/47 (400b + 80)/61 S → X → G → S → ··· 15/47 ≤ b < 3/4 32b/3 S → X → Y → X → ··· 3/4 ≤ b 16b − 4 S → X → X → ···

1

1e-04 1e-06

1e-13 1e-10 1e-7 1e-4 0.1

0.01 1e-04 Upper margin

Lower margin

1

1e-13 1e-10 1e-7 1e-4 0.1

0.01

1e-08 1e-10 1e-12 1e-14

1e-06 1e-08 1e-10 1e-12 1e-14

1e-16

1e-16

1e-18

1e-18 0

5000

10000

15000

20000

0

5000

10000

t

15000

20000

t

300

Total number of elements Σs,a|Ω (s,a)|

1e-13 1e-10 1e-7 1e-4 0.1

250 200

U

L

Total number of elements Σs,a|Ω (s,a)|

Fig. 4. Transition of margins L and U from various initial margins

150 100 50 0 0

5000

10000

15000

20000

t

Fig. 5. Total number of elements

300

1e-13 1e-10 1e-7 1e-4 0.1

250 200 150 100 50 0 0

5000

10000

15000

20000

t

s,a

|Ω X (s, a)|. (Left: X = L, Right: X = U ).

Parallel Reinforcement Learning for Weighted Multi-criteria Model

495

1 0.01 1e-04 Interval width

1e-06 1e-08 1e-10 1e-12

1e-13 1e-10 1e-7 1e-4 0.1

1e-14 1e-16 1e-18 0

5000

10000

15000

20000

t

4500

1

1e-2(Lower) 1e-2(Upper) 1e-9(Lower) 1e-9(Upper)

4000 3500

0.01 1e-04

3000

Interval width

X

Total number of elements Σs,a|Ω (s,a)|

L Fig. 6. Interval width QU (0.2,1) (A, Up) − Q(0.2,1) (A, Up)

2500 2000 1500

1e-06 1e-08 1e-10 1e-12

1000

1e-14

500

1e-16

0

1e-18 0

5000

10000

15000

20000

1e-2 1e-9 0

5000

10000

t

15000

20000

t

300

1

1e-13 1e-10 1e-7 1e-4 0.1

250 200

0.01 1e-04 Interval width

U

Total number of elements Σs,a|Ω (s,a)|

Fig. 7. Fixed-margin (U = L = 10−2 and U = L = 10−9 ). Left: total

algorithm X number of elements s,a |Ω (s, a)| for X = U, L. Right: interval width.

150 100

1e-06 1e-08 1e-10 1e-12

1e-13 1e-10 1e-7 1e-4 0.1

1e-14

50

1e-16 0

1e-18 0

2000

4000

6000 t

8000

10000

0

2000

4000

6000

8000

10000

t

Fig. 8. Average of 100 trials with inappropriate factors ξs = 1.5, ξw = 1.015 for γ = 0.5 Left: total number of elements in upper approximation. Right: interval width.

nonlinear with respect to b (Table 1). Note that the second pattern (S→A→Y) in Table 1 cannot appear on the naive approach in section 1. The proposed algorithm is applied to this task with random actions at and ¯ = (0.2, 1), parameters α = 0.7, (ξs , ξw ) = (1.7, 1.015), (θQ , θΩ ) = (0, 2), β −14 L U −1 min = 10 . The initial margins  =  at t = 0 is one of 10 , 10−4 , 10−7 ,

496

K. Hiraoka, M. Yoshida, and T. Mishima

10−10 , 10−13 . On this task, we can replace convex hulls with upper convex hulls in our algorithm because β is restricted to the upper half plane [6]. We also assume |b| ≤ 10 ≡ bmax and we safely remove the edges on the both end in Fig.2 if the absolute value of their slope is greater than bmax for lower approximation. Averages of 100 trials are shown in Fig.4,5,6. The proposed algorithm is robust to wide range of initial margins. It realizes reduced set sizes and small interval width at the same time; these requirements are trade-off in the conventional fixed-margin algorithm [7] (Fig.7). A problem of the proposed algorithms is sensitivity to the factors ξs , ξw . When they are inappropriate, instability is observed after a long run (Fig.8). Another problem is slow convergence of the interval width QU − QL compared with the fixed-margin algorithm.

6

Conclusion

A parallel RL method with adaptive margins is proposed for the weighted criterion model, and its behaviors are verified experimentally with a basic task. Adaptive margins realize reduced set sizes and accurate results. A problem of the adaptive margins is instability for inappropriate parameters. Though it is robust for initial margins, it needs tuning of factor parameters. Another problem is slow convergence of the interval between upper and lower estimations. These points must be studied further.

References 1. Jaakkola, T., et al.: Neural Computation 6, 1185–1201 (1994) 2. Sutton, R.S., Barto, A.G.: Reinforcement Learning. The MIT Press, Cambridge (1998) 3. Kaneko, Y., et al.: In: Proc. IEICE Society Conference (in Japanese), vol. 167 (2004) 4. Kaneko, N., et al.: In: Proc. IEICE Society Conference (in Japanese), vol. A-2-10 (2005) 5. Natarajan, S., et al.: In: Proc. Intl. Conf. on Machine Learning, pp. 601–608 (2005) 6. Hiraoka, K., et al.: The Brain & Neural Networks (in Japanese). Japanese Neural Network Society 13, 137–145 (2006) 7. Yoshida, M., et al.: Proc. FIT (in Japanese) (to appear, 2007) 8. Preparata, F.P., et al.: Computational Geometry. Springer, Heidelberg (1985) 9. Alexandrov, V.N., Dongarra, J., Juliano, B.A., Renner, R.S., Tan, C.J.K. (eds.): ICCS 2001. LNCS, vol. 2073. Springer, Heidelberg (2001) 10. Fukuda, K.: J. Symbolic Computation 38, 1261–1272 (2004) 11. Fogel, E., et al.: In: Proc. ALENEX, pp. 3–15 (2006)

Convergence Behavior of Competitive Repetition-Suppression Clustering Davide Bacciu1,2 and Antonina Starita2 1

2

IMT Lucca Institute for Advanced Studies, P.zza San Ponziano 6, 55100 Lucca, Italy [email protected] Dipartimento di Informatica, Universit` a di Pisa, Largo B. Pontecorvo 3, 56127 Pisa, Italy [email protected]

Abstract. Competitive Repetition-suppression (CoRe) clustering is a bio-inspired learning algorithm that is capable of automatically determining the unknown cluster number from the data. In a previous work it has been shown how CoRe clustering represents a robust generalization of rival penalized competitive learning (RPCL) by means of M-estimators. This paper studies the convergence behavior of the CoRe model, based on the analysis proposed for the distance-sensitive RPCL (DSRPCL) algorithm. Furthermore, it is proposed a global minimum criterion for learning vector quantization in kernel space that is used to assess the correct location property for the CoRe algorithm.

1

Introduction

CoRe learning has been proposed as a biologically inspired learning model mimicking a memory mechanism of the visual cortex, i.e. repetition suppression [1]. CoRe is a soft-competitive model that allows only a subset of the most active units to learn in proportion to their activation strength, while it penalizes the least active units, driving them away from the patterns producing low firing strengths. This feature has been exploited in [2] to derive a clustering algorithm that is capable of automatically determining the unknown cluster number from the data by means of a reward-punishment procedure that resembles the rival penalization mechanism of RPCL [3]. Recently, Ma and Wang [4] have proposed a generalized loss function for the RPCL algorithm, named DSRPCL, that has been used for studying the convergence behavior of the rival penalization scheme. In this paper, we present a convergence analysis for CoRe clustering that founds on Ma and Wang’s approach, describing how CoRe satisfies the three properties of separation nature, correct division and correct location [4]. The intuitive analysis presented in [4] for DSRPCL is enforced with theoretical considerations showing that CoRe pursues a global optimality criterion for vector quantization algorithms. In order to do this, we introduce a kernel interpretation for the CoRe loss that is used to generalize the results given in [5] for hard vector quantization, to kernel-based algorithms. M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 497–506, 2008. c Springer-Verlag Berlin Heidelberg 2008 

498

2

D. Bacciu and A. Starita

A Kernel Based Loss Function for CoRe Clustering

A CoRe clustering network consists of cluster detector units that are characterized by a prototype ci , that identifies the preferred stimulus for the unit ui and represents the learned cluster centroid. In addition, units are characterized by an activation function ϕi (xk , λi ), defined in terms of a set of parameters λi , that determines the firing strength of the unit in response to the presentation of an input pattern xk ∈ χ. Such an activation function measures the similarity between the prototype ci and the inputs, determining whether the pattern xk belongs to the i-th cluster. In the remainder of the paper we will use an activation function that is a gaussian centered in ci with spread σi , i.e.   ϕi (xk |{ci , σi }) = exp −0.5xk − ci 2 /σi2 . CoRe clustering works essentially by evolving a small set of highly selective cluster detectors out of an initially larger population by means of a competitive reward-punishment procedure that resembles the rival penalization mechanism [3]. Such a competition is engaged between two sets of units: at each step the most active units are selected to form the winners pool, while the remainder is inserted into the losers pool. More formally, we define the winners pool for the input xk as the set of units ui that fires more than θwin or the single unit that is maximally active for the pattern, that is wink = {i | ϕi (xk , {ci , σi }) ≥ θwin } ∪ {i | i = arg max ϕj (xk | {ci , σi })} j∈U

(1)

where the second term of the union ensures that wink is non-empty. Conversely, the losers pool for xk is losek = U \ wink , that is the complement of wink with respect to the neuron set U . The units belonging to the losers pool are penalized and their response is suppressed. The strength of the penalization for the pattern xk , at time t, is regulated by the repetition suppression RSkt ∈ [0, 1] and is proportional to the frequency of the pattern that has elicited the suppressive effect (see [2,6] for details). The repetition suppression is used to define a pseudo-target activation for the units in the losers pool as ϕˆti (xk ) = ϕi (xk , {ci , σi })(1 − RSkt ). This reference signal forces the losers to reduce their activation proportionally to the amount of repetition suppression they receive. The error of the i-th loser unit can thus be written as E ti,k =

1 t 1 (ϕˆi (xk ) − ϕi (xk , {ci , σi }))2 = (−ϕi (xk , {ci , σi })RSkt )2 . 2 2

(2)

Conversely, in order to strengthen the activation of the winner units, we set the target activation for the neurons ui (i ∈ wink ) to M , that is the maximum of the activation function ϕi (·). The error, in this case, can be written as t

E i,k = (M − ϕi (xk , {ci , σi })).

(3)

To analyze the CoRe convergence, we give an error formulation that accumulates the residuals in (2) and (3) for a given epoch e: summing up over all CoRe units in U and the dataset χ = (x1 , . . . , xk , . . . , xK ) yields

Convergence Behavior of Competitive Repetition-Suppression Clustering

Je (χ, U ) =

I  K 

δik (1 − ϕi (xk )) +

i=1 k=1

I  K 

499

 2 (e|χ|+k) (1 − δik ) ϕi (xk )RSk (4)

i=1 k=1

where δik is the indicator function for the set wink and where {ci , σi } has been omitted from ϕi to ease the notation. Note that, in (4), we have implicitly used the fact that the units can be treated as independent. The CoRe learning equations can be derived using gradient descent to minimize Je (χ, U ) with respect to the parameters {ci , σi } [2]. Hence, the prototype increment for the e-th epoch can be calculated as follows ⎡ ⎤

K (e|χ|+k) 2 e  ⎣δik ϕi (xk ) (xk − ci ) − (1 − δik ) ϕi (xk )RSk cei = αc (xk − cei )⎦ (σie )2 σie k=1

(5) where αc is a suitable learning rate ensuring that Je decreases with e. Similarly, the spread update can be calculated as σie = ασ

K  k=1

δik ϕi (xk )

 e 2 xk − cei 2 (e|χ|+k) 2 xk − ci  − (1 − δ )(ϕ (x )RS ) . (6) i ik k k (σie )3 (σie )3

As one would expect, unit prototypes are attracted by similar patterns (first term in (5)) and are repelled by the dissimilar inputs (second term in (5)). Moreover, the neural selectivity is enhanced by reducing the Gaussian spread each time the corresponding unit happens to be a winner. Conversely, the variance of loser neurons is enlarged, reducing the units’ selectivity and penalizing them for not having sharp responses. The error formulation introduced so far can be restated by exploiting the kernel trick [7] to express the CoRe loss in terms of differences in a given feature space F . Kernel methods are algorithms that exploit a nonlinear mapping Φ : χ → F to project the data from the input space χ onto a convenient, implicit feature space F . The kernel trick is used to express all operations on Φ(x1 ), Φ(x2 ) ∈ F in terms of the inner product Φ(x1 ), Φ(x2 ) . Such inner product can be calculated without explicitly using the mapping Φ, by means of the kernel κ(x1 , x2 ) = Φ(x1 ), Φ(x2 ) . To derive the kernel interpretation for the CoRe loss in (4), consider first the formulation of the distance dFκ of two vectors x1 , x2 ∈ χ in the feature space Fκ , induced by the kernel κ, and described by the mapping Φ : χ → Fκ , that is dFκ (x1 , x2 ) = Φ(x1 ) − Φ(x2 )2Fκ = κ(x1 , x1 ) − 2κ(x1 , x2 ) + κ(x2 , x2 ). The kernel trick [7] have been used to substitute the inner products in feature space with a suitable kernel κ calculated in the data space. If κ is chosen to be a gaussian kernel, then we have that κ(x, x) = 1. Hence dFκ can be rewritten as dFκ = Φ(x1 ) − Φ(x2 )2Fκ = 2 − 2κ(x1 , x2 ). Now, if we take x1 to be an element of the input dataset, e.g. xk ∈ χ, and x2 to be the prototype ci of the i-th CoRe unit, we can rewrite dFκ in such a way to depend on the activation function ϕi . Therefore, applying the substitution κ(xk , ci ) = ϕi (xk , {ci , σi }) we obtain ϕi (xk , {ci , σi }) = 1 − 12 Φ(xk ) − Φ(ci )2Fκ . Now, if we substitute this result in the formulation of the CoRe loss in (4), we obtain

500

D. Bacciu and A. Starita

1  δik Φ(xk ) − Φ(ci )2Fκ + 2 i=1 k=1

 2 I  K  1 1 (e|χ|+k) 2 + (1 − δik ) RSk 1 − Φ(xk ) − Φ(ci )Fκ (7) 2 i=1 2 I

Je (χ, U ) =

K

k=1

Equation (7) states that CoRe minimizes the feature space distance between the prototype ci and those xk that are close in the kernel space induced by the activation functions ϕi , while it maximizes the feature space distance between the prototypes and those xk that are far from ci in the kernel space.

3

Separation Nature

To prove the separation nature of the CoRe process we need to demonstrate that, given a a bounded hypersphere G containing all the sample data, then after sufficient iterations of the algorithm the cluster prototypes will finally either fall into G or remain outside it and never get into G. In particular, those prototypes remaining outside the hypersphere will be driven far away from the samples by the RS repulsion. We consider a prototype ci to be far away from the data if, for a given epoch e, it is in the loser pool for every xk ∈ χ. To prove CoRe separation nature we first demonstrate the following Lemma. Lemma 1. When a prototype ci is far away from the data at a given epoch e, then it will always be a loser for every xk ∈ χ and will be driven away from the data samples. Proof. The definition of far away implies that, given cei , ∀xk ∈ χ. i ∈ loseek , where the e in the superscript refers to the learning epoch. Given the prototype update in (5), we obtain the weight vector increment Δcei at epoch e as follows cei

= −ασ

K  k=1



(e|χ|+k)

ϕi (xk )RSk σie

2 (xk − cei ).

(8)

As a result of (8), the prototype ce+1 is driven further from the data. On the i other hand, by definition (1), for each of the data samples there exists at least one winner unit for every epoch e, such that its prototype is moved towards the samples for which it has been a winner. Moreover, not every prototype can be deflected from the data, since this would make the first term of Je (χ, U ) (see (4)) grow and, consequently, the whole Je (χ, U ) will diverge since the loser error term in (4) is lower bounded. However, this would contradict the fact that Je (χ, U ) decreases with e since CoRe applies gradient descent to the loss function. Therefore, there must exist at least one winning prototype cel that remains close to the samples at epoch e. On the other hand cei is already far away from the samples and, by (8), ce+1 will be further from the data and i won’t be a winner for any xk ∈ χ. To prove this, consider the definition of wink

Convergence Behavior of Competitive Repetition-Suppression Clustering

501

in (1): for ce+1 to be a winner, it must hold either (i) ϕi (xk ) ≥ θwin or (ii) i i = arg maxj∈U ϕj (xk , λj ). The former does not hold because the receptive field area where the firing strength of the i-th unit is above the threshold θwin does not contain any sample at epoch e. Consequently, it cannot contain any sample at epoch e + 1 since its center ce+1 has been deflected further from the data. The i latter does not hold since there exist at least one prototype, i.e. cl , that remains close to the data, generating higher activations than unit ui . As a consequence, a far away prototype ci will be deflected from the data until it reaches a stable point where the corresponding firing strength ϕi is negligible.  Now we can proceed to demonstrate the following Theorem 1. For a CoRe process there exist an hypersphere G surrounding the sample data χ such that after sufficient iterations each prototype ci will finally either (i) fall into G or (ii) keep outside G and reach a stable point. Proof. The CoRe process is a gradient descent (GD) algorithm on Je (χ, U ), hence, for a sufficiently small learning step, the loss decreases with the number of epochs. Therefore, being Je (χ, U ) always positive the GD process will converge to a minimum J ∗ . The sequences of prototype vectors {cei } will converge either to a point close to the samples or to a point of negligible activation far away from the data. If a unit ui has a sufficiently long subsequence of prototypes {cei } diverging from the dataset then, at a certain time, will no longer be a winner for any sample and, by Lemma 1, will converge at a point far away from the data. The attractors for the sequence {cei } of the diverging units lie at a certain distance r from the samples, that is determined by those points x where the gaussian unit centered in x produces a negligible activation in response to any pattern xk ∈ χ. Hence, G can be chosen as any hypersphere surrounding the samples with radius smaller than r. On the other hand, since Je (χ, U ) decreases to J ∗ , there must exist at least one prototype that is not far away from the data (otherwise the first term of Je (χ, U ) in (4) will diverge). In this case, the sequences {cei } must have accumulation points close to the samples. Therefore any hypersphere G enclosing all the samples will also surround the accumulation points of {cei } and, after a certain epoch E, the sequence will be always within such hypersphere.  In summary, Theorem 1 tells that the separation nature holds for a CoRe process: some prototypes are possibly pushed away from the data until their contribution to the error in (4) becomes negligible. Far away prototypes will always be losers and will never head back to the data. Conversely, some prototypes will converge to the samples, heading to a saddle point of the loss Je (χ, U ) by means of a gradient descent process.

4

Correct Division and Location

Following the convergence analysis in [4] we now turn our attention to the issues of correct division and location of the weight vectors. This means that the

502

D. Bacciu and A. Starita

number of prototypes falling into G will be nc , i.e. the number of the actual clusters in the sample data, and they will finally converge to the centers of the clusters. At this point, we leave the intuitive study presented for DSRPCL [4], introducing a sound analysis of the properties of the saddle points identified by CoRe, giving a sufficient and necessary condition for identifying the global minimum of a vector quantization loss in feature space. 4.1

A Global Minimum Condition for Vector Quantization in Kernel Space

The classical problem of hard vector quantization (VQ) in Euclidean space is to determine a codebook V = v1 , . . . , vN minimizing the total distortion, calculated by Euclidean norms, resulting from the approximation of the inputs xk ∈ χ by the code vectors vi . Here, we focus on a more general problem that is vector quantization in feature space. Given the nonlinear mapping Φ and the induced feature space norm  · Fκ introduced in the previous sections, we aim at optimizing the distortion K N 1  1 min D(χ, ΦV ) = δik Φ(xk ) − Φvi 2Fκ K i=1

(9)

k=1

where ΦV = {Φv1 , . . . , ΦvN } represents the codebook in the kernel space and 1 δik is equal to 1 if the i-th cluster is the closest to the k-th pattern in the feature space Fκ , and is 0 otherwise. It is widely known that VQ generates a Voronoi tessellation of the quantized space and that a necessary condition for the minimization of the distortion requires the code-vectors to be selected as the centroids of the Voronoi regions [8]. In [5], it is given a necessary and sufficient condition for the global minimum of an Euclidean VQ distortion function. In the following, we generalize this result to vector quantization in feature space. To prove the global minimum condition in kernel space we need to extend the results in [9] (Proposition 3.1.7 and 3.2.4) to the most general case of a kernel induced distance metric. Therefore we introduce the following lemma. Lemma 2. Let κ be a kernel and Φ : χ → Fκ a map into the corresponding feature space Fκ . Given a dataset χ = x1 , . . . , xK partitioned into N subsets Ci , K 1 define the feature space mean Φ = χ k=1 Φ(xk ) and the i-th partition centroid K 1  Φvi = |Ci | k∈Ci Φ(xk ), then we have K  k=1

Φ(xk ) − Φχ 2Fκ =

N   i=1 k∈Ci

Φ(xk ) − Φvi 2Fκ +

N 

|Ci |Φvi − Φχ 2Fκ . (10)

i=1

Proof. Given a generic feature vector Φ1 , consider the identity Φ(xk ) − Φ1 = (Φ(xk ) − Φvi ) + (Φvi − Φ1 ): its squared norm in feature space is Φ(xk ) − Φ1 2Fκ = Φ(xk ) − Φvi 2Fκ + Φvi − Φ1 2Fκ + 2(Φ(xk ) − Φvi )T (Φvi − Φ1 ).

Convergence Behavior of Competitive Repetition-Suppression Clustering

503

Summing over all the elements in the i-th partition we obtain    Φ(xk ) − Φ1 2Fκ = Φ(xk ) − Φvi 2Fκ + Φvi − Φ1 2Fκ k∈Ci

k∈Ci



+2

k∈Ci

(Φ(xk ) − Φvi ) (Φvi − Φ1 ) T

k∈Ci

=



Φ(xk ) − Φvi 2Fκ + |Ci |Φvi − Φ1 2Fκ .

(11)

k∈Ci

 The last term in (11) vanishes since k∈Ci (Φ(xk ) − Φvi ) = 0 by definition of Φvi . Now, applying the substitution Φ1 = Φχ and summing up for all the N partitions yields K 

Φ(xk ) − Φχ 2Fκ =

N  

Φ(xk ) − Φvi 2Fκ +

i=1 k∈Ci

k=1

N 

|Ci |Φvi − Φχ 2Fκ (12)

i=1

N

where the left side of equality holds since

i=1

Ci = χ and

N i=1

Ci = ∅ .



Using the results from Lemma 2 we can proceed with the formulation of the global minimum criterion by generalizing the results of Proposition 1 in [5] to vector quantization in feature space. g } be a global minimum solution to the probProposition 1. Let {Φv1g , . . . , ΦvN lem in (9), then we have

N 

|Cig |Φvig − Φχ 2Fκ ≥

N 

i=1

|Ci |Φvi − Φχ 2Fκ

(13)

i=1

g for any local optimal solution {Φv1 , . . . , ΦvN } to (9), where {C1g , . . . , CN } and g g {C , . . . , C } are the χ partitions corresponding to the centroids Φ = 1/|C 1 N v i| i   k∈Cig Φ(xk ) and Φvi = 1/|Ci | k∈Ci Φ(xk ) respectively, and where Φχ is the dataset mean (see definition in Lemma 2). g } is a global minimum for (9) we have Proof. Since {Φv1g , . . . , ΦvN

N  

Φ(xk ) − Φvig 2Fκ ≤

i=1 k∈Cig

N  

Φvi − Φχ 2Fκ

(14)

i=1 k∈Ci

for any local minimum {Φv1 , . . . , ΦvN }. From Lemma 2 we have that K 

Φ(xk ) − Φχ 2Fκ =

k=1

Φ(xk ) − Φvig 2Fκ +

i=1 k∈Cig

k=1 K 

N  

Φ(xk ) − Φχ 2Fκ =

N  

Since (14) holds, we obtain

|Cig |Φvig − Φχ 2Fκ (15)

i=1

Φ(xk ) − Φvi 2Fκ +

i=1 k∈Ci

N

N 

g g i=1 |Ci |Φvi

N 

|Ci |Φvi − Φχ 2Fκ . (16)

i=1

−

Φχ 2Fκ

≥

N i=1

|Ci |Φvi − Φχ 2Fκ 

504

4.2

D. Bacciu and A. Starita

Correct Division and Location for CoRe Clustering

To evaluate the correct division and location properties we first analyze the case when the number of units N is equal to the true cluster number nc . Consider the loss in (4) as being decomposed into a winner and a loser dependent term, i.e. Je (χ, U ) = Jewin (χ, U )+Jelose (χ, U ). By definition, Jewin (χ, U ) = nc K i=1 k=1 δik (1 − ϕi (xk )) must have at least one minimum point. Applying the necessary condition ∂Jewin (χ, U )/∂ci = 0 we obtain an estimate of the prototypes by means of fixed point iteration, that is N e k=1 δik ϕi (xk )xk ci =  . (17) N k=1 δik ϕi (xk ) When the number of prototypes equals the number of clusters, the fixed point iteration in (17) converges by positioning each unit weight vector close to the true cluster centroids. In addition, it can be shown that (17) approximates a local minima of the kernel vector quantization loss in (9). To prove this, consider the CoRe loss formulation in kernel space (7): we have Jewin (χ, U ) = 1 nc K 2 i=1 k=1 δik Φ(xk ) − Φ(ci )Fκ , where ci is estimated by (17). 2 Now, consider the VQ loss in (9): a necessary condition for its  minimization requires the computation of the cluster centroids as Φvi = |C1i | k∈Ci Φ(xk ). The exact calculation of Φvi requires to know the form of the implicit nonlinear mapping Φ to solve the so-called pre-image problem [10], that is determining z such that Φ(z) = Φvi . Unfortunately, such a problem is insolvable in the general case [10]. However, instead of calculating the exact pre-image we can search an approximation by seeking z minimizing ρ(z) = Φvi − Φ(z)2Fκ , that is the feature space distance between the centroid in kernel space and the mapping of its approximated pre-image. Rather than optimizing ρ(z), it is easier to minimize the distance between Φvi and its orthogonal projection onto the span Φ(z). Due to space limitations, we omit the technicalities of this calculation (see [10] for further details). It turns out that the minimization of ρ(z) reduces to the the evaluation of the gradient of ρ (z) = Φvi , Φ(z) . By substituting the definition of Φvi and applying the kernel trick we obtain   ρ (z) = (1/|Ci |) κ(xk , xj ) + κ(z, z) + (1/|Ci |) κ(xk , z) k,j∈Ci

k∈Ci

where κ(z, z) = 1 since we are using a gaussian kernel. Differentiating ρ (z) with respect to z and solving by fixed point iteration yields  e−1 )xk k∈Ci κ(xk , z e z =  (18) e−1 ) k∈Ci κ(xk , z that is the same as the prototype estimate obtained in (17) for gaussian kernels centered in z e . The indicator function δik in (17) is not null only for those points xk for which unit ui was in the winner set. This does not ensures the partition conditions over χ, since, by definition of wink , some points can be associated with

Convergence Behavior of Competitive Repetition-Suppression Clustering

505

two or more winners. However, by (6) we know that the variance of the winners tends to reduce as learning proceeds. Therefore, using the same arguments by Gersho [8] it can be demonstrated that, after a certain epoch E, the CoRe winners competition will become a WTA process where δik will be ensuring the partition conditions over χ. Summarizing, the minimization of the CoRe winners error Jewin (χ, U ) generates an approximate solution to the vector quantization problem in feature space in (9). As a consequence, the prototypes ci become a local solution satisfying the conditions of Proposition 1. Hence, substituting the definition of Φχ in the results of Proposition 1 we obtain that {c1 , . . . , cnc } is an approximated global minimum for (9) if and only if nc  K  |Ci | i=1 k=1

K

Φ(ci ) − Φ(xk )2Fκ ≥

nc  K  |C˜i | i=1 k=1

K

Φ(˜ ci ) − Φ(xk )2Fκ

(19)

holds for every {˜ c1 , . . . , c˜nc } that are approximated pre-images of a local minimum for (9). In summary, a global optimum to (9) should minimize the feature space distance between the prototypes and samples belonging to their cluster while maximizing the weight vector distance from the sample mean, or, equivalently, the distance from all the samples in the dataset χ. The loser component Jelose (χ, U ) in the kernel CoRe loss (7) depends on the term (1 − (1/2)Φ(ci ) − Φ(xk )2Fκ that maximizes the distance between the prototypes ci and those xk that do not fall in the respective Voronoi sets Ci . Hence, Jelose (χ, U ) produces a distortion in the estimate of ci that pursues the global optimality criterion except for the fact that it discounts the repulsive effect of the xk ∈ Ci . In fact, (19) suggests that ci has to be repelled by all the xk ∈ χ. On the other hand, the estimate ci is a linear combination of the xk ∈ Ci : applying the repulsive effect in (19) would subtract their contribution, either canceling the attractive effect (which would be catastrophic) or simply scaling the magnitude of the learning step without changing the final direction. Hence, the CoRe loss makes a reasonable assumption discarding the repulsive effect of the xk ∈ Ci when calculating the estimate of ci . Summarizing, CoRe locates the prototypes close to the centroids of the nc clusters by means of (17), escaping from local minima of the loss function by approximating the global minimum condition of Proposition 1. Finally, we need to study the behavior of Je (χ, U ) as the number of units N varies with respect to the true cluster number nc . Using the same motivations in [4], we see that the winner-dependent loss Jewin tends to reduce as the the number of units increases. However, if the number of units falling into G is larger than nc there will be a number of clusters that are erroneously split. Therefore, the samples from these clusters will tend to produce an increased level of error in Jelose contrasting the reduction of Jewin . On the other hand, Jelose will tend to reduce when the number of units inside G is lower than nc . This however will produce increased levels of Jewin since the prototype allocation won’t match the underlying sample distribution. Hence, the CoRe error will have its minimum when the number of units inside G will approximate nc .

506

5

D. Bacciu and A. Starita

Conclusion

The paper presents a sound analysis of the convergence behavior of CoRe clustering, showing how the minimization of the CoRe cost function satisfies the properties of separation nature, correct division and location [4]. As the loss reduces to a minimum, the CoRe algorithm is shown to converge allocating the correct number of prototypes to the centers of the clusters. Moreover, it is given a sound optimality criterion that shows how CoRe gradient descent pursues a global minimum of the vector quantization problem in feature space. The results presented in the paper hold for a batch gradient descent process. However, it can be proved that, under Ljung’s conditions [11], they can be extended to stochastic (online) gradient descent. Moreover, we plan to investigate further the properties of the CoRe kernel formulation, extending the convergence analysis to a wider class of activation functions other than gaussians, i.e. normalized kernels.

References 1. Grill-Spector, K., Henson, R., Martin, A.: Repetition and the brain: neural models of stimulus-specific effects. Trends in Cognitive Sciences 10(1), 14–23 (2006) 2. Bacciu, D., Starita, A.: A robust bio-inspired clustering algorithm for the automatic determination of unknown cluster number. In: Proceedings of the 2007 International Joint Conference on Neural Networks, pp. 1314–1319. IEEE, Los Alamitos (2007) 3. Xu, L., Krzyzak, A., Oja, E.: Rival penalized competitive learning for clustering analysis, rbf net, and curve detection. IEEE Trans. on Neur. Net. 4(4) (1993) 4. Ma, J., Wang, T.: A cost-function approach to rival penalized competitive learning (rpcl). IEEE Trans. on Sys., Man, and Cyber 36(4), 722–737 (2006) 5. Munoz-Perez, J., Gomez-Ruiz, J.A., Lopez-Rubio, E., Garcia-Bernal, M.A.: Expansive and competitive learning for vector quantization. Neural Process. Lett. 15(3), 261–273 (2002) 6. Bacciu, D., Starita, A.: Competitive repetition suppression learning. In: Kollias, S., Stafylopatis, A., Duch, W., Oja, E. (eds.) ICANN 2006. LNCS, vol. 4131, pp. 130–139. Springer, Heidelberg (2006) 7. Scholkopf, B., Smola, A., Muller, K.R.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Comp. 10(5), 1299–1319 (1998) 8. Yair, E., Zeger, K., Gersho, A.: Competitive learning and soft competition for vector quantizer design. IEEE Trans. on Sign. Proc. 40(2), 294–309 (1992) 9. Spath, H.: Cluster analysis algorithms. Ellis Horwood (1980) 10. Scholkopf, B., Mika, S., Burges, C.J.C., Knirsch, P., Muller, K.R., Ratsch, G., Smola, A.J.: Input space versus feature space in kernel-based methods. IEEE Trans. on Neur. Net. 10(5), 1000–1017 (1999) 11. Ljung, L.: Strong convergence of a stochastic approximation algorithm. The Annals of Statistics 6(3), 680–696 (1978)

Self-Organizing Clustering with Map of Nonlinear Varieties Representing Variation in One Class Hideaki Kawano, Hiroshi Maeda, and Norikazu Ikoma Kyushu Institute of Technology, Faculty of Engineering, 1-1 Sensui-cho Tobata-ku Kitakyushu, 804-8550, Japan [email protected]

Abstract. Adaptive Subspace Self-Organizing Map (ASSOM) is an evolution of Self-Organizing Map, where each computational unit defines a linear subspace. Recently, its modified version, where each unit defines an linear manifold instead of the linear subspace, has been proposed. The linear manifold in a unit is represented by a mean vector and a set of basis vectors. After training, these units result in a set of linear variety detectors. In another point of view, we can consider the AMSOM represents the latent commonality of data as linear structures. In numerous cases, however, these are not enough to describe the latent commonality of data because of its linearity. In this paper, the nonlinear variety is considered in order to represent a diversity of data in a class. The effectiveness of the proposed method is verified by applying it to some simple classification problems.

1

Introduction

The subspace method is popular in pattern recognition, feature extraction, compression, classification and signal processing.[1] Unlike other techniques where classes are primarily defined as regions or zones in the feature space, the subspace method uses linear subspaces that are defined by a set of normalized basis vectors. One linear subspace is usually associated with one class. An input vector is classified to a particular class if its projection error into the subspace associated with one class is the minimum. The subspace method, as compared to other pattern recognition techniques, has advantages in applications where the relative intensities or energies of the vector components are more important than the overall level of the signal. It also provides an economical representation for groups of vectors with high dimensionality, since one can often use a small set of basis vectors to approximate the subspace where the vectors reside. Another paradigm is to use is use a mixture of local subspace to collectively model the data space. Adaptive-Subspace Self-Organizing Map (ASSOM)[2][3] is a mixture of local subspace method for pattern recognition. ASSOM, which is an evolution M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 507–516, 2008. c Springer-Verlag Berlin Heidelberg 2008 

508

H. Kawano, H. Maeda, and N. Ikoma

of Self-Organizing Map (SOM),[4] consists of an input layer and a competitive layer arranging some computational units in a line or a lattice structure. Each computational units defines a subspace spanned by some basis vectors. ASSOM creates a set of subspaces representations by competitive selection and cooperative learning. In SOM, a set of reference vectors is spatially organized to partition the input space. In ASSOM, a set of reference sub-models is topologically ordered, with each sub-model responsible for describing a specific region of the input space by its local principal subspace. The ASSOM is attractive not only because it inherits the topographic representation property in the SOM, but also because the learning results of ASSOM can faithfully describe the the core features of various transformation groups. The simulation results in the reference [2] and the reference [3] have illustrated that different feature filters can be self-organized to different low-dimensional subspaces and a wavelet type representation does emerge in the learning. Recently, Adaptive Manifold Self-Organizing Map (AMSOM) which is a modified version of ASSOM has been proposed.[5] AMSOM is the same structure as the ASSOM, except for the way to represent each computational unit. Each unit in AMSOM defines an affine subspace which is composed of a mean vector and a set of basis vectors. By incorporating a mean vector into each unit, the recognition performance has been improved significantly. The simulation results in the reference [5] have been shown that AMSOM outperforms linear PCA-based method and ASSOM in face recognition problem. In both ASSOM and AMSOM, a local subspace in each unit can be adapted by linear PCA learning algorithms. On the other hand, it is known that there are a number of advantages in introducing nonlinearities into a PCA type network with reproducing kernels.[6][13] For example, the performance of the subspace method is affected by the dimensionality for the intersections of subspaces.[1] In other words, the dimensionality of subspace should be as possible as low in order to achieve successful performance. it is, however, not enough to describe variation in a class of patterns by low dimensional subspace because of its linearity. From this consideration, we propose a nonlinear extended version of the AMSOM with the reproducing kernels. The proposed method could be expected to construct nonlinear varieties so that effective representation of data belonging to the same category is achieved with low dimensionality. The effectiveness of the proposed method is verified by applying it to some simple pattern classification problems.

2

Adaptive Manifold Self-Organizing Map (AMSOM)

In this section, we give a brief review of the original AMSOM. Fig.1 shows the structure of the AMSOM. It consists of an input layer and a competitive layer, in which n and M units are included respectively. Suppose i ∈ {1, · · · , M } is used to index computational units in the competitive layer, the dimensionality of the input vector is n. The i-th computational unit constructs an affine subspace, which is composed of a mean vector μ(i) and a subspace spanned by H basis

Self-Organizing Clustering with Map

x1

xj

509

xn

Input Layer

1

M

i

Competitive Layer

Fig. 1. A structure of the Adaptive Manifold Self-Organizing Map (AMSOM)

input vector :

relative input vector :

x ~ ~ x=

φ

φ

φ^ mean vector : μ

x^

Subspace

Origin

Fig. 2. Affine Subspace in a computational unit (i)

vectors bh , h ∈ {1, · · · , H}. First of all, we define the orthogonal projection of a input vector x onto the affine subspace of i-th unit as x ˆ(i) = μ(i) +

H 

T (i)

(i)

(φ(i) bh )bh ,

(1)

h=1

where φ(i) = x − μ(i) . Therefore the projection error is represented as x ˜(i) = φ(i) −

H 

T (i)

(i)

(φ(i) bh )bh .

(2)

h=1

Figure 2 shows a schematic interpretation for the orthogonal projection and the projection error of a input vector onto the affine subspace defined in a unit i. The AMSOM is more general strategy than the ASSOM, where each computational unit solely defines a subspace. To illustrate why this is so, let us consider a very simple case: Suppose we are given two clusters as shown in Fig.3(a). It

510

H. Kawano, H. Maeda, and N. Ikoma

x2

x2 class 1 class 2

manifold 1

b1

class 1 class 2

μ1

O

x1

O

μ2

x1 b2

manifold 2

(b)

(a)

Fig. 3. (a) Clusters in 2-dimensional space: An example of the case which can not be separated without a mean value. (b) Two 1-dimensional affine subspaces to approximate and classify clusters.

is not possible to use one dimensional subspaces, that is lines intersecting the origin O, to approximate the clusters. This is true even if the global mean is removed, so that the origin O is translated to the centroid of the two clusters. However, two one-dimensional affine subspaces can easily approximate the clusters as shown in Fig.3(b), since the basis vectors are aligned in the direction that minimizes the projection error. In the AMSOM, the input vectors are grouped into episodes in order to present them to the network as an input sets. For pattern classification, an episode input is defined as a subset of training data belonging to the same category. Assume that the number of input vectors in the subset is E, then an episode input ωq in the class q is denoted as ωq = {x1 , x2 , · · · , xE } , ωq ⊆ Ωq , where Ωq is a set of training patterns belonging to the class q. The set of input vectors of an episode has to be recognized as one class, such that any member of this set and even an arbitrary linear combination of them should have the same winning unit. The training process in AMSOM has the following steps: (a) Winner lookup. The unit that gives the minimum projection error for an episode is selected. The unit is denoted as the winner, whose index is c. This decision criterion for the winner c is represented as  E   (i) 2 c = arg min ||˜ xe || , (3) i

where i ∈ {1, · · · , M }.

e=1

Self-Organizing Clustering with Map

(b) Learning. For each unit i, and for each xe , update μ(i)   μ(i) (t + 1) = μ(i) (t) + λm (t)hci (t) xe − μ(i) (t) ,

511

(4)

where λm (t) is the learning rate for μ(i) at learning epoch t, hci (t) is the neighborhood function at learning epoch t with respect to the winner c. Both λm (t) and hci (t) are monotonic decreasing function with respect to t. f in f in t/tmax In this paper, λm (t) = λini and hci (t) = exp(−|c − i|/γ(t)), m (λm /λm ) ini f in f in t/tmax γ(t) = γ (γ /γ ) are used. Then the basis vectors are updated as (i)

(i)

(i)

bh (t + 1) = bh (t) + λb (t)hci (t)

(i)

φe (t)T bh (t) , φ(i) e (t), (i) ˆ(i) ||φ (t)||||φ (t)|| e e

(5)

(i)

where φe (t) is the relative input vector in the manifold i updated the mean (i) ˆ(i) vector, which is represented by φe (t) = xe − μ(i) (t + 1), φ e (t) is the orthogonal projection of the relative input vector, which is represented by H (i) (i) T (i) ˆ(i) φ e (t) = h=1 (φ (t) bh (t))bh (t) and λb (t) is the learning rate for the basis vectors, which is also monotonic decreasing function with respect to t. f in f in t/tmax In this paper, λb (t) = λini is used. b (λb /λb ) After the learning phase, a categorization phase to determine the class association of each unit. Each unit is labeled by the class index for which is selected as the winner most frequently when the input data for learning are applied to the AMSOM again,

3 3.1

Self-Organizing Clustering with Nonlinear Varieties Reproducing Kernels

Reproducing kernels are functions k : X 2  → R which for all pattern sets {x1 , · · · , xl } ⊂ X

(6)

give rise to positive matrices Kij := k(xi , xj ). Here, X is some compact set in which the data resides, typically a subset of Rn . In the field of Support Vector Machine (SVM), reproducing kernels are often referred to as Mercer kernels. They provide an elegant way of dealing with nonlinear algorithms by reducing them to linear ones in some feature space F nonlinearly related to input space: Using k instead of a dot product in Rn corresponds to mapping the data into a possibly high-dimensional dot product space F by a (usually nonlinear) map Φ : Rn  → F , and taking the dot product there, i.e.[11] k(x, y) = (Φ(x), Φ(y)) .

(7)

By virtue of this property, we shall call Φ a feature map associated with k. Any linear algorithm which can be carried out in terms of dot products can be made

512

H. Kawano, H. Maeda, and N. Ikoma

nonlinear by substituting a priori chosen kernel. Examples of such algorithms include the potential function method[12], SVM [7][8] and kernel PCA.[9] The price that one has to pay for this elegance, however, is that the solutions are only obtained as expansions in terms of input patterns mapped into feature space. For instance, the normal vector of an SV hyperplane is expanded in terms of Support Vectors, just as the kernel PCA feature extractors are expressed in terms of training examples, Ψ=

l 

αi Φ(xi ).

(8)

i=1

3.2

AMSOM in the Feature Space

The AMSOM in the high-dimensional feature space F is considered. In the proposed method, an varieties defined by a computational unit i in the competitive layer take the nonlinear form Mi = {Φ(x)|Φ(x) = Φ(μ(i) ) +

H 

(i)

ξΦ(bh )},

(9)

h=1

where ξ ∈ R. Given training data set {x1 , · · · , xN }, the mean vector and the basis vector in a unit i are represented by the following form Φ(μ(i) ) =

N 

(i)

(10)

(i)

(11)

αl Φ(xl ),

l=1

(i)

Φ(bh ) =

N 

βhl Φ(xl ),

l=1 (i)

(i)

respectively. αl in Eq.(10) and βhl in Eq.(11) are the parameters adjusted by learning. The derivation of training procedure in the proposed method is given as follows: (a) Winner lookup. The norm of the orthogonal projection error onto the i-th affine subspace with respect to present input xp is calculated as follows: (i)

||Φ(˜ xp ) ||2 = k(xp , xp ) +

H 

(i) 2

Ph

h=1

−2

N 

αl k(x, xl ) + 2

l=1

−2

H  N  h=1 l=1

+

N  N 

(i) (i)

αl1 αl2 k(xl1 , xl2 )

l1 =1 l2 =1 H  N 

N 

(i)

Ph αl1 βhl2 k(xl1 , xl2 )

h=1 l1 =1 l2 =1 (i)

Ph βhl k(x, xl ),

(12)

Self-Organizing Clustering with Map

513

(i)

where Ph means the orthogonal projection component of present input xp (i) into the basis Φ(bh ) and it is calculated by (i)

Ph =

N 

N  N 

(i)

βhl k(xp , xl )−

l=1

(i) (i)

αl1 βhl2 k(xl1 , xl2 ).

(13)

l1 =1 l2 =1

The reproducing kernels used generally are as follows: = (xTs xt )d d ∈ N, T d = (xs xt + 1) d ∈ N,   ||xs −xt ||2 k(xs , xt ) = exp − 2σ2 σ ∈ R, k(xs , xt ) k(xs , xt )

(14) (15) (16)

where N and R are the set of natural numbers and the set of reals, respectively. Eq.(14), Eq.(15) and Eq.(16) are referred as to homogeneous polynomial kernels, non-homogeneous polynomial kernels and gaussian kernels, respectively. The winner for an episode input ωq = {Φ(x1 ), · · · , Φ(xE )} is decided by the same manner as the AMSOM as follows:  E   (i) 2 c = arg min ||Φ(˜ xe ) || , i ∈ {1, · · · , M }. (17) i

e=1 (i)

(i)

(b) Learning. The learning rule for αl and βhl are as follows: ⎧ (i) ⎪ f or l = e ⎨−αl (t)λm (t)hci (t) (i) Δαl = −α(i) (t)λm (t)hci (t) + λm (t)hci (t) , ⎪ ⎩ l f or l = e ⎧ (i) (i) −α (t + 1)λb (t)hci (t)Th (t) ⎪ ⎪ ⎪ l ⎪ f or l = e ⎨ (i) Δβhl = −α(i) , (t + 1)λ (t)h (t)T (t) b ci l ⎪ ⎪ ⎪ +λ (t)h (t)T (t) b ci ⎪ ⎩ f or l = e where

(i)

T (t) = ˆ(i) Φ(φ e (t))) =

 H  N h=1

(i)

l=1

||Φ(φ(i) e )(t)|| = k(xe , xe ) − 2

N  l=1

(19)

(i)

Φ(φe (t))T Φ(bh (t)) , (i) ˆ(i) ||Φ(φ e (t))||||Φ(φe (t))|| βhl k(xe , xl ) −

(18)

N  N 

(20)

2  12 αl1 βkl2 k(xl1 , xl2 ) ,

l1 =1 l2 =1

(i)

αl k(xe , xl ) +

N  N 

(21)  12 (i) (i) αl1 αl2 k(xl1 , xl2 ) ,

l1 =1 l2 =1

(22)

514

H. Kawano, H. Maeda, and N. Ikoma (i)

T Φ(φ(i) e (t)) Φ(bh (t)) =

N 

βhl k(xe , xl ) −

l=1

N  N 

(i) (i)

αl1 βhl2 k(xl1 , xl2 ). (23)

l1 =1 l2 =1

In Eqs.(18) and (19), λm (t), λb (t) and hci (t) are the same parameters as mentioned in the AMSOM training process. After the learning phase, a categorization phase to determine the class association of each unit. The procedure of the categorization is the same manner as mentioned in previous section.

4

Experimental Results

Data Distribution 1. To analyze the effect of reducing the dimensionality for the intersections of subspaces by the proposed method, the data as shown in Fig.4(a) is used. For this data, although a set of each class can be approximated by 1-dimensional linear manifold in the input space R2 , the intersections of subspace could be happend between class 1 and class 2, and between class 2 and class 3 , even if the optimal linear manifold for each class can be obtained. However, the linear manifold in the high-dimensional space, that is the nonlinear manifold in input space, can be expected to classify the given data by reduction effect of the dimensionality for the intersections of subspaces. As the result of simulation, the given input data are classified correctly by the proposed method. Figure 4(b) and (c) are the decision regions constructed by AMSOM and the proposed method, respectively. In this experiment, 3 units were used in the competitive layer. The class associations to each unit are class 1 to unit 1, class 2 to unit 2, class 3 to unit 3, respectively. In this simulation, the experimental parameters are assigned as follows: the f in ini totoal number of epochs tmax = 100, λini = 0.1, m = 0.1, λm = 0.01, λb f in ini ini λb = 0.01, γ = 3, γ = 0.03, both in AMSOM and in the proposed method in common, H = 1 in AMSOM, and H = 2, k(x, y) = (xT y + 1)3 in the proposed method. From the experiment, it was shown that the the proposed method has the ability to reduce the dimensionality for intersections of subspaces. Data Distribution 2. To verify that the proposed method has the ability to describe the nonlinear manifolds, the data as shown in Fig.5(a) is used. This case is of impossible to describe each class by a linear manifold, that is 1-dimensional affine subspace. As the result of simulation, the given input data are classified correctly by the proposed method. Figure 5(b) and (c) are the decision regions constructed by AMSOM and the proposed method, respectively. In this experiment, 3 units were used in the competitive layer. The class associations to each unit are class 1 to unit 3, class 2 to unit 2, class 3 to unit 1, respectively. In this simulation, the experimental parameters are assigned as follows: the

Self-Organizing Clustering with Map 5

5

4

4

Unit 3

3

3

2

2

1

Unit 1

1

Unit 1

Unit 2

0

x2

x2

515

-1

-2

-2

-3

-3

-4

-4

-5 -5 -4 -3 -2 -1

0

x1

1

2

3

4

Unit 2

0

-1

Unit 3

-5 -5 -4 -3 -2 -1

5

(b)

(a)

0

x1

1

2

3

4

5

(c)

Fig. 4. (a) Training data used in the second experiment. (b) Decision regions learned by AMSOM and (c) the proposed method.

4

4

4

class 1 class 2 class 3

3

2

2

2

1

1

1

0

x2

3

x2

x2

3

0

0

-1

-1

-1

-2

-2

-2

-3

-3

-4

-3

-4 -4

-3

-2

-1

0

1

x1

(a)

2

3

4

-4 -4

-3

-2

-1

0

x1

(b)

1

2

3

4

-4

-3

-2

-1

0

1

2

3

4

x1

(c)

Fig. 5. (a) Training data used in the second experiment. (b) Decision regions learned by AMSOM and (c) the proposed method. f in ini totoal number of epochs tmax = 100, λini = 0.1, m = 0.1, λm = 0.01, λb f in λb = 0.01, γ ini = 3, γ ini = 0.03, both in AMSOM and in the proposed method in common, H = 1 in AMSOM, and H = 2, k(x, y) = (xT y)2 in the proposed method. From the experiment, it was shown that the the proposed method has the ability to extract the suitable nonlinear manifolds efficiently.

5

Conclusions

A clustering method with map of nonlinear varieties was proposed as a new pattern classification method. The proposed method has been extended to a nonlinear method easily from AMSOM by applying the kernel method. The effectiveness of the proposed method were verified by the experiments. The proposed algorithm has highly promising applications of the ASSOM in a wide area of practical problems.

516

H. Kawano, H. Maeda, and N. Ikoma

References 1. Oja, E.: Subspace Methods of Pattern Recognition. Research Studies Press (1983) 2. Kohonen, T.: Emergence of Invariant-Feature Detectors in the Adaptive-Subspace Self-Organizing Map. Biol.Cybern 75, 281–291 (1996) 3. Kohonen, T., Kaski, S., Lappalainen, H.: Self-Organizing Formation of Various invariant-feature filters in the Adaptive Subspace SOM. Neural Comput 9, 1321– 1344 (1997) 4. Kohonen, T.: Self-Organizing Maps. Springer, Berlin, Heidelberg, New York (1995) 5. Liu, Z.Q.: Adaptive Subspace Self-Organizing Map and Its Application in Face Recognition. International Journal of Image and Graphics 2(4), 519–540 (2002) 6. Saitoh, S.: Theory of Reproducing Kernels and its Applications. In: Longman Scientific & Technical, Harlow, England (1988) 7. Cortes, C., Vapnik, V.: Support Vector Networks. Machine Learning 20, 273–297 (1995) 8. Vapnik, V.: The Nature of Statistical Learning Theory, 2nd edn. Springer, New York, Berlin, Heidelberg (1995) 9. Sch¨ olkopf, B., Smola, A.J., M¨ uler, K.R.: Nonlinear Component Analysis as a Kernel Eigenvalue Problem, Technical Report 44, Max-Planck-Institut fur biologische Kybernetik (1996) 10. Mika, S., R¨ atsch, G., Weston, J., Sch¨ olkopf, B., M¨ uler, K.R.: Fisher discriminant analysis with kernels. Neural Networks for Signal Processing IX, 41–48 (1999) 11. Boser, B.E., Guyon, I.M., Vapnik, V.N.: A Training Algorithm for Otimal Margin Classifiers. In: Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory, pp. 144–152 (1992) 12. Aizerman, M., Braverman, E., Rozonoer, L.: Theoretical Foundations of the Potential Function Method in Pattern Recognition Learning. Automation and Remote Control 25, 821–837 (1964) 13. Sch¨ olkopf, B., Smola, A.J.: Learning with Kernels. The MIT Press, Cambridge (2002)

An Automatic Speaker Recognition System P. Chakraborty1, F. Ahmed1, Md. Monirul Kabir2, Md. Shahjahan1, and Kazuyuki Murase2,3 1

Department of Electrical & Electronic Engineering, Khulna University of Engineering and Technology, Khulna-920300, Bangladesh 2 Dept. of Human and Artificial Intelligence Systems, Graduate School of Engineering 3 Research and Education Program for Life Science, University of Fukui, 3-9-1 Bunkyo, Fukui 910-8507, Japan [email protected], [email protected]

Abstract. Speaker Recognition is the process of identifying a speaker by analyzing spectral shape of the voice signal. This is done by extracting & matching the feature of voice signal. Mel-frequency Cepstrum Co-efficient (MFCC) is the feature extraction technique in which we will get some coefficients named Mel-Frequency Cepstrum coefficient. This Cepstrum Coefficient is extracted feature. This extracted feature is taken as the input of Vector Quantization process. Vector Quantization (VQ) is the typical feature matching technique in which VQ codebook is generated by providing predefined spectral vectors for each speaker to cluster the training vectors in a training session. Finally test data are provided for searching the nearest neighbor to match that data with the trained data. The result is to recognize correctly the speakers where music & speech data (Both in English & Bengali format) are taken for the recognition process. The correct recognition is almost ninety percent. It is comparatively better than Hidden Markov model (HMM) & Artificial Neural network (ANN). Keywords: MFCC- Mel-Frequency Cepstrum Co-efficient, DCT: Discrete cosine Transform, IIR: - Infinite impulse response, FIR: - Finite impulse response, FFT: - Fast Fourier Transform, VQ: - Vector Quantization.

1 Introduction Speaker Recognition is the process of automatic recognition of the person who is speaking on the basis of individual information included in speech waves. This paper deals with the automatic Speaker recognition system using Vector Quantization. There are another techniques for speaker recognition such as Hidden Markov model (HMM), Artificial Neural network (ANN) for speaker recognition. We have used VQ because of its less computational complexity [1]. There are two main modulesfeature extraction and feature matching in any speaker recognition system [1, 2]. The speaker specific features are extracted using Mel-Frequency Cepstrum Co-efficient (MFCC) processor. A set of Mel-frequency cepstrum coefficients was found, which are called acoustic vectors [3]. These are the extracted features of the speakers. These M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 517–526, 2008. © Springer-Verlag Berlin Heidelberg 2008

518

P. Chakraborty et al.

acoustic vectors are used in feature matching using vector quantization technique. It is the typical feature matching technique in which VQ codebook is generated using trained data. Finally tested data are provided for searching the nearest neighbor to match that data with the trained data. The result is to recognize correctly the speakers where music & speech data (Both in English & Bengali format) are taken for the recognition process. This work is done with about 70 spectral data. The correct recognition is almost ninety percent. It is comparatively better than Hidden Markov model (HMM) & Artificial Neural network (ANN) because the correct recognition for HMM & ANN is below ninety percent. The future work is to generate a VQ codebook with many pre-defined spectral vectors. Then it will be possible to add many trained data in that codebook in a training session, but the main problem is that the network size and training time become prohibitively large with increasing data size. To overcome these limitations, time alignment technique can be applied, so that continuous speaker recognition system becomes possible. There are several implementations for feature matching & identification. Lawrence Rabiner & B. H. Juang proposed Mel- frequency cepstrum co-efficient (MFCC) method to extract the feature & Vector Quantization as feature matching technique [1]. Lawrence Rabiner & R.W. Schafer discussed the performance of MFCC Processor by following several theoretical concepts [2]. S. B. Davis & P. Mammelstein described the characteristics of acoustic speech [3]. Linde A Buzo & R. Gray proposed the LBG Algorithm to generate a VQ codebook by splitting technique [4]. S. Furui describes the speaker independent word recognition using dynamic features of speech spectrum [5]. S. Furui also proposed the overall speaker recognition technology using MFCC & VQ method [6].

2 Methodology A general model for speaker recognition system for several people is shown in fig: 1. the model consists of four building blocks. The first is data extraction that converts a wave data stored in audio wave format into a form that is suitable for further computer processing and analysis. The second is pre-processing, which involves filtering,

Fig. 1. Block diagram of speaker recognition system

An Automatic Speaker Recognition System

519

removing pauses, silences and weak unvoiced sound signal and detect the valid speech signal. The third block is feature extraction, where speech features are extracted from the speech signal. The selected features have enough information to recognize a speaker. Here a class label is assigned to each word uttered by each speaker by examining the extracted features and comparing them with classes learnt during the training phase. Vector quantization is used as an identifier.

3 Pre-processing A digital filter is a mathematical algorithm implemented in hardware and/or software that operates on a digital input signal to produce a digital output signal for the purpose of achieving a filtering objective. Digital filters often operate on digitized analog signals or just numbers, representing some variable, stored in a computed memory. Digital filters are broadly divided into two classes, namely infinite impulse response (IIR) and finite impulse response (FIR) filters. We chose FIR filter because, FIR filters can have an exactly linear phase response. The implication of this is that no phase distortion is introduced into the signal by the filter. This is an important requirement in many applications, for example data transmission, biomedicine, digital audio and image processing. The phase responses of IIR filters are non-linear, especially at the band edges. When a machine is continuously listening to speech, a difficulty arises when it is trying to figure out to where a word starts and stops. We solved this problem by examining the magnitude of several consecutive samples of sound. If the magnitude of these samples is great enough, then keep those samples and examine them later. [1]

Fig. 2. Sample Speech Signal

Fig. 3. Example of speech signal after it has been cleaned

One thing that is clearly noticeable in the example speech signal is that there is lots of empty space where nothing is being said, so we simply remove it. An example speech signal is shown before cleaning in Figure 2, and after in Figure 3. After the empty space is removed from the speech signal, the signal is much shorter. In this case the signal had about 13,000 samples before it was cleaned. After it was run through the clean function, it only contained 2,600 samples. There are several advantages of this. The amount of time required to perform calculations on 13,000 samples is much larger than that required for 2,600 samples. The cleaned sample now contains all the important data that is required to perform the analysis of the speech. The sample produced from the cleaning process is then fed in to the other parts of the ASR system.

520

P. Chakraborty et al.

4 Feature Extraction The purpose of this module is to convert the speech waveform to some type of parametric representation (at a considerably lower information rate) for further analysis and processing. This is often referred as the signal-processing front end. A wide range of possibilities exist for parametrically representing the speech signal for the speaker recognition task, namely: • • • • •

Linear Prediction Coding (LPC), Mel-Frequency Cepstrum Coefficients (MFCC), Linear Predictive Cepstral Coefficients (LPCC) Perceptual Linear Prediction (PLP) Neural Predictive Coding (NPC)

Among the above classes we used MFCC, because it is the best known and most popular. MFCC’s are based on the known variation of the human ear’s critical bandwidths with frequency; filters spaced linearly at low frequencies and logarithmically at high frequencies have been used to capture the phonetically important characteristics of speech. This is expressed in the mel-frequency scale, which is linear frequency spacing below 1000 Hz and a logarithmic spacing above 1000 Hz [1, 2]. 4.1 Mel-Frequency Cepstrum Processor A diagram of the structure of an MFCC processor is given in Figure 4. The speech input is typically recorded at a sampling rate above 10000 Hz. This sampling frequency was chosen to minimize the effects of aliasing in the analog-to-digital conversion. These sampled signals can capture all frequencies up to 5 kHz, which cover most energy of sounds that are generated by humans. As been discussed previously, the main purpose of the MFCC processor is to mimic the behavior of the human ears. In addition, rather than the speech waveforms themselves, MFCC’s are shown to be less susceptible to mentioned variations [5, 6].

Fig. 4. Block diagram of the MFCC processor

4.1.1 Frame Blocking In this step the continuous speech signal is blocked into frames of N samples, with adjacent frames being separated by M (M < N). The first frame consists of the first N samples. The second frame begins M samples after the first frame, and overlaps it by

An Automatic Speaker Recognition System

521

N - M samples. Similarly, the third frame begins 2M samples after the first frame (or M samples after the second frame) and overlaps it by N - 2M samples. This process continues until all the speech is accounted for within one or more frames. Typical values for N and M are N = 256 and M = 100. 4.1.2 Windowing Next processing step is windowing. By means of windowing the signal discontinuities, at the beginning and end of each frame, is minimized. The concept here is to minimize the spectrum distortion by using the window to taper the signal to zero at the beginning and end of each frame.If we define the window as w(n), 0 ≤ n≤ N - 1 , where N is the number of samples, then the result of windowing is the signal

yl (n) = xl (n) w(n), 0 ≤ n ≤ N − 1

(1)

The followings are the types of window method: ▪ Hamming ▪ Rectangular ▪ Barlett (Triangular) ▪ Hanning

▪ Kaiser ▪ LancZos ▪ Blackman-Harris

Among all the above, we used Hamming Window method most to serve our purpose for ease of mathematical computations, which is described as:

⎛ 2π n ⎞ w ( n ) = 0 . 54 − 0 .46 cos ⎜ (2) ⎟, 0 ≤ n ≤ N − 1 ⎝ N −1⎠ Besides this, we’ve also used Hanning window and Blackman-Harris window. As an example, a Hamming window with 256 samples is shown here.

Fig. 5. Hamming window with 256 speech sample

4.1.3 Fast Fourier Transform (FFT) Fast Fourier Transform, converts a signal from the time domain into the frequency domain. The FFT is a fast algorithm to implement the Discrete Fourier Transform (DFT) which is defined on the set of N samples {xn}, as follow:

Xn =

N −1

∑x k =0

k

e − 2 π jkn / N ,

n = 0 ,1, 2 ,..., N − 1

(3)

522

P. Chakraborty et al.

We use j here to denote the imaginary unit, i.e. j = √ (-1). In general Xn’s are complex numbers. The resulting sequence {Xn} is interpreted as follow: the zero frequency corresponds to n = 0, positive frequencies 0
mel ( f ) = 2595 * log 10(1 + f / 700)

(4)

One approach for simulating the subjective spectrum is to use a filter bank, spaced uniformly on the mel scale. That filter bank has a triangular band pass frequency response, and the spacing as well as the bandwidth is determined by a constant mel frequency interval. The modified spectrum of S (ω) thus consists of the output power of these filters when S (ω) is the input. The number of mel spectrum coefficients, K, is typically chosen as 20 [3]. 4.1.5 Cepstrum In this step the log mel-spectrum is converted back to the time domain. The cepstral representation of the speech spectrum provides a good representation of the local spectral properties of the signal for the given frame analysis. Because the melspectrum coefficients (and so their logarithm) are real numbers, we convert them to the time domain using the Discrete Cosine Transform (DCT). In this step we find Mel Frequency Cepstrum Coefficient (MFCC). This set of coefficients is called an acoustic vector. Therefore each input utterance is transformed into a sequence of acoustic vectors.

5 Speech Feature Matching There are several methods of Feature matching, which are stated below: • Template models − Vector Quantization (VQ) − Dynamic Time Wrapping (DTW) • Stochastic models − Gaussian Mixture Models (GMM) − Hidden Markov Modeling (HMM) • Neural Networks (NNs) • Support Vector Machines (SVMs) [5] We used VQ approach due to ease of implementation and high accuracy.

An Automatic Speaker Recognition System

5.1

523

Vector Quantization

The objective of VQ is the representation of a set of feature vectors by a set,

x ∈ X ⊆ ℜk

Y = { y1 ,........, y N C } , of NC reference vectors in ℜ k . Y is called

codebook and its elements codewords. The vectors of X are called also input patterns or input vectors. So, a VQ can be represented as a function: q : X → Y . The knowledge of q permits us to obtain a partition S of X constituted by the NC subsets Si (called cells):

S i = { x ∈ X : q( x ) = y i }

i = 1, ….,NC

(5)

In brief, VQ is a process of mapping vectors from a large vector space to a finite number of regions in that space. Each region is called a cluster and can be represented by its center, called a codeword. The collection of all codewords is called a codebook. Figure 6 shows a diagram to illustrate this recognition process. In the figure, only two speakers and two dimensions of the acoustic space are shown. The circles refer to the acoustic vectors from the speaker 1 while the triangles are from the speaker 2. In the training phase, a speaker-specific VQ codebook is generated for each known speaker by clustering his/her training acoustic vectors. The result codewords (centroids) are shown in Figure 6 by black circles and black triangles for speaker 1 and 2, respectively. The distance from a vector to the closest codeword of a codebook is called a VQ-distortion. In the recognition phase, an input utterance of an unknown voice is “vector-quantized” using each trained codebook and the total VQ distortion is computed. The speaker corresponding to the VQ codebook with smallest total distortion is identified.

Fig. 6. Diagram illustrating vector quantization codebook formation

One speaker can be discriminated from another based of the location of centroids. After the enrolment session, the acoustic vectors extracted from input speech of a speaker provide a set of training vectors. As described above, the next important step is to build a speaker-specific VQ codebook for this speaker using those training

524

P. Chakraborty et al.

vectors. There is a well-know algorithm, namely LBG algorithm [Linde, Buzo and Gray, 1980], for clustering a set of L training vectors into a set of M codebook vectors. The LBG VQ design algorithm is an iterative algorithm which alternatively solves the above two optimality criteria. The algorithm requires an initial codebook c(0). This initial codebook is obtained by the splitting method. In this method, an initial codevector is set as the average of the entire training sequence. This codevector is then split into two. The iterative algorithm is run with these two vectors as the initial codebook. The final two codevectors are splitted into four and the process is repeated until the desired number of codevectors is obtained. The algorithm is summarized here [4]. 1. Design a 1-vector codebook: This is the centroid of the entire set of training vectors (hence, no iteration is required here). 2. Double the size of the codebook by splitting each current codebook yn according to the rule.

3.

4. 5. 6.

yn+ = yn (1+ ∈)

(6)

yn− = yn (1− ∈)

(7)

Where n varies from 1 to the current size of the codebook and ∈ is a splitting Parameter (we choose ∈= 0.01 ) Nearest neighbor search: for each training vector, find the codeword in the current codebook that is closest (in terms of similarity measurement), and assign that vector to the corresponding cell (associated with the closest codeword). Centroid Update: update the code word in each cell using the centroid of the training vectors assigned to that cell. Iteration1: repeat steps 3 and 4 until the average distance falls below a preset threshold. Iteration 2: repeat steps 2, 3 and 4 until a codebook size of M is designed. [1,2].

5.2 The Testing Procedure The recognition algorithm can be summarized by the following steps. Step 1 : Unknown speakers speech is recorded first. Step 2 : The starting and endpoint is detected and speech should go through the filtering process. : Step 3 Speech features are extracted from the speech signal which is used to create the testing Vector (acoustic vector) for that utterances. : Step 4 The testing vector is then fed into the vector quantizer Step 5 : The predefined knowledge is used by the vector quantization to calculate the spectral distortion(distance) for each utterance and smallest distance value is selected. : Step 6 The smallest distance value is compared with a threshold value and a decision of whether the unknown speaker to be recognized or not is made[6].

An Automatic Speaker Recognition System

525

6 Result In this work, the utterances of several speakers are taken and the data are divided in music (rock and melody) and sample voice data (Bengali and English).Each sample data is taken to train the Vector Quantizer and then all the utterances are used for recognition or testing. The input of the VQ is obtained by the frequency analysis for the given input utterances. The detail of the VQ is specified by representing the input in the form of Matrix. We’ve taken about 70 data for which the result is shown in percentage below. Data Type Music Speech (English) Speech (Bengali)

Correct Recognition 86 % 92 % 91 %

False Inclusion 6% 3% 4%

False Rejection 8% 5% 5%

7 Conclusion This paper deals with the automatic Speaker recognition system using Vector Quantization. There are two main modules, feature extraction and feature matching. The speaker specific features are extracted using Mel-Frequency Cepstrum Coefficient (MFCC) processor. A set of Mel-frequency cepstrum coefficients was found, which are called acoustic vectors. These are the extracted features of the speakers. These acoustic vectors are used in feature matching using vector quantization technique. There are another techniques for feature matching such as Hidden Markov model (HMM), Artificial Neural network (ANN) for speaker recognition. We’ve used VQ as its computational complexity is less than others. Vector Quantization is the typical feature matching technique in which VQ codebook is generated using trained data. Finally tested data are provided for searching the nearest neighbor to match that data with the trained data. The result is to recognize correctly the speakers where music & speech data (Both in English & Bengali format) are taken for the recognition process. The correct recognition is almost ninety percent. It is comparatively better than Hidden Markov model (HMM) & Artificial Neural network (ANN) because the correct recognition for HMM & ANN is below ninety percent. The future work is to generate a VQ codebook with many pre-defined spectral vectors. Then it will be possible to add many trained data in that codebook in a training session, but the main problem is that the network size and training time become prohibitively large with increasing data size. To overcome these limitations, time alignment technique can be applied, so that continuous speaker recognition system becomes possible. Acknowledgement. This work was supported by grants to Shahjahan from KUET, and to KM from the Japanese Society for the Promotion of Science, the Yazaki Memorial Foundation for Science and Technology, and the University of Fukui.

526

P. Chakraborty et al.

References 1. Rabiner, L., Juang, B.H.: Fundamentals of Speech Recognition. Prentice Hall, Englewood Cliffs (1978) 2. Rabiner, L., Schafer, R.W.: Digital Processing of Speech Signals. Prentice Hall, Englewood Cliffs (1978) 3. Davis, S.B., Mermelstein, P.: Comparison of Parametric representations for monosyllabic word recognition in Continuously Spoken Sentences. IEEE Transactions on Acoustics Speech, Signal Processing ASSP-28(4) (August 1980) 4. Buzo, L.A., Gray, R.: An Algorithm for Vector Quantizer Design. IEEE Transactions on Communications 28, 84–95 (1980) 5. Furui, S.: Speaker Independent Isolated word Recognition using Dynamic Features of Speech Spectrum. IEEE Transactions on Acoustic, and Speech Signal Processing ASSP34(1), 52–59 (1986) 6. Furui, S.: An Overview of Speaker Recognition Technology. In: ESCA Workshop on Automatic Speaker Recognition, Identification & Verification, pp. 1–9 (1994)

Modified Modulated Hebb-Oja Learning Rule: A Method for Biologically Plausible Principal Component Analysis Marko Jankovic1, Pablo Martinez1, Zhe Chen2, and Andrzej Cichocki1 1

Laboratory for Advanced Brain Signal Processing, RIKEN Brain Science Institute, Wako-shi, Saitama, 351-0198, Japan {elmarkoni,cia}@brain.riken.jp 2 Neuroscience Statistics Research Laboratory, Departement of Anesthesia and Critical Care, Massachusetts General Hospital, Harvard Medical School, Boston, MA 02114, USA [email protected]

Abstract. This paper presents Modified Modulated Hebb-Oja (MHO) method that performs principal component analysis. Method is based on implementation of Time-Oriented Hierarchical Method applied on recently proposed principal subspace analysis rule called Modulated Hebb-Oja learning rule. Comparing to some other well-known methods for principal component analysis, the proposed method has one feature that could be seen as desirable from the biological point of view – synaptic efficacy learning rule does not need the explicit information about the value of the other efficacies to make individual efficacy modification. Simplicity of the “neural circuits” that perform global computations and a fact that their number does not depend on the number of input and output neurons, could be seen as good features of the proposed method. The number of necessary global calculation circuit is one. Some similarity to a part of the frog retinal circuit will be suggested, too. Keywords: Principal component analysis, time oriented hierarchy, Stiefel submanifold, local learning rules.

1 Introduction Neural networks provide a way for parallel on-line computations of the principal component analysis (PCA) or principal subspace analysis (PSA). Due to their parallelism and adaptivity to input data, such algorithms and their implementations in neural networks are potentially useful in feature extraction and data compression. It is well known that in the first step of any pattern recognition scheme, which is the representation of the objects from a usually large amount of raw data, some preprocessing and data compression are essential. In that case a minimal loss of information is a central objective. Principal subspace analysis (PSA) and principal component analysis (PCA) are powerful and popular tools that are used in statistical signal processing and data compression and they have objective to reduce the M. Ishikawa et al. (Eds.): ICONIP 2007, Part I, LNCS 4984, pp. 527–536, 2008. © Springer-Verlag Berlin Heidelberg 2008

528

M. Jankovic et al.

complexity of input data and at the same time keep as much as possible of the input data information. Within last years various PCA and PSA learning algorithms have been proposed and mathematically investigated (see [1-5], [8-15], [17-28]). Most of the proposed algorithms are based on local Hebbian learning - due to locality it has been argued that these algorithms are biologically plausible. In [11], biologically inspired PSA methods, named Modulated Hebbian (MH) and Modulated Hebb-Oja learning rules have been introduced. Major objectives for the methods derivation were: – to obtain a network which has a learning rule for individual synaptic efficacy that requires the least possible amount of explicit information about the other synaptic efficacies, especially those related to other neurons – minimization of the neural hardware that is necessary for implementation of the proposed learning rule. In this paper modification of MHO algorithm, MMHO algorithm is analyzed. MMHO algorithm performs PCA. New algorithm is obtained by implementation of time-oriented hierarchical method (TOHM) [14] on MHO method. TOHM method is general method that transforms PSA methods into PCA methods. It is based on learning on approximate principal Grassman/Stiefel submanifold [14]. Section 2 reviews basic theory of modulated Hebb (MH) and modulated Hebb-Oja (MHO) learning rules and suggests its computational circuit in the context of retinal processing. Section 3 is devoted to the introduction of new MMHO learning rule. Section 4 contains some simulation results. In Section 5 suggests speculative role of the proposed method for early visual processing in part of the frog retinal circuit. Section 6 gives some conclusion.

2 Modulated Hebb Learning Rule and Computational Circuit 2.1 Modulated Hebbain Rule: The Case N=K Let x ∈ ℜK denotes the input random variables with zero mean, and let y = WTx ∈ ℜN denotes the output, where W ∈ ℜKxN denotes the synaptic weight matrix. It is assumed N=K. In scalar form, for the kth output, we have yk =wkTx (k = 1, …, N), where wk ∈ ℜK denotes the column vector of W. Modulated Hebbian (MH) learning rule is introduced in [11] and analyzed in [13][15]. Here we will give a little bit different interpretation. MH rule can be derived as a gradient descent learning rule for minimization of the following cost function

(

J (W ) = E x − Wy

(

4

) = E⎛⎜⎜ ⎛⎜⎝ x − WW (

⎝

) (

T

2 2⎞ ⎞ x ⎟ ⎟ ⎠ ⎟⎠

= E⎛⎜ tr (C ) − 2tr W T CW + tr W T CWW TW ⎝ under assumption W TW = I ,

)) ⎞⎟⎠ 2

(1)

Modified Modulated Hebb-Oja Learning Rule

529

or in compact notation

⎧⎛ N N ⎞ ⎪ J NLoPCA (W ) = E ⎨⎜ ∑ xi2 − ∑ yi2 ⎟ ⎜ ⎟ ⎪⎩⎝ i =1 i =1 ⎠

2⎫

⎪ ⎬. ⎪⎭

(2)

It leads to the following gradient descent algorithm: N ⎛ N ⎞ W (i + 1) = W (i) + γ (i )⎜ xk2 (i ) − y k2 (i ) ⎟ x (i ) y T (i ). ⎜ ⎟ k =1 ⎝ k =1 ⎠

∑

∑

(3)

It is probably noticed that only an implicit constraint WTW=I is used - there is no explicit constraint. As it is shown in [13], that this algorithm, named Modulated Hebbian learning (MH), will lead toward a solution which insures WTW=I, although no additional terms, that should keep weight vectors orthonormal, are added. Also, it is interesting to notice that algorithm can be seen as a variant of the moving threshold algorithm, which is a form similar to the BCM learning rule [3], where the moving threshold is equal to the power contained in input signals. From equation (3) we can see that the original Hebb learning rule modulated with a single signal is applied. The network structure of interest is depicted on Fig. 1. The proposed learning method posses the locality and homogeneity. These are desirable properties for implementing artificial neural networks in parallel hardware [21]. Comparing to some other approaches (e.g. [19] and [26]) in this case we have solution in which update learning rule for individual synaptic efficacy does not require information about the explicit value of the other synaptic efficacies. Also, the number of circuits that perform global calculations (in this case summators) is two, regardless of the dimension of input and output vectors. 2.2 Modulated Hebb-Oja Rule: Case N < K

The MH learning rule is of low practical interest since the output has the same dimensionality as the input (for instance it can be used for statistical orthogonalization of a matrix). In the case when the number of output neurons is lower than the number of input neurons, or if N
(

)(

(

))

W (i + 1) = W (i ) + γ (i) x (i ) T x (i ) − y (i ) T y (i) x (i) y (i ) T − W (i )diag y (i ) y (i )T .

(4)

It is called Modulated Hebb Oja (MHO) learning rule. This learning rule generates a solution W, whose range is equal to the subspace spanned by N biggest eigenvectors of input signal covariance matrix C (C= E(tr(xxT)). Structure of the neural network of

530

M. Jankovic et al.

Input x x1

x2

xK

Σ

y1

y2

Output y

yN

Σ Fig. 1. MH(O) principal subspace network (small empty circle at the end of an arrow denotes the square function)

interest, can also be presented by Fig. 1. The only difference is that some additional calculations at the output are necessary. In [15] was shown that matrix W(i) has bounded elements Wkn (i).

3 Introduction of the New PCA Learning Rule Now it will be explained how we can transform MHO algorithm into PCA algorithm. First we will recall the definition of Grassman and Stiefel manifold that can be found in [7]. Grassman manifold is defined as the space of matrices W∈ O KxN ⊂ ℜKxN (N≤K) such that WTW=I and a homogeneous function J: O KxN→ ℜ such that J(W)=J(WQ) for any NxN orthogonal matrix Q. Stiefel manifold is defined as the space of matrices W∈ O KxN ⊂ ℜKxN (N≤K) and a function J: O KxN→ ℜ. Some learning algorithms can be seen as learning on Stiefel manifold if the weight matrix is kept automatically orthonormal at each update step. Such algorithms are known as strongly orthonormaly constrained (SOC) algorithms (e.g. [9]). Here we will use recently proposed time oriented hierarchical method (TOHM) (or more general GTOHM [14]) in order to transform PSA algorithm MHO into PCA method. The method is based on the following idea, that Each neuron tries to do what is the best for his family, and then what is the best for itself. We shall call this idea “the family principle”. In other words, the algorithm consists of two parts: the first part is responsible for the family-desirable feature learning and the second part is responsible for the individual-neuron-desirable feature learning. The second part is taken with a weight coefficient which is, by absolute

Modified Modulated Hebb-Oja Learning Rule

531

value, smaller than 1. This means that some time-oriented hierarchy in realization of the family and individual parts of the learning rules is made. In order to realize “the family principle”, we propose the following class of learning rules that can be used for parallel extraction of principal components, defined by the following equation ΔW PCA = ΔW PSA + D(i)IPSPCAorSMCA ,

(5)

where ΔWPCA represents modification of the weights, ΔWPSA defines family part of the learning rule (that is PSA), IPSPCAorSMCA represent individual part of the learning rule (single unit PCA or MCA algorithm) and D(i) is diagonal matrix which diagonal elements can be functions of time (in the case of homogeneous algorithm D(i, i) = α). In the most general case all individual parts can be implemented as different learning rules. It is interesting to note that individual part can pursue minor component while the whole algorithm pursue principal components. The intuition is as follows: we use two times scales; so if α is sufficiently small, then the term multiplied by α does not affect the PSA learning direction. When algorithm reaches the principal subspace, then part of the algorithm that is multiplied by α will perform learning on approximately Grasmman/Stiefel principal submanifold (definitions are given in [14]), and will rotate weight vectors toward principal components. By implementation of the proposed principle new learning rule can be written in the form

)(

(

w k (i + 1) = w k (i ) + γ (i) x (i ) y k (i ) − w k (i ) y k2 (i ) x (i ) − y (i ) ⎛ + αγ (i) x (i ) y k (i ) − w k (i ) y k2 (i ) ⎜ x (i ) ⎜ ⎝

(

)

2

2

−

k

⎞

j =1

⎠

∑ y 2j (i )⎟⎟,

2

)

k = 1,.., N .

(6)

We can see that we actually have a system of equations that have same family part of the learning rule, and all individual parts of the learning rules are different. By implementation of the same method that was proposed in [15], it can be shown that synaptic vectors have bounded norms under some reasonable assumptions. It can be proved that the stable points of the algorithm are principal eigenvectors of the input signal covariance matrix. However, the proof is lengthy and will be omitted here. The sketch of the proof is as follows: – difference equations (6) can be related to their differential counterparts by implementation of stochastic approximation [16], [20]; – it is possible to show that all equations actually represent eigenvector equations for the set of matrices that have eigenvectors equal to eigenvectors of the input signal covariance matrix C= E(tr(xxT); – if the weight matrix is full rank for all t, then columns of the synaptic matrix will converge toward some of the principal eigenvectors of the input covariance matrix; – then, using the method proposed in [15] it is possible to prove that those eigenvectors will correspond to principal eigenvectors of the input covariance matrix.

532

M. Jankovic et al.

4 Simulation Results Now we will examine the small scale numerical simulations results. The number of inputs was K = 5 and the number of output neurons was N = 3. Artificial zero-mean vectors with uncorrelated elements were generated by the following equations: x(1, i ) = .45 sin(i/2); x(2, i ) = .45 ((rem(i,23) - 11)/9).^5; x(3, i ) = .35sin(i/17.8); x(4, i ) = .145 ((rand(1,1) < .5) * 2 - 1). ⋅ log(rand(1,1) + .5); x(5, i ) = .18 randn(1,1) . Input signal is constructed as s = 0.47*mix*x, where mixing matrix mix is defined as mix = - .5 + rand( K ). In Fig.2 cosine of the angles between column vectors of matrix W and numerically calculated eigenvectors of input signal covariance matrix were used as illustration of the algorithm efficiency. Learning rate was chosen constant, γi = 3.45, for the first 15000 iterations, and then was set at γi = 0.115. Individual part was taken as α=0.5. Initial value for matrix W was selected as Winit=-.5+rand(5,3). In Fig. 3 we can see that proposed method can be efficiently used for extraction of the deterministic input signals (under the assumption that mixing matrix is orthogonal). First column in Fig. 3 represents original signals, second column represents input signals obtained from original signals by multiplication with orthonormal matrix mix ( mix = mix (mixTmix)-.5), and last column represents output, or reconstructed signals, obtained by implementation of MMHO. 1.4

1.2

cos(wi, pcai)

1

0.8

0.6

0.4

0.2

0

0

0.5

1 1.5 Number of iterations

2

2.5 x 10

4

Fig. 2. Cosine of angles between column vectors of W and three principal eigenvectors of input covariance matrix versus the number of iterations

Modified Modulated Hebb-Oja Learning Rule

Original signals

Input

0.5

Output

1

1

0

0

0

-0.5

-1

-1

0

50

100

150

200

250

2

0

50

100

150

200

250

0.5

0

0

0

-0.5

-0.5

50

100

150

200

250

0

50

100

150

200

250

0.5

0.5

0.2

0

0

0

-0.5

0

50

100

150

200

250

-0.5

0.1

0.5

0

0

-0.1

0

50

100

150

200

250

0.5

-0.5

0

50

100

150

200

250

0

50

100

150

200

250

0

50

100

150

200

250

0.5

-2

0

533

0

50

100

150

200

250

0

50

100

150

200

250

0

50

100

150

200

250

-0.2

0.5

0

0

-0.5

-0.5

0

50

100

150

200

250

Fig. 3. Blind signal extraction of deterministic components (mixing matrix is orthonormal)

5 Comparison to a Part of the Frog Retinal Wiring Equation (6) can be rewritten as

)(

(

2

w k (i + 1) = w k (i ) + γ (i ) x (i ) y k (i) − w k (i ) y k2 (i ) (1 + α ) x (i) − y (i )

2

)

⎛ k −1 ⎞ − αγ (i) x (i ) y k (i) − w k (i ) y k2 (i ) ⎜ y 2j (i) + y k2 (i ) ⎟, k = 1,.., N . ⎜ j =1 ⎟ ⎝ ⎠

(

)∑

(7)

Proposed learning rule can be implemented by the network structure shown in Fig. 4. By analyzing the proposed structure, we can see that some degree of similarity with the part of the frog’s retinal wiring exists [6]. We can make the following analogies: • the inputs x1,…,xK can be viewed as photoreceptors • the outputs y1,…,yN correspond to bipolar cells (BC) • circuit for calculation of the input energy marked as Σ can be viewed as horizontal cell • units labeled as Ak and which calculate the output partial powers ⎛ k −1 ⎞ Ak (i ) = ⎜ y 2j (i ) + y k2 (i ) ⎟ ⎜ j =1 ⎟ ⎝ ⎠

∑

could be related to the amacrine cells (A