Lecture Notes in Electrical Engineering Volume 67
Zhigang Zeng · Jun Wang (Eds.)
Advances in Neural Network Research and Applications
ABC
Prof. Zhigang Zeng Department of Control Science and Engineering Huazhong University of Science and Technology Wuhan, Hubei, 430074 China E-mail:
[email protected] Prof. Jun Wang Department of Mechanical & Automation Engineering The Chinese University of Hong Kong Shatin, New Territories Hong Kong Tel.: +852 26098472 Fax: +852-26036002 E-mail:
[email protected]
ISBN 978-3-642-12989-6
e-ISBN 978-3-642-12990-2
DOI 10.1007/978-3-642-12990-2 Library of Congress Control Number: 2010926919 c 2010 Springer-Verlag Berlin Heidelberg This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typeset & Coverdesign: Scientific Publishing Services Pvt. Ltd., Chennai, India. Printed on acid-free paper 987654321 springer.com
Preface
This book is a part of the Proceedings of the Seventh International Symposium on Neural Networks (ISNN 2010), held on June 6-9, 2010 in Shanghai, China. Over the past few years, ISNN has matured into a well-established premier international symposium on neural networks and related fields, with a successful sequence of ISNN series in Dalian (2004), Chongqing (2005), Chengdu (2006), Nanjing (2007), Beijing (2008), and Wuhan (2009). Following the tradition of ISNN series, ISNN 2010 provided a high-level international forum for scientists, engineers, and educators to present the state-of-the-art research in neural networks and related fields, and also discuss the major opportunities and challenges of future neural network research. Over the past decades, the neural network community has witnessed significant breakthroughs and developments from all aspects of neural network research, including theoretical foundations, architectures, and network organizations, modeling and simulation, empirical studies, as well as a wide range of applications across different domains. The recent developments of science and technology, including neuroscience, computer science, cognitive science, nano-technologies and engineering design, among others, has provided significant new understandings and technological solutions to move the neural network research toward the development of complex, large scale, and networked brain-like intelligent systems. This long-term goals can only be achieved with the continuous efforts from the community to seriously investigate various issues on neural networks and related topics. Toward this end, the ISNN 2010 provided a great platform for the community to share their latest research results, discuss critical future research directions, stimulate innovative research ideas, as well as facilitate worldwide multi-disciplinary collaborations. ISNN 2010 received numerous submissions from about thousands of authors in about 40 countries and regions (Australia, Belgium, Brazil, Canada, China, Colombia, Czech Republic, Finland, France, Germany, Hong Kong, India, Iran, Italy, Japan, Lebanon, Lithuania, Macao, Malaysia, Mexico, Netherlands, New Zealand, Norway, Peru, Poland, Portugal, Singapore, Slovenia, South Africa, South Korea, Spain, Sri Lanka, Taiwan, Thailand, Tunisia, Uganda, United Arab Emirates, United Kingdom, and USA) across six continents (Asia, Europe, North America, South America, Africa, and Oceania). Based on the rigorous peer-reviews by the program committee members and the reviewers, 108 high-quality papers were selected for publications in Lecture Notes in Electrical Engineering (LNEE) Proceedings. These papers cover all major topics of the engineering designs and applications of neural network research. In addition to the contributed papers, the ISNN 2010 technical program included four plenary speeches by Andrzej Cichocki (RIKEN Brain Science Institute, Japan), ChinTeng Lin (National Chiao Tung University, Taiwan), DeLiang Wang (Ohio State University, USA), Gary G. Yen (Oklahoma State University, USA). We would like to take this opportunity to express our sincere thanks to the Shanghai Jiao Tong University, The Chinese University of Hong Kong, Microsoft Research Asia, and the National Natural Science Foundation of China for their
VI
Preface
sponsorship, to the IEEE Shanghai Section, IEEE Computational Intelligence Society, International Neural Network Society, Asia Pacific Neural Network Assembly, European Neural Network Society, and International Association for Mathematics and Computers in Simulation for their technical co-sponsorship. We would also like to sincerely thank the chairs for their contributions to the symposium, members of the Advisory Committee and Steering Committee for their guidance, and the members of the Organizing Committee, Special Sessions Committee, Publication Committee, Publicity Committee, Finance Committee, Registration Committee, and Local Arrangements Committee for all their great efforts and time in organizing such an event. We would like especially to express our deepest gratitude to the members of the International Program Committee and all reviewers for their professional review of the papers; their expertise guaranteed the high quality of the technical program of the ISNN 2010. Furthermore, we would also like to thank Springer for publishing the proceedings in the prestigious series of Lecture Notes in Electrical Engineering. Meanwhile, we also would like to express our heartfelt appreciations to the plenary and panel speakers for their vision and discussions on the latest research development in the field as well as critical future research directions, opportunities, and challenges. Finally, we would also like to thank all the speakers, authors, and participants for their great contribution and support that made ISNN 2010 a great success.
Zhigang Zeng Jun Wang
ISNN 2010 Organization
Sponsor and Organizer Shanghai Jiao Tong University
Co-organizers The Chinese University of Hong Kong Microsoft Research Asia
Technically Co-sponsors IEEE Shanghai Section IEEE Computational Intelligence Society International Neural Network Society European Neural Network Society Asia Pacific Neural Network Assembly International Association for Mathematics and Computers in Simulation
Financial Co-sponsor National Natural Science Foundation of China
General Chairs Jun Wang Bao-Liang Lu
Hong Kong, China Shanghai, China
Organizing Committee Chair Jianbo Su
Shanghai, China
Program Committee Chairs Liqing Zhang Zhigang Zeng James T.Y. Kwok
Shanghai, China Wuhan, China Hong Kong, China
VIII
Organization
Special Sessions Chairs Si Wu Qing Ma Paul S. Pang
Shanghai, China Kyoto, Japan Auckland, New Zealand
Publications Chairs Hongtao Lu Yinling Wang Wenlian Lu
Shanghai, China Shanghai, China Shanghai, China
Publicity Chairs Bo Yuan Xiaolin Hu Qingshan Liu
Shanghai, China Beijing, China Nanjing, China
Finance Chairs Xinping Guan Xiangyang Zhu
Shanghai, China Shanghai, China
Registration Chairs Fang Li Gui-Rong Xue Daniel W.C. Ho
Shanghai, China Shanghai, China Hong Kong, China
Local Arrangements Chairs Qingsheng Ren Xiaodong Gu
Shanghai, China Shanghai, China
Advisory Committee Chairs Xiaowei Tang Bo Zhang Aike Guo
Hangzhou, China Beijing, China Shanghai, China
Organization
IX
Advisory Committee Members Cesare Alippi, Milan, Italy Shun-ichi Amari, Tokyo, Japan Zheng Bao, Xi'an, China Dimitri P. Bertsekas, Cambridge, MA, USA Tianyou Chai, Shenyang, China Guanrong Chen, Hong Kong Andrzej Cichocki, Tokyo, Japan Ruwei Dai, Beijing, China Jay Farrell, Riverside, CA, USA Chunbo Feng, Nanjing, China Russell Eberhart, Indianapolis, IN, USA David Fogel, San Diego, CA, USA Walter J. Freeman, Berkeley, CA, USA Kunihiko Fukushima, Osaka, Japan Xingui He, Beijing, China Zhenya He, Nanjing, China Janusz Kacprzyk, Warsaw, Poland Nikola Kasabov, Auckland, New Zealand Okyay Kaynak, Istanbul, Turkey Anthony Kuh, Honolulu, HI, USA Frank L. Lewis, Fort Worth, TX, USA Deyi Li, Beijing, China Yanda Li, Beijing, China
Chin-Teng Lin, Hsinchu, Taiwan Robert J. Marks II, Waco, TX, USA Erkki Oja, Helsinki, Finland Nikhil R. Pal, Calcutta, India Marios M. Polycarpou, Nicosia, Cyprus José C. Príncipe, Gainesville, FL, USA Leszek Rutkowski, Czestochowa, Poland Jennie Si, Tempe, AZ, USA Youxian Sun, Hangzhou, China DeLiang Wang, Columbus, OH, USA Fei-Yue Wang, Beijing, China Shoujue Wang, Beijing, China Paul J. Werbos, Washington, DC, USA Cheng Wu, Beijing, China Donald C. Wunsch II, Rolla, MO, USA Youlun Xiong, Wuhan, China Lei Xu, Hong Kong Shuzi Yang, Wuhan, China Xin Yao, Birmingham, UK Gary G. Yen, Stillwater, OK, USA Nanning Zheng, Xi'an, China Yongchuan Zhang, Wuhan, China Jacek M. Zurada, Louisville, KY, USA
Program Committee Members Haydar Akca Alma Y. Alanis Bruno Apolloni Sabri Arik Vijayan Asari Tao Ban Peter Baranyi Salim Bouzerdoum Martin Brown Xindi Cai Jianting Cao Yu Cao Jonathan Chan
Chu-Song Chen Liang Chen Sheng Chen Songcan Chen YangQuan Chen Yen-Wei Chen Zengqiang Chen Jianlin Cheng Li Cheng Long Cheng Zheru Chi Sung-Bae Cho Emilio Corchado
X
Organization
Jose Alfredo F. Costa Ruxandra Liana Costea Sergio Cruces Baotong Cui Chuanyin Dang Mingcong Deng Ming Dong Jixiang Du Andries Engelbrecht Meng Joo Er Jufu Feng Chaojin Fu Wai-Keung Fung John Gan Junbin Gao Xiao-Zhi Gao Xinping Guan Chen Guo Chengan Guo Ping Guo Abdenour Hadid Honggui Han Qing-Long Han Haibo He Hanlin He Zhaoshui He Akira Hirose Daniel Ho Noriyasu Homma Zhongsheng Hou Chun-Fei Hsu Huosheng Hu Jinglu Hu Junhao Hu Sanqing Hu Guang-Bin Huang Tingwen Huang Wei Hui Amir Hussain Jayadeva Minghui Jiang Tianzi Jiang Yaochu Jin Joarder Kamruzzaman Shunshoku Kanae Qi Kang Nik Kasabov
Okyay Kaynak Rhee Man Kil Kwang-Baek Kim Sungshin Kim Mario Koeppen Rakhesh Singh Kshetrimayum Edmund Lai Heung Fai Lam Minho Lee Chi-Sing Leung Henry Leung Chuandong Li Fang Li Guang Li Kang Li Li Li Shaoyuan Li Shutao Li Xiaoli Li Xiaoou Li Xuelong Li Yangmin Li Yuanqing Li Yun Li Zhong Li Jinling Liang Ming Liang Pei-Ji Liang Yanchun Liang Li-Zhi Liao Wudai Liao Longnian Lin Guoping Liu Ju Liu Meiqin Liu Yan Liu Hongtao Lu Jianquan Lu Jinhu Lu Wenlian Lu Jian Cheng Lv Jinwen Ma Malik Magdon Ismail Danilo Mandic Tiemin Mei Dan Meng Yan Meng
Organization
Duoqian Miao Martin Middendorf Valeri Mladenov Marco Antonio Moreno-Armendáriz Ikuko Nishkawa Stanislaw Osowski Seiichi Ozawa Shaoning Pang Jaakko Peltonen Vir V. Phoha Branimir Reljin Qingsheng Ren Tomasz Rutkowski Sattar B. Sadkhan Toshimichi Saito Gerald Schaefer Furao Shen Daming Shi Hideaki Shimazaki Michael Small Qiankun Song Jochen J. Steil John Sum Roberto Tagliaferri Norikazu Takahashi Ah-hwee Tan Ying Tan Toshihisa Tanaka Dacheng Tao Ruck Thawonmas Xin Tian Christos Tjortjis Ivor Tsang Masao Utiyama Marc Vanhulle Bin Wang Dan Wang Dianhui Wang Lei Wang Liang Wang Rubin Wang
Wenjia Wang Wenwu Wang Xiaoping Wang Xin Wang Yinglin Wang Yiwen Wang Zhanzhan Wang Zhongsheng Wang Zidong Wang Hau-San Wong Kevin Wong Wei Wu Cheng Xiang Hong Xie Songyun Xie Rui Xu Xin Xu Guirong Xue Yang Yang Yingjie Yang Yongqing Yang Jianqiang Yi Dingli Yu Jian Yu Xiao-Hua Yu Bo Yuan Kun Yuan Pong C Yuen Xiaoqin Zeng Changshui Zhang Jie Zhang Junping Zhang Kai Zhang Lei Zhang Nian Zhang Dongbin Zhao Hai Zhao Liang Zhao Qibin Zhao Mingjun Zhong Weihang Zhu
Reviewers Ajith Abraham Alma Y. Alanis NG Alex
Jing An Sung Jun An Claudia Angelini
Nancy Arana-Daniel Nancy Arana-Daniel Kiran Balagani
XI
XII
Organization
Tao Ban Simone Bassis Anna Belardinelli Joao Roberto Bertini Junior Amit Bhaya Shuhui Bi Xuhui Bo Salim Bouzerdoum N. Bu Qiao Cai Xindi Cai Hongfei Cao Yuan Cao Jonathan Chan Wenge Chang Benhui Chen Bo-Chiuan Chen Chao-Jung Chen Chu-Song Chen Cunbao Chen Fei Chen Gang Chen Guici Chen Junfei Chen Lei Chen Min Chen Pin-Cheng Chen Sheng Chen Shuwei Chen Tao Chen Xiaofen Chen Xiaofeng Chen Yanhua Chen Yao Chen Zengqiang Chen Zhihao Chen Jianlin Cheng K. H. Cheng Lei Cheng Yu Cheng Yuhu Cheng Seong-Pyo Cheon Zheru Chi Seungjin Choi Angelo Ciaramella Matthew Conforth
Paul Christopher Conilione Paleologu Constantin Jose Alfredo F. Costa Ruxandra Liana Costea Fangshu Cui Zhihua Cui James Curry Qun Dai Xinyu Dai Spiros Denaxas Jing Deng Xin Deng Zhijian Diao Ke Ding Jan Dolinsky Yongsheng Dong Adriao Duarte Doria Neto Dajun Du Jun Du Shengzhi Du Wei Du Qiguo Duan Zhansheng Duan Julian Eggert Yong Fan Chonglun Fang Italia De Feis G C Feng Qinrong Feng Simone Fiori Chaojin Fu Jun Fu Zhengyong Fu Zhernyong Fu Sheng Gan Shenghua Gao Fei Ge Vanessa Goh Dawei Gong Weifeng Gu Wenfei Gu Renchu Guan Chengan Guo Jianmei Guo Jun Guo Ping Guo
Xin Guo Yi Guo Juan Carlos Gutierrez Caceres Osamu Hasegawa Aurelien Hazart Hanlin He Huiguang He Lianghua He Lin He Wangli He Xiangnan He Zhaoshui He Sc Ramon Hernandez Esteban Hernandez-Vargas Kevin Ho Xia Hong Chenping Hou Hui-Huang Hsu Enliang Hu Jinglu Hu Junhao Hu Meng Hu Sanqing Hu Tianjiang Hu Xiaolin Hu Zhaohui Hu Bonan Huang Chun-Rong Huang Dan Huang J. Huang Kaizhu Huang Shujian Huang Xiaodi Huang Xiaolin Huang Zhenkun Huang Cong Hui GuoTao Hui Khan M. Iftekharuddin Tasadduq Imam Teijiro Isokawa Mingjun Ji Zheng Ji Aimin Jiang Changan Jiang Feng Jiang
Organization
Lihua Jiang Xinwei Jiang Gang Jin Ning Jin Yaochu Jin Krzysztof Siwek Yiannis Kanellopoulos Enam Karim Jia Ke Salman Khan Sung Shin Kim Tae-Hyung Kim Mitsunaga Kinjo Arto Klami Mario Koeppen Adam Kong Hui Kong Qi Kong Adam Krzyzak Jayanta Kumar Debnath Kandarpa Kumar Sarma Franz Kurfess Paul Kwan Darong Lai Jiajun Lai Jianhuang Lai Wei Lai Heung Fai Lam Paul Lam Yuan Lan Ngai-Fong Law N. K. Lee Chi SingLeung Bing Li Boyang Li C. Li Chaojie Li Chuandong Li Dazi Li Guang Li Junhua Li Kang Li Kelin Li Li Li Liping Li Lulu Li Manli Li
Peng Li Ping Li Ruijiang Li Tianrui Li Tieshan Li Xiaochen Li Xiaocheng Li Xuelong Li Yan Li Yun Li Yunxia Li Zhenguo Li Allan Liang Jinling Liang Pei-Ji Liang Li-Zhi Liao Wudai Liao Hongfei Lin Qing Lin Tran Hoai Lin Bo Liu Chang Liu Chao Liu Fei Liu Hongbo Liu Jindong Liu Lei Liu Lingqiao Liu Nianjun Liu Qingshan Liu Wei Liu Xiangyang Liu Xiwei Liu Yan Liu Yanjun Liu Yu Liu Zhaobing Liu Zhenwei Liu Jinyi Long Jinyi Long Carlos Lopez-Franco Shengqiang Lou Mingyu Lu Ning Lu S F Lu Bei Lv Jun Lv
XIII
Fali Ma Libo Ma Singo Mabu Danilo Mandic Qi Mao Tomasz Markiewicz Radoslaw Mazur Tiemin Mei Bo Meng Zhaohui Meng Marna van der Merwe Martin Middendorf N. Mitianoudis Valeri Mladenov Alex Moopenn Marco Moreno Loredana Murino Francesco Napolitano Ikuko Nishkawa Tohru Nitta Qiu Niu Qun Niu Chakarida Nukoolkit Sang-Hoon Oh Floriberto Ortiz Stanislaw Osowski Antonio de Padua Braga Antonio Paiva Shaoning Pang Woon Jeung Park Juuso Parkkinen Michael Paul Anne Magály de Paula Canuto Zheng Pei Jaakko Peltonen Ce Peng Hanchuan Peng Jau-Woei Perng Son Lam Phung Xiong Ping Kriengkrai Porkaew Santitham Prom-on Dianwei Qian Lishan Qiao Keyun Qin Meikang Qiu
XIV
Organization
Li Qu Marcos G. Quiles Mihai Rebican Luis J. Ricalde Jorge Rivera Haijun Rong Zhihai Rong Tomasz Rutkowski Jose A. Ruz Edgar N. Sanchez Sergio P. Santos Renato José Sassi Chunwei Seah Nariman Sepehri Caifeng Shan Shiguang Shan Chunhua Shen Furao Shen Jun Shen Yi Shen Jiuh-Biing Sheu Licheng Shi Qinfeng Shi Xiaohu Shi Si Si Leandro Augusto da Silva Angela Slavova Sunantha Sodsee Dandan Song Dongjin Song Doo Heon Song Mingli Song Qiang Song Qiankun Song Kingkarn Sookhanaphibarn Gustavo Fontoura de Souza Antonino Staiano Jochen Steil Pui-Fai Sum Jian Sun Jian-Tao Sun Junfeng Sun Liang Sun Liming Sun Ning Sun
Yi Sun Shigeru Takano Mingkui Tan Ke Tang Kecheng Tang Y Tang Liang Tao Yin Tao Sarwar Tapan Ruck Thawonmas Tuan Hue Thi Le Tian Fok Hing Chi Tivive Christos Tjortjis Rutkowski Tomasz Julio Tovar Jianjun Tu Zhengwen Tu Goergi Tzenov Lorenzo Valerio Rodrigo Verschae Liang Wan Min Wan Aihui Wang Bin Wang Bo Hyun Wang Chao Wang Chengyou Wang Dianhui Wang Guanjun Wang Haixian Wang Hongyan Wang Huidong Wang Huiwei Wang Jingguo Wang Jinghua Wang Lan Wang Li Wang Lili Wang Lizhi Wang Min Wang Ming Wang Pei Wang Ruizhi Wang Xiaolin Wang Xiaowei Wang Xin Wang
Xu Wang Yang Wang Ying Wang You Wang Yunyun Wang Zhanshan Wang Zhengxia Wang Zhenxing Wang Zhongsheng Wang Bunthit Watanapa Hua-Liang Wei Qinglai Wei Shengjun Wen Young-Woon Woo Ailong Wu Chunguo Wu Jun Wu Qiang Wu Si Wu Xiangjun Wu Yili Xia Zeyang Xia Cheng Xiang Linying Xiang Shiming Xiang Xiaoliang Xie Ping Xiong Zhihua Xiong Fang Xu Feifei Xu Heming Xu Jie Xu LinLi Xu Rui Xu Weihong Xu Xianyun Xu Xin Xu Hui Xue Jing Yang Liu Yang Qingshan Yang Rongni Yang Shangming Yang Wen-Jie Yang Wenlu Yang Wenyun Yang Xubing Yang
Organization
Yan Yang Yongqing Yang Zi-Jiang Yang John Yao Jun Yao Yingtao Yao Keiji Yasuda Ming-Feng Yeh Xiao Yi Chenkun Yin Kaori Yoshida WenwuYu Xiao-Hua Yu Kun Yuan Weisu Yuan Xiaofang Yuan Zhuzhi Yuan Zhuzhu Yuan P C Yuen Masahiro Yukawa Lianyin Zhai Biao Zhang Changshui Zhang Chen Zhang
Secretariat Jin Gang Kan Hong Qi Kong Qiang Wang Qiang Wu Rong Zhou Tianqi Zhang
Dapeng Zhang Jason Zhang Jian Zhang Jianbao Zhang Jianhai Zhang Jianhua Zhang Jin Zhang Junqi Zhang Junying Zhang Kai Zhang Leihong Zhang Liming Zhang Nengsheng Zhang Nian Zhang Pu-Ming Zhang Qing Zhang Shaohong Zhang Tao Zhang Teng-Fei Zhang Ting Zhang Xian-Ming Zhang Yuyang Zhang Hai Zhao Qibin Zhao
Xiaoyu Zhao Yi Zhao Yongping Zhao Yongqing Zhao Ziyang Zhen Chengde Zheng Lihong Zheng Yuhua Zheng Caiming Zhong Mingjun Zhong Shuiming Zhong Bo Zhou Jun Zhou Luping Zhou Rong Zhou Xiuling Zhou Haojin Zhu Song Zhu Wenjun Zhu Xunlin Zhu Yuanming Zhu Wei-Wen Zou Xin Zou Pavel Zuñiga
XV
Table of Contents
Prediction and Forecasting A Novel Prediction Mechanism with Modified Data Mining Technique for Call Admission Control in Wireless Cellular Network . . . . . . . . . . . . . . Chen-Feng Wu, Yu-Teng Chang, Chih-Yao Lo, and Han-Sheng Zhuang
1
The Study of Forecasting Model of Rock Burst for Acoustic Emission Based on BP Neural Network and Catastrophe Theory . . . . . . . . . . . . . . . Yunyun Xu and Dongqiang Xu
11
Application of Wavelet Neural Network to Prediction of Water Content in Crude Oil . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Huifen Niu, Cuiling Liu, Jinqi Wang, and Xiaowen Sun
21
Prediction of Urban Heat Island Intensity in Chuxiong City with Backpropagation Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Wujun Xi and Ping He
29
Research on the Fouling Prediction of Heat Exchanger Based on Wavelet Relevance Vector Machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lingfang Sun, Rina Saqi, and Honggang Xie
37
Electricity Price Forecasting Using Neural Networks Trained with Entropic Criterion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jianhua Zhang, Jingyue Wang, Rui Wang, and Guolian Hou
47
Least Square Support Vector Machine Ensemble for Daily Rainfall Forecasting Based on Linear and Nonlinear Regression . . . . . . . . . . . . . . . . Jiansheng Wu, Mingzhe Liu, and Long Jin
55
Estimating Portfolio Risk Using GARCH-EVT-Copula Model: An Empirical Study on Exchange Rate Market . . . . . . . . . . . . . . . . . . . . . . . . . . Zongrun Wang, Yanbo Jin, and Yanju Zhou
65
Forecasting Financial Time Series via an Efficient CMAC Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chi-Jie Lu and Jui-Yu Wu
73
Forecasting Daily Cash Turnover of Bank with EWMA and SVR . . . . . . Wei-min Ma and Wei Lu Financial Distress Prediction Model via GreyART Network and Grey Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ming-Feng Yeh, Chia-Ting Chang, and Min-Shyang Leu
83
91
XVIII
Table of Contents
Risk Assessment Model Based on Immune Theory . . . . . . . . . . . . . . . . . . . . Tao Liu, Li Shang, and Zhifeng Hu
101
Short-Term Load Forecasting Based on Bayes and RS . . . . . . . . . . . . . . . . Yanmei Li
109
Fuzzy Neural Networks Nonlinear System Modeling with a New Fuzzy Model and Neural Compensation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cruz Vega Israel and Wen Yu
117
A Research of Fuzzy Neural Network in Ferromagnetic Target Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Caipeng Wu, Jiahao Deng, and Yanli Yang
129
Multiple T-S Fuzzy Neural Networks Soft Sensing Modeling of Flotation Process Based on Fuzzy C-Means Clustering Algorithm . . . . . . Jiesheng Wang, Yong Zhang, and Shifeng Sun
137
A Using Reliability Evaluation Model for Diesel Engine Based on Fuzzy Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ying-Kui Gu and Kai-Qi Huang
145
Fuzzy Sliding Mode Control with Perturbation Estimation for a Piezoactuated Micromanipulator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Qingsong Xu and Yangmin Li
153
A Credit Risk Rating Model Based on Fuzzy Neural Network . . . . . . . . . . Ke-Jun Zhu, Pin-Chang Chen, and Yu-Teng Chang
161
Interval-Valued Fuzzy Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Wenyi Zeng and Jiayin Wang
173
Research on Fuzzy Preference Relations-Based MAS for Decision Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Weijin Jiang and Qing Jiang
185
Optimization and Planning Study of Stochastic Demand Inventory Routing Problem with Soft Time Windows Based on MDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Wei Zeng and Qian Zhao
193
An Agent-Based Approach to Joint Procurement Modeling with Virtual Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chih-Yao Lo, Hung-Teng Chang, Yu-Teng Chang, and Hsiu-Yun Hu
201
Table of Contents
Interactive Hybrid Evolutionary Computation for MEMS Design Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ying Zhang and Alice M. Agogino
XIX
211
Genetic Algorithms for Traffic Grooming in Optical Tree Networks . . . . . Shu-tong Xie and Li-fang Pan
219
Multi-sensor Multi-target Tracking with OOSM . . . . . . . . . . . . . . . . . . . . . . Cheng Cheng and Jinfeng Wang
227
Hopfield Neural Network Guided Evolutionary Algorithm for Aircraft Penetration Path Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Nan Wang, Lin Wang, Xueqiang Gu, Jing Chen, and Lincheng Shen Fuzzy Material Procurement Planning with Value-at-Risk . . . . . . . . . . . . . Gao-Ji Sun Radial Basis Function Network for Endpoint Detection in Plasma Etch Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Shu-Kun Zhao, Min-Woo Kim, Yi-Seul Han, Se-Youn Jeon, Yun-Keun Lee, and Seung-Soo Han A Novel Cellular Neural Network and Its Applications in Motion Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yan Cao, Feng Zhang, Xuewu Wu, Sheng Lu, Yi Li, Lei Sun, and Shuai Li Evaluation of Enterprise ERP System Based on Neural Network Optimized by Ant Colony . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hanmei Wang, Dongxiao Niu, and Chengkai Cai
235 245
253
265
275
Pattern Recognition Visual Attention-Based Ship Detection in SAR Images . . . . . . . . . . . . . . . . Ying Yu, Zhenghu Ding, Bin Wang, and Liming Zhang
283
Recognizing Multi-ships Based on Silhouette in Infrared Image . . . . . . . . Jun-Wei Lv, Bo Wang, and Dong-Mei Wang
293
SAR Images Feature Extraction and Recognition Based on G2DCDA . . . Liping Hu, Hongwei Liu, and Ping Zhou
301
Bridge Detection and Recognition in Remote Sensing SAR Images Using Pulse Coupled Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Zhenming Peng, Shijun Liu, Guiyou Tian, Zhang Chen, and Tao Tao
311
Approaches to Robotic Vision Control Using Image Pointing Recognition Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tian-Ding Chen
321
XX
Table of Contents
A Hybrid Evolutionary Approach to Band Selection for Hyperspectral Image Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hao Wu, Jiali Zhu, Shijin Li, Dingsheng Wan, and Lin Lin Impulsive Environment Sound Detection by Neural Classification of Spectrogram and Mel-Frequency Coefficient Images . . . . . . . . . . . . . . . . . . Peerapol Khunarsa, Chidchanok Lursinsap, and Thanapant Raicharoen Classification and Control of Cognitive Radios Using Hierarchical Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sheng Chen, Xiaochen Li, Qiao Cai, Nansai Hu, Haibo He, Yu-Dong Yao, and Joseph Mitola Identifying Spatial Patterns of Land Use and Cover Change at Different Scales Based on Self-Organizing Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hao Wu, Xiaoling Chen, Zhan Li, Sheng Wang, Wei Cui, and Qian Meng Colon Tumor Microarray Classification Using Neural Network with Feature Selection and Rule-Based Classification . . . . . . . . . . . . . . . . . . . . . . Narissara Eiamkanitchat, Nipon Theera-Umpon, and Sansanee Auephanwiriyakul
329
337
347
355
363
Signal and Image Processing Dual Channel Speech Denoising Based on Sparse Representation . . . . . . . Gongxian Sun, Feng Gao, Jun Lv, and Ming Xiao Frequency-Domain Blind Separation of Convolutive Speech Mixtures with Energy Correlation-Based Permutation Correction . . . . . . . . . . . . . . . Li-Dan Wang and Qiu-Hua Lin A Blind Broadband Beamforming Method for Speech Enhancement . . . . Dongxia Wang, Fuliang Yin, and Fuming Sun
373
381
391
Algorithm and Simulation Research for Blind Nonlinear System Identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hongqiu Teng and Weizheng Ruan
397
Study on Digital Image Correlation Using Artificial Neural Networks for Subpixel Displacement Measurement . . . . . . . . . . . . . . . . . . . . . . . . . . . . Xiao-yong Liu, Qing-chang Tan, and Rong-li Li
405
Tree Modeling through Range Image Segmentation and 3D Shape Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mingrui Dai, Hongjun Li, and Xiaopeng Zhang
413
Table of Contents
XXI
Combining Bag of Words Model and Information Theoretic Method for Image Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Xue Bai and Siwei Luo
423
Fast Fuzzy c-Means Clustering Algorithm with Spatial Constraints for Image Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yanling Li and Gang Li
431
Application of Support Vector Machines to Airborne Hyper-Spectral Image Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ming-Der Yang, Kai-Siang Huang, Ji-Yuan Lin, and Pei Liu
439
Research for the Identification Method of the Image Definition Based on a W-N Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Miaofen Zhu, Guojin Chen, Yongning Li, and Wanqiang Wang
445
Analysis of Texture Images Generated by Olfactory System Bionic Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chen Fang, Jin Zhang, Shangwu Zhu, Guang Li, and Rulong Wang
453
Do Neural Networks Have True Power for Natural Language Processing? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Qing Ma
461
Robust Channel Identification Using FOCUSS Method . . . . . . . . . . . . . . . Zhaoshui He and Andrzej Cichocki
471
Human Head Modeling Using NURBS Method . . . . . . . . . . . . . . . . . . . . . . Songyun Xie, Ningfei Li, and Zhuo Lv
479
Risk Sensitive Unscented Particle Filter for Bearing and Frequency Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Peng Li, Shenmin Song, and Xinglin Chen
485
Fully Complex Multiplicative Neural Network Model and Its Application to Channel Equalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kavita Burse, R.N. Yadav, and Sushil Chandra Shrivastava
493
Robotics and Control Visual Navigation of a Novel Economical Embedded Multi-mode Intelligent Control System for Powered Wheelchair . . . . . . . . . . . . . . . . . . . Jizhong Liu, Xuepei Wu, Jiating Xia, Guanghui Wang, and Hua Zhang Neural Networks L2 -Gain Control for Robot System . . . . . . . . . . . . . . . . . . Zhi-gang Yu, Yong-liang Shen, Shen-min Song, and Da-wei Zhang
503
513
XXII
Table of Contents
Neural Network Control of Spacecraft Formation Using RISE Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Shicheng Wang, Haibo Min, Fuchun Sun, and Jinsheng Zhang A Simplified Modular Petri Net for the Walking Assistant Robot . . . . . . . Zhen Zhang, Qixin Cao, Chuntao Leng, and Peihua Chen
521
529
Omni-directional Vision Based Tracking and Guiding System for Walking Assistant Robot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Xiaoxiao Zhu, Qixin Cao, Hongbing Tan, and Aolin Tang
537
Dynamic Eye-in-Hand Visual Servoing with Unknown Target Positions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hesheng Wang, Weidong Chen, and Yun-hui Liu
545
Optimum Motion Control for Stacking Robot . . . . . . . . . . . . . . . . . . . . . . . . Xiaoming Zhang, Nan Luan, Zhong Dong, and Liming Chen Passive Target Tracking Using an Improved Particle Filter Algorithm Based on Genetic Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yue Liang, Zhong Liu, and Guodong Zhang Large-Scale Structure Assembly by Multiple Robots Which May Be Broken . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Masayuki Otani, Kiyohiko Hattori, Hiroyuki Sato, and Keiki Takadama Real-Time Five DOF Redundant Robot Control Using a Decentralized Neural Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ramon Garcia-Hernandez, Edgar N. Sanchez, Maarouf Saad, and Eduardo Bayro-Corrochano
551
559
567
577
Improving Transient Response of Adaptive Control Systems Using Multiple Neural Network Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Xiao-li Li, Xiang Yu, and Yan Zhang
587
An Information Theoretic Approach for Design MIMO Networked Control Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jianhua Zhang and Hong Wang
595
An Engineering Solution for Decoupling Control of Aircraft Motion Using Affine Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tsurng-Jehng Shen
603
Black-Box Input-Output Identification of a Class of Nonlinear Systems Using a Discrete-Time Recurrent Neurofuzzy Network . . . . . . . . . . . . . . . . Marcos A. Gonz´ alez-Olvera and Yu Tang
615
Table of Contents
XXIII
Passivity Analysis of Stochastic Neural Networks with Mixed Time-Varying Delays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Qinggao He and Qiankun Song
623
A Novel Recovering Initial Conditions Method from Spatiotemporal Complex Dynamical System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Longquan Dai, Xiaoyun Kang, and Minfen Shen
631
An Intelligent Control Scheme for Nonlinear Time-Varying Systems with Time Delay . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sun Zhou, Guoli Ji, Wei Lin, and Zijiang Yang
639
Master-Slave Chaos Synchronization of Uncertain Nonlinear Gyros Using Wavelet Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chien-Jung Chiu, Chun-Fei Hsu, Tsu-Tian Lee, and Jang-Zern Tsai
647
Transportation Systems WNN-Based Intelligent Transportation Control System . . . . . . . . . . . . . . . Chiu-Hsiung Chen, Ya-Fu Peng, and Tsai-Sheng Kao
655
Incident Detection in Urban Road . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Wu Bing-Fei, Kao Chih-Chung, Chen Chao-Jung, Li Yen-Feng, Chen Ying-Han, and Yang Cheng-Yen
663
An Efficient Web-Based Tracking System through Reduction of Redundant Connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Bing-Fei Wu, Ying-Han Chen, Chao-Jung Chen, Chih-Chung Kao, and Po-Chia Huang An Embedded All-Time Blind Spot Warning System . . . . . . . . . . . . . . . . . Bing-Fei Wu, Chao-Jung Chen, Yen-Feng Li, Cheng-Yen Yang, Hai-Chang Chien, and Chia-Wei Chang Design of Autonomous Parallel Parking Using Fuzzy Logic Controller with Feed-Forward Compensation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Bo-Chiuan Chen, Yi-Wen Huang, Shiuh-Jer Huang, and Bo-Jhao Liu Telematics Services through Mobile Agents . . . . . . . . . . . . . . . . . . . . . . . . . . Jonathan Lee, Shin-Jie Lee, Hsi-Min Chen, and Wen-Tin Lee Multi-agent System Model for Urban Traffic Simulation and Optimizing Based on Random Walk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yu Cheng, Tao Zhang, and Jianfei Wang Vehicle Detection Using Bayesian Enhanced CoBE Classification . . . . . . . Zhong Zheng, Shen Xu, and Yi L. Murphey
671
679
687
695
703 713
XXIV
Table of Contents
Vibration Analysis of a Submarine Model Based on an Improved ICA Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Wei Cheng, Zhousuo Zhang, and Zhengjia He
721
A Hierarchical Salient-Region Based Algorithm for Ship Detection in Remote Sensing Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Fukun Bi, Feng Liu, and Lining Gao
729
Industrial Applications Turning Tool Wear Monitoring Based on Fuzzy Cluster Analysis . . . . . . . Hongtao Chen, Sui Huang, Dengwan Li, and Pan Fu Part-Machine Clustering: The Comparison between Adaptive Resonance Theory Neural Network and Ant Colony System . . . . . . . . . . . Bo Xing, Wen-Jing Gao, Fulufhelo V. Nelwamondo, Kimberly Battle, and Tshilidzi Marwala
739
747
Fault Diagnosis of Bearings Based on Time-Delayed Correlation and Demodulation as Well as B-Spline Fuzzy Neural Networks . . . . . . . . . . . . . Pan Fu, Li Jiang, A.D. Hope, and Weiling Li
757
Fast and Noninvasive Determination of Viscosity of Lubricating Oil Based on Visible and Near Infrared Spectroscopy . . . . . . . . . . . . . . . . . . . . Lulu Jiang, Yu Zhang, Fei Liu, Lihong Tan, and Yong He
765
Chattering-Free Adaptive Wavelet Neural Network Control for a BLDC Motor via Dynamic Sliding-Mode Approach . . . . . . . . . . . . . . . . . . . . . . . . . Chun-Fei Hsu and Chih-Hu Wang
773
A New BPSO Algorithm and Applications in Interruptible Load Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ping Huang, Pengcheng Li, Yao Zhang, Jinyang Yu, and Yongquan Yuan
783
Force Identification by Using Support Vector Machine and Differential Evolution Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Zhichao Fu and Wei Cheng
793
Soft Sensor Modeling of Ball Mill Load via Principal Component Analysis and Support Vector Machines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jian Tang, Lijie Zhao, Wen Yu, Heng Yue, and Tianyou Chai
803
An Approach Based on Hilbert-Huang Transform and Support Vector Machine for Intelligent Fault Diagnosis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chang Che and Dan Hu
811
Table of Contents
XXV
Real-World Applications Study on Factors of Floating Women’s Income in Jiangsu Province Based on Bayesian Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yingyu Ge, Chunping Li, and Qin Yin
819
Variation Trend Analysis of Groundwater Depth in Area of Well Irrigation in Sanjiang Plain Based on Wavelet Neural Network . . . . . . . . . Hong Ding, Dong Liu, and Fei-fei Zhao
829
A Petri-Net Modeling Method of Agent’s Belief-Desire-Intention and Its Application in Logistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Weifeng Zhu and Qi Fei
837
Supply Chain Flexibility Assessment by Multivariate Regression and Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ananda S. Jeeva and William W. Guo
845
An Intelligent Mobile Location-Aware Book Recommendation System with Map-Based Guidance That Enhances Problem-Based Learning in Libraries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chin-Ming Chen and Yu-Chieh Yang
853
Applying Least Squares Support Vector Regression with Genetic Algorithms for Radio-Wave Path-Loss Prediction in Suburban Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kuo-Ping Lin, Kuo-Chen Hung, Jen-Chang Lin, Chi-Kai Wang, and Ping-Feng Pai
861
A Probabilistic Neural Network Approach to Modeling the Impact of Tobacco Control Policies by Gender . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Xiaojiang Ding, Chung-Hsing Yeh, and Susan Bedingfield
869
The BPNN-Fuzzy Logic Pre-control of an Underground Project in City Center of Shanghai . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Zhenzhen Yin, Youliang Chen, and Peng Wang
877
Optimal Parameter Inversion of Marine Water Quality Model Using a BPNN Data-Driven Model — A Case Study on DIN . . . . . . . . . . . . . . . . . Mingchang Li, Bin Zhou, Shuxiu Liang, and Zhaochen Sun
885
Determination of Sugar Content of Instant Milk-Tea Using Effective Wavelengths and Least Squares-Support Vector Machine . . . . . . . . . . . . . . Fei Liu and Yong He
893
Sports Video Summarization Based on Salient Motion Entropy and Information Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Bo-Wei Chen, Jhing-Fa Wang, Jia-Ching Wang, and Chen-Yu Chen
901
XXVI
Table of Contents
A Neural Network Based Algorithm for the Retrieval of Precipitable Water Vapor from MODIS Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Shenglan Zhang, Lisheng Xu, Jilie Ding, Hailei Liu, and Xiaobo Deng A Neural Network Based Approach to Wind Energy Yield Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Piers R.J. Campbell, Faheem Ahmed, Haydar Fathulla, and Ahmad D. Jaffar
909
917
Research on New Intelligent Business-Oriented Decision-Making Model Based on MA and GA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Weijin Jiang and Qing Jiang
925
Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
933
A Novel Prediction Mechanism with Modified Data Mining Technique for Call Admission Control in Wireless Cellular Network Chen-Feng Wu, Yu-Teng Chang, Chih-Yao Lo, and Han-Sheng Zhuang Department of Information Management, Yu Da University, No. 168, Hsueh-fu Rd, Tanwen Village, Chaochiao Township, Miaoli County, 361 Taiwan {cfwu,cyt,jacklo,97104504}@ydu.edu.tw
Abstract. It is an important issue to allocate appropriate resources to mobile calls for wireless cellular networks owe to scarce wireless spectrums. The call admission control (CAC) will maintain better performance metrics of mobile call such as call dropping probability (CDP) and call blocking probability (CBP) if the future utilization of wireless spectrums can be predicted and provided to the decision of CAC. Therefore, a prediction mechanism which can predict most information such as system utilization is proposed in this paper. The techniques of data mining and pattern matching which adopts gradient to fuzz time series data for representations of chain code are applied to mining a possible repetitive pattern. Our proposed prediction mechanism can provide prediction information in advance whether the repetitive time series pattern of information exists or not. Furthermore, an update of confident level will be conducted continuously for performing each prediction in the proposed scheme. Our proposed mechanism is developed and tested with four cases which can be regarded as using scenarios of wireless cellular network. The experimental results show that the proposed scheme can capture repetitive time series patterns and perform following predictions with these repetitive time series patterns. Besides, the required storage is less than traditional schemes and lower computation power is required for the proposed scheme. Keywords: Call Admission Control (CAC), data mining, time series, pattern matching, prediction.
1 Introductions Although there has been a rapid development in wireless cellular communications, the QoS guarantee remains one of the most challenging issues [1]. One of the key elements in providing QoS guarantees is an effective CAC policy, which not only has to ensure that the network meets the QoS of the newly arriving calls if accepted, but also guarantees that the QoS of the existing calls does not deteriorate. The variable user mobility has made that it becomes more complex to predict the appropriate cell for handoff. The past research [2] showed the impact of mobility on cellular network and provided a modeling method for configuring cellular networks to study the dynamics of mobility. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 1–9. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
2
C.-F. Wu et al.
The improvement of radio bandwidth is always thought as a dynamic channel (code) allocation problem in literatures [3]. Although there were some schemes for bandwidth reservation proposed to reduce the CDP, such as the study [4], seldom literature has developed to satisfy QoS and lower CBP issues at the same time. Furthermore, the past researches mentioned at the above focused on individual mobility prediction, and they may cause mass load focusing on the MSC. According to the study [5], final information is a prediction of users’ number in a given cell, and it leads to use a global approach that only observes variations of system utilization and users’ flows. There are many advantages for [5]: it does not require any control message and additional load for the MSC; cells generate their own statistics independently from others; it is sensitive to geographical constrains and to users’ common habits. According to [12], the final information is a quantity prediction of users in a given cell, and it leads to use a global approach that only observes variations of system utilization and users’ flows. There are many advantages of the scheme presented in [12]: it does not require any control message and additional load for the MSC; cells generate their own statistics independently from others; it is sensitive to geographical constraints and to users’ common habits. Besides, the concept of aggregated history has been also applied to acquire user mobility profile in [13] so that a user mobility profile framework is developed for estimating service patterns and tracking mobile users, including descriptions of location, mobility, and service requirements. Therefore, in order to provide suitable statistical prediction information which may be system utilization, CBP, or other system resources to the CAC, a prediction mechanism which can predict most information is proposed in this paper.
2 Related Works A suitable prediction mechanism can help the CAC scheme to do better decision for call admission, thus the integration of CAC and prediction will effectively reduce the CDP to meet the QoS requirement. 2.1 CAC Overviews Although CAC schemes can be classified into centralized ones that make the decision of call admission in the MSC and distributed ones that make the decision of call admission in individual cell locally, most of prior researches [14], [15] that were based on different theories focused on the developments of centralized CAC schemes. Instead of using centralized CAC schemes, distributed CAC schemes have been proposed in [16], [17]. Based on the local information, Zhang et al. [16] have applied two prediction mechanisms that were historic data and time series prediction to CAC scheme in noncollaborative manner. Unfortunately, the overheads that are caused by predicting the appropriate information for making CAC decision have been still existed. A clustering prediction scheme that is based on data mining concept for user mobility prediction has been proposed in [17], but the related information from neighboring cells is still required to participate in mobility prediction and more computation loads and storage for data mining process are also required. Although the above researches exhibit that the distributed CAC schemes can reduce the traffic loads
A Novel Prediction Mechanism with Modified Data Mining Technique
3
concentrating on the MSC indeed, some drawbacks, such as an efficient prediction scheme, still exist for themselves. 2.2 Time Series Prediction One of the typical statistic models that based on statistical concepts and principles is Box-Jenkins forecasting model [6]. Although a wide spectrum of time series behavior can be modeled by such models, they can not model well when there are noises in the time series such as inaccurate or incomplete data. In the research [7], neural networks that are one of the well-known artificial intelligent techniques have been claimed to be able to achieve better prediction compared to statistical models. However, the study [8] shows that there exit some drawbacks for neural networks, such as hard to pre-select the system architecture, spending much training time and lacking knowledge representation techniques. Besides, the study [9] applies the data mining concept to perform time series prediction. Both adaptive association rule and pattern matching techniques are employed by [9] to mine for repetitive patterns, which can be used to understand the underlying utilization of system utilization, in the time series. Although this model can perform life long adaptation and reinforcement learning for continuous prediction, the disadvantage is the requirement for intensive computation power and huge storage. The proposed prediction module that is based on time series prediction can provide the appropriate information for the CAC algorithm to make the decision of call admission. Comparing to the traditional data mining process for time series prediction scheme, the proposed prediction method reduces calculation overheads and storage requirements significantly.
3 Methodology Data mining is always applied to perform the prediction by analyzing data and look for data behavior pattern owing to its four basic goals: prediction, identification, classification, and optimization. There exist many knowledge representations, and the association rule is commonly used. Association rules correlate the presence of a set of items with another range of values for another set of variables. 3.1 The Proposed Prediction Method The process flow of the proposed prediction method can be summarized in four steps, as shown as follows: Procedure of Prediction Begin Step 1: Time-series data analysis; Step 2: Time-series prediction; Step 3: Compensation prediction; Step 4: Prediction selection; End
4
C.-F. Wu et al.
Step 1: Time Series Data Analysis This step is response for performing data pre-processing and data mining on time series data. The purpose of data pre-processing is to convert the time series data into gradient code which is defined by the application characteristics of prediction. The data mining is to generate chain code pattern, and the learning results are stored into database which will be used for prediction task. The time series data is represented in a chain code format to keep track of the rate-of-change of time series. The rate-of-change of a time series is obtained by calculating the difference between each two temporal point. The following phase of the analysis step performs data mining to detect the periodicity of patterns and other relative information. The process of data mining is based on Eq. (1) to find iterative pattern. ⎛ R (d ) = set ⎜ ⎝
i+ n −1
∑
t=i
⎞ G (t + d ) − G (t ) ⎟ ⎠
(1)
The set function decides whether a repetitive pattern exists or not. If there exists a repetitive pattern of length n that is represented as the size of the pattern window, the n differences between each gradient vectors must be all zero with distance d, where d is the delay (d >= n) and i is the time of the first point in the preceding pattern. Pattern with different distances between the pattern length and the delay are managed in different list. The occurrence times of periodic patterns and the average of compensation values will be recorded in the database. Step 2: Time-Series Prediction After data mining, the prediction step will make use of periodic patterns to predict possible time series data in future. If a segment of a repetitive time series pattern matches with the inputs, it may happen that the following segment of the particular repetitive pattern has the possibility of reoccurrence. The prediction result with highest probability which is decided by counts of these patterns will be adopted. Because the prediction value is only a gradient code, the final output should be adjusted by the average compensation value and decoded. Fig. 1 illustrates the process of performing prediction task. The example that is with prefix pattern [1, 2, -4, 3] of gradient code wants to predict the value G(t+1), and G(t+1) = -2 with the highest probability 54% will be predicted. Moreover, the prediction gradient code will be passed through the compensation and decoding process to obtain the prediction times series data X(t+1).
Fig. 1. An example of performing prediction
A Novel Prediction Mechanism with Modified Data Mining Technique
5
Step 3: Compensation Prediction In order to provide more accurate prediction information, the compensation prediction that is based on Eq. (2) to perform prediction generates different prediction values to supply selection step for choosing the most appropriate one. C P ( t + 1) = α × ( C P ( t ) + ε t ) + (1 − α ) × X ( t )
ε
t
=
1 t
t
∑
i=1
ε
i
ε i = X (i) − i X (i )
(2) (3) (4)
For Eq. (2), the compensation prediction is concerned with the actual time series data X(t) at current time t and the previous prediction result CP(t) at time t-1. The parameter α is used to decide the weight of the value CP(t), and ε t is average difference that is the distance from actual value X(i) to prediction value i X (i ) and is defined by Eq. (3). Step 4: Prediction Section This step is responsible for determining the final prediction result between time series prediction and compensation prediction and updating the confident levels of pattern lists. Because the proposed scheme can dynamically remind the confidence level of pattern lists as time goes on, the pattern lists with low confidence level will be gradually neglected.
Fig. 2. An example of updating the pattern list
The process of updating the pattern list is illustrated in Fig. 2. At time t+1, the actual time series data X(t+1) that is with former pattern [1, 2, 4, -3] is received and transferred into G(t+1) by pre-processing process. If the actual value of G(t+1) is -2, 1, or 6, the count value of the existing pattern list will increase, and the corresponding confident level will be risen at next prediction. A new pattern will be added to the pattern list, if G(t+1) is not an expected value, such as 8. The prediction selection step dynamically adopts one of the prediction results that are from both compensation prediction and time series prediction. For compensation prediction step, the compensation algorithm generates several results with various coefficients αs, and the best prediction result will be chosen according to the average distance.
6
C.-F. Wu et al.
3.2 Design Considerations There are two defined thresholds, the distance threshold and the count threshold, dominating the accuracy of the time series prediction. If the difference between the prediction value and the actual value is larger than the distance threshold, the prediction will be regarded as a prediction miss. The time series prediction will be thought of being inaccurate when the number of continuous prediction misses is more than the count threshold. For compensation prediction step, the compensation algorithm generates several results with various coefficients of αs, and the best prediction result will be chosen according to the average distance. If there has not been any periodic pattern found or the accuracy of the time series prediction is too low, then the best prediction value that is generated by the compensation prediction will be regarded as the output. Otherwise, the prediction result of the time series patterns will be selected and sent out, and the prediction process is also completed simultaneously by the proposed prediction system. For reducing the consumption of computation capability and the demand of storage, some techniques are applied to the implementation. A list that records the possible pattern length is maintained in the database. The pattern lengths with the higher occurrence probability will earlier become the window size of performing pattern detection, and it will avoid running the detecting algorithm for every single length pattern. All of the system average values are computed by Eq.(5) instead of Eq.(6). A v g (n ) =
n − 1 × A v g (n − 1) + n
A v g (n ) =
1 × n
1 × v a lu e ( n ) n
(5)
n
∑
i = 1
v a lu e (i)
(6)
Where Avg(n) is the average of n number of values, and value(n) is the last value in the data stream. Eq.(6) needs to record all of the n values in database; on the contrary, Eq.(5) reduces the storage demand and less computation is needed.
4 Experimental Results There are four artificial test cases designed for verifying the accuracy of the proposed prediction module for system utilization for a given cell, and the time scale is in minute for all simulations. The purposes of these four test cases are to test how well the prediction method can detect the periodic patterns and perform prediction using these patterns, even though the periodic pattern is not found. The simulations start at 00:00, and advanced time unit is 00:01. The simulation results are shown in Fig. 3 to Fig. 6. The prediction method can provide the next three prediction value X(t+1), X(t+2), and X(t+3) at the time t. For the first case, a periodic pattern is successfully found at 00:08, as shown in Fig. 3, since the pattern ‘4, 2,-1, -4’ has appeared twice. The prediction value which produced by the compensation prediction step with the most suitable α is adopted by the prediction method before the pattern detected. While the periodic pattern has been detected, the time series prediction step applies it to perform prediction.
A Novel Prediction Mechanism with Modified Data Mining Technique
7
For the second case, the simulation results of the second case that is the first case of time series added a little bit of random oscillation is shown in Fig. 4. At the same time 00:08, the same pattern has been detected and the prediction value from compensation step is also replaced by the result of time series prediction step at the following time. Obviously, the compensation values of each item in the pattern list are different, and the prediction step adjusts the expected prediction value with the average compensation values. The third case is applied to test a suddenly continuous scenario as shown in Fig. 5. The front part of this time series is the same as the first test case except at 00:09 the utilization increases 10% suddenly and lasts in the following time series. The identical pattern can be detected but the prediction value that is derived from the periodic pattern causes one miss. Whatever, the prediction module can restore the accuracy immediately. In the fourth case, the time series prediction sub-module identifies successfully no existence of periodic pattern found. Thus, the compensation prediction step is instead of time series prediction step to provide the prediction value in the prediction method. Fig. 6 shows that the compensation prediction step can provide better prediction values at the part with fixed and smaller variation. According to the simulation results of the four test cases, the proposed prediction method can perform well in all kinds of time series data. Once the repetitive patterns exist in the time series, the step of time series data analysis can identify the pattern quickly, and accurate prediction values can be provided by the time series prediction step. Contradictorily, if there is not any periodic pattern existing, the proposed prediction module not only successfully identifies that the time series is random but also provides continuously the prediction values in the following times. Actual Value Predict at t-1 Predict at t-2 Predict at t-3
Time series data for utilization (simple case) 100 90
Utilization
80 70 60 50 40
00 :1 8
00 :1 9
00 :1 6
00 :1 7
00 :1 4
00 :1 5
00 :1 2
00 :1 3
00 :1 1
00 :0 9
00 :1 0
00 :0 8
00 :0 7
00 :0 6
00 :0 5
00 :0 3
00 :0 4
00 :0 1
00 :0 2
00 :0 0
30
Time
Fig. 3. The simulation result for system utilization of simple case Actual Value Predict at t-1 Predict at t-2 Predict at t-3
Time series data for utilization (thrashing case) 100 90
70 60 50 40
00 :1 9
00 :1 7
00 :1 8
00 :1 6
00 :1 5
00 :1 4
00 :1 3
00 :1 2
00 :1 0
00 :1 1
00 :0 9
00 :0 8
00 :0 7
00 :0 6
00 :0 5
00 :0 4
00 :0 3
00 :0 2
00 :0 1
30
00 :0 0
Utilization
80
Time
Fig. 4. The simulation result for system utilization of thrashing case
8
C.-F. Wu et al. Actual Value Predict at t-1 Predict at t-2 Predict at t-3
Time series data for utilization (occupation case-2) 100 90
Utilization
80 70 60 50 40
00 :1 9
00 :1 7
00 :1 8
00 :1 6
00 :1 4
00 :1 5
00 :1 3
00 :1 1
00 :1 2
00 :1 0
00 :0 9
00 :0 7
00 :0 8
00 :0 6
00 :0 4
00 :0 5
00 :0 3
00 :0 1
00 :0 2
00 :0 0
30
Time
Fig. 5. The simulation result for system utilization of occupation case Actual Value Predict at t-1 Predict at t-2 Predict at t-3
Time series data for utilization (random case)
oatin iliztU
100 90 80 70 60 50 40 30 20 10 0
:00 00
:01 00
:02 00
:03 00
:04 00
:05 00
:06 00
:07 00
:08 00
:09 00
:10 00
Time
:11 00
:12 00
:13 00
:14 00
:15 00
:16 00
:17 00
:18 00
:19 00
Fig. 6. The simulation result for system utilization of random case
5 Conclusions The simulation results of the prediction method show that the proposed scheme works well for the four different kinds of test cases. According to the simulation results of the time series prediction method, the proposed prediction method can estimate the required resources accurately in advance, so it could be applied in several fields that need to estimate time series based information beforehand for further manipulations, such as applications in finance, Internet traffic, grid computing, and so on.
Acknowledgements This work was supported in part by the National Science Council of Taiwan under the Grants NSC97-2221-E-412-001.
References 1. Akyildiz, I.F., McNair, J., Ho, J., Uzunaliogluh, H., Wang, W.: Mobility management in next-generation wireless systems. In: Proc. of IEEE ICON 1999, vol. 87, pp. 1347–1384. IEEE Press, Los Alamitos (1999) 2. Zaidi, Z.R., Mark, B.L.: Mobility estimation for wireless networks based on an autoregressive model. In: Proc. of IEEE GLOBECOM 2004, vol. 6, pp. 3405–3409. IEEE Press, Los Alamitos (2004)
A Novel Prediction Mechanism with Modified Data Mining Technique
9
3. Kim, Y.C., Lee, D.E., Lee, B.J., Kim, Y.S.: Dynamic channel reservation based on mobility in wireless ATM networks. IEEE Communication magazine, 47–51 (1999) 4. Tajima, J., Imamura, K.: A Strategy for Flexible Channel Assignment in Mobile Communication Systems. IEEE Transactions on Vehicular Technology 37 (1988) 5. Perato, L., Agha, K.A.: Handover Prediction: User Approach versus Cell Approach. In: Proc. of Mobile and Wireless Communications Network 4th International Workshop, pp. 9–11. IEEE Press, Los Alamitos (2002) 6. William, W., Wei, S.: Time Series Analysis: Univariate and Multivariate Methods. Addison Wesley Publication, Reading (1999) 7. Kamitsuji, S., Shibata, R.: Effectiveness of Stochastic Neural Network for Prediction of Fall or Rise of TOPIX. Research Paper, Keio University (2004) 8. Kasabov, N.: Evolving Connectionist Systems. Springer-Verlag London Limited (2003) 9. Yaik, O.B., Yong, C.H., Haron, F.: Time Series Prediction Using Adaptive Association Rules. In: Proc. Of the First International Conference on Distributed Frameworks for Multimedia Applications (DFMA 2005), pp. 310–314. IEEE Press, Los Alamitos (2005) 10. Chen, H., Kumar, S., Kuo, C.-C.J.: Dynamic Call Admission Control Scheme for QoS Priority Handoff in Multimedia Cellular System. In: Proc. of IEEE Wireless Communications and Networking Conference WCNC 2002, vol. 1, pp. 114–118. IEEE Press, Los Alamitos (2002) 11. Gaasvik, P.-O., Cornefjord, M., Svensson, V.: Different methods of giving priority to handoff traffic in a mobile telephone system with directed retry. In: Proc. Of 41st IEEE Vehicular Technology Conference ‘Gateway to the Future Technology in Motion’, pp. 549–553. IEEE Press, Los Alamitos (1991) 12. Hong, D., Rapport, S.S.: Traffic model and performance analysis for cellular mobile radiotelephone systems with prioritized and nonprioritized handoff procedures. IEEE Trans. on Vehicular Technology 35, 77–92 (1986) 13. Akyildiz, I.F., Wang, W.: The predictive user mobility profile framework for wireless multimedia networks. IEEE/ACM Transactions on Networking 12(6), 1021–1035 (2004) 14. Levine, D.A., Akyildiz, I.F., Naghshineh, M.: A resource estimation and call admission algorithm for wireless multimedia networks using the shadow cluster concept. IEEE/ACM Transactions on Networking 5(1), 1–12 (1997) 15. Hou, J., Yang, J., Papavassiliou, S.: Integration of pricing with call admission control to meet QoS requirements in cellular networks. IEEE Transactions on Parallel and Distributed System 19(9), 898–910 (2002) 16. Zhang, T., et al.: Local predictive resource reservation for handoff in multimedia wireless IP networks. IEEE Journal on selected areas in Communications 19(10), 1931–1941 (2001) 17. Tsiligaridis, J., Acharya, R.: A clustering prediction scheme for wireless cellular network. In: Proc. of the 2005 International Symposium on Collaborative Technologies and Systems, pp. 298–304. IEEE Press, Los Alamitos (2005)
The Study of Forecasting Model of Rock Burst for Acoustic Emission Based on BP Neural Network and Catastrophe Theory Yunyun Xu1,2 and Dongqiang Xu2 1 2
Department of Civil Engineering of Tangshan College, Hebei Tangshan 063009, China College of Civil Engineering, Hebei University of Technology, Tianjin 300132, China
Abstract. Forecasting model of the rate of rock burst acoustic emission time series has been established by BP neural network, and it is combined with the catastrophe theory to determine whether the rock burst. And then the experimental data recorded are used for examining the model. The results show that the degree of prediction accuracy is high, and it proves that the prediction model of rock burst is feasible. Keywords: rock burst, acoustic emission, forecasting model, BP neural network, catastrophe theory.
1 Introduction Rock burst is a usual geologic disaster, which occurs in rock of underground works of high ground stress. The reason for rock burst is that stress field in wall rock redistributes, caused by unloading in the process of excavation, and it results in elastic strain energy stored in hard and brittle wall rock being released suddenly, and wall rock being burst, loosen, peeled, ejected or even thrown[1]. The process of rock burst preparation and development is that surrounding rocks in mining space or roadway deform rapidly and fracture, from flow to sudden change. Rock acoustic emission is an intuitive performance of rock micro breakdown, and it is caused by rock crack propagation under load [2]. Acoustic emission method is to predict the occurrence of rock burst based on the parameters in each stage of deformation such as acoustic emission rate, energy rate, and wave shape, and so on, which is the most direct means of detecting and predicting rock burst. Prediction of rock burst is to mainly determine the possibility of rock burst in a large area in the next period[3]. For a long time, the analysis of acoustic emission time series monitored and forecasting trends has become the primary means of predicting rock burst [4-5]. Gray system and the neural network model are the most important prediction methods [6]. But prediction results show that the gray model is suitable for exponential growth in time-series, and in dealing with geotechnical engineering timeseries prediction problem, gray model should be used with caution [7-8].In this paper, the rock burst prediction method by means of neural network theory and catastrophe theory is discussed on the basis of the test data of rock burst of acoustic emission. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 11–19. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
12
Y. Xu and D. Xu
2 The Establishment of Forecasting Model of Time Series Neural Network 2.1 The Introduction of BP Neural Network Error Back Propagation(BP) neural network is one of the more mature and widely used models. It is a former feed-forward network, mainly composed of input layers, hidden layers and output layers [9]. Network structure is shown in Fig.1.
Fig. 1. BP network topology
BP network is a kind of guiding learning network, which relies on adjusting the connection weights between layers of neurons (nodes) to "memory" the learning samples. And it automatically creates the highly nonlinear mapping relationship between the inputs and outputs by studying a large number of samples. It has been proved that a three-layer BP network includes m input nodes, 2m +1 hidden layer nodes and n output nodes, which can achieve any non-linear mapping from Rm to Rn [9]. 2.2 Process of Network Training Learning process of BP algorithm is composed of front-propagation and backpropagation. In the process of front-propagation, the input information is dealt with the hidden units layer by layer and transmitted to the output layers. If the output layer can not get the desired output, the back-propagation process is transferred, and the errors between actual value and network output return along the path to the original links. By modifying the link weights of every layer’s neurons, the errors reduce, and then the process of front- propagation is transferred, iterating until the error is less than the value given. (1) Design network structure to adapt to the issue (network layers and nodes of network layers), and choose reasonable network learning parameters (learning rate η and impulse parameters α ) . (2) Set the initial value of network connection weights and thresholds w ji (0) , θ j (0) , which are the random number in the range of [-1,+1].
The Study of Forecasting Model of Rock Burst for Acoustic Emission
13
p ( p = 1,2, " , K ) by the steps of (4) - (8). Sample data include input vector x = ( x1 , x 2 , " , x n ) and desired output vector (3) Deal with every study sample
t p = (t p1 , t p 2 , " , t pm ) . (4) Calculate the actual outputs of nodes layer by layer in forward direction. n
O pj = f j (∑ w ji O pj + θ j ) i =1
,
f ( x) =
1 1 + e−x
(1)
In the Formula 1, f (x ) is an active function of the S-type. (5) Calculate output network errors.
⎤ 1 k ⎡1 m E = ∑ ⎢ ∑ (t pj − O pj ) 2 ⎥ k p =1 ⎣ 2 j =1 ⎦ (6) If
(2)
E ≤ EG (the system average error margin) or iterations number reaches the
specified number, the study ends. Otherwise, the back-propagation turns to step (7). (7) Calculate reversely the error signal of each node in network layer by layer.
δ pj = O pj (1 − O pj )(t pj − O pj )
(3)
m
δ pj = O pj (1 − O pj )∑ δ pl w jl
(4)
l =1
(8) Modify the network connection weights and node thresholds.
Δw ji (t + 1) = η × δ pj × O pj + αΔw ji (t ) Δθ j (t + 1) = η × δ pj + αΔθ j (t )
,
(5)
(9) Turn to step (3). 2.3 Establishment of Time Series Neural NetworkForecasting Model Suppose that there is a time-series X (i ) , i = 1,2, " , N ; and N is the number of observation points. Geotechnical engineering time-series forecasting model can be described as the following:
X (t ) = ϕ[ X (t − 1), " , x(t − p )] In the formula, ϕ [.] is a non-linear function, and p is the order of the model. The building process of geotechnical engineering time-series forecasting model is to find ϕ [.] [10].
14
Y. Xu and D. Xu
From the time series X (i ) ,
i = 1,2, " , N , N − p samples can be constructed.
Input Layer Vector No1 Sample No t − p Sample No N − p Sample
Output Layer Vector
X (1), " , X ( p − 1), X ( p ) X (t − p ), " , X (t − 2), X (t − 1) X ( N − p ), " , X ( N − 2), X ( N − 1)
X ( p + 1) X (t ) X (N )
N − p samples constructed are used for the network shown in Fig.1 for learning and training, and then a stable network structure, connection weights and thresholds can be got. The time series forecasting model of geotechnical engineering is established based on artificial neural network. The input vectors are inputted into the network, using the stable network structure, connection weights and thresholds, so that the samples can be predicted.
3 Catastrophe Cusp Model and Yield Mechanism Cusp theory of the catastrophe theory is the most widely used in the range of mechanics [11]. It has two control variables and a state variable. Its regular function of potential function in the form is as the following:
v ( x) = x 4 + px 2 + qx
(6)
So phase space is three-dimensional, in which x is the state variable, p and q are control variables, and x is the time variable. The corresponding set M of all critical points for E, is called the equilibrium surface (shown in Fig. 2). M satisfying:
grad xV = 4 x 3 + 2 px + q = 0
Fig. 2. Balance surface and control surface
(7)
The Study of Forecasting Model of Rock Burst for Acoustic Emission
15
The graphics of M in ( x, p, q ) space is a smooth surface with a fold, so in different regions, the number of equilibrium position is different. It is easy to be proved that the potential function corresponding to the middle lobe takes the maximum (that is to say, grad x ( grad xV ) < 0 ), whose equilibrium position is unstable; the potential function corresponding to the upper and lower lobe takes the minimum, whose equilibrium position is stable. Clearly, there are vertical tangent points at the curve surface, that is, near the point to satisfy Formula 8 in Fig.2, the number of equilibrium position is different. These points are called sudden change points or singular points, whose projection at the control variables p, q plane constitutes a bifurcation set B (also called bifurcation). This set includes all the state variables, and makes x produce a sudden change.
8 p 3 + 27q 2 = 0 grad x ( grad xV ) = 12 x 2 + 2 p = 0 Eliminate
(8)
x from Formula7 and Formula 8, and the equation is got: 8 p 3 + 27q 2 = 0
(9)
Through the above analysis, the instability discriminant can be drawn as the following:
F = 8 p 3 + 27 q 2
(10)
If control point ( p, q ) changes, corresponding point position changes on the curve surface accordingly. But when the control points trajectory cross over the bifurcation set 8 p + 27 q = 0 , corresponding points must jump across the middle of leaves, that is, rock mass is unstable. The relation is described as the following: 3
If If If
2
F > 0 , the rock mass near measuring point is steady; F = 0 , the rock mass near measuring point is critical; F < 0 , the rock mass near measuring point is unstable.
From Formula 7, it can be seen that the number sequences increase in the form of an exponent, but this kind of exponential function is not suitable for the requirements of catastrophe theory. Because any single variable function can be spread by the Taylor formula, v(t ) can be spread by the Taylor series, and truncated to 5. Then we can get: 4
v(t ) = ∑ ai t i i =0
(11)
16
Y. Xu and D. Xu
∂ i v(t ) In Formula 11, a i = | t =0 , and suppose x = t − n by using of variable ∂t i substitution, we can get:
v( x) = x 4 + px 2 + qx
(12)
For the consistency of the neural network model and the mutation model, it can be obtained:
p = −6n 2 + a 2 / a 5 n = a3 / 5a5
(13)
p = −6 n 2 + a 2 / a 5
(14)
q = 8n 2 − 2na 2 / a 5 + a1 / a 5
(15)
Inserting AE parameters at the moment to predict in Formula 12, adopting the leastsquares method, we can get a i . And then, inserting a i in Formula 13 to Formula 15, we can get the value of p and q , which can be inserted in Formula 10 to calculate the value of F. At last, we can forecast the rock burst circumstances based on the value of F.
4 Application of BP Neural Network – Mutation Model in Rock Burst Prediction of Acoustic Emission To verify the accuracy of the neural network - mutation model in rock burst prediction, we use the data recorded on the experimental to test the model. The white marble specimens, whose integrity is good, are used for testing. In the entire failure process of the specimen, the acoustic emission event rate has a bimodal nature and when the stress reaches about 80% of ultimate stress, it reaches high-value soon until the stress reaches 95% ultimate stress or so. During this time from 80% to 95% ultimate stress, the acoustic emission event rate fluctuations little and has an upward trend. The acoustic emission parameters of white marble specimens after 85% of ultimate stress for some time are shown in Table 1 for every 5 seconds. The corresponding data of rate of acoustic emission time series at 190-260second are used as a model; and the corresponding data of rate of acoustic emission time series at 265-270second are used for testing. We insert the modeling data in the network to operate. The model order is 3, the input layer nodes number using the network is 2, hidden layer node number s is 5, and output layer nodes number is 1. After 15000 iterations, the network converges, and the steady network structure,
The Study of Forecasting Model of Rock Burst for Acoustic Emission
17
connection weights and threshold are got. The fitted and measured values are compared in Table 1. Table 1. Comparison of fitted and measured values Monitoring Time (s)
Monitoring value-AE event rate (time/m)
Predicting value-AE event rate (time/m)
Relative error (%)
190
85
87
2.35
195
90
93
-3.23
200
100
100
0
205
109
103
5.50
210
98
105
-7.14
215
105
107
-1.90
220
111
108
-0.93
225
113
110
2.70
230
109
112
-2.75
235
115
113
1.74
250
115
118
2.60
255
118
120
-1.69
260
124
126
-1.61
Table 2. Predictions comparison Monitoring time (s)
Measured value
Prediction value
Relative error%
265
129
133
-3.10
270
135
132
2.22
From Table 1, the neural network fitting is good, and the relative average error is only 2.62%. The rates of acoustic emission at 265s and 270s are predicted, and compared with the record values to test the correctness of the model. Those results are shown in Table2. It can be seen that for the samples without training, the neural network model has very high prediction accuracy and it is an effective way to solve the problem of rock burst time series. Use AE parameters shown in Table1 at the moment to calculate the value of F for predicting the burst of rock. The result is shown in Table 3.
18
Y. Xu and D. Xu Table 3. Rock burst prediction
Time(s)
p
q
F
Rock burst prediction
245
-5.7
51.69
56096.9
No burst
250
-5.65
-58.55
91091.9
No burst
255
-6.65
-29.30
20837.2
No burst
260
-7.695
-105.36
290512.1
No burst
265
-98.81
-35.08
-7686526.1
burst
From the values shown in Table 3, it can be seen that rock may burst in holes at 265 seconds when the specimens is under load. In reality, at 269 seconds the rock fragments in holes appear, followed by rupture of the specimens. It shows that the forecasting results are very consistent with the actual situation. Several other specimens are used to predict, and the forecast is successful.
5 Conclusion The studies show that artificial neural network method for prediction of rock burst has provided a new and effective method. In this paper, the forecasting model of the rate of rock burst acoustic emission time series has been established by using BP neural network, and it was combined with the catastrophe theory to determine whether the rock burst. The results show that the degree of prediction accuracy is high, and it proves that the prediction model of rock burst is feasible, and has important application value. The model described in this article can forecast the occurrence of rock burst, but it can not predict the level and scale of rock burst. Many factors must be considered comprehensively to predict rock burst, so it is necessary to develop rock burst prediction experts system to grasp comprehensively and accurately the laws of rock burst, and to take appropriate measures to prevent rock burst.
References 1. Wang, W., W.E.Y.: Summary and Development Analysis of Rock Burst Prediction Technology (in Chinese). Mining Safety and Environmental Protection 30(4), 12–15 (2003) 2. Zhang, Y.B., Kang, Z.Q., Jiang, G.H., Xu, D.Q.: Rock Burst Mechanism Based on Rock Damage and Acoustic Emission Theory (in Chinese). Metal Mine 12, 79–82 (2007) 3. Pan, C.L., Cao, X.B., Cao, P.: Rock Burst Prediction Method (in Chinese). Non-ferrous Mining and Metallurgy 13(6), 3–5 (1997)
The Study of Forecasting Model of Rock Burst for Acoustic Emission
19
4. Tang, Q.B., Cao, P.: The Analysis of Rock Burst Prediction Method (in Chinese). Jiang Xi Nonferrous Metals 6, 2–4 (1997) 5. Yang, T., Li, G.W.: Study Based on Prior Knowledge of the Rock Burst Prediction (in Chinese). Chinese Journal of Rock Mechanics and Engineering 19(4), 429–431 (2000) 6. Peng, Q., Zhang, R., Xie, H.P.: Prediction Model for Rock Burst Based on Acoustic Emission Time Series (in Chinese). Rock and Soil Mechanic 30(5), 1436–1440 (2009) 7. Lu, X.Q., Liang, X.H., Lu, X.J.: Neural Network and Its Application to Predication of Nonlinear Time Sequence (in Chinese). Systems Engineering-theory and Practice 6, 97–99 (1997) 8. Zhang, Y.X., Lu, S.L.: Application of Artificial Neural Network to Engineering Time Sequence Forecasting Problem (in Chinese). Journal of NanJing University of Science and Technology 21(6), 522–525 (1997) 9. Li, X.A., Zhang, X.H.: Introduction to Neural Network and Neural Computation (in Chinese). Northwestern Polytechnical University Press, Xi an (1994) 10. Zhang, Y.X.: Primary Research on Forecasting Problem of Time Sequence in Geotechnical Engineering (in Chinese). Chinese Journal of Rock Mechanics and Engineering 17(5), 552–558 (1998) 11. Ling, F.H.: Catastrophe Theory and Its Application (in Chinese). Shanghai Jiaotong University Press, Shang Hai (1987)
Application of Wavelet Neural Network to Prediction of Water Content in Crude Oil Huifen Niu1, Cuiling Liu1, Jinqi Wang2, and Xiaowen Sun1 1
Computer and Information Engineering College, Beijing Technology and Business University, 100048 Beijing, China 2 Earth Sciences College, Daqing Petroleum Institute, 163318 Daqing, China
[email protected],
[email protected],
[email protected],
[email protected]
Abstract. In the course of oilfield development, accurate measurement of water content in crude oil has always been playing important role in practicing development adjustment and enhancing the effects of stimulation operation, and moreover it determines the development perspectives of oilfield. After research of the method of measuring the rate of water content, and the non-linear mapping relation between the rate of water content and impact factors of crude oil, a model of predicting the rate of water content in crude oil about vertical well based on wavelet neural network is proposed. Results of the simulation in MATLAB indicated that the way of WNN (wavelet neural network) has better convergent rate, prediction precision, learning ability and generalization ability than the traditional BP neural network. The method of WNN can predict the water content in crude oil with high precision and it owns much more powerful theoretical guide and much better application effects. It will have a broad application in the future. Keywords: wavelet neural network, BP neural network, crude oil, water content, prediction model.
1 Introduction The water content of oil is the parameter which is not only important in oil gas and oilfield measure of production, but also indispensable in researching the development perspectives of oilfield. If the inspection accuracy of the rate of water content in crude oil can be improved, the production parameters will be more optimized and production accounting will be more reasonable during the storage and transportation process of the oilfield and oil gas[1]. Lots of work on the rate of water content in crude oil has been researched by many domestic and foreign scholars for a long time. In order to improve the measure precision, many kinds of methods have been proposed and many types of equipment about testing the rate of the water content have been designed. But it is hard to set up accurate mathematical models for all situations due to the complex nonlinear relationship of the factors which cause the effect of measuring the rate of the water content in crude and complicated work conditions. With the rapid development of computer and artificial intelligence technology, the prediction models about the rate of Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 21–27. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
22
H. Niu et al.
the water content in crude oil based on ANN(artificial neural network) have been put forward recently[2]. The ANN is suitable for the non-linear prediction, and in theory it can approach the non-linear sequences in any precision. But the ANN is hard to decide the structure of the net in the application, and it like slow convergence of learning and liability of dropping into local minimum [3]. The WNN has been a new analysis method of modeling in recent years, which is established with the combination of the wavelet transformation and the traditional ANN thoughts. It is the progression formed by transformation of shifting and dilation with wavelet decomposition, which has the same attribute of approximating function as wavelet decomposition. Meanwhile, because of introducing two new variables named non-integer dilation factor and shift factor, it has more freedom of motion than wavelet transform, more flexible and effective function approximation capability. It can effectively overcome some defects of normal ANN models. In this paper, after intensive analysis and research on the data of flux and the water content which get from multi-sensors, a prediction model of water content in crude oil based on WNN is established, and compared with the result of the traditional BP neural network.
2 Water Content in Crude Oil Measurement System The oil-water mixture flows between the inner conductor and the outer conductor of the coaxial line sensor, which is used as the measuring device. It also becomes the carrier of the spread of electromagnetic energy. Choosing appropriate parameters of the sensor can ensure the electromagnetic wave in the coaxial line propagate in the form of TEM (Transverse Electromagnetic). It can be got the message of water content by measuring the phase characteristics of the electromagnetic waves spreading in the coaxial line sensor, changing the characteristics of the voltage signal into a frequency signal, and then using the signal amplification circuit and A/D converter. The object of the measurement is well mixed wet crude, whose water content is from 0% to 100% and its gas content is certain[4-6]. The Coaxial-Phase Technology crude oil water content measurement system diagram is shown in Fig. 1.
Fig. 1. Schematic Diagram of Water Content in Crude Oil Measurement System
Application of Wavelet Neural Network to Prediction of Water Content in Crude Oil
23
3 Wavelet Neural Network Model 3.1 Basic Theory The wavelet neural network uses the shifting and dilation of the mother wavelet to constitute the wavelet base, and takes the wavelet as the excitation function of a hidden lay. This network takes the attribute of evaluation index which is used to describe the attribute of the evaluation object as the input vector of the network. And it takes the corresponding evaluation as the output vector of the network. It can batchprocessing by using the conjugate gradient method, and adjust the wavelet exponential and neural weight by self-adapting. Use enough samples to training the neural network, and through the study of the samples the wavelet neural network like a “black box”, it saved the experience of the expertise and some mathematical inference mechanism. And it established the nonlinear mapping relation from the attribute of the evaluation object to the output. In this way, we can use the wavelet which has been trained to evaluate the new object. 3.2 Wavelet Neural Network If towards ψ ( t ) ∈ L2 ( R ) and Ψ ( ω ) which is the Fourier transformation of ψ (t ) meet the admissibility condition of wavelet function[7]: cψ
=
∫
+∞
ω
− ∞
− 1
Ψ (ω )
2
dω
< ∞
(1)
Then ψ (t ) is be called basic wavelet, and it also referred to as mother wavelet. After
the ψ (t ) shift b, then flex a in different delay, it generates the Wavelet bases: ψ
a ,b
(t ) =
⎛ t − b ⎞ ⎟ a ⎝ ⎠
1
ψ ⎜
a
(2)
Where, a, b——the dilation factor and shift factor ofψ a ,b (t ) , a, b ∈ R , a ≠ 0 . Given non-linear time sequence transformation function as wavelet transformation is: 1
WT x ( a , b ) ≤ x (t ),ψ a ,b (t ) ≥
a
∫
+∞
−∞
⎛t −b⎞ x (t )ψ ⎜ ⎟dt ⎝ a ⎠
x(t ) ∈ L2 ( R) , the
(3)
The water content in crude oil sequence can be obtained through the method which is linear superposition, use the finite term of the wavelet levels to approach the real testing data of the water content of crude oil: g (x) =
L
∑
k =1
⎛ ⎜ w kψ ⎜ ⎜ ⎜ ⎝
n
∑
i=1
u
ki
⎞ x (i) − bk ⎟ ⎟ ⎟ ak ⎟ ⎠
(4)
24
H. Niu et al.
Where, g(x)—the prediction value of y; u ki ,
wk —weight coefficient; ak , bk —the
dilation factor and shift factor; n—dimension of input vector; L—number of wavelet basis. The structural diagram of wavelet neural network is shown in Fig. 2 (combined with the actual needs, here gives the situation of two input vectors and one output vector).
Fig. 2. Structural Diagram of Wavelet Neural Network
For N samples of the input and output that is (xt, yt)( t=1, 2, …N), the aim is to
u ki , wk , ak , bk and L to make the gt(x) and the yt u is best fitting. The network parameter ki can be optimized by the energy function of
make sure the network parameter
minimum mean-square error: E
= 0 . 5 [y t − g t ( x ) ]
2
t
(5)
The wavelet basis adopt the Morlet mother wavelet which is : ψ ( T ) = cos( 1 . 75 T ) exp( − 0 . 5 T
2
(6)
)
This wavelet is gaussian wave which is cosine modulation, it has high precision and strong antijamming feature. The calculating method in detail of this network is introduced as follow: 1. 2. 3.
Initialization of the network parameters; Input the training samples xt, and matching desired output value yt; The network self-learning, use the current network parameter to figure out the output of the network:
g
t
( x ) =
L
∑
k = 1
w
k
ψ
⎛ ⎜ ⎜ ⎜ ⎜ ⎝
n
∑
u
ki
x (i ) −
i = 1
a
k
b
k
⎞ ⎟ ⎟ ⎟ ⎟ ⎠
(7)
Application of Wavelet Neural Network to Prediction of Water Content in Crude Oil
4. 5. 6.
Figure out the instantaneous value of gradient vector; Back propagation, modify the network parameter; Calculation the error N
E = 0 .5 ∑
t =1
7.
25
[g t ( x ) −
yt
]2
(8)
If E < E max (one positive value has be preset) or it gets the maximum train epochs, then stop training; else order E=0, return to step (2).
4 The Prediction Model Simulation Prepare oil samples with calibration water content using artificial ration, and then use the three-phase flow dynamic mixed devices to simulate the actual oil well. According to the nonlinear mapping relation between the water content and its influencing factors, taking the frequencies of the coaxial line water detector based on phase and the measured value of the turbine flowmeter as the input vectors of the WNN, taking corresponding known water content in crude oil as the output vector. Get the node number of latent layer by auto-optimization function of the WNN. So the node number of latent layer is intended to be 8 wavelet function to build the WNN prediction model. In order to increase the precision of prediction, we need normalization processing the data firstly by using function (7), because the water content is between 0~1, so it does not need to normalization[8]. x i' =
x i − x min + 0 . 1 x max − x min + 0 . 1
(9)
Choose the former 44 groups of the known data as the training samples, and the remaining 11 groups as the testing samples. Momentum coefficient λ =0.5, learning rate η =0.3, learning error E=0.01, and simulate in matlab7.1. In order to make known the effects of the prediction of WNN, this paper also use traditional BP to predict the water content in crude oil, and compared these two methods. Fig. 3 and Fig. 4 are the fitting chart between the real data and the data predicted by BP and the fitting chart between the true value and the prediction value of WNN. From these two figures we can see that the data predicted by WNN is much more approach the real data. Tab. is the simulated result of BP and the simulated result of WNN. From table 1, we can see that the relative error and mean square error (MSE) of the prediction model based on WNN is much smaller than the relative error and mean square error (MSE) of the prediction model based on BP, the precision of prediction has been remarkably improved. So, we can find out that the predict precision and the data fitting of the prediction model of water content in crude oil based on WNN are both better than the model based on BP neural network.
Ⅰ
26
H. Niu et al.
Fig. 3. The fitting chart between the true value and the prediction value of BP
Fig. 4. The fitting chart between the true value and the prediction value of WNN
Application of Wavelet Neural Network to Prediction of Water Content in Crude Oil
27
Table 1. The Simulated Result of BP Neural Network and The Simulated Result of WNN
number
Measured of water content%
1 2 3 4 6 6 7 8 9 10 11
0 10 20 30 40 50 60 70 80 90 100
BP neural network Prediction Relative of water error% content% 3.10 ∞ 13.56 35.6 24.46 22.3 40.32 34.4 48.62 21.5 53.75 7.5 57.73 5.4 63.16 9.8 70.13 12.3 78.13 13.2 102.16 2.2
MSE
0.5224
WNN neural network Prediction Relative of water error% content% 0 0.0 10. 90 9.0 17.39 13.1 33.12 11.0 37.08 7.3 52.32 4.6 63.12 5.2 69.18 1.2 82.96 3.7 92.12 2.4 101.14 1.14
MSE
0.0562
5 Conclusion This paper proposed the prediction model of water content in crude oil based on wavelet neural network, use the nonlinear wavelet basis replace the excitation function of the neuron in neural network. It combines advantage of the wavelet and the traditional neural network, and through the calculation function such as flexible and shifting to multi-scale analysis the data. The result of the prediction shows that the wavelet neural network prediction model has the following advantages: much better convergence and higher prediction precision than BP network, and have stronger capability of study and popularize. So it can provide the on-line prediction and has a broad application.
References 1. Ming-yi, C., Xin-rong, Y.: Information Merge Technology of Multiple Transducers and Its Application in Moisture Measurement of Oil Products. J. Process Automation Instrumentation 23(8), 2–3 (2002) 2. Shun-tian, L., Yang, S.: The Systems Analysis and Design Based on Matlab-Neural Network. Xidian University Press, Xi’an (1999) 3. Tsaih, R., Hsu, Y.-s., Lai, C.C.: Forecasting S&P 500 Stock Index: Futures with a Hybrid AI System. J. Decision Support System, 161–174 (1998) 4. Jin-qi, W., Xi-fu, Q., Kui-yong, Z.: Coaxial Transmission Line Phase Method for Measuring Water Content of Oil Well. J. Chinese Journal of Scientific Instrument 23(1), 75–76 (2002) 5. Jin-qi, W., Xi-fu, Q., Jian-ming, C.: Fhase measurement of water content in oil well. J. Journal of Harbin Institute of Technology 34(2), 245–247 (2002) 6. Jin-qi, W., Xi-fu, Q., Ying-hua, Y.: Test Study of Water Cut Tool in Oil Well Based on Phase Method. Acta Metrologica Sinica 25(4), 366–368 (2004) 7. Xin, W., Lu, Z., Xiang, L.: Simulation and Application of MATLAB Neural Network. Science Press (2003) 8. Li-qun, H.: Artifical Neural Network Course Book. Beijng University of Posts and Telecommunications Press, Beijing (2006)
Prediction of Urban Heat Island Intensity in Chuxiong City with Backpropagation Neural Network Wujun Xi and Ping He Department of Geography and Tourism Management, Chuxiong Normal University, Chuxiong, China
[email protected],
[email protected]
Abstract. In order to improve prediction accuracy of urban heat island intensity, we chose 9 main influencing factors from 1981 to 2006 and predicted urban heat island intensity in Chuxiong city in 2006 with backpropagation neural network. The predicted value was 2.1815 . Compared with its measured value, residual error was 0.1042, relative error was 4.5588%. It was superior to results of SCGM model and GM(1,1) model. The result shows that backpropagation neural network is effective to predict urban heat island intensity.
℃
Keywords: Backpropagation neural network, Prediction, Urban heat Island intensity, Chuxiong.
1 Introduction An urban heat island (UHI) is an urban area which is significantly warmer than its surrounding rural areas [1]. Urban heat island is one of the most important features of urban climate and atmospheric environment, urban human factors and local meteorological condition cause urban heat island to form [2]. In 1820, The British chemist Lake Howard put forward urban heat island firstly, by observing the urban centre and surrounding town areas [3]. After that, many researchers paid great attention to it, and the attention increased in recent years (Figure 1).
Fig. 1. Academic attention to urban heat island effect in China from 1996 to 2008
Data source: www.cnki.net. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 29–36. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
30
W. Xi and P. He
Correlation study included urban heat island intensity[4-5], Sequential variation of urban heat island[6-7], spatial distribution characteristic and pattern of urban heat island [8-9], monitoring method of urban heat island[10-11], influencing factors of urban heat island[12-15], formation mechanism of urban heat island[16], influence of urban heat island and countermeasures[17-18]. However, the researches of prediction of urban heat island intensity were relatively less, and the method was mainly regression and gray system [18-21]. Professor Zhi CHEN predicted urban heat island intensity in Xian with SCGM model, the relative error was 5.0% [20]. Professor Ping HE predicted urban heat island intensity in Chuxiong with GM(1,1) model, the relative error was 7.97%[21]. When they predicted urban heat island intensity by using regression and gray system, they commonly used urban heat island intensity data, but not considered other factors. In order to remedy the monotony of them, we used backpropagation neural network to synthesize the relevant influencing factors to predict urban heat island intensity in Chuxiong city.
2 A Survey of Chuxiong City 2.1 Natural Situation of Chuxiong City Chuxiong city locates on North-central of Yunnan Province, 101.32°E, 25.01°N, the land area of 4,433 square kilometers, among which, the built-up areas is 23 square kilometers. It has a population of 500,000. The altitude is between 1700 meters to 2300 meters. Its mean annual temperature is 15.6 , and average annual rainfall is 864 millimeters. Chuxiong belongs to subtropical monsoon climate [22].
℃
2.2 Sequential Variation of Urban Heat Island in Chuxiong City According to the professor Ping He's research [6], in the years prior to 1988, the urban heat island intensity in Chuxiong city was below 1 , since then, the intensity was going up over 1 . It increased quickly and reached 2 in 2006 (Figure 2).
℃
℃ ℃
Fig. 2. Variation of annual heat island intensity in Chuxiong city from 1959 to 2000
The maximum value of heat island intensity in Chuxiong city from 1980 to 2000 happened in spring (from March to April), the minimum happened in summer (from June to July) (Figure 3).
Prediction of Urban Heat Island Intensity in Chuxiong City
31
Fig. 3. Variation of monthly heat island intensity in Chuxiong city
3 Prediction of Urban Heat Island Intensity in Chuxiong City with Backpropagation Neural Network 3.1 Influencing Factors of Urban Heat Island in Chuxiong City Because urban human factors and local meteorological condition cause urban heat island to form, so influence factors of urban heat island include relative humidity, air pressure, aqueous vapor pressure, wind velocity, low cloud amount, total cloud amount, sunshine duration, prevailing wind, greenhouse gases, atmospheric pollution, urban underlying surface, urban population, power consumption, coal consumption, number of motor vehicles, industrial pattern, land use type, density of population, water area, road area, building density and urban green coverage ratio, etc[20,21,24,25]. Considering influencing factors’relevancy and data’s acquisition, we chose 4 meteorological elements (relative humidity, wind velocity, total cloud amount, sunshine duration) and 5 human factors (urban population, urban underlying surface area, power consumption, number of motor vehicles and urban green coverage ratio) to analyze urban heat island intensity in Chuxiong city. 3.2 Prediction Process with Backpropagation Neural Network Data preparation. Convert 9 factors and urban heat island intensity data in Chuxiong city from 1981 to 2006 into non-dimensional quantities, all factors’ measured values as reference value 1. Table 1 was the processing results. Table 1. The influencing factors’ values of urban heat island intensity in Chuxiong city from 1981 to 2006 Year
Urban heat island intensity
Urban population
Urban underlying surface area
Urban green coverage ratio
Number of motor vehicles
1981
1.0000
1.0000
1.0000
1.0000
1.0000
1982
1.0000
0.9677
0.9362
1.0784
0.9928
1983
1.0000
1.0622
0.8511
1.0963
1.0870
1984
0.8571
1.2191
1.0851
0.9940
1.1726
1985
0.8571
1.1660
1.0638
1.0273
1.4039
1986
1.4286
1.2674
1.0638
1.1671
1.7280
32
W. Xi and P. He Table 1. (continued) 1987
2.2857
1.2885
1.0638
1.0554
1.9845
1988
2.0000
1.3226
1.3617
1.0656
2.3234
1989
2.1429
1.4746
1.3617
1.0656
2.4559
1990
2.1429
1.5572
1.3191
1.0656
2.8267
1991
2.1429
1.5499
1.7234
1.0656
3.0661
1992
2.1429
1.6971
1.7234
1.2447
3.4622
1993
2.8571
1.7372
2.6809
1.1679
3.9689
1994
2.7143
1.9483
2.8085
1.1765
4.0659
1995
2.5714
1.8281
2.8936
2.1381
4.8725
1996
2.8571
1.9296
2.8723
2.0946
6.1463
1997
3.2857
1.9751
3.0000
2.1475
6.8072
1998
3.0000
2.0821
2.8723
1.8824
7.0374
1999
2.8571
2.1664
2.8723
2.0350
8.0073
2000
2.8571
2.2027
3.5745
1.9241
8.2236
2001
2.5714
2.3715
3.0851
2.0716
8.4146
2002
3.0000
2.2386
3.6170
2.1824
9.0101
2003
2.2857
2.3707
3.6383
2.1483
9.9578
2004
1.8571
2.3615
3.8298
2.1560
11.6458
2005
2.1429
2.4351
4.4681
2.2165
14.0174
2006
2.2857
2.7815
4.2553
2.1876
15.1821
Year 1981
Power consumption 1.0000
Sunshine duration 1.0000
Wind velocity 1.0000
Relative humidity 1.0000
Total cloud amount 1.0000
1982
0.9954
0.8590
1.0000
1.0290
0.9474
1983
1.1382
0.7948
1.0625
1.0145
1.0877
1984
1.1648
0.9771
1.0625
1.0000
0.8421
1985
1.2453
0.8484
0.9375
1.0435
1.0351
1986
1.4415
0.9158
0.9375
0.9565
0.9123
1987
1.3941
0.9151
1.0000
0.9565
0.8772
1988
1.5710
0.8775
1.0000
0.9565
1.0351
1989
1.6595
0.8588
0.7500
0.9565
0.9825
1990
1.8474
0.6352
0.6875
1.0580
1.1053
1991
1.9153
0.8053
0.6875
0.9855
0.9123
1992
2.3170
0.7183
0.8750
1.0000
0.9649
1993
2.1374
0.7223
0.7500
0.9855
0.9649
1994
2.4260
0.8171
0.7500
1.0000
0.8947
Prediction of Urban Heat Island Intensity in Chuxiong City
33
Table 1. (continued) 1995
2.5505
0.7704
0.9375
0.9710
1.0702
1996
3.0175
0.7759
0.8750
1.0435
0.9649
1997
2.8170
0.8852
0.8125
0.9710
1.0000
1998
3.0634
0.7654
0.6875
0.9710
0.9474
1999
3.4123
0.7150
0.6250
0.9855
1.0877
2000
3.3011
0.6787
0.6250
0.9710
1.0702
2001
3.4886
0.6144
0.5625
1.0145
1.2105
2002
3.9863
0.6812
0.4375
0.9855
0.9649
2003
3.9070
0.8734
1.3750
0.9420
0.8070
2004
4.5524
0.6433
1.3750
0.9130
1.0526
2005
5.7407
0.6713
1.5625
0.8841
1.0526
2006
7.3056
0.7999
1.3750
0.9275
1.0175
Data source: Chuxiong city yearbook (1994-2007), Lucheng town statistical station.
Prediction with backpropagation neural network. Analysis tool was DPS Data processing system [23], it provided a rapid way to compute with backpropagation neural network. First, we imported all non-dimensional quantities into DPS Data processing system, then made backpropagation neural network analyze with 9 influencing factors and urban heat island intensity from 1981 to 2005. After repeated experiments, we finally determined essential parameters: the number of hidden network Layer was 1, minimum training ratio was 0.1, dynamic parameter was 0.4, sigmoid was 0.9, error excepted was 0.0001, maximum iteration number was 5000. When iteration number reached about 2300, the result matched error excepted, and the computation stopped, fitting residual error was 0.00094(Figure 4).
Fig. 4. Backpropagation neural network fitting curve
34
W. Xi and P. He
We predicted urban heat island intensity in Chuxiong city with 9 influencing factors in 2006 as observed data, the predicted value was 2.1815 . Compared with the measured value of urban heat island intensity in Chuxiong city in 2006, residual error was 0.1042, relative error was 4.5588%.
℃
3.3 Predicted Result and Analysis Compared with these researchers’ predict results[20,21], our predicted relative error was only 4.5588%. It showed that backpropagation neural network is effective to predict urban heat island intensity. The result is superior to results of SCGM model and GM(1,1) model.
4 Conclusion Prediction the development tendency of urban heat island intensity is more important now. However, the prediction accuracy was not very satisfied. In order to improve prediction accuracy of urban heat island intensity, we chose 9 influencing factors of urban heat island as main influencing factors by analyzing the urban heat island situation in Chuxiong city, the factors included relative humidity, wind velocity, total cloud amount, sunshine duration, urban population, urban underlying surface area, power consumption, number of motor vehicles and urban green coverage ratio. We made our backpropagation neural network analysis with 9 influencing factors and urban heat island intensity from 1981 to 2005, and predicted urban heat island intensity in Chuxiong city with these factors in 2006 as observed data, the predicted value was 2.1815 . Compared with the measured value of urban heat island intensity in Chuxiong city in 2006, the residual error was 0.1042, and the relative error was 4.5588%. Compared with the mentioned researchers’ predict results, our result is superior to the results of SCGM model and GM(1,1) model. It shows that backpropagation neural network is effective to predict urban heat island intensity, and it used many kinds of influencing factors synthetically to analyze urban heat island intensity. It has remedied the defect of predicting with unitary urban heat island intensity values, and has promoted the prediction accuracy.
℃
Acknowledgments. This study was supported by the National Natural Science Foundation of China (Grant No. 40965002), and Natural Science Foundation of Yunnan province (Grant No. 2005D0077M). And Hui Chen made a contribution to data collection.
References 1. Han, S., Guo, J., Huang, S., Bian, H.: Study on the Evolution of Urban Heat Island in Tianjin city (in Chinese). Ecology and Environment 16(2), 280–284 (2007) 2. Zhou, S., Shu, J.: Urban Climatology (in Chinese), p. 315. China Meteorological Press, Beijing (1994) 3. Howard, L.: Climate Deduced from Meteorological, vol. 1(3), pp. 1–24. Harvey and Darton, London (1833)
Prediction of Urban Heat Island Intensity in Chuxiong City
35
4. Wang, J., Wang, K., Wang, P.: Urban Heat (or Cool) Island over Beijing from Modis Land Surface Temperature (in Chinese). Journal of Remote Sensing 11(3), 330–339 (2007) 5. Li, F., Ma, A., Ding, Y., Yang, J., Jiao, J., Liu, L.: Research on Urban Heat Island Effect Based on Landsat Data (in Chinese). Remote Sensing Technology and Application 24(4), 553–558 (2009) 6. He, P., Li, H., Shu, J., Lu, X., Sun, J.: The Analysis of the Urban Climate in Chuxiong of Yunnan: Comparison of Urban Heat Island Effect in China (in Chinese). ACTA Geographica Sinica 58(5), 712–720 (2003) 7. Song, Y., Zhang, S.: The Study on Heat Island Effect in Beijing During Last 40 Years. Chinese Journal of Eco-Agriculture 11(4), 126–129 (2003) 8. Zhang, Y., He, Y., Ma, Y., Liu, Y., Li, Y., Dou, J., et al.: Characteristics of Vertical Distribution of Urban Heat Island Effect in Kunming City (in Chinese). Plateau Meteorology 21(6), 604–609 (2002) 9. Zheng, Z., Liu, W., Wang, Y.: Distributive Character of Urban Heat Island Effect in the Beijing Region (in Chinese). Journal of Nanjing Institute of Meteorology 29(5), 694–699 (2006) 10. Jian, W., Samei, S.: Research on the Method of Monitoring the Urban Heat Island by Remote Sensing Technique of Satellite (in Chinese). Modern Scientific Instruments (12), 27–29 (2004) 11. Rongbo, X., Ouyang, Z., Zhaoming, Z., Xiaoke, W., Weifeng, L., Hua, Z.: Advances in Methodology of Urban Heat Island (in Chinese). Meteorological Monthly 31(11), 3–6 (2005) 12. Cheng, C., Wu, N., Guo, S., Li, S., Liu, D.: A Study on the Interaction Between Urban Heat Island and Vegetation Theory, Methodology, and Case Study (in Chinese). Research of Soil and Water Conservation 11(3), 172–174 (2004) 13. Huang, J., Liu, X., Li, Q.: The Study Of Relationship Between Heat Island Effect and Population in Cities Over South of China (in Chinese). Journal of Tropical Meteorology 20(6), 713–722 (2004) 14. Ma, Y., Tiyip, T., Huang, Y., Yang, J.-L.: Effects of Landscape Pattern Change on Urban Heat-island Effect in Arid Areas—A Case Study in Urumqi (in Chinese). Arid Zone Research 23(1), 172–176 (2006) 15. Zhou, K., Ye, Y., Peng, S., Su, J.: Total suspended particulates and its effects on urban heat island (in Chinese). Ecology and Environment 15(2), 381–385 (2006) 16. Dai, X., Zhang, L., Guo, Z., Wu, J., Li, X., Zhu, Y.: Mechanism of formation of urban heat island effect and its spatial pattern in Shanghai (in Chinese). Acta Ecologica Sinica 29(7), 3995–4004 (2009) 17. Zhu, Z., Wang, M.: The Harms and Countermeasures of Urban Heat Island Effect (in Chinese). Pollution Control Technology 22(2), 94–96 (2009) 18. Wen, J., Bao, J., Zhang, Z.: Ecological Measure Analysis of Reducing Urban Heat Island (in Chinese). Ecological Economy (2), 151–153 (2008) 19. Wei, H., Zhao, Y., Chen, Y.: Grey Assessment and Prediction of the Urban Heat Island Effect in Lanzhou City (in Chinese). Journal of Desert Research 29(3), 571–573 (2009) 20. Zhi, C., Bingfeng, Y., Wangyang, H., Xilian, L., Linxiang, Q.: Grey Assessment and Prediction of the Urban Heat Island Effect in City (in Chinese). Journal of Xi’an Jiaotong University 38(9), 985–988 (2004) 21. He, P., Chen, H., Li, H., Xi, W.: Grey Analysis of the Urban Heat Island Effect Factors of the Medium-sized City of Chuxiong on Yunnan Plateau (in Chinese). Progress in Geography 28(1), 25–32 (2009)
36
W. Xi and P. He
22. Chuxiong Online . A Survey of Chuxiong, http://www.cxs.gov.cn/file_read.aspx?id=14944 23. Tang, Q.Y., Feng, M.G.: DPS Data processing system: Experimental design, statistical analysis, and data mining (in Chinese), pp. 932–935. Science Press, Beijing (2007) 24. Li, Z., Gong, C., Hu, Y., Yin, Q., Kuang, D.: The Progress of the Remote Sensing Research on Urban Heat Island (in Chinese). Remote Sensing Information (8), 100–105 (2009) 25. Wang, X., Bai, J.: The Development Prospect of the Urban Heat Island Effect (in Chinese). Journal of Foshan University (Natural Science Edition) 26(1), 53–56 (2008)
Research on the Fouling Prediction of Heat Exchanger Based on Wavelet Relevance Vector Machine Lingfang Sun, Rina Saqi, and Honggang Xie School of Automation Engineering, Northeast Dianli University, Jilin City, 132012, China
[email protected]
Abstract. Based on the relevance vector machine and wavelet theory, a new machine learning method–wavelet relevance vector is introduced. The method performs wavelet transform to decompose the original series into some filtered series, and a relevance vector machine whose kernel functions is wavelet function models each of them. Then, this method is used to predict fouling thermal resistance of Heat exchanger. Construction of wavelet relevance vector machine prediction model is presented. Simulations show that wavelet relevance vector machine requires dramatically fewer kernel functions and it can get high prediction precision, and work in the paper offers a new method for the research of heat exchanger fouling. Keywords: Fouling thermal resistance, Wavelet Transform, Relevance Vector Machine, Prediction.
1 Introduction Fouling exists in nature, daily life and most of the industrial process. Accumulation of fouling not only degrades heat transfer efficiency of heat exchanger, increases greatly initial capital and maintenance costs, but also causes overheat on certain local areas of heat exchanger, and threaten the safety of heat exchange equipment severely. The economic losses caused by fouling are as high as 0.25 of Gross National Product in a developed country [1]. In our country, the partial fouling losses have reached 0.11 of Gross Domestic Product only in electric power system [2], if considering other industry, the whole fouling losses is more than the developed country’s. So minimizing the harm of fouling becomes an effective avenue of save energy. As the foundation of the fouling research, research on the fouling prediction has become of one main direction since the late 1980s, however, for the many influence factors, fouling prediction based on the traditional mechanics analysis can not satisfy the actual needs. In recent years, the artificial intelligence is applied to fouling research, such as neural network [3,4], support vector machine (SVM) [5] and so on, and gets great results, but these methods also show some limitations at the same time. Tipping improved on the SVM and formulated the relevance vector machine (RVM) [6,7]. The RVM is a probabilistic model similar to the widespread SVM, but the training takes place in a Bayesian framework and predictive distributions of the outputs instead of point estimates are obtained. Relevance vector machine draws Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 37–45. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
38
L. Sun, R. Saqi, and H. Xie
hyper-parameters into run, implements automatic estimation of ‘nuisance’ parameters, and its kernel function doesn’t need to satisfy ‘Mercer’ condition, its calculate complexity is dramatically reduced and the operating speed is advanced. So the RVM has been applied to much industrial process recently [8,9]. In the paper, we constructs the wavelet relevance vector machine(WRVM) regression prediction model with the wavelet transform and relevance vector machine, and uses to analyze and predicts the heat exchanger fouling increase.
2 The Principles of Wavelet Relevance Vector Machine 2.1 Wavelet Transform Wavelet analysis is a multi-resolution analysis method both in time domain and frequency domain. On the low frequency part, the frequency resolution of wavelet transform is high, but the time domain resolution is low. On the high frequency part, the time domain resolution is high, but the frequency resolution of wavelet transform is low. So wavelet analysis is called a mathematic microscope. Wavelet function ϕ (t ) is called mother wavelet. a,b(t) can be obtained through compressing and expanding ϕ (t ) : −1
ψ a ,b ( t ) = a
2
ψ(
t−b ) a
(1)
Where a,b(t) is a successive wavelet; a is scale or frequency factor, a time factor; b R, R is the domain of real number. Wavelet transform is defined as:
∈
WTx (a,τ ) =
1 a
∫
+∞
−∞
x(t )Ψ ∗ (
∈R, a≠0; b is
t −τ )dt a
(2)
The equivalent formula in frequency domain: WTx (a ,τ ) =
a 2π
∫
+∞
−∞
X (ω )Ψ ∗ ( aω )e jωτ d ω
(3)
Fourier transformation of x(t) and ¸ (t) are X(¹) and ¸(¹) respectively. Let frequency factor a and time factor b be discrimination: a=2j and the discrete binary wavelet transform is written as: WT f ( j , k ) = 2
−
j 2
∫ f (t ) Ψ
∗
⎡t − 2 j k ⎤ ⎢ 2 j ⎥dt ⎣ ⎦
τ=2 k. So j
(4)
Where, k is translation factor, and j is wavelet decomposition level. Its binary wavelet reconstruction is formed as follows: f (t ) = ∑ c j ,k ϕ j ,k (t ) + ∑∑ d j , k φ j , k (t ) = A j (t ) + ∑ D j (t ) k
k
j
j
(5)
Research on the Fouling Prediction of Heat Exchanger
39
Where, Aj and Dj is the trend part (background information) and detail part (detail information) respectively. The trend series contains the slowest dynamic, and the detail series contains the dynamic at a certain intermediate scale. 2.2 Relevance Vector Machine
∈
∈
Given a training set {xn , tn }nN=1 , with xi Rn for the input, yi R for the target output, N for the number of training sample. Form this ‘training’, the regression problem is to find a mapping f from the input space to the output space. The purpose of the RVM is to find a relevance regression model based on the Bayesian framework:
t=¶w+¦
(6)
Here, ¹ is parameter vector, ¶ is the N×(N+1) design matrix with Φ = [φ1 , φ2 ,..., φM ]
(7)
Φ nm = K ( xn , xm )
(8)
Wherein
Due to the Bayesian inference, the likelihood of the complete date set can be written as: p(t w , σ 2 ) = (2πσ 2 ) − N 2 exp{− t − Φw
…
2
(2σ 2 )}
(9)
…,
wN). Where, t = (t1, t2, , tN), w = (w1, w2, Generally, with multi-parameter in the regression model, its maximum likelihood estimation may lead to severe over-fitting. And that, on the relevance vector machine, owing to the Bayesian inference makes model more general. The Gaussian prior distributions are chosen: M
p( w, α ) = (2π )− M 2 ∏ α m1 2 exp(−(α m wm2 ) 2) m −1
(10)
Where α = {α1 , α 2 ,...α M } is a vector of hyper parameters that governs the prior over the weight. The value of the hyper parameter is the key to the model sparsity and it also controls the strength of influence of the prior distribution on the each parameter. On the basis of prior distribution and marginal likelihood, the posterior distribution over the weights can be written using Bayes inference: p(w t ,α ,σ 2 ) = = (2π ) − ( N +1) 2 Σ
p ( y | w, σ 2 ) p ( w | α ) ~ N ( μ , Σ) p( y | α , σ 2 ) −1 2
exp{−( w − μ )T Σ −1 ( w − μ ) 2}
(11)
40
L. Sun, R. Saqi, and H. Xie
Where, the posterior covariance and mean are respectively given by: Σ = (σ −2 Φ T Φ + A) −1
(12)
μ = σ −2ΣΦT
(13)
With A = diag (α 0 , α1...α M ) . In practice, we can find that the majority of parameters are automatically set to zero during the leaning process. And the corresponding basis function can thus be ‘pruned’. To the RVM regression model, the non-zero parameters are the relevance vectors (like SV). By integrating the parameters, we obtain the marginal likelihood for the hyper parameters: p (t α , σ 2 ) = N (0, C )
(14)
Here, C = σ 2 I + ΦA−1ΦT . For a new input x∗ and output t∗ , we can then compute the predictive distribution is Gaussian: μ∗ = F μ
(15)
σ ∗2 = σ op2 + F T ΣF
(16)
With F = [Φ1 ( x∗ ), Φ 2 ( x∗ ),..., Φ M ( x∗ )]T . We can consider that μ∗ is the prediction to t∗ , and the distribution of t∗ is given, so we can obtain the Confidence interval under some conspicuous level. If given α ∗ , we can gain its 1 − α ∗ Confidence interval:
( μ∗ − t∗α / 2 ( N − k )σ ∗ , μ∗ + t∗a / 2 ( N − k )σ ∗ )
(17)
With N is the number of the sample and k is the number of parameter.
3 Construction of WRVM Prediction Model WRVM is to hang together the wavelet transform and wavelet kernel function with relevance vector machine. The construction of wavelet relevance vector machine have four steps: (1) decompose the original series into some filtered series with the resolution level 3 (6 time series); (2) reconstruct each sub-time series; (3) let wavelet function to be the kernel function of RVM; (4) predict each sub-time series (trend part and detail part), and combine the results of every series, and then we can acquire the final prediction result. The diagrammatic sketch is shown as Fig. 1. Due to that every sub-time series has its own characteristic, so the each predict model corresponding to the sub-time series is distinct.
Research on the Fouling Prediction of Heat Exchanger
41
Original series
x Wavelet decomposition
d1…
aJ
…
dJ
Wavelet Reconstruction
D1…
AJ
…
DJ
Wavelet Kernel RVM
YD1…
YAJ
…
YDJ
Prediction Result
Fig. 1. Method of WRVM
The capability of non-liner regression model can be evaluated by many measurements. The most important measurement to evaluate the quality of model is the accuracy. In this work, we use mean relatively error: ∧
MAPE =
1 n yt − yt ∑ n t =1 yt
(18)
4 Fouling Thermal Resistance Prediction Based on WRVM 4.1 Data Acquisition and Pretreatment Analyzing the system that we need to predict, make sure the characteristic parameters that can reflect the system’s state. We select the fouling thermal resistance as a way to detect the fouling degree. The data we acquired used for the research were from online monitoring equipment [10]. According to the methods in the reference [1], we found that six measuring parameters influenced fouling, i.e. input temperature of working fluid, output temperature of working fluid, flow velocity of working fluid and three temperatures of heat exchanger tube wall. So we use these parameters forming the inputs of WRVM. The data we collected were from the IDCB893 for every one minute, altogether 180 series sample consisted the discrete fouling thermal resistance data. In the same way, another group data were completed, for 180 series sample, for the prediction. These parameters were chosen to be the inputs of the SVM network. They represent themselves for different significations, so they need to be normalized. The process can be done with the function ‘Prestd’ in the Matlab, it is as below, [pn, meanp, stdp] = prestd (x) Here, ‘pn’ stands for the normalized input vector or matrix; ‘meanp’ and ‘stdp’ stand for the mean and the standard deviation of the x. The sample data were divided into two groups; the former 120 data were used for training, the rest ones were used for predicting. The second group was all used for prediction.
42
L. Sun, R. Saqi, and H. Xie
Hyper parameters and deviation was initialized respectively as follows: initAlpha=( 1 ) 2 N
initBeta=var[t] × 0.1
4.2 Structure of Kernel Function The performance of RVM is mainly decided by its kernel function. How to construct suitable kernel function relating on complicated practical application is an important aspect of RVM all the way. Two wavelet functions are chosen in this paper, Mexico wavelet function and Morlet wavelet function respectively. Mexico wavelet function d
xi − xi2
i =1
a2
K ( x, x ' ) = ∏ [(1 −
) exp( −
xi − xi'
2
)]
(19)
xi − xi' xi − xi' )exp(− )) a 2a2
(20)
2a 2
Morlet wavelet function: 2
d
K (x, x' ) = ∏(cos(1.75× i =1
4.3 Results and Discussion The simulation based on wavelet relevance vector machine includes 5steps. Step 1: Apply the function ‘wavedec’ of Matlab, and then decompose the original time series to 4 subsidiary time series. [ci,li]=wavedec (x1c (:,i),2,'db3') Step 2: Apply the function ‘wrcoef’ of Matlab and carry out wavelet reconstruction to each subsidiary time series. A(:,i) = wrcoef('a',ci,li,'db3',2) Step 3: Let Mexico wavelet function and Morlet wavelet function to be the kernel function of relevance vector machine. Through repeatedly experiments, we can select the most contented output. Step 4: In order to cater for the characteristic of the fouling time series, the parameters is setting as follow: The regression time-delay :d=3 The width of wavelet function: a=0.8 The variance of noise: σ 2 = var( y ) × 0.1 The other parameters:
itermax = 50 ,
α max = 1.0e5 , {α } = (ones (1.N + 1) × ( 1 )) 2 2
Step 5: Predict each sub-time series, and combine the results of every series, we can acquire the final prediction result. Corresponding experiment data and prediction results are shown as Fig. 2 – Fig. 8.
Research on the Fouling Prediction of Heat Exchanger
Fig. 2. Prediction curve of trend part (*) and real fouling cure (-)
Fig. 3. Prediction curve of detail part cd1 (*) and real fouling cure (-)
Fig. 4. Prediction curve of detail part cd2 (*) and real fouling cure (-)
Fig. 5. Prediction curve of detail part cd3 (*) and real fouling cure (-)
43
44
L. Sun, R. Saqi, and H. Xie
Fig. 6. Prediction fouling curve (-) and real fouling cure (*)
Fig. 7. The contrastive curve with 95% confidence interval
Fig. 8. Relative error curve
Through simulation, we can obtain that wavelet theory was used two parts, wavelet kernel functions and wavelet transform respectively. As a result, the precision of wavelet relevance vector is higher than standard relevance vector machine, shown as table 1. So the method reported in this paper plays an important role in improving the precision of relevance vector machine in forecasting time series. Table 1. Performance analysis WRVM The number of RVs/SVs Mean relative error
10 0.000457
RVM
SVM
10
92
0.000465
0.000598
Research on the Fouling Prediction of Heat Exchanger
45
5 Conclusions A research on modeling and Predicting the fouling of heat exchanger based on WRVM was introduced in this work. Simulation analysis indicates that the result of prediction has high prediction precision. The main reason is that the wavelet transform was used to extract the trend series and detail series, and make models respectively. It can avoid the over fitting in short term fluctuate. This method took even short time and it has excellent ability and generalization performance and robustness under the noises background. Acknowledgments. Supported by National Basic Research Program of China (2007CB206904)and the Eleventh Five-Year Science & Technology Project of Jilin Province Office of Education ([2007]NO.53).
References [1] Yang, S.R., Xu, Z.M., Sun, L.F.: Heat Exchanger Fouling and Its Countermeasure, 2nd edn. Science Press, Beijing (2004) (in Chinese) [2] Xu, Z.M., Yang, S.R.: Approach to Costing the Utility Boiler Fouling in China. J. of the CSEE. 24, 196–200 (2004) (in Chinese) [3] Radhakrishnan, V.R., Ramasamy, M.: Heat Exchange Fouling Model and Preventive Maintenance Scheduling Tool. Applied Thermal Engineering 27, 2791–2802 (2007) [4] Sun, L.F., Cai, H.D., Zhang, Y.Y., Yang, S.R., Qin, Y.K.: Research on the Fouling Prediction of Heat exchanger Based on Wavelet Neural Network. In: 3rd IEEE International Conference On Cybernetics and Intelligent Systems, Chengdu, pp. 961–964 (2008) [5] Sun, L.F., Zhang, Y.Y., Zheng, X.P., Yang, S.R., Qin, Y.K.: Research on the Fouling Prediction of Heat exchanger Based on Support Vector Machine. In: 2008 International Conference on Intelligent Computation Technology and Automation, Changsha, pp. 240– 244 (2008) [6] Tipping, M.E.: The Relevance Vector Machine. In: Proc. of Advances in Neural Information Processing Systems. MIT Press, Cambridge (2000) [7] Nikolaev, N.: Sequential Relevance Vector Machine Learning from Time Series. In: Proceedings of International Joint Conference on Neural Networks, Montreal (2005) [8] Lexander, S., Richard, E.: Smooth Relevance Vector Machine: A Smoothness Prior Extension of the RVM. Machine Learning 68, 107–135 (2007) [9] Clodoaldo, A.M., Lima, A.L.V., Coelho, S.C.: Automatic EEG Signal Classification for Epilepsy Diagnosis with Relevance Vector Machines. Expert Systems with Application 36, 10054–10059 (2009) [10] Yang, S.R., Xu, Z.M., Sun, L.F.: Automatic On-line Monitor of Fouling Thermal Resistance and Its Application. J. of Engineering Thermophysics 22, 150–152 (2001) (in Chinese)
Electricity Price Forecasting Using Neural Networks Trained with Entropic Criterion Jianhua Zhang, Jingyue Wang, Rui Wang, and Guolian Hou Beijing Key Laboratory of Industrial Process Measurement & Control New Technology and System, North China Electric Power University, Beijing 102206, China
Abstract. Accurate electricity price forecasting is critical to market participants in wholesale electricity markets. The problem becomes more complex because the acquired data series are non-linear and non-Gaussian. In this paper, Multi Layer Perceptrons (MLP) trained with minimizing error entropy (MEE) algorithm is utilized to forecast electricity price. Compared with the conventional MLP with mean square error (MSE) criterion, the proposed approach can achieve better performance in simulated examples. Keywords: Electricity price forecasting, Neural network, Entropy.
1 Introduction Recently, a group of algorithms have been developed to analyze and forecast electricity prices with different aims and time horizons. Arciniegas and Rueda [1] used Takagi-Sugeno-Kang (TSK) fuzzy inference system in forecasting day-ahead prices in Ontario electricity market. Alicia and Antonio [2] introduced an input-output hidden markov model (IOHMM) by finding the probabilistic model of prices and market information, such as the loads, generation kinds and prices. Rodriguez and Anders [3] invented a hybrid forecast model by back propagation neural networks and neuro-fuzzy system for predicting hourly market clearing price in the energy market. In addition, support vector machine (SVM) was also applied into electricity price forecasting field in [4]. And the forecasting method using rough set theory based on collective concept dealing with uncertainty was also proposed in [5]. Among these different techniques of price forecasting, artificial neural networks has been applied by many researchers because of its ability to learn complex and non-linear relationships that are difficult to model with conventional techniques. The mean square error (MSE) has been almost exclusively employed to train ANN systems [6,7]. However, MSE which concentrates on second order statistics was under the linearity and Gaussian distribution assumptions. Most real problems are governed by nonlinear equations and most random phenomena are far from being normally distributed. In this context, the higher-order statistical behavior of the systems is required. Recently, minimizing error entropy (MEE) has been utilized in the training of some adaptive systems [8-10]. As such, minimization of error entropy is employed as a more robust criterion when MLP is utilized to predict electricity prices. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 47–53. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
48
J. Zhang et al.
2 MLP Trained with MEE Criterion A schematic representation of the MLP with MEE criterion is shown in Fig.1. The weights between different layers of MLP are updated by minimizing the entropy of the error e( k ) between the estimated output and the targets.
y ( n)
r ( n)
^
y ( n)
e(n)
Fig. 1. Training MLP using MEE criterion
In this work, the quadratic Renyi entropy Hˆ 2 (e) of error e( k ) between the desired and estimated output is employed to be the performance index, and it can be obtained from data samples {e1(k), e2 (k),..., eN (k)} by utilizing Parzen window with Gaussian kernel function.
⎡ 1 Hˆ 2 (e) = − log ⎢ 2 ⎣N where the kernel function
κσ ( x) =
N
⎤
N
∑∑ κσ (e (k ) − e (k )) ⎥ i
i =1 j =1
1 2πσ
j
⎦.
(1)
− x2
e
2σ 2
.
For the purpose of computing, the estimation of quadratic Renyi entropy at time k can be calculated by a sliding window of samples [8].
⎡1 Hˆ 2 (e) = − log ⎢ ⎣W where
V ( e) =
1 W
k -1
⎤
∑ κσ (e(i) − e(k )) ⎥⎦ = − log V (e)
(2)
i = k -W
k -1
∑ κσ (e(i) - e(k)) is named as information potential in [9], and
i = k -W
W is the width of sliding window.
Electricity Price Forecasting Using Neural Networks Trained with Entropic Criterion
49
In this work, a q − p − l MLP is used to forecast electricity prices. The weights can be updated by using the steepest descent algorithm as follows.
wk (n) = wk (n − 1) + η ⋅ where
∂V (e(n)) ∂wk
(3)
∂V (e(n)) is the gradient of the information potential with respect to the ∂wk
weights and η the learning rate. Once the weight updating rules are obtained, the MLP can be trained by sampled electricity prices series. The electricity prices can be predicted by generalizing the trained MLP.
3 Price Forecasting by MLP with MEE Criterion The factors that influence the electricity price were investigated in [11, 12] , for example, power demands, transmission and generation outages, system or regional reserves, bidding strategy and market power, system congestion, ancillary services and market rules and so on. The main factors can be extracted by correlation analysis and experience in order to improve the efficiency of forecasting. The correlation coefficient ρ is defined as follows: −
n
ρ=
−
∑ ( x − x)( y − y) i
i =1
i
−
n
−
n
∑ ( x − x) ∑ ( y − y ) 2
i
i =1
−
x= −
y=
i
i =1
1 n
n
∑x
i
(5)
1 n ∑ yi n i =1
(6)
i =1
−
where
(4) 2
−
xi , yi are the ith observations and x , y the corresponding mean value of n
observation series. The correlation coefficients between the physical variables and prediction variables are listed in Table 1 (The historical price and demand data are selected from a real power market. P and L represent the electricity price and load in different days i and different times t). The MLP used to forecast daily electricity prices is shown in Fig.2.
50
J. Zhang et al. Table 1. Correlation coefficients between physical variables and price
Variables P(iˈt-1) P(iˈt-2) P(iˈt-3) P(iˈt-4) P(i-1ˈt) P(i-2ˈt) P(i-7ˈt) P(i-1ˈt-1) P(i-1ˈt+1)
correlation with price P(iˈt) 0.9235 0.8396 0.7451 0.6224 0.8646 0.7078 0.9770 0.7343 0.6249
Variables L(iˈt-1) L(iˈt-2) L(iˈt-3) L(iˈt-4) L(i-1ˈt) L(i-2ˈt) L(i-7ˈt) L(iˈt) L(i-1ˈt-1)
correlation with price P(iˈt) 0.9217 0.8172 0.7420 0.6178 0.8723 0.7738 0.9711 0.9747 0.7248
Fig. 2. MLP applied for forecasting
Several error indicators are analyzed to evaluate the accuracy of the proposed approaches to forecasting electricity prices. They are the mean absolute percentage error (MAPE), absolute percentage error (APE) and sum squared error (SSE). The MAPE criterion is given by:
δ MAPE =
∧ 1 n Y − Y ( ∑ i i ) Y × 100% n i =1
Y=
(7)
1 n ∑ Yi n i =1
(8) ∧
where
n is the total number of hours, Yi is the actual price at hour i, Yi is the
forecasted price at that hour.
Electricity Price Forecasting Using Neural Networks Trained with Entropic Criterion
51
The absolute percentage error (APE) is defined as follows: ∧
δ APE = (Yi − Yi ) Yi ×100%
(9)
The SSE criterion is the sum of the n square differences between the forecasted and the actual ones. The SSE criterion is given by: n
∧
δ SSE = ∑ (Yi − Yi )2
(10)
i =1
4 Simulation and Analysis In order to test the validity of the proposed algorithm, a 9-21-1 MLP is constructed with the historical data of electricity price and demand in certain region from July 15 to August 15, and then the trained MLP is utilized to investigate the daily price for next day-August 16. In this simulation, the learning rate η = 0.9 , the width of sliding window W = 32 , the total number of hours n = 24 . The information potential during training phase is shown in Fig. 3, it can be seen that the entropy of error is minimized gradually. Compared with the conventional MLP (using MSE criterion), the forecasting electricity prices are shown in Fig. 4 and the corresponding APEs are shown in Fig.5. It is clear that APE with MEE criterion is almost smaller than that one with MSE. In addition, Table 2 shown the value of MAPE and SSE during the whole day, it can be seen that the proposed MLP with MEE criterion can obtain better forecasting than conventional MLP.
Fig. 3. Information potential of error
52
J. Zhang et al.
Fig. 4. Forecasting by MLP-MEE and MLP-MSE
Fig. 5. The APE of MLP-MEE and MLP-MSE Table 2. Statistical Analysis of Errors
Criteria Error MAPE SSE
MLP-MEE
MLP-MSE
5.0609% 7.8988
8.1125% 14.1206
Electricity Price Forecasting Using Neural Networks Trained with Entropic Criterion
53
5 Conclusions In this paper, a MLP with MEE criterion is utilized to predict the next 24 hour electricity prices in the real electricity market. The predicted price can be used for obtaining bidding strategies. Acknowledgments. This work is partially supported by National Natural Science Foundation of China under grants (No. 60674051, 60974029). These are gratefully acknowledged.
References 1. Arciniegas, A.I., Rueda, I.E.A.: Forecasting Short-term Power Prices in the Ontario Electricity Market (OEM) with a Fuzzy Logic Based Inference System. Utilities Policy 16, 39–48 (2008) 2. Gonzalez, A.M., Roque, A.M.S., Gonzalez, J.G.: Modeling and Forecasting Electricity Prices with Input/output Hidden Markov Models. IEEE Trans. Power Syst. 20, 13–24 (2005) 3. Rodriguez, C.P., Anders, G.J.: Energy Price Forecasting in the Ontario Competitive Power System Market. IEEE Trans. on Power Syst. 19, 366–374 (2004) 4. Sun, W., Lu, J.-C., Meng, M.: Application of Time Series Based SVM Model on Next-day Electricity Price Forecasting under Deregulated Power Market. In: Proc. 5th Int. Conf. Machine Learning and Cybernetics, Dalian, China, pp. 13–16 (2006) 5. Xiao, Z., Ye, S.J., Zhong, B., Sun, C.X.: BP Neural Network with Rough Set for Short Term Load Forecasting. Expert Systems with Applications 36, 273–279 (2009) 6. Szkuta, B.R., Sanabria, L.A., Dillon, T.S.: Electricity Price Short-term Forecasting Using Artificial Neural Networks. IEEE Trans. Power Syst. 14, 851–857 (1999) 7. Mandal, P., Senjyu, T., Urasaki, N., Funabashi, T., Srivastava, A.K.: A Novel Approach to Forecast Electricity Price for PJM Using Neural Network and Similar Days Method. IEEE Trans. on Power Systems. 22, 2058–2065 (2007) 8. Erdogmus, D., Principe, J.C., Kim, S.P., Sanchez, J.C.: A Recursive Renyi’s Entropy Estimator. In: Proceedings of IEEE Neural Networks for Signal Processing, Martigny, Valais, Switzerland, pp. 209–217 (2002) 9. Erdogmus, D., Principe, J.C.: An Error–entropy Minimization Algorithm for Supervised Training of Nonlinear Adaptive Systems. IEEE Transactions on Signal Processing 50, 1780–1786 (2002) 10. Erdogmus, D., Principe, J.C.: Comparison of Entropy and Mean Square Error Criteria in Adaptive System Training Using Higher Order Statistics. In: Proc. Second International Workshop on Independent Component Analysis and Blind Signal Separation, Helsinki, Finland (2000) 11. Zhu, J.Z., Jordan, G., Ihara, S.: The Market for Spinning Reserve and Its Impacts on Energy Prices. In: Proc. IEEE Power Engineering Society Winter Meeting, Singapore, pp. 1202–1207 (2000) 12. Kian, A., Keyhani, A.: Stochastic Price Modeling of Electricity in Deregulated Energy Markets. In: Proc. 34th Hawaii Int. Conf. on System Sciences, Hawaii, USA, pp. 832–838 (2001)
Least Square Support Vector Machine Ensemble for Daily Rainfall Forecasting Based on Linear and Nonlinear Regression Jiansheng Wu1 , Mingzhe Liu2 , and Long Jin3 1
Department of Mathematical and Computer Sciences, Liuzhou Teachers College, Liuzhou, 545004, Guangxi, China
[email protected] 2 Institute of Information and Mathematical Sciences, Massey University, Albany, 0629, Auckland, New Zealand
[email protected] 3 Guangxi Climate Center, Naning, 530022, Guangxi, China
[email protected]
Abstract. Accurate forecasting of rainfall has been one of the most important issues in hydrological research. In this paper, a novel nonlinear regression ensemble model is proposed for rainfall forecasting. The model employs Least Square Support Vector Machine (LS-SVM) based on linear regression and nonlinear regression. Firstly, Projection Pursuit (PP) technology and Particle Swarm Optimization (PSO) algorithm are used to obtain the main factors of the rainfall, which optimize projection index from high dimensionality to a lower dimensional subspace. Secondly, using different linear regressions extract linear characteristics of the rainfall system, and using different Neural Network (NN) algorithms and different network architectures extract nonlinear characteristics of the rainfall system. Finally, LS-SVM regression is used for nonlinear ensemble model. This technique is implemented to forecast daily rainfall in Guangxi, China. Empirical results show that the prediction by using the LS-SVM ensemble model is generally better than those obtained using other models presented in this study in terms of the same evaluation measurements. The results suggest that our nonlinear ensemble model can be extended to meteorological applications in achieving greater forecasting accuracy and improving prediction quality. Keywords: Projection Pursuit, Particle Swarm Optimization, Neural Network, Least Square Support Vector Machine Ensemble.
1
Introduction
Rainfall prediction is a challenging task in the climate dynamics and climate prediction theory [1]. Accurate rainfall forecasting is one of the most important issues in hydrological research because early warnings of severe weather can help prevent casualties and damages caused by natural disasters [2,3]. With the Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 55–64. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com
56
J. Wu, M. Liu, and L. Jin
development of science and technology as well as intelligent computing in recent decades, many emerging techniques, such as Neural Networks (NNs), have been used in the rainfall forecasting. These efforts enhanced our understanding of intelligent computing and rainfall forecasting [4,5,6,7]. Short-term precipitation in the climate process is a very complex system, which it is not a simple linear or non-linear system. Therefore, it is difficult to effectively establish a model rainfall forecasting by the pure linear or pure nonlinear model. In other words, it is difficult to determine an effective predictor of the impact of weather systems due to the large dimension and noise in the weather system. The model will be complicated and the forecasting ability will reduce if there are many factors. Currently, modelling such a system focuses on: 1. how to extract sufficient feature information and how to reduce irrelevant information from the spatial data for the rainfall forecasting model; 2. how to set the effective forecasting rainfall modeling. In this paper, a novel method is presented for daily rainfall forecasting. This approach combines Least Square Support Vector Machine (LS–SVM), linear and nonlinear regressions. The Projection Pursuit technology and Particle Swarm Optimization (PP–PSO) are used to select the rainfall factors. Three linear regressions extract linear features and three NNs extract nonlinear features. The rainfall data in Guangxi is used as a case study for the development of daily rainfall forecasting model. The rest of the paper is organized as follows. Section 2 describes the model building process. For further illustration, this work employs the method to set up a prediction model for daily rainfall forecasting in Section 3. Finally, some concluding remarks are drawn in Section 4.
2
The Building Process of the LS-SVM Ensemble Model
Once data are collected, determining variables for modelling becomes possible. A large number of meteorological factors depends each others, and too much variables can enlarge the model scale, increase the complexity of the model. Therefore, the best way is to identify the variables that will save modelling time and reduces the problem space. Other reasons for the difficulties in forecasting rainfall are, such as the complexity of atmosphere-ocean interactions and the uncertainty of the relationship between rainfall and hydro-meteorological variables. The characteristic of rainfall is complex system dynamics under the influence of different meteorological factors. 2.1
Feature Extraction Used by PP–PSO
In the late 1970s, the international statistical society has developed a class of new statistical method to deal with and analysis high-dimensional data, which is called Projection Pursuit (PP) [8], which is suitable for dealing with nonlinear, non-normal distribution data to avoid “dimension disaster”, and it has succeeded in practice in many areas [9]. The basic idea of the method lies in: that
LS–SVM Ensemble for Daily Rainfall Forecasting
57
the computer technology is used to project high-dimensional data into the low dimensional subspace by numerically maximizing a certain objection function or projection index, which can extract the original data structure or characteristics, so as to achieve the goal of the study and analysis of high dimensional data. In rainfall system, X={xij , i = 1, 2, · · · , m, j = 1, 2, · · · , n} are m-dimensional factor of weather,where n is the number of the sample. A linear projection from Rm to Rp ,The matrix of the projectional direction is described as follow A = {ast > 0, s = 1, 2, · · · , p, t = 1, 2, · · · , m}
(1)
where p(p < m) is the number of new dimension for the factor matrix, and the row vector of A are orthogonal to each other. Then the new factor matrix has the form Z = A ∗ X = {zsj , i = 1, 2, · · · , p, j = 1, 2, · · · , n} (2) PP searches for a projection A maximizing a certain objection function. While synthesizing the projection value, the dispersal characteristic of projection value Z requests to be: the partial projection point should be as intensive as possible, it can be best to be some point zone, and in the total the point zone should be as wide as possible. The objective function can be written as max Q(a) = S(a)D(a) (3) s.t. A · AT = 1 Because of the complex nonlinear optimization, it is very difficult to solve the solution projectional direction by the traditional method. In order to extract the information, reduce dimension and decrease noise from the original variables, the following optimization models are used to find the solution projectional direction parameters with PSO. The fitness function of PSO is defined as follows: F itness = Q(a)
(4)
The results can be solved by a standard PSO algorithm for Equation 4. 2.2
Extraction of Linear Features by Three Traditional Linear Regression
Perhaps the most popular mathematical model for making predictions is the multiple linear regression model in the traditional mathematical method, which estimates one random variables using known values of multiple variables. There is a continuous random variable called the dependent variable Y , and a number of independent variables, x1 , x2 , · · · , xp . The goal of regression models is to fit a set of data with an equation, the simplest being a linear equation. The linear regression model is given by yˆ = β0 + β1 x1 + β2 x2 + · · · + βn xp + ε
(5)
where ε , the “noise”variable, is a normally distributed random variable with mean equal to zero and standard deviation σ, {βt , t = 1, 2, · · · , p} are the values
58
J. Wu, M. Liu, and L. Jin
of the coefficients. Unknown parameters values should be estimated from the smallest error sum of squares of the samples. Three types of linear regression models have been made in this paper for parsimony variables, such as stepwise multiple linear (SML) regression, partial least squares (PLS)regression and multi recursive regression method (MRR) to capture the linear characteristics of rainfall system. More detailed introductions to the three linear regression model are referred to the related literature [10,11,12]. 2.3
Extraction of Nonlinear Features by Three NNs Methods
NNs provide an interesting technique that theoretically can approximate any nonlinear continuous function on a compact domain to any designed of accuracy. The network learns by adjusting the interconnections (called weights) among layers. When the network is adequately trained, it is able to generalize relevant output for a set of input data. A valuable property of neural networks is that of generalization, whereby a trained neural network is able to provide a correct matching in the form of output data for a set of previously unseen input data. Learning typically occurs by example through training, where the training algorithm iteratively adjusts the connection weights (synapses). With the work about bias-variance trade-off of Bretiman [13], an NNs regression model consisting of diverse models with much disagreement is more likely to have a good generalization [14]. Therefore, how to generate diverse models is a crucial factor. In this paper, there are three methods for generating diverse models, such as multi–layer perceptions based on the Levenberg–Marquard algorithms (MLP–NN), the radial basis function network network (RBF–NN) and General Regression Neural Network (GR–NN) for nonlinear regression model. Readers interested in a more detailed introduction about these neural network algorithms are referred to the related literature [15]. 2.4
Nonlinear Ensemble Based on LS-SVM
LS-SVM is a least squares modification to the Support Vector Machine [16]. The major advantage of LS-SVM is that it is computationally very cheap while it still possesses some important properties of the SVM. In this section we will briefly discuss the LS-SVM method for a regression task. For more detailed information see [16]. We let {xi , i = 1, 2, · · · , N } be the output of linear and nonlinear forecasting predictors, {yi , i = 1, 2, · · · , N } be the aggregated output and the goal is to estimate a regression function f . Basically we define a N -dimensional function space by defining the mapping ϕ = [ϕ1 , ϕ2 , · · · , ϕN ]T according to the measured ˆ = ω T ϕ(x) + b where ω is a weight points. The LS-SVM model is of the form f (x) vector and b is a bias term. The optimization problem is the following: ⎧ N ⎨ min J(ω, ) = 12 ω T ω + γ 12 2i (6) i=1 ⎩ s.t. yi = ω T ϕ(xi ) + b + i , i = 1, 2, · · · , N
LS–SVM Ensemble for Daily Rainfall Forecasting
59
where the fitting error is denoted by i . Hyper-parameter γ controls the tradeoff between the smoothness of the function and the accuracy of the fitting. This optimization problem leads to a solution: ˆ = f (x)
N
αi K(x, xi ) + b
(7)
i=1
where αi is for the coefficients and K(x, xi ) = ϕT (x)ϕ(xi ) is the kernel. A common choice for the kernel is the Gaussian RBF: K(x, xi ) = e
x−xi 2 2σ2
(8)
The above-mentioned method can be summed up as follows: firstly, this paper is to extract the feature of the original meteorological factors from high dimension space to lower dimension subspace,and construct a forecasting variable by PSO algorithm optimization projection index. Secondly, three traditional linear regression methods are used to capture rainfall series linear characteristics. Thirdly, three different NNs models is trained to capture nonlinear patterns. Finally, the LS-SVM method are used for ensemble, which incorporate Linear model with Nonlinear model in an adaptive manner(LS–SVM–LNE). The basic flow diagram can be shown in Fig.1.
Randomly initialize population position and velocities
Initial value of projection directions
Calculate fitnessvalure by the original datd sets
Data sets (the main factor of weather) Yes
S atis fy s toping criteria?
Generate new factors of rainfall
Stepwise Muliple Regression Partial Leas t S quares Regres s ion
Multi Recursive Regression Levenberg-Marquard Neural Network
LS-SVM Ensemble
No Update particle best Update global best
Radial Basis Function Network Network
Update particle velocity Update particle position
General Regression Neural Network
Fig. 1. A Flow Diagram of The Proposed Nonlinear Ensemble Forecasting Model
3
Experiments Analysis
In this paper, PSO parameters are set as follows: the iteration times are 100; the population is 100; the minimum inertia weight is 0.1; the maximum inertia weight is 0.9 and the learning rate is 2.0. NNs parameters are set as follows: the number of neurons in the hidden layer range from 3. The learning rate is 0.9; the momentum factor is 0.7; the iteration times are 1000; the global error is 0.001.
60
3.1
J. Wu, M. Liu, and L. Jin
Empirical Data
Real–time ground rainfall data have been obtained in June from 2003 to 2008 in Guangxi by observing 89 stations, which 144 samples are modeled from 2003 to 2007 in June, other 30 samples are tested modeling in June of 2008. Method of modeling is one-step ahead prediction, that is, the forecast is only one sample each time and the training samples is an additional one each time on the base of the previous training. Due to the complex terrain of Guangxi and inhomogeneous rainfall, the region has been divided into three regional precipitation based on historical precipitation data by the cluster analysis method to reduce the difficulty of forecasting. Statistics for each district in the average daily precipitation is used as the forecasting object. Fig.2 shows three region maps.
Fig. 2. The Group Average Region Map of Guangxi Rainfall
In the first region as an example to show the process of modeling, it is very important to select the appropriate forecasting factors for model, different climatic variability and its effect have been discussed many times in the literature. In this paper, first of all, the candidate forecasting factors are selected from the numerical forecast products based on 48h forecast field, which includes:(1) the 17 conventional meteorological elements and physical elements from the T213 numerical products of China Meteorological Administration, the data cover the latitude from 15◦ N to 30◦ N, and longitude from 100◦ E to 120◦ E, with 1◦ ×1◦ resolution, altogether there are 336 grid points.(2) the fine-mesh precipitation data from the East Asia of Japanese Meteorological Agency, the data cover the latitude from 15◦ N to 30◦ N, and longitude from 100◦E to 120◦ E, with 1.25◦ × 1.25◦ resolution, altogether there are 221 grid points. We can get 60 variables as the main forecasting factors. Finally, a new information matrix can be extract form 60 variables by PP– PSO so we get 3 variables as the predictors in the first region. Fig.3 shows the data–sets of the main impact factors, which 60 variables and 144 samples. Fig.4 shows the data–sets of the predictors including 3 variables and 144 samples by PP–PSO extraction.
LS–SVM Ensemble for Daily Rainfall Forecasting
61
1
1
0.5
0.5
0
0
−0.5 −0.5
−1 1 1
0.5 0.5
0 0
−0.5
0.5 0
−0.5 −1
−1
−1
Fig. 3. The Main Factors Data–sets
3.2
−1 1
−0.5
0
0.5
1
Fig. 4. The Predictors Data–sets
The Performance Evaluation of the Model
In order to measure effectiveness of the proposed method, five types of errors, such as, the Maximum Absolute Error (MAE), the Root Mean Squares Error(RMSE), the Pearson Relative Coefficient (PRC), the Error Value more than 25mm (F1 ) and the Error Value less than 5mm (F2 ) which have been found in many papers, are also used here. Interested readers can be referred to [4] for more details. For the purpose of comparison, we have also built three other ensemble forecasting models: (1) Simple Average Ensemble (SAE-LE) - all the available linear forecasting output; 1 Tˆ1 = (y1 + y2 + y3 ) (9) 3 (2) Simple Average Ensemble (SAE-NE) - all the available nonlinear forecasting output; 1 Tˆ2 = (y4 + y5 + y6 ) (10) 3 (3) Simple Average Ensemble (SAE-LNE) - all the available linear and nonlinear forecasting output; 1 Tˆ3 = (y1 + y2 + y3 + y4 + y5 + y6 ) 6 3.3
(11)
Analysis of the Results
Fig.5. shows the fitting results of all model for 144 training samples, we can see that the fitting results of SL-SVM-LNE model are best in all models, the SAE–LE can basically fit the historical precipitation trends,the fitting results of the SAE-NE and the SAE-LNE are in good agreement with the actual values. The more important factor to measure performance of a method is to check its forecasting ability.
62
J. Wu, M. Liu, and L. Jin 80
140
Rainfall(mm)
Rainfall(mm)
Actural Rainfall SAE−LE Fitting SAE−NE Fitting SAE−LNE Fitting LS−SVM−LNE Fitting
70
60
Actural Rainfall SAE−LE Forecasting SAE−NE Forecasting SAE−LNE Forecasting LS−SVM−LNE Forecasting T213 Forecasting
120
100
50 80
40 60
30 40
20
20
10
0
0
20
40
60
80
100
120
140
5
10
15
20
Day
Fig. 5. Fitting of 144 Samples in June
25
30 Day
Fig. 6. Forecasting of 30 Samples in June
From Table 1, for the rainfall testing case in June, the MAE and RMSE of the LS–SVM–LNE model is also the smallest in all model. The minimum values of MAE and RMSE indicate that the deviations between original values and forecast values are very small. At the same time, from Table 1, the PRC of the LS–SVM–LNE model is the highest in all model. The accurate efficiency of the proposed model is measured as PRC, The higher values of PRC (maximum value is 1) indicate that the forecasting performance of the LS–SVM–LNE model is effective, which can capture the average change tendency of the cumulative rainfall data. The effective information of the proposed rainfall forecasting model is measured by F1 and F2 . If the Error Value is more than 15mm between the original value and the forecast value, the model predictions don’t provide effective information, if the Error Value is less than 5mm, the model predictions provide a piece of very important effective information for the future forecasting. From Table 1, the LS–SVM-LNE model has no unreliable information. Similarly, the LS–SVM-LNE model has reference information at least 86.67%. These results indicate that the proposed model can provide more useful information, avoid invalid information for the future forecasting. Table 1. A Comparison of Result of Ensemble Models about 30 Testing Samples Results Errors MAE RMSE PRC F1 (F1 /30) · 100% F2 (F2 /30) · 100%
SAE–LE
SAE–NE
SAE–LNE
LS–SVM–LNE
T213
79.4627 24.5914 0.5728 12 40.00% 9 30.00%
68.4590 17.7559 0.7543 6 20.00% 13 43.33%
73.7623 18.5157 0.7062 8 26.67% 16 53.33%
49.8746 13.5746 0.8957 0 0.00% 26 86.67%
62.7980 35.4308 0.0129 11 36.67% 10 33.33%
LS–SVM Ensemble for Daily Rainfall Forecasting
63
Furthermore, the same method is used to train precipitation data and predict precipitation at the other two regions. The experimental results also show that LS–SVM–LNE method is better than other methods. From the experiments presented in this paper, the LS–SVM ensemble forecasting model is superior to the traditional linear regression model, the individual NNs model as well as the linear ensemble models for the test cases of daily rainfall.
4
Conclusions
Accurate rainfall forecasting is crucial for a frequent unanticipated flash flood region to avoid life losing and economic loses. This paper proposes a LS–SVM ensemble forecasting model that combines three traditional linear regression model and three different NNs model to predict daily rainfall based on PP–PSO. In terms of the different forecasting models, empirical results show that the nonlinear ensemble model performs the best for the test cases of three regions Guangxi, China on the basis of different criteria. In the LS–SVM ensemble model test cases, the MAE and RMSE are the lowest and the PRC is the highest. This indicates that the LS–SVM ensemble forecasting model can be used as an alternative tool for daily rainfall forecasting to obtain greater forecasting accuracy and improve the prediction quality further in view of empirical results, and can provide more useful information, avoid invalid information for the future forecasting.
Acknowledgment This work was supported in part by the Key Natural Science Foundation of Guangxi under Grant No.0832019Z, and in part by the Department of Guangxi Education under Grant No. 200807MS098.
References 1. Jiangsheng, W., Long, J.: Forecast Research and Applying of BP Neural Network Based on Genetic Algorithms. Mathematics in Practice and Theory 35(1), 83–88 (2005) 2. Jiansheng, W., Long, J., Mingzhe, L.: Modeling Meteorological Prediction Using Particle Swarm Optimization and Neural Network Ensemble. In: Wang, J., Yi, Z., ˙ Zurada, J.M., Lu, B.-L., Yin, H. (eds.) ISNN 2006. LNCS, vol. 3973, pp. 1202–1209. Springer, Heidelberg (2006) 3. Nasseri, M., Asghari, K., Abedini, M.J.: Optimized Scenario for Rainfall Forecasting Using Genetic Algorithm Coupled with Artificial Neural Network. Expert Systems with Application 35, 1414–1421 (2008) 4. Jiansheng, W., Enhong, C.: A Novel Nonparametric Regression Ensemble for Rainfall Forecasting Using Particle Swarm Optimization Technique Coupled with Artificial Neural Network. In: Yu, W., He, H., Zhang, N. (eds.) ISNN 2009. LNCS, vol. 5553, pp. 49–58. Springer, Heidelberg (2009)
64
J. Wu, M. Liu, and L. Jin
5. Yingni, J.: Prediction of Monthly Mean Daily Diffuse Solar Radiation Using Artificial Neural Networks and Comparison with other Empirical Models. Energy Policy 36, 3833–3837 (2008) 6. Govindaraju, R.S.: Artificial Neural Network in Hydrology, I: Preliminary Concepts. Journal of Hydrologic Engineering 5(2), 115–123 (2000) 7. French, M.N., Krajewski, W.F., Cuykendal, R.R.: Rainfall Forecasting in Space and Time Using a Neural Network. Journal of Hydrology 137, 1–37 (1992) 8. Friedman., J.H., Turkey, J.W.: A Projection Pursuit Algorithm for Exploratory Data Analysis. IEEE Transaction On Computers 3(9), 881–889 (1974) 9. Filzmoser, P., Serneels, S., Croux, C., Van Espen, P.J.: Robust Multivariate Methods: The Projection Pursuit Approach. In: Spiliopoulou, M., Kruse, R., N¨ urnberger, A., Borgelt, C., Gaul, W. (eds.) From Data and Information Analysis to Knowledge Engineering Proceedings of the 29th Annual Conference of the Gesellschaft f¨ ur Klassifikation e.V., University of Magdeburg, vol. 8, pp. 270–277. Springer, Heidelberg (2006) 10. Johenson., D.: Applied Ultivariate Methods for Data Analysts, 2nd edn. Thomson Learning Press, UK (1998) 11. Helland, I.S.: PLS Regression and Statistical Models. Scandivian Journal of Statistics 179, 97–114 (1990) 12. Kenneth, D.S.: Using recursive regression to explore nonlinear relationships and interactions: A tutorial applied to a multicultural education study. Practical Assessment Research and Evaluation 14(3), 1–13 (2009) (Retrieved March 3, 2009) http://pareonline.net/getvn.asp 13. Jiansheng, W.: A Novel Artificial Neural Network Ensemble Model Based on K–nn Nonparametric Estimation of Regression Function and Its Application for Rainfall Forecasting. In: 2nd Internatioal Joint Conference on Computational Sciences and Optimization, pp. 44–48. IEEE Computer Society Press, New York (2009) 14. Benediktsson, J.A., Rveinsson, J., Ersoy, O.K., Swain, P.H.: Parallel Consensual Neural Neural Networks. IEEE Transactions on Neural Networks 8, 54–64 (1997) 15. Fredric, M.H., Ivica, K.: Principles of Neurocomputing for Science & Engineering. McGraw-Hill, New York (2001) 16. Suykens, J., Van Gestel, T., De Brabanter, J., De Moor, B., Vandewalle, J.: Least Squares Support Vector Machines. World Scientific Publishing Press, Singapore (2002)
Estimating Portfolio Risk Using GARCH-EVT-Copula Model: An Empirical Study on Exchange Rate Market Zongrun Wang1, Yanbo Jin2, and Yanju Zhou1 1
2
School of Business, Central South University, Changsha 410083, China Department of Finance, California State University, Northridge 91325, USA
Abstract. This paper introduces GARCH-EVT-Copula model and applies it to study the portfolio risk of exchange rates. Multivariate Copulas including Gaussian Copula, t Copula and Clayton Copula were used to describe the structure and extend the analysis from bivariate to any n-dimension. We apply this methodology to study the returns of a portfolio of four major foreign currencies in China. Our results suggest that the optimal investment allocations are similar across different Copula and confidence levels and the optimal investment concentrates in the USD investment. Generally speaking, t Copula and Clayton Copula can better portray the correlation structure of multiple assets than Normal Copula. Keywords: GARCH-EVT-Copula model, Exchange rate, Portfolio risk, VaR and CVaR.
1 Introduction Value-at-Risk (VaR) is one of the most important measures of the market risk and Conditional VaR (CVaR) has emerged as a supplement to VaR. It provides a more conservative measure of losses relative to VaR. Extreme value theory (EVT) focuses directly on the tails and could therefore potentially give us better estimates and forecasts of risk. But applying EVT to the return series is inappropriate as they are not independent and identically distributed. Thus following the approach of McNeil and Frey [1], we use GARCH model to fit the return series and apply EVT to the residuals rather than to the return series. Several researchers home and abroad have made great progress using this method to measure VaR, such as Hans N. E. Bystrom [2], Malay Bhattacharyya [3], Xiao [4] and so on. On the other hand, when measuring the risk of a multi-asset portfolio, one needs to model the relation between the risk factors. This has become an active line of research in the finance field in recent years. Copulas are widely used along this line of research, because it is able to extract the dependence structure from the joint probability distribution function and, simultaneously, to isolate such dependence structure from the univariate marginal distributions. However, most empirical studies have focused on the equity risk using Gaussian Copula and t Copula. Among the few studies that did focus on the exchange rate risk. The Archimedean Copula is rarely used because of implementation issues and the Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 65–72. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
66
Z. Wang, Y. Jin, and Y. Zhou
analysis was limited to two-dimension [5,6]. Tursunalieva [7] extends [5,6] and shows that the SJC-Copula captures the tail dependence between the exchange rates in pre-euro and post-transition periods, while the rotated Gumbel Copula captures the dependence during the transition period. In China, Wu [8] was the first to analyze two-asset portfolio of foreign exchanges using Archimedean Copula. Sun [9] considered the EUR/RMB and JPY/RMB exchange rates, using variety of Archimedean Copula to fit the data and derived the optimal Copula to capture the two variable’s correlation. Wang [10] uses EVT to measure the exchange rate risk and builds Copula-EVT model and calculates the VaR for the exchange rate portfolio of EUR/CNY and JPY/CNY by Monte Carlo simulation. On the other hand, some researchers combine GARCH model and Copula to dynamically study the correlation and risk of financial assets, such as Jondeau [11,12], Tae-Hwy lee [13], Wu[14] and so on. In this paper, we introduce Clayton Copula to apply to the exchange rate portfolio. The contributions of our paper are two folds. First, in addition to Gaussian Copula and t Copula, we use the multivariate Archimedean Copula, in particular the Clayton Copula, to analyze the asymmetric dependence structure among financial asset returns. Second, we extend the bivariate copula methods in a trivariate context, and use GARCH nested copulas with EVT margins to estimate the risk of fourdimensional exchange rate portfolios.
2 GARCH-EVT Model and Copula Parameter Estimates 2.1 GARCH-EVT Model Recognizing the heteroskedasticity and volatility clustering nature of time series, we first use GARCH (1,1) to model the time series. The model is:
⎧⎪ rt = μ + ε t = μ + σ t zt ⎨ 2 2 ⎪⎩σ t = w + αε t −1 + βσ t −1
(1)
Where w > 0, α ≥ 0, β ≥ 0, α + β < 1 . rt is the actual return, μ is the expected return and σ t is the volatility of the returns on day t . zt comes through the stochastic variables, which are the residuals or the innovations of the process. Conventionally these residuals are assumed to follow a normal distribution. However, the conditional distribution of GARCH models has been shown to have a heavier tail than that of a normal distribution, which provides better approximation to actual financial time series. Furthermore, EVT appears to be an appropriate approach for modeling the tail behavior. However, applying EVT to the random variable rt is inappropriate as rt is not i.i.d. Therefore, our marginal model is built on GARCH-EVT model, in which we use GARCH model to fit the historical return data, then model the new series by the GPD distribution in the lower or upper tail and by the empirical distribution in the remaining part. The distribution of innovation is followed by (see details in Stuart Coles [15]):
Estimating Portfolio Risk Using GARCH-EVT-Copula Model −1/ ξ ⎧N ⎧ L ⎫ ⎪ u L ⎨1 + ξ L u − zt ⎬ zt < u L ⎪ N ⎩ βL ⎭ ⎪⎪ F ( zt ) = ⎨φ ( z ) u L < zt < u R ⎪ −1/ ξ R R ⎪ Nu R ⎧ R u − zt ⎫ zt > u R ⎬ ⎪1 − N ⎨1 + ξ R β ⎩ ⎭ ⎪⎩
67
L
(2)
Where β , ξ are the scale and shape parameter respectively, u R ( u L ) is the upper (lower) threshold. In order to get β , ξ exactly, we must choose the proper threshold. 2.2 Copula Parameter Estimates
The following theorem is known as Sklar’s Theorem. It is the most important theorem about copula functions because it is used in many practical applications. Theorem: Let F be an n-dimensional c.d.f. with continuous margins F1 , F2 , it has the following unique copula representation:
xn ) = C ( F1 ( x1 ), F2 ( x2 ),
F ( x1 , x2 ,
Fn ( xn ))
Fn .Then
(3)
We can see that, for continued multivariate distribution, the univariate marginal distribution and the multivariate dependence structure can be separated. Assuming that the dependence structure does not change with time, we select Gaussian Copula, t Copula and Clayton Copula to estimate the joint distribution. For n-variable Gaussian Copula, Let φ∑ be the standardized multivariate normal distribution with correlation matrix ∑ . It can be defined as follows:
C Gu (u1t , u2t ,
, unt ) = φ∑ (φ −1 (u1t ), φ −1 (u2t ),
, φ −1 (unt ))
(4)
Where φ −1 denotes the inverse of the normal cumulative distribution function. From (2) we know the marginal distribution is Fi ( z ) . Based on the historic data {z1t , z2t znt }, t = 1, 2 T ,we set: ut = (u1t , u2t
unt ) = ( F1 ( z1t ), F2 ( z2t )
ξt = (φ (u1t ), φ (u2t ), φ (unt )) −1
−1
−1
Fn ( znt ))
(5)
So we can get the simple expression: C Gu (ut ) = φ∑ (ξt ) . And using MLE method, ∑ is estimated as follows: T ˆ = 1 ξξ ' ∑ ∑ t t T t =1
(6)
To capture the fat tail property, we introduce the multivariate student’s t Copula which shows more observations in the tails than the Gaussian. Let tv , ∑ be the
68
Z. Wang, Y. Jin, and Y. Zhou
standardized multivariate t distribution with correlation matrix ∑ and v degrees of freedom. The t Copula can be defined as follows: C t (u1t , u2t ,
−1
−1
unt ) = tv , ∑ (tv (u1t ), tv (u2t ),
−1
(7)
tv (unt ))
−1
Where tv denotes the inverse of the Student’s t cumulative distribution function. Based on the historic data and the given degree of freedom v , we set: ut = (u1t , u2t −1
unt ) = ( F1 ( z1t ), F2 ( z2t )
ζ t = (tv (u1t ), t (u2t ), −1 v
Fn ( znt ))
(8)
−1 v
t (unt ))
So we have C t (ut ) = tv , ∑ (ζ t ) . The parameter matrix ∑ is also estimated using MLE method. The initial matrix is the correlation coefficient matrix of multivariate normal ˆ Copula function made from (6). We can get the correlation coefficient matrix ∑ of n +1
multivariate t Copula through the following iterative calculation method:
ζ tζ t ' 1 ⎛v+n⎞ T ˆ k = 1, 2, ∑ k +1 = ⎜ ⎟⋅∑ T ⎝ v ⎠ t =1 1 + 1 ζ ' ∑ ˆ ζ t k t v
(9)
ˆ =∑ ˆ . In addition to the fat tails, we can try to Repeat the above process until ∑ n +1 n check whether asymmetry exists in the tails. We use the Clayton Copulas, one of the Archimedean families better known in capturing left tail dependence. The Clayton Copula’s cumulative distribution functions are defined by the following:
C clayton (u1 , u2 ,
, un ) = (u1−θ + u2 −θ +
+ un −θ − n + 1)
−
1
θ
,θ ≥ 0
(10)
Base on the historic data and (2), we have ui = Fi ( zit ) and can directly estimate the parameter θ of n-variable Clayton Copula using MLE method.
3 Simulation Algorithm and Portfolio Risk Analysis 3.1 Simulation Algorithm
ˆ First, for ∑
derived from (6), we do Cholesky decomposition and then
have ∑ = A ' A . Second, draw independent n-dimensional vectors from the multiple normal distribution x = ( x1 , x2 ,
, xn ) ' , xi ∼ N (0,1) , set y = A ' x ,then
z = ( F (φ ( y1 )), F2−1 (φ ( y2 )),
Where Fi −1
, i = 1, 2,
−1 1
, F1−1 (φ ( yn )))
(11)
, n is the inverse of distribution of Fi in (2). Third, repeat the
above step and simulate M times, we can get vector ( z1m , z2 m , , znm ) ' m = 1, 2 M . Then restoring it into (1) we can get M returns at the time t+1. The returns residuals’
Estimating Portfolio Risk Using GARCH-EVT-Copula Model
69
joint distribution is this Gaussian Copula. The returns can be defined by rt +1 = (r1m , r2 m , , rnm ) ' = ( μ1 + z1mσ 1,t +1 , μ 2 + z2 mσ 2,t +1 , , μ n + znmσ n ,t +1 ) ' , Where σ i ,T +1 , μi , i = 1, 2,
, n are calculated by GARCH (1,1) model. ˆ derived from (9) we firstly do Cholesky decomposition. On the other hand, for ∑ After that, generate n random vectors which are i.i.d, then produce random vector s subjecting to Chi-square distribution and independent from x , and set: z = ( F1−1 (t (
,
y1 s/v
y2
)), F2−1 (φ (t (
s/v
))),
, Fi −1 (φ (
yi s/v
)))
(12)
Where Fi −1 i = 1, 2, , n is the inverse of distribution of Fi in (2). Finally, repeat the above step M times, we can also get the vector and the simulating returns via t Copula at the time t+1. A hurdle for practical implementation of any multivariate Archimedean Copula was the absence of an efficient method to generate returns. Marshall and Olkin proposed an alternative method, which is computationally straightforward than the conditional distribution approach. It requires the generation of an additional variable. 1 1 ˆ −1 = t − θ − 1 , Considering the inverse function of Gamma( ,1) ’s LT transformation Ga
θ
it is different from the generator φ (t ) =
1
θ
(t −θ − 1) of Clayton Copula just a constant
and it does not affect generating Clayton Copula. So the algorithms of simulating returns via Clayton Copula are as follows: First, Generate M random variables Ym , m = 1, 2 M which obey the function 1 Gamma( ,1) , where the parameter θ is estimated using MLE. Second, simulate
θ
independent
uniform
random
variables y1 , y2 ,
, yn .Third,
Let uim = Ga(
− ln( yi ) ) , i = 1, 2, Ym
u = (u1 , u2 ,
, un ) is the n-dimension Clayton Copula with the parameter θ , then
, n, m = 1,
M , then the joint distribution of
−1
zim = Fi (uim ), i = 1, 2, , n, m = 1, 2, , M . Finally, similar to the last step of simulating via Gaussian Copula or t Copula above, we can also get M returns via Clayton Copula at the time t+1. 3.2 Portfolio Risk Analysis 3.2.1 VaR and CVaR of the Equally-Weighted Portfolio As we know, the above algorithms have simulated the returns at time t+1 of ndimension time series using the three Copula functions to describe the correlation structure. Now suppose we want to compute the empirical VaR and CVaR of an equally-weighted portfolio. Assuming there are n-dimension assets we can invest and
70
Z. Wang, Y. Jin, and Y. Zhou
1 1 1 the weight in each asset is the same, that is w = [ , , , ]' , according to the M n n n returns samples simulated above, we can get M total returns of the portfolio: Re turns = rt +1 * w = (r1m , r2 m , , rnm ) * w . As a result, we can get the empirical distribution and thus the VaR and CVaR of the equally-weighted portfolio. 3.2.2 Optimal Portfolio with Minimum Risk The discussion above focused on estimating the VaR and CVaR of an equallyweighted portfolio. However, for individual investors, one of the major concerns is to minimize risk of the investment portfolio. As a result, the following algorithm computes the optimal weights of each asset which minimizes the portfolio risk. Assuming that the weight in individual assets within a portfolio is w = [ w1 , w2 , , wn ]' ,
where 0 ≤ wi ≤ 1 and w1 + w2 + + wn = 1 . Needless to say, there are many weight vectors that satisfy these conditions. For each given investment weight vector w , we can get M returns of the portfolio; that is, Re turns = rt +1 * w = (r1n , r2 n , , rnm ) * w and compute VaR of portfolio under a given confidence. After getting all VaR values corresponding to all the weight vectors, we can find the minimum VaR and return back to get the weight in each asset; that is, min VaRα = risk ( w1 , w2 , , wn ) , also we can get the CVaR value under this given investment proportion.
4 Empirical Studies in Foreign Exchange Market We choose four foreign exchange rates including USD/CNY, EUR/CNY, JPY/CNY, HKD/CNY to demonstrate the application of GARCH-EVT-Coupla model. As we know, China has started implementing the new exchange rate policy since July 21,2005. The data series used here are from July 25, 2005 to July 25, 2008.1. We use the daily logarithmic returns defined as rt = ln Pt − ln Pt −1 . The data series give us 732 observations for each exchange rate. 4.1 Summary Statistics
The descriptive statistics of daily logarithmic returns show that all series have heavy tails and they do not follow normal distribution. Take USD/CNY as an example, the ADF test result of USD/CNY indicates that there is no unit root and that the series follow a stationary stochastic process. Then we perform ARCH LM test to find that USD/CNY is a stationary stochastic process, but with heteroskedasticity. And the analysis on the other three exchange rates shows similar properties. As a result, we can use GARCH (1,1) model as in (1) to study the exchange rate movement. 4.2 GARCH-EVT-Copula Application
We first establish GARCH-EVT model to the each RMB exchange rate series and get the marginal distribution of residuals. Second use Multivariate Gaussian Copula, t 1
Data source: http://www.safe.gov.cn/model_safe/index.html
Estimating Portfolio Risk Using GARCH-EVT-Copula Model
71
Copula and Clayton Copula to describe the correlation of the residuals and obtain the Copula parameters. Finally adopt Monte Carlo to simulate returns considering the correlation structure, calculate the risk value under the same investment proportion and find the optimal investment proportion under the given confidence. As described in earlier sections, we can determine the threshold μ . The other parameters can be obtained using the method of MLE. We find that GPD fits the residuals of all series perfectly. After obtaining the parameters of GPD and the residuals i = 1, 2,3, 4, t = 1, 2 T , we substitute zit into (2) and get the marginal distribution ui = f ( zi ) . According to the parameter estimation method in the section 2.1 we can get the parameters of Copulas. That is the correlation matrix ∑ of Gaussian Copula, ∑ and freedom degree v of t Copula, the parameter θ of Clayton Copula. Applying the simulation algorithm described in section 2.2 above, we can simulate the returns at the time t+1 considering correlation structure using the three types of Copula. When the investment proportion of the four foreign exchanges were equal we can calculate VaR and CVaR of the portfolio. In addition, we can find the optimal investment proportion to minimize the portfolio risk. The results indicate that under the same confidence level, VaR(CVaR) calculated from t Copula and Clayton Copula is less than that from Gaussian Copula. That is because t Copula and Clayton Copula consider the tail correlation but the Gaussian Copula does not. In addition, the optimal portfolio weights under minimum portfolio risk are similar across different Copula and confidence levels. The investment concentrates on USD, although the price of USD/CNY has decreased since 2007 and the volatility has increased. However, the volatility of USD/CNY is still smaller than that of EUR/CNY and JPY/CNY, as a result, during the sample period of July 2005-July 2008, USD investment accounted for the largest proportion in the optimal investment under minimum portfolio risk. With higher confidence level, the proportion of USD investment becomes smaller and the proportions of the other foreign currencies become larger. When α = 0.9 , VaR using Clayton Copula is larger than those using t Copula and Gaussian Copula. However, with higher confidence level, Clayton Copula better captures the correlations and results in VaR that is similar to the result using t Copula. As a result, under the higher confidence level, we can choose t Copula and Clayton Copula to calculate the portfolio risk.
,
5 Conclusions In this paper we introduce the GARCH-EVT-Copula model and use it to measure the risk of foreign exchange portfolio with multiple foreign exchanges. When modeling the correlation structure we not only apply the conventional Gaussian Copula and t Copula, but also the multivariate Clayton Copula which was rarely used because the complexity involved in programming. We find that for an equally-weighted foreign exchange portfolio, VaR(CVaR) calculated from t Copula and Clayton Copula is less than that computed from the Gaussian Copula. This is because t Copula and Clayton Copula model the tail correlation but the Gaussian Copula does not. On the other hand, when minimizing portfolio risk, optimal portfolio weights are similar across different copula and
72
Z. Wang, Y. Jin, and Y. Zhou
different confidence levels. The optimal investment tends to concentrate in the investment of USD. With higher confidence level, weight in USD decreases while weight in EUR and JPY increases. In addition, we show that t Copula and Clayton Copula provide better approximation to capture portfolio risk compared to Gaussian Copula. These findings can help investors better manage the exchange rate risk. Acknowledgments. The work was supported by the National Natural Science Foundation of China under the grant No. 70973145 and 70771114.
References 1. Frey, R., McNeil, A.J.: Estimation of tail-related risk measures for heteroscedastic financial time series: an extreme value approach. Journal of Empirical Finance 7, 271–300 (2000) 2. Bystrom, H.N.E.: Managing extreme risks in tranquil and volatile markets using conditional extreme value theory. International Review of Financial Analysis 13, 133–152 (2004) 3. Malay, B., Gopal, R.: Conditional VaR using EVT- Towards a planned margin scheme. International Review of Financial Analysis 17, 382–395 (2008) 4. Xiao, Z., Fu, X.X., Zhang, B.: Dynamic VaR Risk Measures Based on EVT-POTFIGARCH. Nankai management review 11(4), 100–104 (2008) (in Chinese) 5. Patton, A.J.: Modelling time-varying exchange rate dependence using the conditional Copula. Working Paper of London School of Economics & Political Science (2001) 6. Patton, A.J.: Modeling asymmetric exchange rate dependence. International Economic Review 2(47), 527–556 (2006) 7. Tursunalieva, A., Silvapulle, P.: Assessing and modeling the changes in dependence between exchange rates. Working Paper of Monash University (2007) 8. Wu, Z.X., Ye, W.Y.: Risk Analysis of Foreign Exchange Markets by Copula. Chinese management science 4, 1–5 (2004) (in Chinese) 9. Sun, L.J., Bai, M.Y.: The Research of Optimal Archimdean Copula Based Exchange Rate Correlation. Data Analyses 3(2), 91–102 (2008) 10. Wang, Z.R., Chen, C.: The exchange rate risk of Chinese yuan: using VaR and ES based on extreme value theory. Journal of Applied Statistics 37(2), 265–282 (2010) 11. Eric, J., Micheal, R.: Copula-GARCH model of Conditional Dependency of Financial Series: the Copula-GARCH Model. Working Paper (2002) 12. Eric, J., Micheal, R.: The Copula-GARCH model of Conditional Dependency: An international stock market application. Journal of International Money and Finance 25, 827–853 (2006) 13. Lee, T.H., Long, X.D.: Copula-based multivariate GARCH model with uncorrelated dependent errors. Journal of Econometrics 150, 207–218 (2009) 14. Wu, Z.X., Chen, M., Ye, W.Y.: Risk Analysis of Portfolio by Copula-GARCH. Journal of Systems Engineering Theory and Practice 2(8), 45–52 (2006) (in Chinese) 15. Coles, S.: An Introduction to Statistical Modeling of Extreme Values. Series in Statistics, New York (2001)
Forecasting Financial Time Series via an Efficient CMAC Neural Network Chi-Jie Lu1 and Jui-Yu Wu2,* 1
Department of Industrial Engineering and Management, Ching Yun University, Taiwan
[email protected] 2 Department of Business Administration, Lunghwa University of Science and Technology, Taiwan
[email protected]
Abstract. Cerebellar model articulation controller neural network (CMAC NN) has many advantages, such as very fast learning, reasonable generalization capability and robust noise resistance. Thus, CMAC NNs are conventionally used in robot control. To solve financial time series forecasting, this paper presents an efficient CMAC NN scheme. The proposed CMAC NN transforms continuous values of input variables to discrete indexes by using a quantization operator. To enhance generalization ability, the CMAC NN employs high quantization resolution and a large generalization size. To perform many-to-few mappings, the CMAC NN uses an efficient and fast hashing code based on bitwise XOR operator. The proposed CMAC NN was used to Nikkei 225 closing cash indexes collected from Japanese stock market. The forecasting results of the proposed CMAC NN were compared with those of support vector regression (SVR), which is statistical/ machine learning algorithm. Experimental results indicate that the performance of the CMAC NN was better than SVR in the tested case. Therefore, the CMAC NN may be considered as an efficient tool for forecasting financial time series. Keywords: cerebellar model articulation controller, neural network, stock price prediction, support vector regression.
1 Introduction Over the past three decades, there has been a growing interest in financial time series forecasting [1, 2]. Stock index forecasting is regarded as a challenging task of the financial time series prediction process since the stock market is a complex, evolutionary, and nonlinear dynamic system [3]. Many techniques have been applied to the domain of time series prediction, and are traditionally categorized as statistical methods and spectral analysis [4, 5]. Zhang et al. [6] found that the traditional statistical methods based on linear models, such as Box-Jenkins approach, have difficulty in measuring the performance of real systems with nonlinear behavior. Rigoze et al. [7] compared the advantages and drawbacks of four classical spectral analytical methods. These approaches are difficult to apply for general practitioners; Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 73–82. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
74
C.-J. Lu and J.-Y. Wu
that is, prior knowledge of signal processing is required to adjust specific parameters for these spectral analytical methods. Artificial neural networks (ANNs) which can accurately model nonlinear systems. Unlike traditional statistical models, NNs are data-driven and non-parametric models. A cerebellar model articulation controller (CMAC) is a supervised NN. Albus [8, 9] first introduced the CMAC NN based on the functions of the human cerebellum. The CMAC NN has the advantages of very fast learning, reasonable generalization ability and robust noise resistance [10]. It has been successfully used in many applications, such as control [11, 12], fault detection [13, 14] and classification [15, 16]. Moreover, CMAC NNs have been used to predict, such as electricity price forecasting and power system marginal price forecasting [17, 18]. However, this paper attempted to extend the application field to stock price forecasting. This paper presented a CMAC NN scheme for stock index prediction. Moreover, this paper compared the performance of the CMAC NN and support vector regression (SVR), which is a novel machine learning algorithm, and is increasing attention to solve nonlinear regression estimation problems.
2 Literature Review 2.1 CMAC NN A CMAC NN comprises five cells, input space ( X ), sensory cell ( S ), association cell ( A ), physical memory cell ( P ) and output cell ( Y ), and transforms input values into output values using a series of mappings, as follows: (1) X → S A quantization operator is used to transform components of input vectors into discrete quantization indexes. Each component xi of input pattern j is quantized individually by:
⎡⎛ N sij = ⎢⎜ max i min ⎜ ⎣⎢⎝ xi − xi
⎤ ⎞ min ⎥ − 1, i = 1, 2,… , n j = 1, 2,… , Dtotal ⎟⎟ × xij − xi ⎥⎦ ⎠
(
)
sij = 0, if sij < 0 where sij = quantization index of component xi of input pattern j xij = component xi of input pattern j ximin = minimum value of input vector xi ximax = maximum value of input vector xi Ni = quantization resolution of input vector xi n = dimension of input space Dtotal = total number of input patterns
[⋅]
= rounded number
(1)
(2)
Forecasting Financial Time Series via an Efficient CMAC Neural Network
75
The indexes sij depends on ximin , ximax and Ni . Larger values of Ni improve the accuracy of representation, but require a large weight table w . (2) S → A A sensory cell comprises random tables [19]. The size of each random table is calculated by: Ci = Ni + g − 1, i = 1, 2,… , n
(3)
where
Ci = size of the random table of input vector xi g = generalization size The indexes sij are mapped to random table i , and segment mappings are created based on g . The quantization and segment mappings generate natural interpolation, and give the CMAC NN the ability to generalize [20]. A large g increases generalization ability, but reduces the approximation accuracy. (3) A → P → Y A vector a in cell A corresponds to a table w in cell P . The table w stores many weights. An actual output value in cell Y can be computed by the matrix operation:
yo = aT w
(4)
where ⎡ a1 ⎤ ⎢ ⎥ a a = ⎢ 2⎥ , w = ⎢ ⎥ ⎢ ⎥ ⎣ ak ⎦
⎡ w1 ⎤ ⎢ ⎥ ⎢ w2 ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ wk ⎦
y o = actual output, k = size of w and T = transpose of a matrix. 2.2 SVR The SVR is an adaptation of recently introduced statistical/ machine learning theory based classification paradigm namely, support vector machines. For illustrating the concept of SVR, a typical regression problem is formulated. Consider a dataset G = ( xij , yd , j ) ( i = 1, 2,..., n , j = 1, 2,..., Dtotal ), where xij are input patterns and yd , j is desired output. The goal of the regression analysis is to determine a function f (x) , so as to predict accurately the yd , j . The regression problem can be classified as linear and nonlinear regression problems. As the nonlinear regression problem is more difficult to deal with, SVR was mainly developed for dealing with the nonlinear regression problem. SVR algorithms have been successfully applied in time series prediction, such as and financial time series forecasting [21, 22]. When constructing a
76
C.-J. Lu and J.-Y. Wu
forecasting model, one major concern is increasing the accuracy of predictions. This would be especially beneficial in areas like stock price forecasting, where even the smallest improvement in prediction accuracy can have a positive impact on investments. For building SVR forecasting model, the LIBSVM package proposed by Chang and Lin [23] is used in this paper since the package can be easy to find and use.
3 Method A CMAC NN was developed and used to Nikkei 225 closing cash index dataset collected from Japanese stock market. Figure 1 shows the pseudo-code of the proposed CMAC NN, which is described below. begin Training stage: h←0 Step 1: quantize and normalize Step 2: create random tables and weight table While h < epochmax do for j = 1,2,…, Dtrain do Step 3: generate address table Step 4: calculate of actual output Step 5: learning rule endFor Step 6: evaluate training accuracy h ← h +1 endWhile Testing stage: Step 7: measure testing accuracy end
Fig. 1. The pseudo-code of the proposed CMAC NN
Step 1: Quantize and normalize Equations (1) and (2) are used to quantize components of input patterns. Each desired output
yd , j ( j = 1, 2,… , Dtotal ) undergoes normalization, as follows: yd' , j =
yd , j − ydmin ydmax − ydmin
( Emax − Emin ) + Emin ,
j = 1, 2,… , Dtotal
where
yd' , j = normalized desired output j ydmin = minimum value of desired output vector Yd = [ yd ,1 , yd ,1 ,… yd , Dtotal ]T ydmax = maximum value of Yd Emin = minimum value of expected output Emax = maximum value of expected output The values [ Emin , Emax ] are generally set to [0.2, 0.8].
(5)
Forecasting Financial Time Series via an Efficient CMAC Neural Network
77
Step 2: Create random tables and weight table w Equation (3) is used to compute the size of each random table, and the random tables are then created. To control the range of the index of generated address table, each random table consists of uniform random numbers created from the interval [ 0, k / 2 ]. A table w is generated based on k as defined in Section 2, with initial weights set to “0”. To reduce the probability of hash collision, this paper used a large value of k = 10000. Step 3: Generate address table To implement many-into-few mapping, the proposed CMAC NN used a hash coding based on bitwise XOR operator [24]. The detail description of the hash coding can be found in the literature [15]. Step 4: Calculate actual output Equation (4) is used to measure the values yo, j ( j = 1, 2,… , Dtrain ). Step 5: Learning rule Equation (6) is employed to modified the weights in w , as follows: wl (h + 1) = wl (h) +
β ( yd − yo ) g
, l = 1, 2,… , k
h = 1, 2,… , epochmax
(6)
where β = learning rate Step 6: Evaluate training accuracy A normalized root mean square error (NRMSE) is used as a performance index to evaluate a training accuracy, as follows: Dtrain
∑
NRMSE =
j =1
( yd' , j − yo, j )2
(7)
Dtrain
where Dtrain = number of input patterns for training Steps 3–6 are repeated until the termination condition epochmax is met. Step 7: Measure testing accuracy Input pattern j ( j = Dtrain + 1, Dtrain + 2,… , Dtotal ) is used for testing. The actual outputs of recall of the CMAC NN can be calculated using the w obtained in the training stage (Steps 1−6), and then the generalization accuracy of the CMAC NN can be measured by a testing NRMSE.
4 Experimental Results For evaluating the performance of the proposed CMAC NN forecasting model, the daily Nikkei 225 closing cash index dataset is used in this paper. In forecasting Nikkei
78
C.-J. Lu and J.-Y. Wu
225 closing cash index, the Nikkei 225 index future prices are used as forecasting variables since the changes of the future prices lead price changes of the cash market [25, 26]. Four forecasting variables (Singapore Exchange-Derivative Trading Limited, Osaka Securities Exchange, Chicago Mercantile Exchange and the previous day’s cash market closing index) are used for predicting the Nikkei 225 closing cash index. The daily data of futures and cash prices from December 6, 2004 to December 30, 2008 of the Nikkei 225 cash index provided by Bloomberg are collected in this paper. There are totally 1000 data points in the dataset and the daily Nikkei 225 closing cash prices are shown in Fig. 2. The first 800 data points (80% of the total sample points) are used as the training sample while the remaining 200 data points (20% of the total sample points) are used as the testing sample.
19000 17000 Prices
15000 13000 11000 9000 7000 5000 1
100
200
300
400
500
600
700
800
900
1000
Data points
Fig. 2. The daily Nikkei 225 closing cash prices from December 6, 2004 to December 30, 2008
The proposed CMAC NN was coded in MATLAB. This paper employed epochmax = 200 as a termination condition, β = 0.1 and k = 10000. The optimal settings of parameters ( Ni , g ) for the proposed CMAC NN was evaluated by using Ni = {30, 40, 50} and g = {10, 20, 30, 40, 50}. Table 1 lists the model selection results of the proposed CMAC NN. When using a large value of Ni = 50, increasing g decreased the training accuracy, but enhanced the generalization ability of the CMAC NN. When using small values of Ni = 10 and Ni = 20 against various g = {10, 20, 30, 40, 50}, the poor training and generalization accuracies were found. As shown in Table 1, the CMAC NN obtained the best training and testing RMSEs by using the parameter settings ( Ni , g ) = (50, 40), which are high quantization and a large generalization size. For building SVR forecasting model, the LIBSVM package proposed by Chang and Lin [23] is used in this paper. The original dataset is first scaled into the range of [−1.0, 1.0] when using LIBSVM package. The parameter setting of regularization constant (C) and precision parameter ( ε ) which generate the minimum forecasting root mean square error (RMSE) is considered as the best parameter setting. In the modeling of the SVR model, C=1.25 and ε =0.0019 can be obtained. Since C=1.25 is near C=21 and ε =0.0019 is close to ε =2-9, the parameter set (C=21, ε =2-9) is used as the starting point of grid search for searching the best parameters. The testing results
Forecasting Financial Time Series via an Efficient CMAC Neural Network
79
of the SVR model with combinations of different parameter sets are summarized in Table 2. From Table 2, it can be found that the parameter set (C=23, ε =2-9) gives the best forecasting result (minimum testing RMSE) and is the best parameter set for SVR model in forecasting Nikkei 225 closing cash index. Table 1. Model selection results of the proposed Table 2. Model selection results of the SVR CMAC NN model-Nikkei 225 forecasting model-Nikkei 225
The prediction performance is evaluated using the following performance measures, namely, the root mean square error (RMSE), mean absolute difference (MAD), mean absolute percentage error (MAPE), directional accuracy (DA), correct up trend (CP) and correct down trend (CD). The definitions of these criteria can be found in Table 3. RMSE, MAD and MAPE are measures of the deviation between actual and predicted values. Table 3. Performance measures and their definitions
Metrics
Calculation
Metrics
Dtotal
RMSE
RMSE =
∑ ( yd , j − yo , j ) 2 j =1
Dtotal Dtotal
MAD
MAD =
∑ j =1
∑
MAPE =
yd , j − yo , j
j =1
CP
yd , j
⎧1 dj = ⎨ ⎩0 CD =
yd , j − yo , j Dtotal
⎧1 dj = ⎨ ⎩0 CP =
Dtotal Dtotal
MAPE
DA
Calculation 100 DA = Dtotal
CD
⎧1 dj = ⎨ ⎩0
Dtotal
∑ d j , where j =1
(yo , j − yo , j −1 )( yd , j − yd , j −1 ) ≥ 0 otherwise 100 Dtotal ∑ dj n1 j =1
, where
(yo , j − yo , j −1 )( yd , j − yd , j −1 ) ≥ 0 otherwise 100 Dtotal ∑ dj n2 j =1
, where
(yo , j − yo , j −1 )( yd , j − yd , j −1 ) ≥ 0 otherwise
* n1 is number of data points belong to up trend and n2 is number of data points belong to down trend.
80
C.-J. Lu and J.-Y. Wu
The Nikkei 225 closing cash index forecasting results using the SVR and proposed CMAC NN models are computed and listed in Table 4. From Table 4, it can be found that the RMSE, MAD and MAPE of the CMAC NN model are, respectively, 55.36, 35.50 and 0.35%. It can be observed that these values are smaller than those of SVR models. It indicates that there is a smaller deviation between the actual and predicated values using the proposed CMAC NN model. Moreover, compared to the SVR models, the CMAC NN model has the highest DA (directional accuracy), CP (correct up trend) and CD (correct down trend) ratios which are 81.58%, 83.33% and 79.62%, respectively. DA, CP and CD provide a good measure of the consistency in prediction of the price direction. Thus, the proposed CMAC NN model provides better forecasting results than the SVR model in terms of prediction error and prediction accuracy in the tested dataset. Table 4. The Nikkei 225 closing cash prices forecasting results using the proposed the SVR and proposed CMAC NN models Metrics
RMSE
MAD
MAPE
DA
CD
CP
SVR
61.58
43.20
0.39%
78.95%
81.67%
75.93%
CMAC NN
55.36
35.50
0.35%
81.58%
83.33%
79.62%
Models
5 Conclusion Stock index forecasting is a challenging task and has drawn serious attention during past three decades. This paper developed an efficient CMAC NN to forecast Nikkei 225 closing cash index. This paper compared the forecasting results of the proposed CMAC NN with SVR. Experimental results indicate that the CMAC NN was superior to SVR for the tested dataset. Moreover, the CMAC NN is much easier to use than traditional statistical and spectral analytical methods. Thus, the CMAC NN may be considered as financial tool for stock price prediction. Future work is to compare the performance of the CMAC NN with SVR using the dataset TAIEX closing cash indexes collected from Taiwanese stock market.
Acknowledgements The authors would like to thank the National Science Council of the Republic of China, Taiwan for financially supporting this research under Contract No. NSC 982221-E-262-014-.
References 1. De Gooijer, J.G., Hyndman, R.J.: 25 Years of Time Series Forecasting. International Journal of Forecasting 22, 443–473 (2006) 2. Lawrence, M., Goodwin, P., O’Connor, M., et al.: Judgmental Forecasting: A Review of Progress over the Last 25 Years. International Journal of Forecasting 22, 493–518 (2006)
Forecasting Financial Time Series via an Efficient CMAC Neural Network
81
3. Atsalakis, G.S., Valavanis, K.P.: Surveying Stock Market Forecasting Techniques - Part II: Soft Computing Methods. Expert Systems with Applications 36, 5932–5941 (2009) 4. Chatfield, C.: Time-Series Forecasting. Chapman & Hall/CRC, New York (2001) 5. Priestley, M.B.: Spectral Analysis and Time Series. Academic Press, New York (1981) 6. Zhang, G., Patuwo, B.E., Hu, M.Y.: Forecasting with Artificial Neural Networks: The State of Art. International Journal of Forecasting 14, 35–62 (1998) 7. Rigozo, N.R., Echer, E., Nordemann, D.J.R., et al.: Comparative Study between for Classical Spectral Analysis Methods. Applied Mathematics and Computation 168, 411– 430 (2005) 8. Albus, J.S.: A New Approach to Manipulator Control: The Cerebellar Model Articulation Controller (CMAC). ASME Journal of Dynamic Systems, Measurement, and Control 97, 220–227 (1975) 9. Albus, J.S.: Data Storage in the Cerebellar Model Articulation Controller (CMAC). ASME Journal of Dynamic Systems, Measurement, and Control 97, 228–233 (1975) 10. Wong, Y.F., Sideris, A.: Learning Convergence in the Cerebellar Model Articulation Controller. IEEE Transactions on Neural Networks 3, 115–121 (1992) 11. Lin, C.M., Chen, L.Y., Chen, C.H.: RCMAC Hybrid Control for MIMO Uncertain Nonlinear Systems Using Sliding-Mode Technology. IEEE Transactions on Neural Networks 18, 708–720 (2007) 12. Peng, Y.F.: Robust Intelligent Sliding Model Control Using Recurrent Cerebellar Model Articulation Controller for Uncertain Nonlinear Chaotic Systems. Chaos, Solitons & Fractals 39, 150–167 (2009) 13. Wang, S., Jiang, Z.: Valve Fault Detection and Diagnosis Based on CMAC Neural Networks. Energy and Buildings 36, 599–610 (2004) 14. Hung, C.P., Wang, M.H.: Diagnosis of Incipient Faults in Power Transformers Using CMAC Neural Network Approach. Electric Power Systems Research 71, 235–244 (2004) 15. Wu, J.Y., Lu, C.J.: Applying Classification Problems via a Data Mining Approach Based on a Cerebellar Model Articulation Controller. In: 1st Asian Conference on Intelligent Information and Database Systems, pp. 61–66. Dong Hoi City, Vietnam (2009) 16. Lin, C.J., Lee, J.H., Lee, C.Y.: A Novel Hybrid Learning Algorithm for Parametric Fuzzy CMAC Networks and Its Classification Applications. Expert Systems with Applications 35, 1711–1720 (2008) 17. Zhou, H., Chen, J., Wu, H., et al.: CMAC-Based Short-Term Electricity Price Forecasting. In: Sixth International Conference on Advances in Power System Control, Operation and Management, Hong Kong, pp. 348–353 (2003) 18. Qiaolin, D., Jing, T., Jianxin, L.: Application of New FCMAC Neural Network in Power System Marginal Price Forecasting. In: The 7th International Power Engineering Conference, Singapore, pp. 1–57 (2005) 19. Lee, J.: Measurement of Machine Performance Degradation Using a Neural Network Model. Computers in Industry 30, 193–209 (1996) 20. Handelman, D.A., Lane, S.H., Gelfand, J.J.: Integrating Neural Networks and KnowledgeBased Systems for Intelligent Robotic Control. IEEE Control Systems Magazine 10, 77–87 (1990) 21. Huang, C.L., Tsai, C.Y.: A Hybrid SOFM-SVR with a Filter-Based Feature Selection for Stock Market Forecasting. Expert Systems with Applications 36, 1529–1539 (2009) 22. Lu, C.J., Lee, T.S., Chiu, C.C.: Financial Time Series Forecasting Using Independent Component Analysis and Support Vector Regression. Decision Support Systems 47, 115– 125 (2009)
82
C.-J. Lu and J.-Y. Wu
23. Chang, C.C., Lin, C.J.: Libsvm: A Library for Support Vector Machines, http://www.csie.ntu.edu.tw/~cjlin/libsvm 24. Zobrist, A.L.: A New Hashing Method with Application for Game Playing. Technical report, Computer Sciences Department, University of Wisconsin (1969) 25. Lee, T.S., Chen, N.J.: Investigating the Information Content of Non-Cash-Trading Index Futures Using Neural Networks. Expert Systems with Applications 22, 225–234 (2002) 26. Lee, T.S., Chiu, C.C.: Neural Network Forecasting of an Opening Cash Price Index. International Journal of Systems Science 33, 229–237 (2002)
Forecasting Daily Cash Turnover of Bank with EWMA and SVR Wei-min Ma and Wei Lu School of Information Technology of Beijing Normal University Zhuhai Campus, Zhuhai 519085, China
Abstract. This paper present one forecasting method with Exponential Weighted Moving Average(EWMA) and Support Vector Regression (SVR). The daily cash turnover of the banks is time-serial data, banks need to forecasting daily cash turnover for banking reserve. First, the time series is preprocessed with EWMA method. The EWMAs with different coefficients are selected for forecasting features. And then SVR is used in the transformed dataset with EWMA for forecasting. The experimental result shows that the EWMA can improve the forecasting accuracy, and the SVR is more effective than other method such as 1NN and MLP. Statistical correlation of SVR between the forecasted and actual values is much higher than other method such as 1-NN and MLP. Keywords: Time series, EWMA, SV-regression, SRM.
1
Introduction
Forecasting time-series is important in several domains[1][2] and a large number of studies have used classical statistical method for forecasting series behavior. Advanced methods such as neural networks[3], genetic algorithms[4], Markov model[5] and fuzzy methods have also been frequently used[6]. The one nearest neighbor(1-NN) method is one local approximation technique. Unlike global approximators, local approximation techniques are dependent on the number of free parameters. The well-known methods of local approximation for time-series forecasting include chart representations, nearest neighbor methods[7] and pattern imitation techniques. [8] applied local approximation method in dynamic time-serial forecasting. The Multi-Layer Perceptron(MLP)[9] is a kind of neural net that is capable of generating arbitrary decision boundaries. But when the data change remarkably, the forecast precision of these method will descend obviously. This paper present one forecasting method with Exponential Weighted Moving Average(EWMA)[10]and Support Vector Regression(SVR)[11][12] [13] which can forecast the time series steadily. Firstly we use exponential weighted moving average method(EWMA) to preprocess the time series data. Exponential weighted moving average lends more weight to recent data than the simple moving average, which actually drops old data from its sample. And we also calculated the ratio of original data to
Corresponding author.
Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 83–90. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com
84
W.-m. Ma and W. Lu
different EWMA. By comparing, we find that using ratio data can improve the forecasting precision. After the preprocessing, we use SVR method for forecasting. SVMs(Support Vector Machines) are a recently developed class of classifier architectures, derived from the structural risk minimization principle by Vapnik[11]. A promising aspect of the SVM classifiers is that, SVM training involves selecting a small subset of critical data points(known as support vectors) from the original training data base. SVMs are also used to estimate real-valued functions, which is usually denote as SVR. By experiment, we find that SVR is better than 1-NN and MLP. The rest of paper is organized as following: section 2 describes the preprocessing of the original data and forecasting method with EWMA and SVR; the section 3 gives experiment result; and section 4 gives conclusion and future work.
2 2.1
Forecasting Method with EWMA and SVR Domain Description
Our problem is to model the current daily cash turnover of the time-series with past states.The data example is: Table 1. The example of data CDVH …
OULJLQDO Bank Transaction 5DQN No. Date Weekday Records Saturday Sunday Monday Wednesday ... … …. … …
… … … … … …
Daily Cash Turnover …
We denote our data as the form (x, y), where x represent some effective factors such as date, bank No., debit or credit, and so on, y represent daily cash turnover. If we choose to represent daily cash turnover of time-series as vector y = {y1 , y2 , · · · , yn } where n is the total number of points in the series. Often, we also represent such a series as a function of time, e.g. yn = yt , yn−1 = yt−1 , and so on. Then the state of the time-series represents its current daily cash turnover value yt . One simple method of forecasting may be based on identifying the closest neighbor of yt in the past data, say yj , and forecasting yt+1 on the basis of yj+1 . In this paper, we will also refer to states as patterns used in the matching process. The technique of matching structural primitives is based on the premise that the past repeats itself. The dynamic behavior of time-series can be efficiently forecasted by using local approximation. For this purpose, a map between current states and the nearest neighbor past states can be generated for forecasting. The aim of a pattern matching algorithm is to find the closest match of patter ρ in the historical data and use this for forecasting yt+1 . 2.2
Exponential Weighted Moving Average
Exponential smoothing is the most common model-free means of forecasting a future realization of a time series[10]. It requires the specification of a smoothing
Forecasting Daily Cash Turnover of Bank with EWMA and SVR
85
factor which is usually chosen from the data to minimize the average squared residual of previous one-step ahead forecasts. The most basic exponential smoother is the exponentially weighted moving average(EWMA). It has the attraction of being model-free and based on a simple algebraic formula, in terms of a smoothing factor, and therefore enjoys common usage as an ad hoc forecasting procedure. The goal is to estimate the underlying trend in a scatterplot without the use of restrictive models. The exponentially weighted moving average has the advantage of giving the most weight to the most current value. Also it only requires maintaining the value of the previous day’s average rather than the data for the entire period being averaged, so it is very compact. Following above, let y = {y1 , y2 , · · · , yt } be a time series observed of daily cash turnover at equally-spaced time points T1 , T2 , · · · , Tt . We consider the problem of using these data to forecast yt+1 . One of the simplest model-free approaches to this problem is the exponentially weighted moving average(EWMA). It forecasts yt+1 using a weighed average of past observations with geometrically declining weights: t−1 y t+1 = (1 − ω) ω i yt−i (1) i=0
where 0 < ω < 1 is commonly referred to as the smoothing constant. An advantage of using geometric weights is that equ. (1) admits the recurrence formula y t+1 = ωyt + (1 − ω)yt
(2)
with y 1 = y1 . Notice that the sum of the weights in (1)is 1 − ω t which is close to 1 when it is large enough. We use formula (2) to preprocess our bank data. The ω is called EXPONENT, which is the element in the formula that determines the period of the moving average, can set different value. The calculation of the EXPONENT is done by dividing 2 by the number of days in the period being averaged. For example, the EXPONENT for a 200-day moving average is determined by dividing 2 by 200. The result is 0.01 (2/200=0.01). Because our data capacity is not very large, so we let ω takes five values:0.1, 0.2, 0.25, 0.3, 0.4, corresponding to 20, 10, 8, 20/3, 5 days of the moving average. We notate the calculating result of (2) as y(1) for ω = 0.1, y(2) for ω = 0.2, and so on. In respecting that the value of original data or exponential weighted moving average data is very big, we consider using the ratios of them for forecasting. For example, we can create new data y1 = {y11 = y1 /y(1)1 , y21 = y2 /y(1)2 , · · · , yt1 = yt /y(1)t }. We can see that it improves the forecasting precision by latter experiment. 2.3
Support Vector Regression Algorithm
Support vector regression is based on the Structural Risk Minimization(SRM) principle from statistical learning theory [11]. The idea of SRM is to find a hypothesis for which one can guarantee the lowest probability of error. In SVR, this
86
W.-m. Ma and W. Lu
principle is translated to find a separating hyperplane with the largest distance to the closest training samples. Given the training data (xi , yi ), i = 1, · · · , n, xi ∈ Rd , yi ∈ R. SVR maps the input vectors x into a high-dimensional feature space H through some mapping function φ : Rd → H, and constructs an optimal separating hyperplane in this space. In the linear case, φ(x) is an identity function; in the nonlinear case, the mapping φ(·) is performed by a kernel function K(·, ·) which defines an inner product in H. In ε-SV regression Vapnik[11][13], our goal is to find a function f (x) that has at most ε deviation from the actually obtained targets yi for all the training data, and at the same time, is as flat as possible. The function given by an SVR is: f (x) = ω · φ(x) + b. and therefore f (x) =
l
(αi − α∗i )k(xi , x) + b.
(3)
(4)
i=1
where the threshold b is computed as follows: b = yi − ω · φ(xi ) − ε, αi ∈ (0, C) b = yi − ω · φ(xi ) + ε, α∗i ∈ (0, C).
(5)
The SVR acts well in estimating real-valued functions, regression estimation and signal processing[11] [14]. 2.4
The Forecasting Flow
We first preprocess original data with some basic operations, such as separating the data according to bank No., removing outlines, processing missing value, and so on. So the whole processing flow is shown in Figure 1:
2ULJLQDO'DWD
%DVLF3UHSURFHVV
(:0$SURFHVV
0HDVXUH5HVXOWV
11 0/37HVWLQJ 690 7HVWLQJ GDWD 7UDLQLQJGDWD 11
0/37UDQLQJ 690
Fig. 1. Forecasting processing flow chart
Forecasting Daily Cash Turnover of Bank with EWMA and SVR
3 3.1
87
Experiment Results Experimental Data and Performance Measures
The data comes from the bank, we do experiment for every branch, and forecast the daily cash turnover for the debit and credit respectively. We select randomly 4/5 of the data as training set, the rest as testing set. We use the following measures for estimating the accuracy of the forecasting.In these equations, we denote f as the forecasted value , a the actual value, and a the average of the actual value.
i (fi −f )(ai −a) (1) The correlation coefficient SSFFSAA . where SF A = , SF = n−1 2 2 f −f ( ) i (a −a) i i , SA = i n−1 . It measures the statistical correlation between the n−1 forecasted and actual values. (2) Mean absolute error.
|a1 − f1 | + |a2 − f2 | + · · · + |an − fn | n
(6)
It is the average of the difference between forecasted and actual value in all test case; it is the average forecasting error. (3) Root mean-squared error. (a1 − f1 )2 + (a2 − f2 )2 + · · · + (an − fn )2 (7) n If this number is significantly greater than the mean absolute error, it means that there are test cases in which the forecasting error is significantly greater than the average forecasting error. (4) Relative absolute error. |a1 − f1 | + |a2 − f2 | + · · · + |an − fn | |a1 − a| + |a2 − a| + · · · + |an − a|
(8)
It is the total absolute error made relative to what the error would have been if the forecasting simply had been the average of the actual values. (5) Root relative squared error. (a1 − f1 )2 + (a2 − f2 )2 + · · · + (an − fn )2 (9) (a1 − a)2 + (a2 − a)2 + · · · + (an − a)2 It is the total squared error made relative to what the error would have been if the forecasting had been the average of the absolute value. In latter four error measurements, a lower value means a more precise model, with a value of 0 depicting the statistically perfect model.
88
3.2
W.-m. Ma and W. Lu
Primary Results
First we compare the results of different preprocessing data. The results are showed on the table 2. Table 2. The measures of different data Original ratio1 ratio2 &RUUHODWLRQFRHIILFLHQW 5HODWLYHDEVROXWHHUURU 5RRWUHODWLYHVTXDUHGHUURU
ratio3
ratio4
ratio5
In table 2, row original represents using original data, row ratio1 represents using data y1 , row ratio2 of using data y2 , and so on. From the table, we can see that the statistical correlation using ratios increase remarkably than using original data, the error measures using ratios reduce much. And using data y1 can obtain best result, corresponding to take EXPONENT ω = 0.1 in formula (2). It is showed that taking 20 days for smoothing region acts best on our data. We can also show the results of 1-NN, MLP and SVR methods on table 3. Table 3. The measures of different methods 1-NN MLP SVR (SMO) &RUUHODWLRQFRHIILFLHQW 0HDQDEVROXWHHUURU 5RRWPHDQVTXDUHGHUURU 5HODWLYHDEVROXWHHUURU 5RRWUHODWLYHVTXDUHGHUURU
We select the parameter of SVR algorithm, and the table is the best result. From the table, we can see the the forecasting method with EWMA and SVR is much better than other two methods. The correlation of using SVR is much higher than using other two methods, and the error measures of using SVR is the lowest. This prove SVR has better generalization than 1-NN and MLP, for it’s based on Structural Risk Minimization, and other methods are based on Experimental Risk Minimization. The precisions of different methods on some testing data are showed in figure 2: In these figures, the serial number represent testing data set, and forecasting error is defined by error = |yf − ya |/ya (10) where yf is forecasting value and ya is actual value. The forecasting error can reflect the effect of the methods, we can see that the points represent forecasting error of SVR is below these points represent 1-NN and MLP on most data, which proves the SVR does better than the other two methods.
Forecasting Daily Cash Turnover of Bank with EWMA and SVR
89
IRUHFDVWLQJHUURU
11 0/3 695
VHULDOQXPEHU
Fig. 2. The results of three method
4
Conclusion and Future Work
In this paper we present one forecasting method with EWMA and SVR in the bank cash. The experimental results have showed that the SVR works better than 1-NN and MLP methods, and the forecasting precision with SVR is much better. The correlation of SVR is much higher than the other two methods, and the error measures of SVR is the lowest. This is mainly because SVR is based on Structural Risk Minimization, and the other two methods are based on Experimental Risk Minimization. For the data of every branch of bank is small, about 100 records. And SVR is good for small sample problem, so it has better generalization than the other two methods on our data. We also compare the forecasting effect of using original data and ratio data preprocessed by EWMA methods. The experimental results show that the EWMA methods can smooth the data, consequently improve the precision. The statistical correlation of using ratios increase remarkably, and the error measures reduce much. In the future, we will consider adjusting the parameter of the SVR so that we can obtain better forecasting precision. Also we will apply more preprocessing and learning algorithms on our data, and use some statistic measures to compare the results. Acknowledgments. This paper is supported by the teaching quality engineering foundation of Beijing Normal University Zhuhai Campus.
References 1. Last, M., Klein, Y., Kandel, A.: Knowledge Discovery in Time Series Databases. IEEE Transactions on Systems, Man, and Cybernetics 31B (2001) 2. Han, J., Dong, G., Yin, Y.: Efficient Mining of Partial Periodic Patterns in Time Series Database. In: Fifteenth International Conference on Data Engineering, pp. 106–115. IEEE Computer Society, Sydney (1999)
90
W.-m. Ma and W. Lu
3. Refenes, A.N., Burgess, A.N., Bentz, Y.: Neural Neworks in Financial Engineer: A study in Methodology. IEEE Transactions on Neural Networks 8(6), 1222–1267 (1997) 4. George, S.I.: A Search for Hidden Relationships: Data Mining with Genetic Algorithms 5. Tim, O., Firoiu, L., Cohen, P.: Clustering Time Series with Hidden Markov Models and Dynamic Time Warping. In: IJCAI-1999 Workshop on Sequence Learning (1999) 6. Hadjimichael, M., Arunas, P., Kuciauskas, L., Brody, R.: MEDEX: A Fuzzy System for Forecasting Mediterranean Gale Force Winds. In: IEEE Int. Conf. On Fuzzy Systems, pp. 529–534. IEEE press, New Orleans (1996) 7. Bottou, L., Vapnik, V.N.: Local Learning Algorithms. Neural Computation 4(6), 888–900 (1992) 8. Singh, S., McAtackney, P.: Dynamic Time-Series Forecasting Using Local Approximation. In: Proceedings of the IEEE Tenth International Conference on Tools with Artificial Intelligence, pp. 392–399. IEEE press, Los Alamitos (1998) 9. Hampshire II, J.B., Perlmutter, B.A.: Equivalence Proofs for Multilayer Perceptron Classifiers and the Bayesian Discriminant Function. In: Proceedings of the 1990 Connectionist Models Summer School, San Mateo (1990) 10. Gijbels, I., Pope, A., Wand, M.: Automatic Forecasting via Exponential Smoothing: Asymptotic Properties (1997) (manuscript) 11. Vapnik, V.: The Nature of Statistical Learning Theory. Springer, NY (1995) 12. Cortes, C., Vapnik, V.: Support-Vector Networks. Machine Learning 20(3), 273– 297 (1995) 13. Smola, A., Sch, B.: A Tutorial on Support Vector Regression. NeuroCOLT2 Technical Report NC2-TR-1998-030 (1998) 14. Sch¨ olkopf, B., Smola, A.J.: Learning with Kernels. MIT Press, Cambridge (2002)
Financial Distress Prediction Model via GreyART Network and Grey Model Ming-Feng Yeh1, Chia-Ting Chang2, and Min-Shyang Leu1 1
Department of Electrical Engineering, Lunghwa University of Science and Technology, Taoyuan, 33327 Taiwan {mfyeh,unit484}@mail.lhu.edu.tw 2 Taipei Municipal Wan Fang Senior High School, Taipei, 11696 Taiwan
[email protected]
Abstract. This study attempts to use GreyART network and grey model to construct a financial distress prediction model. The inputs used to train the network are the historical data containing 17 different financial ratios of 22 healthy and 5 distressed Taiwan’s listed banks. With the help of the developed performance index, this study also proposes a growing extraction method for financial variables not only to further improve the classification ability in the training and testing phases, but also to use fewer extracted variables to build the financial distress prediction model. Simulation results show that the optimal condition is the one using four extracted variables as inputs and the vigilance threshold of 0.80. Under this condition, the proposed method generates only two clusters with corresponding classification hit rates of 96.30% and 95.24% for the training and testing results, respectively. Keywords: Financial distress prediction model, Grey model, Grey relational analysis, Adaptive resonant theory, GreyART network.
1 Introduction Since Beaver’s pioneering work in 1966, there are a large amount of studies using financial ratios to predict financial failure. Among these researches, Altman was the first one to use discriminant analysis in predicting bankruptcy by combining financial ratios into a single predictor of corporate bankruptcy [1]. The main appeal of this approach was its ability to reduce a multidimensional problem to a single score. Although discriminant analysis was widely used in the financial distress prediction problem, it has the following potential problems: 1) the distressed and viable firms used to distinguish must be linearly separable, and 2) ratios are treated as completely independent [2-3]. An alternative methodology for the analysis of financial distress is based on neural networks. Owing to the learning capability, neural networks could create a perfect mapping between the input and output values for a set of training data. Once the training process is completed, the patterns found by the neural networks can be used to forecast situations where the outcome (e.g., a firm is in distress or health) is unknown. This also demonstrates that the use of neural networks is possible without any of circumscription that binds discriminant analysis. Research studies on using neural networks for bankruptcy prediction started in 1990 [4], and Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 91–100. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
92
M.-F. Yeh, C.-T. Chang, and M.-S. Leu
are still active now. There have been lots of published literatures in the area of financial distress prediction [5-8], which supported the use of the neural networks over the statistical methods. GreyART network [9], a neural-network architecture combining grey relational analysis [10] and Adaptive resonant theory (ART) [11] network as shown in Fig. 1, is designed to find the underlying structure of a given dataset. Owing to the architecture of ART-based network with flexibility and stability, and GreyART network without the problem of selecting an appropriate learning rate, this study adopts GreyART network to construct the proposed financial distress prediction model. Generally speaking, the testing patterns of a neural network are usually given in advance and could be used to ‘post-examine’ the generalization ability of the network. Although those patterns have never been trained in the training phase, the results cannot actually reflect the forecasting ability of the network. Grey model [10] provides a simple and fast scheme to generate the forecasting values from the current and past data. This study adopts the forecasting values obtained by grey model as the testing patterns to solve this problem. Besides, GreyART network in [9] only focuses on determining the optimal clustering results for a given data set and it might not perform well in classifying the financial health of a firm. This study, therefore, develops a new performance index to solve the classification problem by determining the best result the GreyART network can attain in the sense of that performance index. This study also proposes a growing extraction method for financial variables such that the financial distress prediction model could be constructed by some extracted variables, but has better classification ability in the training and testing phases.
2 Preliminaries 2.1 Grey Relational Analysis Grey relational analysis is a similarity measure for finite sequences with incomplete information [10]. Denote the ith N-dimensional input pattern by xi = (xi(1), xi(2), …, xi(N)) and the weight vector of node j in an ART-based network by yj = (yj(1), yj(2), …, yj(N)), j ∈ {1, 2, …, m}. If xi is viewed as a specific reference pattern and all weight vectors are regarded as the comparative patterns, then the grey relational coefficient between xi and yj at the kth component, k ∈ {1, 2, …, N}, is defined as follows:
(
)
r x i (k ), y j (k ) =
Δ i ,min + ξ ⋅ Δ i , max Δ ij (k ) + ξ ⋅ Δ i ,max
(1)
where Δ ij (k ) = xi (k ) − y j (k ) , Δ i , max = max j max k Δ ij (k ), Δ i ,min = min j min k Δ ij (k ), and
ξ ∈ (0, 1] is the distinguishing coefficient controlling the resolution between Δi,max and Δi,min. The corresponding grey relational grade is g (x i , y j ) =
∑ [wk ⋅ r (x j (k ), y j (k ) )] N
k =1
(2)
Financial Distress Prediction Model via GreyART Network and Grey Model
93
where wk ≥ 0 is the weighting factor and ∑kN=1 wk = 1 . Normally, we select it as wk = 1/N for all k. The best comparative weight vector to the input is the one with the largest relational grade. Note that no matter how large the input pattern is, we can guarantee that 0 < g(xi, yj) ≤ 1. 2.2 GreyART Network GreyART network is a kind of ART-based network that incorporates grey relational analysis into ART network [11]. If each input pattern is viewed as a reference while the templates (weight vectors) of all categories are regarded as comparative ones, then the best comparative weight vector can be easily obtained by grey relational analysis. The node corresponding to the best comparative weight vector is denoted as a winner. Additionally, the vigilance test is designed to check the similarity measurement (the grey relational grade) of the winner. The algorithm of GreyART network is presented as follows: Step 1: Present a new training pattern, say xi. By using grey relational analysis, we can select the winner, which has the largest relational grade maxj{g(xi, yj)} among all weight vectors. The winner is denoted as j*. Step 2: The node j* passes the vigilance test if and only if the grey relational grade g (x i , y j ∗ ) ≥ ρ , where ρ is the vigilance threshold.
Step 3: If the winner fails the vigilance test, a new node unit k is created with weight vector yk = xi. Step 4: If the winner passes the vigilance test, adjust the weight vector of the winner j* by ) y (jnew = ∗
(
(
)
) Num cluster j(∗old ) ⋅ y (jold + xi ∗
(
)
Num cluster j(∗old ) + 1
)
) where Num cluster j(old denotes the number of members in cluster j ∗ . ∗
Fig. 1. The configuration of GreyART network
(3)
94
M.-F. Yeh, C.-T. Chang, and M.-S. Leu
After the learning process is terminated, the final weight vectors are regarded as the estimated cluster centroids. Since 0 < g(xi, yj) ≤ 1, the vigilance threshold ρ is therefore limited to the range (0, 1]. 2.3 Grey Model Let x(0) = (x(0)(1), x(0)(2), …, x(0)(N)) be a raw sequence and x(0)(k) ≥ 0, k = 1, 2, …, N. Then the grey differential equation of GM(1,1) model, i.e., the first order and single variable grey model, is defined as [11]
x (0 ) (k ) + az (1) (k ) = b, k = 2, 3, …, N,
(4)
where z (1) (k ) = 12 [ x (1) (k ) + x (1) (k − 1)], k = 2, 3, …, N, and x (1) (k ) = ∑km=1 x ( 0) (m), k = 1, 2, …, N. The unknown parameters a (the developing coefficient) and b (the grey input) in (4) can be obtained by the least square method as ⎡a ⎤ T −1 T ⎢ ⎥ = (B B ) B z , ⎣b ⎦
(5)
T
T ⎡− z (1) (2) − z (1) (3) " − z (1) ( N )⎤ ( 0) ( 0) ( 0) where B = ⎢ ⎥ and z = [x (2) x (3) " x ( N )] . " 1 1 ⎣ 1 ⎦ Once the developing coefficient and the grey input are obtained, the forecasting values of x(0)(k) and x(1)(k), denoted by xˆ ( 0 ) (k ) and xˆ (1) (k ), respectively, are determined by
b b xˆ (1) (k ) = [ x ( 0) (1) − ]e −a ( k −1) + , k = 1, 2, … a a
(6)
⎧ xˆ (1) (k ) − xˆ (1) (k − 1), k = 2, 3, ... xˆ ( 0) (k ) = ⎨ (0 ) k = 1. ⎩ x (1),
(7)
3 Financial Distress Prediction Model This study adopts GreyART network and GM(1,1) model to construct the proposed financial distress prediction model. The inputs applied to GreyART network for classifying the financial health of a firm or bank are the historical financial data. Each resultant centroid indicates a subclass, rather than a class, and several different subclasses could make up each class. Once the training phase is completed, the final clustering results can be used to indicate the distressful conditions in currently viable banks. 3.1 Data Collection and Variable Selection
The data used in this study are collected from the Taiwan Economic Journal (TEJ) Data Bank [12] covering the period 2004-08. Two classes of listed banks are drawn: a
Financial Distress Prediction Model via GreyART Network and Grey Model
95
class identified as financially distressed banks and a class identified as financially healthy banks, namely the ‘distressed class’ and the ‘healthy class,’ respectively. Additionally, the dataset is divided into two separate sets. Data for the 4th quarter of 2004 to 2006 are used as the training set to train the GreyART network, while data for the 4th quarter of 2007 and the 2nd quarter of 2008 are used as the testing (validation) set. The training set contains 27 banks with 5 banks in the distressed class, while the testing set consists of 21 banks with 1 bank classified as in distress in each year. Seventeen financial ratios used in this study are collected from TEJ Data Bank, which reflect the following five dimensions: structure, management, solvency, growth, and profitability. Table 1 contains a detail description of all selected variables, where xi,j represent the raw data of the jth financial ratio for the ith bank. 3.2 Data Preprocess
To reduce the number of training patterns, data for the training set are generated as follows. Let xi,j(t) represent the raw data of the jth financial ratio for the ith bank at year t, and yi = (yi,1, yi,2, …, yi,N) denote the generated pattern of the ith bank. Then each element of yi can be obtained by the next two strategies: i) Healthy class: If the ith bank is in the healthy class, then its jth element is the average value of all corresponding ratios in this bank, i.e.,
(
)
yi , j = xi , j (t1 ) + xi , j (t 2 ) + " + xi , j (t k ) k ,
(8)
Table 1. Description of research variables Dimensions Structure Management Solvency
Growth
Profitability
Description Debt ratio stockholder / total assets Total assets turnover Equity Turnover Current ratio Revenue growth rate Income from operation growth rate Earning before taxes growth rate Total assets growth rate Stockholder’s equity growth rate Total assets turnover growth rate Return on asset Return on equity rate Gross profit rate Operating profit rate Earning before taxes rate EPS (earning per share)
Denotation x i,1 x i,2 x i,3 x i,4 x i,5 x i,6 x i,7 x i,8 x i,9 x i,10 x i,11 x i,12 x i,13 x i,14 x i,15 x i,16 x i,17
96
M.-F. Yeh, C.-T. Chang, and M.-S. Leu
ii) Distressed class: If the ith bank is in the distressed class, then its jth element is the value when the bank occurred the financial distress, i.e., yi , j = xi , j (t d ),
(9)
where td denotes the year that the ith bank occurred the financial distress. After generating the input patterns by (8) or (9), we uses the data preprocessing method proposed by Hsia and Wu [13] to enable each input pattern to be comparable and, at the same time, normalize each pattern. The data preprocessing formula for yi,j is stated as follows. ⎧ yi , j − Δ j ,min , if the larger, the better ⎪ ⎪ Δ j ,max − Δ j ,min ~ yi , j = ⎨ ⎪ Δ j ,max − yi , j , if the smaller, the better ⎪ Δ j ,max − Δ j ,min ⎩
(10)
where Δj,max = maxiyi,j and Δj,min = miniyi,j. After the data preprocessing, we can guarantee that 0 ≤ ~ yi , j ≤ 1. Note that, in Table 1, a financially healthy banks is normally to pursue higher ratios, except the debt ratio (xi,1). 3.3 Learning and Testing Phases
In the learning phase, the inputs applied to GreyART network are those training patterns after data preprocessing. While the learning phase of the network is completed, some templates could be obtained and regarded as the cluster centroids. Thereafter, the cluster centroids are respectively recognized by auditors, experts, or managers to label them as ‘health’ or ‘distress’. Assume that cα,β represents the cluster centroid of the βth cluster belonging to class α. Then an input pattern, say ~ y i , is in * the class α if g (c ,~ y ) = max{g (c , ~ y )}. (11) α ∗ ,β ∗
i
α ,β
α ,β
i
That is to say, an input pattern is in the class that one of its subclasses has the largest relational grade. In the testing phase, all resultant cluster centroids obtained in the learning phase remain changeless and only the first step of GreyART algorithm is required. Each testing pattern is identified as the class that the corresponding template belongs to. The testing patterns described in Sec. 3.1 have never been used to train the network in the training phase. They could be used to post-examine the ‘generalization’ ability of the network. Since those patterns are actual values, the classification results cannot actually reflect the ‘forecasting’ ability for the case of one- or two-year-ahead. As indicated in [14], constructing a GM(1,1) model requires at least four data, i.e., x (0) = (⊗, x(0)(2), x(0)(3), x(0)(4)), where ⊗ represents a arbitrarily positive number. Let x(0)(k) = xi,j(k+2002), k = 2, 3, and 4. That is to say, the raw sequence x (0) in this study is formed by three ‘historical’ data: xi,j(2004), xi,j(2005) and xi,j(2006). Then using (5) to (7) could obtain one- and two-step ahead forecasting values of x (0),
Financial Distress Prediction Model via GreyART Network and Grey Model
97
i.e., xˆi , j (2007) = xˆ (0 ) (5) and xˆi , j (2008) = xˆ ( 0 ) (6). Once the forecasting values are obtained, those forecasting data are regarded as the testing patterns of GreyART network. By this way, the corresponding classification results can actually represent the forecasting results of the proposed financial distress prediction model. Fig. 2 demonstrates the training and testing accuracies with respect to the vigilance threshold, where the number above the point on the training accuracy indicates the number of clusters resulted from that threshold and the testing accuracy includes the results of the actual and forecasting values. As can be seen, the training accuracy is 96.30% for each vigilance threshold. Also, the larger the value of the threshold, the more the created clusters. In one- or two-step ahead prediction case, generally speaking, the testing result with the actual data as inputs (the generalization ability) is better than that with the forecasting values as inputs (the forecasting ability). Besides, the best forecasting result is 57.14% in both cases. It is a worse result. This study therefore proposes a growing extraction method to extract some significant variables from the given 17 variables to improve the forecasting ability. 3.4 Growing Extraction Method In order to overcome the trade-off between the clustering performance and the classification ability, this study utilizes the following index to evaluate the performance of the clustering results. Let Accuracy(ρ) denote the classification accuracy resulted from the vigilance threshold ρ and PQ(ρ) represent the partition quality caused by the same threshold ρ, where the partition quality (PQ) is defined in Hamerly and Elkan [15] as
Fig. 2. Training and testing accuracies with 17 financial ratios
PQ =
nclass ncluster
nclass
α =1 β =1
i =1
∑ ∑ p(α , β ) 2 ∑ p(α ) 2 ,
(12)
98
M.-F. Yeh, C.-T. Chang, and M.-S. Leu
where nclass is the true number of classes, ncluster is the number of clusters found by the clustering algorithm, p(α,β) and p(α) represent the frequency based probability of a point vector in cluster β belonging to class α, and the actual class probability, respectively. Then, the performance index (PI) is defined as follows. PI ( ρ ) = 100 × Accuracy( ρ ) × PQ( ρ ).
(13)
Among those thresholds, the one with the maximal performance index is termed an optimal threshold in this study. The corresponding cluster centroids and the number of clusters are the optimal results in the sense of the performance index (13). The proposed growing extraction method starts with the training input containing only a single variable. That is to say, in the beginning, the input pattern used to train yi = (~ yi , j ), j = 1, 2, …, 17. Hereafter the jth the GreyART network is of the form ~ variable is termed the active variable. Denote the performance index due to such an input form and a specific vigilance threshold ρ as PIj(ρ). Among all possible active variables, the extracted variable is the one with the maximal performance index. In other words, if the j*th variable is the extracted variable, then it must satisfy PI j∗ ( ρ ) = max j PI ( ρ ).
(14)
Assume that the first extracted variable is the ‘return on equity rate’, i.e., j* = 13. To extract the second variable from the remaining variables, the form of input pattern y i = ( ~yi ,13 , ~ yi , j ), j ≠ 13. According to (14) again, the second extracted becomes ~ variable can be determined. Assume that it is the ‘total assets growth rate’, i.e., j* = 9. Subsequently, the 9th variable is inserted into the second component of the input, i.e., ~ yi = ( ~ yi ,13 , ~yi ,9 , ~ yi , j ), j ≠ 9, 13, for consideration. The remaining extraction process could be done by the similar method. Since the number of components is successively increased, that is the reason why the proposed scheme is termed the growing extraction method. Besides, the final input form also could reveal the significance of a variable in order in which it is extracted.
3.5 Results and Discussion The growing extraction method given in Section 3.4 is under a specific vigilance threshold. To search of an optimal result, the vigilance threshold starts from 0.70 and increases by 0.01 in the simulations. Fig. 3 shows the training and testing accuracies corresponding to the vigilance threshold ρ with the value of 0.80. The ‘variable sequence’ given in the top figure represents the extraction variables in order in which they are given. The x-axis represents the number of variables, that is, if the number of variables is n, then it represents that the first n variables in sequence are selected. Besides, the number above the point on the training accuracy indicates the number of resultant clusters resulted from that number of variables. Fig. 3 reveals that no matter how large the number of extracted variables is, the training accuracy rate is 96.30%, i.e., one financially distressed bank is misclassified into the healthy class. However, the larger the number of variables, the more the created clusters. While using the forecasting data as the testing patterns, it can be seen that the forecasting ability with
Financial Distress Prediction Model via GreyART Network and Grey Model
99
fewer extracted variables is generally better than that with more variables in both oneand two-year ahead prediction cases. In our simulations, the conditions resulting in the optimal result are ρ = 0.80 and the inputs with the form constructed by the first four extracted variables (i.e., return on equity rate, total assets growth rate, stockholder’s equity growth rate, and revenue growth rate). Under these conditions, there are only two clusters, one for the healthy class and the other for the distressed class, being generated with the training accuracy of 96.30%. Both one- and two-year ahead prediction cases can attain the same forecasting accuracy of 95.24% (one distressed bank is misclassified into the healthy class) which is much better than the best forecasting result obtained by use of 17 variables as inputs, i.e., 57.14%. That is to say, the forecasting ability is indeed improved with the proposed variable extraction method.
4 Conclusions The proposed financial distress prediction model is based on GreyART network and grey model. In order to further improve the classification ability in the training and testing phases of GreyART network, this study also proposes a growing extraction method for financial variables to extract some important variables. Simulation results show that the best result the proposed method can attain is used by four extracted variables with the vigilance threshold of 0.80. For this case, there are only two clusters, one for each class, being generated with the corresponding classification hit rates of 96.30% and 95.24% for the training and forecasting results, respectively.
Fig. 3. Training and testing accuracies with the growing extraction method
100
M.-F. Yeh, C.-T. Chang, and M.-S. Leu
Acknowledgments. This work was supported by the National Science Council, Taiwan, Republic of China, under Grant NSC 98-2221-E-262-004.
References 1. Altman, E.I.: Financial ratios, discriminant analysis and the prediction of corporate bankruptcy. J. Financ. 23(4), 589–609 (1968) 2. Deakin, E.B.: A discriminant analysis of predictors of business failure. J. Account. Res. 10(1), 167–179 (1972) 3. Blum, M.: Fail company discriminant analysis. J. Account. Res. 12(1), 1–25 (1974) 4. Odom, M.D., Sharda, R.: A neural network model for bankruptcy prediction. In: Proc. Int. Joint Conf. Neural Netw., vol. 2, pp. 163–168. IEEE Press, New York (1990) 5. Atiya, A.F.: Bankruptcy prediction for credit risk using neural networks: a survey and new results. IEEE Trans. Neural Netw. 12(4), 929–935 (2001) 6. Kumar, P.R., Ravi, V.: Bankruptcy prediction in banks and firms via statistical and intelligent techniques – a review. Eur. J. Operation Res. 180, 1–28 (2007) 7. Tsai, C.F., Wu, J.W.: Using neural network ensembles for bankruptcy prediction and credit scoring. Expert Syst. Appl. 34, 2639–2649 (2008) 8. Chen, W.S., Du, Y.K.: Using neural networks and data mining techniques for the financial distress prediction model. Expert Syst. Appl. 36, 4075–4086 (2009) 9. Yeh, M.F., Chiang, S.S.: GreyART network for data clustering. Neurocomputing 67, 313– 320 (2005) 10. Deng, J.L.: Introduction to grey system theory. J. Grey Syst. 1(1), 1–24 (1989) 11. Carpenter, G.A., Grossberg, S.: ART 2: self-organization of stable category recognition codes for analog input patterns. Appl. Optics. 26(23), 4919–4930 (1987) 12. Taiwan Economic Journal Data Bank, http://www.tej.com.tw/ 13. Hsia, K.H., Wu, J.H.: A study on the data preprocessing in grey relational analysis. J. Chinese Grey Syst. 1(1), 47–54 (1998) 14. Li, B.Q.: Three-data modeling of grey system theory. J. Grey Syst. 2(1), 11–20 (1990) 15. Hamerly, G., Elkan, C.: Learning the k in k-means. In: Proc. 17th Ann. Conf. Neural Inform. Process. Syst. MIT Press, Cambridge (2003)
Risk Assessment Model Based on Immune Theory Tao Liu, Li Shang, and Zhifeng Hu Jiangsu Research & Development Centre for Modern Enterprise Information Software Engineering, Suzhou, China
[email protected], {sl0930,huzf}@jssvc.edu.cn
Abstract. As new field of computer intelligence research, Artificial Immune System inspired by biological immune system, provides a strong paradigm for information processing and problem solving. Artificial immune network theory is an important theory of AIS and has already wildly applied in the fields of data clustering, data analysis and robot control. This research proposes firstly to apply artificial immune network theory into risk assessment of managerial field. According to the comparability of the e-commerce risk problem and the biological immune system, presents specific model construction process relating to case study and testifies the result with data from real test. Keywords: Risk, Assessment, Model, Immune network.
1 Introduction This Artificial Immune Systems (AIS) are the information processing technology, computer technology and other intelligence systems based on the theory and application of biological immune system [1]. The study of AIS has become a focus in the research of intelligence system as well as neural networks, fuzzy logic and genetic algorithms. AIS provide a strong paradigm for information processing and problem solving in computer intelligence research [2][3]. Immune theory [4][5]suggested systematic hypothesis on the activity of immune cell, generation of antibody, immune tolerance, self and non-self identification, immune memory and immune system evolution and defined immune system as adjustment network composed by immune cell or molecule. Inspired by immune theory, researchers constructed many artificial immune model [6-8] and implied them into fields like data clustering, data analysis, robust controlling, computer safety, etc [1-2]. However, artificial immune theory is not used in risk fields. This research provides specific model construction process with the study of risk assessment in risk field. Theoretical analysis and test results show that it is an effective new method of risk appraisal.
2 Design of Immune Network Risk appraisal model proposed in this research is constructed on the immune network. In our adaptive immune response network (AIRN) [8], antigen is the original data and antibody is feature of pattern which displays itself as a data item in the network; the system structure is the population of antibody. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 101–108. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
102
T. Liu, L. Shang, and Z. Hu
2.1 Initialization of Network and Antigen Presentation The antigen in the system can be described in detail as AG
= (agi ...ag I )T , and agi
represents the i-th antigen; agij represents the j-th vector of the i-th antigen. And we have
agij ∈ [0,1]m , i = 1,..., I ; j = 1,..., m . The following vector is generated
randomly:
AB = (ab1 ⋅⋅⋅ abk )T , and abij ∈ [0,1]m , i = 1,..., k ; j = 1,..., m are the
new antibodies. In the expression, abi represents the i-th antigen, and abij represents the j-th vector of the i-th antibody. Designate each element in the antigen to each element in the antibody. For any antigen agi and any antibody abj,, the distance between the two can be calculated as follows: 1/ 2
⎛ m ⎞ d (i, j ) = ⎜ ∑ (agit − ab jt )2 ⎟ ⎝ t =1 ⎠
(1)
.
For any pair of antibodies abi and abj, the distance between the two can be calculated as follows: 1/ 2
⎛ m ⎞ s (i, j ) = ⎜ ∑ (abit − ab jt ) 2 ⎟ ⎝ t =1 ⎠ For any antigen
ab j (ab j ∈ AB, j = 1,
ag i (agi ∈ AG, i = 1,
(2)
.
, k ) , the stimulation sl(i, j) it receives from the
, I ) is:
⎧⎪1 d (i, j ) sl (i, j ) = ⎨ ⎪⎩0
d (i, j ) < Tsl
.
(3)
other
In (3), Tsl is the threshold value of stimulation from antigen to antibody, and I
k
Tsl = ksl ∑∑ d (i, j ) [( I × k ) / 2] , Tsl represents the threshold of the stimulation i =1 j =1
from the antigen to the antibody, and
ksl ∈ (0,1] is a constant.
2.2 Clone and Mutation When the stimulation to antibody from antigen arrives certain degree, the immune system is activated and begins to clone antibody. The number of cloned antibody abi stimulated by antigen agi is:
⎧⎪ ⎢ sl (i, j ) ⎥⎦ d (i, j ) > Tsl . c1 (i, j ) = ⎨ ⎣ other ⎪⎩0
(4)
Risk Assessment Model Based on Immune Theory
In (4):
103
⎣⎢i ⎦⎥ represents the closest natural number to sl(i, j). Abji is the set of clones of
the antibody abj as a result of the stimulation from the antigen agi, and the set of clones of the antibody abj as a result of the stimulation from the antigens is:
C j = Ab j1 ∪ Ab j 2 ∪
∪ Ab jc1 (i , j ) =
c1 ( i , j )
∪
Ab jt .
(5)
t =1
The corresponding set of antibodies AB generates the set of new antibodies C through cloning:
C = C1 ∪ C2 ∪
k
∪ Ct = ∪ Ct .
(6)
t =1
The clone cell needs mutation process to bestow the newly born cell higher affinity with the selected antigen. Suppose Mut (i, j ) = d (i, j ) is the new antibody mutation rate from antibody abj stimulated by antigen agi. Suppose the mutation set of C ′j , then antibody abj stimulated by antigen group is
c′j = c j + mut (i, j ) × (c j − agi ) .
Suppose the mutation set of antibody abj
stimulated by antigen group is C ′j , then the antibody set from mutation is:
C ′ = C1′ ∪ C2′ ∪
k
∪ Ct′ = ∪ Ct′ .
(7)
t =1
The mutation ensures the diversity in the newly generated antibodies. This is an inherent characteristic of the learning mechanism of the immune system. The system improves its recognition rate of antigens through repeated cycling of the above process. The clone suppression starts right after clone mutation is completed. For the set C ′ , the intimacy level between the antigen and the antibody and the antibody cells are computed again after the mutation. Meanwhile, the suppression threshold between the antigen and the antibody Td, and the suppression threshold between the antibodies, Ts , are computed as follows: Td p
constant,
= kd
1 p min iI=1 d (i, j ), kd ∈ (0,1] , and kd is a ∑ p j =1
p
Ts = k s ∑∑ s (i, j ) [ p × ( p − 1) / 2] k s ∈ (0,1] and ks is a constant. i =1 j =1
During the clone suppression stage, the antibody cells with higher values of the intimacy level (as compared with the threshold Td) are deleted at first, and then the memory cells with smaller intimacy levels (as compared with the threshold Ts) are deleted according to the intimacy levels between the antibody cells. The set of
104
T. Liu, L. Shang, and Z. Hu
antibodies upon the completion of this stage is C ′′ , whose elements satisfy the following conditions:
⎧ab j ab j = ⎨ ⎩0
∃agi d (i, j ) < Td
⎧ab j ab j = ⎨ ⎩0
∃abi s (i, j ) > Ts
other
other
.
(8)
.
(9)
After the stage of clone suppression, the new set of antibody C ′′ is smaller than C ′ . Clone selection is then carried out at this stage. Among the antibodies in C ′′ , ξ % of the mutated cells are chosen to act as the memory cells for the antibody, according to their intimacy level with the antigen. ξ % can be specified beforehand. The chosen mutated cells are then put into M, the set of memory cells. 2.3 Termination of Network Training Among the antibodies after clone mutation, the ones with higher antigen stimulation level morph into the memory cells with longevity, while the ones with lower antigen stimulation level die out as the antigens die out. Since the recognition among the antibody cells themselves could activate the autoimmune response, the antibodies that are capable of self recognition have to be suppressed each time after new antibody memory cells are added in order to suppress autoimmune. Denote the number of elements in the memory cell set M (M={m1,m2,…,mz; M ∈ ℜ }) as p1. Define the intimacy maturity threshold of the memory cell as: Z
p1
p1
Ts′ = k s′ ∑∑ s (i, j ) [ p1 × ( p1 − 1) / 2] .
(10)
i =1 j =1
In (10):
k s′ ∈ (0,1] and is a constant. Therefore, after the process of the intimacy
level maturity, the elements in the set of memory cells M should satisfy the following conditions:
⎧⎪ab ab j = ⎨ j ⎪⎩0
s (i, j ) > Ts′ other
i =1,2,..., p1 ; j =1,2,..., p1 .
(11)
During the network training, a certain number of iterations could be specified as the termination condition. As an alternative, the stimulation level between the antigen and the memory cell can be measured upon the completion of each iteration to serve as the termination condition. In other words, for any given antibody agi, an antibody abj can be found in the set of memory cells so that the condition d (i, j ) ≤ Ts′ is satisfied.
Risk Assessment Model Based on Immune Theory
105
3 Definition of Risk Measure Risk assessment of managerial field is quite similar with those in the biological immune system (BIS), both needs to sustain the stability of the system in a dynamic environment. Considering the corresponding relationship between antibody consistence and pathogeny invasion degree, to establish mathematical models of elements such as self body, non-self body, antibody and antigen, establish the risk appraisal model based on antibody consistence. With this model, the type and degree of attacks to risk can be calculated. There are diversified risks and different risk sources would bring different damages. Therefore, when assessing the risks, all the factors and their corresponding importance must be considered. Moreover, risk is calculated according to the antibody information under the stimulation of all kinds of risk factors. When there is no risks, number of antibody is stable; when risk increases, number of antibody increase sharply and vice versa. Thus risk can be judged by the number of antibody stimulated by risk factors. Suppose r or R is the facing risk and 0≤r(R) ≤1. when r(R) =1, it indicates the most great risk for current system; when r(R)=0 means no risk for current system, and bigger r(R) implies higher risks. The degree of r(R) can provide information for antirisk policy-making. Define ni is the number of antibody caused by i risk found by the system, βi is the degree of danger of i attack, ci is the number of antibody under normal network. Risk calculation of the system under i risk:
ri = 1 −
1 . 1 + lg( ni − ci + 1)
(12)
Risk calculation of the whole system:
Ri = 1 −
1 1 + lg( β i ∑ ( ni − ci ) + 1)
.
(13)
i
There exist many indicators which are hard to explain exactly in the risk appraisal system. However, this research solved the problem.
4 Case Study This research used e-commerce risk assessment as an example. E-commerce risk appraisal indicator system is a serial of integrated indicators which are inter-related and can sensitively reflect the situation and problems. The design of the system follows principles of sensitivity, flexibility, integrity and with combination of quantitative and qualitative indicators. With the literature review, this research constructs e-commerce risk appraisal indicator system based on AIRN theory Risk quotient of e-commerce risk is the top level of the system (A). And then the ecommerce risk appraisal. indicator system has four first level indicators: indicator for
106
T. Liu, L. Shang, and Z. Hu
risk caused by the incompleteness of national control system (B1), indicator for the risk in the food production and processing channel (B2), indicator for the risk in the food distribution channel (B3) and indicator for the risk in the food consuming process (B4). All these four first level indicators can be assessed by its second level indicators (C) respectively. Risk appraisal indicators are in Table 1. Table 1. E-commerce risk appraisal indicator system
Final Result (A)
Risk quotient of e-commerce
First Level Indicators (B)
Second Level Indicators (C)
Risk in network(B1)
Physical security (C11) Threaten of network virus(C12) attack of hacker (C13)
Risk caused by transaction (B2)
Validity of electronic contract (C21) Flaw of electronic payment (C22) Preservation of virtual property (C23)
Risk in supply chain collaboration (B3)
Risk in law (B4)
… …
price fluctuations (C31) Uneven in profit distribution (C32)
…
confusion caused by unification policy (C41) Non-performance of market regulatory functions (C42) Financial services imperfect (C43)
…
The value of risk appraisal indicator can be decided by expert evaluation and statistic. Expert evaluation is to select a certain number of consultants and experts who would appraise every indicator according to its level and give specific reference value; statistic method would refer to all the data and draw the value of risk indicator by statistical instrument. In this research, 5 indicators such as Law and Regulation and Operation of Government Body are decided by expert method while the others are by statistic method. Relevant e-commerce risk indicator data from 2008 to 2009 in a company are divided into 45 groups. After test, these data is normally distributed. The other data are also generated and satisfy the normal distribution. Firstly, normalize data. Then calculate the risk indicator of e-commerce risk using the presented model. Threshold value of immune network is set as ksl=1,kd=1,ks=1,k’s=1. Fig.1(a) is the threat degree of risk happened in the attack of hacker; Fig.1(b) is the threat degree of risk happened in the confusion caused by unification policy; Fig.1(c) is the e-commerce risk degree appraisal when there is no threat for risk sources; Fig.1(d) is the e-commerce risk degree appraisal when there is threat for risk sources.
Risk Assessment Model Based on Immune Theory
107
Fig. 1. (a) Threat degree of risk only caused by network virus; (b) Threat degree of risk only caused by Flaw of electronic payment; (c) E-commerce risk degree appraisal when there is no threat for risk sources; (b) commerce risk degree appraisal when there is threat for risk sources
From Fig.1(a)- (d), it can be learned that risk appraisal model based on AIRN can reflect the risk of e-commerce risk and dynamically calculated its degree. Compared with neural network appraisal, this appraisal does not need the training sample of risk in advance. While compared with ambiguous appraisal, this can avoid subjective judgment in the process.
5 Conclusions Artificial immune theory has developed into a new branch of computer intelligence research. This research explores the feasibility of constructing risk assessment model based on immune network theory, designs plans for risk assessment and constructs ecommerce risk assessment model with the study of risk assessment in e-commerce risk field. The result indicated that this model is practical, reliable and capable of providing strategic decision-making tool for policy maker of e-commerce risk supervision. Further, this research abstracts and expands the conception of antibody consistence in biological immune system and applies into e-commerce risk appraisal. Lastly this research establishes e-commerce risk appraisal model based on immune network. Test results show that this method is an effective new method of ecommerce risk appraisal. The risk appraisal model in this research has deep influence to self-nonself, antigen, antibody, immune cell, clone selection, immune memory. It can not only be applied in the e-commerce risk appraisal and restructuring of mathematic model but also in pattern recognition and combinatorial optimization.
Acknowledgment This research was supported by the grants of the National Science Foundation of China (No.60970058), the grants of Natural Science Foundation of Jiangsu Province
108
T. Liu, L. Shang, and Z. Hu
of China (No.BK2009131) and is also sponsored by Qing Lan Project of Jiangsu Province.
References 1. Timmis, J., Hone, A., Stibor, T., Clark, E.: Theoretical Advances in Artificial Immune Systems. Theoretical Computer Science 1, 11–32 (2008) 2. Dasgupta, D.: Advances in Artificial Immune Systems. Computational Intelligence Magazine 4, 40–49 (2006) 3. Castro, D.E., Zuben, V.: The Clonal Selection Algorithm with Engineering Applications. In: The Genetic and Evolutionary Computation Conference, pp. 36–37. Morgan Kaufmann Publishers, San Fransisco (2002) 4. Jerne, N.K.: The immune System. Scientific American 1, 51–60 (1973) 5. Jerne, N.K.: Towards a Network Theory of the Immune System. Annual Immunology 125C, 373–389 (1974) 6. Li, Y.Y., Jiao, L.C.: Quantum-inspired Immune Clonal Algorithm. In: 4th International Conference on Artificial Immune Systems, pp. 304–317. IEEE Press, New York (2005) 7. Mo, H.W., Zuo, X.Q., Xu, L.f.: Immune Algorithm Optimization of Membership Functions for Mining Association Rules. In: Garijo, F.J., Riquelme, J.C., Toro, M. (eds.) ICNC 2006. LNCS, vol. 4222, pp. 92–99. Springer, Heidelberg (2006) 8. Tao, L.: Study on Risk Management of E-business Based on Immune Principle. Report of Post Doctorate, Huazhong University of Science and Technology (2007)
Short-Term Load Forecasting Based on Bayes and RS Yanmei Li School of Business and Administration, North China Electric Power University, Baoding 071003, China
[email protected]
Abstract. In the many influencing factors of load forecasting, some are related to each other, and some are independent, so it is not very necessary of all factors to the conclusions. The paper adaptively selects input features making use of Bayes method and Rough Set, and chose a group of input characteristic set which report most out the reason of output change to train the BP Neural Network model for forecasting, which has been testified to be valid. Keywords: Bayes, Load Forecasting, Rough Set, BP Neural Network.
1 Introduction When building up the short-term load forecasting model, we should first make sure the input of the model.The factors and datas which influence the forecasting result are very much, and not all the influencing factors are necessary.In the factors, some are interrelated, and others are irrelated. How to select the factors from the large numbers of infecting factors and datas which have strongger influences on forecasting value and constitute an effective inputting variable set function are the chief matter before establishing the forecasting model. The paper selects the inputting characteristic value of forecasting model through Bayes and RS to predigest the inputting characteristic values. After simplfying and abstracting the samples, making sure the number of neurons of the inputting values,then optimizing the neurons of hidden layer, through which the training speed and accuracy can be increased. The method has been proved to be valid through an example.
2 Bayes Decision Theory Bayes decision theory concept and method are used for fields such as engineering, management science already by broad field, adaptively selects input features step with Bayes method as follows: 2.1 Ascertains a Priori Probability Distribution The priori probability P (ω j ) represents the estimation to the probability distribution of the variable ω j , it has reflected a priori knowledge to the variable, has included Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 109–116. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
110
Y. Li
practical experience and subjective judgment etc. And that a priori probability scatters in load forecasting , is what be remained to be chosen influencing factor and their probability aggregation , restricting condition is whose necessary probability greater than zero, whose combination is 1. The load at the same time, forecasting (namely "big close small distant" distance according to load forecasting nearest segment of period affects maximal) and "identical date load type similarity" characteristic, the priori probability distribution is Pper = αP dis ⎧ ⎪ P equ = βPdiv . ⎪⎪ P per = γPequ ⎨ ⎪ P + P + P + P =1 ⎪∑ per ∑ dis ∑ equ ∑ div ⎪⎩ Pper > 0, Pdis > 0, Pequ > 0, Pdiv > 0
(1)
In the formula, Pper and Pdis represent the distance from influencing factor of forecasted point far or close; and respectively, Pequ and Pdiv represent the priori probability of influencing factor that having identical or different date type; Take α , β , γ value range being (1, ∞ ) , may look at concrete conditions but fix. When α = 1, β = 1, γ = 1 , the priori probability distribution is an uniform one. 2.2 Ascertains a Likelihood Function The likelihood function has been a condition probability essentially it has reflected sample information, whose function value has been called likelihood rates, has been.
(
)
P x1 , x 2 ," , x N ω j . During the period of load forecasting, if the input variable is selected as ω j , we may make use of N samples x n ( n = 1, " , N ) to calculate the Error E according to the follow formula. Ei =
Rvalid _ j [ f train ] =
1 n ∑ Rvalid _ j [ f train ] . n j =1
1 N valid
∑
x j ∈Z valid
[y
j
(2)
( )] × 100% .
− f train x j yj
(3)
Rvalid _ j [ f train ] be relative mean error in style, from training collection to get regression function f train relativity on effective set; N valid be an effective set Z valid all together sample book number; y j be that actual load value; f train x j be forecasting load value.
( )
Short-Term Load Forecasting Based on Bayes and RS
111
2.3 Calculates Posterior Probability
(
)
The formula of posterior probability P ω j x1 , x 2 ,", x N is
(
)
P ω j x1 , x 2 , ", x N =
(
)( ) . ω )P (ω )
P x1 , x 2 ," , x N ω j P ω j
∑ P(x , x j
1
2
i =1
,", x N
j
(4)
j
Among them, a priori probability and likelihood function action input vectors, a posteriori probability is output vector. Because a posteriori probability has synthesized a priori knowledge and sample book information, ultimateness being to decide which group of influencing factors to choose being input vector is standard. A posteriori probability is increasingly big, the probability that input vector is pitched on is increasingly big. 2.4 Chooses Input Vector ω j According to the actual characteristic being unlike area load, choose M may affect bigger factor composition to load waiting for choosing influencing factor, again out of, the random chooses different influencing factor combination, forms input vector ω j .
3 Rough Sets Reduction Algorithm 3.1 Use the Dicernibility Matrix to Get the Least Reduction The Discernibility Matrix is proposed by the Poland Warsaw famous mathematician Skowron. Utilizing this tool, we can give expression to the whole cannot distinguish relations which consist in the complicated information system. One presupposition to use the dicernibility matrix to get the least reduction is to deal with the antipathic register at first at the data in table pretreatment stage, that is, the discermibility matrix not deal with the incompatibility record. With the pretreatment methods, if the records of the conflict by dividing the number of the total number of records, we can get a measurement of the roughness and this measurement can be a character of the data sheet. Suppose S = (U , R , V , f ) is a information system, R=CYD is property collection, subset C = {ai / i = 1,2, " , m} and D = {d } is Conditional attribute set and decisionmaking attribute set, U = {x1 , x2 ," , x n } is on domain, a k (x j ) is sample, on the
j
a k . Definition system get the dicernibility matrix is
x j is the value
[ ]
M (S ) = mij
n×n
, it’s
i and
the list element is ⎧a k ∈ C , a k (xi ) ≠ a k (x j ) ∧ D(xi ) ≠ D(x j ) . mij = ⎨ i, j = 1,2,", n ⎩ φ , D( xi ) = D(x j )
(5)
112
Y. Li
So in the dicernibility matrix the element collection to the
mij is capable of distinct the whole
xi and x j ; But if xi and x j belong to the same decision-making
class, the value of the
mij is φ in the dicernibility matrix.
For every dicernibility matrix M (S ) corresponding the only distinguish function
f M ( S ) (a1 , a 2 , " , a m ) = ∧{∨ mij ,1 ≤ j < i ≤ n, mij ≠ φ }.
(6)
3.2 Reduction Algorithm Based on the Attribute Dependability Strike all about a history of NP-hard problem, so enlightening information to use simplified calculation to identify the optimal or sub-optimal reduction can be achieved is obviously a method. Many basic steps of heuristic algorithms are incepted by the nuclear of the information systems or table, then some measure of the importance of attributes, choose the most important attributes to the nuclear, until the termination conditions, and we have the information systems or a decision table reduction. To be more exact, includes a reduction of the attribute set. A information system for all the attributes of the decision-making is not as important as in the rough set theory, attribute importance of the dependencies can be reflected. Decision-making property D to the property R (R belongs to C), the dependencies γ (R, D ) defined: γ ( R, D ) =
card (POS R (D )) card (POS C (D ))
(7)
Obviously, 0 ≤ γ (R, D ) ≤ 1, γ (R, D ) give a measure of the D decision-making on the attributes of the dependencies. It reflect the importance of the property R to D. Under the premise known conditions R, a property a ∈ C − R to the decision-making D’s importance SGF (a, R, D ) may be defined as follows: SGF = (a, R, D ) = γ (R + {a}, D ) − γ (R, D )
(8)
SGF = (a, R, D ) reflect that after property a plus R, the rise degree of the dependence between R and D. And in fact, the stronger for the impact of a property attribute R and D dependencies, the greater value of SGF = (a, R, D ) is. 3.3 Reduction Algorithm Based on Conditional Information Entropy This reduction algorithm does not require distinction matrix, directly from the point of view of information theory, the idea of entropy calculated using various attributes the importance of the load. Reduction act and algebra is the same in the information entropy reduction in the importance of also need to set a threshold value greater than the threshold value of the property that is affected relatively large load factors, and
Short-Term Load Forecasting Based on Bayes and RS
113
less than the threshold value of the attribute is considered to be redundant and could be deleted. In this method for the calculation of the im nt attribute set, that is, the various factors affecting load; D represent decision-making set, that is, the load of the forecast day; RC, RD represent separately the equivalence relation in the decision-making domain of the attribute set conditions C and portance of attributes ai: I (a i , D
C represent D;
)=
⎡R ⎤ H⎢ D ⎥ ⎣ RC ⎦
⎡ ⎡ RD ⎤ ⎡ RD ⎤⎤ ⎡RD ⎤ ⎢H ⎢ ⎥ − H ⎢ ⎥⎥ / H ⎢ ⎥ ⎣ RC ⎦⎦ ⎣ RC ⎦ ⎣ ⎣ RC−A ⎦
(9)
represent RD to RC’s condition entropy.
3.4 Attribute Reduction Based on Genetic Algorithm Adaptive genetic algorithm is a method of random search, the search methods, not by a unitary structure or direction, it will more than individual as a possible solution and to consider the search space within the scope of the overall sample, thus leading to the possibility of greater convergence Global Optimal Solutions, Therefore, it was the introduction of the genetic algorithm Rough Set Attribute Reduction. Algorithm by using computer simulation of biological evolution, groups continues to be optimized, in the process of change and to identify the optimal solution. In the genetic algorithm, the design of fitness function is the core of the entire GA algorithm steps, as several genetic operator rely on the adaptation of the chromosome and therefore fitness function of design objectives, determines to a large extent iteration convergence direction, but the rough set of attribute reduction is to achieve the smallest reduction attribute set. In this way, the attribute set to meet in ensuring the accuracy of certain circumstances, at least to the number of attributes that ultimately the results that are needed to meet the classification requirements of the attribute set. Therefore, the design function to the ultimate goal should contain the following two objective classification must meet quality requirements usually must be about SR. function: covered by the reduction of the number of properties to minimize. Then provisions for the fitness function: score(r ) = m − l r + k m R for which chromosome the corresponding attribute set, D for decision-making attributes, the attribute set dependence on R. This function will attribute dependence on the introduction of fitness function, and rely on the attributes of that decision attributes of the corresponding attribute set chromosome depended reflects the attributes of taxonomic capacity.
②
①
4 Case Analysis Artificial neural network has been applied extensivly in load forecasting and decisionmaking because its strong parallel processing function, approximation to arbitrary
114
Y. Li
function, self-learning, self-organization capabilities and self-adaption capabilities, and also consider many influencing factors such as climate, temperature etc. However,ANN has some deficiency and shortage. The learning speed is fixed, so the convergence rate is low and the network need longer training time. For some complicated problem, the more inputting nodes will increase the complexity of the network., which will extend the training time. Although the weight can be converged into some value, it is not necessary the minimum one of the error plane, maybe it will produce a local minimum. Rough Set may eliminate some redundancy through reduction, which is in favour of the classification and simplification of the information, but the calculation is based on some basic concept such as approximation, plus field, minus field and boundary field etc, which will bring deviation by boundary problem. The BP neural network has strong fault-tolerance, but it has no ability of screening and compression on priori knowledge,so the RS and the BP nework are complementary.The paper reduces the inputting variables og BP network by Bayes and RS, then selects training samples and bring the to the network to train, and the trained network will be used to forest. In this paper, a southern city of the 2005 data were analyzed to August 10, 2005 to August 19 points for the entire sample for the study and active load to August 20 for the entire load for the test samples were forecast in August on the 21st load. 55 selected by experience on the condition variable attributes, which is 12 load data, that is, on the 10th to the 19th day the whole point of load; The remaining 43 non-load data, including weather, the date type, sunshine duration, maximum temperature, minimum temperature, average temperature, the biggest humidity, humidity, such as minimum 43 factors, including the date and type of rest days (including weekends and the statutory rest), the weather conditions on the provision of meteorological information is divided into 17 types. The neural network input vector, using the above-mentioned were all rough intensive SR algorithm primaries to the impact of load reduction factors. Table 1 lists a variety of reduction algorithm results. Can be seen, after attribute reduction, the input vector be simplified. Table 1. Comparision of using single attribute reduction method
Method Before reduction
The left variables 42
Hidden layer units 12
Time/min
RSME/%
54
2.7317
Bayes Discernibility matrix Conditional information entropy
30
10
13
2.1478
35
11
12
1.9956
19
14
4
1.3234
Attribute dependability
28
8
10
1.6535
Genetic algorithm
27
13
8
1.8718
Short-Term Load Forecasting Based on Bayes and RS
115
Table 1 has carried out a form from the function forecasting accuracy and training time the forecast to Table 3 kinds method in two aspects comparing, has forecast accuracy among them adopt average proportional error (η ) to be the analytical index , ( P − P ) ⋅ 100 ⎤ ⎡ ⎢∑ ⎥ whose definition as follows: P ⎣ ⎦ Among them, PAi be to forecast Ai
η =
Fi n
n
value, PFi be actual value, n is to forecast number of times. We can see from Table 1, making use of Bayes reduction, the input variables reduces from 43 to 31, calculating time reduces to 13min, the average relative error decrease from 11.11% to 5.82%, not only the attribution was reduced in the maximal degree, but the computing time is shortest relatively, and the calculation error is minimum. It is shown from the training time and accuracy,. The method in the paper is better than traditional BP Neural Network, so it is suitable for the forecasting samples in this area. First of all input and output variables on the regulation of a processing, data processing interval [0,1] between the data. Regulation is one of the many ways here by the following formula: xˆ =
x − xmin . x max − x min
In a 3-layer BP neural network model to predict the hidden layer neurons of the first set to the number of n1 = n + m + a , Among them, m is the output neuron number, n is the number of input neurons, a [1,10] between the constants. Network layer neuron transfer function using S-type tangent function, the output layer neuron transfer function using S-type logarithmic function, because the output function in the interval [0,1] between the output precisely meet the requirements. The learning algorithm uses a faster convergence Levenberg-Marquardt dynamic numerical optimization algorithm. Training error indicators set to 0.01, taking into account the network structure more complicated, the number of training set for 1000, at last we get the comparison results between the way adopted in the paper and traditional BP Neural Network. It shows from the Table 1 that comparing with traditional BP Neural Network, the method based on Bayes and RS not only improves forecasting accuracy, but also iecreases training period significantly. In all the reduction ways of the paper, the conditional information entropy method has the least time and the high precision, and also the minimum forecasting error, which shows that the method put forward in the paper has some application prospect. The simulation data above is all calculated on a same computer with main frequency 2.55GHz and memory 1GB.
5 Conclusion This paper has been submitted one kind of short-term load forecasting method based on Bayes and RS. On the premise of every factor data test result being indicated, being method's turn to be able to think that the forecast is connected with load in synthesis, reach higher forecast accuracy, and be that one kind of effective short period load forecasts method within shorter training time.
116
Y. Li
References 1. Fang, R.: Short-term load forecasting of power system with rough sets and support vector regression. Journal of Huaqiao University (Natural Science) 28, 252–255 (2007) 2. Zhu, W., Shan, M.: A neural network model of risk analysis for public construction project. Journal of Harbin Engineering Unicersity 27, 142–147 (2006) 3. Ying, J., Hua, G., Liu, Y.: Power syetem load forecasting based on v-support vector regression. Journal of Northeast Danli Unicersity (Natural Science) 27, 43–47 (2007) 4. Chen, S.: Several attributes reduction algorithm analysis based on rough set. Wuhan Institute of Industrial Journal 24, 118–121 (2005) 5. Wang, L., Li, Z., Wen, J.: Short-term load forecasting based on support vector regression. Relay 33, 17–21 (2005) 6. Pan, F., Cheng, H., Yang, J.: Power system short-term load forecasting based on SVM. Power System Technology 28, 39–42 (2004) 7. Jiang, Y.: Multi-factor forecast models based on rough sets and BP neural network. Computer Engineering 33, 154–155 (2007) 8. Zhang, Q., et al.: Short-term load forecasting based on rough sets attribute reduction algorithm and support vector machine. Grid Technology 30, 56–60 (2006)
Nonlinear System Modeling with a New Fuzzy Model and Neural Compensation Cruz Vega Israel and Wen Yu Departamento de Control Automtico, CINVESTAV-IPN A.P. 14-740, Av.IPN 2508, Mxico D.F., 07360, Mxico
[email protected]
Abstract. Normal fusion procedure of neural networks and fuzzy systems is to use neural learning techniques to train membership functions of fuzzy system. If no mechanistic prior knowledge can be used, fuzzy systems should be obtained from a set of data. The same data are usually used to train these fuzzy systems in the framework of fuzzy neural networks. But modeling accuracy cannot be improved extraordinary, because neural training and fuzzy modeling use the same data set. In this paper, we propose a new modeling idea, the fuzzy system will not be changed, and modeling error between real plant and the fuzzy system is compensated by a neural network. Another contribution of this paper is fuzzy model is generated automatically by kernel smoothing technique. The third contribution of this paper is a new learning approach for neural compensator is proposed, which assures stable and faster learning. Keywords: fuzzy modeling, neural compensation, satbility.
1
Introduction
Fuzzy system is a popular intelligent modeling method , which cannot only model linguistic variables based on fuzzy rules, but also is simple and highly intuitive , especially for complex industrial processes. But fuzzy model did not seem good under testing conditions for the sake of inadequate experimental data. The production rules cannot produce a good precision in case of lack of information. The structure (number of fuzzy rule) also influences identification capability and accuracy [1]. Resent results show that the fusion procedure of neural networks and fuzzy systems seems to be very effective for nonlinear system modeling [2]. A general combination method is to use neural training for a fuzzy model [3]. From prior knowledge, the structure of fuzzy system can be determined. After neural training, the membership functions of fuzzy systems are changed to match the training data. The process of fuzzy rule extraction for nonlinear systems modeling is called structure identification. Two popular methods can be used to construct fuzzy systems: 1) from experts [4], 2) from observed data [5]. In many industrial processes, no mechanistic prior knowledge can be used to construct fuzzy systems. It is difficult to obtain mathematical models for these process. A common approach is to partition input and output data, it is also called fuzzy Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 117–127. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com
118
C.V. Israel and W. Yu
x (k )
Plant
y (k )
Final model: yˆ (k ) + Δˆ (k ) Δ (k ) e (k )
Fuzzy model
yˆ (k )
G1 ω1 G2
Kernel smoothing
Gm
ω2
Δˆ (k )
ωm RBFNN
Fig. 1. Fuzzy model with neural compensation
grid [3], such as fuzzy C-means clustering [6], mountain clustering [6], and subtractive clustering [7]. Besides clustering approaches, fuzzy models can also be obtained by neural networks method [3], genetic algorithms [4], SVD-QR [8] and support vector machine (SVM) technique [9]. Kernel smoothing is an alternative modeling technique [10], it can find a regression function such that function is best-fit match to data at those data points. Normally, modeling accuracy of kernel smoothing is better than clustering methods, it is faster than SVM and SVD-QR. And also kernel smoothing method is very simple. The idea of kernel regression is putting a set of identical weighted function called Kernel local to each observational data point. The kernel will assign weight to each location based on distance from the data point. The kernel basis function depend only to the radius or width (or variance) from the ‘local’ data point X to a set of neighboring locations x. Kernel regression is a superset of local weighted regression and closely related to moving average and K nearest neighbor (KNN), radial basis function (RBF), neural network and support vector machine (SVM). The first successful kernel smoothing approach is NadarayaWatson estimator [16], which used kernel as a weighting function to estimate f as a locally weighted average. Backpropagation and gradient algorithm are always used for neural network training. Slow convergence and local minimum are main drawbacks. Generally, we have to make some modifications such that learning processes are stable [12]. By using passivity theory, a time-varying learning rate is stable without robust modification [2]. Although the stability of neural identification is guaranteed, the optimal properties are lost. Sometimes they cannot even reach local minima. The time-varying rate defined as η (k) = kc (c is a small positive constant) can give optimally fast convergence in the sense of the misjudgment going to zero proportional to k1 , see [14]. But it results in slow convergence to bad solutions when c and k are small. Therefore, the ”search then converge” approach is pro1 posed by [13], where η (k) = η0 2 . But these two kinds of algorithms c 1+a1 k +a2 ( kc ) cannot guarantee stability.
Nonlinear System Modeling with a New Fuzzy Model
119
In this paper, a new multiple fuzzy models is given, see Fig1. First we use a windows with length of N to collect data, a kernel smoothing support method is applied to obtain an approximator in the time interval [τ1 , τ2 ]. Since the kernel smoothing model has similar form as a fuzzy system, fuzzy rules are constructed and the corresponding fuzzy systems are made. If the same data are usually used to train these fuzzy systems. The modeling accuracy cannot be improved extraordinary, because membership functions training training and fuzzy structure generation use the same data set. In the second stage, the fuzzy system will not be changed, and the modeling error between the real data and fuzzy system will be compensated by a neural network. Then we propose a new learning rate, η0 such as η (k) = 1+a1 c +G(x . It takes the advantages of [2] and [13] and assure k ) k stable and faster learning.
2
Generating Fuzzy Systems via Kernel Smoothing
Consider a nonlinear discrete plant represented by y(k) = f [x (k) , θ] + e (k) T x (k) = [y (k − 1) · · · , y (k − ny ) , u (k − d) , · · · , u (k − d − nu )] ∈ n×1
(1)
and f (·) is an unknown nonlinear difference equation representing the plant dynamics, u (k) and y (k) are measurable scalar input and output, d is the delay, θ is an unknown parameter vector associated with an appropriate but yet to be determined model structure, e (k) is a bounded observation noise, ny and nu are the lengths of output and input, ny + nu = n. In fact, (1) is NARX model [22]. A generic fuzzy model is presented as a collection of fuzzy rules in the following form (Mamdani fuzzy model [15]) Rj : IF x1 is Aj1 and x2 is Aj2 and · · · xn is Ajn THEN y is B j
(2)
We use l(j = 1, 2, · · · , l) fuzzy IF-THEN rules to perform a mapping from the input linguistic vector x = [x1 , · · · , xn ] ∈ n to the output y. From [1] we know, by using product inference, center-average and singleton fuzzifier, the output of the fuzzy logic system can be expressed as ⎛ n ⎞ ⎛ l n ⎞ l y = ⎝ wj μAj ⎠ / ⎝ μAj ⎠ = W ϕ [x (k)] (3) j=1
i=1
i
j=1
i=1
i
where μAj is the membership functions of the fuzzy sets Aji , wpj is the point at i which μBpj = 1. W = [w1 , · · · wl ] , ϕ is l−dimension vector function, the element ϕi =
n i=1
μAj / i
n l j=1
i=1
μAj i
(4)
120
C.V. Israel and W. Yu
If we use Takagi-Sugeno fuzzy model [21][20], the fuzzy rules are Rj : IF x1 is A1j and x2 is A2j and · · · xn is Anj THEN y = pj0 +pj1 x1 +· · · pjn xn (5) where j = 1 · · · l. The output of the fuzzy logic system can be expressed as y =
l
piq0 + piq1 x1 + · · · piqn xn ϕi
(6)
i=1
where ϕi is defined as in (4). (6) can be also expressed in the form of the Mamdani-type (3), y = W ϕ [x (k)] (7)
1 with W (k) = p0 · · · p1n p111 · · · pl11 · · · pl0 · · · pln . Since the Takagi-Sugeno fuzzy model (6) has the same mathematical expression as the Mamdani fuzzy model (3), in this paper we only discuss Mamdani fuzzy system. Generally the fuzzy neural networks (3) can not match the given nonlinear system (1) exactly, the nonlinear system (1) can be represented as y(k) = W ϕ [V, x (k)] + ε (k)
(8)
where ε (k) is defined as modelling error. The identified nonlinear system (1) can also be written as
y(k) = W 0 ϕ V 0 , x (k) + f(k) (9) where fk is modeling error, V 0 and W 0 are sets of known parameters chosen by the user. In general, f(k) ≥ |ε (k)| . From (9) we know the modeling error ft depends on the structure of the fuzzy neural network. Assume we have data of observations [y (k) , x (k)] , taken at time points k ∈ [τ1 , τ2 ]. We assume that the data come from the model f (x) = φ (x) + where φ (x) is an unknown smooth mean response curve, and is error. Our goal is to find the kernel smooth estimate φˆ at some pre-specified time point k. Kernel regression of statistics was derived independently by Nadaraya and Watson [16] with a mathematical foundation given by Parzen’s earlier work on kernel density estimation. Kernel regression is the estimation of the functional relationship between two variables y and t on the interval {0 ≤ ti ≤ T } , i = 1···N yi = y (ti ) + ξ where ξ is a random noise variable with the mean equal to zero. The Nadaraya– Watson kernel regression estimate at a time t is N 2 (t − ti ) 1 i yi k (t − ti ) yi = N (10) , k (t − ti ) = √ exp − 2σ 2 σ 2π i k (t − ti )
Nonlinear System Modeling with a New Fuzzy Model
121
Here the Gaussian density of statistics is the kernel function. For nonlinear system (1) modeling, (10) is yˆ (k) = k [x (k)
N y k[x(k)−x(ti )] i N i i k[x(k)−x(ti )] 2 i ) − x (ti )] = σ√12π exp − x(k)−x(t 2 2σ
(11)
where yˆ (k) is estimated value of y(k) = f [x (k)] . Statistical analysis of the Nadaraya-Watson kernel regression is difficult because it is defined as the ratio of two random variables. In many important applications in signal processing and automatic control, which usually has a constant pdf. With these type of random variables, the summations involving theNkernel function are equivalent to a Monte Carlo integration, N i k (t − ti ) → T . The Nadaraya-Watson kernel regression becomes Priestley-Chao regression, which is defined N N yˆ (ti ) = yi k (t − ti ) (12) T i For system identification yˆ (k) =
N N yi k [x (k) − x (ti )] T i
(13)
The nonlinear plant y(ti ) = f [x (ti ) , θ] + e (ti ) , i = 1 · · · N has only N sparse data in the time interval τ1 ≤ ti ≤ τ2 . We use these data to construct fuzzy rules. For the data set [y(ti ), x (ti )] , we extract fuzzy product rules in the form of (2). x (ti ) is center of the Gaussian function μAj , y(ti ) is the center of the Gaussian i function μB j . Compared (13) with (3), kernel smoothing is equivalent to fuzzy modeling if each data corresponds to one fuzzy rule. However, this kind of fuzzy structure cannot be applied, because there are too many fuzzy rules from kernel smoothing when n is large. A possible way is design a wide kernel function to cover more data, such that these data correspond a rule. A practical method is full width at half maximum. The maximum width is defined as σmax = 2 2 ln (2)σ, σmax is chosen by the total number of the data, for N example σmax = 10 , the width parameter of Gaussian function is σ = √N . 20 2 ln(2)
∗ ∗ We use center points of each Gaussian function xj , yj , j = 1 · · · v to construct the membership functions. For the data set [y (k) , x (k)] , k ∈ [1, N ] , we extract fuzzy product rules in the form of (13) or (2). x∗j is center of the Gaussian function μAj , yj∗ is the center of the Gaussian function μB j . In the time interval i [τ1 , τ2 ] , we use the following automatic kernel smoother to construct fuzzy rules. Step 1: Initialization: select a and σmax . Moreover, calculate kernel. Step 2: Solve kernel smoothing problem according to (13), which has the same form as the fuzzy system is (3). Step 3: Membership function training: we will discuss it in next section
122
C.V. Israel and W. Yu
Step 4: Check phase: check if the network construction should be changed. If no, continuous training go to Step 3. Otherwise, terminate the training precess, go to network construction phase (Step 2 ).
3
Neural Compensation for Fuzzy Models
Since there exists modeling error Δ (k) between the plant output y (k) and the fuzzy model output yˆ (k), a Radial Basis Function (RBF) neural network Δˆ (k) is used to compensate this error, see Fig.1. Define neural compensation error as e (k) = Δˆ (k) − Δ (k)
(14)
Radial Basis Function (RBF) neural networks have recently gained considerable attention. The advantages of the RBF approach, such as the linearity in the parameters and the availability of the fast and efficient training methods, have been noted in several publications, see [11]. The neural compensator is m 2 Xi (k) − ξi ˆ Δ (k) = (15) ωi Gi (X (k)) + b Gi [X (k)] = exp 2σi2 i=1 where m is the nodes number in hidden layer, ωi is the weight, ξi and σi are T the center and the width of Gi (t), respectively, X (k) = x (k)T , Δ (k) . b is a threshold, it can be combined with the first term as w0 = b, G0 (x) = 1, so m Δˆ γ (k) = ωi Gi (x). Finally, the RBF neural compensator is i=0
Δˆ (k) = Ωk G [X (k)]
(16)
where Ωk = [ω1 · · · ωm ] , G is m−dimension vector function. We use this RBF neural networks to compensate the modeling error of the fuzzy model. According to the Stone-Weierstrass theorem, this general nonlinear smooth function (14) can be written as ˆ (k) = Ω ∗ G [X (k)] − μ (k) Δ (k) − Δ where Ω ∗ is optimal weight, μ (k) is the modeling error. Since G is bounded function and the output of the plant is assumed bounded, μ (k) is bounded as μ2 (k) ≤ μ, μ is an unknown positive constant. Neural compensation error can be represented as k G [X (k)] + μ (k) e (k) = Ωk G [X (k)] − Ω ∗ G [X (k)] + μ (k) = Ω
(17)
k = Ωk −Ω ∗ . The following theorem gives a stable fast learning algorithm where Ω of the RBF compensator. The following theorem gives a stable gradient descent algorithm for neural modeling.
Nonlinear System Modeling with a New Fuzzy Model
123
Theorem 1. If we use the RBF neural networks (16) to compensate fuzzy mechanistic model error Δ (k), the following gradient updating with time-varying rate can make the final modeling error e (k) bounded (stable in an L∞ sense) Ωk+1 = Ωk − ηk G [X (k)] e (k) η0
where ηk =
2
1 + G [X (k)] + fication error satisfies
c k
, 0 < η0 ≤ 1, c > 0. The average of the identi-
J = lim sup T →∞
where μ = max μ2 (k) .
(18)
T 1 2 2+c e (k) ≤ μ T 1 − η0
(19)
k=1
k
Proof. We select Lyapunov function as n 2 T Ω Vk = Ω = ω i2 = tr Ω k
(20)
i=1
From the updating law (18) k − ηk G [X (k)] e (k) k+1 = Ω Ω 2 2 ΔVk = Vk+1 − Vk = Ω k − ηk G [X (k)] e (k) − Ωk 2 k G [X (k)] = ηk2 e2 (k) G [X (k)] − 2ηk e (k) Ω
Using (17) and 0 < η ≤ 1, 0 ≤ ηk ≤ η0 ≤ 1, ηk = 2
1 + G [X (k)] +
c k
η0 2
1 + G [X (k)] +
≤1+1+c
c k
,
ΔVk = ηk2 e2 (k) G [X (k)]2 − 2ηk (e (k) [e (k) − μ (k)]) = ηk2 e2 (k) G [X (k)]2 − 2ηk e2 (k) + ηk 2e (k) μ (k) 2 2 2 2 ≤ ηk2 e2 (k) G + ηk e (k) + ηk μ (k) [X (k)] − 2ηk e (k)
(21) 2 = −ηk e2 (k) 1 − ηk G [X (k)] + ηk μ2 (k) η0 (1 − η0 ) e2 (k) + η0 μ2 (k) ≤ −ηk (1 − η0 ) e2 (k) + ηk μ2 (k) ≤ − 2+c 2 2 2 2 Since n min w i ≤ Vk ≤ n max w i and n × max w i are K∞ i , n × min w functions, and (2 + c) e2 (k) is an K∞ -function, η02 μ2 is a K-function. From (17) and (20) we know Vk is the function of e (k) and μ (k) , so Vk admits the smooth ISS-Lyapunov function. The dynamic of the identification error is input-to-state stable. The ”INPUT” is corresponded to the second term of the last line in (21), i.e., the modeling error μ (k) , the ”STATE” is corresponded to the first term of the last line in (21), i.e.., the identification error e (k) . Because the ”INPUT” μ (k) is bounded and the dynamic is ISS, the ”STATE” e (k) is bounded. (21) can be rewritten as ΔVk ≤ −
η0 (1 − η0 ) 2 η0 (1 − η0 ) 2 e (k) + η0 μ2 (k) ≤ − e (k) + η0 μ 2+c 2+c
(22)
124
C.V. Israel and W. Yu
Summarizing (22) from 1 up to T , and by using VT > 0 and V1 is a constant, we obtain T 0) 2 VT − V1 ≤ − η0 (1−η k=1 e (k) + T η0 μ 2+c η0 (1−η0 ) T 2 2 k=1 e (k) ≤ V1 − VT + T η0 μ ≤ V1 + T η0 μ 2+c (19) is established. Remark 1. (18) is the gradient descent algorithm, which the normalizing learning rate ηk is time-varying in order to assure the identification process is stable. This learning law is simpler to use, because we do not need to care about how to select a better learning rate to assure both fast convergence and stability. No previous information is required.
4
Simulation Comparisons
The process of engine at idle has time delays that vary inversely with engine speed and is time-varying due to aging of components and environmental changes such as engine warm-up after a cold start. The measurement of system outputs occurs asynchronously with the calculation of control signals. We assume that the occurrence of plant disturbances, such as engagement of air conditioner compressor, shift from neutral to drive in automatic transmissions, application and release of electric loads, and power steering lock-up, are not directly measured. The dynamic engine model is a two-input and two-output system [17]: · · P˙ = kP mai − mao , N˙ p = kN (Ti − TL ) · (23) mai = 1 + km1 θ + km2 θ2 g (P ) ·
mao = −km3 Np − km4 P + km5 Np P + km6 Np P 2 The system outputs are manifold press P (kPa) and engine speed Np (rpm). The control inputs are throttle angle θ (degree) and the spark advance δ (degree). Disturbances act to the engine in the form of unmeasured accessory torque Td · · (N-m). The variable mai and mao refer to the mass air flow into and out of the manifold. mao is the air mass in the cylinder. The parameter τ is a dynamic transport time delay. The function g (P ) is a manifold pressure influence function. Ti is the engine’s internally developed torque, TL is the load torque. The parameters for an engine model with 1.6 liter, 4-cylinder fuel in N 2 P < 50.6625 √ 1 jected are: g (P ) = + , TL = 263.17 0.0197 101.325P − P 2 P ≥ 50.6625 ·
Td , mao = mao / (120N ) , Ti = −39.22 + 325024mao − 0.0112δ 2 + 0.635δ + 2π 2 2π 0.000102N 2, kP = 42.40, kN = 54.26, km1 = 60 (0.0216 + 0.000675δ) Np − 60 0.907, km2 = 0.0998, km3 = 0.0005968, km4 = 0.0005341, km5 = 0.000001757, τ = 45/Np . Difference technique is used to get the discrete-time states of the · system (23). (23) can be written as x = f (x, u) with x = [P, Np ]T , u = T [θ, δ, Td ] .The discrete-time model for P is P (k + 1) = f1 [P (k) , Np (k) , θ (k) , δ (k) , Td (k)]
Nonlinear System Modeling with a New Fuzzy Model
125
100
50
0
0
200
400
600
800
1000
0
200
400
600
800
1000
100
50
0
Fig. 2. The training and testing results
In order to train the neural model, we select inputs as δ = 30 sin (0.06k) , θ is a square wave with amplitude 20, θ (k) = 20Square(0.04k), Td is a constant, T Td (k) = 10, x0 = [10, 500] . 6 fuzzy rules from kernel smoothing are obtained.The time interval the the hidden layer selection N is chosen as N = 300. We start from Ωk ∈ 1×15 . After then (18) is applied to update the weight Ωk . The final training result is shown in Fig.2. The testing signals are changed as δ = 10, θ (k) = 40 cos(0.05k), Td (k) = 1. The testing result is shown in Fig.2. Now we compare the performances of our fuzzy model with neural compensation (FNC) with neural networks identification (NN) [11], ANFIS fuzzy model (FS) [1], and fuzzy neural model [2] (FNN). The multilayer neural networks as in [11] is Π5,15,1 (one hidden layer with 15 nodes), and its learning rate of backpropagation is fixed as η = 0.05. For FS and FNN, we use 15 rules. The results are shown in Table 1, where the modeling error is calculated as RMS.
Table 1. Comparisons of four identification methods networks case modeling error hidden nodes rule number
NN FS FNN FNC training testing training testing training testing training testing 0.24 0.52 0.21 0.49 0.06 0.43 0.21 0.41 15
15
15
15
15
15
6
6
126
5
C.V. Israel and W. Yu
Conclusion
This paper proposed a new fuzzy model for nonlinear system modeling. Although the fuzzy system plus neural networks based method is widely used for modeling nonlinear systems, there are some new contributions in this paper: the fuzzy model is obtained from kernel smoothing, not from prior knowledge; a new fast training algorithm for the neural compensator is proven to be stable.
References 1. Wang, L.X.: Adaptive Fuzzy Systems and Control. Prentice-Hall, Englewood Cliffs (1994) 2. Yu, W., Li, X.: Fuzzy identification using fuzzy neural networks with stable learning algorithms. IEEE Transactions on Fuzzy Systems 12, 411–420 (2004) 3. Jang, J.S.: ANFIS: adaptive-network-based fuzzy inference system. IEEE Transactions on Systems, Man and Cybernetics 23, 665–685 (1993) 4. Rivals, I., Personnaz, L.: Neural-network construction and selection in nonlinear modeling. IEEE Transactions on Neural Networks 14, 804–820 (2003) 5. Leski, J.M.: TSK-fuzzy modeling based on ε-insensitive learning. IEEE Transactions on Fuzzy Systems 13, 181–193 (2005) 6. Mitra, S., Hayashi, Y.: Neuro–fuzzy rule generation: survey in soft computing framework. IEEE Transactions on Neural Networks 11, 748–769 (2000) 7. Chiu, S.L.: Fuzzy model identification based on cluster estimation. Journal of Intelligent and Fuzzy Systems 2, 159–163 (1994) 8. Chiang, J.-H., Hao, P.-Y.: Support vector learning mechanism for fuzzy rule-based modeling: a new approach. IEEE Transactions on Fuzzy Systems 12, 14–25 (2004) 9. Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines. Cambridge Univ. Press, Cambridge (2000) 10. Wand, M., Jones, M.: Kernel Smoothing. Chapman & Hall, London (1995) 11. Narendra, K.S., Cheng, X.: Adaptive control of discrete-time systems using multiple models. IEEE Transactions on Automatic Control 45, 1669–1686 (2000) 12. Suykens, J.A.K., Vandewalle, J., De Moor, B.: NLq theory: checking and imposing stability of recurrent neural networks for nonlinear modelling. IEEE Transactions on Signal Processing 45, 2682–2691 (1997) 13. Darken, C., Chang, J., Moody, J.: Learning Rate Schedules for Faster Stochastic Gradient Search. In: Proceedings of the IEEE-SP Workshop, pp. 3–12 (1992) 14. Ljung, L., Soderstrom, T.: Theory and Proactice of Recursive System Identification. The MIT Press, Cambridge (1983) 15. Mamdani, E.H.: Application of fuzzy algorithms for control of simple dynamic plant. In: IEEE Proccedings-Control Theory and Applications, vol. 121, pp. 1585– 1588 (1976) 16. Nadaraya, E.A.: On estimating regression. Theory of Probability and its Applications 9, 141–142 (1964) 17. Puskorius, G.V., Feldkamp, L.A.: Neurocontrol of nonlinear dynamics systems with Kalman filter trained recurrent networks. IEEE Transactions on Neural Networks 5, 279–297 (1994) 18. Priestley, M.B., Chao, M.T.: Non-parametric function fitting. J. Royal Statistical Soc. Ser. B 34, 385–392 (1972)
Nonlinear System Modeling with a New Fuzzy Model
127
19. Prokhorov, D., Wunsch, D.: Adaptive critic designs. IEEE Transactions. on Neural Networks 8, 997–1007 (1997) 20. Takagi, T., Sugeno, M.: Fuzzy identification of systems and its applications to modeling and control. IEEE Trans. Syst., Man, Cybern. 15, 116–132 (1985) 21. Baruch, I.S., Lopez, R.B., Olivares, J.-L., Flores, J.M.: A fuzzy-neural multi-model for nonlinear systems identification and control. Fuzzy Sets and System 159, 2650– 2667 (2008) 22. Chen, S., Billings, S., Grant, P.: Recursive hybrid algorithm for nonlinear system identification using radial basis function networks. Int. J. Control 55, 1051–1070 (1992)
A Research of Fuzzy Neural Network in Ferromagnetic Target Recognition Caipeng Wu*, Jiahao Deng**, and Yanli Yang*** National Key Laboratory of Mechatronic Engineering and Control, Beijing Institute of Technology, Beijing 100081, China
[email protected],
[email protected],
[email protected]
Abstract. Based on deeply analyzing the characteristic of the battlefield ferromagnetic targets, according to the problems of magnetic detection system, for instance single detection pattern, low detection resolution and poor antiinterference performance, the Giant Magneto-Impedance(GMI) micro-magnetic sensor in combination with the technology of fuzzy neural networks(FNN) were carried as the core of the magnetic detection system. Take advantage of GMI sensor and FNN to realize accurate recognition of the target in the range of nanotesla magnetic field. In this paper, equable magnetization rotation ellipsoid is used to simulate the tank and military truck, taking the triaxial magnetic moments and semi-focal length, that is M x , M y , M z , c as recognition characteristic quantity , and the FNN is used to recognize the tank and military truck including the categories and motion directions. The method reaches good recognition effect through experimental verification, and it has significance to improve detection range and recognition accuracy. Keywords: Ferromagnetic target recognition, Adaptive neuro-fuzzy inference system, Giant Magneto-Impedance, Micro-magnetic sensors.
1 Introduction The tank, armored vehicle, and military truck are the important ferromagnetic target on the battlefield, and the magnetic detection system that is based on geomagnetic field disturbance generated by the ferromagnetic substance makes use of various magnetic measuring methods to locate and recognize targets. However, the majority of the magnetic fuse detection systems are confronted with the problem of neardistance detecting and poor target identification ability at present, so the early burst and error burst situations happen from time to time. *
**
***
Caipeng Wu is a Ph.D. candidate in Beijing Institute of Technology. His research area is target detection, guidance and control technology. Jiahao Deng is currently a professor in Beijing Institute of Technology. His main research interests include target detection and signal processing, sensing and control technology. Yanli Yang is a Ph.D. candidate and his research interests are target detection and signal processing.
Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 129–136. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
130
C. Wu, J. Deng, and Y. Yang
There are two problems mainly in realizing distant reliable detection according to ferromagnetic moving target on the battlefield. First, the most of the present magnetic fuse systems based on metallic eddy effect, electromagnetic induction or hall effect, exist bad effect in high temperature, poor detection accuracy and so on when conduct target detection; Second, when detection range increase, the high sensitive magnetic sensor is necessary, and the target signal is very weak, so we need reduce detection threshold to measure target signal, which is easily reached by natural or artificial disturbance, then the system has high false alarm probability. In order to solve the above two problems, this paper will design a novel magnetic detection system adopting micro-magnetic sensor based on Giant Magneto-Impedance (GMI) effect and fuzzy neural networks (FNN), then make use of it to detect and recognize ferromagnetic target on the battlefield, especially for tank.
2 Magnetic Detection System Establish The detection system mainly consists of a three-axis GMI magnetic sensor, signal conditioning circuit and the target identification module taking FPGA (Field Programmable Gate Array) as core, as shown in Fig.1.
Fig. 1. Magnetic Detection and Target Recognition System Block Diagram
Taking the three-axis GMI sensor as the core of the sensitive element, detect x (East is positive), y (North is positive), z (Upward is positive). The so-called GMI effect, is referred to a significant change in the ac impedance of a soft magnetic conductor with an ac current when subjected to an applied dc magnetic field, which is announced by Panina and Mohri for the first time in 1994. The GMI sensor initial prototype has been successfully developed abroad, the performance of which achieves the following indexes: detection accuracy reaches 1 μ Oe, reaction speed can reach 1MHz, and the average power consumption is approximately 10mW. Compared with a Giant Magneto-Resistance (GMR) that has a sensitive of ~1%/Oe, the field sensitivity of a typical GMI sensor can reach a value as high as 20~100%/Oe [1-3]. The system control and signal processing module is composed of FPGA and CPLD (Complex Programmable Logic Device). With improvement of FPGA capacity, performance, cost efficiency, and parallel processing ability, it can become an significant hardware realization method for FNN, compared with software implementation which is beneficial to improve real-time performance of the fuse system. The CPLD with design flexibility and high hardware integration is applied to assisted control logic, so it will complete chip selection signal, buffering function of A/D and FPGA, some level conversions function, and other control logic.
A Research of Fuzzy Neural Network in Ferromagnetic Target Recognition
131
3 Fuzzy Neural Network Fuzzy Neural Network, FNN for short, is referred to combine fuzzy logic with neural network appropriately, and the combination starts from 1988, absorbs the merits of the both [4]. It can fully develop the advantages of parallel processing and adaptive learning in neural network, and it has reasoning and decision function using fuzzy logic to human knowledge. If we take the FNN system as a recognition method to fuzzy and complex magnetic characteristic signals from ferromagnetic target on battlefield, we can obtain a magnetic detection system with high recognition rate. TS fuzzy neural network proposed by Takagi and Sugeno is a type of neural network that is the most widely used and deeply researched at present. The FNN based of TS type can realize Adaptive Neuro-Fuzzy Inference System(ANFIS), which is a method using data modeling, that is to say the fuzzy membership functions and fuzzy rules come from the known data’s learning, but not from experiences or intuition. For the ANFIS of the n inputs and single output, x = [ x1 x2 … xn ] T is the inputs defined by fuzzy linguistic variables, and the fuzzy rules proposed by Takagi and Sugeno can be expressed as follows [5, 6]:
: x is A
IF
1
j 1
and … and xn is Anj ,THEN: y j = q j 0 + q j1 x1 + " + q jn xn
Where Ai j (i = 1, 2," , n; j = 1, 2," , mi ) is fuzzy subset of the xi , μ
Ai
j
(1)
( xi ) is its
membership function. Every prescriptive relevance weight is shown as that:
μj = μ
j A1
( x1 ) μ
j A2
( x2 ) " μ
j An
( xn )
(2)
Then the network output can be written as follows: m
y = ∑μj yj
(3)
i =1
4 Magnetic Field Analysis of the Ferromagnetic Target In previous magnetic detection systems, the waveform characteristics of target magnetic signals, such as amplitude, waveform width and slope, were used in target detection [7]. When the projectile-target distance is relatively near, because of the higher signal-noise ratio, we can use frequency filtering and other signal preprocessing method to realize target detection effectively, however, for the situation of farther projectile-target distance and low signal-to-noise, if we only use some time-frequency characteristics of the target signal to wave filter, it is difficult to ensure high recognition accuracy of the detection system. In contrast, using modeling method to research ferromagnetic target fully utilizes the time-frequency characteristics of magnetic signal, the spatial distribution characteristic of magnetic vector and the motion features of target, so it can be called a holographic technique of the target field. If we can build the stable reliable model of ferromagnetic target, utilize the model parameter that is the most refined essential characteristic of target magnetic
132
C. Wu, J. Deng, and Y. Yang
field as the characteristic quantity of target recognition, and combine GMI micromagnetic senor as the core of detection system, the target detection distance will be improved significantly at the situation of high correct recognition rate. 4.1 Target Magnetic Field Model Building
、
The most typical ferromagnetic targets of land battlefield are tank armored vehicle and military truck, all of which are magnetic objects having a certain geometry. Taking tank as an example, the common model building methods include uniform magnetized rotary ellipsoid with array of magnetic dipole modeling method, rectangular parallelepiped with hollow rotary ellipsoid model and a single rotary ellipsoid method. The advantage of former two mixed modeling methods is that they can improve fitting precision on the near field of tank, but the deficiency is the model parameters are too complicated and not stable enough. While for a single rotary ellipsoid method, although the fitting precision on the near field of tank is poor, with distance growing, it becomes higher and higher, and the model parameters are more stable. So when the projectile-target distance is relatively far, we can use a single rotary ellipsoid to simulate the magnetic field of tank that is shown in Fig.2, and take the model parameters as the characteristic quantity of target recognition.
Fig. 2. Modeling schematic diagram for tank magnetic field
In the model of Fig.2, the parameters are defined as follows: the coordinate origin is located in the center of tank, the semi-major axis of the ellipsoid is a = L / 2 ( L is equal to the longitudinal length of the tank), the short half axis is expressed as b = W ⋅ H / 2 ( W , H respectively represent the transverse length, the height of the tank), c is the half focal length and equal to a 2 − b 2 [8].So on the ellipsoid coordinate system, the three-axis components of magnetic field intensity on point P( x, y, z ) can be expressed as: H x = M x a x + M y a y + M z a z , H y = M xbx + M y by + M z bz , H z = M x cx + M y c y + M z cz
(4)
In (4), [ M x , M y , M z ] is the magnetic moment of ellipsoid model, and the coefficient matrices are [ax , a y , az ]T [bx , by , bz ]T and [cx , c y , cz ]T that can be calculated through model geometric parameter.
A Research of Fuzzy Neural Network in Ferromagnetic Target Recognition
133
4.2 Target Characteristic Quantity Extraction
The values of magnetic moment that are obtained in different point are not the same, due to the complex shaped parts of tank and the inhomogeneous distribution of ferromagnetic substance, but we can adopt least square method to get the estimated value of model parameters. In the practical projectile target crossing process, the stepwise regression method is used to estimate the magnetic moment parameters [ M x , M y , M z ] of ferromagnetic target real-timely, and the practical projectile target
—
crossing velocity can be calculated through the method. By comparing the real-time model parameters calculated by the above method and that of knowledge base, the result of accurate target recognizing can be attained.
5 FNN Recognition System Design The ANFIS is a modeling method based on mass data, in which the membership functions and fuzzy rules come from learning the knowing data, so it is known as a recognition system based on data rules. The so-called “data rules” means unify organically the input data (target feature) and the rules that are the “IF-THEN” in the ANFIS to form the rule base, which can perform target recognition. 5.1 Fuzzy Rule Base Construction
According to the analysis above, the “ M x , M y , M z , c → y ” is the expected input-output data couple, where M x , M y , M z , c are the magnetic target model parameters according to rotary ellipsoid modeling, y is the result of target recognition. In fuzzy system, the input and output spaces should be divided into some fuzzy spaces, where there are corresponding membership functions. The input and output parameters need to be localized in the spaces having the biggest membership functions, which based on the corresponding membership of M x , M y , M z , c at different intervals, then a rule is generated from the every satisfying input-output data couple. The confidence principle can be used to solve those conflicting rules, that is selection the rules having higher confidence. Finally, the mapping relation can be determined according to combination of fuzzy rules [9-11]. 5.2 Application Example of Ferromagnetic Target Recognition
The received signals of magnetic detection system on land battlefield are mainly from tank target, common military truck, industrial interference or circuit noise, and natural interference signal, especially for the geomagnetic storm and pulsation. For the latter two interference signals, although the frequency range of signals is so wide that it can cover that of the target magnetic field completely, they can’t pass the examination of rotary ellipsoid model and the fitting accuracy is poor, because their magnetic field distribution can’t be simulated by a uniform magnetic rotary ellipsoid; whereas the
134
C. Wu, J. Deng, and Y. Yang
military truck is very similar to tank in geometry shape, material composition and weight, so the two types of targets is the distinguished emphases. Take a three bits binary array T (T1, T2 , T3 ) as the output of FNN, and there are eight possible outputs, which respectively direct at the four driving directions of tank and military truck. For improving the reliability of target recognition system, we have measured the actual magnetic field values of a certain main battle tank and a “Liberation” truck, and according to different measurement points, in the space of 20m × 15m × 10m including target, we selected 400 groups sample data that is 100 groups data selected in one direction, in which 200 groups data constitute training group, and the other 200 groups data is the testing group of the FNN. All of the data need to be preprocessed to obtain the model parameters M x , M y , M z , c at the four directions, and normalize the
—
model parameters as the inputs of FNN. Some data having been processed are shown in Table 1. Table 1. Partial training samples and the expected outputs
Model parameters(input) Mx
My
Mz
c
0.452 0.437 0.484 0.474 0.075 0.041 0.168 0.162 0.989 0.952
0.851 0.857 0.999 0.999 0.522 0.521 0.379 0.378 0.322 0.018
0.075 0.152 0.797 0.869 0.041 0.172 0.667 0.699 0.240 0.766
0.061 0.079 0.873 0.920 0.011 0.112 0.775 0.796 0.157 0.793
…
T (T1, T2 , T3 )
000 000 100 100 001 001 101 101 011 111 …
5.3 Realization Process of FNN
According to the data of Table 1, we can construct the ANFIS to complete the recognition of tank and truck targets in species and directions, which includes input layer, fuzzy layer, reference layer and conclusion layer. The inputs of FNN is M x , M y , M z and c , and the output is the recognition result. In MATLAB fuzzy logic toolbox, the function of genfis1() is used to train the ANFIS and generate the initial value of TS type ANFIS, and utilizing the function of anfis() to train the initialized network structure. The optimum parameters of membership functions can be calculated through data learning effectively, and make the designed Takagi-Sugeno type FNN simulate the expected or actual relationship of input-output. Fig.3 is the graph of membership functions about M x , M y , M z , c before and after training.
1
Before Training 1
0.5
0.5
0
c
Mx,My and Mz
A Research of Fuzzy Neural Network in Ferromagnetic Target Recognition
0
0 1 0 After Training 1
0.5
0
Mz
My
0.5 0
0.5
1
1
0.5
0.5
0
0
0.5
0
1
0.5
1
0
0.5
1
0
0.5
1
0.5 0
1
c
Mx
1
135
Fig. 3. Membership Function about M x , M y , M z and c Before and After Training
The error change curves of training and testing data is shown in Fig.4. -3
6
x 10
A B
5
training error testing error
B 4 3 2 1 0 0
A 10
20
30
40
50
60
70
80
90
100
Fig. 4. Training and Testing Error Change Curves
From Fig.4, we can see both of the training and testing errors is smaller than 0.002 after function training of 100 times, and the fitting effect is good. The testing error is bigger than the training error, but the correct recognition rate is up to 95%.
6 Conclusion In this paper, according to the problems of magnetic detection system, such as the near detection distance and poor target recognition ability, we proposed the solution that used the new generation magnetic sensor—GMI sensor having better performance as
136
C. Wu, J. Deng, and Y. Yang
the detecting element, and took fuzzy neural network as the recognition tool of ferromagnetic target. It is important to make above research for improving the distance and recognition accuracy of magnetic detection system. Acknowledgments. The work was supported by the National Natural Science Foundation of China under the grant No.60874100, and the Innovation Foundation of Aerospace Science and Technology (CAST200834).
References 1. Panina, L., Mohri, K.: Magneto-impedance effect in amorphous wires. Appl. Phys. Lett. 65, 1189–1191 (1994) 2. Mohri, K., Uchiyama, T., Panina, L.: Recent advances of micro-magnetic sensors and sensing application. Sensors & Actuators A: Physical 59, 1–8 (1997) 3. Kentaro, T., Yichi, H., Masayoshi, E.: Three-axis magneto-impedance effect sensor system for detecting position and orientation of catheter tip. Sensors & Actuators A: Physical 111, 304–309 (2004) 4. Sugeno, M., Yasukawa, T.: A fuzzy logical based approach to qualitative modeling. IEEE Trans. on Fuzzy Systems 1(1), 7–31 (1993) 5. Hairong, S., Pu, H., Lihui, Z.: A New Method to construct Fuzzy Systems Based on Rule Selecting. In: ICMLC 2004, vol. 3, pp. 1855–1858 (2004) 6. Takagi, T., Sugeno, M.: A robust stabilization problem of fuzzy control system and its application to backing up control of truck-trailer. IEEE Trans. on Fuzzy System. 2(2), 119–134 (1994) 7. Chunsheng, L., Qian, X., Shenguang, G.: A modeling algorithm for detection of moving on-water magnetic objects. Journal of China Ordnance 2(26), 192–195 (2005) 8. Pingxian, Y., Shenguang, G.: The physical field of warship, pp. 25–60. Weapon Industry Press (1992) 9. Zhenyuan, J., Ying, Z., Yingdong, S., Wenyan, T.: The application of NN in recognition of vehicles. Journal of Natural Science of Heilongjiang University 1(26), 39–42 (2009) 10. Tao, Z., Tingjin, L., Xuehai, Z.: The recognition method of small underground objective based on fuzzy neural network. Journal of Projectiles, Rockets, Missiles and Guidance 4(27), 316–319 (2007) 11. Hitoshi, L., Masafumi, H.: Adaptive fuzzy inference neural network. Pattern Recognition 10(37), 2049–2057 (2004)
Multiple T-S Fuzzy Neural Networks Soft Sensing Modeling of Flotation Process Based on Fuzzy C-Means Clustering Algorithm Jiesheng Wang, Yong Zhang, and Shifeng Sun School of Electronic and Information Engineering, Liaoning University of Science & Technology, Anshan 114044, China
[email protected],
[email protected],
[email protected]
Abstract. Inspired by the idea of combining multiple models to improve prediction accuracy and robustness, a soft sensing modeling of flotation process based on multiple T-S fuzzy neural networks and fuzzy c-means clustering algorithm (FCM) is proposed. Firstly, the model adopts principal component analysis (PCA) to reduce dimensions of the input variables data composed of texture characteristics of floatation froth image and process variables. FCM algorithm is used for separating a whole training data set into several clusters with different centers and each subset is trained by T-S FNN. The degrees of membership are used for combining several models to obtain the finial soft sensing result. Simulation results show that the proposed modeling is effective in the prediction of indexes and meets the requirement for optimization computation for the flotation process. Keywords: Flotation Process, Texture Characteristic, Multiple T-S Fuzzy Neural Networks, FCM, Soft Sensing, PCA.
1 Introduction In the flotation process, technical indicators such as concentrate grade and flotation recovery of economic are the key to production process control targets. However, it can only be adjusted and be based on an off-line analysis. Because large-time delays, it is difficult to achieve the quality of direct closed-loop control [1]. This estimating problem of process quality indicators of off-line measurement is solved effectively by using inferred estimates (soft-sensing) technology [2]. At present, the single model structure which uses the sample data is most used in the non-linear soft sensor modeling. If we do not restrict the size of the model and the training data is sufficient, a satisfactory model structure and the approximation accuracy are got which are based on neural networks or fuzzy systems modeling. However, as the expansion of the training area, the systemic states become more and more complexity. It will result in low efficiency model optimization and the reducing of accuracy of the model prediction and the significantly ability of robustness and generalization. By considering the divide and conquer, and task decomposition, a number of model structure are proposed by many scholars in order to improve Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 137–144. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
138
J. Wang, Y. Zhang, and S. Sun
prediction accuracy and robustness, such as Multiple Neural Networks (MNN), Stacked NN (SNN) and Distributed NN (DNN), and so on [3-4]. A method which based on principal component analysis (PCA) and fuzzy c-means clustering algorithm (FCM) method of the multi-T-S model structure (the PFM T-S algorithm) is presented in this paper. Simulation studies have shown that the method can significantly improve the model accuracy and robustness.
2 FCM-Based Multiple T-S Fuzzy Neural Network Model Considering a multi-input single-output (MISO) system, the training sample set can be expressed as D = {Y , X | i = 1, 2, " , m} . Y = { y | j = 1, 2, " , n} is the output matrix ( n is the number of the training datum and m is the number of input variables). X = [ x , x , " , x ]′ (i = 1, 2, " , m ) is the input vector. If D is divided into c clusters {D , D , " , D } , it can establish c of sub-model {M , M , " , M } by using a subset {D | i = 1, 2, " , c} . Fig. 1 is the structure diagram of PFM T-S FNN model [4]. j
i
i
1i
1
2
2i
ni
c
1
2
c
i
Fig. 1. PFM T-S Fuzzy neural network model structure
2.1 FCM Clustering Algorithm FCM clustering algorithm [5] divides the feature points of the feature space X = ( x , x , " , x ) into c categories ( 2 ≤ c ≤ n ) , the i class cluster center is 1
2
expressed as
n
vi , any feature point
x
j
belong to the i
category of
membership u ( 0 ≤ u ≤ 1) , and uij Satisfy the following conditions: ij
ij
c
∑u
ij
=1
for
j = 1, 2, " , n
(1)
i =1 n
0<
∑u j =1
ij
for
i = 1, 2, " , c
(2)
Multiple T-S Fuzzy Neural Networks Soft Sensing Modeling of Flotation Process
139
The objective functions of FCM algorithm as follows: c
n
∑∑ (u )
J m (U , V ) =
m
ij
i =1
x j − vi
2
(3)
j =1
Where m > 1 is the index weight to influence the degree of fuzzy membership degree matrix. Clustering problem is to make (3) the smallest degree of membership matrix U = [ u ] and center V = ( v , v , " , v ) . Using FCM, a prior parameters of the classification needed to be determined is the number of data sets. The cluster validity function is used in this paper described as follows [6]: ij
c×n
1
V p (U , c ) =
2
c
c −1 1 n 1 c −1 c 1 n max(uik ) − ∑ ∑ ( ∑ max(uik , u jk )) , K = ∑ i ∑ n k =1 i K i =1 j =i +1 n k =1 i =1
(4)
The clustering criterion is combined with the fuzzy division of compactness and the separation. To get the optimal fuzzy partition, it is combined with the ISODATA clustering algorithm and the proposed criterion function. The algorithm is described as follows: Step 1: cmax ( c ≤ n ): the maximum number of cluster center. T : maximum number of iterations. Index weights m and stop threshold ε > 0 ; m ax
Step2: Center on the clustering number c = 2, 3, " , c , initialize fuzzy clustering max
Center: V = { v , v , " , v 0
10
20
c0
}.
Step 3: For t = 1, 2, " , T , according to the following formula, calculate membership degree matrix and the cluster center:
uij ,
t
⎡ 1 =⎢ ⎢ xi − v j ⎣
⎤ ⎥ 2 ⎥ ⎦
1
⎡ 1 ⎢ ∑ j =1 ⎢ x − v j ⎣ i
a −1
c
⎤ ⎥ 2 ⎥ ⎦
1
a −1
(5)
n
v j, t =
∑ [u i =1 n
ij ,t
]m xi (6)
∑ [uij,t ]m i =1
Where i = 1, 2," , n, j = 1, 2," , c . If V − V < ε , switch to the next step, otherwise repeat step 3; Step 4: According to the formula (4) Calculate V (U , c ) ; If c < c max , switch to t
t −1
p
Step 2; Otherwise, stop iteration. cb is the optimal number of clusters. cb Satisfy the following formula:
140
J. Wang, Y. Zhang, and S. Sun
Vp (U , cb ) = min{Vp (U , c )}, c = 2, 3, " , cmax
(7)
2.2 Flotation Foam Image Texture Feature Selection and Extraction
The state of the production process flotation can be reflected by flotation foam statistical characteristics of image texture, the iterative changes of different gray values which distribute in the space forming the image texture, a gray level relationship exist between the two pixels (the gray space-related features).LCM reflects the relationship of changing characteristics, which image gray on the direction, the adjacent interval and of the magnitude image. That f ( x, y ) pixel of gray-scale I for proceeding, and declination θ , distance δ , gray-scale of the j of Pixel ( x + Δx , y + Δy ) of the probability of simultaneous P (i , j , δ , θ ) .That means the joint distribution statistical form of two-pixel. It can be expressed as follow: P (i , j , δ , θ ) = {[( x , y ), ( x + Δx , y + Δy )] | f ( x , y ) = i
(8)
f (x +Δx, y +Δy) = j; x =0,1,", Nx −1; y =0,1,"Ny −1}
(9)
Where i , j = 0,1, " , L − 1 , x , y is pixel coordinates in the image, L is the gray series of the image, N and N indicated rows and columns of the image number. According to the definition above, GLCM is defined by the i row and column j element. That x
y
all the θ directions of the image and separation as δ , the gray is on the difference frequency of the i and j pixel. θ connects the two clockwise and the pixel x Axis angle. Its value is usually 0o, 45o, 90o and 135o. Three characteristic quantities of the calculation of the statistical characteristics texture energy (E), entropy (ENTS), and moment of inertia (I) are based on the gray co-occurrence matrix, are defined as follows [7]:
E=
L −1 L −1
∑∑ Pˆ (i, j ) i=0
∑∑ Pˆ (i, j) log
ENTS = −
I =
2
Pˆ (i, j )
(11)
L −1 L −1
L −1
∑n { ∑ ∑ 2
n=0
(10)
j=0
i =0 j =0
Pˆ (i , j )}
(12)
n= i− j
Three texture parameters of the flotation froth image can be calculated by using Equation (10) - (12). They can reflect the amount of pulp in the reagent addition and indirectly reflects the state of the production in the flotation process. This article use
Multiple T-S Fuzzy Neural Networks Soft Sensing Modeling of Flotation Process
141
image texture characteristics and process variables as the input variables consisting of PFM T-S fuzzy neural network soft-sensor model of the input variables. 2.3 Dimensionality Reduction Based on PCA
If input vector dimension of T-S model is too large, it would make a great network topology and training become very complex. We can use PCA on the T-S model to reduce model high-dimensional of input vector dimensionality. Using Principal Component Analysis [8], we selected the process of input vector principal component of the T-S model as follows: (1) Data collection operation, a number of the image texture statistical characteristic energy (E), entropy (ENTS), and moment of inertia (I) and the corresponding time period to the ore grade, the ore flow, the ore concentration, give mine size and pharmaceutical flow measurements X ∈ R in flotation process, m : process number of variables (in this case 8), n : a sample of data vectors. In order to avoid the impact of different dimensions of process variables on the results and ease calculate. It is , standard necessary to normalize the data processing. Using medium vector deviation vector σ , the normalized process variables are defined as follows: n ×m
μ
xij = ( xij − μ j ) / σ j (i = 1, " ,n,j = 1, " ,m)
(13)
Define X is Hutchison normalized process variable vector. Covariance matrix is ∑ . (2) Calculate the m characteristic values λ of the covariance matrix ∑ and the j
orthogonal eigenvectors
Pj
of the corresponding units;
(3) Calculate the j principal component t : j
t j = XP j
(14)
(4) Calculate the principal component model: T T T Xˆ = t1 P1 + t2 P2 + " + t j Pj
Where
tj ∈ R
n
is the principal component,
Pj ∈ R
m
(15)
is the load vector, and
Pj
is the
characteristic roots of matrix ∑ of the matrix t arranged in descending. Each pair j , P are the corresponding eigenvectors P by the corresponding eigenvectors of
j
j
j
Eigen values λ of the descending order. The first pair intercepted the largest amount of information of the load vector of all the decomposition and principal component vectors, the rest of inferior race and so on. T = [t , " , t ] is the main element matrix, P = [ P , " , P ] is the load matrix, and then the equation above can be expressed as: j
1
1
m
m
T Xˆ = T P
(16)
142
J. Wang, Y. Zhang, and S. Sun
(5) Calculate the previous k (1 ≤ k ≤ m ) variance contribution rate ηk : k
η k = ∑ λi i =1
m
∑λ
(17)
i
i =1
Then determine the main element k of the PCA model number. Because there is a high correlation between the variables, if general previous k ( k << m ) principal component of the cumulative contribution rate more than 85%, we can think that k principal component reflect the main information of the process. We can choose the k vectors corresponding to the main element in the T matrix as the training input vector of the TS fuzzy neural network. In this paper, by using principal component analysis cope with input vector data of soft-sensor model flotation process, the variance contribution rate of the former four principal components will have reached 92.54%, So choose the previous four principal components as an input of each T-S model. It dot not only preserve the information characteristics of the original various, but also simplify the model structure. 2.4 T-S Fuzzy Neural Network Model
Fuzzy data-based modeling problem can usually be described as follows[6]: Given n pairs of input-output model P ( x, y ) and modeling error indicators ε > 0 . By Optimization fuzzy model F (θ , w, p ) .we can obtain the minimum number of fuzzy rules p and the optimal model parameters (Including the rules antecedent membership function parameters θ and the rule consequent linear parameters w ), which makes the modeling error function E = y − F satisfy the inequality E (θ , w) < ε . According to s historical data sets Victoria {P1 , P2 ," , Pn } , a MISO fuzzy model can be expressed as having the following form of fuzzy rule sets:
:
Ri IF x 1 is A i1 and x 2 is A i2 " and x s is A is , Then y i = zi ( x ) Where,
x = ( x1 , x2 , " , x s ) ∈ U 1 × U 2 × " × U s
(18)
Are linguistic variables; Ai j is fuzzy sets on
the domain of discourse U ∈ R ( j = 1, 2, " , s ) ; Ri (i = 1, 2," , p ) :the i rule; yi ∈ V : the i rule output. z i (x) has the following two forms: Single-valued ( zi ( x ) = bi ), called as j
the Mamdani model; Linear function ( zi ( x ) = bi 0 + ∑ j =1 bi j x j ), called as Takagis
Sugeno (T-S) model. T-S used in this paper fuzzy neural network model diagram as shown below. The model input X r , r = 1," , 4 , each T-S model input is divided into three membership function X r u , u = 1," , 3 ; we use concentrate grade and flotation recovery as a model output ( f h , h = 1, 2 ). For each T-S sub-model, we can use gradient descent method of least squares mixed model to adopt the before and after the pieces of parameters: First, we fix antecedent parameters, prior to the calculation, and identify the last pieces by using least squares linear parameter; Second, we use gradient descent method to adjust the non-linear parameters of fuzzy rules, the algorithm in the reference [9] as described.
Multiple T-S Fuzzy Neural Networks Soft Sensing Modeling of Flotation Process
143
3 Industrial Experiments Collect historical data of production of the flotation process and choose 700 group historical data with uniform and representative. Then we divided the pre-processing datum into two parts. The first part used as the training data by using PMM T-S model and a simple model of T-S model in the paper, then we learn and train the model. The second part is used to verify the soft-sensor model performance. Fig. 2 and Fig.3 are the prediction output of concentrate grade and flotation recovery.
Fig. 2. The predictive output of concentrate grade
Fig. 3. The predictive output of flotation recovery
It can be seen from Fig. 2 and Fig.3 that the multi-TS FNN soft-sensor model is based on PCA and FCM algorithm is better than a single TS soft-sensor model at economic and technical indicators (concentrate grade and flotation recovery). It has higher prediction accuracy and generalization capabilities. The forecast error of concentrate grade is between ±0.3 percent and the flotation recovery of the forecast error is between ±0.1 percent. The simulation results show that the higher prediction accuracy of the model-based prediction model is helpful to stabilize the flotation of
144
J. Wang, Y. Zhang, and S. Sun
pharmaceutical operations and improve operational recovery rate. It has a certain practical value for the flotation process and laid the foundation for optimal control.
4 Conclusions In this paper, a multi-T-S fuzzy neural network soft-sensor model based on principal component analysis and fuzzy c-means clustering algorithm is presented. According to extrapolation of the actual operating data estimates, it shows that the method has a good tracking rate and high estimation accuracy and fully meets computing requirements of the real-time optimal flotation reagent addition system. Acknowledgments. This work is supported by the Scientific Research Foundation of the Higher Education Institutions of Education Bureau of Liaoning Province, China (Grant No. 20060432) and the Program for the Innovative Research Team of Education Bureau of Liaoning Province, China (Grant No. 2008T091).
References 1. Zhang, Y., Wang, J.S., Wang, W.: Application of Expert Control Method to Flotation Process. Control and Decision 19, 1271–1274 (2004) (in Chinese) 2. Zhu, X.F.: Soft-sensing Technique and Its Applications. Journal of South China University of Technology 30, 61–67 (2002) (in Chinese) 3. Wu, J., Xu, Y., Ou, H.Y.: Modular Fuzzy Logic System Based on Principal Component Regression Analysis. Journal of Xi’an Jiaotong University 36, 1311–1314 (2002) (in Chinese) 4. Zhong, W., Yu, J.S.: Study on Soft Sensing Modeling Via FCM-based Multiple Models. Journal of East China University 26, 83–87 (2002) (in Chinese) 5. Bezdek, J.C.: Pattern Recognition with Fuzzy Objective Function Algorithm. Plenum Press, New York (1981) 6. Chen, M.Y., Linkens, D.A.: Rule-base Self-generation and Simplification for Data-driven Fuzzy Models. Fuzzy Sets and Systems 142, 243–265 (2004) 7. Bartolacci, G., Pelletier Jr., P., Tessier Jr., J.: Application of Numerical Image Analysis to Process Diagnosis and Physical Parameter Measurement in Mineral Processes. Minerals Engineering 114, 734–747 (2006) 8. Ipek, H., Ankara, H.: The Application of Statistical Process Control. Minerals Engineering 12, 827–835 (1999) 9. Takagi, T., Sugeno, M.: Fuzzy Identification of Systems and Its Applications to Modeling and Control. IEEE Transaction on Systems, Man, and Cybernetics 15, 116–132 (1985)
A Using Reliability Evaluation Model for Diesel Engine Based on Fuzzy Neural Network Ying-Kui Gu and Kai-Qi Huang School of Mechanical and Electronical Engineering, Jiangxi University of Science and Technology, Ganzhou 341000 China
[email protected]
Abstract. Using reliability evaluation for diesel engine is a complex, dynamic and uncertain process. In order to make an objective and right evaluation, and offer a stronger decision-making tool for designer and user of diesel engine, the neural-network-driven fuzzy reasoning mechanism of using reliability evaluation was developed based on the detail analysis of engine using reliability in the case that there is no sufficient quantitative information or the information is fuzzy and imprecise, where a feedforward neural network was used to replace fuzzy evaluation in the fuzzy system. Applications show that the evaluation result can be used as references for the improvement of reliability and maintainability of engines, and for the establishment of maintenance strategy. Keywords: Neural network, fuzzy logic, fuzzy reasoning, reliability evaluation, multiple attribute decision making.
1 Introduction As the momentum of automobile, engineering machine, ship and power plant units, diesel engine plays an important role in the national economy. With the fast development of science and technology, diesel engine is increasingly developing towards large scale, high speed, high precision, and its working performance has been constantly improved, the automation degree is also higher. On the one hand, the productivity of labour as well as quality of product is greatly improved, and the cost of production as well as consumption of energy is reduced. But on the other hand, once certain part or certain link of diesel engine breaks down, there will be shut down or off production, thus enormous economic loss would be directly or indirectly caused. Therefore, it is very imperative to evaluate the using reliability of diesel engine. However, it is very difficult to evaluate the using reliability of diesel engine because of the complexity, fuzziness and dynamic uncertainty of evaluation process. Fuzzy logic and neural network provide stronger tools for decision-making modeling and analyzing under the fuzzy and uncertain environment [1-3]. In this paper, we presented an approach to evaluate the using reliability of diesel engine using fuzzy logic and neural network. First, the using reliability indices for diesel engine were Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 145–152. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
146
Y.-K. Gu and K.-Q. Huang
analyzed in detail. Second, we developed a fuzzy-logic-based reasoning mechanism to analyze the using reliability under the fuzzy and imprecise design environment. Then, a neural-network-driven fuzzy reasoning system was developed to improve the reasoning efficiency, where the neural network was used to replace fuzzy evaluation in the fuzzy reasoning system. Finally, a case was used to illustrate the proposed neural-network-driven fuzzy reasoning.
2 Using Reliability Indices Analysis for Diesel Engine The main using reliability indices for diesel engine are listed as follows. (1) Mean Time to First Failure (MTTFF) The point estimate value Tˆf of MTTFF and confidence lower limit value T fL of interval estimation are as follows. 1 n Tˆ f = [∑ t i + ( N − n)t . n i =1
(1)
n
T fL =
2[∑ t i + ( N − n) ⋅ t i =1
χ 12−a (2n + 2)
.
(2)
where N is the output of diesel engine. n is the number of diesel engine that break down (not include minor failure). t i is the time to first failure of ith diesel engine, km (h). t is test cut-off distance (or time), km (h). a is significance level, the suggested value a = 0.1 . (2) Mean Time Between Failure (MTBF) The point estimate value Tˆb of MTBF and confidence lower limit value TbL of interval estimation are as follows. Nt . Tˆb = n TbL =
χ
2 1− a
2 Nt . ( 2 n + 2)
(3)
(4)
where N is the output of diesel engine. n is the number of diesel engine that break down (not include minor failure, that is not include 4 kinds of faults). t is test cut-off distance (or time), km (h). a is significance level, the suggested value a = 0.1 . (3) Use Availability (UA) N
A=
∑U
i
i =1
N
∑ (U i =1
i+
Gi )
=
MTBF . MTBF + MTTR
(5)
A Using Reliability Evaluation Model for Diesel Engine Based on Fuzzy Neural Network 147
where N is the output of diesel engine. U i is running time of ith diesel engine, h. Gi is the stopping time of ith diesel engine because of fault (include failure diagnosis time, preparing time, repairing time), h. MTBF is mean time between failure, h. MTTR is mean time to repair, h. (4) Equivalent Failure Rate (EFR) D=
1000 4 ∑ ε j rj . Nt j =1
(6)
(5) Mean Maintenance Cost Ratio of Each Year (MMCR) When reliability of diesel engine is measured from economical efficiency, it can be measured by mean maintenance cost ratio of each year. M =
C ma int enance × 100% . C price
(7)
where C ma int enance is the mean maintenance cost of each diesel engine for the first year, unit: yuan. C price is the price of diesel engine, unit: yuan. (6) Comprehensive Evaluation Value (CEV) Q = 100 −
(MTBF) 0 Nt 0
r
∑K e . i =1
(8)
i i
where (MTBF) 0 is the target value of reliability on diesel engine’s product or the MTBF value of congeneric advanced diesel engine’s product, unit: h. N is output of diesel engine . t 0 is test cut-off distance (or time), km (h). k i is coefficient of injury about ith fault, the specific value can be obtained from following table, ei is coefficient of injury about ith fault, ei =
2 ⋅ t0 , where t i is accumulative work t0 + ti
mileage of diesel engine for ith fault (or time), km (h). Each index can be obtained by reliability trial or cumulative observation, while using reliability on diesel engine can be synthetically measured by all the indices, thus hierarchy model of using reliability on diesel engine can be established as shown in Figure 1. Evaluation result
MTTFF
1# engine
MTBF
UA
2# engine
EFR
CEV
MMCR
3# engine
…
n# engine
Fig. 1. Hierarchical respresentation of using reliability evaluation for engine
148
Y.-K. Gu and K.-Q. Huang
3 The Using Reliability Evaluation Method Based on Fuzzy Logic Reliability evaluation process is a complex problem. There exist many factors, which influence the evaluation result of the engine reliability. In order to simplify the problem and also to illustrate the approach, only the following three important indices are considered in this paper, i.e. (1) use availability, (2) mean maintenance cost ratio of each year, (3) comprehensive evaluation value. All these indices are fairely vague and fuzzy and can be represented linguistically. Thus, fuzzy sets and fuzzy logic are ideally suited for representing these indices. These indices form the input of the model and the output are the desirability of the reliability evaluation results. Figure 1 illustrates the hierarchical linguistic representation. Since these representations are fuzzy and approximate, neural network learning is a necessity to improve this approximate model.
Fuzzy membership
VL
L
M
H
0.25
0.5
0.75
VH
1 0.5
0
1.0 UA, MMCR, CEV
Fig. 2. Membership functions of reliability indices
The indices are represented lingusitically in Figure 2. each index is divided into several different grades based on a universe of discourse of [0,1]. Each grade is represented by a triangular membership function. The three indices, use availability, mean maintenance cost ratio of each year and comprehensive evaluation value has the linguistic grade: VL (very low), L (low), M (medium), H (high), VH (very high) respectively. The problem is to evaluate the engine reliability based on the three indices. Notice that this is a multiple criteria problem and, in general, the three indices cannot mutually compensate. Therefore, the output variable can also be divided into different fuzzy levels. The connections between the input indices and the output evaluation results can be made by the use of fuzzy rules. Table 1 illustrates these fuzzy if-then rules. For example, Rule 1: if the use availability is VH, and the mean maintenance cost ratio of each year is VL and the comprehensive evaluation value is VH, then the grade of the output is A1. Rule 2: if the use availability is VH, and the mean maintenance cost ratio of each year is VL and the comprehensive evaluation value is H, then the grade of the output is A2.
A Using Reliability Evaluation Model for Diesel Engine Based on Fuzzy Neural Network 149 Table 1. Fuzzy if-then rules I/O Sample 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
Input (if) MMCR VL VL L L VL M M L M VL VL L L VL M M L M VL VL L L VL M M L M
UA VH VH VH VH VH VH VH VH VH H H H H H H H H H M M M M M M M M M
Out (then) Ranking Classic A1 A2 A3 A4 A5 A6 A7 A8 A9 B1 B2 B3 B4 B5 B6 B7 B8 B9 C1 C2 C3 C4 C5 C6 C7 C8 C9
CEV VH H VH H M VH H M M VH H VH H M VH H M M VH H VH H M VH H M M
The expression if-and-then is in fuzzy logic terms. To simplify the illustration, only three levels of each index with a total of 27 rules are considered in Table 1. the different output grades for these 27 rules are shown in Figure 3. The output ranking has three classes designated as A, B and C, and each class has five grades. For example, class A is divided into A1, A2, A3, A4 and A5.
Fuzzy membership
Class
C5 B5(C4) B4(C3) B3(C2) A5(B2) A4(B1) A3
A2
A1
1 0.5
0
0.6
0.65
0.7
0.75
0.8
0.85
0.9
0.95
Fig. 3. The membership function for output ranking score
0.1
150
Y.-K. Gu and K.-Q. Huang
The fuzzy rules or the interconnections between the input and the output can first formulated approximately based on the vague knowledge of the actual system. These approximate interconnections can be refined for any given system by actual learning using neural network. The training sample sets can be obtained from the actual system experts.
4 BP-Neural-Network-Driven Fuzzy Evaluation Model Because back-propagation neural network has the ability to learn by examples, it has been used in pattern matching, pattern classification and pattern recognition. Therefore, it can be used to establish the neural-network-driven fuzzy evaluation model for engine reliability. The BP network is constructed in three layers. Nine nodes in input layer, five nodes in hidden layer, and one node in the output layer. In the training process, a hybrid learning algorithm that combined unsupervised learning and supervised gradient-descent learning procedures was used to build the fuzzy rules and train membership functions. Through assigning values to the input variables, calculating the membership function measures of the input nodes, and achieving the output membership function using fuzzy reasoning, the correct data set can be achieved. All the fuzzy rules are involved and can be encoded in the neural network. When a set of input data is provided to the neural network, these encoded rules are activated to different degree at the same time. The output membership function can be generated by the neural network computation, and it is related to all the fuzzy rules. To obtain the training samples, the center of the area defuzzification method is used. The center of the area defuzzification equation for discrete system can be represented by [4,5]
∑ f μ (f ) n
j
F=
j =1 n
0
j
∑ μ0 ( f j )
.
(9)
j =1
where n is the number of quantization levels, and μ 0 ( f j ) is the membership at the quantized level j. using the above equation and the fuzzy if-then rules in Table 1 with the help of Figure 3, the 54 training samples can be obtained.
5 Case Study Take the reliability evaluation of YC6100 as an example to illustrate the established BP-neural-network model. There are 10 engines waiting for evaluating their reliability. The reliability index and its evaluation value for these engines were listed in Table 2.
A Using Reliability Evaluation Model for Diesel Engine Based on Fuzzy Neural Network 151 Table 2. The input and output of the training sample I/O Index sample 1# engine 2# engine 3# engine 4# engine 5# engine 6# engine 7# engine 8# engine 9# engine 10# engine
Use availability VH 0.92 0 0.88 0 0.95 0 0.86 0.78 0.75 0.65
H 0.08 0.89 0.12 0.93 0.05 0.96 0.14 0.22 0.25 0.35
M 0 0.11 0 0.07 0 0.04 0 0 0 0
Input Mean maintenance cost ratio of each year VL L M 0.85 0.15 0 0.67 0.33 0 0 0.95 0.03 0.83 0.17 0 0 0.92 0.08 0.74 0.26 0 0 0.76 0.24 0.68 0.32 0 0.65 0.35 0 0.62 0.38 0
Output Comprehensive evaluation value VH 0.85 0 0.87 0.82 0.79 0 0.75 0 0.65 0
H 0.15 0.87 0.13 0.18 0.21 0.88 0.25 0.82 0.35 0.86
M 0 0.13 0 0 0 0.12 0 0.18 0 0.14
Value
Rank
0.9321 0.8564 0.9023 0.8765 0.9125 0.8128 0.8956 0.7986 0.7887 0.7469
1 6 3 5 2 7 4 8 9 10
Based on the analysis in Section 2 and the established neural network model in Section 3 and 4, and the network precision of training parameter is 10-6 and the study rate is 0.001. The ranking order are expressed as: R1 > R5 > R3 > R7 > R4 > R2 > R6 > R8 > R9 > R10. The output result shows that the 1# engine has higher using reliability.
6 Conclusions In order to decrease the influence of the fuzzy and uncertain factors during evaluation process, the fuzzy logic and neural network were combined to establish the evaluation model for diesel engine reliability. The evaluation result of engine using reliability can be obtained by training the network. Applications show that The result can be used as references for the improvement of reliability and maintainability of engines, and for the establishment of maintenance strategy, and it also lay a theoretical foundation for studying the improvement of evaluation method for the engine reliability as well.
Acknowledgments This research was partially supported by the Jiangxi Provincial Natural Science Foundation under the contract number 2007GQC0654 and the Science Foundation of Education Commission of Jiangxi Province under the contract number GJJ09245.
References 1. Gu, Y.K., Yang, Z.Y.: TS-neural-network-based maintenance decision model for diesel engine. In: Liu, D., Fei, S., Hou, Z.-G., Zhang, H., Sun, C. (eds.) ISNN 2007. LNCS, vol. 4491, pp. 557–565. Springer, Heidelberg (2007)
152
Y.-K. Gu and K.-Q. Huang
2. Su, J., Ao, C.L., Jiao, Y.: Evaluation of tractor reliability based on Fuzzy Neural Network. Journal of Northeast Agricultural University 39, 99–103 (2008) 3. Hsu, C.H., Jiang, B.C., Lee, E.S.: Fuzzy Neural Network Modeling for Product Development. Mathematical and Computer Modelling 29, 71–81 (1999) 4. Gu, Y.K., He, X.W.: Neural-network-driven fuzzy optimum selection for mechanism schemes. In: Liu, D., Fei, S., Hou, Z., Zhang, H., Sun, C. (eds.) ISNN 2007. LNCS, vol. 4492, pp. 275–283. Springer, Heidelberg (2007) 5. Sun, J., Kalenchuk, D.K., Xue, D., Gu, P.: Design candidate identification using neural network-based fuzzy reasoning. Robotics and Computer Integrated Manufacturing 16, 383–396 (2000)
Fuzzy Sliding Mode Control with Perturbation Estimation for a Piezoactuated Micromanipulator Qingsong Xu and Yangmin Li Department of Electromechanical Engineering, Faculty of Science and Technology, University of Macau, Av. Padre Tom´ as Pereira, Taipa, Macao SAR, China {qsxu, ymli}@umac.mo http://www.sftw.umac.mo/~ yangmin/
Abstract. This paper is concentrated on control system design for an XY parallel micromanipulator with piezoelectric actuation. The decoupled property of the manipulator enables the employment of single-inputsingle-output controllers for the two working axes. To compensate for nonlinear hysteresis effect stemming from piezoelectric actuator, the dynamics with Bouc-Wen hysteresis model is derived and identified for the system. Afterwards, a real-time fuzzy sliding mode control with perturbation estimation (FSMCPE) along with fuzzy switching control is proposed to attenuate the chattering phenomenon. Experimental studies reveal that the designed FSMCPE is superior to conventional SMCPE in terms of both positioning and tracking performances of the micromanipulator system, which validates the effectiveness of the presented control system design as well. Keywords: Micropositioning, parallel manipulator, fuzzy sliding mode control.
1
Introduction
Recently, XY parallel micromanipulators have been paid extensively attentions [1,2] due to their promising applications in micro-/nano-manipulation fields such as scanning probe microscopy, biological cell manipulation, and so on. Nevertheless, majority of existing manipulators have either an uncomplicated structure along with a coupled motion or a decoupled motion which is at the expense of a complicated structure. To benefit both controller design and prototype fabrication in practice, a parallel micromanipulator with both a decoupled motion and an uncomplicated structure is desirable. Moreover, it is suggested [3] that the micromanipulator is preferred to be designed with both input and output decoupling properties in virtue of actuation isolation and decoupled output motion. This totally decoupling property is necessary for some situations where the platform is under-actuated or sensory feedback of end-effector positions is not allowed [4]. One of such XY parallel micromanipulators has been recently designed Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 153–160. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com
154
Q. Xu and Y. Li
by the authors. The objective of the current research is to design an appropriate controller to enable a sub-micron accuracy for the micromanipulator. In view of the advantages of high stiffness as well as high output force, piezoelectric actuator (PZT) is widely used for the drives of micromanipulators. However, a piezoactuated manipulator is a highly nonlinear system due to the hysteresis and creep behaviors introduced by the PZT. In the literature, various types of approaches have been exploited to alleviate the hysteresis effect. Particularly, sliding mode control (SMC) has been employed in the piezoactuated manipulators since the SMC is an effective and simple way to deal with model imperfection and uncertainties for nonlinear systems. As a nonlinear control strategy, SMC drives the system’s state trajectory onto a specified sliding surface and maintains the trajectory on this surface for the subsequent time. However, in conventional SMC design, a priori knowledge of the bounds on system uncertainties has to be acquired. As a consequence, the controller based on the above knowledge tends to be overconservative, which may result in poor tracking performance and undesirable oscillations in control signal. To overcome such drawback, SMC with perturbation estimation (SMCPE) has been proposed [5] as an enhanced version of the conventional SMC. It is shown that the need to know the uncertainty bounds is removed in the SMCPE and is replaced with a strategy of online estimation of perturbations. Nevertheless, due to the discontinuity of the signum function, high-frequency chattering phenomenon may occur in the control input, which is a severe problem especially when the state trajectory is close to the sliding surface. To attenuate the chattering phenomenon, the boundary layer technique is usually adopted by replacing the signum function with saturation function in the switching control term. The chattering can be avoided with the saturation function. As the cost, sliding mode performances will be compromised at the same time [6]. Additionally, in selecting the boundary layer, a tradeoff between the chattering and tracking error is always required. In this research, an alternative approach employing a fuzzy SMCPE controller with fuzzy switching control is proposed to reduce the chattering effect in the conventional SMCPE. The efficiency of the presented fuzzy SMCPE controller will be demonstrated by experimental studies performed on an XY micromanipulator prototype in the following sections.
2
System Description
The experimental setup of the XY parallel micromanipulator is shown in Fig. 1, where the XY stage is fabricated from the alloy Al 7075. In particular, the stage has a compact size owing to a two-layered mechanism design. It is actuated with two 20 μm-stroke PZT (model PAS020 produced by Thorlabs, Inc.) through displacement amplifiers. A PCI-based D/A board (PCI-6703 from National Instruments Corp.) is adopted to produce an analog voltage, which is then amplified by a two-axis piezo-amplifier (BPC002 from the Thorlabs) to provide a voltage of 0–75 V for the drives of the PZTs. The output displacements of the mobile platform are measured by two laser displacement sensors (Microtrak II,
FSMCPE for a Piezoactuated Micromanipulator
Plant
Real-Time Controller
Prototype
PZT
D/A (PCI-6703)
Piezo amplifier
Laser sensor
PC (Malab/Simulink, Real-Time Windows Target Toolbox)
155
A/D (PCI-6034E)
Sensor controller
Fig. 1. Schematic diagram of apparatus connection
head model: LTC-025-02, from MTI Instruments, Inc.), whose analog voltage outputs are read simultaneously by the PC through a PCI-based data acquisition board (PCI-6034E from the National Instruments). It can be calculated that the resolution of the position detection system is 0.038 μm. Besides, the software of Matlab/Simulink with Real-Time Windows Target toolbox is employed to implement the real-time control of the micromanipulator system. Open-loop tests show that a workspace range of 138×128 μm2 can be attained by the XY micromanipulator, and the cross-talk between the two working axes is less than 5%. Thus, the two-axis motions are well-decoupled, which allows the employment of two SISO controllers for x- and y-axes, respectively.
3
Dynamic Modeling and Identification
It is known that a piezoactuated system is nonlinear mainly attributed to the complicated hysteresis effects. Due to the decoupled property of the XY stage, the x- and y-axes can be treated independently. For brevity, only the x-axis motion is described in this paper. The dynamic model of the entire micropositioning system with Bouc-Wen hysteresis model can be established as follows: m¨ x + bx˙ + kx = k(du − h) + p h˙ = αdu˙ − β|u|h ˙ − γ u|h| ˙
(1) (2)
where the parameters m, b, k, and x represent the mass, damping coefficient, stiffness, and x-axis displacement of the XY stage, respectively; d is the piezoelectric coefficient, u denotes the input voltage, and h indicates the hysteretic loop in terms of displacement whose magnitude and shape are determined by parameters α, β, and γ. Besides, p denotes the overall perturbation of the system arising from model parameter uncertainties, unmodeled dynamics, and other unknown terms. By neglecting the perturbation, the seven parameters (m, b, k, d, α, β, and γ) of the system model with Bouc-Wen hysteresis can be identified simultaneously by an optimization algorithm. Specifically, with a 0.2-Hz multi-amplitude sine-wave voltage applied to the PZT, the primary output motion of the stage is recorded and collected for the purpose of model identification. By employing
156
Q. Xu and Y. Li
the particle swarm optimization (PSO) [7], the following model parameters are identified for the x-axis motion: m = 0.1040 kg, b = 1.8625×104 N s/m, k = 1.9920×105 N/m, d = 1.8347×10−5 m/V, α = 0.0786, β = 0.0995, and γ = 0.0008. Because the Bouc-Wen model cannot represent nonsymmetric hysteresis exactly, there exist a maximal error of 7.0% between the identified model and experimental result. In order to compensate for the model errors and other uncertainties, a SMCPE is designed below.
4
Conventional SMCPE Design
Based on the perturbation estimation strategy [5], the system model (1) becomes: m¨ x(t) + bx(t) ˙ + kx(t) = k[du(t) − h(t)] + pest (t) + p(t)
(3)
with the estimated perturbation: x(t) + bx(t) ˙ + kx(t) − k[du(t − T ) − h(t − T )]. pest (t) = m¨
(4)
where p(t) = p(t) − pest (t) represents an error between the system’s real perturbation and its estimation. To design a SMCPE controller, an error coordinate is defined as: e(t) = x(t)− xd (t), where xd represents the desired position trajectory, and t denotes the time variable. In view of the linear second-order dynamic system, a first-order sliding surface is defined as s = e˙ + λe, where λ (λ > 0) is a design parameter. It can be proved that, the tracking error e satisfies lim e(t) = 0 and the sliding variable t→∞ meets the sliding condition ss˙ < 0 if the control law is given by: u = ueq + usw =
1 pest m m b ( − λ)x˙ + (x + h) − + (¨ xd + λx˙ d ) − η sgn(s) (5) kd m d kd kd
p| ε where η > | kd + kd , the last term in (5) represents the switching or corrective control usw and the rest is the equivalent control term ueq . Besides, sgn(·) represents the signum function, η is a positive switching gain, and ε is an arbitrary positive constant. Additionally, inserting (4) into (5), gives
1 m u(t) = u(t − T ) + [h(t) − h(t − T )] − [¨ e(t) + λe(t)] ˙ − η sgn(s(t)). d kd
(6)
Furthermore, due to the discontinuity of the signum function sgn(s), chattering may occur in the control input. To alleviate the chattering phenomenon, the boundary layer technique is usually adopted by replacing the signum function in switching control term usw in (6) with the saturation function: sgn(s) for |s| > δ sat(s) = (7) s/δ for |s| ≤ δ where the positive constant δ represents the boundary layer thickness, which ensures that s is always bounded by δ. The chattering can be avoided with the
FSMCPE for a Piezoactuated Micromanipulator
157
saturation function. Nevertheless, sliding mode performances will be compromised as the cost. Besides, in the selection of parameter δ, a tradeoff between the chattering and tracking error is required. In the next section, an alternative approach employing a fuzzy SMCPE controller with fuzzy switching control is proposed to reduce the chattering effect.
5
Fuzzy SMCPE Design
In this research, the sliding variable s which measures the closeness to the sliding surface is employed as the input variable to design a fuzzy switching controller, which will replace the switching control term usw in the control signal and regulate this control term on-line to improve the control performance. A fuzzy controller consists of three modules as described below. Fuzzification: The input variable s is imported into the fuzzy controller and classified by the input membership functions in the fuzzification module. Specifically, the input space is classified into five fuzzy sets, which are specified by the selected triangular membership functions. A membership function associated with a fuzzy set maps an input value to its appropriate membership value. The degree of an input variable belonging to a fuzzy set is denoted by a membership value between 0 and 1. Particularly, the input fuzzy sets are: F (s) = {Fs1 , Fs2 , · · · , Fs5 } = {N B, N M, ZO, P M, P B}
(8)
where the labels of fuzzy sets N B, N M , ZO, P M , and P B represent negative big, negative medium, zero, positive medium, and positive big, respectively. Fuzzy inference engine and fuzzy rules: The input information is converted into the output by the fuzzy inference engine module through the fuzzy rules. In this research, the Mamdani’s fuzzy inference method is adopted. The output space is classified into five fuzzy sets: F (u) = {Fu1 , Fu2 , · · · , Fu5 } = {N B, N M, ZO, P M, P B}
(9)
where the meanings of fuzzy sets labels N B, N M , ZO, P M , and P B are the same as the input fuzzy sets. Besides, the triangular-type membership functions are chosen for the output sets. Selecting appropriate fuzzy rules is the major task in fuzzy controller design. According to the sliding surface, switching control effort is determined to satisfy the sliding condition ss˙ < 0, so that the system state is forced to move toward the origin along the sliding surface. From empirical knowledge of sliding mode controller design, a large switching control term will drive the state trajectory to approach the sliding surface s = 0 quickly. However, it tends to excite large chattering at the same time. Thus, when the sliding variable s is negative or positive large, the switching control term usw should be increased accordingly, and vice versa. Hence, the rules which determines the switching control output (6−i) can be established as Ri : If s is Fsi then usw is Fu , for i = 1, 2, · · · , 5.
Degree of membership
Q. Xu and Y. Li
NB
1
NM ZO PM
0.4
PB
0.3 (a)
0.5
0.2 0.1
0 −2
−1.5
NB
1
0.5
−1
−0.5 NM
0 s
0.5
1
ZO
PM
PB
1.5
2
usw
Degree of membership
158
0
(c)
−0.1 −0.2
(b)
−0.3 0 −0.6
−0.4
−0.2
0 usw
0.2
0.4
0.6
−0.4 −2
−1.5
−1
−0.5
0 s
0.5
1
1.5
2
Fig. 2. Input (a) and output (b) membership functions, and the control signal (c) of the fuzzy controller
The fuzzy values of the rule parameters N B, N M , ZO, P M , and P B are tuned by the trial and error procedure through experiments, and the input and output membership functions are depicted in Figs. 2(a) and 2(b), respectively. The fuzzy inference engine performs a mapping from the input fuzzy sets to output fuzzy sets based on the established IF-THEN fuzzy logic rules. Defuzzification module: The input for the defuzzification process is the output fuzzy sets obtained above, and the output is a single crisp number which represents the output of the fuzzy controller. Using the centroid defuzzification method, Mamdani (minimum function) implication in the rule base, and maximum aggregation method (μu ), the output of the fuzzy controller can be derived as: N i=1 usw μu uf s = (10) N i=1 μu where N is the total number of fuzzy rules. The control signal of the fuzzy controller with respect to the input variable is plotted in Fig. 2(c), which exhibits a shape similar to the saturation function. By replacing the switching control in (6) with the fuzzy controller output, a fuzzy SMCPE control input can be expressed as: 1 m u(t) = u(t − T ) + [h(t) − h(t − T )] − [¨ e(t) + λe(t)] ˙ + uf s (s(t)). d kd
6
(11)
Experimental Studies
Based on the fuzzy SMCPE controller designed above, several experiments are conducted to demonstrate the performance of the micromanipulator. The implemented closed-loop sampling time is 0.003 seconds for the real-time control. Concerning the fuzzy SMCPE (FSMCPE) controller, the sliding surface is defined with λ = 2 × 104 . For comparisons, a conventional SMCPE with saturation-function switching control is also implemented with the parameters
FSMCPE for a Piezoactuated Micromanipulator
14 SMCPE
25
FSMCPE
12
Desired SMCPE FSMCPE
20
Position x (μm)
Position x (μm)
10 8 6 10.5 11
4
10
10 2
9
9.5 4
8 0
7 0
−2 0
0.5 1
4.5
4
15 10 5 0
5
1 2 3 Time (sec.)
−5 0
5
1
(a)
5
4
5
FSMCPE 1.5
Control effort (V)
Position error x (μm)
4
2 SMCPE FSMCPE
0.5 0 −0.5
1 0.5 0
−1 −1.5 0
2 3 Time (sec.)
(b)
1.5 1
159
1
2 3 Time (sec.)
(c)
4
5
−0.5 0
1
2 3 Time (sec.)
(d)
Fig. 3. (a) Step responses obtained with SMCPE and fuzzy SMCPE (FSMCPE); 0.4Hz sinusoidal motion tracking results (b), tracking errors (c), and FSMCPE control effort (d)
of λ = 2 × 104 , δ = 0.50, and η = 0.32, which gives a switching control signal shape like the one produced by the fuzzy controller as shown in Fig. 2(c). First, a 10-μm step response is experimentally tested. The control results with the two types of controllers are depicted in Fig. 3(a). It is observed that the conventional SMCPE produces a response with a longer settling time. In contrast, the fuzzy SMCPE controller gives a rapid response, which is at the expense of a larger overshoot value. In addition, at the steady-state, both controllers can maintain the positioning error within ±0.5 μm. Second, the sinusoidal motion tracking performance of the micromanipulator is verified. Given a 0.4-Hz sine-wave signal with peak-to-peak (p-p) amplitude of 20 μm, the tracking results of the micromanipulator are shown in Figs. 3(b)–(c). It is seen that the SMCPE can only suppress the p-p error within ±1.3 μm. On the contrary, the FSMCPE achieve a p-p error of ±0.8 μm, which means that the tracking error has been reduced by 38% with respect to the conventional SMCPE control. The low-level tracking error also implies that the hysteresis has been substantially compensated by the FSMCPE approach. In addition, the control effort of FSMCPE as plotted in Fig. 3(d) indicates that the chattering phenomena have been eliminated. It is noticeable that above performances for the parallel micromanipulator are obtained within the input rate of 0.4 Hz. The limitation mainly lies in the relatively low sampling frequency due to the hardware restrictions. Additional
160
Q. Xu and Y. Li
experiments show that the sub-micron accuracy of the stage is degraded as the increasing of input rate. Once a higher bandwidth is achieved, a high-rate tracking control can be implemented even with the same control method proposed in this paper.
7
Conclusions
The studies presented in this research shows that the fuzzy sliding mode control with perturbation estimation technique can improve the tracking performance of a piezo-driven XY parallel micromanipulator. Based on a relatively simple Bouc-Wen hysteresis model, the dynamic model of the system is established and identified by PSO approach. The fuzzy controller is employed to replace the conventional switching control in SMCPE so as to overcome the limitation of boundary layer technique. Experimental results show that the hysteresis effect has been significantly reduced and chattering phenomenon has been suppressed by the proposed fuzzy SMCPE substantially, which leads to a sub-micron accuracy motion tracking of the micromanipulator. In the future works, a highbandwidth control will be carried out to improve the positioning performance of the system by increasing input rate. Acknowledgments. The authors appreciate the fund support from the research committee of University of Macau under grant no.: UL016/08-Y2/EME/ LYM01/FST and Macao Science and Technology Development Fund under grant no.: 016/2008/A1.
References 1. Pham, H.H., Chen, I.M.: Evaluation of resolution of flexure parallel mechanisms for ultraprecision manipulation. Rev. Sci. Instrum. 75(9), 3016–3024 (2004) 2. Li, Y., Xu, Q.: A novel design and analysis of a 2-DOF compliant parallel micromanipulator for nanomanipulation. IEEE Trans. Automat. Sci. Eng. 3(3), 248–254 (2006) 3. Li, Y., Xu, Q.: Design and analysis of a totally decoupled flexure-based XY parallel micromanipulator. IEEE Trans. Robot. 25(3), 645–657 (2009) 4. Awtar, S., Slocum, A.H.: Constraint-based design of parallel kinematic XY flexure mechanisms. ASME J. Mech. Des. 129(8), 816–830 (2007) 5. Elmali, H., Olgac, N.: Sliding mode control with perturbation estimation: A new approach. Int. J. Contr. 56, 923–941 (1992) 6. Kim, S.W., Lee, J.J.: Design of a fuzzy controller with fuzzy sliding surface. Fuzzy Sets and Systems 71(3), 359–367 (1995) 7. Xu, Q., Li, Y.: Error analysis and optimal design of a class of translational parallel kinematic machine using particle swarm optimization. Robotica 27(1), 67–78 (2009)
A Credit Risk Rating Model Based on Fuzzy Neural Network Ke-Jun Zhu1, Pin-Chang Chen1,2, and Yu-Teng Chang2 1
China University of Geosciences, Wuhan Hubei 430074, China 2 Yu Da University, Miaoli 36143, Taiwan
[email protected], {chenpc,cyt}@ydu.edu.tw
Abstract. This study integrates the characteristics of credit risk rating and artificial intelligence technology into a credit risk rating model based on fuzzy neural network. The combination of fuzzy theory and neural network provides a good foundation for credit risk rating, making this model with fewer parameters, faster learning and less training samples. This study confirms that fuzzy neural network is an effective method for credit risk rating. The results of this study can solve the shortcomings in existing credit risk rating model and provide more information for decision-making reference. Keywords: fuzzy theory, neural network, credit risk rating.
1 Introduction Credit risk has significant influence on the financial market. It can directly affect various activities in modern economy [1]. When rating the credit risk of companies or individuals, an accurate and efficient prediction model is important to financial institutions, so as to determine an effective loan solution to reduce the rate of non-performing loans [2]. This study examines the shortcomings and deficiencies in models and methods of current credit risk rating models, and combines the characteristics of credit risk rating with artificial intelligence technology to develop a credit risk rating model based on fuzzy neural network. This study is intended to construct a fuzzy neural network model through input and output data sets, and optimize the parameters in membership function through a separate back-propagation algorithm. According to the combination of fuzzy theory and neural network, the function is not based on intuition or experience, but on any given values, which is especially important for this proposed model with complex characteristics [3]. In addition, the membership function and fuzzy rules are acquired automatically, so that the application object of fuzzy inference is not limited to the models with pre-determined characteristics of some model variables [4]. The traditional neural network has no specific weight of physical meaning, but given the physical meaning of inference parameters in fuzzy theory, which increases the transparency of the black box Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 161–171. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
162
K.-J. Zhu, P.-C. Chen, and Y.-T. Chang
and expresses neural network’s inference function appropriately [5]. The fuzzy neural inference has a problem that there are too many fuzzy rules, and it is inferior to back-propagation neural network in multi-dimensional images [6]. Fuzzy neural network can effectively utilize the advantages of both fuzzy theory and neural network, and make up for their shortages. The use of fuzzy neural network for credit risk rating not only can improve the learning and expression abilities of the rating index system, but also improve the accuracy of rating at the same time, achieving the purpose of more efficient credit risk rating [7].
2 Literature Review Fuzzy theory and neural network are two distinct areas of research, but they have the same purpose because they both are used to study artificial intelligence. They are equivalent in theory, but in practice, they have their own advantages and disadvantages. Fuzzy theory focuses on human brain’s macro functions, which can be expressed with language and concept. According to the membership function and serial fuzzy inference rules defined by human, it deals with all kinds of fuzzy information with the logical inference [8]. Neural network focuses mainly on the microstructure of the human brain, and studies the generation and formation of human’s wisdom from physical structure. Through the nonlinear dynamics on neural network with self-learning, self-organizing and self-adaptive functions, it processes the information which cannot be verbalized [9]. Fuzzy theory and neural network are two new methods for information processing, they are self-adaptive estimators that are not related to the model, with a certain degree of expertise in dealing with uncertain information and the information contained noise [10]. Fuzzy theory and neural network have their own advantages and disadvantages. The advantages of neural network are its self-learning and self-adaptive abilities as well as its parallel processing capacity and good fault tolerance. However, in the expression of knowledge and interpretation of rules learned, there are obvious deficiencies and its learning speed is usually unsatisfactory. The advantages of fuzzy theory include it can expresses knowledge naturally and easily, deal with uncertain information, express knowledge with fuzzy and natural language, and easy to use expertise. Fuzzy theory can realize fuzzy inference of knowledge with simple computing. However, fuzzy theory has poor learning ability, and it is very weak in knowledge acquisition [11]. Fuzzy theory and neural network are correlated and complementary in many aspects. It is theoretically proven that fuzzy theory can approach a nonlinear function with any precision, while neural network has the imaging capacity. It means that both two have close connection. If fuzzy theory and neural network are combined, which supplements each other, field of information processing will rise to a new realm. In such a combined fuzzy neural network, neural network simulates brain’s topology, that is, the “hardware”; and fuzzy theory simulates the thinking capacity of fuzzy information processing, namely “software” [12]. As a result, fuzzy neural network absorbs the essence of the biological cybernetics, it is distributed and parallel in storage and processing of information. In this way, all kinds of neurons support and supplement information mutually based on information sharing. It can collect inputs from nearby neurons connected to the input, calculate the
A Credit Risk Rating Model Based on Fuzzy Neural Network
163
output and then pass to higher or the other neurons, thus accomplish stronger fault tolerance and association ability. The overall performance is not affected severely when partial neurons are damaged. The output is not distorted seriously if input signal is subject to a certain degree of noise pollution. It proves that fuzzy neural network has a unique ability and can be widely used in many research fields [13].
3 Methodology Due to the complexity and high nonlinearity of credit risk, there is a wide deviation between traditional linear model and actual system. This study is intended to use non-linear and generalization capacity according to fuzzy theory and neural network, establish a credit risk rating model based on fuzzy neural network, which can achieve risk assessment and prediction. The model is based on the financial risk model and company scale rating, coupled with the consideration on companies’ fundamental factors, so that the rating factors are comprehensive, and the results are more scientific and accurate. When constructing the financial risk model, due to neural network’s self-learning, self-adaptive capacity, parallel processing capability and good fault tolerance, it can obtain more accurate non-linear outputs from various financial indexes. Fuzzy theory expresses knowledge naturally and easily, which is able to process uncertain information. It can express knowledge with fuzzy and natural language, as well as use expertise to assess the company scale. Therefore, both fuzzy theory and neural network are used in credit risk rating to obtain fewer parameters, faster learning and less training samples. The research methodology of this study is described as follows: Step 1: Collect examples of industrial enterprises, substitute companies’ credit-related data into the method proposed by this study to confirm the effectiveness. Step 2: Build a financial risk model with back-propagation neural network. Compared with traditional methods of statistical analysis, neural network is more accurate and with lower predictive risk and smaller error. SPSS statistical software is used to carry out factor analysis on company balance sheet and income statement. Step 3: Build company scale rating with fuzzy theory. Based on the concept of corporate risk rating model, the company scale can be divided into four types, which are small, medium, large, and extra large. Such vague language is particularly suited to be realized with the concept of fuzzy theory. Based on expertise, parameters of the model are adjusted continuously according to the actual output, so as to determine the scope of the scale rating. Step 4: Carry out credit risk rating with fuzzy neural network. Based on the financial risk rating from Step 2 and scale rating from Step 3, the fundamental factors are added in the final credit risk rating along with the description of the index system of corporate risk rating. Such input is companies’ total score rated by the experts from quality, strength, environment, credit status, critical incidents, and leader’s quality, and then it is naturalized into a value among [0,1]. A corporate credit risk rating model is constructed based on the above-mentioned rules and fuzzy neural network. Step 5: Analyze the data and propose a conclusion.
164
K.-J. Zhu, P.-C. Chen, and Y.-T. Chang
4 Data Analysis 4.1 Acquisition and Preprocessing of Sample Data Since business environments and scopes in different industries vary greatly, their financial and non-financial indexes are not comparable. Thus, the same industry with short-term loan as sample data are selected by this study to avoid the problems mentioned above. The data of this study are collected from the credit department of a bank. A total of 250 samples are collected. The sample data has to be preprocessed first. In the actual application, the sample cannot be increased unlimitedly, so the selected samples must be representative. Since the data obtained by this study has larger capacity, there are a wider range of indicators. The double and triple standard deviation tests are conducted to remove data anomalies, and 250 sample data are obtained. Among those, 200 are selected as training sample data and 50 are test sample data. As this rating model is applied as a corporate risk rating for commercial banks through a more comprehensive system of indexes, it is obvious that application of fuzzy neural network is a relatively new attempt. However, too much input information in the fuzzy neural network will lead to explosive growth of training time. This study focuses on the financial risks and scale, and then applies fuzzy neural network to obtain the final results of credit risk rating. 4.2 Construct the Financial Risk Model Based on Back-Propagation Neural Network Compared with the traditional methods of statistical analysis, artificial neural network is more accurate with lower predictive risk and smaller error. SPSS statistical software is used to perform factor analysis on company’s balance sheet and income statement. The main purpose is to determine the smallest number of factors which can interpret the relationship between observed variables. According to the characteristics of eigenvalue criteria, under the current conditions of sample data, the number of factors is 5. Also, from the factor loading matrix, it is clear that the economic implications of each factor are obvious; the specific results are the sample data in this study. In a three-layer back-propagation neural network model, the first layer is the input layer and the number of neurons is 5. The input layer includes the data input of profitability, growth capacity, operating capacity, short-term solvency and long-term solvency, respectively. The second layer is the hidden layer. The third layer is the output layer and the number of neuron is 1. After completing the model training, it can be used to solve practical financial risk rating. The MATLAB software is applied for back-propagation neural network, the transfer function of hidden layer is tansig, the transfer function of neuron in output layer is logsig, this is because the elements in target vector are located between [0, 1], meeting the output requirements of logsig. Back-propagation neural network structure is as shown in Fig. 1.
A Credit Risk Rating Model Based on Fuzzy Neural Network
165
Fig. 1. Back-Propagation Neural Network Structure
4.3 Construct the Company Scale Rating Based on Fuzzy Inference System According to expertise, company scale has a direct impact on business credit. It is determined by the company's total income and total assets. In this study, the Mamdani fuzzy theory with fuzzy generator and eliminator is applied to build the fuzzy inference model. According to the concept of corporate risk rating model, the company scale can be divided into four types, which are small, medium, large, and extra large. Such vague language is particularly suited to realize with the fuzzy inference system. Based on expertise, the parameters of the scale rating model are adjusted continuously according to the actual output, so the scope of scale rating is determined: 0 small 0.27; 0.27<medium 0.587; 0.587
≦
≦
≦
≦
≦
(1) Establish corresponding data sets into fuzzy inference system for storage, modification and management of the fuzzy theory (2) Determine the input and output linguistic variables and linguistic values (a) Two inputs: the total assets and total income; (b) One output: company scale; (c) The corresponding fuzzy sets: T (total assets) = {small, medium, large, extra large}; T (total income) = {low, medium, high, extra high}; T (company scale) = {small, medium, large, extra large}; (3) Determine the membership function of linguistic value, including the types and parameters of the membership function, as shown in Table 1. (a) gaussmf (Gaussian membership function): (1) (b) gauss2mf (two-sided-Gaussian membership function):
166
K.-J. Zhu, P.-C. Chen, and Y.-T. Chang
(2)
(c) trimf (triangle membership function):
(3)
Table 1. Types and Parameters of the Membership Function Linguistic Variable total assets
total income
company scale
Linguistic Value small medium large extra large low medium high extra high small medium large extra large
Membership Function gaussmf gauss2mf gauss2mf gaussmf gaussmf gauss2mf gauss2mf gaussmf trimf trimf trimf trimf
Parameter [0.8, 0] [3.6, 2.75] [0.8, 2.75] [8, 27.5] [12, 27.5] [16, 80] [0.8, 0] [3.6, 2.75] [0.8, 2.75] [8, 27.5] [12, 27.5] [16, 80] [-0.1, 0.0846, 0.2635] [0.2667, 0.4538, 0.6385] [0.632, 0.7778, 0.9154] [0.9154, 1, 1.1]
(4) Determine the fuzzy rules Fuzzy inference rules are usually derived from expert knowledge, fuzzy rules can be specified as follows: (a) If (total assets is small) or (total income is low) then (company scale is small) (b) If (total assets is medium) and (total income is not low) then (company scale is medium) (c) If (total assets is not small) and (total income is medium) then (company scale is medium) (d) If (total assets is large) and (total income is high) then (company scale is large) (e) If (total assets is large) and (total income is extra high) then (company scale is large)
A Credit Risk Rating Model Based on Fuzzy Neural Network
167
(f) If (total assets is extra large) and (total income is high) then (company scale is large) (g) If (total assets is extra large) and (total income is extra high) then (company scale is extra large) (5) Determine fuzzy computing, including fuzzy inference method, fuzzy method, and defuzzification methods (a) Fuzzification of input The fuzzification of input is similar to a corresponding query table or function calculation. The first step to establish a fuzzy inference system is to choose the system input variables and to determine these inputs which must be definite values based on corresponding membership function. The fuzzy inference system uses seven fuzzy inference rules, which regarded inputs as sets of different fuzzy senses. Before using these fuzzy rules, input variables should be fuzzified to obtain their corresponding fuzzy sets. (b) Synthesis operation of input fuzzy sets When the input is fuzzified, the level that it meets corresponding fuzzy rules is known. However, one condition of fuzzy rules is multi-input, so composition operation is required to consider and analyze comprehensively. The input could obtain a value to express its satisfactory level on the rules, and then the value is used in output function. Any reasonable and sound fuzzy synthesis method can be realized with “and” and “or”. In the realization of this model, “and” operation uses the min method, “or” uses the max method. (c) Fuzzy implication The input for calculation of fuzzy implication is from previous synthesis operation of input fuzzy sets, the single value obtained is the fuzzy set. Output is the conclusion fuzzy set derived based on fuzzy rules. Fuzzy implication is the expression problem of fuzzy rules. The MATLAB software contains two methods of calculation. This study selected the min method. (d) Synthesis of output Synthesis of output is the process to integrate the fuzzy sets output from all fuzzy rules. The method of synthesis is not related to the order as well as the synthesis order of rule does not affect the results. After tests, the max method is taken. (e) Defuzzification The input of defuzzification is used as fuzzy set, as well as the output is a value. After the fuzzy inference, the membership function for the scope of output variable is obtained, so it is necessary to carry out defuzzification to make the output a specific value. (6) Obtain the model output Model output is obtained through the training sample data. The input and output surface of fuzzy inference can clearly reflect the fuzzy rules. 4.4 Application of Fuzzy Neural Network in Corporate Credit Risk Rating (1) Model construction with fuzzy neural network Companies’ financial risk and scale ratings are obtained based on fuzzy theory and neural network. Companies’ fundamental factors are added in the final credit risk rating, such input is companies’ total score rated by the experts from quality, strength,
168
K.-J. Zhu, P.-C. Chen, and Y.-T. Chang
environment, credit status, critical incidents, and leader’s quality. And then, it is naturalized into a value among [0,1]. In this way, the rating model for corporate credit risk can be constructed based on fuzzy neural network. On the basis of Takagi-Sugeno inference rules, the model has three inputs, which are financial risk, fundamental risk and company scale. The model has one output, which is the default probability of companies. The default probability refers to the degree of likelihood that the borrower of a loan or debt will not be able to make the necessary scheduled repayments or to fulfill relevant obligations. Under normal circumstances, it means the one-year default probability. It is based on a large number of historical data and obtained with logistic regression analysis. Before loaning to the lender, commercial banks will decide the amount of loan based on the default probability. When conducting a rating, such output data should carried out training for the model, and expertise can be used for reference. The network architecture of fuzzy neural network model has three inputs, one output; the fuzzy layer contains 12 neurons, fuzzy rules layer has 60 rules, which constructs the fuzzy neural network model. (2) Model training Fuzzy toolbox of the MATLAB software provides a command function ANFIS, which obtains membership function parameters of fuzzy inference system through self-adaptive neural network learning. When training the fuzzy system model with training samples, ANFIS can construct fuzzy sets and fuzzy rules adaptively based on samples and relevant given parameters, and adjust the parameters of membership function. Modification in membership function curve of input variables before and after identification fully reflects the self-learning ability of this fuzzy neural network. According to the model, the training data is the mean square error curve, perf = 6.1421e-005, and it demonstrates a good training effect as shown in Fig. 2.
Fig. 2. Model Training
A Credit Risk Rating Model Based on Fuzzy Neural Network
169
(3) Model testing After completing model training, the test sample is applied into the trained ANFIS fuzzy neural network for corporate credit rating. There are 50 test samples which tested with the MATLAB software. The error dependence graph between the actual values and output values is shown in Fig. 3. The average error is 0.0094.
Fig. 3. Error Dependence Graph
4.5 Data Analysis and Assessment According to historical data of the bank, the companies in this study are divided into six levels, according to the default probability which are distributed as follows: 0 very small 0.008; 0.008
≦
≦
≦
≦
≦
≦
≦
Table 2. The Rating Classification of Output Values and Actual Values No 1
Output 0.0092
Actual 0.0132
0.6230
Rating quite small suspicious
2
No 26
Output 0.0075
0.6452
Rating quite small suspicious
27
0.0221
3 4
0.0023 0.6418
very small suspicious
0.0034 0.6484
very small suspicious
28 29
0.9465 0.0104
5
0.8294
loss
0.8237
loss
30
0.8342
Rating quite small quite small loss quite small loss
Actual 0.0064
Rating very small
0.0258
general
0.9484 0.0120
loss quite small loss
0.8367
170
K.-J. Zhu, P.-C. Chen, and Y.-T. Chang Table 2. (continued)
No 6
Output 0.0145
7 8 9
0.8561 0.0015 0.0093
10
0.0094
11 12 13 14
0.4683 0.1018 0.2365 0.9811
15 16 17 18
0.0449 0.0748 0.0052 0.0221
19 20
Rating quite small loss very small quite small quite small attention attention attention loss
Actual 0.0157
Rating quite small loss very small very small
No 31
Output 0.0698
Rating general
Actual 0.0309
Rating General
32 33 34
0.0073 0.1258 0.6674
very small attention suspicious
0.0032 0.1322 0.6633
very small attention suspicious
35
0.0514
general
0.0455
general
0.4600 0.0812 0.2729 0.9737
quite small attention attention attention loss
36 37 38 39
0.1892 0.0545 0.1284 0.0159
0.2452 0.0559 0.1325 0.0081
0.0300 0.0756 0.0035 0.0293
general attention very small general
40 41 42 43
0.5276 0.0776 0.8485 0.1475
attention general attention quite small suspicious general loss attention
0.9477 0.0037
general attention very small quite small loss very small
attention general attention quite small attention attention loss attention
0.9454 0.0010
loss very small
44 45
0.2510 0.0142
21 22
0.7986 0.0049
suspicious very small
0.7894 0.0057
46 47
23
0.0145
0.0161
24 25
0.8765 0.6412
quite small loss suspicious
suspicious quite small quite small loss suspicious
0.8781 0.0024 0.0078 0.0088
0.8721 0.6438
0.5436 0.0681 0.8346 0.1459
0.6802 0.7654
attention quite small suspicious suspicious
0.2204 0.0107 0.6708 0.8001
attention quite small suspicious suspicious
48
0.3505
attention
0.3807
attention
49 50
0.0431 0.1333
general attention
0.0467 0.1458
general attention
5 Conclusions Due to the complexity and high nonlinearity of credit risk, there is a wide deviation between traditional linear model and actual system. This study uses non-linear and generalization capacity based on fuzzy theory and neural network, establishes a credit risk rating model on a basis of fuzzy neural network, which can achieve risk rating and prediction. This model is based on the financial risk model and company scale rating, coupled with the consideration on companies’ fundamental factors, which makes the rating factors more comprehensive, the rating results more scientific and accurate. When constructing the financial risk model, due to neural network’s self-learning, self-adaptive capacity, parallel processing capability and strong fault tolerance, it can obtain more accurate non-linear outputs from various financial indexes. Fuzzy theory expresses knowledge naturally and easily, it is able to process uncertain information, which expresses knowledge with fuzzy and natural language and is easy to use expertise, so it is appropriate to rate the company scale. The combination model of fuzzy theory and neural network provides a good foundation for credit risk rating with fuzzy neural network, making this model with fewer parameters, faster learning, as well as less training samples.
A Credit Risk Rating Model Based on Fuzzy Neural Network
171
This study proposes the combined model based on fuzzy theory and neural network. This model fully utilizes the advantages of both methods, and it rates companies’ credit with a more comprehensive rating index system. On the one hand, it makes the rating index more comprehensive; on the other hand, it makes the model as simple and reliable as possible. In addition, the powerful MATLAB software is used as the development tool in this study, which completes the leap from theory to practice. The results of this study demonstrate an effective model for credit risk rating. This study has combined with practice and theory, verified the use of theoretical models, so as to facilitate the future inferences and follow-up research. Future studies should focus on improving the model proposed in this study, and expanding the sample set to improve generalization ability. It is intended for future researchers to learn and research for a more comprehensive rating index system, as well as to improve the algorithm for better training speed and accuracy.
References [1] Forest, L., Aguais, S.: Implementing a comprehensive credit risk management system: the case study of Hanvit Bank. Hanvit Bank 16(2), 16–30 (2001) [2] Eklund, T., Larsen, K., Berhardsen., E.: Model for analysis credit risk in the enterprise sector. Norges Bank 72(3), 99–106 (2001) [3] Kosko, B.: A Dynamical Approach to Machine Intelligence: Neural Networks and Fuzzy Systems. Prentice-Hall, Englewood Cilffs (1992) [4] Zadeh, L.A.: Fuzzy sets. Information and Control 8, 338–353 (1965) [5] Wang, B.K., Bodnovich, T., Selvi, Y.: Neural network applications in business: A review and analysis of the literature (1988-1995). Decision Support Systems 19, 301–320 (1997) [6] Gomez, F., Schmidhuber, J., Miikkulainen, R.: Accelerated neural evolution through cooperatively coevolved synapses. The Journal of Machine Learning Research 9, 937–965 (2008) [7] Jackson, P., Perraudin, W.: Regulatory implications of credit risk modeling. Journal of Banking and Finance 24(1), 1–24 (2000) [8] Zhang, H., Tam, C.M., Shi, J.: Application of fuzzy logic to simulation for construction operations. Journal of Computing in Civil Engineering 17(1), 38–45 (2003) [9] Haykin, S.: Neural networks: a comprehensive foundation, 2nd edn. Prentice Hall, Upper Saddle River (1999) [10] Kuo, R.: A decision support system for stock market through integration of fuzzy neural network and fuzzy delphi. Applied Artificial Intelligence 12(6), 501–520 (1998) [11] Feng, Y., Lei, Y., Li, Y.: Research on economic simulation based on neural network. In: Proceedings of 2001 International Conference on Management Science & Engineering, Harbin, P. R. China, August 2001, pp. 18–20 (2001) [12] Kuo, R., Wu, P., Wang, C.: An intelligent sales forecasting system through integration of artificial neural networks and fuzzy neural networks with fuzzy weight elimination. Neural Networks 15(7), 909–925 (2002) [13] Leng, G., McGinnity, T., Prasad, G.: An approach for on-line extraction of fuzzy rules using a self-organizing fuzzy neural network. Fuzzy Sets and Systems 150, 211–243 (2005)
Interval-Valued Fuzzy Control Wenyi Zeng1, and Jiayin Wang2 1 College of Information Science and Technology, Beijing Normal University, Beijing, 100875, P.R. China [email protected] 2 School of Mathematical Sciences, Beijing Normal University, Beijing, 100875, P.R. China
Abstract. In this paper, we introduce the concept of interval-valued fuzzy control. Based on the idea of the interpolation mechanism of fuzzy control, we propose the inference algorithm of interval-valued fuzzy inference and mathematical model of interval-valued fuzzy control, investigate the interpolation mechanism of interval-valued fuzzy control. Finally, we use a simulation experiment of interval-valued fuzzy control to illustrate our proposed algorithm reasonable. Keywords: Interval-valued fuzzy set, interpolation mechanism, intervalvalued fuzzy inference, interval-valued fuzzy control.
1
Introduction
Since the fuzzy set was introduced by Zadeh [16], many new approaches and theories treating imprecision and uncertainty have been proposed. Some of these theories, such as the intuitionistic fuzzy sets theory pioneered by Atanassove [1, 2], the generalized theory of uncertainty (GTU) introduced by Zadeh [20], are extensions of the classic fuzzy set theory, and have allowed people to deal with uncertainty and information in a much broader perspective. Another wellknown generalization of an ordinary fuzzy set is the interval-valued fuzzy set, which was first introduced by Zadeh [17, 18, 19]. Since this was introduced, many researchers have investigated this topic and have established some meaningful conclusions. For example, Deschrijver [8] investigated the arithmetic operators of the interval-valued fuzzy set theory, and Zeng [21, 23] investigated decomposition theorem and representation theorem of the interval-valued fuzzy set, respectively. Moreover, some researchers has pointed out that there is a strong connection between Atanassov’s intuitionistic fuzzy sets and the interval-valued fuzzy sets. For more details, readers can refer to [7]. Fuzzy inference is an important topic in fuzzy set theory. Many researchers have investigated it from different points of view and proposed many different mathematical models of fuzzy inference. These different fuzzy inference models
This work is supported by grants from the National Natural Science Foundation of China (10971243). Corresponding author.
Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 173–183. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com
174
W. Zeng and J. Wang
and inference algorithms have been applied rapidly in many areas such as fuzzy control and pattern recognition according to their feature. For example, Wang [14] proposed true-value flow inference theory and inference algorithm, Zeng [22] proposed inner product truth-valued flow inference algorithm, Wu [15] investigated the principle and method of fuzzy inference. Applying fuzzy inference into fuzzy control, scholars have done a lot of meaningful works. For example, Li [11] proposed the interpolation mechanism of fuzzy control and applied fuzzy control in the research of inverted pendulums. Li [12, 13] investigated adaptive fuzzy controllers based on variable universe and the relationship between fuzzy controllers and PID controllers, respectively. Since the interval-valued fuzzy set was introduced, researchers have started to investigate fuzzy inference of interval-valued fuzzy sets. Some researchers have proposed some different approximate reasoning methods of interval-valued fuzzy sets. For example, Gorzalczany [9, 10] proposed fuzzy inference methods of interval-valued fuzzy sets and discussed its some properties, respectively. Bustince [3] proposed approximate reasoning models of interval-valued fuzzy sets based on the interval-valued fuzzy relations. Chen [4, 5] investigated the bidirectional approximate reasoning methods of interval-valued fuzzy sets based on the similarity measure of interval-valued fuzzy sets and direction index of matching function. Chun [6] applied the similarity-based bidirectional approximate reasoning method in decision-making systems. In this paper, we introduce the concept of interval-valued fuzzy control, propose the inference algorithm of interval-valued fuzzy inference and the mathematical model of interval-valued fuzzy control, investigate the interpolation mechanism of interval-valued fuzzy control. Finally, we use a simulation experiment of interval-valued fuzzy control to illustrate our proposed algorithm reasonable. The organization of our work is as follows. In section 2, some basic notions of interval-valued fuzzy set are reviewed. In section 3, we propose the mathematical model of interval-valued fuzzy control and investigate the interpolation mechanism of interval-valued fuzzy control. In section 4, we use a simulation experiment of interval-valued fuzzy control to illustrate our proposed algorithm reasonable. The conclusion is given in the last section.
2
Some Notions
Throughout this paper, we use X and Y to denote the input discourse set and the output discourse set, respectively, IVFSs(X) and IVFSs(Y ) stand for the set of all interval-valued fuzzy subsets in X and Y , respectively. A and B express the interval-valued fuzzy set in X and Y , respectively, the operation “c” stands for the complement operation of interval-valued fuzzy set or fuzzy set, ∅ stands for the empty set. Let L([0, 1]) be the set of all closed subintervals of the interval [0, 1]. Especially, for an arbitrary element a ∈ [0, 1], we assume that a is the same as [a, a], namely, a = [a, a]. Then, according to Zadeh’s extension principle [16], − + for any a = [a , a ], b = [b− , b+ ] ∈ L([0, 1]), we can popularize some operations such as , , and c to L([0, 1]), and have a b = [a− b− , a+ b+ ],
Interval-Valued Fuzzy Control
a
t∈W
b = [a− at = [
b − , a+
t∈W
a− t ,
b+ ], ac = [1 − a+ , 1 − a− ],
t∈W
at = [
a− t ,
t∈W
175
a+ t ] and
t∈W
a+ t ], where W denotes an arbitrary index set. Further-
t∈W
more, we have a = b ⇐⇒ a− = b− , a+ = b+ , a b ⇐⇒ a− b− , a+ b+ and a < b ⇐⇒ a b and a = b; then there exists a minimal element 0 = [0, 0] and a maximal element 1 = [1, 1] in L([0, 1]). We call a mapping, A : X −→ L([0, 1]) an interval-valued fuzzy set in X. For every A ∈ IVFSs(X) and x ∈ X, then A(x) = [A− (x), A+ (x)] is the degree of membership of an element x to the interval-valued fuzzy set A. Thus, fuzzy sets A− : X → [0, 1] and A+ : X → [0, 1] are called low and upper fuzzy sets of the interval-valued fuzzy set A, respectively. For simplicity, we denote A = [A− , A+ ]. F (X) and P(X) stand for the set of all fuzzy sets and crisp sets in X, respectively. Specially, if X = {x1 , x2 , . . . , xn }, then A(xi ) = [ai1 , ai2 ], for every xi ∈ X, i = 1, 2, · · · , n. Thus, we have A = {(x1 , [a11 , a12 ]), (x2 , [a21 , a22 ]), . . . , (xn , [an1 , an2 ])} where [ai1 , ai2 ](0 ≤ ai1 ≤ ai2 ≤ 1, i = 1, 2, · · · , n) is the degree of membership of an element xi to the interval-valued fuzzy set A. If A, B ∈IVFSs(X), then the following operations can be found in Zeng [21]. A ⊆ B if and only if ∀x ∈ X, A− (x) ≤ B − (x) and A+ (x) ≤ B + (x), A = B if and only if ∀x ∈ X, A− (x) = B − (x) and A+ (x) = B + (x), (A)c = [(A+ )c , (A− )c ], (A ∩ B)(x) = [A− (x) ∧ B − (x), A+ (x) ∧ B + (x)], ∀x ∈ X, (A ∪ B)(x) = [A− (x) ∨ B − (x), A+ (x) ∨ B + (x)], ∀x ∈ X.
3
Interpolation Mechanism of Interval-Valued Fuzzy Control
Recently, Li [11, 12, 13] has pointed out that some common fuzzy inference algorithms in fuzzy control can be attributed to some kind of interpolation method, and has proved that fuzzy control algorithm based on the implication operator R(a, b) = 0 ∨ (a + b − 1) can be attributed some kind of interpolation system. To introduce the nomenclatures, we will consider a reasonably simple intervalvalued fuzzy control system with one-input and one-output. Let X = [a, b] be the universe of input variable x, and Y = [c, d] be the universe of output variable y. Denoting A = {Ai }(1≤i≤n) , B = {B i }(1≤i≤n) , Ai ∈ IVFSs(X), B i ∈ IVFSs(Y ), and A, B are linguistic variables, thus, we can form the fuzzy inference rules as follows: if x is Ai then y is B i where i = 1, 2, · · · , n; x ∈ X, y ∈ Y .
176
W. Zeng and J. Wang
For a given interval-valued fuzzy set A , according to Zadeh’s compositional rule of inference, then we have the mathematical model of interval-valued fuzzy inference in the following. Model n
B =
A ◦ Ri =
i=1
n
A ◦ (Ai → B i )
i=1
where Ri (x, y) = (Ai → B i )(x, y) = [Ri − (x, y), Ri + (x, y)], Ri − (x, y) = max(0, Ai − (x) + Bi − (y) − 1), Ri + (x, y) = Ai + (x) ∧ Bi + (y), i = 1, 2, · · · , n. Hence, the conclusion of interval-valued fuzzy inference can be determined as follow. − + B = [B , B ] −
where B (y) =
n
n
(A− (x) ∧ Ri − (x, y)), B (y) = +
x∈X i=1
(A+ (x) ∧
x∈X i=1
Ri + (x, y)). In general, the input to the fuzzy logic system is crisp or real quantity, thus, it must be fuzzified. We will adopt the following most frequently used fuzzification system for every x = [x , x ], 1, x = x A (x) = 0, x = x
Then, we will obtain an interval-valued fuzzy set A , A = [A− , A+ ], where A − = A + = A . Using the above fuzzification system, we have B = [B − , B + ], where n n − + Ri − (x , y), B (y) = Ri + (x , y), To obtain a crisp quantity, B (y) = i=1
i=1
the interval-valued fuzzy set B will be defuzzified by the use of the following centroid method: − + yB (y)dy yB (y)dy + − Y Y , y = y = + − B (y)dy B (y)dy Y
Y
where
n
−
B (y) =
−
−
(max(0, Ai (x ) + Bi (y) − 1))
i=1
B (y) = +
n
(Ai + (x ) ∧ Bi + (y))
i=1
Thus, for the interval-valued fuzzy control system with one-input and oneoutput, we find that the interval-valued fuzzy control system can be transformed
Interval-Valued Fuzzy Control
177
two kinds of general fuzzy control systems. Namely, they are two different kinds of fuzzy inference rules, respectively. These two kinds of fuzzy inference rules are illustrated as follows. ifxisAi − thenyisBi − , i = 1, 2, · · · , n and ifxisAi + thenyisBi + , i = 1, 2, · · · , n
4
Simulation Experiment of Interval-Valued Fuzzy Control
For convenience, we use the mathematical model of controlled object, x˙ + x = u(t), to do the simulation experiment, where x(0) = −1, u(t) is out action. The control target is to trace pulse signal, 1, t > 0, r(t) = 0, else For u(t) ≡ 0, we can obtain the response curve for being controlled object, then the response curve for being controlled object is listed as Fig. 1. t
In our experiment, we use ID controller, u(t) = KI
e(τ )dτ , where KI =
0
10 is proportion coefficient, e(t) = r(t) − x(t) is system error, the response curve 3 for the system is listed as Fig. 2. 0
−0.1
−0.2
Response curve
−0.3
−0.4
−0.5
−0.6
−0.7
−0.8
−0.9
−1
0
1
2
3
4
5 Time (s)
6
7
8
9
Fig. 1. The response curve for being controlled object
10
178
W. Zeng and J. Wang
2
1.5
Response curve
1
0.5
0
−0.5
−1
0
1
2
3
4
5 Time (s)
6
7
8
9
10
Fig. 2. The response curve for the system on ID
In the following, we give two kinds of general membership functions, their membership functions are listed as follows and their membership function curves are listed as Fig. 3. For k > 1, ⎧ k x−a ⎪ ⎪ , x ∈ [a b], ⎪ ⎪ ⎨ b−a k
w minus mf (x, [a b c]) = x−c ⎪ , x ∈ [b c], ⎪ ⎪ ⎪ ⎩ b−c 0, else ⎧x−a ⎪ ⎪ ⎨ b − a , x ∈ [a b], trimf (x, [a b c]) = x − c , x ∈ [b c], ⎪ ⎪ ⎩ b−c 0, else In the design of interval-valued fuzzy controller, we choose input variable as t error integral, x(t) = e(τ )dτ , output variable as control quantity u(t), where 0
input variable universe X = [−3, 3], output variable universe U = [−10, 10]. Fuzzy partition of input variable universe is listed as Fig. 4. We use five intervalvalued fuzzy sets, A1 , A2 , A3 , A4 , A5 , to denote such linguistic variables as “NL”, “NM”, “ZR”, “PM”, “PL”, respectively. These membership functions are listed as follows, respectively. + A1 (x) = A− 1 (x), A1 (x) , + A− 1 (x) = w minus mf (x, [−3 − 3 − 1.5]), A1 (x) = trimf (x, [−3 − 3 − 1.5]), + A2 (x) = A− 2 (x), A2 (x) ,
Interval-Valued Fuzzy Control
179
1
Membership Function
k=1 k>1
0.5
0
a
b x
c
Fig. 3. Membership function curves + A− 2 (x) = w minus mf (x, [−3 − 1.5 0]), A2 (x) = trimf (x, [−3 − 1.5 0]), − A3 (x) = A3 (x), A+ 3 (x) , + A− 3 (x) = w minus mf (x, [−1.5 0 1.5]), A3 (x) = trimf (x, [−1.5 0 1.5]), − A4 (x) = A4 (x), A+ 4 (x) , + A− 4 (x) = w minus mf (x, [0 1.5 3]), A4 (x) = trimf (x, [0 1.5 3]), − A5 (x) = A5 (x), A+ 5 (x) , + A− 5 (x) = w minus mf (x, [1.5 1.5 3]), A5 (x) = trimf (x, [1.5 1.5 3]).
Similarly, for the fuzzy partition of output universe, we use five intervalvalued fuzzy sets, B 1 , B 2 , B 3 , B 4 , B 5 , to denote such linguistic variables as “NL”, “NM”, “ZR”, “PM”, “PL”, respectively. Their membership functions are listed as follows, respectively. B 1 (u) = B1− (u), B1+ (u) , B1− (u) = w minus mf (u, [−10 −10 −5]), B1+ (u) = trimf (u, [−10 −10 −5]), B 2 (u) = B2− (u), B2+ (u) , B2− (u) = w minus mf (u, [−10 − 5 0]), B2+ (u) = trimf (u, [−10 − 5 0]), B 3 (u) = B3− (u), B3+ (u) , B3− (u) = w minus mf (u, [−5 0 5]), B3+ (u) = trimf (u, [−5 0 5]), B 4 (u) = B4− (u), B4+ (u) , B4− (u) = w minus mf (u, [0 5 10]), B4+ (u) = trimf (u, [0 5 10]), B 5 (u) = B5− (u), B5+ (u) , B5− (u) = w minus mf (u, [5 10 10]), B5+ (u) = trimf (u, [5 10 10]).
180
W. Zeng and J. Wang
Membership Function
1
0.66
k=1.5
0.33
0 −3
−1.5
0 x
1.5
3
Fig. 4. Psedu-fuzzy partition of A− and A+ on input universe X
Then, based on five kinds of fuzzy inference rules in the following, we can construct the rule base for interval-valued fuzzy inference. Rule 1: if x(t) is A1 then u(t) is B 1 ; Rule 2: if x(t) is A2 then u(t) is B 2 ; Rule 3: if x(t) is A3 then u(t) is B 3 ; Rule 4: if x(t) is A4 then u(t) is B 4 ; Rule 5: if x(t) is A5 then u(t) is B 5 . According to the interpolation mechanism of interval-valued fuzzy control, we can obtain the control function curve for the interval-valued fuzzy inference system: u = F (x) = F − (x), F + (x) where F − (x) =
5 i=1
+ A− i (x)yi and F (x) =
5 i=1
A+ i (x)yi , yi are the peak points of
fuzzy sets both Bi− and Bi+ on output universe. For the convenience, we order k = 1.5, then we can obtain their control function curve on interval-valued fuzzy controller which is listed as Fig. 5, and the response curve for the system on interval-valued fuzzy controller which is listed as Fig. 6.
Interval-Valued Fuzzy Control
181
10
u
5
0
u− u+ −5
−10 −3
−1.5
0 x
1.5
3
Fig. 5. Control function curve on interval-valued fuzzy controller 2
u− u+ 1.5
Response curve
1
0.5
0
−0.5
−1
0
1
2
3
4
5 Time (s)
6
7
8
9
10
Fig. 6. The response curve for the system
5
Conclusion
Considering the importance of fuzzy inference and fuzzy control, in this paper, we introduce the concept of interval-valued fuzzy control. Based on the idea on the interpolation mechanism of fuzzy control proposed by Li [11], we propose the mathematical model of interval-valued fuzzy control and investigate the interpolation mechanism of interval-valued fuzzy control. Firstly, we investigate the mathematical model of interval-valued fuzzy inference and obtain an interval-valued fuzzy set as the conclusion of the interval-valued fuzzy inference. Secondly, we use an interval to denote the response value of the interval-valued
182
W. Zeng and J. Wang
fuzzy control, and obtain an interval belt of the response curve of interval-valued fuzzy control, not as a single value and a single curve. Finally, we use a simulation experiment of interval-valued fuzzy control to illustrate our proposed algorithm reasonable. We believe that the design of interval-valued fuzzy controller will induce more and more fuzzy technology. These technology will be extensively applied in many fields such as pattern recognition, image processing, approximate reasoning, fuzzy control, and so on.
Acknowledgements This work is supported by grants from the National Natural Science Foundation of China (10971243). And the authors are grateful to the anonymous referees for their great comments and valuable suggestions which helped us improve this paper.
References [1] Atanassov, K.: Intuitionistic fuzzy sets. Fuzzy Sets and Systems 20, 87–96 (1986) [2] Atanassov, K.: Intuitionistic Fuzzy Sets: Theory and Applications. PhysicaVerlag, Heidelberg (1999) [3] Bustince, H., Burillo, P.: Mathematical analysis of interval-valued fuzzy relations: Application to approximate reasoning. Fuzzy Sets and Systems 113, 205–219 (2000) [4] Chen, S.M., Hsiao, W.H., Jong, W.T.: Bidirectional approximate reasoning based on interval-valued fuzzy sets. Fuzzy Sets and Systems 91, 339–353 (1997) [5] Chen, S.M., Hsiao, W.H.: Bidirectional approximate reasoning for rule-based systems using interval-valued fuzzy sets. Fuzzy Sets and Systems 113, 185–203 (2000) [6] Chun, M.G.: A similarity-based bidirectional approximate reasoning method for decision-making systems. Fuzzy Sets and Systems 117, 269–278 (2001) [7] Deschrijver, G., Kerre, E.E.: On the relationship between some extensions of fuzzy set theory. Fuzzy Sets and Systems 133, 227–235 (2003) [8] Deschrijver, G.: Arithmetic operators in interval-valued fuzzy set theory. Information Sciences 177, 2906–2924 (2007) [9] Gorzalczany, M.B.: A method of inference in approximate reasoning based on interval-valued fuzzy sets. Fuzzy Sets and Systems 21, 1–17 (1987) [10] Gorzalczany, M.B.: An interval-valued fuzzy inference method -some basic properties. Fuzzy Sets and Systems 31, 243–251 (1989) [11] Li, H.X.: Interpolation mechanism of fuzzy control. Science in China, Ser. E 41(3), 312–320 (1998) [12] Li, H.X.: Adaptive fuzzy controllers based on variable universe. Science in China, Ser. E 42(1), 10–20 (1999) [13] Li, H.X.: Relationship between fuzzy controllers and PID controllers. Science in China, Ser. E 42(2), 215–224 (1999) [14] Wang, P.Z., et al.: Mathematical theory of truth-valued flow inference. Fuzzy Sets and Systems 72, 221–238 (1995) [15] Wu, W.M.: Principle and Method of Fuzzy Inference. Guizhou Science and technology Press, Guizhou (1994) (in Chinese)
Interval-Valued Fuzzy Control
183
[16] Zadeh, L.A.: Fuzzy sets, Inform. Control 8, 338–353 (1965) [17] Zadeh, L.A.: The concept of a linguistic variable and its application to approximate reasoning(I). Inform. Sci. 8, 199–249 (1975) [18] Zadeh, L.A.: The concept of a linguistic variable and its application to approximate reasoning (II) 8, 301–357 (1975) [19] Zadeh, L.A.: The concept of a linguistic variable and its application to approximate reasoning(III). Inform. Sci. 9, 43–80 (1975) [20] Zadeh, L.A.: Toward a generalized theory of uncertainty(GTU)—-an outline. Information Sciences 172, 1–40 (2005) [21] Zeng, W.Y., Shi, Y.: Note on interval-valued fuzzy set. In: Pal, N.R., Kasabov, N., Mudi, R.K., Pal, S., Parui, S.K. (eds.) ICONIP 2004. LNCS, vol. 3316, pp. 20–25. Springer, Heidelberg (2004) [22] Zeng, W.Y., Li, H.X.: Inner product truth-valued flow inference, International Journal of Uncertainty. Fuzziness and Knowledge-Based Systems 13(6), 601–612 (2005) [23] Zeng, W.Y., Shi, Y., Li, H.X.: Representation theorem of interval-valued fuzzy set, Internat. J. Uncer. Fuzzi. Knowloedge-Based Systems 14, 259–269 (2006)
Research on Fuzzy Preference Relations-Based MAS for Decision Method Weijin Jiang1 and Qing Jiang2 1
School of Computer and Electronic Engineering, Hunan University of Commerce, Changsha 410205, China [email protected] 2 School of Computer, South University of Technology, Guangzhou 510092, PRC Abstract. The use of multiple intelligent agents in the decision support systems has become common. However, there are many questions in this field, such as the consensus of the agents and the method on choosing the best decision. In order to solve these problems, this paper provides a method, which uses the fuzzy aggregation operator to aggregate the opinions of the agents, ranks the alterative with the fuzzy preference relations, and then chooses the best alternative. This approach can be widely used in many fields. Keywords: Agent, Fuzzy preference relations, Fuzzy satisfaction function.
1
Introduction
The use of multiple intelligent agents in the decision support systems has become common. Distributional artificial intelligence technology took one kind of establishment in the distributional control structure foundation intelligent method integration method, by its unique superiority by the people was being gradually known and takes [1-5]. But also brings some questions along with it. First, because person's thought has the inherent fuzziness, the person is accustomed to uses the natural language description correlation concept, when the agent provides a quantity, the explicit policy-making result [6-8], the person very difficultly makes to it conforms with the intuition the judgment, Moreover at very many times, the agent also is according to incompletely, the indefinite information makes the decisionmaking, in this kind of situation [9], the agent very is also difficult to provide a precise policy-making result, therefore in this article the agent decision-making result indicated with the fuzzy collection [10], causes its output is not a simple numeral, but is a subordination degree, namely a fuzzy concept. The second question lies in: Because various agent looked the question the angle quite same not less than, can obtain also is different, like this very possibly causes the decision-making which some main bodies makes to be mutually contradictory. Then, how obtains to conform to the objective policy-making result then to become a key question [11]. Some literature proposed through neglects achieves the result with the majority main body policy-making result contradictory main body policy-making result method the uniformity. But in very many situations this method is not suitable. The third question is: How obtains finally the reasonable policy-making plan. In the article agent use fuzzy collection expresses the policy-making result [12-15], then becomes a difficulty to the fuzzy number arrangement. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 185–192. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
186
W. Jiang and Q. Jiang
Above in order to solve the problem, the author proposed one kind of fuzzy policymaking result polymerization algorithm, and through fuzzy relates by chance to the polymerization result carries on the arrangement [16], obtains finally the reasonable policy-making plan. This article content arrangement as follows: The second part of introduction fuzzy intelligence main body and its the polymerization algorithm, the third part proposed based on fuzzy number arrangement method which fuzzy relates by chance, the fourth part of simulation result, finally summarizes the full text.
2 Fuzzy Agent Since 1965 Zadeh proposed the fuzzy collection theory, the fuzzy collection obtained the widespread application in many domains. The fuzzy collection distinguishes to the classics gathers mainly lies in a subordination function. The classics gather discuss use two values functions to take it subordinates a function:
⎧1x∈A ⎩0x∈A
μA = ⎨
A fuzzy collection subordination function finally will map [0, 1] on, namely: . The such fuzzy collection may express some fuzzy concepts well, for example: Young, middle age, concept and so on old age. Thus even more conforms to person's thinking mode. The fuzzy collection subordination function type has very many kinds, quite commonly used has the triangle to subordinate a function, a trapezoid subordination function, a subordination function. This article supposition already knew policy-making in the policy-making question plan collection S= {S1, S2... Sm}, various agent embarks from own angle to each policy-making plan puts forward the proposal, then the so-called fuzzy agent decision-making is refers, various agent does not produce the accurate judgement, which plan uses, but is the use fuzzy collection theory produces the fuzzy judgement to various policy-making plan. In fuzzy process, subordination function selection count for much. This article is the simplification operation, a selection triangle subordination function. A typical triangle subordination function like figure 1 shows.
μ 1
a
c
b
x
Fig. 1. Triangular fuzzy numbers
In order to coordinate various agent the work, this article first the supposition various intelligences main body all can real-time produce the policy-making result.
Research on Fuzzy Preference Relations-Based MAS for Decision Method
187
The author proposed the method is: Various main bodies after analyze the data which itself obtains, produces own to the corresponding policy-making plan policy-making opinion, the natural policy-making opinion produces by the fuzzy collection form, in this foundation, the use fuzzy polymerization operator gets together each agent to the specific policy-making plan policy-making opinion, finally, carries on these polymerization results the arrangement, like this may obtain the best policy-making plan, but this plan also is the multi- agent unification policy-making result. Then the question key lies in seeks an appropriate polymerization operator. Generally says, fuzzily gets together the operator is a solid mapping function:
∫ : [0,1]
→ [0,1] [8]
∫
below has the nature:
∫ ( 0)
= 0 ∫ (1) = 1 , Among
them, the parameter 0, 1 is the n Uygur vector. Many literature proposed the corresponding polymerization operator, for example, Yanger proposed OWA [6] (Ordered Weighted Averaging) the method, this operator use language classifier has carried on the fuzzy number polymerization; D. DuBois [7] and so on proposed in Multicnieria decision uses the Choquet integral method to take the polymerization operator, this operator use fuzzy measure, the use fuzzy Choquet integral method carries on the polymerization computation. These methods have obtained the success in many domains, but they all have a very big flaw, namely computation complex is very high, like this in the timely request quite high situation, the military situation, cannot answer the purpose in particular. Such author considered uses the simple weighing method, concrete method as follows: (1) Acts according to various agent the important degree difference, entrusts with a power for each agent ωi ; (2) Carries on the power value normalized processing: '
ωi =
ωi'
∑ω i =1
;
, i=1…n
n
' i
(3) Supposes the ith agent to the kth policy-making plan policy-making result ~
is R ki Then all agents to the kth policy-making plan policy-making polymerization result is: ~
R ki =
)
ck ,
~
~ ~ ⎤ 1 ⎡ n ( 1 R ) 1 . Among for parameter, R + μω − μ ki = ak, bk, ki ∏ i ⎥ μ ⎢⎣ i = 1 ⎦
(
(
)
R ki = aki, bki, cki .
3 The Model of Multi-agent Decision-Making In obtains the main body after all policy-making plan policy-making result, next step must choose the best policy-making plan, takes the final policy-making result. Then
188
W. Jiang and Q. Jiang
must carry on the arrangement to various policy-making plan policy-making result. The fuzzy number arrangement does not look like fine arrangement such simple, is direct. Has very many literature to the fuzzy number arrangement to propose many methods [2-3], these methods mostly concentrate to two aspects: (1) defines a arrangement function, it will blur the number to map in the real axis, then will carry on the arrangement; (2) the second method is establishes one kind of fuzzy function, the use fuzzy by chance function carries on the arrangement by chance. Revolves many fuzzy numbers arrangements method which these two aspects proposed, although has all obtained the success in the literature, but does not have one method all to obtain satisfaction in each kind of situation the arrangement result. The main question lies in the arrangement result which some algorithms produces is violates the intuition [4], and has some methods to have the certain limit to the fuzzy number shape, For example Nakamura and so on proposes based on the arrangement method which fuzzy relates by chance[3]. Compares says, person and so on KeonMyung Lee proposed the use degree of satisfaction function carries on the fuzzy number arrangement the algorithm first is lets the person in the intuition accept, and it has not requested, the result to the fuzzy number shape which obtains also is satisfying, therefore this article mainly uses this method to carry on the fuzzy number arrangement. Under separately introduces several main concepts. The fuzzy degree of satisfaction function is one kind of fuzzy measure, it produced two fuzzy numbers to conform to some comparison relations degree, here comparison relations was the arithmetic comparison relations which in the usual significance said{< = >} Under produces the correlation definition.
,, 。
Defines 1: Fuzzy collection Aγ-resolution collection D gamma (A), definition as follows:
D γ (A) = {x | x = ηγ , x ∈ Supp(A ), n ∈ N} Among
them,
Supp
(A)
is
;
a fuzzy collection A collection, N is the integer collection. Is worthy of looking
Supp(A) = {x | μ A (x) > 0} at from its definition, D γ (A) will continual real number field separate.
Defines 2: Fuzzy degree of satisfactionis ϕ , Supposes A, B is two fuzzy numbers,
ϕ(ARB) )expresses A, B satisfies relates R the degree, Obviously has: ϕ(ARB) ∈ [0,1] . Here about R is{< = >} ϕ (ARB) =1 expresses A, B completely satisfies relates R, On the contrary, ϕ ARB =0 expresses A, B completely does not satisfy relates R.
Then
,, 。
(
)
Defines 3: Fuzzy degree of satisfaction function Sγ
(A
j
Research on Fuzzy Preference Relations-Based MAS for Decision Method
S γ (Ai < Aj) =
j
min(y ~ γ )
y= j
x =d
∑ ∑μ I
Ai
x =i y= j
( x ) ⊗ μ A i ( y) (1)
J
∑∑μ
189
Ai
( x ) ⊗ μ A i ( y)
Among them, Imin=minDγ (Ai), Imin=maxDγ (Ai), Jmin=minDγ (Ai), Imax=maxDγ (Ai), ⊗ for T- nom operator, in article, Chooses min operator which Zadeh proposed. Similar may define: Defines 4: Fuzzy degree of satisfaction function Sγ j
min(y ~ γ )
y= j I
x=d J
∑ ∑μ
S γ (Ai > Aj) =
∑∑μ x =i y = j
S γ (Ai = Aj) =
x =d J
∑∑ μ
(
x =i y = j
j
( x ) ⊗ μ A i ( y) (2)
Ai
j min(y ~ γ )
y= j I
i
Ai
( x ) ⊗ μ A i ( y)
Defines 5: Fuzzy degree of satisfaction function Sγ
∑ ∑μ
(A >A ),defines as follows:
Ai
(A =A ),defines as follows: i
j
( x ) ⊗ μ A i ( y) (3)
Ai
( x ) ⊗ μ A i ( y)
) (
) (
)
Defines by above obviously has: Sγ AiAj =1. In this section, will define in the above fuzzy degree of satisfaction function foundation Fuzzy preference relations. The supposition various intelligences main body to various policy-making plan policy-making opinion polymerization result is: A={A1 A2 …An} Ai is the agent to the policy-making plan i decision-making opinion polymerization result, n is the policy-making plan integer which mutually competes. Then fuzzy relates the definition is by chance:
, ,
,
Defines 6: About fuzzy degree of satisfaction function Sγ Fuzzy preference relations Rs is:
R s ( Ai , A j ) = S γ ( Ai > A j ) +
Rs : A × A → [0,1],
Fuzzily by chance relates satisfies:
(1)R (A ,A )+R R (A ,A )=1, ∀A , A (2)R (A ,A )=0.5, ∀A ∈ A 。 s
i
j
s
i
j
s
i
j
i
j
∈ A
1 S γ ( Ai = A j ) (4) 2
;
i
By definition which fuzzy relates by chance below, may obtain the conclusion. As shown in table 1:
190
W. Jiang and Q. Jiang Table 1. Conclusion
Rs (Ai, Aj) value scope
Conclusion
1
The plan i completely surpasses plan j
[0.5, 1]
The plan i surpasses plan j
0.5
The plan i and plan j is equal
[0, 0.5]
Plan j surpasses the plan i
0
Plan j completely surpasses the plan i
After had fuzzily has related by chance, obviously was allowed through fuzzy to relate by chance to polymerization result A carries on the arrangement, obtained the best policy-making plan. The literature [4] center has produced the sort algorithm, but considered the question the particularity, namely this article main goal lies in found the best policy-making plan, but certainly not nice other plans fit and unfit quality orders. Such literature [4] center produces the sort algorithm appeared too complex; thereupon here proposed simple sought the most superior plan the algorithm. Algorithm 1 Seeks most superior policy-making plan. Step1: Input intelligence main body policy-making polymerization result A, as well as parameter γ; Step2: Calculates fuzzy degree of satisfaction function Sγ, and fuzzy relates Rs by chance; Step3: To all Ai, Aj A calculates O (Ai) = min R i ( A i , A j ) ;
∈
Step4
: N = ⎧⎨⎩A
i
i= j
| 0(Ai) = max 0(A j )⎫⎬, alternative = i ; A J ∉A ⎭
Step5: Returns to most superior policy-making plan alternative. Algorithm conclusion.
4 Simulation and Discussion In order to confirm the algorithm the accuracy, a this article structure simulation environment, the supposition in a policy-making question, altogether had 6 agent, its important degree respectively is: (0.8, 0.5, 0.4, 0.6, 0.2, 0.7), After the normalized weight is: (0.25, 0.16, 0.13, 0.19, 0.06, 0.21), the policy-making marquis chooses the plan altogether to have 4, namely S= {S1, S2, S3, S4}, after the stroke operation, the decision system chooses the final fuzzy appraisal to various marquises which the plan obtains for (A1, A2, A3, A4), Respectively is: A1= (0.2, 0.3, 0.55), A2= (0.12, 0.48, 0.9), A3= (0.45, 0.6, 0.65), A4= (0.71, 0.9, 1.0), its result like figure 2 shows. Obtains fuzzily through the fuzzy degree of satisfaction functional calculus relates like table 2 to show by chance.
Research on Fuzzy Preference Relations-Based MAS for Decision Method Table 2. Fuzzy Preference Relations(
191
γ=0.01)
Rs
A1
A2
A3
A4
O
A1
0.500
0.618
0.200
0.007
0.007
A2
0.380
0.500
0.010
0.150
0.010
A3
0.800
0.990
0.500
0.000
0.000
A4
0.993
0850
1.000
0.500
0.850
Simulation result plan A4 most melts, then the final policy-making plan is A4. ha rt
3
1
0.2
0.4
0.6
4
0.8
1.0
Fig. 2. A1,A2,A3, A4fuzzy value result
5 Conclusions The multi- agent technology development, causes the multi- main bodies technology to be allowed to enhance policy-making the accuracy and the policy-making efficiency, displays the multi- main bodies decision-making well the resources superiority. This article in sums up experience in the foundation, proposed based on multi- agent decision-making technology which fuzzy relates by chance, causes policy-making the result to be more reasonable, the experiment indicated, this method has the certain confidence level. This article provides the method may in the widespread domain application. Because its policy-making efficiency greatly enhances, the timely enhancement, enables it to have the very good application prospect in the tactical decision-making. Waits for question which further studies: Various agent weight factor determination, at present has the subjective factor; Various agent fuzzy number production, namely the fuzzy subordination function expression also has the certain subjectivity.
192
W. Jiang and Q. Jiang
Acknowledgments This work is supported by the Society Science Foundation of Hunan Province of China No. 07YBB239, and the Important Natural Science Foundation of Hunan Province of China No. 06JJ2033.
References 1. Bredin, J., Kotz, D., Rus, D., Maheswaran, R.T., Imer, C., Basar, T.: Computational markets to regulate mobile-agent systems. Autonomous Agents and Multi-Agent Systems 6(3), 235–263 (2003) 2. Lind, J.: Issues in Agent-Oriented Software Engineering. In: The First International Workshop on Agent-Oriented Software Engineering (AOSE 2000) (2000) 3. Fortemps, P., Roubens, M.: Ranking and defuzzification methods based on area Z compensation. Fuzzy Sets and Systems 82, 319–330 (1996) 4. Scherrer, B., Charpillet, F.: Cooperative Co-learning: A Model-based Approach for solving MultiAgent Reinforcement Problems. In: Proceedings 14th IEEE International Conference on Tools with Artificial Intelligence, pp. 463–468 (2002) 5. Nakanura, K.: Preference relations on a set of fuzzy utilities as basis for decision making. Fuzzy Sets and Systems 20, 147–162 (1986) 6. Jiang, W.J., Wang, P.: Research on Distributed Solution and Correspond Consequence of Complex System Based on MAS. Journal of Computer Research and Development 43(9), 1615–1623 (2006) 7. Liu, j., Jing, H., Tang, Y.Y.: Multi-agent oriented constraint satisfaction. Artificial Intelligence 136(1), 101–144 (2002) 8. Yager, R.R.: Quantifier Guided Aggregation Using OWA Operations. International Journal of Intelligent Systems 11, 49–73 (1996) 9. Weicai, Z., Jin, L., Mingzhi, X., et al.: A multi agent genetic algorithm for global numerical optimization. IEEE Transactions on System, Man, and Cybernetics 34(2), 1128– 1141 (2004) 10. Bredin, J., Kotz, D., Rus, D., Maheswaran, R.T., Imer, C., Basar, T.: Computational markets to regulate mobile-agent systems. Autonomous Agents and Multi-Agent Systems 6(3), 235–263 (2003) 11. Busoniu, L., Babuska, R., De Schutter, B.: A Comprehensive Survey of Multiagent ReinforcementLearning. IEEE Transactions On System, Man, And Cybernetics-Part C: Application And Review 38(2), 156–172 (2008) 12. Gou, Y., Huan, J.Z., Rong, H.: Adaptive grid job allocation with genetic algorithm. In: Future genre. comp. sys., vol. 21, pp. 151–161. Elsevier Press, London (2005) 13. Matsui, M.: A generalized model of convey-serviced production station (CSPS). Journal of Japan IndustrialManagement Association 44(1), 25–32 (1993) 14. Yamada, T., Satomi, K., Matsui, M.: Strategic selection of assembly system under viable demands. Assembly Automation 26(4), 335–342 (2006) 15. Dubois, D., Grabisch, M., Modave, F., et al.: Relating decision under uncertainty and multieniteria decision making models. In: IJCAI, Nagoya, Japan (1997) 16. Jiang, W.J., Wang, P., Zhang, L.: Research on Grid Resource Scheduling Algorithm Based on MAS Cooperative Bidding Game. Chinese Science F 52(8), 1302–1320 (2009)
Study of Stochastic Demand Inventory Routing Problem with Soft Time Windows Based on MDP Wei Zeng and Qian Zhao Institute of Systems Engineering, Huazhong University of Science and Technology, Wuhan, Hubei 430074, P.R. China [email protected]
Abstract. Compared with related studies of stochastic demand inventory routing problem (IRP), this paper takes the constraints of soft time windows into consideration. Specially, penalties for violating time window limits are included in value function; upper limit of time window is dependent on inventory level. These assumptions of IRP are more practical. Firstly, the paper represents stochastic demand inventory routing problem with soft time windows (IRPSTW) as a discrete time Markov decision process model. Then the paper proposes a solution with considering constraints of soft time windows. Finally the application of this solution is illustrated with a numerical example. Keywords: Inventory route problem, Soft time window, Stochastic demand, Markov decision process.
1 Introduction Inventory routing problem (IRP) that is a problem of how to make decisions regarding integrated optimization of inventory and transportation, is a core problem to be solved when implementing vendor managed inventory (VMI) [1]. Generally, the IRP has the following three problems to be solved [2]: ─ Frequency and time of replenishment. For a given planning horizon, when to replenish at each customer need to be decided. If planning horizon is not given, it is necessary to decide how frequently to execute replenishment service at each customer. ─ How much to replenish at customers. The supplier needs to decide how much to replenish at each customer. ─ Delivery route. The distribution vehicles need to be arranged optimally on the transportation route and service the customers in proper order. The first two problems are related to inventory control, and the last one is a vehicle routing problem (VRP). This paper studies the stochastic demand inventory routing problem with soft time windows (IRPSTW), while most related studies do not consider the constraints of time windows. For example, toward stochastic demand and short-term IRP, Federgruen et al study a kind of single product and single day planning horizon IRP. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 193–200. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
194
W. Zeng and Q. Zhao
The IRP is divided into inventory allocation problem and traveling salesman problem (TSP) based on nonlinearity integer programming model [3]. Golden et al defined demand urgency as the ratio of inventory capability to practical inventory level. First, the supplier calculates each customer’s demand urgency and chooses a customer to be delivered directly based on demand urgency. Then formulate and solve a traveling salesman problem (TSP) by iteration [4]. Jaillet et al use a rolling horizon framework to solve the IRP with a period of two weeks [5]. Minkoff et al represents the stochastic demand and long-term IRP as a Markov decision process (MDP). To overcome the calculation difficulty brought from huge state space, they put forward a heuristic algorithm based on the idea of decomposition [1], [6]. Anily et al study replenishment strategies of indefinite horizon IRP in order to minimize long-run average transportation and inventory costs. A collection of regions (sets of retailers) is specified which cover all outlets. Each time of the retailers in a given region receives a delivery, this delivery is made by a vehicle who visits all other outlets in the region as well [7]. Similarly to this paper, Li et al study the IRP with constraints of hard time windows, which means the IRP with constraints of time windows should be inviolable. In fact, it is generally unpractical to assume that the constraints of time windows are inviolable [8]. The paper studies the stochastic demand inventory routing problem with soft time windows. In other words, the penalties for violating time windows are taken into consideration. If your paper is intended for a conference, please contact your conference editor concerning acceptable word processor formats for your particular conference.
2 Problem Definition 2.1 Problem Hypothesis In order to facilitate to research stochastic demand IRP under the constraints of soft time windows, we have the following assumptions without loss of generality: • The entire logistics system delivers only a kind of production. • The demand of each customer is mutual independent and its probability distribution is deterministic without change over time in a planning period. • The supplier monitors the customer's inventory level and knows each customer's demand probability distribution. • Out of stock is allowed. Demand loss occurs in case of stockout. • The vehicle fleet size is limited; all vehicles have the same maximum capacity and the same cost characteristics. • Ignoring the time of loading and unloading goods. No demand occurres from acquiring the customers’ inventory level to making a decision on delivery quantity d i to the customer i . • Each vehicle can perform several delivery tasks per day. Each customer may receive several deliveries every day. • Do not consider inventory cost of distribution center. • Transportation costs is only depended on travel distance.
Study of Stochastic Demand IRPSTW Based on MDP
195
• Customer’s inventory cost per day is proportional to the maximum storage capacity of that day. • Storage capacity of distribution center is infinite. • Each customer's inventory state is discrete. • Consider the constraints of soft time windows. Specially, delivery vehicles should delivery goods to customer within a specified time window. If violating the constraints of time windows, should pay penalties. 2.2 Problem Description Consider products are delivered from a distribution center to N customers using a fleet of M homogeneous vehicles. The storage capacity of each customer is Ci (i = 1, 2,..., N ) . The maximum capacity of each vehicle is CV . Delivery vehicles
will deliver products to clients within a specified time window[ ETi , LTi ] , ETi is the earliest start time customer allows and LTi is the latest start time. If delivery time violates the time window limit, there will be a penalty for it. The process is modeled in discrete time t = 0, 1,..., and the discrete time periods are called days. Customers' demands on different days are independent random vectors with a joint probability distribution F . Distribution center makes decisions every day regarding which customer’s inventory to replenish, how much to deliver at each customer and how to combine customers into vehicle routes. Suppose everyday’s decisions should be completed to ensure that all vehicles available at the beginning of the next day. The revenues and costs of the logistics system include: a reward of r ( di ) when the supplier delivers quantity d i to customer i ; the travel cost ci , j on the arcs (i, j ) of the distribution network that is the shortest distance between the customer i and j; penalty pi ( si ) for inventory shortage assuming the shortage quantity at customer i is si ; each customer's inventory holding cost hi ( xi + d i ) assuming the inventory at customer i is xi at the beginning of the day and quantity d i is decided to deliver at customer i ; for customer i , assuming the penalty is mi if delivery time violates lower time limit and the penalty is li for violating upper time limit. The objective is to choose an optimal distribution policy that maximizes the expected discounted value (revenues minus costs) over an infinite time horizon.
3 MDP Model In the paper the IRP is formulated as a discrete infinite-stage Markov decision process model. The MDP model is described as follows: 1. Decision phase: Each decision phase is defined as a day, which is denoted as T = {t | 0,1, 2,...} . 2. State space: The state variable x = {x1 , x2 ,..., xN } represents the current inventory level at each customer. Thus the discrete state space is
196
W. Zeng and Q. Zhao
χ = [0,1,..., C1 ] × [0,1,..., C2 ] × ... × [0,1,..., C N ] Let X it ∈ {0,1,...Ci } denote the inventory level at customer i at time t . Let Xt ≡ ( X1t , X 2t ,...X Nt ) ∈ χ denote the state at time t . 3. Action space: For any state x , action space A( x) is the set of any feasible decisions in state x . Any feasible decision a ∈ A( x) made in state about
x contains information
─ which customer’s inventory to replenish? ─ how much to deliver at each custom? ─ how to combine customers into vehicle routes?
The set A( x) is determined by various constrains, such as the limited number M of vehicles that can be used each day, the limited quantity CV that can be delivered by each vehicle, the maximum inventory levels Ci that are allowed at each customer i and so on. For any decision a , let d i (a ) denote the quantity of products that are delivered at customer i while executing the decision a . 4. Transition probability: Let U it denote the demand of customer i at time t . Then the amount of products used by customer i at time t is denoted as min { X it + di ( At ),U it } . Thus the inventory shortage at customer i at time t is
given by Sit = max {U it − ( X it + di ( At )),0} , the inventory level of customer i at time t + 1 is given by X i ,t +1 = max { X it + di ( At ) − U it , 0} . Markov transition matrix
Q is determined by the known joint probability distribution F of customer’s
demands U it . For any state x ∈ χ , any decision a ∈ A( x) and any subset B ∈ χ , u ( x, a, B ) ≡ {U ∈ R+N : (max { x1 + d1 (a ) − U1 , 0} ,..., max { x N + d N ( a ) − U N , 0} ∈ B} ,
then Q ⎡⎣ B x, a ⎤⎦ ≡ F [u ( x, a, B ) ] . Thus P ⎡⎣ X t +1 ∈ B X t = x, At = a ⎤⎦ = Q ⎡⎣ B x, a ⎤⎦ ≡ F [u ( x, a, B ) ]
5. Rewards: Let g ( x, a ) denote the expected single stage reward if the process is in state x and decision a ∈ A( x ) is executed. For any decision a and arc (i, j ) , let kij ( a ) denote the number of times that arc (i, j ) is traversed by a vehicle while
executing the decision a . Let gi denote whether delivery task i is below the lower limit of time window. g i is set to 1 when task i is below the lower limit of time window, otherwise g i is set to 0. Let qi denote whether task i exceeds the upper limit of time window. qi is the set to 1 when task i exceeds the upper limit of time window, otherwise qi is set to 0.
Study of Stochastic Demand IRPSTW Based on MDP N N g ( x, a ) ≡ ∑ r ( d ( a )) − ∑ c k ( a ) − ∑ h ( x + d ( a )) i i ij ij i i i i =1 i =1 (i , j ) N N − ∑ E ⎡ p (max U − ( x + d ( a )), 0 ) ⎤ − ∑ ( g m + q l ) ⎥ F ⎣⎢ i i0 i i i i ii ⎦ i =1 i =1
{
197
(1)
}
6. Objective function: The objective is to maximize the expected total discounted value over an infinite horizon. Let α ∈ [ 0,1] denote the discount factor. Let ∏ denote the set of policies. Policy π ∈ Π prescribes a decision π ( x ) ∈ A( x ) based on the information contained in the current state x of the process only. And for any policy π and any state x ∈ χ , the expected value V π ( x) can be expressed as: ⎡∞ ⎤ V π ( x ) ≡ E π ⎢ ∑ α t g ( X t , π ( X t )) X 0 = x ⎥ ⎣ t =0 ⎦ π = g ( x, π ( x)) + α ∑ V ( y )Q ⎡⎣ y x, π ( x) ⎤⎦
(2)
y∈χ
Then the optimal expected value can be given by: ⎧ ⎫ V ∗ ( x) = supV π ( x) = sup ⎨ g ( x, a) + α ∑V * ( y)Q ⎣⎡ y x, a ⎦⎤ ⎬ a∈ A( x ) ⎩ π ∈∏ y∈χ ⎭
(3)
∗
If a policy π ∗ which makes V π = V ∗ exists, then it is the optimal policy.
4 Solve the Model Because the state space of MDP model is too large, the traditional solving methods for MDP will not be adopted in this paper. Otherwise, an algorithm similar to reference [1] will be proposed. Its algorithm flowchart is shown in Fig.1. Compared with reference [1] and related studies of IRP, this paper takes the constraint of time window into consideration. Given a current state x = ( xi , x j ) , the constraints of time window of customer i and customer j are determined according to the inventory level of each customer. Suppose that travel route is given, arrival time ti at customer i can be calculated by the travel distance and travel speed of vehicle. Comparing ti with the upper and lower limit of time windows, whether delivery task i violates the constraint of time window can be determined. When customer i and customer j are combined into one vehicle route, there are two cases: arrive at customer i from the distribution center, then arrive at customer j ; arrive at customer j from the distribution center, then arrive at customer i . The optimal sequence of visiting can be determined by comparing these two cases. However, when the number of customers on a vehicle route increases, the problem will be more complicated to solve. Considering that there are usually not too
198
W. Zeng and Q. Zhao
many customers on a vehicle route, it is assumed that no more than 3 customers will be combined on a vehicle route in the following example.
Solving IRP
Building approximation value function
Dividing customer set into subsets Given a policy, simulating the entire IRP to determine parameters of sub-problems
Solving the sub-problems
Determining approximate function through subproblems joint optimization
Building the approximate function
Solving optimal policy of each stage
Solving the policy of direct distribution Adding the rest customers into the direct delivery routes
Solving value function and select the optimal policy
Stopping selecting the optimal policy of each stage
Solving IRP ends
Fig. 1. Algorithm flowchart
5 Example Study Given the following example, all customers are served by a distribution center. The inventory and order costs of the supplier are not considered. The logistics system has 3 customers whose maximum inventory level is 6. There are 2 vehicles whose capacity is 6. It is assumed that each vehicle at most executes one delivery task per day. The example data is shown in Table. 1 and Table.2. In addition, CV = 6 , α = 0.9 , N = 3 , M = 2 , C12 = 80 , C13 = 60 , C23 = 40 ,
V = 90km / h , where Ci , j denotes the distance between customer i and j, for example
Study of Stochastic Demand IRPSTW Based on MDP
199
C13 represents the distance between customer 1 and 3. Let the upper limits of time windows dependent on the customers’ inventory level. In table 2 according to inventory level 0, 1, 2, 3, 4, 5 and 6, upper limits of time windows are given respectively. Table 1. Example data i
Ci
1
fi
ci
ri
pi
hi
0
70
20
45
5
0
0
140
15
45
5
0.5
0.5
120
25
20
5
0
1
2
3
4
5
6
6
0
0.5
0.5
0
0
0
2
6
0
0
0
0.5
0.5
3
6
0
0
0
0
0
Table 2. Time window limits lower limit of time window
0
1
2
3
4
5
1
8:00
8:20
8:30
8:40
20:00
20:00
2
8:10
8:20
8:25
8:40
9:00
3
8:00
8:20
8:20
8:30
8:50
i
6
Lower limit penalty
Upper limit penalty
20:00
20:00
50
120
9:30
20:00
20:00
50
260
9:20
10:00
20:00
50
210
upper limit of time window
Without loss of generality, only the computational result of state (5, 1, 1) is given here. The optimal value of state V (5,1,1) is -3120.75, the optimal policy is that the distribution vector is (1, 2, 5), which means delivery quantity at customer 1, 2 and 3 is 1, 2, 5 respectively; there are two vehicle routes, 0-1-3 and 2-0, which means customer 1 and customer 3 are combined into the same route, customer 2 is on another route. If there is no constraint of time window and other parameters of the example remain unchanged, the optimal policy is that the distribution vector is (0, 3, 5) and the optimal routes are 0-2, 0-3 in the state (5, 1, 1). Compared the example’s computational results with constraints and without constraints of time windows, the obvious influence on the solution of IPR with constraints of time windows is shown.
6 Conclusion Compared with the researches of stochastic demand IPR, this paper takes the influence of constraints of soft time windows into consideration. The penalty for violating the time window limit is included in the value function; upper limit of time
200
W. Zeng and Q. Zhao
windows is dependent on inventory level. These assumptions of the IRP are more practical. Firstly, the stochastic demand inventory routing problem with soft time windows (IRPSTW) is formulated as a discreet time Markov decision process model in this paper. Based on the algorithm similar to reference [1], a solution with considering constraint of soft time window is proposed. According to the research, the constraints of soft time windows have obvious influence on optimal policy, so it is meaningful to do the research of IRP with constraints of time windows. Acknowledgements. This research is supported by the Natural Science Foundation of China (Grant No. 70971048).
References 1. Kleywegt, A.J., Nori, V.S., Savelsbergh, M.W.P.: Dynamic Programming Approximations for a Stochastic Inventory Routing Problem. Transportation Science 38(1), 42–70 (2004) 2. Qu, W., James, H.B., Iyogun, P.: An Integrated Inventory Transportation System with Modified Periodic Policy for Multiple Products. European Journal of Operational Research 115(2), 254–269 (1999) 3. Federgruen, A., Zipkin, P.: A Combined Vehicle Routing and Inventory Allocation Problem. Operations Research 32(5), 1019–1036 (1984) 4. Golden, B., Assad, A., Dahl, R.: Analysis of a Large Scale Vehicle Routing Problem with an Inventory Component. Large Scale Systems 7(2), 181–190 (1984) 5. Jaillet, P., Bard, J.F., Huang, L., Dror, M.: Delivery Cost Approximations for Inventory Routing Problems in a Rolling Horizon Framework. Transportation Science 36(3), 292–300 (2002) 6. Minkoff, A.S.: A Markov Decision Model and Decomposition Heuristic for Dynamic Vehicle Dispatching. Operation Research 41, 77–90 (1993) 7. Anily, S., Federgruen, A.: One Warehouse Multiple Retailer Systems with Vehicle Routing Costs. Management Science 36(1), 92–114 (1990) 8. Zhao, D., Li, J.: An algorithm for Stochastic Demand Inventory Routing Problem with Time Windows (in Chinese). Science Paper Online, http://www.paper.edu.cn
An Agent-Based Approach to Joint Procurement Modeling with Virtual Organization Chih-Yao Lo1,2, Hung-Teng Chang2, Yu-Teng Chang2, and Hsiu-Yun Hu2 1
School of management, Huazhong University of Science & Technology 2 Department of Information Management, Yu Da University {jacklo, cht, cyt and 97310022}@ydu.edu.tw
Abstract. The research aims at profit maximization, and firstly, according to the purchase methods for one-to-many purchase situation, the research installs purchase agent system model of virtual organization to solve the problems of purchase possibly faced by the virtual organization on supply chains. The research uses the results of value allocation model of virtual organization researched by AHP to install the module of value distribution agent system. In the last step, the research uses the joint procurement cycle and quantity estimation model made by back-propagation neural network algorism to install the model of enterprise demand estimation agent system and the model joint procurement agent. The goal aims at construction of the optimal solutions and methods to the negotiation between buyers, the joint procurement between the raw material suppliers, the item purchase assignment of jointly the goods with various shoppers, distribution of the purchase goods and so on. Keywords: Virtual organization, Joint procurement, Back-propagation neural networks.
1 Introduction The Internet technology-framed information technology has fundamentally changed the production pattern and life style in human society in which through information technology and means, the quality of exchange and acquisition pattern of knowledge and technology has changed. The results of change not only show the convenience and rapidity but also lead to the continuous emergence of new ideas, models and methods that challenge almost all traditional ideas, models and methods. One of most outstanding results is the “virtualization” brought by the information technology. Under the guidance of virtualized conception, a series of virtualized forms including virtual community, virtual society, virtual school, virtual enterprise and virtual organization have emerged. Among them, the most important are virtual enterprise and virtual organization. Because for a society, the enterprise is the cell of economic system and economic system in turn is the basis of social existence and development. Therefore, the research on virtual enterprise and virtual organization has important practical meaning and far-reaching strategic meaning for a society, nation or region. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 201–210. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
202
C.-Y. Lo et al.
Despite the fact that virtual organization has already existed and continues to develop, the researches on the related theory of virtual organization are stagnant and become an important factor that may affect the application of virtual organization. The acceleration of theory research on virtual organization is a front burner. Under such background and importance, the research systemizes and quantifies the purchase pattern in the virtual organization of supply chain and under the support of enterprise theory, use analytic hierarchy process (AHP) to install the virtual organization of multiple agent system (MAS) and back-propagation network to estimate the optimization of purchase cycle, purchase quantity in the agent system. In the last step, we will use MAS-related technology to install the platform of multiple agent system of virtual organization with a hope that it could achieve the goal of maximizing the benefits generated by virtual organization.
2 Literature Review Followings are introduction of core theoretical basis and extended application adopted by the research: 2.1 Virtual Organization Virtual organization, or called virtual enterprises, virtual corporation, extended enterprise, virtual integration, networked enterprises, dynamic alliance, etc., some researches even include E-commerce into the scope of research on virtual organization. But no matter what the name is, they all mean temporary alliance of some independent commercial processes or business organizations under the condition of increasing enhancement of information standard [1-2]. The alliance might be a value chain in which the member business organizations contribute their core capabilities, realize the share of resources, information, knowledge, technology in the different rings of the value chain to adapt themselves to the rapid-changing demands from markets under the environment of information. Certainly, the alliance could be a “enterprise clusters” or other forms [3-4]. The existence of virtual organization is an objective fact and many corporations have successfully taken advantage of virtual organization to gain competitive edges. Among them, Nike is a typical example. From virtual organization, an enterprise may achieve the goals of operation and gain the incomparable speed of development and performance in other enterprises of same type in the same times. Many international well-known large companies like Dell emerged in recent one or two decades that they may go beyond many traditional giants in the same industry. One of the important reasons is the use of virtual organization. Without virtual organization, enterprises will continue to adopt the existed operation model whereas new enterprises will follow the path that the successful enterprises have walked. In this way, enterprises will be difficult, or even impossible to catch up to or go beyond those enterprises that have absolute competitive edges. If there is a possibility, it should be theoretical possibility that has little practical meaning [5-6].
An Agent-Based Approach to Joint Procurement Modeling with Virtual Organization
203
2.2 Auction Mechanism Auction is an economic mechanism or transaction pattern for determining product prices in markets. It means that in a transaction between a monopoly seller (monopoly buyer) and a group of buyers (sellers), because the monopoly seller (monopoly buyer) is not clear about the demands of products from buyers (sellers) that gather buyers (sellers) to bid mutually to determine the most appropriate resource allocation or selling price. Generally speaking, when the value of a product does not have a direct and objective standard to recognize, it is the best time to apply the auction system. McAfee & McMillan stressed that auction was a market system that tried to determine resource allocation and price based on the bidding of participants in a market under clear rules [7]. Blumrosen and Nisan argued that basically auction was a mechanism of allocation and pricing of rare resources [8]. In auction theory, generally the market mechanism could be divided into two categories: 1-side auction and 2-side auction. The 1-side auction could be further divided into single auction and multiple auctions according to auction single article or multiple articles. The single auction could be further divided into open auction and sealed bids auction depending on whether the bidding is open or not. Open auction could be divided into Dutch auction and English auction whereas sealed bids auction could be divided into the first price sealed bids and the second price sealed bids. In this section, we like to introduce the four major types of 1-side auction: English auction, Dutch auction, the first price sealed bids and the second price sealed bids. Open auction means that the bidding price from a bidder is known to every participant bidder. Based on the pattern of price change, open auction could be divided into Dutch auction and English auction. Sealed-bid auction means that every bidder could bid one price only and will not make the bidding prices public. Sealed-bid auction could be divided into two patterns: the first price sealed bids and the second price sealed bids [9-10]. 2.3 Back-Propagation Network (BPN) After Chang et al. deliver their article on back-propagation method, Back-Propagation Network has been the most representative, most popularly applied model in artificial neural network learning models. Its basic theory is to use the conception of gradient steepest descent method to minimize the error function. The network model is a supervised learning network that will be suitable for the application of diagnosis, classification and prediction. BPN will repeatedly carry out two steps in the learning stage as follows [11]: 1.
2.
Forward Pass: The network system will carry out feed-forward for each learning data of outside example and compute corresponding excited status value by order all hidden nodes and output nodes; then compare the actual input value and expected output value to obtain an error value. Revere Pass: Its purpose is to adjust the weight and bias of each layer. It involves the phenomena of backward pass operation of error value in network system. The error value from output echelon will back pass to each hidden node and properly adjust the linking weight between output node and hidden node and the bias between output nodes. Then the error value will further to pass to each input node. Likewise, the linking weight between hidden node and input node and the
204
C.-Y. Lo et al.
bias between hidden nodes will be adjusted. The learning process of network is the process of minimizing errors.
3 Methodology and Steps Under the support of enterprise theory, the research systemized the purchasing methods of virtual organization on supply chains. The research is planning to install complete research framework based on set research scope as Fig. 1.
Fig. 1. Research Framework
Followings are descriptions of implement steps and the process of practical installment of models: Step I: According to the purchase methods for one-to-many purchase situation to install purchase agent system model of virtual organization, the research tried to solve the problems of purchase possibly faced by the virtual organization on supply chains. In the process of auction, the participants are the party A who invites the bids – a virtual enterprise, bidder B – a raw material supplier ( B is a set, Bi is the ith raw material supplier, i = 1,2,..., n ). The indicator of bidding task is x ∈ X and we select
X as a standard interval [0,1] . The participation restrictions that should be satisfied include:
An Agent-Based Approach to Joint Procurement Modeling with Virtual Organization
205
A. The premise for designing bids
Per (Tar aution Tr ) = Cr (Tr ) − Cr (Tr − T aution ) > 0
(1)
B. The premise for participating bids:
Per (T aution Tq ) = Cq (T aution ∪ Tq ) − Cq (Tq ) > 0
(2)
Per represents profit, C represents cost function and T represents bidding task. Because every participant is rational, in the bidding process, Bi is Among them,
inclined to offer lower bidding price (in the research,
Bi is inclined to offer l higher
bidding price). Meanwhile, because the bidders could take measures like concealment, alliance, the profit of the party who invites the bids will be affected. In fact, for the bidding prices by the bidder b = (b1 ,..., bn ) , A hopes to secure the real estimation of
Ri
Bi . Thus it could reflect the real estimation of bidding task T by each bidder and it
goes without saying the meaning. Based on “Revelation Principle”, we could give a design principle of auction mechanism. Under such principle, the bidding price by each bidder is certainly the real estimation of T . Suppose that the real estimation by the set B of n bidders is
Ri = (i = 1,2,..., n) , and satisfy the standard distribution on [0,1] . We design following auction mechanism: The bidder Bi declares that its bidding price is
bi , but the actual dealing price is b b pri = i . The probability of successful bidding for the bidder Bi is qi = n i , θ ∑ bk k =1
n
∑q
i
= 1 . Assume the bidding price bi of Bi is linear homogeneous, that
i =1
is
bi = ki Ri . Then the expected profit for the bidding price bi by Bi is E=
bi n
∑b k =1
× (R −
bi
)=
θ
k
ki Ri n
∑b
× (V −
ki Ri
θ
)
(3)
k
k =1
For a rational bidder, he hopes that his expected profit is maximum, that is
max [ ki
Ri2 n
∑b k =1
Its first order condition is obtained through
×
kiθ − ki2
θ
]
k
ki derivation to above formula. That is
(4)
206
C.-Y. Lo et al.
ki =
θ
(5)
2
= 1 , bi = Ri . Therefore, we could learn that when θ = 2 , all bidders will give their real estimation of T and not care how So called “telling the truth” by the bidder is ki
others tender their bids. Based above-mentioned formulas, we designed a new auction patter in which the bidder tender their bids individually and their probability of success bidding is a strict increase function and the dealing price is half of the bidding price. So it could prove that under such auction mechanism, a rational bidder will “tell the truth”. Step II: Using the results of value allocation model of virtual organization researched by AHP to install the module of value distribution agent system. In the process of realization of profit distribution agent, based on the principle of profit distribution of supply chain of contribution and risks-sharing, it is possible to set a value that could be accepted by related parties through the consultations between cooperative enterprises that participate in the distribution. Therefore, in the research, we firstly assume that there are n cooperative enterprises participating in the profit distribution of the virtual organization. Suppose the final distributable total amount is I that could be expressed in the form of distributable part of various products, accessories and services in the market realization, that is, the distributable part in the supply chain in which tax, necessary public expenditures of supply chain have been deducted from the final realized total income of market sales. m
The total cost
Ci paid by Enterprise i is Ci = ∑ Csj Q j + C pi . Among them, m j =1
is the type of task shouldered by Enterprise unit cost of Task
i in the virtual organization. Csj is the
j , Q j is the quantity of Task j , C pi is the extra cost paid by
Enterprise i . It is the quantification of contribution by the enterprise to the virtual organization. The extra cost could be obtained from the joint procurement plan and control process of virtual organization. Then the total operation cost of virtual n
organization could be expressed as
C = ∑ Ci . Therefore, the rate of return could be i =1
computed through the formula
ε=
I −C 100% . C
Because the purpose for enterprise to participate in a virtual organization is to acquire more profit than without participation, we introduce here the social average rate of return ε for comparisons. Based on profit distribution project of contribution and risk-sharing, the first things we should consider are the cost input by the enterprise in virtual organization and corresponding social average rate of return before distributing extra return. Let the extra return be
R , then R = 1 − (1 + ε )C . Besides, due to the
difference of the type of task, technological conditions shouldered by each cooperative enterprise in the virtual organization, the risk degrees shouldered by each cooperative
An Agent-Based Approach to Joint Procurement Modeling with Virtual Organization
207
enterprise in the virtual organization are different accordingly. Suppose the risk degree of task shouldered by Enterprise i in the virtual organization is γ i , then
γi > 0
n
,
∑γ
=1
i
(6)
i =1
Likewise, the investments by each cooperative enterprise in their own infrastructure for the purpose of integral benefit of virtual organization are different. Let λi be the ratio of investment in the field by Enterprise i in the virtual organization, then
λi > 0
n
,
∑λ
i
=1
(7)
i =1
We could conduct the evaluation of risk degree
γi
that each cooperative enterprise
should shoulder by adopting some evaluation methods through supply chain risk management and sub-system control and λi could evaluate the cooperative enterprises through the performance evaluation sub-system of virtual organization and meanwhile, further to confirm based on the actual investment. The existence of uncertain factors in the operation process of virtual organization would make the rate of return positive or negative. In a certain stage of opportunity realization, the final investment used in the distributable part could be greater or smaller than the operation cost C in virtual organization in the same stage. Therefore, the research uses following three situations to consider supply chain profit distribution strategies of opportunity contribution and risk-sharing: (1) I < C , (2)
C < I < (1 + ε )C , (3) (1 + ε )C < I . The research take the third situation, the rate of return of virtual organization is significantly greater the social average rate of return, as the method. Under the situation that the rate of return of virtual organization is significantly greater the social average rate of return, in addition to recovering cost, calculating the yield of input cost, it is necessary to distribute the extra profit of virtual organization based on the return of investment risks, the size of contribution, the size of shouldered task risk and enterprise’s own infrastructure, then Vi could be expressed as
V i Ci H Ci
C Ci DR n pi ER J iJR Oi OR C ¦ C pi
(8)
i 1
The formula thus links the risk shouldered by the enterprise in the virtual organization and investment risk in the return of the enterprise and realizes risk diversification and equal sharing. Meanwhile, it links directly the contribution to the virtual organization by each cooperative enterprise and its profit distribution, individual activity of virtual organization in different opportunity realization periods and progress and operation revenue of virtual organization. Finally, authentically, equally distribute the profit
208
C.-Y. Lo et al.
according to the different contributions made to the virtual organization in each operation period. Step III: Use the joint procurement cycle and quantity estimation model made by back-propagation neural network algorism to install the model of enterprise demand estimation agent system and the model joint procurement agent. 1. Install the Prototype of Demand Estimation System of Back-Propagation Neural Network according to Back-Propagation Neural Network Algorism. 2. Establishment of Profit-Maximized Objective function. 3.
H / Ri
H / Ri
R =1
k =1
Max{Pi × max{ ∑ Qik , Di } − C (1 + hi ) ∑ Qik − Bi H / Ri
× max{( ∑ QiK −Di ),0} − (CTi + K ) × H / Ri } K =1
Pi = unit price, C = purchase cost, hi = yearly holding cost rate, Bi = short supply cost, CTi = transportation cost, Di = total quantity of real demand during H period, Qik = The Estimated Quantity of Order during the kth
Among them,
Replenishment Period, obtained from the estimation of back-propagation neural network, H = calculation cycle, Ri = best replenishment cycle (within H cycle),
H / Ri = replenishment times, k = cost of order 4.
5.
6.
7.
Compare the maximum profits obtained from each purchase cycle and select the cycle of the biggest profit as the best cycle for the enterprise to purchase a raw material and thus obtain the most optimal purchase batches; Adopt highest common factor method and the lowest common multiple method to select the highest common factor or the lowest common multiple as common purchase cycle before substitute the common purchase cycle into the prototype of estimation system to estimate the purchase quantity of the common purchase; Substitute common purchase cycle and quantity parameter obtained from Step 4 into the profit objective function established by Step 2 and compare the profit during the period that it had not joined the joint purchase; Look for case enterprises in a proper industry and introduce their sales history into the method introduced by the research plan to conduct empirical verification and confirm the method has a certain extent of benefit for join purchase.
4 Validation and Results As a whole, the main contributions of the research are following points: 1.
Establish a joint procurement model of multi-echelon supply chain with multiple agent conception and overcome the existing information uncertainty in practical environment and the problems of information transparency degree and openness through the functions of each module;
An Agent-Based Approach to Joint Procurement Modeling with Virtual Organization
2.
3.
4.
209
The research introduces an integral evaluation mechanism that combines the quantitative factors such as delivery dates, pricing, evaluation of history of delivery dates and qualitative factors such as production quality, partnership, cooperation intentions evaluation to conduct effective analyses for policy-decision through the operation of function. It will make order assignment policy-decision support model more complete that we could give an overall consideration and draw up the best order assignment policy-decision and help a virtual organization to plan the best order production assignment and schedule. Use enterprise’s data of production and sales history to train back-propagation neural network to obtain the best weighted value and bias value of back-propagation neural network used by the enterprise in estimating purchase quantity in different cycles; use recent enterprise’s data of production and sales and the best weighted value and bias value of back-propagation neural network to confirm the accuracy and reliability of the neural network and complete the design of back-propagation neural network algorism of the best purchase cycle and quantity for the enterprise; then substitute the estimated purchase quantity of the enterprise in different cycles into the objective function of maximum profit and find out best purchase cycle and quantity. The results of the simulated analyses of various parameters and related set situations are line with dynamic changes of real environment. Therefore, the simulation of the system could help the enterprise to conduct effective bidding and draw up the best order assignment policy-decision in the competitive and cooperative multi-echelon supply chains.
5 Conclusions The application of virtual organization and agent system is not limited to the occasion of commercial negotiation that the follow-up researcher could combine other fields, such as education, society to further extend the development. On the technological aspect, the results of the research could not only provide the references for industry in practices and virtual organization in drawing up strategies but also help the industry in cultivating talents and capability of the development of demand estimation theory and the drawing-up of joint procurement strategies. Furthermore, the results could be helpful to the establishment of agent database that will benefit follow-up researchers to find out strategy plan of virtual organization and agent system.
References 1. Mowshowitz, A.: Virtual Organization. Communications of the ACM 40(9), 30–37 (1997) 2. Hale, R., Whitlaw, P.: Towards the Virtual Organization. McGraw-Hill, London (1997) 3. Venkatraman, N., Henderson, J.C.: Real Strategies for Virtual Organizing. Sloan Management Review, 33–47 (1998) 4. Saabeel, W., Verduijn, T.M., Hagdorn, L., Kumar, K.: A Model of Virtual Organization: A Structure and Process Perspective. EJOV 4(1), 1–17 (2002) 5. Lau, C., Hennry, W., Wong, T.T.: Partner Selection and Information Infrastructure of a Virtual Enterprise Network. IJCIM 14(2), 186–193 (2002)
210
C.-Y. Lo et al.
6. Cloutier, L., Frayret, J.M., D’amours, S., Espinasse, B., Montereuil, B.: A Commitmentoriented Framework for Networked Manufacturing Co-ordination. IJCIM 14(6), 522–534 (2001) 7. Vytelingum, P., Cliff, D., Jennings, N.R.: Strategic Bidding in Continuous Double Auctions. Artificial Intelligence 172(14), 1700–1729 (2008) 8. Blumrosen, L., Nisan, N.: Informational Limitations of Ascending Combinatorial Auctions. Journal of Economic Theory (2009) (in press); Corrected Proof 9. Shim, S., Lee, B.: Applying Experimental Online Auctions in Marketing Research for Multi-channel Firms. Expert Systems with Applications 37(3), 2505–2516 (2009) 10. Bu, T.M., Sun, Q., Qi, A.W.: Unconditional Competitive Auctions with Copy and Budget Constraints. Theoretical Computer Science 393(1-3), 1–13 (2008) 11. Chang, P.C., Liu, C.H., Lin, J.L., Fan, C.Y., Ng, C.S.: A Neural Network with a Case Based Dynamic Window for Stock Trading Prediction. Expert Systems with Applications 36(3), 6889–6898 (2009)
Interactive Hybrid Evolutionary Computation for MEMS Design Synthesis Ying Zhang1 and Alice M. Agogino2 1
School of Electrical and Computer Engineering, Georgia Institute of Technology, GA, USA [email protected] 2 Department of Mechanical Engineering, University of California at Berkeley, CA, USA [email protected]
Abstract. An interactive hybrid evolutionary computation (IHC) process for MEMS design synthesis is described, which uses both human expertise and local performance improvement to augment the performance of an evolutionary process. The human expertise identifies good design patterns, and local optimization fine-tunes these designs so that they reach their potential at early stages of the evolutionary process. At the same time, the feedback on local optimal designs confirms and refines the human assessment. The advantages of the IHC process are demonstrated with micromachined resonator test cases. Keywords: interactive hybrid computation, MEMS synthesis, evolutionary computation, optimal design.
1 Introduction Evolutionary computation based on a multi-objective genetic algorithm (MOGA) has been used for microelectromechanical system (MEMS) design synthesis [1, 2]. Incorporating gradient-based local optimization at the end of the evolutionary process, referred to herein as the “standard hybrid GA process”, has been shown to improve the performance of the evolutionary MEMS design synthesis process [3]. This paper explores the integration of local optimization into intermediate generations of the evolutionary process in order to allow the refinement of promising designs at an early stage of the evolutionary process and guide search towards more promising design regions earlier. This method applies local optimization either to the whole population [4] or to individuals with the best performances [5] in intermediate generations. Applying local improvement to the whole population may be computationally prohibitive on large complex problems like MEMS design synthesis. On the other hand, limiting local optimization to the best individuals based on the quantitative fitness function does not guarantee local improvement of all designs with good patterns as these patterns have not yet reached their potential. Some of these designs Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 211–218. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
212
Y. Zhang and A.M. Agogino
could evolve to much better design solutions if fine-tuned or if given enough time to evolve in the evolutionary process. Interactive evolutionary computation (IEC) is a model of using the symbiosis of human expertise with evolutionary algorithms for a user-directed search, which has been used to evaluate objectives of MEMS designs that are difficult to model [6]. During the IEC process, human experts assess all individuals subjectively and assign fitness values based on their judgment of the potential for performance or a recognition of innovative design patterns. The reliance on human judgment to guide the evolutionary process has both potential strengths and weaknesses. On one hand, human expertise can be effectively exploited for guidance of a naive evolutionary process. On the other hand, including a human in the evolutionary loop introduces the need for rapid convergence to prevent human fatigue, and human assessment tends to have a component of subjectivity and non-linearity of focus over time [7]. To solve the human fatigue problem, Kamalian, et al., suggested a method with the human in a supervisory role, providing evaluation only every nth-generation during the IEC process [6]. To reduce the influence of the subjectivity and non-consistency of human assessment on the evolutionary process, we present an interactive hybrid computation (IHC) algorithm that uses local optimization to fine-tune all selected designs by a designer so that these designs reach their potential at early stages of the synthesis process. The performances of these locally-optimized designs can also be beneficial in confirming and refining the human’s subjective assessment of these designs over time. The advantages of the IHC process are demonstrated on the synthesis of surface-micromachined resonators.
2 Interactive Hybrid Computation (IHC) Process IHC exploits human expertise in identifying potentially good design patterns, using the results of local optimization to refine fitness values of strong designs and reduce the influence of subjectivity and non-consistency of the human evaluation. During the IHC process, a human designer evaluates all the designs or a subset of them, such as those in the Pareto optimal set for multi-objective fitness functions, before applying local improvements. To reduce human fatigue, human input is only applied at certain predefined generations during the hybrid evolutionary processes. In our MEMS resonator test cases, IHC provides the designer with multiple design concepts within a graphical user interface (Fig. 1.) at predefined generations to evaluate all the designs or a subset of them in the current generation. If the designer identifies a promising design pattern, he/she can click the ‘local_Opt’ button, to enable the MEMS synthesis process to apply gradient-based local optimization to the design. After gradient-based local optimization has been performed, the geometrical parameters and performances of the optimized designs are fed back into the genotype and the phenotype domains of the GA process to replace the original designs.
Interactive Hybrid Evolutionary Computation for MEMS Design Synthesis
213
Fig. 1. Graphical User Interface during IHC Processes
3 Evaluation of the IHC Process 3.1 Experimental Design Two experiments were conducted to evaluate the IHC process, comparing the proposed IHC process with: (1) the standard hybrid GA process; and (2) a random hybrid GA process. The random hybrid GA process randomly selects similar number of designs from the Pareto optimal set of predefined intermediate generations and applies local optimization. The second experiment is to test if the benefits of IHC are from human interaction or intermediate local optimization. A series of resonator design cases are used as test cases for the experiment. The resonator design is similar to the design used in [3] and [8] for purposes of comparison. These test cases are symmetric designs without angle constraints, which provide a relatively large design search space. The multiple design objectives considered are to: (1) minimize the error from the target resonant frequency, f_target (Designs within an acceptable error range, 5%, are considered feasible designs.); (2) minimize the design area; and (3) satisfy the stiffness ratio requirement (Kx/Ky ≥ k_target). Six test cases were designed for the comparison between the IHC and the standard hybrid GA processes. Two cases were the same as the test cases used in [3] and [8], and four other cases were developed by randomly generating design objective goals with natural resonant frequency values between 1,000Hz and 10,000Hz, and stiffness ratios between 1 and 10. The objective values of these six test cases are listed in Table 1.
214
Y. Zhang and A.M. Agogino Table 1. Design Objective Goals for the Test Cases (in addition to area minimization)
Test case 1 Test case 2 Test case 3 Test case 4 Test case 5 Test case 6
Natural resonant frequency (Hz) 10,000 10,000 2,587 5,109 8,393 9,551
Stiffness ratio requirement 1 10 5 1 5 3
All test cases have an I-shaped center mass with a fixed geometry, which has one 100μm x 100μm rectangular mass plate attached with four 30μm x 50μm beams. The comb drives connected to the center mass also have fixed geometries. Each comb drive has 11 fingers, 50μm long by 4μm wide with a 3μm gap, plus a 4μm wide spine. The parameters of springs connected to the center mass are free parameters. The parameter constraints for the springs are the same as the constraints discussed in [3] and [8] except the upper bound of the beam length is extended to 300μm for the test cases with target natural resonant frequencies higher than 5,500Hz and 800μm for the test cases with target natural resonant frequencies lower than that. All hybrid computation design synthesis processes are applied to the six test cases three times. The maximum number of generation for each process is 50 and each generation has 100 populations. The designs that are in the overall Pareto set, have a frequency deviation within 5% of the target frequency and satisfy the stiffness ratio requirement are tallied after each design synthesis process. In the first experiment, designers interact every 10th generation to select designs to apply the local improvement procedure during the IHC process (IHC_10). The objective function of the gradient-based local optimization used during the IHC process is formulated as the sum of the normalized design area and frequency deviation, plus a large penalty if the design violates the stiffness ratio requirement. In the second experiment, designers interact every 5th generation of the first 25 generations to select designs for the local performance improvement during the IHC process (IHC_5), resulting in a similar human fatigue level as that during the IHC_10 process. A total of 263 designs were selected by humans during the IHC_5 processes for all six test cases with three runs for each case, which means, on average, approximately 3 designs were selected at each computer-human interaction. To be comparable and give the random GA process a slight advantage in terms of the number of locally-optimized designs, the random hybrid GA process randomly selected 5 designs from the Pareto set every 5th generation during the first 25 generations to apply the gradient-based local optimization. At the end of all of the all evolutionary synthesis processes, another gradient-based local optimization is applied to all the tallied feasible designs generated. Since all feasible designs satisfy the following requirements: the frequency deviation is within 5% of the target frequency, and the stiffness ratio (Kx/Ky) is not less than the requirement, the local optimization only needs to minimize the design area while keeping all other requirements as inequality constraints.
Interactive Hybrid Evolutionary Computation for MEMS Design Synthesis
215
3.2 Results and Discussion The statistical analysis of the area of the feasible designs on the overall Pareto sets of the first experiment is shown in Table 2. Table 2. Statistical Analysis of the Area of Pareto Optimal and Feasible Designs Resulting from the MEMS Synthesis Processes Synthesis Minimum Average Area Area (E-07 m2) (E-07 m2) Process Standard 1.7056 4.1585 Case 1 Hybrid GA 1.6665 2.0077 IHC_10 Standard 1.8426 2.1778 Case 2 Hybrid GA 1.7997 2.0045 IHC_10 Standard 4.9434 10.1455 Case 3 Hybrid GA 3.8010 5.5189 IHC_10 Standard 4.0256 17.0716 Case 4 Hybrid GA 2.7589 3.1272 IHC_10 Standard 1.9569 2.1818 Case 5 Hybrid GA 1.8093 2.0918 IHC_10 Standard 1.7444 2.1292 Case 6 Hybrid GA 1.5723 2.0302 IHC_10 Note: Statistical significant improvements shown as shaded areas.
ANOVA pvalue 0.0234
0.2539
0.0310
0.0283
0.4902
0.6321
The results show that the IHC process is significantly better than the standard hybrid GA process using the Wilcoxon Matched-Pairs Signed-Ranks [9] (p=0.025) for both the minimum design area and the average design area. The analysis of variance (ANOVA) between the IHC final feasible populations and the standard hybrid GA final feasible populations shows that the IHC process is significantly better than the standard hybrid GA process for cases 1, 3 and 4 with p ≤ 0.031. The improvement of the IHC process is not statistically significant for cases 2, 5 and 6 using ANOVA over the entire final Pareto optimal population, although the results do show improvement for both the minimum and average areas. The reason for the lack of statistically significant improvement in the Pareto optimal populations in cases 2,5 and 6 may be due to the fact that these cases have both a high target frequency goal and a high stiffness ratio requirement, reducing the feasible search space. The best designs generated from the IHC process for different test cases have either springs turning inward or high beam densities in the rectangular bounding box around the device (not including two large anchor pads connected to comb drives). With fixed center masses, both the high target frequency and the high stiffness ratio requirement evolves to consistently push the springs outward, reducing the probability of
216
Y. Zhang and A.M. Agogino
generating novel new patterns and thus minimizing the value of the human interaction in identifying promising new patterns. Table 3. Statistical Analysis of the Area of Pareto Optimal and Feasible Designs Resulting from the Random Hybrid GA process Minimum Average Sign1* Sign2 ** area area (E-07m2) (E-07m2) Random_5 1.6854 2.1700 1 + + - IHC_5 1.6573 1.7888 Random_5 1.8568 2.1353 2 - + - IHC_5 1.6718 1.8273 Random_5 5.8068 11.1288 3 - - IHC_5 3.3859 4.9742 Random_5 2.6139 11.1723 4 + + - IHC_5 2.3948 2.9165 Random_5 1.8545 2.0142 5 + + + + IHC_5 1.8758 2.2097 Random_5 1.7534 1.9053 6 - + - + IHC_5 1.5420 1.9866 * Comparison between the random and standard hybrid GA processes. ** Comparison between the random hybrid GA process and the IHC_5 process. Note: Statistical significance shown as shaded areas. Standard hybrid GA results for minimum and average areas listed in Table 2. Case
Synthesis Process
The statistical analysis of the results of the second experiment is shown in Table 3. In the ‘sign1’ and ‘sign2’ columns, the first symbol ‘+/-’ represents the comparison between the minimum areas, and the second symbol represents the comparison between the average areas. The symbol ‘+’ is used if the area resulting from the random hybrid GA process is smaller than that resulting from the other process, sign ‘-’ is used otherwise. Based on the results of the best generated designs, the IHC_5 process is statistically better than the random hybrid GA process using the Wilcoxon Matched-Pairs Signed-Ranks (p<0.05), although the random hybrid GA process could occasionally outperform the IHC process (case 5). ANOVA only shows significant improvement of the Pareto optimal population for the IHC_5 process for case 1 (p=0.0245) and case 3 (p=0.0086). Comparisons are also performed between the random hybrid GA process and the standard hybrid GA processes. The random hybrid GA process shows statistically significant improvement over the standard hybrid GA processes using the Wilcoxon Matched-Pairs Signed-Ranks (p<0.05) in terms of the average area. Using ANOVA, the random hybrid GA process only shows significant improvement over the standard hybrid GA process for case 1 (p=0.0287). The results show that, in general, there appears to be a benefit in applying local optimization at intermediate designs, even if performed on a random subset of the Pareto set. Although human selection performed better than random selection as determined by the Wilcoxon Matched-Pairs Signed-Ranks test in terms of the minimum design area, the ANOVA test only showed statistically improvements over
Interactive Hybrid Evolutionary Computation for MEMS Design Synthesis
217
all feasible designs for two cases – cases 1 and 3. This suggests that some automation procedure that could lessen the human load during intervention might have merit.
4 Conclusion and Future Research An interactive hybrid computation algorithm is developed, which integrates a computer-human interaction and a local improvement procedure into the standard hybrid GA process. The performance of the IHC process is evaluated based on a series of MEMS resonator test cases. Compared with the standard hybrid GA process, the improvement of the IHC process is statistically significant in terms of the best and average final designs using the Wilcoxon Matched-Pairs Signed-Ranks test. The research results also demonstrate that IHC process strongly impacts the stochastic search process when the feasible search space is much larger and includes potentially good design patterns. Compared with a random hybrid GA process, the improvement of the IHC process is statistically significant in terms of the best generated designs, but not the averages. This suggests that some automated selection procedure has the potential to reduce the human load and still have relatively good performance. During the IHC process, about half of the human-selected designs were not in the Pareto optimal sets, indicating that Pareto optimal sets may not be the ideal subset for human evaluation. This suggests an interesting new research direction in developing an intelligent pattern-recognition algorithm for simulating human expertise that could be used in the IHC process. With the successful development of this technique, computers would have the potential to take on more of the pattern recognition burden, reducing the number of times the human designer would be needed for evaluation in the IHC process and allowing a more efficient portion of the design space to be locally optimized.
Acknowledgements This research was conducted in part through NSF grant CCR-DES/CC-0306557. The authors wish to acknowledge the contributions of Raffi Kamalian in working with us on the interactive user interface design.
References 1. Zhou, N., Agogino, A.M., Pister, K.S.J.: Automated Design Synthesis for Micro-ElectroMechanical Systems (MEMS). In: Proceedings of the ASME Design Engineering Technical Conference, vol. 2, pp. 267–273 (2002) 2. Zhang, Y., Kamalian, R., Agogino, A.M., Séquin, C.H.: Design Synthesis of Microelectromechanical Systems Using Genetic Algorithms with Component-Based Genotype Representation. In: Proceedings of GECCO (Genetic and Evolutionary Computation Conference), vol. 1, pp. 731–738 (2006)
218
Y. Zhang and A.M. Agogino
3. Zhang, Y., Kamalian, R., Agogino, A.M., Séquin, C.H.: Hierarchical MEMS Synthesis and Optimization. In: Smart Structures and Materials 2005: Smart Electronics, MEMS, BioMEMS, and Nanotechnology. Proceedings of SPIE, vol. 5763, CD ROM, Paper # 5763_12, pp. 96–106 (2005) 4. Oh, I.-S., Lee, J.-S., Moon, B.-R.: Hybrid Genetic Algorithms for Feature Selection. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(11), 1424–1437 (2004) 5. Zhou, Y., Li, J., Hao, L.: Shape Inversion of Metallic Cavities Using Hybrid Genetic Algorithm Combined with Tabu List. Electronics Letters 39(3), 280–281 (2003) 6. Kamalian, R., Zhang, Y., Takagi, H., Agogino, A.M.: Reduced Human Fatigue in Interactive Evolutionary Computation For Micromachine Design. In: Proceedings of ICML2005, the Fourth International Conference on Machine Learning and Cybernetics, International Machine Learning Society, pp. 5666–5671 (2005) 7. Caleb-Solly, P., Smith, J.: Interactive Evolutionary Strategy Based Discovery of Image Segmentation Parameters. In: Proceedings of the 6th International Conference on Adaptive Computing in Design and Manufacture, UK (April 2004) 8. Kamalian, R., Agogino, A.M., Takagi, H.: The Role of Constraints and Human Interaction in Evolving MEMS Designs: Microresonator Case Study. In: Proceedings of DETC/DAC, 2004 Design Engineering Technical Conference, Design Automation Track, Paper # DETC2004-57462, CD ROM (2004) ISBN # I710CD 9. Siegel, S., Castellan, N.J.: Nonparametric Statistics for the Behavioral Sciences, 2nd edn. McGraw-Hill, London (1988)
Genetic Algorithms for Traffic Grooming in Optical Tree Networks Shu-tong Xie1 and Li-fang Pan2 1
School of Computer Engineering, Jimei University, Xiamen, 361021, P.R. China 2 School of Science, Jimei University, Xiamen, 361021, P.R. China
Abstract. To deal with the changing traffic in SONET/WDM tree networks, reconfigurable grooming of non-deterministic dynamic traffic (REGNT) is introduced. The full-fit case of REGNT is studied according to different network demand. Then two heuristic algorithms are developed to tackle the case by combining genetic algorithms with traffic-splitting heuristics. To evaluate the algorithms, the theoretical bounds are also derived. Computer simulations show that the proposed algorithms can achieve good results in reducing the number of additional ADMs and wavelengths to fit new traffic. Keywords: Traffic grooming, Genetic algorithm, Reconfiguration, WDM Tree networks.
1 Introduction Synchronous optical network (SONET) networks incorporated with wavelength division multiplexing (WDM) are widely used in today's optical network infrastructures. Due to large gap between the capacity of a wavelength (light path) and a traffic request, it is necessary to assign reasonably several traffic requests to a wavelength to make the best of capacity of wavelengths. The problem of how to combine different low-speed traffic streams onto high-speed streams to minimize the number of SONET add/drop multiplexers (ADMs) is referred to as traffic grooming (TG) problem and has been proved NP-complete [1]. Recently, to solve the TG problem in tree networks, the authors [2] first decomposed a tree topology into stars, then used greedy algorithm to search the optimal results. The authors [3] proposed a genetic algorithm tackling the dynamic traffic directly the TG problem in tree and star networks. On the other hand, more and more researchers paid attention to grooming of dynamic traffic [4-9]. In [4, 5], the authors researched the dynamic traffic which takes place after a ring network establish. So we attempt to study the new kind of dynamic traffic grooming problem in tree networks, in which the traffic would change after the configuration of the network has been finalized. Based on the current configuration (assignments of ADMs and Wavelengths), we have to reconfigure the virtual or even physical topology of the network to satisfy the new traffic pattern. By doing so, it is possible that only few traffic requests are blocked in the establishment of new traffic patterns. We has study the TG problem only Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 219–226. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
220
S.-t. Xie and L.-f. Pan
reconfiguring the virtual topology of the tree networks in [9]. So in this paper the TG problem reconfiguring the physical topology of the tree networks are studied. The remainder of this paper is organized as follows. We define the general properties of REGNT problem and tree networks, and address the theoretical bound in section 2. In section 3, two heuristic algorithms are described in detail, which are used to deal with the full-fit cases of REGNT. Section 4 gives the computer simulations along with necessary discussions. In the last section, we summarize this paper.
2 Problem Definition The solution of dynamic TG problem is composed of the following two phrases [9]: 1) The old traffic matrices are known (first traffic pattern). Assumed that there are two traffic matrices describing the initial dynamic traffic in the network. The algorithm proposed in [3] can be used to realize the strictly non-blocking grooming in tree networks. By using it, the current configuration of the tree network can be obtained. Then the assignment of ADMs and wavelengths in the network is recorded. 2) A new traffic pattern comes (second traffic pattern). The objective of the full-fit case of REGNT is to minimize the number of additional ADM as well as wavelength to fit all new traffic to the network. We investigated TG problem in tree networks with bi-directional fiber with following properties, and used some notations following [9]: 1) There are n nodes numbered 0, 1, 2, …, n-1 in the tree networks. 2) The physical topology of the tree network is a binary tree. 3) Internal nodes and leaf nodes in the tree networks. The internal nodes need to be installed an OXC [6]. One wavelength add/drop multiplexer (WADM) was installed in each leaf node. And then each node should be equipped with an ADM for each wavelength if the traffic carried by the wavelength needs to be added/dropped at that node. 4) The shortest route between any two nodes is used to carry traffic. 5) The number of low-speed traffic streams a wavelength can carry is referred to as the traffic granularity g. 6) The first traffic pattern has two traffic matrices which are represented by matrix R1[i, j] and R2[i, j]. And the new (second) traffic pattern is only a traffic matrix which are represented by R3[i, j]. Each element rij of these matrices represents the traffic request originating form node i and terminating at node j. 2.1 Upper Bound Theoretical bounds are necessary to evaluate our results, so we derived the upper bound for it. The objective of the best-fit case is to place as much traffic as possible to the network by using the old configuration of the network [9]. The old (first) traffic pattern has two traffic matrices which are represented by R1[i, j] and R2[i, j], and the new traffic pattern (new traffic matrix) is R3[i, j]. We define a traffic matrix Rmax[i, j] as rmaxij = Max (r1ij, r2ij), then let D1[i, j] = R3[i, j] − Rmax[i, j]. If some elements (traffic requests) are negative in D1[i, j] we set them as zero and finally generate D2[i, j]. And the sum of all
Genetic Algorithms for Traffic Grooming in Optical Tree Networks
221
elements in D2[i, j] is our upper bound in best-fit because the part of new traffic matrix, R3[i, j] − D2[i, j], are sure to be groomed in the original networks even in worst situation but another part of it, D2[i, j], may be blocked [9]. Based on these definitions, we can derive the upper bounds for full-fit case. Take D2[i, j] as a static traffic pattern, and use the similar method applied to deduce the upper bounds in ring or tree [3, 8]. The calculated process is that: we checked all the elements in D2[i, j]. If one request of the traffic requests pair (rij and rji) is not equal zero, we use a dedicated wavelength to carry them or it, and also add two ADMs on this wavelength; On the other hand, if both requests rij and rji are equal to zero, they do not need any wavelength and ADMs. At last the amount of wavelengths required and the sum of ADMs on all wavelengths on all nodes are the upper bounds on number of wavelengths and ADMs in full-fit respectively. The calculated method of number of ADMs can be expressed by:
⎧1 if rij ≠ 0 or rji ≠ 0 ADM ij = ADM ji = ⎨ ⎩0 if rij = 0 and rji = 0
(1)
Where ADMij represents an ADM on node i used to add/drop the traffic request pairs rij and rji. And the upper bound on number of ADMs required can be expressed as n −1 n −1
1 M max = ∑∑ ADM ij
(2)
i = 0 j ≠i
So the upper bound on number of used wavelengths can be expressed as 1 Wmax =
1 M max
2
(3)
However, there is another method to derive the upper bounds for number of ADMs and wavelengths in full-fit case, respectively [3, 8]. Firstly, if we can drop the minimum number of wavelengths at each node the upper bound on number of ADMs can also be obtained by 2 M max = n × Wmin
(4)
Where Wmin is the lower bound on number of wavelengths for D2[i, j]. And the upper bound on the number of wavelengths in full-fit is express as [3]
⎧⎪(n 2 -1)/4 n is odd 2 Wmax =⎨ 2 n is even ⎪⎩n / 4
(5)
Two methods to derive the upper bounds of number of ADMs and wavelengths are subjected to either n or special traffic matrix, or g, or the combination of several factors before. So the upper bound of ADMs and wavelengths is the small one of them, given by
222
S.-t. Xie and L.-f. Pan 1 2 M max = Min( M max , M max )
(6)
1 2 Wmax = Min(Wmax , Wmax )
(7)
3 Algorithms Due to its powerful search ability in solving combinatorial optimization problems, we designed two GAs for searching the optimal solutions of the full-fit case in tree networks. GA with and without traffic splitting heuristic are proposed to solve the full-fit case in REGNT problem, which is denoted by GA-splitting and GA-non-splitting, respectively. The effectiveness of traffic splitting method in traffic grooming has been investigated in detail in [4, 7]. So the traffic splitting (traffic-dividing [7]) method will be embedded into the genetic algorithms to help further improve the grooming performance for REGNT problem. 3.1 Framework of GAs Fig.1 shows the flow chart of the GA, in which the criterion in this paper is the maximum number of iteration times. We choose the (μ+λ)-strategy as reproducing method [8]. And an order-based representation for chromosome [8] is used to code the chromosome, which, in nature, is a list of all the traffic requests. Each gene in the chromosome represents the corresponding traffic request in the traffic matrix. The position each gene lies in gives the decoding order of the traffic requests. The realization of crossover, mutation, chromosome decoding and selection are described in the following subsections.
Fig. 1. Flow chart of genetic algorithms
3.2 Crossover and Mutation Operators The GAs proposed in this paper use the same genetic operators. Firstly, we use the crossover operator proposed by Xu et al in [8], i.e. the order-mapped crossover (OMX), which is efficient to executes and do not produce illegal offspring. Secondly, we
Genetic Algorithms for Traffic Grooming in Optical Tree Networks
223
employ the inversion mutation operator to mutate the individuals. This operator randomly selects two points in a parent, and inverses the genes between two points to produce an offspring. 3.3 Approaches of Chromosome Decoding and Fitness Assignment
Ⅱ
is the chromosome decoding method of GA-splitting which is applied to Algorithm groom the new traffic pattern. Firstly Algorithm (Chromosome decoding for GA-splitting for best-fit case [9]) is embedded into the Algorithm to groom the new traffic by using the old configuration of the network. Then Algorithm adds some necessary ADMs and wavelengths to groom the remaining traffic.
Ⅰ
Ⅱ
Ⅱ
Ⅱ
Algorithm (Chromosome decoding for GA-splitting in full-fit case)
Ⅰ
was embedded to groom traffic requests; Here Algorithm Begin with a new wavelength; While there is still a gene bg un-assigned if(current wavelength can accommodate traffic[bg] according to gene bg ) { pack traffic[bg] into current wavelength; add corresponding ADMs; exchange gene bg with bun-assigned; un-assigned++; } else{ for( g2=un-assigned; g2<=last remaining gene; g2++) { if((the current wavelength can accommodate the traffic[bg2]) && (corresponding ADMs on the source/termination nodes of traffic[bg2] )) { assign traffic[bg2] to the current wavelength; exchange gene bun-assigned with bg2; un-assigned++; } } Add an ADM to a node on current wavelength to groom the remained traffic and exchange the according genes’ position and also increase the un-assigned; // traffic splitting method for( g2= last remaining gene; g2>un-assigned; g2-- ) { if((the current wavelength can accommodate the part of traffic[bg2]) && (there are ADMs on the source/termination nodes of traffic[bg2] )) assign a part of traffic[bg2] to the wavelength; { } leave the other part in traffic[bg2]; } begin with a new wavelength; } EndWhile
224
S.-t. Xie and L.-f. Pan
Ⅱ
Algorithm is a first-fit greedy algorithm. It always tries to assign the first-encountered traffic request in the sequence into the current wavelength. When the current wavelength cannot accommodate this traffic request, the algorithm will try to assign the next traffic request in the sequence to the current wavelength without adding any ADMs. After all the traffic requests are checked and not any more traffic can be packed into the current wavelength without adding ADMs, in order to assign more traffic to current wavelength we continue to employ another traffic assignment method in which only adding a new ADM to one of nodes on the current wavelength, and sharing another ADM(s) already in other node(s) to carry traffic requests. The traffic-splitting method is used to groom more traffic when the current wavelength can not accommodate any traffic request as a whole any more. After assigning the traffic with splitting method and the current wavelength can no longer accommodate any more traffic, the algorithm stops packing the current wavelength and begins with a new wavelength. The process goes on until all the traffic requests are groomed. The GA-non-splitting for full-fit can develop easily from GA-splitting by ignoring the traffic splitting process in Algorithm . We can assign the fitness value to each individual after the above decoding process. The objective in full-fit case is to minimize the number of additional ADMs while as few additional wavelengths as possible being used to fit all the new traffic. So the fitness value is determined by the number of additional ADMs and wavelengths required for each individual after decoding. The individual that requires fewer additional ADMs gets higher fitness value. If some individuals need to add the same number of ADMs, we give those requiring fewer wavelengths a higher fitness value; if the number of required wavelengths of the two is still the same again, we assign a same fitness value to them. At last, the μ individuals with the highest fitness value are selected as parents of the next generation from all the μ + λ candidates in the current generation.
Ⅱ
4 Simulation Results and Analysis The proposed two algorithms are tested in tree networks with 8-15 nodes. All traffic matrices are generated at random. The traffic granularity is g= 16 or 24 in the tests. In the simulations, the population and offspring sizes are both set to 200 and the algorithm stops after 500 generations. Each problem is tested 10 times with different random initial populations and the average results are given in this paper. For each individual, the crossover and the mutation rates are set to 0.6 and 0.4, respectively. As shown in Fig.2, both GA-splitting and GA-non-splitting get good solutions. And the number of additional ADMs obtained by GA-splitting is still fewer than that got by GA-non-splitting both for g=16 and for g=24. Specially, in several cases the number of additional ADM is just less 4 and wavelengths is less 3, which is the networks provider desired for. Further more, both algorithms can yield small number of additional wavelengths which is less 4 in most cases.
Genetic Algorithms for Traffic Grooming in Optical Tree Networks
225
80 60
Number of additional ADMs
60
Number of additional wavelengths
Upper bound non-splitting splitting
70
50 40 30 20 10
Upper bound non-splitting splitting
50
40
30
20
10
0
0 8
10
12
14
8
16
10
12
(a)
16
(b)
60
60
Upper bound non-splitting splitting
50
Number of additional wavelengths
Number of additional ADMs
14
Number of nodes
Number of nodes
40
30
20
10
Upper bound non-splitting splitting
50
40
30
20
10
0
0 8
10
12
14
8
16
10
12
14
16
Number of nodes (d)
Number of nodes (c)
Fig. 2. Results of non-splitting method and splitting method. (a) The number of additional ADMs vs. nodes for g=16; (b) The number of corresponding additional wavelengths vs. nodes for g=16; (c) The number of additional ADMs vs. nodes for g=24; (d) The number of corresponding additional wavelengths vs. nodes for g=24. 90
60
Number of additional wavelengths
Number of additional ADMs
80 70
Upper bound non-splitting splitting
60 50 40 30 20
50
Upper bound non-splitting splitting
40
30
20
10
10 0
2
4
6
Traffic pattern (a)
8
10
0 0
2
4
6
8
10
traffic pattern (b)
Fig. 3. Number of additional ADMs (a) and corresponding wavelengths (b) vs. different pairs of traffic pattern for n=15 and g=16
We also generate ten different traffic pattern pairs for both algorithms to see the statistical performance of them. From Fig.3, we can clearly see that the number of additional ADMs is small. For the results of GA-non-splitting, the number of additional
226
S.-t. Xie and L.-f. Pan
ADMs is at most 20 far smaller than the upper bound in all cases, and the number of additional wavelengths is less 5. On the other hand, the GA-splitting only uses less half of number of additional ADMs of GA-non-splitting to groom all the new traffic. Thus the both algorithms are efficient in solving the full-fit case in REGNT problem.
5 Conclusion We solve the full-fit case of REGNT problem in WDM tree networks by developing two genetic algorithms for it. And the traffic splitting heuristic is also introduced to further improve performance of our algorithm. What’s more, to evaluate the performance of the algorithms the theoretical bounds on ADMs and wavelengths are derived. Computer simulations show that all genetic algorithms can get good results. Further more, GA-splitting incorporated with the traffic splitting heuristic can further save number of additional ADMs and wavelengths as well but consuming a bit more running time, so as to get optimal grooming results. All the results show our algorithms are effective in solving REGNT problem. Acknowledgements. This work is supported by the Important Sci-Tech Special Projects of Fujian Province under Grant No. 200812230002.
References 1. Chiu, A.L., Modiano, E.H.: Traffic grooming algorithms for reducing electronic multiplexing costs in WDM ring networks. J. Lightwave Technol. 18, 2–12 (2000) 2. Huang, S., Dutta, R., Rouskas, G.N.: Traffic grooming in path, star, and tree networks: Complexity, bounds, and algorithms. IEEE. J. Sel. Area. Comm. 24, 66–81 (2006) 3. Liu, K.-H., Xu, Y., Huang, D.-S., Cheng, M.: Grooming of dynamic traffic in WDM star and tree networks using a genetic algorithm. Photonic. Netw. Commun. 15, 111–121 (2008) 4. Zhang, S., Ramamurthy, B.: Dynamic Traffic Grooming Algorithms for Reconfigurable SONET Over WDM Networks. IEEE. J. Sel. Area. Comm. 21, 1165–1172 (2003) 5. Xie, S.-T., Xu, Y.: Reconfigurable grooming of dynamic traffic in SONET/WDM ring networks. J. High Speed Netw. 16, 261–273 (2007) 6. Zhu, K., Mukherjee, B.: Traffic grooming in an optical WDM mesh network. IEEE. J. Sel. Area. Comm. 20, 122–133 (2002) 7. Liu, K.-H., Xu, Y.: A new approach to improving the grooming performance with dynamic traffic in SONET rings. Comput. Netw. 46, 181–195 (2004) 8. Xu, Y., Xu, C.S., Wu, B.X.: Strictly nonblocking grooming of dynamic traffic in unidirectional SONET/WDM rings using genetic algorithms. Comput. Netw. 41, 227–245 (2003) 9. Xie, S.T., Guo, Y.b., Xu, Y., Liu, K.h.: Grooming of Dynamic Traffic in WDM Tree Networks Using Genetic Algorithms. In: Yu, W., He, H., Zhang, N. (eds.) ISNN 2009, PART 2. LNCS, vol. 5552, pp. 424–431. Springer, Heidelberg (2009)
Multi-sensor Multi-target Tracking with OOSM Cheng Cheng and Jinfeng Wang Zhengzhou Institute of Aeronautical Industry Management No.2, Daxuezhong Road, Zhengzhou 450015, China
Abstract. In multi-sensor multi-target tracking systems, the arrivals of “out-ofsequence” measurement (OOSM) can occur even in the absence of communication time delays. A optimal OOSM update algorithm are derived to solve one-lag as well as multi-lag OOSM update problems. In order to extend the OOSM update algorithms to multi-sensor multi-target tracking in clutter, the probabilistic data association (PDA) have been incorporated into the OOSM update algorithms with economic storage and efficient computation based on the nonsingularity assumption of some special matrices. The simulation results shows that PDA with the OOSM update algorithms have compatible RMS errors to the in-sequence PDA filter. Keywords: Target tracking, Multi-sensors, Simulation.
1 Introduction Estimation fusion has been investigated for more than two decades. Target Tracking and Motion Analysis demand practical fusion systems for both military and nonmilitary applications. In multi-sensor target tracking systems, measurements from the same target can arrive out of sequence, called the out-of-sequence measurements (OOSM). Such “out-of-sequence” measurement arrivals can occur even in the absence of communication time delays [1]. The resulting problem — how to update the current state estimate with an “older” measurement — is a nonstandard estimation problem. Although OOSM update is a small problem from theoretical point of view, solving this problem has a significant impact in target tracking and many other engineering applications.
2 Out-of-Sequence Measurement Update Suppose there is no constraint. Then the OOSM update problem can be simply solved by rerun the Kalman filter with all data from the OOSM occurrence time to its arrival time[2]. But we do have limited storage, how do we perform an efficient OOSM update with limited storage? Many OOSM update algorithms have been proposed. They can be classified into two classes in terms of the delayed time of this OOSM: single-lag and multi-lag. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 227–233. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
228
C. Cheng and J. Wang
Suppose the delayed measurement is
zd , where the sampling time is
td (tk −1 < td < tk ) . When l = 1, zd is a one-step delayed (single-lag) OOSM; when l > 1, zd is a multi-step delayed (multi-lag) OOSM. The OOSM update for multi-step delayed problem is much harder than the onestep delay case. The main reason is that we have limited information storage. For one step globally optimal update, the information of z K can be characterized through
xˆ k|k and xˆ k|k-1 , so we can have an OOSM update algorithm having equivalent performance with in-sequence update using the information
xˆ k|k-1 , zd , z K . By
generalizing to the multi-step delayed OOSM update problem, we can not based only ˆ k|k and xˆ k|k-1 to characterize all observations from on current information such as x OOSM occurrence time to its arrival time.
3 Optimal Update with OOSM In a distributed multiple-sensor tracking system, observations produced by the sensors typically arrive at a fusion center with a random time delay due to communication delay. The state equations are usually obtained in continuous time and then discretized. The sensor may provide a “time stamp” with each measurement. In centralized multi-sensor tracking systems, all these measurements are sent to the fusion center for processing. There are usually different time delays in transmitting data to the fusion center. This can lead to situations where measurements from the same target arrive out of sequence. In this case, a measurement produced at time t k is received at the fusion center and is used to produce an updated track state estimate and covariance matrix for that time t k . Then, a delayed observation z d produced at a prior time td (tk −1 ≤ td < tk −l +1 , l = 1, 2,...) is received at the fusion center. This could occur if the observation produced at time t d was subject to a longer transmission delay than the delay associated with the observation produced at the later time t k .
Fig. 1. The OOSM
zd
arrives after the last processed measurement z k
There are some methods for updating the state estimate globally optimally with an out-of-sequence measurement (OOSM) within one-step time delay (i.e., l = 1, referred
Multi-sensor Multi-target Tracking with OOSM
229
to as one-step update) for a system with nonsingular state transition matrix [3], and multi-step OOSM updating using augmented state smoothing [4]. Also, one-step suboptimal updating algorithms using stored information have been proposed for systems with invertible state transition matrix [5]; multi-step update was discussed in [6] without any discussion of the optimality. The globally optimal update algorithm and the optimal update algorithm with limited information are optimal in the linear minimum mean-square error (LMMSE) sense. When he required condition holds, our optimal update algorithm given limited information reduces to the suboptimal algorithms of [7], which provides a simple proof of the optimality of these generally suboptimal algorithms. The dynamics and measurement models assumed are given by
where
x j = Fj,j-1x j-1 + w j,j-1
(1)
z j =H j x j +v j
(2)
Fj,j-1 is the state transition matrix from time t j-1 to t j and w j,j-1 is (the
cumulative effect of) the process noise for the interval
[t j-1 , t j ] . The process noise
w j,j-1 and the measurement noise v j are white and have zero mean and covariances
C wj,j-1 =cov(w j,j-1 )= Q j,j-1
(3)
C vj = cov(v j ) = R j
(4)
t d is in the sampling interval t k-l ≤ td < t k-l+1 , where l = 1, 2, " . Which means that the OOSM z d is l lags behind. Similar to (1), we have
Suppose time
x k = Fk,d x d + w k,d
z d = H d x d + vd
(5) (6)
At time t k the LMMSE estimator is
xˆ k|k E* (x k |z k ) = arg minPk|k Pk|k MSE[xˆ k|k ] Where
(7) (8)
x = x - x , Letting C xz =cov(x, z) and Cz =var(z) , xˆ E* (x|z) = x+ C xz C-1z (z - z)
(9)
ˆ = E(xx') MSE(x)
(10)
230
C. Cheng and J. Wang
In the above, z
k
k {zi }i=1 is
the measurement sequence through t k . If the inverse
-1 z
C does not exist, it can be simply replaced with the unique Moorre-Penrose C-1z . Subsequently, an earlier measurement at ˆ k|k and error covariance Pk|k have been time t d arrives after the state estimate x pseudoinverse (MP inverse in short)
calculated. We want to update this estimate with the earlier measurement
z d , that is,
to calculate the LMMSE estimator
xˆ k|k,d = E* (x k |Ωk , z d ) Pk|k,d = MSE[xˆ k|k,d ] Where
(11) (12)
Ωk stands for the information available for update with the OOSM z d .
It follows from the proposed constrained optimal OOSM update Algorithm, that
where E
*
* xˆ ik|k,d =E* (x k |xˆ k|k , zid ) = Kz i
(13)
i Γ)R(K Γ)'+Q − ΓRΓ ' Pk|k,d = MSE(xˆ ik|k,d ) = (Kk,d
(14)
(x k |xˆ k|k , z id ) is the LMMSE without prior and z*i = ⎡⎣ xˆ 'k|k ( zdi ) '⎤⎦ ' , with
md
zd = ∑ β di zi* ,we have i =1
xˆ k|k,d =β d0 xˆ k|k +Kz d Let
(15)
Γ)R(K Γ)'++Q − ΓRΓ ' then Pkc|k ,d = (Kk,d c Pk|k,d =β d0 Pk|k +[1+β d0 ]Pk|k,d
(16)
4 Performance Evaluation Consider a discretized continuous time kinematic system driven by white noise with power spectral density q, known as constant velocity model or white-noise acceleration model in target tracking, described by the equations (1) and (2). where
⎡1 T ⎤ Fj = ⎢ ⎥ ⎣0 1 ⎦
H j = [1,0]
⎡T 3 / 3 T 2 / 3⎤ var( w j ) = Q = ⎢ 2 ⎥ q, var(v j )=R=1 T ⎦ ⎣T / 2
Multi-sensor Multi-target Tracking with OOSM
231
where T is the sampling interval. The prior information is
⎡ R xˆ0|0 = x = [200km, 0.5km / sec]', P0|0 = ⎢ ⎣R / T
R /T ⎤ 2 R / T 2 ⎥⎦
and the maneuver index is λ = qT3 /R . In order to consider multi-lag delay as well as single-lag OOSM update, we choose a series of OOSMs z d . These OOSMs occurred at d = (l + 1) n and arrived at (l +1) n + l with n = 1, 2, . . ., corresponding to l-lag delayed OOSMs, where l = 1, 2, . . .. For cluttergeneration we use a poisson model with a spatial density λ to get the number of false measurements in the validation region:
μF (m) = e- λ V
∈
λV m!
By choosing λV [0, 1], we can simulate different clutter densities. We use PD = 1. Residual based χ2 (99.9) test is used for testing tracking divergence. In simulation results, there is less than 10% track loss. We show RMS errors over 1000 monte Carlo runs for the OOSM updated states at (l +1) n+l with n = 1, 2, . . ., where KF — in-sequence Kalman filter without clutter; IS-PDAF— in-sequence PDA filter; IG-PDAF — in-sequence PDA filter ignoring the OOSMs; UD-PDAF—PDA with globally optimal OOSM Update. We design the OOSM model by choosing var (vj) = R/10 at j = (l + 1) n with n = 1, 2, . . . , which means the OOSMs are more accurate than the in-sequence measurements. In the following, we show the RMS errors for the 1-lag, 2-lag and 4lag OOSM update problems at time k = 29 with λ V ∈{0, 0.25, 0.5, 0.75, 1} in Fig.2, Fig.3 and Fig.4 respectively.
Fig. 2. RMS errors with 1-lag OOSMs
232
C. Cheng and J. Wang
Fig. 3. RMS errors with 2-lag OOSMs
Fig. 4. RMS errors with 4-lag OOSMs
As shown in these figures, when there is no clutter, i.e., λV = 0, PDA with globally optimal OOSM Update Algorithm has the same performance as the KF. For target tracking in clutter, the PDA with OOSM update Algorithm yields RMS errors close to the KF, which indicates that through OOSM update, the performance has significant improvement especially for small-lag OOSMs. For large-lag OOSMs, by ignoring them, the performance does not suffer much even if the OOSMs have much better accuracy. The RMS errors of UD-PDAF is very close to that of the in-sequence PDA filter. It also shows that the performance of IS-PDAF, IG-PDAF and UD-PDAF deteriorates as the clutter becomes heavier.
5 Conclusion In this paper, we proposed using PDA with optimal OOSM Update Algorithm in the presence of clutter. Simulation results show that the PDA with the OOSMs update in clutter performs significantly better than ignoring the OOSMs, especially for smalllag OOSMs. Its performance is close to the in-sequence PDA update for OOSMs with various lags and under mild clutter where the PDA filter has less than 10% track loss.
Multi-sensor Multi-target Tracking with OOSM
233
In summary, the PDA incorporating the two OOSM update algorithms has (1) an efficient processing structure; (2) an efficient memory structure; (3) an efficient computational structure.
References 1. Challa, S., Robin, J.E., Wang, X.: A Bayesian Solution and Its Approximations to Out-ofSequence Measurement Problems. Journal of Information Fusion 4, 185–199 (2003) 2. Challa, S., Wang, X., Legg, J.: Track-to-Track Fusion using Out-of-Sequence Tracks. In: International Conf. on Information Fusion, pp. 919–926 (2007) 3. Bar-Shalom, Y., Li, X.R.: Estimation and Tracking: Principles, Techniques, and Software. Artech House, Boston (2004) 4. Challa, S., Evans, R., Legg, J.: A Fixed Lag Smoothing Framework for OOSI Problems. Journal of Communications in Information and Systems 2, 327–350 (2002) 5. Thomopoulos, S., Zhang, L.: Distributed Filtering with Random Sampling and Delay. In: 27th IEEE Conf. Decision and Control, pp. 539–545 (2005) 6. Yang, W.i.: Multisensor data Fusion With Application. Xidian University Press, Xi‘an (2004) 7. Bar-Shalon, Y.: On the Track-to-Track Correlation Problem. IEEE Transactions on Automatic Control 26, 571–572 (1981)
Hopfield Neural Network Guided Evolutionary Algorithm for Aircraft Penetration Path Planning Nan Wang, Lin Wang, Xueqiang Gu, Jing Chen, and Lincheng Shen Mechatronics and Automation School of National University of Defense Technology, 410073, Changsha, China [email protected]
Abstract. This paper proposes a Hopfield neural network guided evolutionary algorithm for aircraft penetration path planning. The combination of the algorithms benefits from the advantages of each and is intended to achieve a fast and adaptive path searching mechanism. The HNN works as guidance for the EA path planner by restricting the searching area around the gradient of the states of the network. Meanwhile the complex penetration factors are well integrated into the algorithm by EA so making the combined algorithm applicable to penetration path planning. Extensive simulations are conducted to demonstrate the effectiveness of the propose algorithm. Keywords: Hopfield NN, evolutionary algorithm, penetration, path planning, HNN guided EA.
1 Introduction Penetration path planning is a key issue to maintain the survivability of aircrafts. It constructs a survivable penetration trajectory through enemy defense area, which takes into account terrain data, threats’ distribution and mission goals. With the development of air-defense systems, threat environment becomes more complex, and many traditional path planning algorithms are no longer applicable. However, there is still little work being reported to deal with this challenging problem. The key factors contained in the penetration path planning problem in modern battlefield environments include: a) Terrain mask. Survivability is enhanced by masking the penetrating aircraft from the threats. The terrain mask analysis is of great importance for aircraft to penetrate the threats’ envelopes. b) Terrain following. Terrain following is widely used by aircrafts to fly at a low altitude to get masked from threats. Whereas terrain with high elevations or low clearance levels should be avoided in path generation. c) Threat uncertainty. Threats in a modern battlefield typically have three types: known, mobile and unknown. The mobile and unknown threats are actually more fatal to aircrafts if not taken into consideration. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 235–243. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
236
N. Wang et al.
d) Other criteria and restrictions. The path planning algorithm should be capable of incorporating time limits, fuel usage and the aircraft maneuverability into planning procedure. The current widely used path planning algorithms, such as Voronoi diagram [1], neural network [2], artificial potential [3] and probabilistic roadmap [4], are fast and efficient, but encounter difficulties to incorporate different types of factors into their planning models. Evolutionary algorithm (EA) [5] is adaptable to various compounds of factors, but suffers from lower speed and efficiency. Though some efforts have been made towards improvement and hybrid of these methods [6][7], there are still steps to be taken to make them applicable in complex mission environments. In this paper, a Hopfield neural network (HNN) guided evolutionary path planning algorithm is proposed. The benefits of Hopfield neural network are that it can be calculated fast and can guarantee target arrival and avoid obstacles and local minima. The benefits of EA, on the other hand, are that it can deal with various factors such as terrain effect, threat uncertainty and other criteria and restrictions. The combination of the two results in a robust and effective algorithm for aircraft penetration path planning under modern battlefield environments. This paper is organized as following: Hopfield neural network for path planning is introduced first. Then, the modeling of penetration factors is illustrated and HNN guided EA is presented. Extensive experiments are carried out to demonstrate the effectiveness of the proposed algorithm.
2 HNN for Path Planning Hopfield neural network is a potential based network which consists of a serious of neurons. Each neuron is connected to n adjacent neurons, as shown in Fig. 1. The connections between the neurons are weighted, in which wij stands for the weight of the connection from neuron j to neuron i. Commonly, the weights are symmetrical (wij=wji) and zero self-coupling (wii=0).
Fig. 1. The Topology of HNN
The topology of HNN makes it suitable for path planning in 2-D C-space for target pursuit and obstacle avoidance. In this paper, an improved HNN algorithm [8] is adopted. The algorithm quickly sets up a numerical network from the end cell (target) to the edge of the C-space using distance transformation based serial simulation [9].
HNN Guided Evolutionary Algorithm for Aircraft Penetration Path Planning
237
Then the optimal path could be found following the state gradients of the network using steepest ascent principle. The planning procedure of the improved HNN algorithm is as follows. First the planning space is expressed by the Hopfield network, i.e., each cell in C-space is regarded as a neuron. Each free neuron (non-obstacle neuron) can be connected with n adjacent neurons (In this the paper, n = 8), as shown in Fig. 1. Then, the network propagation starts from the end neuron that consequently updates the output value of adjacent neurons by distance transformation based serial simulation, as shown in (1) and (2), where Xi represents the output of the ith neuron, f(X) represents the state of the X, I is the input value to the end neuron, Di is 0 for obstacle neurons and 1 for others, k is the iteration number, NEi is the set of adjacent neurons of i, A>2n. When all the states of the neurons are updated, the optimal path from the start cell to the end cell can be found quickly following the steepest gradient descent of the state values. ⎧ Di × f ( X ) / A + I / A, for end neuron X i ( k + 1) = ⎨ ⎩ Di × f ( X ) / A, other neurons
f (X ) =
∑
j < i ∩ j∈NEi
wij X j ( k + 1) +
∑
j >i ∩ j∈NEi
wij X j ( k )
(1)
(2)
3 Modeling of Penetration Factors 3.1 Terrain Mask Analysis Terrain mask is usually represented by the line-of-sight maps (also called intervisibility map) of individual threats. This paper adopts a non-ray-tracing algorithm to calculate the line-of-sight map. This algorithm is very fast and only need to be executed once to attain masked area at different fly height. Details of the algorithm are described in [10]. 3.2 Terrain Following Analysis The quality of terrain following is determined by the clearance levels. The clearance levels (Δhi) means the vertical distance between the lowest height (Hi) that the aircraft can be seen and the terrain elevation (hi). The clearance level is taken as one of the main criteria that imposed on the grids of the map, as shown in (3). ⎧ if Δhi ≤ H F ⎪1 C j =⎪ ⎨ ⎪ ⎪ ⎩ H F / Δhi else
(3)
where Cj is the clearance merit of the jth grid. 3.3 Threat Modeling As mentioned in section 1, threats are classified into known, mobile and unknown types. In this paper, the danger index at any position inside a known threat envelope is calculated as a function of the aircraft position relative to the threat site, terrain mask, as well as the difference in altitude between threat site and aircraft.
238
N. Wang et al.
For mobile and unknown threats, the “area-intervisibility” is adopted to approximately measure the danger index. The area-intervisibility of a given position implies the probability that aircraft at that position would be detected (under line-of-sight) and attacked by mobile and unknown threats. Finally, the mixed danger index of a given grid on the map is calculated as weighted sum of the danger index of known threats and the area-intervisibility, as shown in (4). This is the second main criteria that imposed on the grids of the map. j Dmix = wk × ∑ Di / N tht + wu × Pint
(4)
i
where Djmix is the mixed danger index of the jth grid; Di is the danger index of threat i; Ntht is the number of corresponding threats; Pint is the area-intervisibility of the given position; wk and wu are weights and wk + wu =1. 3.4 Other Criteria and Restrictions The typical mission requirements such as time limit and fuel usage are evaluated based on the length of the total path and the given velocity of the aircraft, which can be taken as criteria and constrains accordingly. The maneuverability of the aircraft is represented by the maximum turning angle, climb and dive angle of the aircraft, which is widely used by path planning algorithms.
4 HNN Guided EA 4.1 EA Based Path Planning Algorithm Typical EA based path planning algorithms use gene sequences to model the path of the aircraft, in which each gene represents the position or movement of the aircraft. In this paper, a fixed-length gene sequence representing the movement actions of the aircraft is adopted. The movement actions are defined as Table 1. Each gene adopts integer coding and has a value of 1-3. Examples of gene sequence under this scheme are shown in Fig. 2. Table 1. Movement actions defined in the proposed EA
Action A1 A2 A3
Meaning no change left turn right turn
Value 1 2 3
The dimensionally of the planning space of the proposed algorithm is 2.5D. This means that the actual space is two dimensional, but warped to the surface of terrain using terrain following. Path planning in 2D space is much faster than 3D space, and is more compatible with the HNN scheme mentioned above.
HNN Guided Evolutionary Algorithm for Aircraft Penetration Path Planning
239
Fig. 2. Gene sequence examples
To evaluate the criteria and constrains of the candidate movement actions, the actions is mapped into a plane trajectory using (5) and (6). ⎧ ⎪ φi (tk ) = φi (tk −1 ) − θ ⎪ ⎪ ⎪ ⎨φi (tk ) = φi (tk −1 ) + θ ⎪ ⎪ ⎪ ⎪ ⎩tk = k ×Tstep
if there is a left turn action if there is a right turn action
⎧ x(tk ) = x(tk −1 ) + v ×sin(φ (tk )) ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ y (tk ) = y(tk −1 ) + v× cos(φ(tk ))
(5)
(6)
where Φ is the its heading angle and Φ=0 means flying towards north; θ is the maximum turning angle in each step; Tstep is the time step of each movement; v is the given penetration velocity; x and y are the 2D position for the aircraft at time tk. When mapped, the plane trajectory is further transferred into a 3D trajectory by calculating the profile of the path as shown in (7). To accommodate the climb and dive angle constrains mentioned in section 3.4, the profile is calibrated using the protuberant points in z. z(tk ) = HF + Z (x(tk ), y(tk ))
(7)
where z is the fly height sequence of the aircraft profile; HF is the terrain following altitude; Z(x, y) is the terrain elevation of position (x, y). The planning procedure of the proposed EA algorithm is described as follows: Algorithm 1: EA based path planning Pre-determined phase: (1) Initialize corresponding parameters and EA arguments (2) Generate a set of M gene sequences Mutation-offspring phase: (3) Set offspring counter gm=1 (4) Repeat (5) Apply the mutation operator to the M gene sequences and generate another M sequences (6) Map the 2M sequences into 2M trajectories (7) Evaluate the criteria and constrains of the 2M trajectories based on penetration factors (8) Choose the best M out of 2M trajectories (9) gm=gm+1 (10) Until the terminal iteration condition is reached or the offspring time is used up, i.e., gm>Gm (11) Select the optimal path from the trajectories as the result
240
N. Wang et al.
4.2 HNN Guided EA Algorithm HNN has the advantage of target pursuit and obstacle avoidance in path planning, which is good for guiding the direction of EA to concentrate on the paths towards target while avoiding sharp terrain and lethal threats as obstacles. The HNN guidance occurs at the mapping step of the proposed EA. Instead of searching as the ‘left direction’ and ‘right turn’ portion in (5), the HNN is used to guide the EA as (8) shows: ⎧φi (tk −1 ) − θ if no η p & left turn ⎪ ⎪ ⎪ ⎪ φ (t ) + θ if no η p & right turn ⎪ ⎪ i k −1 φi (tk ) = ⎪ ⎨ ⎪ if η p − φi (t k −1 ) ≤ θ & turn ηp ⎪ ⎪ ⎪ ⎪ φ (t ) + θ ×sign(η p − φi (tk −1 )) if η p − φi (tk −1 ) > θ & turn ⎪ ⎪ ⎩ i k −1
(8)
where ηp is the gradient direction of the states of the HNN. If the HNN guided heading angle is within the aircraft maneuverability bounds, it is selected as the movement. If not, the mapping mechanism calculates a heading angle that towards the gradient of the HNN as much as possible. Therefore, the searching area is guided around the gradient of the HNN that improves the efficiency of pure EA algorithm in path planning. 4.3 Overall Planning Algorithm The planning procedure of the overall algorithm for penetration path planning is as follows: Algorithm 2: HNN guided EA based path planning (1) Initialize mission environment and threat parameters (2) Analyze the falls of the terrain and mark the grids with sharp falls as obstacles (3) Analyze the threats on the battlefield and identify the most lethal threats to be avoided (4) Build the HNN on the 2D data grid map (5) Analyze the terrain mask and the area-intervisibility to calculate the mixed danger index Dmix of each grid on the map (6) Perform EA pre-determined phase (7) Repeat (8) Conduct mutation-offspring (9) Map the movement actions into trajectories using the HNN guidance (10) Evaluate the criteria and constrains of the trajectories (11) Perform trajectories selection (12) Until the terminal iteration condition is reached or the offspring time is used up (12) Select the optimal path from trajectories the with the best evaluation value as the result
HNN Guided Evolutionary Algorithm for Aircraft Penetration Path Planning
241
5 Simulation Results To demonstrate the performance of the HNN guided EA algorithm, several experiments are carried out under typical mission scenarios, as shown in Fig. 3. The results of pure HNN, pure EA and the HNN guided EA under scenario (a) and (b) are shown in Fig. 4 and Fig. 5 respectively. Table 2 gives a comparison of the performance of results of the proposed algorithms.
(a)
(b)
Fig. 3. Test mission scenario, where blue rings represent threat rings, and the area filled with green color represents the intervisibility area of threats at aircraft height 200m
(a) HNN
(b) EA
(c) HNN+EA
Fig. 4. Best results of the three algorithms under mission scenario (a)
(a) HNN (no solution)
(b) EA
(c) HNN+EA
Fig. 5. Best results of the three algorithms under mission scenario (b)
242
N. Wang et al. Table 2. Comparison of results of different algorithms
(a) HNN Scenario (a) 0 0 72.5 (b) EA To Minimize Scenario (a) Obstacle constrain 0 Goal arrival constrain 20.8 Grid criteria value 8.1 (c) HNN guided EA To Minimize Scenario (a) Obstacle constrain 0 Goal arrival constrain 0 Grid criteria value 7.8 To Minimize Obstacle constrain Goal arrival constrain Grid criteria value
Scenario (b) / no solution / no solution / no solution Scenario (b) 1 23.5 20.2 Scenario (b) 0 0 19.5
The convergence performance of the proposed HNN guided EA under two scenarios tests is shown in Fig. 6 below.
Fig. 6. The convergence of the HNN guided EA (500 iterations), where the upper line in blue represents the grid criteria values, and the lower in red represents the sum of constrains
6 Conclusion In this paper, the characteristics of the aircraft penetration path planning are analyzed through factors of terrain, threat and other mission requirements and restrictions. Towards the efforts of implementing robust and efficient methods to solve the penetration path planning problem, a HNN guided EA algorithm is proposed, in which the efficiency of EA searching is improved by adding HNN guidance to the mapping step of the EA. The HNN guidance serves as heuristic knowledge of target pursuit and avoiding sharp terrain and lethal threats. Experiments demonstrate that the proposed algorithm does well in solving the penetration path planning problem compared to pure HNN and pure EA, which implies that hybrid methods may be more competitive in solving complex path planning problems.
HNN Guided Evolutionary Algorithm for Aircraft Penetration Path Planning
243
References 1. Bortoff, S.A.: Path planning for UAVs. In: Proceedings of the American Control Conference, Chicago, Illinois, pp. 364–368 (1991) 2. Weiß, B., Naderhirn, M., del Re, L.: Corporative UAV Formation Flying with Obstacle/Collision Avoidance. IEEE Transactions on Control Systems Technology 15, 672–679 (2007) 3. Kitamura, Y., Tanaka, T., Kishino, F., Yachida, M.: 3-D Path Planning in a Dynamic Environment using an Octree and an Artificial Potential Field. In: IEEE International Conference on Intelligent Robots and Systems, pp. 474–481 (1995) 4. Pettersson, P.O., Doherty, P.: Probabilistic Roadmap Based Path Planning for an Autonomous Unmanned Aerial Vehicle. In: The Workshop on Connecting Planning and Theory with Practice, ICAPS (2004) 5. Rathbum, D., Capozzi, B.: Evolutionary Approaches to Path Planning under Uncertain Environments. In: 1st Unmanner Aerospace Vehicles, Systems, Technologies, and Operations Conference and Workshop, AIAA (1994) 6. Du, Z., Qu, D., Xu, F., Xu, D.: A Hybrid Approach for Mobile Robot Path Planning in Dynamic Environments. In: Proceedings of the 2007 IEEE International Conference on Robotics and Biomimetics, pp. 1058–1063 (2007) 7. Han, J.D., Campbell, M.: Artificial Potential Guided Evolutionary Path Plan for Target Pursuit and Obstacle Avoidance. In: AIAA Guidance, Navigation, and Control Conference and Exhibit (2003) 8. Wang, N., Gu, X., Chen, J., Shen, L., Ren, M.: A Hybrid Neural Network Method for UAV Attack Route Integrated Planning. In: Yu, W., He, H., Zhang, N. (eds.) ISNN 2009, Part III. LNCS, vol. 5553, pp. 226–235. Springer, Heidelberg (2009) 9. Fan, C., Lu, Y., Liu, H., Huang, S.: Path Planning for Mobile Robot Based on Neural Networks (in Chinese). Computer Engineering and Applications 8, 86–89 (2004) 10. Leavitt, C.A.: Real-Time In-Flight Planning. In: Proceedings of the IEEE 1996 National Conference on Aerospace and Electronics, vol. 1, pp. 83–89 (1996) 11. Rana, A.S., Zalzala, A.M.S.: A Neural Networks Based Collision Detection Engine for Multi-Arm Robotic Systems. In: 5th International conference on artificial neural networks, pp. 140–145 (1997) 12. Ellips, M., Davoud, S.: Classic and Heuristic Approaches in Robot Motion Planning–A Chronological Review. Proceedings of World Academy of Science, Engineering and Technology 23, 101–106 (2007) 13. Lagoudakis, M.G.: Hopfield Neural Network for Dynamic Path Planning and Obstacle Avoidance 14. Capozzi, B., Vagners, J.: Evolving (semi)-autonomous vehicles. In: AIAA Conf. Guidance, Navigation and Control (2001) 15. Yang, G., Vikram, K.: Optimal path planning for unmanned air vehicles with kinematic and tactical constraints. In: Proceedings of the 41st IEEE Conference on Decision and Control, vol. 2, pp. 1301–1306 (2002)
Fuzzy Material Procurement Planning with Value-at-Risk Gao-Ji Sun College of Mathematical and Physical Science, Lishui University, Lishui 323000, China [email protected]
Abstract. Based on credibility theory, this paper presents a class of two-stage fuzzy programming with value-at-risk (VaR) to deal with material procurement planning (MPP) problem. Since the MPP problem usually includes continuous fuzzy variable parameters with infinite supports, it is inherently an infinite-dimensional optimization problem that can rarely be solved directly. To overcome this difficultly, this paper introduces an approximation approach (AA) which can turn the infinitedimensional optimization problem into a finite-dimensional optimization one. Furthermore, in order to solve the proposed MPP model, a hybrid algorithm is designed which combines AA, neural network (NN) and particle swarm optimization (PSO). Additionally, one numerical example is also presented to illustrate the effectiveness of the designed algorithm. Keywords: Material procurement planning, Neural network, Credibility theory, Approximation approach, Particle swarm optimization.
1
Introduction
MPP deals with the problem that purchasing the right quantity of material from the right supplier at the right time, it is an important component of production planning. Since a reasonable MPP decision can reduce the material procurement cost or avoid the risk of material procurement, it attracts many researchers’ interest. However, in the practical MPP problem, lots of parameters such as material demand, spot market material unit price have the uncertainty. In order to deal with the MPP problems under the stochastic environment, some MPP models were proposed based on probability theory. The interested readers can refer to [1,2,3], they assumed that the uncertain parameters are random variables with known probability distribution functions. It is well-known that the acquisition of probability distribution functions relies on a large number of statistical data. However, there is no enough statistical data in many practical MPP problems, these MPP problems should be treated as fuzzy optimization problems. There are some researchers handled fuzzy MPP problems which take fuzzy set [4] or possibility theory [5] as the theoretical foundation, the related literatures contain Lai & Hwang [6] and Mula & Poler [7], etc. Based on possibility measure, Liu & Liu [8] presented a self-dual set function called credibility measure, to Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 245–252. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com
246
G.-J. Sun
measure fuzzy event. Furthermore, Liu [9] developed credibility theory in fuzzy decision systems. Recent researches about this theory included Liu [10,11], Liu & Tian [12], Sun & Liu [13] and Sun, Liu & Lan [14] etc. In addition, credibility theory has attracted much attention and been applied in many fields such as engineering optimization, economics and management problems. The purpose of this paper is to take credibility theory as the theoretical foundation of fuzzy programming, and present a class of two-stage fuzzy programming with VaR to deal with MPP problems. Because of the MPP problem usually includes fuzzy variable parameters defined through possibility distributions with infinite supports. Therefore, in order to solve the proposed MPP problem, an AA [11] is used to turn the infinite-dimensional model into a finite-dimensional one, and the AA can establish several convergence relationships between the original MPP model and the approximating one. Thence it is feasible to solve the approximating model directly instead of the original one. However, the approximating model is neither linear nor convex, conventional optimization algorithms cannot be applied. As a result, a hybrid algorithm by integrating the AA, NN and PSO is designed to solve it. One numerical example is also given to show the effectiveness of the designed hybrid algorithm. The remainder of this paper is organized as follows. Section 2 proposes the two-stage fuzzy MPP model with VaR. Section 3 applies the AA to turn the original model into an approximating one and designs a hybrid algorithm to solve the approximating MPP model. Furthermore, a numerical example is also proposed to illustrate the effectiveness of the designed algorithm. Section 4 gives the conclusions.
2
Fuzzy MPP Model with VaR
Since many real-life MPP problems include fuzzy variable parameters, the firststage decisions are often infeasibility, hence second-stage or recourse actions are required. Additionally, VaR is an important criteria to model optimization problems involving risk. Therefore, the intent of this paper is to propose a class of two-stage fuzzy MPP model with VaR. In order to propose a reasonable MPP model, we will summarize the assumptions and notations, respectively. Assumptions • The source of material contains three components: in effect contracts, pending contracts and spot market, in which in effect contracts are long-term contractual commitments; pending contracts are new supply contracts that can be selected to negotiate, and the purchaser holds the election. • The fuzzy variable parameters include spot market material unit price and material demand. They are mutually independent [15] fuzzy variables with known possibility distributions. • The in effect contracts have a fixed material unit price and supply quantity range, respectively. These assumptions are also suitable for the pending
Fuzzy Material Procurement Planning with Value-at-Risk
247
contracts. In addition, the total amount of material purchased over two stages cannot less than material demand. Notations xi : material amount purchased from the ith in effect contract in first-sage; δk : binary variable equals to 1 if the kth pending contract is signed and equals to 0 otherwise; xk : material amount purchased from the kth pending contract in first-sage; yi : material amount purchased from the ith in effect contract in second-stage; yk : material amount purchased from the kth pending contract in second-sage; y: material amount purchased from the spot market; ci : material unit price of the ith in effect contract; li : lower bound of the ith in effect contract’s supply quantity range; ri : upper bound of the ith in effect contract’s supply quantity range; ck : material unit price of the kth pending contract; lk : lower bound of the kth pending contract’s supply quantity range; rk : upper bound of the kth pending contract’s supply quantity range; ξ1 (γ): spot market material unit price under fuzzy event γ; ξ2 (γ): material demand under fuzzy event γ. We assume that there are m in effect contracts and n pending contracts. The first-sage decision variables contain (xi , δk , xk ) and the variables (yi , yk , y) are the second-stage decisions, where i = 1, 2, · · · , m; k = 1, 2, · · · , n. For convenience, we denote the first-stage decision variables (xi , δk , xk ) by vector x and the secondstage decision variables (yi , yk , y) by vector y. The parameters (ci , li , ri , ck , lk , rk ) are fixed value, and (ξ1 , ξ2 ) are fuzzy variable parameters. Using the assumptions and notations above, the two-stage fuzzy MPP model can be formulated as follows. min QαVaR (x) s.t. li ≤ xi ≤ ri , i = 1, 2, · · · , m (1a) δk lk ≤ xk ≤ δk rk , k = 1, 2, · · · , n (1b) (1c) xi ≥ 0, xk ≥ 0, δk = 0 or 1,
(1)
m n ci xi + ck xk + Q (x, ξ(γ)) ≤ ϕ ≥ α , where QαVaR (x)=inf ϕ | Cr γ | i=1
i=1
α is a preselected credibility level which belongs to (0, 1), Q(x, ξ(γ)) is the optimal objective value of second-stage under fuzzy event γ, and it is obtained by the following programming m n Q(x, ξ(γ)) = min i=1 ci yi + k=1 ck yk + ξ1 (γ)y i = 1, 2, · · · , m (2a) s.t. li ≤ xi + yi ≤ ri , k = 1, 2, · · · , n (2b) δk lk ≤ xk + yk ≤ δk rk , n m i=1 (xi + yi ) + k=1 (xk + yk ) + y ≥ ξ2 (γ) (2c) (2d) yi ≥ 0, yk ≥ 0, y ≥ 0.
(2)
248
G.-J. Sun
For a fixed budget ϕ0 , we take the maximal credibility of fuzzy procurement cost less than ϕ0 as the risk level of budget funds ϕ0 . The objective of proposed fuzzy MPP model (1)–(2) is to find the minimal budget fund such that its risk level exceeding the preselected credibility level α is met. In addition, the constraints (1a, 1b, 2a, 2b) express that the material amount purchased from each supply contract must satisfy the contract’s restrictions. The constraint (2c) indicates that the total amount of material purchased over two stages cannot less than material demand. The constraints (1c) and (2d) determine the domains of decision variables.
3
Hybrid Algorithm and Numerical Example
The intent of this section is to design an effective algorithm for solving the proposed two-stage fuzzy model (1)–(2). Furthermore, we test the designed algorithm through a numerical example. 3.1
Hybrid Algorithm
1
1
0.8
0.8 Possibility distribution
Possibility distribution
According to the two-stage fuzzy model (1)–(2), it is easy to see that for any given first-stage feasible decision x and every realization (ξ1 (γ), ξ2 (γ)), the secondstage programming (2) is a linear programming that can be solved by simplex algorithm. In order to solve the two-stage fuzzy MPP model, it is required to evaluate the function Qx : x → QαVaR (x) repeatedly. Since any computer cannot compute infinite times, the value of QαVaR (x) at x cannot be obtained when (ξ1 , ξ2 ) are continuous fuzzy vector. Therefore, we should use AA to turn the continuous fuzzy vector (ξ1 , ξ2 ) into a sequence of discrete fuzzy vector (ζp,1 , ζp,2 ) which named the discretization of original fuzzy vector. About the AA in details, the interested readers can refer to Liu [11]. Here we only compare a triangular fuzzy variable ξ = (0, 1, 2) and its discretization ζp when p = 5, p = 10 via Fig 1, in which the line represents the possibility distribution of ξ, and the points represent the possibility distribution of ζp .
0.6
0.4
0.2
0
0.6
0.4
0.2
0
0.5 1 1.5 Fuzzy variable and its discretization when p=5
2
0
0
0.5 1 1.5 Fuzzy variable and its discretization when p=10
2
Fig. 1. The comparison of possibility distribution of ξ and its discretization ζp
Fuzzy Material Procurement Planning with Value-at-Risk
249
Because of the discretization (ζp,1 , ζp,2 ) obtained via AA can converge to (ξ1 , ξ2 ), thence we can replace the continuous fuzzy vector (ξ1 , ξ2 ) by its discretization. As a result, the original infinite-dimensional optimization problem is turned into a finite-dimensional optimization one. However, for any given first-stage decision x and each realization (ζp,1 (γ), ζp,2 (γ)), we have to solve the second-stage programming (2) via simplex algorithm one time, it is a timeconsuming process. In order to speed up the solution process, we desire to replace the objective function Qx by a trained NN. In this paper, we employ the fast BP algorithm to train a feed-forward NN [16]. After the NN is well-trained, we can embed it into PSO to produce a hybrid algorithm. The PSO algorithm is a population based algorithm that invented by Kennedy & Eberhart [17]. Recently the PSO algorithm has attracted much attention and been successfully applied in the fields of evolutionary computing, unconstrained continuous optimization problems and many other fields, the interested reader can refer to [18]. PSO uses a particle to indicate a possible solution of the optimization problem, and generates pop size particles to formulate a colony. It searches for the optimal solution by updating generations. The best previous position encountered by the ith particle is denoted as Pi , the particle that attained the best previous position among all the individuals of the colony is denoted as Pg , and t to be the iteration counter. The colony is manipulated according to the following equations: Vi (t + 1) = ωVi (t) + c1 r1 (Pi (t) − Xi (t)) + c2 r2 (Pg (t) − Xi (t))
(3)
Xi (t + 1) = Xi (t) + Vi (t + 1)
(4)
where i = 1, 2, · · · , pop size; ω is called the inertia coefficient; c1 and c2 are learning rates and usually c1 = c2 = 2; r1 and r2 are are two independent random numbers generated randomly in the unit interval [0,1]. Now we only provide the summary about the process of hybrid algorithm. A Hybrid Algorithm Step 1. Generate a set of input-output data for the objective function Qx of approximating MPP model. Step 2. Train an NN to approximate the approximating objective function Qx by the generated input-output data. Step 3. Initialize pop size particles with random positions and velocities, and evaluate the objective values of all the particles by the trained NN. Step 4. Set Pi of the ith particle and its objective value equal to its current position and objective value, and set Pg and its objective value equal to the position and objective value of the best particle in the initial colony. Step 5. Update the velocity and position of each particle according to the formulas (3) and (4), respectively. In addition, calculate the objective values of all new generated particles by the trained NN. Step 6. For the ith particle, compare the current objective value with that of its Pi . If the current objective value is smaller than that of Pi , then replace Pi and its objective value by the current position and objective value.
250
G.-J. Sun
Step 7. Find the best particle of the current swarm with the smallest objective value. If its objective value is smaller than that of Pg , then renew Pg and its objective value with the position and objective value of the current best particle. Step 8. Repeat Step 5 to Step 7 until a stopping criterion is satisfied, and return the Pg and its objective value as the optimal solution and the optimal value. 3.2
Numerical Example
To show the feasibility and effectiveness of the hybrid algorithm. we provide a MPP model with the credibility level α = 0.9, the unit price of the material on the spot market is triangular fuzzy variable ξ1 = (8, 9, 10), the material demand is a trapezoidal fuzzy variable ξ2 = (1000, 1200, 1500, 1600). In addition, the fuzzy variable ξ1 and ξ2 are supposed to be mutually independent. The other related parameters are collected in Table 1. Table 1. The parameters of numerical example l1 50 r1 80 c1 10
l2 50 r2 80 c2 10
l3 80 r3 120 c3 9
l4 80 r4 120 c4 9
l5 100 r5 150 c5 8
l6 100 r6 150 c6 8
l1 50 r1 100 c1 10
l2 50 r2 100 c2 10
l3 50 r3 100 c3 9
l4 100 r4 150 c4 9
l5 100 r5 150 c5 8
l6 100 r6 150 c6 8
First, using AA with 2000 sample points, we generate 3000 input-output data for the objective function Qx . Then we use the 3000 input-output data to train an NN (12 input neurons representing the value of first-stage decision x, 15 hidden neurons and 1 output neurons representing the value of Qx ) to approximate function Qx . Finally, the well-trained NN is embedded into a PSO to search for the optimal solution of MPP model. A run of the hybrid PSO algorithm with 30 particles and 500 generations produces the following optimal solution: x1 =50.0000, x2 =50.0000, x3 =80.0000, x4 =80.0000, x5 =100.0000, x6 =100.0000, x1 =0.0000, x2 =0.0000, x3 =0.0000, x4 =0.0000, x5 =150.0000, x6 =100.0000, and its optimal value is 12952.1656. Based on general knowledge, we know that the contracts whose unit price is lower should be selected. So according to the parameters in our numerical example, we can see that the result above is reasonable, which shows that the designed hybrid algorithm is effective.
4
Conclusions
In this paper, we proposed a class of two-stage fuzzy programming to handle MPP problem with VaR criteria based on credibility theory. Since the fuzzy
Fuzzy Material Procurement Planning with Value-at-Risk
251
variable parameters had continuous possibility distribution with infinite support that cannot be solved directly. Therefore, we used the AA to turn the original infinite-dimensional optimization problem into an approximating one. However, the approximating optimization problem is a mixed-integer program that is neither linear nor convex, traditional optimization algorithms cannot be used to solve it. In order to solve the two-stage fuzzy programming problem, a hybrid algorithm which combines AA, NN and PSO, was designed. Finally, we provided one numerical example to illustrate the feasibility and effectiveness of the designed algorithm, and the computational results showed that the proposed hybrid algorithm is feasibility and effectiveness. Acknowledgments. The work was supported by the Research Project of Zhejiang Educational Commission Grant No.Y200909282.
References 1. Fabian, T., Fisher, J., Sasieni, W., Yardeni, A.: Purchasing Raw Material on A Fluctuating Market. Oper. Res. 7, 107–122 (1959) 2. Kingsman, B.G.: Purchasing Raw Materials with Uncertain Fluctuating Prices. Euro. J. Oper. Res. 25, 358–372 (1986) 3. Bonser, J.S., Wu, S.D.: Procurement Planning to Maintain Both Short-Term Adaptiveness and Long-Term Perspective. Manage. Sci. 47, 769–786 (2001) 4. Zadeh, L.A.: Fuzzy Sets. Information and Control 8, 338–353 (1965) 5. Dubois, D., Prade, H.: Possibility Theory. Plenum Press, New York (1988) 6. Lai, Y.J., Hwang, C.L.: Fuzzy Mathematical Programming: Methods and Applications. Springer, Berlin (1992) 7. Mula, J., Poler, R., Garcia-Sabater, J.P.: Material Requirement Planning with Fuzzy Constraints and Fuzzy Coefficients. Fuzzy Sets Syst. 158, 783–793 (2007) 8. Liu, B., Liu, Y.K.: Expected Value of Fuzzy Variable and Fuzzy Expected Value Models. IEEE Trans. Fuzzy Syst. 10, 445–450 (2002) 9. Liu, B.: Uncertainty Theory: An Introduction to Its Axiomatic Foundations. Springer, Berlin (2004) 10. Liu, Y.K.: Fuzzy Programming with Recource. Int. J. Uncertainty Fuzziness Knowl.-Based Syst. 13, 381–413 (2005) 11. Liu, Y.K.: Convergent Results about the Use of Fuzzy Simulation in Fuzzy Optimization Problems. IEEE Trans. Fuzzy Syst. 14, 295–304 (2006) 12. Liu, Y.K., Tian, M.: Convergence of Optimal Solutions about Approximation Scheme for Fuzzy Programming with Minimum-Risk Criteria. Comput. Math. Appl. 57, 867–884 (2009) 13. Sun, G.J., Liu, Y.K.: Fuzzy Minimum-Risk Material Procurement Planning Problem. In: 4th International Conference on Natural Computation, pp. 629–633. IEEE Press, Jinan (2009) 14. Sun, G.J., Liu, Y.K., Lan, Y.F.: Optimizing Material Procurement Planning Problem by Two-Stage Fuzzy Programming. Comput. Ind. Eng. 58, 97–107 (2010) 15. Liu, Y.K., Gao, J.: The Independence of Fuzzy Variables with Applications in Fuzzy Random Optimization. Int. J. Uncertainty Fuzziness Knowl.-Based Syst. 15, 1–20 (2007)
252
G.-J. Sun
16. Scarselli, F., Tsoi, A.C.: Universal Approximation Using Feedforward Neural Networks: A Survey of Some Existing Methods, and Some New Results. Neural Networks 11, 15–37 (1998) 17. Kennedy, J., Eberhat, R.C.: Particle Swarm Optimization. In: Proccedings of the IEEE International Conference on Neural Networks, Piscataway, NJ, pp. 1942–1948 (1995) 18. Kennedy, J., Eberhat, R.C., Shi, Y.: Swarm Intelligence. Morgan Kaufmann Publishers, San Francisco (2001)
Radial Basis Function Network for Endpoint Detection in Plasma Etch Process Shu-Kun Zhao, Min-Woo Kim, Yi-Seul Han, Se-Youn Jeon, Yun-Keun Lee, and Seung-Soo Han Digital Information Lab., 8507, Dept. of Information Engineering, Myongji University, San 28 Namdong, Yongin, Gyeonggido, 449-728, Korea [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]
Abstract. In the semiconductor manufacturing process, the endpoint of plasma etch process can be determined by the graphics based detection in order to avoid the loss of over-etching and under-etching. Our approach in current study can be conducted as one way to real-time monitor and judge the endpoint instead of observing it manually. When the endpoint occurs, this system can improve the etch processes and provide instant shutdown recommendations. This method makes use of Radial Basis Function (RBF) network’s functional approximation in time-series modeling and in pattern classification. By training with enough samples, the judge will be more accurate. All the samples are probed with optical emission spectroscopy (OES) sensor in real plasma etch process and for both network training and test. Keywords: Etch Process, OES, RBF, Neural Network, Endpoint Detection.
1 Introduction This study was on the plasma etching, which is widely used in the semiconductor manufacturing and the need for in-situ process monitoring of plasma etching is becoming greater as the technology advances. Extremely tight control of all process parameters must be maintained to increase throughput and reproducibility. The greatest need for plasma process monitoring arises in the determination of the etch endpoint for a given process, which can reduce the degree of overetching and underetching [3]. There are other techniques that are used to obtain information about reactive ion plasmas, such as ellipsometry, mass spectroscopy, Langmuir probe measurements, laser induced fluorescence, coherent anti-stokes spectroscopy, and infrared/visible region absorption spectroscopy, but these methods are primarily used for research since they are experimentally more demanding than the previous techniques. Etch is the process of selectively removing unneeded material from the wafer surface by using either chemical or physical means. The fundamental goal of etching is to accurately reproduce the mask features on the resist-coated wafer. Two basic types of etch processes are used in semiconductor manufacturing: dry etch and wet Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 253–263. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
254
S.-K. Zhao et al.
etch [5]. Dry etching is method that etching causing physical reaction in plasma state using gas. And wet etching using chemical reaction, angle of repose attains in order that chemical component and ingredient that wish to etch because material to be etched causes chemical reaction have melted. Etch rate is the speed at which material is removed from the wafer surface during etching (see Fig. 1)
Etch Rate =
ΔT t
(1)
Where:
ΔT = amount of material removed (Å or μm ) t = time elapsed during etch (typically minutes) ΔT ΔT
Fig. 1. Etch Rate
Observing formula (1.1), obviously, at the same situation, the time elapsed is a critical factor that influences the etching profile. For a typical etch process, the required film thickness is etched away and then the etch process continues until a specific over-etch time has arrived. The over-etch time must be long enough so that the etched layer has been removed at every position on the wafer surface, but must also be short enough so that the etch layer is not unacceptably eroded or the layer underlying the etch layer has not been etched [3]. Optical Emission Spectroscopy (OES) is used when monitoring etching process from optical signals into electronic signals. It is also one of the oldest and most popular methods of plasma etch monitoring. Fundamentally, OES is a bulk measure of the optical radiation of the plasma species. Since emissions can emanate from etch reactants as well as products, OES measurements are most often used to obtain the average optical intensity at a particular wavelength above the wafer [7]. While OES is an excellent tool for monitoring plasma emission intensity, a primary issue with its use is the large dimensionality of the spectroscopic data [2].
Fig. 2. General OES device. Light which come out in Plasma is frequency of very wide extent, but area that use in OES most 200 ~ 900nm.
Radial Basis Function Network for Endpoint Detection in Plasma Etch Process
255
Light flow, which comes out in plasma, needs monochromater to do dispersal and optical system and photodetector to intromit light. General OES device is as Fig.2 shown. Light which comes out in Plasma comes by spectroscope through direction or optical fiber. Then, photocurrent that is proportional in the century reaching to PM detector being spread, and only light which has decided conclusion of state examination passes aperture to be way out because diffraction occurs in spectroscope. OES that is used both number to detect change of characteristic spectrum by relative change of inside structure material. Therefore, spectrum of main material that is detected between etching detected endpoint by doing real time monitoring. As Fig.3 shown, the endpoint is the very point, after which the data will fall rapidly. It is marked with circle. As a result, use observed OES graph to trace the color changes on the wafer surface, and then can determine whether the end of the reaction. At the end of the etch process, the gas mixture again resembles its mixture prior to the start of the process. Therefore, the etch endpoint is characterized by a sharp change in the intensity of the endpoint trace.
Fig. 3. Plasma etch intensity (OES): above 4 figures are the intensity values – time graph. The normal line shows the original data, and the bold one shows the curve after filtering. (For details about the filtering, refer to the Methodology part)
Fig. 3 enumerates 4 samples, which were recorded in the real plasma etching experiments under the same condition. In this paper, we propose a novel approach to
256
S.-K. Zhao et al.
model the endpoint shape signature based on the Radial Basis Function Networks. Our approach can directly capture a process engineer’s knowledge about endpoint characteristics, for instance, the nature of the endpoint between polynomial segments, the shape of an example pattern, the expected location in time of the endpoint. This paper is structured as follows: in section 2, we explain the RBF network; in section 3, we provide the methodology details – introduction to network design, also including training data collection and choosing the target matrix; the simulation details and results when applying RBF networks are provided in section 4 and 5, while the conclusions of this study are drawn in section 6.
2 Radial Basis Function Network Radial Basis Function Network emerged as a variant of artificial neural network in late 80’s. However, their roots are entrenched in much older pattern recognition techniques as for example potential functions, clustering, functional approximation, spline interpolation and mixture models [1]. The construction of a RBF network in its most basic form involves three entirely different layers. The input layer is made up of source nodes (sensory units). The second layer is a hidden layer realizing radial basis functions with high enough dimensions, which serves a different purpose from that in a multilayer perceptron. The output layer supplies the response of the network to the activation patterns applied to the input layer. The transformation from the input space to the hidden-unit space is nonlinear, whereas the transformation from the hidden-unit space to the output space is linear. Given a set N different points of N real numbers {d i ∈ R
1
{xi ∈ R p | i = 1, 2,...., N } and a corresponding set
| i = 1, 2,...., N } , find a function F : R N → R1 that
satisfies the interpolation condition:
F ( xi ) = d i , i = 1, 2,......N
(2)
Note that for strict interpolation as specified here, the interpolating surface (i, e., function F) is constrained to pass through all the training data points. The radial basis functions technique consists of choosing a function F that has the following form (Powell M.J.D, 1985): N
F (x) = ∑ wi ⋅ ϕ ( x − xi i =1
Where
{ϕ ( x − x ) | i = 1, 2,..., N } i
)
(3)
is a set of N arbitrary (general nonlinear)
functions, known as radial basis functions, and i denotes a norm that is usually taken to be Euclidean. The known data points xi ∈ R p , i = 1, 2,..., N are taken to be the centers of the RBFs.
Radial Basis Function Network for Endpoint Detection in Plasma Etch Process
257
Inserting the interpolation conditions of (2),(3), we obtain the following set of simultaneous linear equations for the unknown coefficients (weight) of the expansion {w}:
⎡ ϕ11 ϕ12 .... ϕ1N ⎢ ⎢ ϕ21 ϕ22 .... ϕ2 N ⎢ ⎢ ⎣ ϕ N 1 ϕ N 2 .. ϕ NN
⎤ ⎥ ⎥ ⎥ ⎥ ⎦
⎡ w1 ⎤ ⎡ d1 ⎤ ⎢w ⎥ ⎢d ⎥ ⎢ 2⎥ = ⎢ 2⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ wN ⎦ ⎣ d N ⎦
(4)
where
ϕij = ϕ ( x j − xi ), j , i = 1, 2,.....N
(5)
Let
d = [d1 , d 2 ,.....d N ]T w = [ w1 , w2 ,.....wN ]T
(6)
The N-by-1 vectors d and w represent the desired response vector and linear weight vector, respectively. Let φ denote an N-by-N matrix with elements ϕij :
φ = {ϕij | j , i = 1, 2,....N }
(7)
We call this matrix the interpolation matrix. We may then rewrite (4) in the compact form
φw = x
(8)
There is a class of radial-basis functions that enjoys the following remarkable property(Light, 1992): Let x1 , x2 ,.....xN be distinct points in R p Then the N-by-N interpolation matrix
φ
whose jith element is
ϕij = ϕ ( x j − xi )
is positive definite.
This theorem is more powerful than a previous result due to Micchelli(1986) stating that the interpolation matrix φ is nonsingular. Light’s theorem applies to the following cases (among others): 1.
Inverse multiquadrics (Hardy, 1971)
ϕ (r ) =
1 for some c > 0, and, r ≥ 0 (r + c 2 )1/ 2 2
(9)
258
2.
S.-K. Zhao et al.
Gaussian functions
ϕ (r ) = exp(−
r2 ) for σ > 0 , and r ≥ 0 2σ 2
(10)
Theoretical investigations and practical results, however, seem to show that type of nonlinearity ϕ (.) is not crucial to the performance of RBF networks (Powell, 1988). Returning to the implication of Light’s theorem, we note that, provided that the data points are all distinct, the interpolation matrix φ is positive definite, and so we may solve (8) for the weight vector w, obtaining
w = φ −1d
(11)
where φ is the inverse of the interpolation matrix φ . Although in theory we are always assured a solution to the strict interpolation problem, in practice we cannot solve (8) when the matrix φ is arbitrarily closet to singular. At this point, −1
regularization theory can help by perturbing the matrix φ to φ + λ I , as we shall see in Regularization Theory. The approximation capabilities of RBF networks have been studied in [9, 11]. Due to their nonlinear approximation properties, RBF networks can build complicated mappings contrasted with multilayer perceptron [10]. Furthermore, a typical RBF network has only two weight layers and also has generalized simple learning algorithms, so the training speed is faster than multilayer perceptron. While modeling the network, it is different when approximating time-series and in pattern classification [4]. In the first case, the network inputs represent data samples at certain past time-laps, while the network has only one output representing a signal value as shown in Fig. 1. In a pattern classification application the inputs represent feature entries, while each output corresponds to a class as shown in Fig 2. The hidden units correspond to subclasses in this case [11].
3 Methodology To replace the traditional manual control to avoid bring about errors and improve performance. There are basically two ways to replace manual control: 1. Abstract and summarize from the experiences of manual control into general rules or fuzzy rules, and then construct the corresponding exclusive controller or fuzzy controller; 2. Use artificial neural network to study the behaviors of human control when it is difficult to express knowledge. That means, model the artificial control, and then take advantage of neural network controller instead of manual control. This study of traditional controller and then gradually replacing it with the neural network controller is called the neural network supervisory control. Neural network as a parallel computing model, take advantages of traditional modeling methods do not have. It also have a good ability of nonlinear mapping, in terms of that it is not necessary to ask for much empirical knowledge of modeled object. Generally, not have to foreknow about the structure parameters and dynamic
Radial Basis Function Network for Endpoint Detection in Plasma Etch Process
259
characteristics such as knowledge of objects. Instead just need the object inputs and outputs. The network itself will learn to achieve input and output mapping. First of all, observing the filtered data collection, shown in Fig. 3, can find that, if want to design a real-time monitoring system, we should filter the new probe data with previous ones after get it. After then, input the new filtered data together with a number of previous existing filtered data as input of RBF. Here, how many reference data needed, depends on the properties of the observed data: OES data fall rapid at the endpoint, and time is very short, approximately 0.5 ~ 1.0sec. The width of sample, the number of neurons in the RBF network input layer, will affect the network performance. This factor should be concerned. After input data into the RBF network, can get one output corresponds to the current data pattern and network condition. This output can be a scale number or a vector. Considering the complexity of the output in different cases, could design training data and it corresponding target output matrixes of the RBF network. Here we use a scale value as the target of each line of training input. Observing the sample data, in the rapid increase phase, mostly corresponds to the reaction beginning stages while plasma gas mixture. The training target is set at -1.0. During etch process, OES of intensity value trends in relatively flat shape. The training target is set at 0.0. And at the last of etch process, endpoint occurs, the observation value fall rapidly, while set the corresponding training target at 1.0. After then, the OES value preserves in relatively flat at lower.
Fig. 4. Concentration of blips: the samples mainly concentrated in high-value areas (about 17.0 ~ 26.0) and low-value area (about 0.0 ~ 7.0). Between them, scatters a few scattered points (besieged with smaller ellipse). The points are precisely the points corresponding to the rapid declining – endpoint.
The Fig. 4 shows that, according to the distribution of points can be considered concentrated in high-value areas and low-value areas have a similar distribution (similar to normal distribution), while in the rapid decline area, sampling points, show the distribution of uniform, and the slope of the fitted straight is very high (changes fast). Through the above generalization, we can design a system based on RBF neural network that can be suitable for real-time monitoring of plasma etch process endpoint detection.
260
S.-K. Zhao et al.
Fig. 5. System model. The 1st layer is a filter; the 2nd layer is a sequence restoring the current filtered data and a part of previous ones; the 3rd layer is a RBF network, which returns the judgment result.
Fig. 5 shows the design of the system model. The system consists of 3 layers: The first layer is a filter used to filter the system error brought by the new received data. Here we use a very simple mean filter. Namely, whenever receive a new data at time t, do the average of the measured data together with the previous n data. Then, take the mean value as the standard input data at time t. Here, call n the system of filter coefficients. If the total number of received data is less than n, average the input the data has been measured as the standard input data at time t.
Fig. 6. The mean filtering method
Next is the input layer. In this system, input layer is equivalent to a queue that has a certain length. The length of the queue is as same as the number of data which is used for training network. Actually, either training data or real data is standard input. Given the length of input l, when calculate the filtered value at time t, deliver the value to the next RBF network together with previous l – 1 filtered data. The last layer is one RBF network. RBF network use radbas function (MATLAB) as activation function:
radbas (n) = e− n
2
(12)
MATLAB supplies two functions that can be used to generate one RBF network: newrb and newrbe. Function newrbe is different from newrbe, newrbe establishes a RBF network with little error and rapidly according to the characteristics of input vectors. The functions have a same parameter SPREAD. Parameter SPREAD decides the accuracy of interpolation. With increasing SPREAD, RBF network adds more
Radial Basis Function Network for Endpoint Detection in Plasma Etch Process
261
neurons to adapt rapid changes in the curve. But a large number of neurons will take long time to train it and also might cause hard to computing. Therefore, it needs to try several SPREAD values in simulation. There are two sets of data will be input the system and validate whether the endpoint can be detected.
4 Simulation and Result Generate the Training Data Via the method introduced in Methodology, the data for training RBF network is generated. Table 1 shows the example of training data. Upper part is collected from the beginning of sample 1; and lower part is collected from the endpoint of sample 1. Table 1. Example of training data Input P matrix
Target
-3.083333333 -0.930555556 5.814814815 11.43055556 12.56111111 -0.930555556 5.814814815 11.43055556 12.56111111 18.12777778 5.814814815 11.43055556 12.56111111 18.12777778 21.88888889 11.43055556 12.56111111 18.12777778 21.88888889 21.57222222 12.56111111 18.12777778 21.88888889 21.57222222 18.22777778 18.12777778 21.88888889 21.57222222 18.22777778 17.87777778 21.88888889 21.57222222 18.22777778 17.87777778 17.31111111 21.57222222 18.22777778 17.87777778 17.31111111 17.07222222 25.45555556 24.05 22.54444444 20.05555556 17.62222222 24.05 22.54444444 20.05555556 17.62222222 13.8 22.54444444 20.05555556 17.62222222 13.8 10.4 20.05555556 17.62222222 13.8 10.4 10.4 6.316666667 17.62222222 13.8 10.4 6.316666667 5.4 13.8 10.4 6.316666667 5.4 4.688888889 10.4 6.316666667 5.4 4.688888889 2.833333333 6.316666667 5.4 4.688888889 2.833333333 3.272222222
-1 -1 -1 -1 0 0 0 0 0 0 1 1 1 1 1 0
After then, input data into the system. Follow is a part of simulation code. At each time-step, a OES value is delivered to the filter, then stored at input sequence. In final step, RBF network will judge whether current value is the endpoint. net = newrbe(P, T, 5.0); [ROW, COL] = size(test1); for i = 1 : ROW y = sim(net, (test1(i,:)')); disp(y(1,1)); if y(1,1) > 0.99999999 end end disp('ENDPOINT:'); disp(i);
break;
262
S.-K. Zhao et al.
The output will show the time-step of endpoint
Fig. 7. The detection result. In the graph, the circle besieges the area is the endpoint detected manually, and the straight line vertical to the time-step axis marks the endpoint detected by simulation result.
In some cases, if the parameter SPREAD is 1.0, the endpoint cannot be found. By increasing it to 5.0, after training, the endpoint can be detected. So if RBF is used in practice, the parameter SPREAD should be validated in several values. System Performance Evaluation If consider the strict real-time performance to the system, it is necessary to refer to the influences of filtering. That will be balanced by adjusting filtering coefficient. In current study, the filtering method will cause the detection a little delay to the endpoint, and the delay is decided by used filtering coefficient.
tdelay = α ⋅ coef filter
(13)
Where:
tdelay is the delay cause by filtering. α is the very factor reflect the relationship between delay and filtering coefficient. coef filter is the filtering coefficient used in system. And also one median RBF neural network with higher approximation capability [5] [8] will be considered and compared with current result.
5 Conclusion In this study we provide a specific application of the Radial Basis Function networks in semiconductor fabrication. Due to the marked properties, for instance, localization, functional approximation, interpolation, cluster modeling and quasi-orthogonality, RBF can be used in many applications. The feasibility of RBF network applications
Radial Basis Function Network for Endpoint Detection in Plasma Etch Process
263
into semiconductor equipment and production processes was verified. The endpoint can be detected accurately for the other two test samples. Acknowledgments. This work is financially supported by Ministry of Knowledge Economy (10031812-2008-11).
References 1. Tou, J.T., Gonzalez, R.C.: Pattern Recognition. Addison-Wesley, Reading (1974) 2. Hong, S.J., May, G.S., Park, D.C.: Neural network modeling of reactive ion etching using optical emission spectroscopy data. IEEE Transactions 16(4), 598–608 (2003) 3. Czebiniak, J.M.: End Point Detection of Plasma Etching Using Optical Methods. In: Annual Microelectronic Engineering Conference (May 2006) 4. Bors, A.G.: Introduction of the Radial Basis Function Network. University of York, UK 5. Quirk, M., Serda, J.: Semiconductor Manufacturing Technology. Pearson Education International Presses, ISBN 978-7-121-08944-2 6. Haykin, S.: Neural Networks – A Comprehensive Foundation. Prentice Hall, Englewood Cliffs (1998); ch. 7: Radial-Basis Function Networks 7. May, G.S., Spanos, C.J.: Fundamentals of Semiconductor Manufacturing and Process Control. IEEE, Wiley-Interscience 8. Bors, A.G., Pitas, I.: Median radial basis functions neural network. IEEE Trans. on Neural Networks (1996) 9. Park, J., Sandberg, J.W.: Universal approximation using radial basis functions network. Neural Computation 3 (1991) 10. Haykin, S.: Neural Networks: A comprehensive Foundation. Prentice Hall, Upper Saddle River (1994) 11. Poggio, T., Girosi, F.: Networks for approximation and learning. Proc. IEEE 78(9) (1990)
A Novel Cellular Neural Network and Its Applications in Motion Planning Yan Cao1, Feng Zhang1, Xuewu Wu1, Sheng Lu1, Yi Li1, Lei Sun2, and Shuai Li3 1
Artillery Academy, Hefei 230037, China Anhui Agriculture University, Hefei 230036, China 3 Institute of Intelligent Machines, Chinese Academy of Sciences, Hefei 230031, China [email protected] 2
Abstract. A Novel Cellular Neural Network (CNN) entitled the shortest path CNN (SP-CNN) is proposed in this paper. Compared with general CNN, it is distinguished in the network structure and neural dynamics. As a result of these distinctions, SP-CNN has a good performance in motion planning for mobile robots. By mapping environment information to parameters in this neural network, motion planning can be transformed to the state evolvement of SPCNN and the generated state represents information of the optimal path. The proposed method generates the best solution in static environments in real time. Extensive simulations about the above mentioned aspects demonstrate the effectiveness of the proposed approach. Keywords: CNN, SP-CNN, Mobile robot, Motion planning.
1 Introduction Motion planning is a crucial topic for mobile robot navigation. Mobile robots move from current position to the target position through motion planning. With the quick development of mobile robotics, it is required for some applications that mobile robots have the capability of planning their motions within dynamic environments or unknown environments in real time. This requirement is often made for exploration robots like planetary exploration rovers, field exploration robots, rescue robots, etc. In this context, a novel cellular neural network (CNN) entitled the shortest path CNN (SP-CNN) is proposed to solve the problem. The proposed method generates the optimal solution in real time not only in a static environment but also in an unknown environment according to its currently acquired map. Many studies have been performed in the motion planning field. Artificial potential field (APF) method [1], vector field histogram (VFH) method [2] and some other behavior based methods [3] are simple and have a low computational cost. In some simple environments, they may satisfy the requirement. However, APF, VFH and all the other behavior base motion planning approaches cannot generate the globally optimal solution. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 265–273. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
266
Y. Cao et al.
In a complex environment, the simple ‘attraction-repulsion’ mechanism of APF cannot result in satisfactory solution. It often drops into local minima or suffers from Oscillations in the presence of obstacles in such an environment [4].Various novel potential field methods, like Laplace potential field method [5], viscous fluid method [6], magnetic field method [7], electrostatic potential field method [8], dynamic field method [9], etc., are proposed to overcome the above mentioned shortcomings. These methods can generate a feasible path even in complex environments. However, they have a high computational cost and the generated path is not globally optimal. In recent twenty years, more and more researchers focus their attention on the study of neural network based motion planning approaches. Siemiatkowska B proposed a highly parallel cellular neural network based motion planning method. Experiments reveal that the proposed method can find a free path in unknown and varied environments [10]. R. Glasius and A. Komoda proposed a discrete neural network model and a continuous neural network model for motion planning [11] [12]. Yang and Meng proposed a new biological inspired neural network approach for motion planning [13][14]. It is more general and more powerful than the model of R. Glasius et al. However, all the previously mentioned neural networks based motion planning approaches cannot ensure the generated path is globally optimal if the related parameters are not properly chosen [15]. In this paper, a novel CNN entitled SP-CNN is proposed to solve the problem of real-time motion planning. Similar to other neural networks based motion planning, neurons in SP-CNN are also locally connected. Different from other model, SP-CNN has continuously changed neural state values. When SP-CNN becomes steady, the state value of each neuron is equal to the distance from the target. Thus, the state value of each neuron in SP-CNN has an excellent physical meaning. Heuristic information is applied to determine the initial state of SP-CNN to increase the efficiency. This paper is organized into four sections. In section 2, the model of SP-CNN is introduced and it is applied to motion planning problem. In section 3, the applications of SP-CNN for motion path in various environments are supported by computer simulations, which demonstrate the effectiveness of the proposed model. In section 4, several conclusions are drawn.
2 The SP-CNN Model 2.1 Neural Dynamics CNN was firstly presented in 1988 by Chua and Yang [16]. This kind of network is topologically organized with identical dynamical systems. The cells are only locally connected to its neighbors. Adjacent cells interact with each other directly while cells not in the immediate neighborhood interact indirectly through cells connected with them. This model has a simple nonlinear dynamics and is very well suitable for hardware implementations. The neurons in this network correspond to vertexes while connections between neighbor neurons correspond to edges in graph theory. Thus definitions in graph theory can be extended to SP-CNN. In the following part, some term coming from the graph theory will be used. In SP-CNN, there are three types of
A Novel Cellular Neural Network and Its Applications in Motion Planning
267
cellular neurons: target neurons, obstacle neurons and free neurons. Compared with original CNN, SP-CNN has different neural dynamics. This difference endows SPCNN with some new features. Each cellular neuron has a state x, an input u, and an output y. Here we always choose y=x, so y will not appear in the following part of this paper. The dynamics of the i-th neuron in SP-CNN is characterized as follows:
xi = − Axi + Af (ui ) Where
(1)
xi is the state of the i-th neuron. ui is the input of the i-th neuron. A is a
positive constant which determines the dynamic performance of the network. The larger the value of A, the more fast the response speed. However, if A is too large, high frequency noises in
ui results in severe consequence for the output of this
network system. The function
f (ui ) is defined as:
⎧G f (ui ) = ⎨ ⎩u i
ui ≥ G
(2)
otherwise
Where G is an enough large constant, which is larger than the distance between the target neurons and free neurons. The input of the i-th neuron the target neuron, else if it is the obstacle neuron,
ui is equal to zero if it is
ui is equal to G. Otherwise, it is
equal to the minimal value of the sum of the j-th neuron’s state and the distance between the i-th and the j-th neurons (here the j-th neuron is a neighbor of the i-th neuron).This can be expressed as follows: ⎧ ⎪⎪0 ui = ⎨G ⎪min{x + d } i− j ⎪⎩ j∈Ωi j
Where
target neurons obstacle neurons free neurons
(3)
d i − j is the weight of edge connecting the i-th and the j-th neurons. Ωi is the
neighbor set of the i-th neuron. Actually, the state of each neuron converges to a positive number finally. That is to say, when the time is enough large the expression of ui for free neurons can be expressed as follows:
ui = [xi1 + di−i1, xi 2 + di−i 2 ,..., xij + di−ij ,...]
−
−∞
=lim( ∑ (xij + di−ij )−n ) n→∞
ij∈Ωi
1 n
(4)
Where i1,i2,...,ij,…are all neighbor neurons of the i-th neuron. In equation (4), the input of a free neuron is expressed in the form of negative infinite-norm. According to the neural dynamic of SP-CNN, the input of target neurons is clamped to 0 and the input of obstacle neurons is clamped to G. The input of a free neuron is the output of a competitive layer. The input is the sum of the state of this
268
Y. Cao et al.
neighbor neuron and a bias. The bias is the weight of the corresponding edge in topology. The value of it can be chosen according to applications. For a free neuron, the first layer is a competitive layer while the second layer is a dynamic local dynamic layer. For SP-CNN the initial state of the i-th neuron is chosen as follows:
xi0 = random;
xi0 ∈ (-∞, +∞)
(5)
In a static environment, SP-CNN approaches steady gradually with the elapse of time. In practice, an index is required to determine whether the network is steady under certain accuracy level. Here we define an index entitled absolute rate of change (ARC) like this:
max { lim
for all i
dt → 0
xi (t + dt ) − xi (t ) } dt
(6)
ARC is an index revealing the difference between current states of the whole neural network and the stable states. It is not larger than the rate of change of any neuron in this network, so the change of the state value of any neuron is very small if ARC becomes very small and thus each neuron becomes steady. When ARC is enough small, we think the whole network has become steady. In practice, when the expression of (7) becomes true, SP-CNN is regarded to be steady.
ARC ≤ δ
(7)
Where δ is the accuracy level. In search algorithms, heuristic information is often adopted to improve the effectiveness. Generally the heuristic algorithm can find the best solution faster than Dijkstra’s algorithm if the heuristic information is properly defined. Heuristic information can also be used for a dynamic system. The system becomes steady in a short time if its initial state is close to the steady state. Thus, if we choose proper initial state according to the prior knowledge, the state which is very close to the steady state, time consumption is expected to reduce. In SP-CNN, heuristic information can be employed to accelerate the convergence. 2.2 Motion Planning through SP-CNN The purpose of motion planning is to find a free path connecting the target position and the start position and the generated path satisfy certain optimal condition. Motion planning is based on a navigation map. Different maps result in different paths. Here SP-CNN is the path planner, so we should transform the map to expressions which can be recognized by SP-CNN. The configuration space of the mobile robot is divided into grids. This way, we get a grid-based map. There are several types of grids in such a grid-based map as follows: Target grids: The grid identifies the positions of robots in configuration space. Obstacle grids: The grids in configuration space correspond to obstacles in workspace. This can be expressed in a set:
A Novel Cellular Neural Network and Its Applications in Motion Planning
269
Oc = {q ∈ C | R(q ) ∩ Ow ≠ ∅} Oc represents obstacle grids in configuration space; C represents all grids in configuration space; R ( q ) represents the positions occupied by the robot in workspace corresponding to the grid q in configuration space; Ow represents the Where
obstacle positions in workspace. Free grids: The grids in configuration space correspond to spaces except for obstacles in workspace. This can be expressed in a set:
Fc = C − Oc The grid based-map is also a kind a graph from the view of graph theory. The SPCNN used for motion planning is designed to have the same topological structure as the grid-based map. The weight of an edge (as mentioned in equation (3)) in SP-CNN is equal to that of its corresponding edge in the grid-based map. Thus, a vertex in the grid-based map identically corresponds to a neuron in SP-CNN. Two vertices in the grid-based map are adjacent if they are connected by an edge. Since the grid-based map is locally connected by edges, the corresponding neural network with the same topological structure is also locally connected. Until now, we get a SP-CNN with corresponds to the given grid-based map. That is to say, the transform from a gridbased map to a SP-CNN is realized. The output of SP-CNN is the state values of every neuron in the network. It carries information about the optimal path in the particular environment. In order to find the optimal path, it is necessary to carry out the inverse transformation from the state values of SP-CNN back to the path in the grid-based map. For the given i-th grid corresponding to the i-th neuron, the next grid (the j-th grid, which corresponds to the j-th neuron) is obtained by:
j = arg min{x j + di − j } j∈Ωi
Where
(8)
d i − j is the distance between the i-th grid and the j-th grid in the grid-based
map. It is equal to the distance between the i-th neuron and the j-th neuron in the corresponding SP-CNN. Ωi is the neighbor set of the i-th grid. Equation (8) means that the next grid is the current grid’s neighbor grid, the corresponding neuron of which is characterized by the minimal value of x j + d i − j . In steady state, the state values of neurons in SP-CNN indicate some important information. If the state value of certain neuron approaches 0, the corresponding grid must be a target grid. If the state value of certain neuron .is equal to G in steady state, there must be no feasible path from the corresponding grid to one of target grids. This grid may be an obstacle neuron. Otherwise it is a free neuron, all paths from which to target grids are obstructed by obstacle grids. This will be revealed in the following simulations. Compared with most of other neural network path planning methods [10]-[15], SPCNN has an important property: the steady states of SP-CNN are equal to the distance
270
Y. Cao et al.
to the target position. With this distinctive feature, the mobile robot can predict the distance from current grid to the target grid. Also SP-CNN is a type of parallel network. It has relatively low time consumption. In dynamic or unknown environments, the navigation map updates rapidly with the increase of environment information. Traditional off-line planning methods have bad performances in this situation. As to SP-CNN, the change of navigation map leads to the change of state values in SP-CNN and the states indicate the globally optimal motion based on the current map, so SP-CNN based motion planning performs successfully in unknown or dynamic environments, the map of which changes rapidly.
2
1
Fig. 1. The weight of edge in the2-D square Fig. 2. Motion planning in the U-shape grid-based map environment
3 Simulations To demonstrate the effectiveness of this proposed SP-CNN model in motion planning, simulations are conducted. Without losing generalization, the proposed approach is applied to a point mobile robot in a 2-D workspace. Of course, SP-CNN based motion planning is not confined to a point mobile robot in 2-D environments. For robots in higher dimensional environment with the consideration of the robot’s shape and its nonholomic constraints, only mapping from environment to neural networks is different from the simulations referred in this paper while the fundamental principle of the neural network dynamic is the same. In the following simulations, the environment is represented by 2-D grid-based maps. The grids are uniform cell squares. Eight grids adjacent to a grid is regarded as its neighbors and edges exist between them. The weight of each edge is equal to the Euclidean distance between the centers of the adjacent neighbor grids. As shown in Fig.1, the weight of edge is equal to 1 unit for grids in the same row or in the same column. For adjacent grids neither in the same row nor in the same column, the weight is Euclidean distance between the centers of the grids.
2 units. It is equal to the
3.1 U-Shape Environment Some methods encounter dead-lock in U-shape environments. The mobile robot drops into local minima and stops at that position instead of going to the target position. To
A Novel Cellular Neural Network and Its Applications in Motion Planning
271
demonstrate the effectiveness of SP-CNN, the proposed model is applied to solve the U-shape problem shown in Fig.1. The neural network is a 50x50 topologically ordered neuronal network. By choosing A = 10, G=2500, at time t=10s we obtain a globally optimal solution (blue solid points) from start (green solid point) to goal (red solid point) shown in Fig.2. After forming steady states, the distance from certain neuron to the target neuron is equal to its state value. The state value of the neuron corresponding to the start position is equal to 60.0416 finally. It is also equal to the distance from the start to the goal position. This can be calculated according to the path shown in Fig.2. The steady states of all neurons with the red solid point as the target neuron are shown in Fig.3. In this figure, we use pseudo color to display the state of each neuron. The deep the color is, the far the neuron is from the target neuron. For neurons with no free path to goal position, the state values are very large. In order to reflect the comparative state values of neurons, they are reduced in this figure without losing dimension difference between neurons. The state values of all neurons form a potential filed, the potential of which is equal to the distance to the target position. Some other neural networks based motion planning approaches also form a potential field when become steady. However, their potential has no precise physical meanings although they can reflect the distance qualitatively.
Fig. 3. The steady state values of all neurons in Fig. 4. Employing heuristic information V.S. the U-shape environment after value reduction no heuristic inforzmation
3.2 Employing Heuristic Information V.S. No Heuristic Information In this simulation, the environment map, the target position and neural network configuration are the same as those of the U-shape environment respectively as shown in Fig.3. In this simulation, we compared situation with prior knowledge with situation without prior knowledge. In concrete, for situation with prior knowledge, the initial state values are chosen as follows: for obstacle neurons, the initial state values are equal to G; for other neurons, the initial state values are equal to their Euclidean distance to the target. For situation without prior knowledge in this simulation, the initial state values are equal to zero. The y-axis in Fig.4 represents ARC as defined in expression (7).
272
Y. Cao et al.
In Fig.4, ARC of the situation with heuristic information drops more rapid than the situation without heuristic information and it takes less time for the former one to achieve steady states. Of course, heuristic information for SP-CNN is not confined to the Euclidean distance from certain neuron to the target neuron. Other heuristic information is also possible. To some extent, the evaluation of the adopted heuristic information depends on the application. The closer the heuristic information is to the final steady states, the lower the time consumption is. No matter what initial state values are, the final steady state for a particular environment are identical. This indicates that if the heuristic information is far from the final steady state values, it takes more time to reach the steady states indicating the shortest path for the network instead of achieving other state values. This is not the same as A* path planning approach for which inaccurate heuristic information may generate a false path.
Fig. 5. The generated path in a labyrinth environment
3.3 Labyrinth Environment The proposed SP-CNN model does not suffer from undesired local-minima even in complicated labyrinth environments as shown in Fig.5. The neural network is a 50 x 50 topologically ordered neuron network. By choosing A = 10, G=2500, at time t=20s we obtain a globally optimal solution (blue solid points) from start (green solid points) to goal (red solid point). The distance of 113.4975 between start and goal is derived according to the state value of the neuron corresponding to the start. From this figure we can see the proposed model find the shortest path in the labyrinth environment.
4 Conclusion In this paper, a novel CNN entitled SP-CNN is proposed. Each neuron in the neural network only has local connection with its neighbors and it has a competitive layer and dynamic layer composed two-layer architecture. Based on the topological structure of the grid-based map, the proposed model is applied to solve real-time motion planning problem in various environments. Theoretical analysis reveals that the model proposed by Willms and Yang [15] is the discrete form of SP-CNN by
A Novel Cellular Neural Network and Its Applications in Motion Planning
choosing A =
273
1 . With SP-CNN, motion planning in workspaces with different dt
traverse cost is able to be solved. Effectiveness of the proposed SP-CNN is illustrated by simulation results.
References 1. Li, S., Meng, M.Q.H.: SP-NN: a novel neural network approach for path planning. Inst. of Intell. Machines, Chinese Acad. of Sci., Hefei (2008) 2. Borenstein, J., Koren, Y.: The vector field histogram-fast obstacle avoidance for mobile robots. IEEE Transactions on Robotics and Automation 7, 278–288 (1991) 3. Murphy, R.: Introduction to AI robotics. MIT Press, Cambridge (2000) 4. Yoram, K., Johann, B.: Potential Field Methods and Their Inherent Limitations for Mobile Robot Navigation. In: Proceedings of the IEEE International Conference on Robot Automation, California, pp. 1398–1404 (1991) 5. Sato, K.: Deadlock-free motion planning using laplace potential field. Advanced Robotics 5, 449–461 (1993) 6. Louste, C., Liegeois, A.: Near optimal robust path planning for mobile robots: the viscous fluid method with friction. Journal of Intelligent and Robotic Systems 27, 99–112 (2000) 7. Singh, L., Wen, J., Stephanou, H.: Motion planning and dynamic control of a linked manipulator using modified magnetic field. In: Proc. IEEE International Conference on Robotics and Automation, Albuquerque, NM, pp. 1142–1147 (1997) 8. Tsourveloudis, N.C., Valavanis, K.P., Hebert, T.: Autonomous vehicle navigation utilizing electrostatic potential fields and fuzzy logic. IEEE Transactions on Robotics and Automation 17, 490–497 (2001) 9. Wong, B., Spetsakis, M.: Scene reconstruction and robot navigation using dynamic fields. Autonomous Robots 8, 71–86 (2000) 10. Siemiatkowska, B.: Cellular Neural Network for Mobile Robot Navigation. In: Proceedings of the Third IEEE International Workshop on Cellular Neural Networks and their Applications, pp. 285–290 (1994) 11. Glasius, R., Komoda, A., Gielen, S.C.A.M.: Neural network dynamics for path planning and obstacle avoidance. Neural Networks 8, 125–133 (1995) 12. Glasius, R., Komoda, A., Gielen, S.C.A.M.: A biologically inspired neural net for trajectory formation and obstacle avoidance. Biological Cybernetics 74, 511–520 (1996) 13. Yang, S.X., Meng, M.: Neural network approaches to dynamic collision-free trajectory generation. IEEE Trans. on Systems, Man, and Cybernetics, Part B 31, 302–318 (2001) 14. Yang, S.X., Meng, M.: An efficient neural network method for real-time motion planning with safety consideration. Robot. Auton. Syst. 32, 115–128 (2000) 15. Willms, R., Yang, S.X.: An efficient dynamic system for real-time robot-path planning. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 36, 755–766 (2006) 16. Chua, L.O., Yang, L.: Cellular neural networks: theory and applications. IEEE Trans. on Circuits and Systems 35, 1257–1290 (1988)
Evaluation of Enterprise ERP System Based on Neural Network Optimized by Ant Colony Hanmei Wang, Dongxiao Niu, and Chengkai Cai Economics and Management, North China Electric Power University Beijing 102206, China [email protected]
Abstract. Based on the basic theory of ERP (Enterprise Resource Planning), this paper summarized a series of indicators which reflected the effect of the ERP system. Then we used BP neural network and ant colony algorithm together to evaluate the effect of the ERP system. After we get the scores of the evaluation of the subsystems, we can find which subsystem is the weakest, so we can improve that subsystem and make the whole system better. The empirical results show that the ant colony network not only has the extensive mapping capability as the neural network, but also has the high-speed and global convergence features as the ant colony algorithm. When we use this model, we can get better accuracy and efficiency. Keywords: ERP system, Ant Colony Algorithm, BP neural network, evaluation.
1 Introduction The ERP system was from America and Europe countries which have the modern industrial manufacturing methods. After 50 years of continuous application, development and innovation, this system becomes a useful method to manage the company. We are in an information age, in order to beat competitors, companies must use advanced manage and technical methods to obtain and utilize information. The ERP system is a useful method which can provide and utilize information. The ERP system becomes more and more popular in modern companies. Many articles focus the methods of evaluating the ERP system. Comprehensive evaluation methods which we commonly use include composite index evaluation method, fuzzy comprehensive evaluation method, multivariate statistical analysis methods, DEA method and so on. With the rise of evolutionary computation study, more and more scholars use evolutionary computation and BP neural network together to solve the evaluation questions. Evolutionary computation has strong global convergence ability and robustness, when we use these two ways together, this new method not only has the extensive mapping capability, but also get more fast convergence speed and more strong learning ability. Ant colony algorithm is a kind of random search optimization algorithm which simulates the behavior of the ant colony. Ant colony algorithm has strong robustness, convergence and parallelism and so on. Therefore, ant colony algorithm can also optimize the neural network [1]. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 275–282. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
276
H. Wang, D. Niu, and C. Cai
Based on the basic theory of ERP, this paper establishes the evaluation system of the ERP system at first. In order to make the results more accurate, we use BP neural network and ant colony algorithm together to train the weights of network. After training, we use this neural network to evaluate an enterprise’s ERP system. The result indicates that this neural network has better accuracy and efficiency.
2 Evaluation Indicators System 2.1 The Concept of ERP In the concept of ERP, we define that the living environment of the enterprise is a supply chain which includes suppliers, enterprises, distribution network and customers which work together. And within the enterprise, the whole system was divided into several subsystems which work together, such as finance department, marketing department, manufacturing department, services and maintenance department, engineering and technical department. In all links, in order to manage and utilize logistics, information flows, capital flows, workflows, the ERP system makes the full use of modern information technology to build information network system. Through continuous improvement of process optimization and reorganization, the ERP can get more accurately date and more availably decision-making behaviors. These results can improve the level of management and final improve economic efficiency and competitiveness of enterprises [2]. 2.2 Determine the Evaluation Indicators System of the ERP System The ERP system in every enterprise has many subsystems; we can get the operational information and application information through the evaluation of each subsystem, then we will know which subsystem is the worst and which part is needed to be improved. We analyze the characteristics of ERP systems, building twelve indicators of decision evaluation to each subsystem. The indicators are as follows: (1) reliability of the system; (2) scalability of the system; (3) maintainability of the System; (4) security confidentiality of the system; (5) advanced level of the system; (6) economy level of the system; (7) sharing capacity of the system; (8) availability of the system; (9) system-friendly; (10) system user satisfaction; (11) friendship degree of the system; (12) effectiveness of the system. During the evaluation process, these twelve indicators are not independent; they have some inner-relationships, we must consider them in whole [3].
3 Ant Colony Neural Network 3.1 Ant Colony Algorithm Ant colony algorithm (ACA) is a bionic optimization algorithm which can simulate the foraging behavior of ant communities. This algorithm uses a parallel self-catalytic mechanism of positive feedback, and has strong robustness, good distributed computer system, and is easy to combine with other algorithms. It has demonstrated its excellence in the aspects of complex optimization and intelligent search.
Evaluation of Enterprise ERP System Based on Neural Network Optimized
277
The basic idea of the algorithm is: Assume that the number of parameter “θ” in the neural network is n. First of all, sort the parameters, set them as “θ1, θ2, ..., θn”. As for the parameter “θi” (1 ≤ i ≤ n) , set it as some random values whose number is “N”, and then the collection “Iiθ” is established. Each ant starts from the first collection, according to the state of the pheromone of each element, randomly selects an element from each collection “Iiθ”, and accordingly adjusts the pheromone of the selected elements. When the ants complete all the choices of the elements of all collections, it reaches the food source. That is, they have selected a set of network parameters in all the collections. Next the ants will return to the nest along the original path. At the same time, the ants will adjust the pheromone of the selected elements of the selected elements. Repeat the above steps. When all the ants converge to the same path, or achieve a given iteration number, the search will end, which means that the optimal solution of the network parameters has been found. The main steps of the ant gray forecasting model are as follows: Step 1: Initialization: consider the pheromone of the element “j” in collection “Iiθ”(1 ≤ i ≤ n-1) as “τj (Iiθ)(t)”. Make τj (Iiθ)(t) =C (1 ≤ j ≤ N) and the largest cycles is NCmax, and the number of ants is m. All the ants are in the nest. Step 2: Start all the ants. Starting from the first Collection, each ant chooses an element in turn from each Collection in accordance with the following rules. In the end, the selected elements can be considered as a set of network parameters. Path selection rules: as for the collection “Iiθ”, consider each ant as “k” (k =1, 2, …, m). Each ant selects its element “j” randomly according to the random probability in following formula. This process will be end until all the ants reach food source.
Pr ob(τ k j ( Iθ i )) = [τ j ( Iθ i )] /
N ∑
τ u ( Iθ i )
u =1
(1)
Step 3: When all the ants have selected an element in each collection and returned to the nest along the original path, the pheromone of the selected elements will be adjusted accordingly according to the following formula. Consider the number of time units that the process have cost as “n”.
τ j ( Iθ i )(t + n) = ρτ j ( Iθ i )(t ) + Δτ j ( Iθ i )
(2)
In the above formula, the parameter “ ρ ” (0 ≤ ρ <1) stands for the persistent of pheromone, then “1- ρ ” stands for the elapsing degree of pheromone. Δτ j ( I θ i ) =
m ∑
k =1
Δτ k j ( I θ i )
(3)
Δτ k j ( Iθ i ) means the pheromone that the ant of number “k” leaves in the element “j” of collection Iiθ. k
j (I i )
Q/e
If the ant K select in this cycle
0
Otherwise
(4)
278
H. Wang, D. Niu, and C. Cai
In the above formula: “Q” is a constant which can adjust the adjusting speed of the pheromone; “e” is the maximum output error of all training samples. It can be defined as e =
n
∑
k =1
( xk − xk′ ) 2 .
“n” is the number of samples that the ants collected. “ xk ” and “ xk′ ”are the actual output and expected output of the neural network. Thus, the smaller the error is, the more the corresponding pheromone increases. Step 4: Repeat the above steps, when all the ants converge to the same path, or the number of cycles meet the follow conditions, the search will end. 3.2 Ant Colony Neural Network Model
Consider the information of the evaluation indicators of the ERP system as the neural network input vector “X = (X1, X2,…, Xn)”; consider the results that can represent the comprehensive evaluation as the neural network output “Y”; establish a three-tier neural network. Train this network with a sufficient number of samples, and train the network weights through the ant colony algorithm. Therefore, we can find a set of optimal weights. Eventually this network can be seen as an effective assessment tool for an ERP system, and can do corresponding comprehensive evaluation of different objects. The model diagram is shown in Figure 1.
1
x1
11
x2 xn
Output layer
1
Input layer
Y
21
2
Hidden layer
Fig. 1. The topology structure of BP neural network
They are as follows: 1) Input layer: input evaluation indicators of the enterprise ERP system, that is, X = (X1, X2,…, Xn) T; the node “I” of the input layer will pass information directly to the hidden layer nodes, so the output and input are equal [4]. 2) The input of the hidden layer is netj =
n
∑
i=0
vijxi
The output of the hidden layer is yj = f (netj ) In the above formula, “vij” is the weight of input layer node “i” and the hidden layer node “j”, j = 1,2,.., m; i =1, 2,…, n. “v0j”is the threshold of the node “j”; “xi” is the input of the node “i” of the output layer, and bring in the threshold with “x0 = -1”. “f(x)” is the unipolar sigmoid function.
Evaluation of Enterprise ERP System Based on Neural Network Optimized
f(x) = 1/(1+exp(-λx) )
279
(5)
3) The output layer has only one node. The input is net =
m ∑
j =0
wjyj . The output is o = f (net ) .
In the above formula, “wj” is the weight between the hidden layer node “j” and the output layer node “j”; “wo” is the threshold of the node “j”; “yj” is the output of the node “i” of the hidden layer, and bring in the threshold with “yo = - 1”. “f (x)” is the unipolar sigmoid function. f (x) = 1 / (1 + exp (-λx) )
(6)
4 Empirical Analysis 4.1 Evaluation Criteria of Enterprise ERP System
The evaluation indicators of enterprise ERP are just the twelve indicators that were shown in chapter two: (1) reliability of the system; (2) scalability of the system; (3) maintainability of the System; (4) security confidentiality of the system; (5) advanced level of the system; (6) economy level of the system; (7) sharing capacity of the system; (8) availability of the system; (9) system-friendly; (10) system user satisfaction; (11) friendship degree of the system; (12) effectiveness of the system. The criteria of the evaluation indicators are shown in Table 1. Table 1. Evaluation criteria
Ⅰ
Evaluation Criteria Class t1 85 t2 95 t3 96 t4 100 t5 100 t6 96 t7 95 t8 92 t9 85 t10 95 t11 92 t12 98
Ⅱ
Class 86 85 80 90 83 93 90 88 80 90 88 90
Ⅲ
Class 76 69 78 69 80 68 80 75 70 72 70 80
Ⅳ
Class 69 62 62 68 61 68 73 70 68 68 65 70
Ⅴ
Class 56 53 52 59 56 58 60 65 62 60 60 60
4.2 Preprocessing of Raw Data
Raw data will be transformed into [-1, 1] interval value through the following formula. xmid = The desired target:
x max + x min xi − xmid , xˆ i = 1 2 ( x max − x min) 2
(7)
280
H. Wang, D. Niu, and C. Cai
Ⅰ
The desired target of more than class standards of training samples and testing samples is the value between 0.8 and 1; the desired target of class and class of training samples and test samples is the value between 0.6 and 0.8, and so on. The network output ranges that determine the class , class , class , class and class are: 0.80 1; 0.6 0. 8; 0.4 0. 6; 0.2 0.4; 0 0.2. After pretreatment, the criteria of the evaluation indicators are shown in Table 2.
Ⅴ
~
~
~
~
Ⅰ
~
Ⅰ Ⅲ
Ⅱ
Ⅱ Ⅳ
Table 2. Evaluation criteria after pretreatment
Ⅰ
Evaluation Criteria Class t1 0.85 t2 0.95 t3 0.96 t4 1 t5 1 t6 0.96 t7 0.95 t8 0.92 t9 0.85 t10 0.95 t11 0.92 t12 0.98
Ⅱ
Class 0.86 0.85 0.80 0.90 0.83 0.93 0.90 0.88 0.80 0.90 0.88 0.90
Ⅲ
Class 0.76 0.69 0.78 0.69 0.80 0.68 0.80 0.75 0.70 0.72 0.70 0.80
Ⅳ
Class 0.69 0.62 0.62 0.68 0.61 0.68 0.73 0.70 0.68 0.68 0.65 0.70
Ⅴ
Class 0.56 0.53 0.52 0.59 0.56 0.58 0.60 0.65 0.62 0.60 0.60 0.60
4.3 Design of Ant Colony Neural Network and Analysis of Experimental Results
1)Build the neural network based on ant colony: First of all, generate five hundred samples through a rigorous mathematical model according to the evaluation criterions, and there are one hundred samples between two evaluation criterions. Select four hundred and fifty samples randomly as training samples, and use the remaining fifty samples to test the neural network [5]. Network structure: the number of the network input layer neurons depends on the number of the evaluation indicators, which can be set to twelve. The nodes of the hidden layer neurons can be set to ten, and the nodes of the output layer is one, which is the assessment result. First, BP algorithm is used for training. The learning step is five, and the weight convergence factor is 0.0001, and the error convergence of factor is 0.00001. The activation function of the hidden layer and output layer is the nonlinear sigmoid function. The number of the network connection weights is m = 75, and ant colony algorithm parameters ρ = 0.7, S = 30, Q = 100. The neural network parameter Pi is a random number among – 10 and 10. “N” can be taken as 40. 2)Test results and analysis: Take an ERP system of a provincial coal industry as an example, and conduct a comprehensive evaluation of its various subsystems, and further show the application of the ant colony neural network model. The ERP system of the enterprise consists of seven subsystems. Do the network training and testing with the normalized raw data.
Evaluation of Enterprise ERP System Based on Neural Network Optimized
281
Now the trained network can be used in seven subsystems, and the obtained network output and the evaluation results can be shown in Table 3. Table 3. Network output and the evaluation results Criteria t1 t2 t3 t4 t5 t6 t7 t8 t9 t10 t11 t12 expected output network output class
1 0.78 0.68 0.79 0.71 0.73 0.79 0.76 0.73 0.70 0.75 0.68 0.77 0.54 0.5375
Ⅲ
Samples to Be Evaluated 2 3 4 0.94 0.71 0.80 0.86 0.67 0.74 0.99 0.81 0.80 0.94 0.75 0.72 0.95 0.65 0.76 0.97 0.69 0.67 0.97 0.72 0.65 0.96 0.72 0.78 0.82 0.70 72 0.94 0.66 0.72 0.90 0.60 0.73 0.95 0.68 0.78 0.98 0.33 0.58 0.9833 0.3421 0.5804
Ⅰ
Ⅳ
Ⅲ
5 0.76 0.85 0.91 0.92 0.86 0.78 0.89 0.77 0.72 0.70 0.62 0.78 0.72 0.7183
Ⅱ
6 0.50 0.35 0.58 0.51 0.61 0.49 0.47 0.62 0.60 0.62 0.55 0.50 0.18 0.1792
Ⅴ
Fig. 2. Error comparison
From the above figure, we can see that the output curve of ant colony neural network is almost coincided with the desired output curve, and the error curve is nearly at zero scale line. These results show that the neural network based on ant colony that used in ERP system evaluation has higher accuracy, speed and network convergence [5]. If train the network with traditional BP algorithm, to achieve a similar accuracy, it requires about 10,000 times. Through the test (shown in Table 4), we can see that the number of ant neural network learning is far less than the number of BP neural network, and it has higher convergence speed and computing efficiency.
282
H. Wang, D. Niu, and C. Cai Table 4. Comparison of two methods
Methods BP Neural Network Ant Colony Neural Network
Learning times 30000 4200
Run Time Accuracy Probability 124 seconds 97.38% 60 seconds 97.59%
5 Conclusion This paper used ant colony neural network to evaluate ERP system. We built a model, and evaluated a provincial coal enterprise’s ERP system which has seven subsystems. After the empirical analysis, we can get the evaluation scores and classes of the subsystems and find that the effect of subsystem 6 is the weakest, so we should improve that subsystem and make the whole system better. In the end, we contrast BP neural network and ant colony neural network; the results show that ant colony neural network is better than BP neural network. So ant colony neural network is a more feasible way when we evaluate ERP system. Using this model, we can reduce the evaluation workload and improve the rationality of evaluation results. In a word, this method can be promoted. Acknowledgments. Thank the predecessors whose research results pointed out the direction for our paper and provided us with a wealth of reference materials. Meanwhile, thank the teachers and friends who gave a lot of guidance and help for our paper writing.
References 1. Jie, W., Guozhong, W., Li Yanchang, M.S.: Ant combination of gray neural network model in power load forecasting. J. Power System Protection and Control 137, 48–52 (2009) 2. Xianfeng, J., Xiangtong, Z., Zhu Jinghua, M.S.: Evaluation of company ERP system based on genetic nervous network. J. China Coal. 133, 15–18 (2007) 3. Li Jinyang, M.S.: Studying on the ERP Implementation Impact by BP NN Evaluation. J. Science Mosaic, 25–27 (2008) 4. Chuanxu, C., Liu Wenzhe, M.S.: Enterprise’s ERP Executive Risk Appraisal Neural Network Model. J. Journal of Xi’an Aerotechnical College 126, 52–55 (2008) 5. Shan, Z., He Jiannong, M.S.: Research on Optimized Algorithm for BP Neural Networks. J. Computer and Modernization 161, 73–75 (2009)
Visual Attention-Based Ship Detection in SAR Images Ying Yu1, Zhenghu Ding1, Bin Wang1,2, and Liming Zhang1 1
2
Department of Electronic Engineering, Fudan University, Shanghai 200433, P.R. China The Key Lab of Wave Scattering and Remote Sensing Information (Ministry of Education), Fudan University, Shanghai 200433, P.R. China [email protected], {zhding,wangbin,lmzhang}@fudan.edu.cn
Abstract. Satellite-based synthetic aperture radar (SAR) is a powerful tool for ship detection because it can work in all weather conditions, day and night. However, speckles and heterogeneous regions in SAR images pose great challenges on automatic detection of ships. This paper introduces a bottom-up visual attention model and proposes a visual attention based method for ship detection in SAR images. The proposed method is very simple and fast in computation, and has powerful capability of targets detection. The analysis of the detection performance over both simulated and real images confirms the robustness of the proposed algorithm. Keywords: Ship detection, Synthetic aperture radar (SAR), Visual attention, Saliency map, Pulsed cosine transform.
1 Introduction Ship detection by radars has been used to monitor fishing vessels, oil pollution, and traffic activities in the ocean. It is believed that the synthetic aperture radar (SAR) can provide useful information for ship detection because SAR sensors can work in nearly all weather conditions, day or night [1]. Ship signature in SAR images is determined by many scattering mechanisms including direct reflection from areas vertical to the radar beam, corner reflections and multiple reflections from the ship and sea surface [2]. Usually, two challenges exist with ship detection in SAR images. First, the coherence of backscattered signal can cause bright spots over SAR images. Second, the heterogeneities of SAR images caused by different wind conditions, current boundaries, backscattering alteration due to the presence of ship wakes, oil slicks, etc., pose extra challenges on ship detection [3]. The conventional method of ship detection uses constant false alarm rate (CFAR) in which the threshold value automatically adapts to the statistical distribution of the signal in a detection window (e.g., [3][4]). The main disadvantage of CFAR is that if the signature of a ship has a similar intensity level as its surroundings, the extraction of ships becomes very difficult. Other approaches use cross correlation of subapertures [5] or explore a wavelet-based technique [6]. Polarimetric whitening filter (PWF) can optimally combine the elements of scattering matrix to reduce speckles in polarimetric SAR [7]. Polarization cross-entropy is proposed based on the eigendecomposition of the polarimetric coherence matrix [8]. However, these approaches Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 283–292. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
284
Y. Yu et al.
cannot be used in single polarization SAR data. An exhaustive description of the ship detection algorithms proposed in literature can be found in [1]. In human visual system, there exists a bottom-up attention selection mechanism that can guide our eyes rapidly gaze towards salient objects in a clustered scene [9][10]. A lot of recent papers (e.g., [11]-[13]) have investigated the computational model of bottom-up visual attention. It has been noted that the human eye is superior to existing automatic algorithms in observing a slick in the context of the surrounding sea and, surprisingly, some vessels undetected by conventional techniques are visible by eye [6][14]. Inspired by this rationale, this paper designs a simple and very effective ship detection scheme based on human visual attention. Experiments accomplished over full-resolution SAR images show the effectiveness of this new approach. The rest of this paper is organized as follows. Section 2 describes the visual attention based approach for ship detection in SAR images. Section 3 presents the experiments, where our method is tested over simulated and real data. Finally, section 5 concludes the whole paper.
2 Visual Attention-Based Ship Detection It is believed that bottom-up visual attention acts like a “spotlight” that can rapidly shift across the entire visual field and selects a small area from the entire visual scene [9][10]. Only the attended part of input sensory information is allowed to reach shortterm memory and visual awareness [15][16]. Based on the neurobiological studies in human visual system, some researchers in fields of electronic engineering or computer intelligence have attempted to find a computational model to implement the human-like visual attention system. Itti et al. [11] proposed a biologically plausible model of bottom-up attention selection. The model implements and expands the generation mechanism of attention selection proposed by Koch and Ullman [10]. Different from Itti’s model, Fourier transform (FT) based approaches have been recently proposed [13][17][18]. The gist of these FT-based approaches is that the operation of flattening the amplitude spectrum is almost equal to the center-surround operation in Itti’s spatial domain model. These approaches can detect saliency areas precisely with less computational cost. After that, Yu et al. [19][20] noted that Hebbian-based neural networks can implement bottom-up attention selection, and
Fig. 1. An example of the attention model to compute the saliency map of a natural scene. Left: natural scene. Right: saliency map.
Visual Attention-Based Ship Detection in SAR Images
285
proposed the pulsed principal component analysis (P2CA) transform and the pulsed cosine transform (PCT). Fig. 1 gives an example of the PCT model to compute the saliency map of a natural scene. The saliency map provides the information that the bright areas attract more visual attention than dark areas. The following subsections describe our visual attention-based technique for ship detection in SAR images. We employ the PCT model to generate a saliency map due to its good performance in eye fixation prediction and fast speed in computation. 2.1 Preprocessing More often than not, ship signatures in SAR images are fairly bright because of backscattering coherence from the ships. So, we can set a threshold to remove some speckles in a preprocessing stage. Given the SAR image M, the threshold can be calculated as follows:
TF = μ SAR + λ ⋅ σ SAR ,
(1)
where μSAR and σSAR are the image’s mean and its standard deviation, respectively, and the parameter λ determines the extent to which the speckles will be removed. Note that the larger λ is, the more speckles can be masked out but at risk of removing the ship signatures. In this paper, the parameter λ is set to 0.8, which guarantees the ship signature will out be masked out in the preprocessing stage. After the threshold value TF is obtained, the flood image is computed by:
⎧M ( x, y ), when M ( x, y ) ≥ TF . when M ( x, y ) < TF ⎩ TF ,
M Flood ( x, y ) = ⎨
(2)
Using the flood image as input, our attention model can obtain a better saliency map because a part of speckles have been “submerged” in this procedure. 2.2 The Attention Model This subsection presents a PCT-based attention model that can produce the bottom-up saliency map of an input SAR image. Within the saliency map, ship signatures can be easily detected. Given the flood image MFlood, the computational procedure to calculate the bottom-up saliency map is as follows:
P = sign(C (M Flood )) , F = C −1 (P) ,
(3) (4)
where C and C–1 denote the two dimensional discrete cosine transform [21] and its inverse transform respectively. The notation “sign(.)” is the signum function. Note that F is set to zero at locations with negative values. It carries the saliency information of the flood image. Then, a smooth and nice-looking saliency map is computed by:
286
Y. Yu et al.
M Saliency = G ∗ F 2 ,
(5)
where G is a two dimensional Gaussian low-pass filter. Eq. (3) is called pulsed cosine transform (PCT) in [19]. Note that it only retains the signs of the DCT coefficients and discards the amplitude information. This procedure mimics the lateral surround suppression among cortical cells tuned to similar features. So, it can generate the saliency information of an input scene. A detailed description and discussion about this model has been shown in [19] and [20]. Note that the saliency map experiences a gray scale normalization process before the detection stage. 2.3 Ship Detection After the saliency map is obtained, a threshold level could be fixed to discriminate ships from their background. However, since a threshold value assigned suitable in any case can not be considered, we need to find a calculation scheme of this threshold, which is suitable in all the SAR images and provides a correct detection with no false alarms. In order to detect the ships automatically, such a detection threshold level can be easily calculated as:
TD = μ SM + c ⋅ σ SM .
(6)
where μSM and σSM are the saliency map’s mean and its standard deviation respectively, and c is a parameter empirically adjusted. Finally, a ship detection map is computed by:
⎧1, when M Saliency ( x, y ) ≥ TD . ⎪⎩0, when M Saliency ( x, y ) < TD
D( x, y ) = ⎪⎨
(7)
Usually, saliency maps with ship signatures have small deviation, whereas saliency maps without any ship signature have large deviation. So, given a proper parameter c and calculated by Eq. (6), the detection threshold for a ship-free saliency map can become large enough to avoid false alarms. In this paper, the c is empirically set to 8. 2.4 Algorithm Summary To sum up, the proposed visual attention-based ship detection algorithm for SAR images comprises the following three stages: 1) Generating the flood image: Given a SAR image M, we calculate a threshold value TF using Eq. (1), and calculate the flood image MFlood using Eq. (2). 2) Generating the Saliency map: After the flood image is obtained, we calculate its saliency map MSaliency using Eqs. (3)-(5). 3) Detecting ship signatures: After the saliency map is obtained, we calculate a detection threshold value TD using Eq. (6), and finally accomplish the ship detection using Eq. (7).
Visual Attention-Based Ship Detection in SAR Images
287
3 Experimental Results This section presents the experiments accomplished over simulated and real SAR data. A set of meaningful experimental results are shown and discussed to demonstrate the effectiveness of the proposed method for ship detection. For comparison, we give the detection results by a conventional CFAR method of which the false alarm rate is set to 0.5%. For our method, the λ in Eq. (1) is set to 0.8 and the c in Eq. (6) is set to 8. Note that some papers have used a contrast parameter, called the significance, to measure the capability of target enhancement (e.g., [6][14][22]). In this paper, we also employ this criterion to measure our saliency map’s capability of enhancing the targets. Similar to the definition in other literatures, the significance is defined as: Significance =
θT − μ B . σB
(8)
where μB and σB are the background’s mean and its standard deviation, respectively, and T is the peak value of the target.
θ
Exp. 1: In the first test, our method is tested over simulated data. Visible ship images are deliberately embedded in speckles by lowering the image intensity to a similar level as its surroundings. Fig. 2(a) and (b) illustrate the ground truth of three ships and the simulated SAR image (400×280 pixels). Fig. 2(c) gives the detection result by CFAR. As can be seen, CFAR method has some false alarms. Fig. 2(d) and (e) give the saliency map and the detection result of our method respectively. As can be seen, our method can highlight all three ships in the saliency map and detect them successfully. As compared with the SAR image, the significance of the saliency map has largely increased, which means that the targets pop out from the background. Exp. 2: In the second test, our method is tested over a set of ALOS HH polarized Lband SAR images. Unfortunately, we cannot provide simultaneous ground truths at present, and the referred targets are visually interpreted from the SAR images. The test results are shown in Fig 3, 4, 5 and 6. Fig. 3(a) concerns a SAR image (400×400 pixels) in which five ships are present (pointed by the arrows). Fig. 3(b) gives the detection result of conventional CFAR.
(a)
(b)
(c)
(d)
(e)
Fig. 2. Test on simulated data. (a) Ground truth. (b) Simulated SAR data (Significance = 1.54). (c) Detection result of CFAR. (d) Saliency map generated by our attention model (Significance = 15.16). (e) Detection result of our method.
288
Y. Yu et al.
(a)
(b)
(c)
(d)
Fig. 3. Test on SAR data. (a) ALOS SAR (Significance = 9.68). (b) Detection result of CFAR. (c) Saliency map generated by our attention model (Significance = 87.83). (d) Detection result of our method.
Fig. 3(c) and (d) give the saliency map and the detection result of our method respectively. As can be seen, our method can find all five ships successfully while the CFAR method causes many false alarms. Fig. 4(a) shows a SAR image (280×400 pixels) in which a ship is present. This case is somewhat difficult because the speckle is very strong. Fig. 4(b)-(d) show the detection result of CFAR, the saliency map and the detection result of our method respectively. As can be seen, the ship’s location pops out in the saliency map and can be perfectly detected by our method. The conventional CFAR method would cause false alarms even though it works at a very low false alarm rate. Fig. 5(a) gives a SAR image (500×256 pixels) within heterogeneous regions in which two ships are present. It should be a difficult task. Fig. 5(b)-(d) give the detection result of CFAR, the saliency map and the output of our method respectively. As can be seen, our method highlights the ships in the saliency map and detects them perfectly. Although the CFAR method finds ships, it cannot discriminate them from the false alarms. Fig. 6(a) gives another SAR image (400×300 pixels) within heterogeneous regions in which a ship is present. Fig. 6(b) gives the detection result of CFAR. Fig. 6(c) and (d) show the saliency map and the detection result of our method respectively. As can be seen, our method can find the ship successfully while CFAR has false alarms.
Visual Attention-Based Ship Detection in SAR Images
(a)
(b)
(c)
(d)
289
Fig. 4. Test on a SAR image with strong speckles. (a) ALOS SAR (Significance = 2.38). (b) Detection result of CFAR. (c) Saliency map generated by our attention model (Significance = 18.32). (d) Detection result of our method.
(a)
(b)
(c)
(d)
Fig. 5. Test on a SAR image within heterogeneous regions. (a) ALOS SAR (Significance = 2.52). (b) Detection result of CFAR. (c) Saliency map generated by our attention model (Significance = 11.56). (d) Detection result of our method.
290
Y. Yu et al.
(a)
(b)
(c)
(d)
Fig. 6. Test on another SAR image within heterogeneous regions. (a) ALOS SAR (Significance = 3.91). (b) Detection result of CFAR. (c) Saliency map generated by our attention model (Significance = 15.53). (d) Detection result of our method.
(a)
(b)
(c)
(d)
Fig. 7. Test on target-free SAR images. (a) ALOS SAR. (b) Detection result of CFAR. (c) Saliency map generated by our attention model. (d) Output of our method.
Exp. 3: In order to verify the reliability of our method, we employ two target-free SAR images in the last test. Fig. 7 gives two SAR images (256×256 pixels) without any ship signature. The first image is within homogeneous regions, and the second image comprises inhomogeneous regions. In this test, the saliency map generated by our attention model has no dominant locations, and hence the detection threshold value calculated by Eq. (6) will become relatively large. So, there is no false detection in the output of our method. Note that CFAR still has false alarms in this case.
4 Conclusion A visual attention-based technique for ship detection in SAR images has been presented in this paper. The proposed method is automatic, simple and robust against
Visual Attention-Based Ship Detection in SAR Images
291
the speckles and heterogeneous regions in SAR images. The ship signatures can pop out from its backgrounds in the saliency map, and the threshold level for ship discrimination can be easily adjusted, which allows a robust detection. Further work will consist in testing the algorithm for inland applications. Acknowledgments. This work was supported in part by the National High Technology Research and Development Program of China (863 Program) under Grant 2009AA12Z115, in part by the National Natural Science Foundation of China under Grant 60672116, and in part by the Shanghai Leading Academic Discipline Project (No. B112).
References 1. Crisp, D.J.: The state-of-the-art in ship detection in synthetic aperture radar imagery. Defence Sci. Technol. Org., Port Wakefield, South Australia, Research Report DSTO-RR0272 (May 2004), http://www.dsto.defence.gov.au/corporate/reports 2. Eldhuset, K.: An automatic ship and ship wake detection system for spaceborne SAR images in coastal regions. IEEE Trans. Geosci. Remote Sensing 34(4), 1010–1019 (1996) 3. Zhang, F., Wu, B.: A scheme for ship detection in inhomogeneous regions based on segmentation of SAR images. Int. J. Remote Sens. 29(19), 5733–5747 (2008) 4. Liao, M., Wang, C., Wang, Y., Jiang, L.: Using SAR images to detect ships from sea clutter. IEEE Geosci. Remote Sens. Lett. 5(2), 194–198 (2008) 5. Ouchi, K., Tamaki, S., Yaguchi, H., Iehara, M.: Ship detection based on coherence images derived from cross correlation of multilook SAR images. IEEE Geosci. Remote Sens. Lett. 1(3), 184–187 (2004) 6. Tello, M., Lopez-Martinez, C., Mallorqui, J.: A novel algorithm form ship detection in SAR imagery based on the wavelet transform. IEEE Geosci. Remote Sens. Lett. 2(2), 201– 205 (2005) 7. Novak, L.M., Burl, M.C.: Optimal speckle reduction in polarimetric SAR imagery. In: Asilomar Conference on Signals, Systems and Computers, vol. 2, pp. 781–793 (1988) 8. Chen, J., Chen, Y., Yang, J.: Ship detection using polarization cross-entropy. IEEE Geosci. Remote Sens. Lett. 6(4), 723–727 (2009) 9. Treisman, A.M., Gelade, G.: A feature-integration theory of attention. Cogn. Psychol. 12(1), 97–136 (1980) 10. Koch, C., Ullman, S.: Shifts in selective visual attention: towards the underlying neural circuitry. Hum. Neurobiol. 4(4), 219–227 (1985) 11. Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998) 12. Bruce, N.D., Tsotsos, J.K.: Saliency based on information maximization. In: Annual Conference on Neural Information Processing Systems (2005) 13. Hou, X., Zhang, L.: Saliency detection: a spectral residual approach. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE Press, New York (2007) 14. Tello, M., Lopez-Martinez, C., Mallorqui, J.: A novel approach for the automatic detection of punctual isolated targets in a noisy background in SAR imagery. In: European Radar Conference, pp. 1–4. IEEE Press, New York (2005) 15. Crick, F., Koch, C.: Constraints on cortical and thalamic projections: the no-strong-loops hypothesis. Nature 391, 245–250 (1998)
292
Y. Yu et al.
16. Desimone, R., Duncan, J.: Neural mechanisms of selective visual attention. Annu. Rev. Neurosci. 18, 193–222 (1995) 17. Guo, C., Ma, Q., Zhang, L.: Spatio-temporal saliency detection using phase spectrum of quaternion Fourier transform. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE Press, New York (2008) 18. Bian, P., Zhang, L.: Biological plausibility of spectral domain approach for spatiotemporal visual saliency. In: Köppen, M., Kasabov, N., Coghill, G. (eds.) ICONIP 2008. LNCS, vol. 5506, pp. 251–258. Springer, Heidelberg (2009) 19. Yu, Y., Wang, B., Zhang, L.: Pulse discrete cosine transform for saliency-based visual attention. In: 8th IEEE International Conference on Development and Learning, pp. 1–6. IEEE Press, New York (2009) 20. Yu, Y., Wang, B., Zhang, L.: Hebbian-based neural networks for bottom-up visual attention systems. In: 16th International Conference on Neural Information Processing (2009) 21. Ahmed, N., Natarajan, T., Rao, K.: Discrete cosine transform. IEEE Trans. Comput. 23, 90–93 (1974) 22. Gambardella, A., Nunziata, F., Migliaccio, M.: A physical full-resolution SAR ship detection filter. IEEE Geosci. Remote Sens. Lett. 5(4), 760–763 (2008)
Recognizing Multi-ships Based on Silhouette in Infrared Image Jun-Wei Lv, Bo Wang, and Dong-Mei Wang Department of Control Engineering, Naval Aeronautical and Astronautical University, Yantai 264001, P.R. China Tel.: 0086-0-13105356236 [email protected]
Abstract. Recognizing multi-warship in infrared image is very important to missile imaging guidance, an algorithm is suggested in this paper to recognize one ship among the other ones in infrared images. Four silhouette features are defined to describe infrared ships. A rule based on BP neural network with these four features is set up to distinguish one ship from the other ones. A complete algorithm is presented. This algorithm was simulated in computer with twenty-five images. The results showed this algorithm is valid when the sides of the ships face the camera. Keywords: Multi-ship recognizing, Ship silhouette features, Infrared image guidance.
1 Introduction When a missile attacks a warship, at first, the missile is brought to a designated region on a sea by another mode (airplane, ship, middle guidance, etc.), then its Homing Guidance system starts to work, it searches and recognizes the ship to find the warship to b