Lecture Notes in Electrical Engineering Volume 67
Zhigang Zeng · Jun Wang (Eds.)
Advances in Neural Network Research and Applications
ABC
Prof. Zhigang Zeng Department of Control Science and Engineering Huazhong University of Science and Technology Wuhan, Hubei, 430074 China E-mail:
[email protected] Prof. Jun Wang Department of Mechanical & Automation Engineering The Chinese University of Hong Kong Shatin, New Territories Hong Kong Tel.: +852 26098472 Fax: +852-26036002 E-mail:
[email protected]
ISBN 978-3-642-12989-6
e-ISBN 978-3-642-12990-2
DOI 10.1007/978-3-642-12990-2 Library of Congress Control Number: 2010926919 c 2010 Springer-Verlag Berlin Heidelberg This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typeset & Coverdesign: Scientific Publishing Services Pvt. Ltd., Chennai, India. Printed on acid-free paper 987654321 springer.com
Preface
This book is a part of the Proceedings of the Seventh International Symposium on Neural Networks (ISNN 2010), held on June 6-9, 2010 in Shanghai, China. Over the past few years, ISNN has matured into a well-established premier international symposium on neural networks and related fields, with a successful sequence of ISNN series in Dalian (2004), Chongqing (2005), Chengdu (2006), Nanjing (2007), Beijing (2008), and Wuhan (2009). Following the tradition of ISNN series, ISNN 2010 provided a high-level international forum for scientists, engineers, and educators to present the state-of-the-art research in neural networks and related fields, and also discuss the major opportunities and challenges of future neural network research. Over the past decades, the neural network community has witnessed significant breakthroughs and developments from all aspects of neural network research, including theoretical foundations, architectures, and network organizations, modeling and simulation, empirical studies, as well as a wide range of applications across different domains. The recent developments of science and technology, including neuroscience, computer science, cognitive science, nano-technologies and engineering design, among others, has provided significant new understandings and technological solutions to move the neural network research toward the development of complex, large scale, and networked brain-like intelligent systems. This long-term goals can only be achieved with the continuous efforts from the community to seriously investigate various issues on neural networks and related topics. Toward this end, the ISNN 2010 provided a great platform for the community to share their latest research results, discuss critical future research directions, stimulate innovative research ideas, as well as facilitate worldwide multi-disciplinary collaborations. ISNN 2010 received numerous submissions from about thousands of authors in about 40 countries and regions (Australia, Belgium, Brazil, Canada, China, Colombia, Czech Republic, Finland, France, Germany, Hong Kong, India, Iran, Italy, Japan, Lebanon, Lithuania, Macao, Malaysia, Mexico, Netherlands, New Zealand, Norway, Peru, Poland, Portugal, Singapore, Slovenia, South Africa, South Korea, Spain, Sri Lanka, Taiwan, Thailand, Tunisia, Uganda, United Arab Emirates, United Kingdom, and USA) across six continents (Asia, Europe, North America, South America, Africa, and Oceania). Based on the rigorous peer-reviews by the program committee members and the reviewers, 108 high-quality papers were selected for publications in Lecture Notes in Electrical Engineering (LNEE) Proceedings. These papers cover all major topics of the engineering designs and applications of neural network research. In addition to the contributed papers, the ISNN 2010 technical program included four plenary speeches by Andrzej Cichocki (RIKEN Brain Science Institute, Japan), ChinTeng Lin (National Chiao Tung University, Taiwan), DeLiang Wang (Ohio State University, USA), Gary G. Yen (Oklahoma State University, USA). We would like to take this opportunity to express our sincere thanks to the Shanghai Jiao Tong University, The Chinese University of Hong Kong, Microsoft Research Asia, and the National Natural Science Foundation of China for their
VI
Preface
sponsorship, to the IEEE Shanghai Section, IEEE Computational Intelligence Society, International Neural Network Society, Asia Pacific Neural Network Assembly, European Neural Network Society, and International Association for Mathematics and Computers in Simulation for their technical co-sponsorship. We would also like to sincerely thank the chairs for their contributions to the symposium, members of the Advisory Committee and Steering Committee for their guidance, and the members of the Organizing Committee, Special Sessions Committee, Publication Committee, Publicity Committee, Finance Committee, Registration Committee, and Local Arrangements Committee for all their great efforts and time in organizing such an event. We would like especially to express our deepest gratitude to the members of the International Program Committee and all reviewers for their professional review of the papers; their expertise guaranteed the high quality of the technical program of the ISNN 2010. Furthermore, we would also like to thank Springer for publishing the proceedings in the prestigious series of Lecture Notes in Electrical Engineering. Meanwhile, we also would like to express our heartfelt appreciations to the plenary and panel speakers for their vision and discussions on the latest research development in the field as well as critical future research directions, opportunities, and challenges. Finally, we would also like to thank all the speakers, authors, and participants for their great contribution and support that made ISNN 2010 a great success.
Zhigang Zeng Jun Wang
ISNN 2010 Organization
Sponsor and Organizer Shanghai Jiao Tong University
Co-organizers The Chinese University of Hong Kong Microsoft Research Asia
Technically Co-sponsors IEEE Shanghai Section IEEE Computational Intelligence Society International Neural Network Society European Neural Network Society Asia Pacific Neural Network Assembly International Association for Mathematics and Computers in Simulation
Financial Co-sponsor National Natural Science Foundation of China
General Chairs Jun Wang Bao-Liang Lu
Hong Kong, China Shanghai, China
Organizing Committee Chair Jianbo Su
Shanghai, China
Program Committee Chairs Liqing Zhang Zhigang Zeng James T.Y. Kwok
Shanghai, China Wuhan, China Hong Kong, China
VIII
Organization
Special Sessions Chairs Si Wu Qing Ma Paul S. Pang
Shanghai, China Kyoto, Japan Auckland, New Zealand
Publications Chairs Hongtao Lu Yinling Wang Wenlian Lu
Shanghai, China Shanghai, China Shanghai, China
Publicity Chairs Bo Yuan Xiaolin Hu Qingshan Liu
Shanghai, China Beijing, China Nanjing, China
Finance Chairs Xinping Guan Xiangyang Zhu
Shanghai, China Shanghai, China
Registration Chairs Fang Li Gui-Rong Xue Daniel W.C. Ho
Shanghai, China Shanghai, China Hong Kong, China
Local Arrangements Chairs Qingsheng Ren Xiaodong Gu
Shanghai, China Shanghai, China
Advisory Committee Chairs Xiaowei Tang Bo Zhang Aike Guo
Hangzhou, China Beijing, China Shanghai, China
Organization
IX
Advisory Committee Members Cesare Alippi, Milan, Italy Shun-ichi Amari, Tokyo, Japan Zheng Bao, Xi'an, China Dimitri P. Bertsekas, Cambridge, MA, USA Tianyou Chai, Shenyang, China Guanrong Chen, Hong Kong Andrzej Cichocki, Tokyo, Japan Ruwei Dai, Beijing, China Jay Farrell, Riverside, CA, USA Chunbo Feng, Nanjing, China Russell Eberhart, Indianapolis, IN, USA David Fogel, San Diego, CA, USA Walter J. Freeman, Berkeley, CA, USA Kunihiko Fukushima, Osaka, Japan Xingui He, Beijing, China Zhenya He, Nanjing, China Janusz Kacprzyk, Warsaw, Poland Nikola Kasabov, Auckland, New Zealand Okyay Kaynak, Istanbul, Turkey Anthony Kuh, Honolulu, HI, USA Frank L. Lewis, Fort Worth, TX, USA Deyi Li, Beijing, China Yanda Li, Beijing, China
Chin-Teng Lin, Hsinchu, Taiwan Robert J. Marks II, Waco, TX, USA Erkki Oja, Helsinki, Finland Nikhil R. Pal, Calcutta, India Marios M. Polycarpou, Nicosia, Cyprus José C. Príncipe, Gainesville, FL, USA Leszek Rutkowski, Czestochowa, Poland Jennie Si, Tempe, AZ, USA Youxian Sun, Hangzhou, China DeLiang Wang, Columbus, OH, USA Fei-Yue Wang, Beijing, China Shoujue Wang, Beijing, China Paul J. Werbos, Washington, DC, USA Cheng Wu, Beijing, China Donald C. Wunsch II, Rolla, MO, USA Youlun Xiong, Wuhan, China Lei Xu, Hong Kong Shuzi Yang, Wuhan, China Xin Yao, Birmingham, UK Gary G. Yen, Stillwater, OK, USA Nanning Zheng, Xi'an, China Yongchuan Zhang, Wuhan, China Jacek M. Zurada, Louisville, KY, USA
Program Committee Members Haydar Akca Alma Y. Alanis Bruno Apolloni Sabri Arik Vijayan Asari Tao Ban Peter Baranyi Salim Bouzerdoum Martin Brown Xindi Cai Jianting Cao Yu Cao Jonathan Chan
Chu-Song Chen Liang Chen Sheng Chen Songcan Chen YangQuan Chen Yen-Wei Chen Zengqiang Chen Jianlin Cheng Li Cheng Long Cheng Zheru Chi Sung-Bae Cho Emilio Corchado
X
Organization
Jose Alfredo F. Costa Ruxandra Liana Costea Sergio Cruces Baotong Cui Chuanyin Dang Mingcong Deng Ming Dong Jixiang Du Andries Engelbrecht Meng Joo Er Jufu Feng Chaojin Fu Wai-Keung Fung John Gan Junbin Gao Xiao-Zhi Gao Xinping Guan Chen Guo Chengan Guo Ping Guo Abdenour Hadid Honggui Han Qing-Long Han Haibo He Hanlin He Zhaoshui He Akira Hirose Daniel Ho Noriyasu Homma Zhongsheng Hou Chun-Fei Hsu Huosheng Hu Jinglu Hu Junhao Hu Sanqing Hu Guang-Bin Huang Tingwen Huang Wei Hui Amir Hussain Jayadeva Minghui Jiang Tianzi Jiang Yaochu Jin Joarder Kamruzzaman Shunshoku Kanae Qi Kang Nik Kasabov
Okyay Kaynak Rhee Man Kil Kwang-Baek Kim Sungshin Kim Mario Koeppen Rakhesh Singh Kshetrimayum Edmund Lai Heung Fai Lam Minho Lee Chi-Sing Leung Henry Leung Chuandong Li Fang Li Guang Li Kang Li Li Li Shaoyuan Li Shutao Li Xiaoli Li Xiaoou Li Xuelong Li Yangmin Li Yuanqing Li Yun Li Zhong Li Jinling Liang Ming Liang Pei-Ji Liang Yanchun Liang Li-Zhi Liao Wudai Liao Longnian Lin Guoping Liu Ju Liu Meiqin Liu Yan Liu Hongtao Lu Jianquan Lu Jinhu Lu Wenlian Lu Jian Cheng Lv Jinwen Ma Malik Magdon Ismail Danilo Mandic Tiemin Mei Dan Meng Yan Meng
Organization
Duoqian Miao Martin Middendorf Valeri Mladenov Marco Antonio Moreno-Armendáriz Ikuko Nishkawa Stanislaw Osowski Seiichi Ozawa Shaoning Pang Jaakko Peltonen Vir V. Phoha Branimir Reljin Qingsheng Ren Tomasz Rutkowski Sattar B. Sadkhan Toshimichi Saito Gerald Schaefer Furao Shen Daming Shi Hideaki Shimazaki Michael Small Qiankun Song Jochen J. Steil John Sum Roberto Tagliaferri Norikazu Takahashi Ah-hwee Tan Ying Tan Toshihisa Tanaka Dacheng Tao Ruck Thawonmas Xin Tian Christos Tjortjis Ivor Tsang Masao Utiyama Marc Vanhulle Bin Wang Dan Wang Dianhui Wang Lei Wang Liang Wang Rubin Wang
Wenjia Wang Wenwu Wang Xiaoping Wang Xin Wang Yinglin Wang Yiwen Wang Zhanzhan Wang Zhongsheng Wang Zidong Wang Hau-San Wong Kevin Wong Wei Wu Cheng Xiang Hong Xie Songyun Xie Rui Xu Xin Xu Guirong Xue Yang Yang Yingjie Yang Yongqing Yang Jianqiang Yi Dingli Yu Jian Yu Xiao-Hua Yu Bo Yuan Kun Yuan Pong C Yuen Xiaoqin Zeng Changshui Zhang Jie Zhang Junping Zhang Kai Zhang Lei Zhang Nian Zhang Dongbin Zhao Hai Zhao Liang Zhao Qibin Zhao Mingjun Zhong Weihang Zhu
Reviewers Ajith Abraham Alma Y. Alanis NG Alex
Jing An Sung Jun An Claudia Angelini
Nancy Arana-Daniel Nancy Arana-Daniel Kiran Balagani
XI
XII
Organization
Tao Ban Simone Bassis Anna Belardinelli Joao Roberto Bertini Junior Amit Bhaya Shuhui Bi Xuhui Bo Salim Bouzerdoum N. Bu Qiao Cai Xindi Cai Hongfei Cao Yuan Cao Jonathan Chan Wenge Chang Benhui Chen Bo-Chiuan Chen Chao-Jung Chen Chu-Song Chen Cunbao Chen Fei Chen Gang Chen Guici Chen Junfei Chen Lei Chen Min Chen Pin-Cheng Chen Sheng Chen Shuwei Chen Tao Chen Xiaofen Chen Xiaofeng Chen Yanhua Chen Yao Chen Zengqiang Chen Zhihao Chen Jianlin Cheng K. H. Cheng Lei Cheng Yu Cheng Yuhu Cheng Seong-Pyo Cheon Zheru Chi Seungjin Choi Angelo Ciaramella Matthew Conforth
Paul Christopher Conilione Paleologu Constantin Jose Alfredo F. Costa Ruxandra Liana Costea Fangshu Cui Zhihua Cui James Curry Qun Dai Xinyu Dai Spiros Denaxas Jing Deng Xin Deng Zhijian Diao Ke Ding Jan Dolinsky Yongsheng Dong Adriao Duarte Doria Neto Dajun Du Jun Du Shengzhi Du Wei Du Qiguo Duan Zhansheng Duan Julian Eggert Yong Fan Chonglun Fang Italia De Feis G C Feng Qinrong Feng Simone Fiori Chaojin Fu Jun Fu Zhengyong Fu Zhernyong Fu Sheng Gan Shenghua Gao Fei Ge Vanessa Goh Dawei Gong Weifeng Gu Wenfei Gu Renchu Guan Chengan Guo Jianmei Guo Jun Guo Ping Guo
Xin Guo Yi Guo Juan Carlos Gutierrez Caceres Osamu Hasegawa Aurelien Hazart Hanlin He Huiguang He Lianghua He Lin He Wangli He Xiangnan He Zhaoshui He Sc Ramon Hernandez Esteban Hernandez-Vargas Kevin Ho Xia Hong Chenping Hou Hui-Huang Hsu Enliang Hu Jinglu Hu Junhao Hu Meng Hu Sanqing Hu Tianjiang Hu Xiaolin Hu Zhaohui Hu Bonan Huang Chun-Rong Huang Dan Huang J. Huang Kaizhu Huang Shujian Huang Xiaodi Huang Xiaolin Huang Zhenkun Huang Cong Hui GuoTao Hui Khan M. Iftekharuddin Tasadduq Imam Teijiro Isokawa Mingjun Ji Zheng Ji Aimin Jiang Changan Jiang Feng Jiang
Organization
Lihua Jiang Xinwei Jiang Gang Jin Ning Jin Yaochu Jin Krzysztof Siwek Yiannis Kanellopoulos Enam Karim Jia Ke Salman Khan Sung Shin Kim Tae-Hyung Kim Mitsunaga Kinjo Arto Klami Mario Koeppen Adam Kong Hui Kong Qi Kong Adam Krzyzak Jayanta Kumar Debnath Kandarpa Kumar Sarma Franz Kurfess Paul Kwan Darong Lai Jiajun Lai Jianhuang Lai Wei Lai Heung Fai Lam Paul Lam Yuan Lan Ngai-Fong Law N. K. Lee Chi SingLeung Bing Li Boyang Li C. Li Chaojie Li Chuandong Li Dazi Li Guang Li Junhua Li Kang Li Kelin Li Li Li Liping Li Lulu Li Manli Li
Peng Li Ping Li Ruijiang Li Tianrui Li Tieshan Li Xiaochen Li Xiaocheng Li Xuelong Li Yan Li Yun Li Yunxia Li Zhenguo Li Allan Liang Jinling Liang Pei-Ji Liang Li-Zhi Liao Wudai Liao Hongfei Lin Qing Lin Tran Hoai Lin Bo Liu Chang Liu Chao Liu Fei Liu Hongbo Liu Jindong Liu Lei Liu Lingqiao Liu Nianjun Liu Qingshan Liu Wei Liu Xiangyang Liu Xiwei Liu Yan Liu Yanjun Liu Yu Liu Zhaobing Liu Zhenwei Liu Jinyi Long Jinyi Long Carlos Lopez-Franco Shengqiang Lou Mingyu Lu Ning Lu S F Lu Bei Lv Jun Lv
XIII
Fali Ma Libo Ma Singo Mabu Danilo Mandic Qi Mao Tomasz Markiewicz Radoslaw Mazur Tiemin Mei Bo Meng Zhaohui Meng Marna van der Merwe Martin Middendorf N. Mitianoudis Valeri Mladenov Alex Moopenn Marco Moreno Loredana Murino Francesco Napolitano Ikuko Nishkawa Tohru Nitta Qiu Niu Qun Niu Chakarida Nukoolkit Sang-Hoon Oh Floriberto Ortiz Stanislaw Osowski Antonio de Padua Braga Antonio Paiva Shaoning Pang Woon Jeung Park Juuso Parkkinen Michael Paul Anne Magály de Paula Canuto Zheng Pei Jaakko Peltonen Ce Peng Hanchuan Peng Jau-Woei Perng Son Lam Phung Xiong Ping Kriengkrai Porkaew Santitham Prom-on Dianwei Qian Lishan Qiao Keyun Qin Meikang Qiu
XIV
Organization
Li Qu Marcos G. Quiles Mihai Rebican Luis J. Ricalde Jorge Rivera Haijun Rong Zhihai Rong Tomasz Rutkowski Jose A. Ruz Edgar N. Sanchez Sergio P. Santos Renato José Sassi Chunwei Seah Nariman Sepehri Caifeng Shan Shiguang Shan Chunhua Shen Furao Shen Jun Shen Yi Shen Jiuh-Biing Sheu Licheng Shi Qinfeng Shi Xiaohu Shi Si Si Leandro Augusto da Silva Angela Slavova Sunantha Sodsee Dandan Song Dongjin Song Doo Heon Song Mingli Song Qiang Song Qiankun Song Kingkarn Sookhanaphibarn Gustavo Fontoura de Souza Antonino Staiano Jochen Steil Pui-Fai Sum Jian Sun Jian-Tao Sun Junfeng Sun Liang Sun Liming Sun Ning Sun
Yi Sun Shigeru Takano Mingkui Tan Ke Tang Kecheng Tang Y Tang Liang Tao Yin Tao Sarwar Tapan Ruck Thawonmas Tuan Hue Thi Le Tian Fok Hing Chi Tivive Christos Tjortjis Rutkowski Tomasz Julio Tovar Jianjun Tu Zhengwen Tu Goergi Tzenov Lorenzo Valerio Rodrigo Verschae Liang Wan Min Wan Aihui Wang Bin Wang Bo Hyun Wang Chao Wang Chengyou Wang Dianhui Wang Guanjun Wang Haixian Wang Hongyan Wang Huidong Wang Huiwei Wang Jingguo Wang Jinghua Wang Lan Wang Li Wang Lili Wang Lizhi Wang Min Wang Ming Wang Pei Wang Ruizhi Wang Xiaolin Wang Xiaowei Wang Xin Wang
Xu Wang Yang Wang Ying Wang You Wang Yunyun Wang Zhanshan Wang Zhengxia Wang Zhenxing Wang Zhongsheng Wang Bunthit Watanapa Hua-Liang Wei Qinglai Wei Shengjun Wen Young-Woon Woo Ailong Wu Chunguo Wu Jun Wu Qiang Wu Si Wu Xiangjun Wu Yili Xia Zeyang Xia Cheng Xiang Linying Xiang Shiming Xiang Xiaoliang Xie Ping Xiong Zhihua Xiong Fang Xu Feifei Xu Heming Xu Jie Xu LinLi Xu Rui Xu Weihong Xu Xianyun Xu Xin Xu Hui Xue Jing Yang Liu Yang Qingshan Yang Rongni Yang Shangming Yang Wen-Jie Yang Wenlu Yang Wenyun Yang Xubing Yang
Organization
Yan Yang Yongqing Yang Zi-Jiang Yang John Yao Jun Yao Yingtao Yao Keiji Yasuda Ming-Feng Yeh Xiao Yi Chenkun Yin Kaori Yoshida WenwuYu Xiao-Hua Yu Kun Yuan Weisu Yuan Xiaofang Yuan Zhuzhi Yuan Zhuzhu Yuan P C Yuen Masahiro Yukawa Lianyin Zhai Biao Zhang Changshui Zhang Chen Zhang
Secretariat Jin Gang Kan Hong Qi Kong Qiang Wang Qiang Wu Rong Zhou Tianqi Zhang
Dapeng Zhang Jason Zhang Jian Zhang Jianbao Zhang Jianhai Zhang Jianhua Zhang Jin Zhang Junqi Zhang Junying Zhang Kai Zhang Leihong Zhang Liming Zhang Nengsheng Zhang Nian Zhang Pu-Ming Zhang Qing Zhang Shaohong Zhang Tao Zhang Teng-Fei Zhang Ting Zhang Xian-Ming Zhang Yuyang Zhang Hai Zhao Qibin Zhao
Xiaoyu Zhao Yi Zhao Yongping Zhao Yongqing Zhao Ziyang Zhen Chengde Zheng Lihong Zheng Yuhua Zheng Caiming Zhong Mingjun Zhong Shuiming Zhong Bo Zhou Jun Zhou Luping Zhou Rong Zhou Xiuling Zhou Haojin Zhu Song Zhu Wenjun Zhu Xunlin Zhu Yuanming Zhu Wei-Wen Zou Xin Zou Pavel Zuñiga
XV
Table of Contents
Prediction and Forecasting A Novel Prediction Mechanism with Modified Data Mining Technique for Call Admission Control in Wireless Cellular Network . . . . . . . . . . . . . . Chen-Feng Wu, Yu-Teng Chang, Chih-Yao Lo, and Han-Sheng Zhuang
1
The Study of Forecasting Model of Rock Burst for Acoustic Emission Based on BP Neural Network and Catastrophe Theory . . . . . . . . . . . . . . . Yunyun Xu and Dongqiang Xu
11
Application of Wavelet Neural Network to Prediction of Water Content in Crude Oil . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Huifen Niu, Cuiling Liu, Jinqi Wang, and Xiaowen Sun
21
Prediction of Urban Heat Island Intensity in Chuxiong City with Backpropagation Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Wujun Xi and Ping He
29
Research on the Fouling Prediction of Heat Exchanger Based on Wavelet Relevance Vector Machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lingfang Sun, Rina Saqi, and Honggang Xie
37
Electricity Price Forecasting Using Neural Networks Trained with Entropic Criterion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jianhua Zhang, Jingyue Wang, Rui Wang, and Guolian Hou
47
Least Square Support Vector Machine Ensemble for Daily Rainfall Forecasting Based on Linear and Nonlinear Regression . . . . . . . . . . . . . . . . Jiansheng Wu, Mingzhe Liu, and Long Jin
55
Estimating Portfolio Risk Using GARCH-EVT-Copula Model: An Empirical Study on Exchange Rate Market . . . . . . . . . . . . . . . . . . . . . . . . . . Zongrun Wang, Yanbo Jin, and Yanju Zhou
65
Forecasting Financial Time Series via an Efficient CMAC Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chi-Jie Lu and Jui-Yu Wu
73
Forecasting Daily Cash Turnover of Bank with EWMA and SVR . . . . . . Wei-min Ma and Wei Lu Financial Distress Prediction Model via GreyART Network and Grey Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ming-Feng Yeh, Chia-Ting Chang, and Min-Shyang Leu
83
91
XVIII
Table of Contents
Risk Assessment Model Based on Immune Theory . . . . . . . . . . . . . . . . . . . . Tao Liu, Li Shang, and Zhifeng Hu
101
Short-Term Load Forecasting Based on Bayes and RS . . . . . . . . . . . . . . . . Yanmei Li
109
Fuzzy Neural Networks Nonlinear System Modeling with a New Fuzzy Model and Neural Compensation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cruz Vega Israel and Wen Yu
117
A Research of Fuzzy Neural Network in Ferromagnetic Target Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Caipeng Wu, Jiahao Deng, and Yanli Yang
129
Multiple T-S Fuzzy Neural Networks Soft Sensing Modeling of Flotation Process Based on Fuzzy C-Means Clustering Algorithm . . . . . . Jiesheng Wang, Yong Zhang, and Shifeng Sun
137
A Using Reliability Evaluation Model for Diesel Engine Based on Fuzzy Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ying-Kui Gu and Kai-Qi Huang
145
Fuzzy Sliding Mode Control with Perturbation Estimation for a Piezoactuated Micromanipulator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Qingsong Xu and Yangmin Li
153
A Credit Risk Rating Model Based on Fuzzy Neural Network . . . . . . . . . . Ke-Jun Zhu, Pin-Chang Chen, and Yu-Teng Chang
161
Interval-Valued Fuzzy Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Wenyi Zeng and Jiayin Wang
173
Research on Fuzzy Preference Relations-Based MAS for Decision Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Weijin Jiang and Qing Jiang
185
Optimization and Planning Study of Stochastic Demand Inventory Routing Problem with Soft Time Windows Based on MDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Wei Zeng and Qian Zhao
193
An Agent-Based Approach to Joint Procurement Modeling with Virtual Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chih-Yao Lo, Hung-Teng Chang, Yu-Teng Chang, and Hsiu-Yun Hu
201
Table of Contents
Interactive Hybrid Evolutionary Computation for MEMS Design Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ying Zhang and Alice M. Agogino
XIX
211
Genetic Algorithms for Traffic Grooming in Optical Tree Networks . . . . . Shu-tong Xie and Li-fang Pan
219
Multi-sensor Multi-target Tracking with OOSM . . . . . . . . . . . . . . . . . . . . . . Cheng Cheng and Jinfeng Wang
227
Hopfield Neural Network Guided Evolutionary Algorithm for Aircraft Penetration Path Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Nan Wang, Lin Wang, Xueqiang Gu, Jing Chen, and Lincheng Shen Fuzzy Material Procurement Planning with Value-at-Risk . . . . . . . . . . . . . Gao-Ji Sun Radial Basis Function Network for Endpoint Detection in Plasma Etch Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Shu-Kun Zhao, Min-Woo Kim, Yi-Seul Han, Se-Youn Jeon, Yun-Keun Lee, and Seung-Soo Han A Novel Cellular Neural Network and Its Applications in Motion Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yan Cao, Feng Zhang, Xuewu Wu, Sheng Lu, Yi Li, Lei Sun, and Shuai Li Evaluation of Enterprise ERP System Based on Neural Network Optimized by Ant Colony . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hanmei Wang, Dongxiao Niu, and Chengkai Cai
235 245
253
265
275
Pattern Recognition Visual Attention-Based Ship Detection in SAR Images . . . . . . . . . . . . . . . . Ying Yu, Zhenghu Ding, Bin Wang, and Liming Zhang
283
Recognizing Multi-ships Based on Silhouette in Infrared Image . . . . . . . . Jun-Wei Lv, Bo Wang, and Dong-Mei Wang
293
SAR Images Feature Extraction and Recognition Based on G2DCDA . . . Liping Hu, Hongwei Liu, and Ping Zhou
301
Bridge Detection and Recognition in Remote Sensing SAR Images Using Pulse Coupled Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Zhenming Peng, Shijun Liu, Guiyou Tian, Zhang Chen, and Tao Tao
311
Approaches to Robotic Vision Control Using Image Pointing Recognition Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tian-Ding Chen
321
XX
Table of Contents
A Hybrid Evolutionary Approach to Band Selection for Hyperspectral Image Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hao Wu, Jiali Zhu, Shijin Li, Dingsheng Wan, and Lin Lin Impulsive Environment Sound Detection by Neural Classification of Spectrogram and Mel-Frequency Coefficient Images . . . . . . . . . . . . . . . . . . Peerapol Khunarsa, Chidchanok Lursinsap, and Thanapant Raicharoen Classification and Control of Cognitive Radios Using Hierarchical Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sheng Chen, Xiaochen Li, Qiao Cai, Nansai Hu, Haibo He, Yu-Dong Yao, and Joseph Mitola Identifying Spatial Patterns of Land Use and Cover Change at Different Scales Based on Self-Organizing Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hao Wu, Xiaoling Chen, Zhan Li, Sheng Wang, Wei Cui, and Qian Meng Colon Tumor Microarray Classification Using Neural Network with Feature Selection and Rule-Based Classification . . . . . . . . . . . . . . . . . . . . . . Narissara Eiamkanitchat, Nipon Theera-Umpon, and Sansanee Auephanwiriyakul
329
337
347
355
363
Signal and Image Processing Dual Channel Speech Denoising Based on Sparse Representation . . . . . . . Gongxian Sun, Feng Gao, Jun Lv, and Ming Xiao Frequency-Domain Blind Separation of Convolutive Speech Mixtures with Energy Correlation-Based Permutation Correction . . . . . . . . . . . . . . . Li-Dan Wang and Qiu-Hua Lin A Blind Broadband Beamforming Method for Speech Enhancement . . . . Dongxia Wang, Fuliang Yin, and Fuming Sun
373
381
391
Algorithm and Simulation Research for Blind Nonlinear System Identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hongqiu Teng and Weizheng Ruan
397
Study on Digital Image Correlation Using Artificial Neural Networks for Subpixel Displacement Measurement . . . . . . . . . . . . . . . . . . . . . . . . . . . . Xiao-yong Liu, Qing-chang Tan, and Rong-li Li
405
Tree Modeling through Range Image Segmentation and 3D Shape Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mingrui Dai, Hongjun Li, and Xiaopeng Zhang
413
Table of Contents
XXI
Combining Bag of Words Model and Information Theoretic Method for Image Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Xue Bai and Siwei Luo
423
Fast Fuzzy c-Means Clustering Algorithm with Spatial Constraints for Image Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yanling Li and Gang Li
431
Application of Support Vector Machines to Airborne Hyper-Spectral Image Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ming-Der Yang, Kai-Siang Huang, Ji-Yuan Lin, and Pei Liu
439
Research for the Identification Method of the Image Definition Based on a W-N Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Miaofen Zhu, Guojin Chen, Yongning Li, and Wanqiang Wang
445
Analysis of Texture Images Generated by Olfactory System Bionic Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chen Fang, Jin Zhang, Shangwu Zhu, Guang Li, and Rulong Wang
453
Do Neural Networks Have True Power for Natural Language Processing? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Qing Ma
461
Robust Channel Identification Using FOCUSS Method . . . . . . . . . . . . . . . Zhaoshui He and Andrzej Cichocki
471
Human Head Modeling Using NURBS Method . . . . . . . . . . . . . . . . . . . . . . Songyun Xie, Ningfei Li, and Zhuo Lv
479
Risk Sensitive Unscented Particle Filter for Bearing and Frequency Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Peng Li, Shenmin Song, and Xinglin Chen
485
Fully Complex Multiplicative Neural Network Model and Its Application to Channel Equalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kavita Burse, R.N. Yadav, and Sushil Chandra Shrivastava
493
Robotics and Control Visual Navigation of a Novel Economical Embedded Multi-mode Intelligent Control System for Powered Wheelchair . . . . . . . . . . . . . . . . . . . Jizhong Liu, Xuepei Wu, Jiating Xia, Guanghui Wang, and Hua Zhang Neural Networks L2 -Gain Control for Robot System . . . . . . . . . . . . . . . . . . Zhi-gang Yu, Yong-liang Shen, Shen-min Song, and Da-wei Zhang
503
513
XXII
Table of Contents
Neural Network Control of Spacecraft Formation Using RISE Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Shicheng Wang, Haibo Min, Fuchun Sun, and Jinsheng Zhang A Simplified Modular Petri Net for the Walking Assistant Robot . . . . . . . Zhen Zhang, Qixin Cao, Chuntao Leng, and Peihua Chen
521
529
Omni-directional Vision Based Tracking and Guiding System for Walking Assistant Robot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Xiaoxiao Zhu, Qixin Cao, Hongbing Tan, and Aolin Tang
537
Dynamic Eye-in-Hand Visual Servoing with Unknown Target Positions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hesheng Wang, Weidong Chen, and Yun-hui Liu
545
Optimum Motion Control for Stacking Robot . . . . . . . . . . . . . . . . . . . . . . . . Xiaoming Zhang, Nan Luan, Zhong Dong, and Liming Chen Passive Target Tracking Using an Improved Particle Filter Algorithm Based on Genetic Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yue Liang, Zhong Liu, and Guodong Zhang Large-Scale Structure Assembly by Multiple Robots Which May Be Broken . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Masayuki Otani, Kiyohiko Hattori, Hiroyuki Sato, and Keiki Takadama Real-Time Five DOF Redundant Robot Control Using a Decentralized Neural Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ramon Garcia-Hernandez, Edgar N. Sanchez, Maarouf Saad, and Eduardo Bayro-Corrochano
551
559
567
577
Improving Transient Response of Adaptive Control Systems Using Multiple Neural Network Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Xiao-li Li, Xiang Yu, and Yan Zhang
587
An Information Theoretic Approach for Design MIMO Networked Control Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jianhua Zhang and Hong Wang
595
An Engineering Solution for Decoupling Control of Aircraft Motion Using Affine Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tsurng-Jehng Shen
603
Black-Box Input-Output Identification of a Class of Nonlinear Systems Using a Discrete-Time Recurrent Neurofuzzy Network . . . . . . . . . . . . . . . . Marcos A. Gonz´ alez-Olvera and Yu Tang
615
Table of Contents
XXIII
Passivity Analysis of Stochastic Neural Networks with Mixed Time-Varying Delays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Qinggao He and Qiankun Song
623
A Novel Recovering Initial Conditions Method from Spatiotemporal Complex Dynamical System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Longquan Dai, Xiaoyun Kang, and Minfen Shen
631
An Intelligent Control Scheme for Nonlinear Time-Varying Systems with Time Delay . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sun Zhou, Guoli Ji, Wei Lin, and Zijiang Yang
639
Master-Slave Chaos Synchronization of Uncertain Nonlinear Gyros Using Wavelet Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chien-Jung Chiu, Chun-Fei Hsu, Tsu-Tian Lee, and Jang-Zern Tsai
647
Transportation Systems WNN-Based Intelligent Transportation Control System . . . . . . . . . . . . . . . Chiu-Hsiung Chen, Ya-Fu Peng, and Tsai-Sheng Kao
655
Incident Detection in Urban Road . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Wu Bing-Fei, Kao Chih-Chung, Chen Chao-Jung, Li Yen-Feng, Chen Ying-Han, and Yang Cheng-Yen
663
An Efficient Web-Based Tracking System through Reduction of Redundant Connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Bing-Fei Wu, Ying-Han Chen, Chao-Jung Chen, Chih-Chung Kao, and Po-Chia Huang An Embedded All-Time Blind Spot Warning System . . . . . . . . . . . . . . . . . Bing-Fei Wu, Chao-Jung Chen, Yen-Feng Li, Cheng-Yen Yang, Hai-Chang Chien, and Chia-Wei Chang Design of Autonomous Parallel Parking Using Fuzzy Logic Controller with Feed-Forward Compensation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Bo-Chiuan Chen, Yi-Wen Huang, Shiuh-Jer Huang, and Bo-Jhao Liu Telematics Services through Mobile Agents . . . . . . . . . . . . . . . . . . . . . . . . . . Jonathan Lee, Shin-Jie Lee, Hsi-Min Chen, and Wen-Tin Lee Multi-agent System Model for Urban Traffic Simulation and Optimizing Based on Random Walk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yu Cheng, Tao Zhang, and Jianfei Wang Vehicle Detection Using Bayesian Enhanced CoBE Classification . . . . . . . Zhong Zheng, Shen Xu, and Yi L. Murphey
671
679
687
695
703 713
XXIV
Table of Contents
Vibration Analysis of a Submarine Model Based on an Improved ICA Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Wei Cheng, Zhousuo Zhang, and Zhengjia He
721
A Hierarchical Salient-Region Based Algorithm for Ship Detection in Remote Sensing Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Fukun Bi, Feng Liu, and Lining Gao
729
Industrial Applications Turning Tool Wear Monitoring Based on Fuzzy Cluster Analysis . . . . . . . Hongtao Chen, Sui Huang, Dengwan Li, and Pan Fu Part-Machine Clustering: The Comparison between Adaptive Resonance Theory Neural Network and Ant Colony System . . . . . . . . . . . Bo Xing, Wen-Jing Gao, Fulufhelo V. Nelwamondo, Kimberly Battle, and Tshilidzi Marwala
739
747
Fault Diagnosis of Bearings Based on Time-Delayed Correlation and Demodulation as Well as B-Spline Fuzzy Neural Networks . . . . . . . . . . . . . Pan Fu, Li Jiang, A.D. Hope, and Weiling Li
757
Fast and Noninvasive Determination of Viscosity of Lubricating Oil Based on Visible and Near Infrared Spectroscopy . . . . . . . . . . . . . . . . . . . . Lulu Jiang, Yu Zhang, Fei Liu, Lihong Tan, and Yong He
765
Chattering-Free Adaptive Wavelet Neural Network Control for a BLDC Motor via Dynamic Sliding-Mode Approach . . . . . . . . . . . . . . . . . . . . . . . . . Chun-Fei Hsu and Chih-Hu Wang
773
A New BPSO Algorithm and Applications in Interruptible Load Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ping Huang, Pengcheng Li, Yao Zhang, Jinyang Yu, and Yongquan Yuan
783
Force Identification by Using Support Vector Machine and Differential Evolution Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Zhichao Fu and Wei Cheng
793
Soft Sensor Modeling of Ball Mill Load via Principal Component Analysis and Support Vector Machines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jian Tang, Lijie Zhao, Wen Yu, Heng Yue, and Tianyou Chai
803
An Approach Based on Hilbert-Huang Transform and Support Vector Machine for Intelligent Fault Diagnosis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chang Che and Dan Hu
811
Table of Contents
XXV
Real-World Applications Study on Factors of Floating Women’s Income in Jiangsu Province Based on Bayesian Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yingyu Ge, Chunping Li, and Qin Yin
819
Variation Trend Analysis of Groundwater Depth in Area of Well Irrigation in Sanjiang Plain Based on Wavelet Neural Network . . . . . . . . . Hong Ding, Dong Liu, and Fei-fei Zhao
829
A Petri-Net Modeling Method of Agent’s Belief-Desire-Intention and Its Application in Logistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Weifeng Zhu and Qi Fei
837
Supply Chain Flexibility Assessment by Multivariate Regression and Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ananda S. Jeeva and William W. Guo
845
An Intelligent Mobile Location-Aware Book Recommendation System with Map-Based Guidance That Enhances Problem-Based Learning in Libraries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chin-Ming Chen and Yu-Chieh Yang
853
Applying Least Squares Support Vector Regression with Genetic Algorithms for Radio-Wave Path-Loss Prediction in Suburban Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kuo-Ping Lin, Kuo-Chen Hung, Jen-Chang Lin, Chi-Kai Wang, and Ping-Feng Pai
861
A Probabilistic Neural Network Approach to Modeling the Impact of Tobacco Control Policies by Gender . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Xiaojiang Ding, Chung-Hsing Yeh, and Susan Bedingfield
869
The BPNN-Fuzzy Logic Pre-control of an Underground Project in City Center of Shanghai . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Zhenzhen Yin, Youliang Chen, and Peng Wang
877
Optimal Parameter Inversion of Marine Water Quality Model Using a BPNN Data-Driven Model — A Case Study on DIN . . . . . . . . . . . . . . . . . Mingchang Li, Bin Zhou, Shuxiu Liang, and Zhaochen Sun
885
Determination of Sugar Content of Instant Milk-Tea Using Effective Wavelengths and Least Squares-Support Vector Machine . . . . . . . . . . . . . . Fei Liu and Yong He
893
Sports Video Summarization Based on Salient Motion Entropy and Information Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Bo-Wei Chen, Jhing-Fa Wang, Jia-Ching Wang, and Chen-Yu Chen
901
XXVI
Table of Contents
A Neural Network Based Algorithm for the Retrieval of Precipitable Water Vapor from MODIS Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Shenglan Zhang, Lisheng Xu, Jilie Ding, Hailei Liu, and Xiaobo Deng A Neural Network Based Approach to Wind Energy Yield Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Piers R.J. Campbell, Faheem Ahmed, Haydar Fathulla, and Ahmad D. Jaffar
909
917
Research on New Intelligent Business-Oriented Decision-Making Model Based on MA and GA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Weijin Jiang and Qing Jiang
925
Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
933
A Novel Prediction Mechanism with Modified Data Mining Technique for Call Admission Control in Wireless Cellular Network Chen-Feng Wu, Yu-Teng Chang, Chih-Yao Lo, and Han-Sheng Zhuang Department of Information Management, Yu Da University, No. 168, Hsueh-fu Rd, Tanwen Village, Chaochiao Township, Miaoli County, 361 Taiwan {cfwu,cyt,jacklo,97104504}@ydu.edu.tw
Abstract. It is an important issue to allocate appropriate resources to mobile calls for wireless cellular networks owe to scarce wireless spectrums. The call admission control (CAC) will maintain better performance metrics of mobile call such as call dropping probability (CDP) and call blocking probability (CBP) if the future utilization of wireless spectrums can be predicted and provided to the decision of CAC. Therefore, a prediction mechanism which can predict most information such as system utilization is proposed in this paper. The techniques of data mining and pattern matching which adopts gradient to fuzz time series data for representations of chain code are applied to mining a possible repetitive pattern. Our proposed prediction mechanism can provide prediction information in advance whether the repetitive time series pattern of information exists or not. Furthermore, an update of confident level will be conducted continuously for performing each prediction in the proposed scheme. Our proposed mechanism is developed and tested with four cases which can be regarded as using scenarios of wireless cellular network. The experimental results show that the proposed scheme can capture repetitive time series patterns and perform following predictions with these repetitive time series patterns. Besides, the required storage is less than traditional schemes and lower computation power is required for the proposed scheme. Keywords: Call Admission Control (CAC), data mining, time series, pattern matching, prediction.
1 Introductions Although there has been a rapid development in wireless cellular communications, the QoS guarantee remains one of the most challenging issues [1]. One of the key elements in providing QoS guarantees is an effective CAC policy, which not only has to ensure that the network meets the QoS of the newly arriving calls if accepted, but also guarantees that the QoS of the existing calls does not deteriorate. The variable user mobility has made that it becomes more complex to predict the appropriate cell for handoff. The past research [2] showed the impact of mobility on cellular network and provided a modeling method for configuring cellular networks to study the dynamics of mobility. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 1–9. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
2
C.-F. Wu et al.
The improvement of radio bandwidth is always thought as a dynamic channel (code) allocation problem in literatures [3]. Although there were some schemes for bandwidth reservation proposed to reduce the CDP, such as the study [4], seldom literature has developed to satisfy QoS and lower CBP issues at the same time. Furthermore, the past researches mentioned at the above focused on individual mobility prediction, and they may cause mass load focusing on the MSC. According to the study [5], final information is a prediction of users’ number in a given cell, and it leads to use a global approach that only observes variations of system utilization and users’ flows. There are many advantages for [5]: it does not require any control message and additional load for the MSC; cells generate their own statistics independently from others; it is sensitive to geographical constrains and to users’ common habits. According to [12], the final information is a quantity prediction of users in a given cell, and it leads to use a global approach that only observes variations of system utilization and users’ flows. There are many advantages of the scheme presented in [12]: it does not require any control message and additional load for the MSC; cells generate their own statistics independently from others; it is sensitive to geographical constraints and to users’ common habits. Besides, the concept of aggregated history has been also applied to acquire user mobility profile in [13] so that a user mobility profile framework is developed for estimating service patterns and tracking mobile users, including descriptions of location, mobility, and service requirements. Therefore, in order to provide suitable statistical prediction information which may be system utilization, CBP, or other system resources to the CAC, a prediction mechanism which can predict most information is proposed in this paper.
2 Related Works A suitable prediction mechanism can help the CAC scheme to do better decision for call admission, thus the integration of CAC and prediction will effectively reduce the CDP to meet the QoS requirement. 2.1 CAC Overviews Although CAC schemes can be classified into centralized ones that make the decision of call admission in the MSC and distributed ones that make the decision of call admission in individual cell locally, most of prior researches [14], [15] that were based on different theories focused on the developments of centralized CAC schemes. Instead of using centralized CAC schemes, distributed CAC schemes have been proposed in [16], [17]. Based on the local information, Zhang et al. [16] have applied two prediction mechanisms that were historic data and time series prediction to CAC scheme in noncollaborative manner. Unfortunately, the overheads that are caused by predicting the appropriate information for making CAC decision have been still existed. A clustering prediction scheme that is based on data mining concept for user mobility prediction has been proposed in [17], but the related information from neighboring cells is still required to participate in mobility prediction and more computation loads and storage for data mining process are also required. Although the above researches exhibit that the distributed CAC schemes can reduce the traffic loads
A Novel Prediction Mechanism with Modified Data Mining Technique
3
concentrating on the MSC indeed, some drawbacks, such as an efficient prediction scheme, still exist for themselves. 2.2 Time Series Prediction One of the typical statistic models that based on statistical concepts and principles is Box-Jenkins forecasting model [6]. Although a wide spectrum of time series behavior can be modeled by such models, they can not model well when there are noises in the time series such as inaccurate or incomplete data. In the research [7], neural networks that are one of the well-known artificial intelligent techniques have been claimed to be able to achieve better prediction compared to statistical models. However, the study [8] shows that there exit some drawbacks for neural networks, such as hard to pre-select the system architecture, spending much training time and lacking knowledge representation techniques. Besides, the study [9] applies the data mining concept to perform time series prediction. Both adaptive association rule and pattern matching techniques are employed by [9] to mine for repetitive patterns, which can be used to understand the underlying utilization of system utilization, in the time series. Although this model can perform life long adaptation and reinforcement learning for continuous prediction, the disadvantage is the requirement for intensive computation power and huge storage. The proposed prediction module that is based on time series prediction can provide the appropriate information for the CAC algorithm to make the decision of call admission. Comparing to the traditional data mining process for time series prediction scheme, the proposed prediction method reduces calculation overheads and storage requirements significantly.
3 Methodology Data mining is always applied to perform the prediction by analyzing data and look for data behavior pattern owing to its four basic goals: prediction, identification, classification, and optimization. There exist many knowledge representations, and the association rule is commonly used. Association rules correlate the presence of a set of items with another range of values for another set of variables. 3.1 The Proposed Prediction Method The process flow of the proposed prediction method can be summarized in four steps, as shown as follows: Procedure of Prediction Begin Step 1: Time-series data analysis; Step 2: Time-series prediction; Step 3: Compensation prediction; Step 4: Prediction selection; End
4
C.-F. Wu et al.
Step 1: Time Series Data Analysis This step is response for performing data pre-processing and data mining on time series data. The purpose of data pre-processing is to convert the time series data into gradient code which is defined by the application characteristics of prediction. The data mining is to generate chain code pattern, and the learning results are stored into database which will be used for prediction task. The time series data is represented in a chain code format to keep track of the rate-of-change of time series. The rate-of-change of a time series is obtained by calculating the difference between each two temporal point. The following phase of the analysis step performs data mining to detect the periodicity of patterns and other relative information. The process of data mining is based on Eq. (1) to find iterative pattern. ⎛ R (d ) = set ⎜ ⎝
i+ n −1
∑
t=i
⎞ G (t + d ) − G (t ) ⎟ ⎠
(1)
The set function decides whether a repetitive pattern exists or not. If there exists a repetitive pattern of length n that is represented as the size of the pattern window, the n differences between each gradient vectors must be all zero with distance d, where d is the delay (d >= n) and i is the time of the first point in the preceding pattern. Pattern with different distances between the pattern length and the delay are managed in different list. The occurrence times of periodic patterns and the average of compensation values will be recorded in the database. Step 2: Time-Series Prediction After data mining, the prediction step will make use of periodic patterns to predict possible time series data in future. If a segment of a repetitive time series pattern matches with the inputs, it may happen that the following segment of the particular repetitive pattern has the possibility of reoccurrence. The prediction result with highest probability which is decided by counts of these patterns will be adopted. Because the prediction value is only a gradient code, the final output should be adjusted by the average compensation value and decoded. Fig. 1 illustrates the process of performing prediction task. The example that is with prefix pattern [1, 2, -4, 3] of gradient code wants to predict the value G(t+1), and G(t+1) = -2 with the highest probability 54% will be predicted. Moreover, the prediction gradient code will be passed through the compensation and decoding process to obtain the prediction times series data X(t+1).
Fig. 1. An example of performing prediction
A Novel Prediction Mechanism with Modified Data Mining Technique
5
Step 3: Compensation Prediction In order to provide more accurate prediction information, the compensation prediction that is based on Eq. (2) to perform prediction generates different prediction values to supply selection step for choosing the most appropriate one. C P ( t + 1) = α × ( C P ( t ) + ε t ) + (1 − α ) × X ( t )
ε
t
=
1 t
t
∑
i=1
ε
i
ε i = X (i) − i X (i )
(2) (3) (4)
For Eq. (2), the compensation prediction is concerned with the actual time series data X(t) at current time t and the previous prediction result CP(t) at time t-1. The parameter α is used to decide the weight of the value CP(t), and ε t is average difference that is the distance from actual value X(i) to prediction value i X (i ) and is defined by Eq. (3). Step 4: Prediction Section This step is responsible for determining the final prediction result between time series prediction and compensation prediction and updating the confident levels of pattern lists. Because the proposed scheme can dynamically remind the confidence level of pattern lists as time goes on, the pattern lists with low confidence level will be gradually neglected.
Fig. 2. An example of updating the pattern list
The process of updating the pattern list is illustrated in Fig. 2. At time t+1, the actual time series data X(t+1) that is with former pattern [1, 2, 4, -3] is received and transferred into G(t+1) by pre-processing process. If the actual value of G(t+1) is -2, 1, or 6, the count value of the existing pattern list will increase, and the corresponding confident level will be risen at next prediction. A new pattern will be added to the pattern list, if G(t+1) is not an expected value, such as 8. The prediction selection step dynamically adopts one of the prediction results that are from both compensation prediction and time series prediction. For compensation prediction step, the compensation algorithm generates several results with various coefficients αs, and the best prediction result will be chosen according to the average distance.
6
C.-F. Wu et al.
3.2 Design Considerations There are two defined thresholds, the distance threshold and the count threshold, dominating the accuracy of the time series prediction. If the difference between the prediction value and the actual value is larger than the distance threshold, the prediction will be regarded as a prediction miss. The time series prediction will be thought of being inaccurate when the number of continuous prediction misses is more than the count threshold. For compensation prediction step, the compensation algorithm generates several results with various coefficients of αs, and the best prediction result will be chosen according to the average distance. If there has not been any periodic pattern found or the accuracy of the time series prediction is too low, then the best prediction value that is generated by the compensation prediction will be regarded as the output. Otherwise, the prediction result of the time series patterns will be selected and sent out, and the prediction process is also completed simultaneously by the proposed prediction system. For reducing the consumption of computation capability and the demand of storage, some techniques are applied to the implementation. A list that records the possible pattern length is maintained in the database. The pattern lengths with the higher occurrence probability will earlier become the window size of performing pattern detection, and it will avoid running the detecting algorithm for every single length pattern. All of the system average values are computed by Eq.(5) instead of Eq.(6). A v g (n ) =
n − 1 × A v g (n − 1) + n
A v g (n ) =
1 × n
1 × v a lu e ( n ) n
(5)
n
∑
i = 1
v a lu e (i)
(6)
Where Avg(n) is the average of n number of values, and value(n) is the last value in the data stream. Eq.(6) needs to record all of the n values in database; on the contrary, Eq.(5) reduces the storage demand and less computation is needed.
4 Experimental Results There are four artificial test cases designed for verifying the accuracy of the proposed prediction module for system utilization for a given cell, and the time scale is in minute for all simulations. The purposes of these four test cases are to test how well the prediction method can detect the periodic patterns and perform prediction using these patterns, even though the periodic pattern is not found. The simulations start at 00:00, and advanced time unit is 00:01. The simulation results are shown in Fig. 3 to Fig. 6. The prediction method can provide the next three prediction value X(t+1), X(t+2), and X(t+3) at the time t. For the first case, a periodic pattern is successfully found at 00:08, as shown in Fig. 3, since the pattern ‘4, 2,-1, -4’ has appeared twice. The prediction value which produced by the compensation prediction step with the most suitable α is adopted by the prediction method before the pattern detected. While the periodic pattern has been detected, the time series prediction step applies it to perform prediction.
A Novel Prediction Mechanism with Modified Data Mining Technique
7
For the second case, the simulation results of the second case that is the first case of time series added a little bit of random oscillation is shown in Fig. 4. At the same time 00:08, the same pattern has been detected and the prediction value from compensation step is also replaced by the result of time series prediction step at the following time. Obviously, the compensation values of each item in the pattern list are different, and the prediction step adjusts the expected prediction value with the average compensation values. The third case is applied to test a suddenly continuous scenario as shown in Fig. 5. The front part of this time series is the same as the first test case except at 00:09 the utilization increases 10% suddenly and lasts in the following time series. The identical pattern can be detected but the prediction value that is derived from the periodic pattern causes one miss. Whatever, the prediction module can restore the accuracy immediately. In the fourth case, the time series prediction sub-module identifies successfully no existence of periodic pattern found. Thus, the compensation prediction step is instead of time series prediction step to provide the prediction value in the prediction method. Fig. 6 shows that the compensation prediction step can provide better prediction values at the part with fixed and smaller variation. According to the simulation results of the four test cases, the proposed prediction method can perform well in all kinds of time series data. Once the repetitive patterns exist in the time series, the step of time series data analysis can identify the pattern quickly, and accurate prediction values can be provided by the time series prediction step. Contradictorily, if there is not any periodic pattern existing, the proposed prediction module not only successfully identifies that the time series is random but also provides continuously the prediction values in the following times. Actual Value Predict at t-1 Predict at t-2 Predict at t-3
Time series data for utilization (simple case) 100 90
Utilization
80 70 60 50 40
00 :1 8
00 :1 9
00 :1 6
00 :1 7
00 :1 4
00 :1 5
00 :1 2
00 :1 3
00 :1 1
00 :0 9
00 :1 0
00 :0 8
00 :0 7
00 :0 6
00 :0 5
00 :0 3
00 :0 4
00 :0 1
00 :0 2
00 :0 0
30
Time
Fig. 3. The simulation result for system utilization of simple case Actual Value Predict at t-1 Predict at t-2 Predict at t-3
Time series data for utilization (thrashing case) 100 90
70 60 50 40
00 :1 9
00 :1 7
00 :1 8
00 :1 6
00 :1 5
00 :1 4
00 :1 3
00 :1 2
00 :1 0
00 :1 1
00 :0 9
00 :0 8
00 :0 7
00 :0 6
00 :0 5
00 :0 4
00 :0 3
00 :0 2
00 :0 1
30
00 :0 0
Utilization
80
Time
Fig. 4. The simulation result for system utilization of thrashing case
8
C.-F. Wu et al. Actual Value Predict at t-1 Predict at t-2 Predict at t-3
Time series data for utilization (occupation case-2) 100 90
Utilization
80 70 60 50 40
00 :1 9
00 :1 7
00 :1 8
00 :1 6
00 :1 4
00 :1 5
00 :1 3
00 :1 1
00 :1 2
00 :1 0
00 :0 9
00 :0 7
00 :0 8
00 :0 6
00 :0 4
00 :0 5
00 :0 3
00 :0 1
00 :0 2
00 :0 0
30
Time
Fig. 5. The simulation result for system utilization of occupation case Actual Value Predict at t-1 Predict at t-2 Predict at t-3
Time series data for utilization (random case)
oatin iliztU
100 90 80 70 60 50 40 30 20 10 0
:00 00
:01 00
:02 00
:03 00
:04 00
:05 00
:06 00
:07 00
:08 00
:09 00
:10 00
Time
:11 00
:12 00
:13 00
:14 00
:15 00
:16 00
:17 00
:18 00
:19 00
Fig. 6. The simulation result for system utilization of random case
5 Conclusions The simulation results of the prediction method show that the proposed scheme works well for the four different kinds of test cases. According to the simulation results of the time series prediction method, the proposed prediction method can estimate the required resources accurately in advance, so it could be applied in several fields that need to estimate time series based information beforehand for further manipulations, such as applications in finance, Internet traffic, grid computing, and so on.
Acknowledgements This work was supported in part by the National Science Council of Taiwan under the Grants NSC97-2221-E-412-001.
References 1. Akyildiz, I.F., McNair, J., Ho, J., Uzunaliogluh, H., Wang, W.: Mobility management in next-generation wireless systems. In: Proc. of IEEE ICON 1999, vol. 87, pp. 1347–1384. IEEE Press, Los Alamitos (1999) 2. Zaidi, Z.R., Mark, B.L.: Mobility estimation for wireless networks based on an autoregressive model. In: Proc. of IEEE GLOBECOM 2004, vol. 6, pp. 3405–3409. IEEE Press, Los Alamitos (2004)
A Novel Prediction Mechanism with Modified Data Mining Technique
9
3. Kim, Y.C., Lee, D.E., Lee, B.J., Kim, Y.S.: Dynamic channel reservation based on mobility in wireless ATM networks. IEEE Communication magazine, 47–51 (1999) 4. Tajima, J., Imamura, K.: A Strategy for Flexible Channel Assignment in Mobile Communication Systems. IEEE Transactions on Vehicular Technology 37 (1988) 5. Perato, L., Agha, K.A.: Handover Prediction: User Approach versus Cell Approach. In: Proc. of Mobile and Wireless Communications Network 4th International Workshop, pp. 9–11. IEEE Press, Los Alamitos (2002) 6. William, W., Wei, S.: Time Series Analysis: Univariate and Multivariate Methods. Addison Wesley Publication, Reading (1999) 7. Kamitsuji, S., Shibata, R.: Effectiveness of Stochastic Neural Network for Prediction of Fall or Rise of TOPIX. Research Paper, Keio University (2004) 8. Kasabov, N.: Evolving Connectionist Systems. Springer-Verlag London Limited (2003) 9. Yaik, O.B., Yong, C.H., Haron, F.: Time Series Prediction Using Adaptive Association Rules. In: Proc. Of the First International Conference on Distributed Frameworks for Multimedia Applications (DFMA 2005), pp. 310–314. IEEE Press, Los Alamitos (2005) 10. Chen, H., Kumar, S., Kuo, C.-C.J.: Dynamic Call Admission Control Scheme for QoS Priority Handoff in Multimedia Cellular System. In: Proc. of IEEE Wireless Communications and Networking Conference WCNC 2002, vol. 1, pp. 114–118. IEEE Press, Los Alamitos (2002) 11. Gaasvik, P.-O., Cornefjord, M., Svensson, V.: Different methods of giving priority to handoff traffic in a mobile telephone system with directed retry. In: Proc. Of 41st IEEE Vehicular Technology Conference ‘Gateway to the Future Technology in Motion’, pp. 549–553. IEEE Press, Los Alamitos (1991) 12. Hong, D., Rapport, S.S.: Traffic model and performance analysis for cellular mobile radiotelephone systems with prioritized and nonprioritized handoff procedures. IEEE Trans. on Vehicular Technology 35, 77–92 (1986) 13. Akyildiz, I.F., Wang, W.: The predictive user mobility profile framework for wireless multimedia networks. IEEE/ACM Transactions on Networking 12(6), 1021–1035 (2004) 14. Levine, D.A., Akyildiz, I.F., Naghshineh, M.: A resource estimation and call admission algorithm for wireless multimedia networks using the shadow cluster concept. IEEE/ACM Transactions on Networking 5(1), 1–12 (1997) 15. Hou, J., Yang, J., Papavassiliou, S.: Integration of pricing with call admission control to meet QoS requirements in cellular networks. IEEE Transactions on Parallel and Distributed System 19(9), 898–910 (2002) 16. Zhang, T., et al.: Local predictive resource reservation for handoff in multimedia wireless IP networks. IEEE Journal on selected areas in Communications 19(10), 1931–1941 (2001) 17. Tsiligaridis, J., Acharya, R.: A clustering prediction scheme for wireless cellular network. In: Proc. of the 2005 International Symposium on Collaborative Technologies and Systems, pp. 298–304. IEEE Press, Los Alamitos (2005)
The Study of Forecasting Model of Rock Burst for Acoustic Emission Based on BP Neural Network and Catastrophe Theory Yunyun Xu1,2 and Dongqiang Xu2 1 2
Department of Civil Engineering of Tangshan College, Hebei Tangshan 063009, China College of Civil Engineering, Hebei University of Technology, Tianjin 300132, China
Abstract. Forecasting model of the rate of rock burst acoustic emission time series has been established by BP neural network, and it is combined with the catastrophe theory to determine whether the rock burst. And then the experimental data recorded are used for examining the model. The results show that the degree of prediction accuracy is high, and it proves that the prediction model of rock burst is feasible. Keywords: rock burst, acoustic emission, forecasting model, BP neural network, catastrophe theory.
1 Introduction Rock burst is a usual geologic disaster, which occurs in rock of underground works of high ground stress. The reason for rock burst is that stress field in wall rock redistributes, caused by unloading in the process of excavation, and it results in elastic strain energy stored in hard and brittle wall rock being released suddenly, and wall rock being burst, loosen, peeled, ejected or even thrown[1]. The process of rock burst preparation and development is that surrounding rocks in mining space or roadway deform rapidly and fracture, from flow to sudden change. Rock acoustic emission is an intuitive performance of rock micro breakdown, and it is caused by rock crack propagation under load [2]. Acoustic emission method is to predict the occurrence of rock burst based on the parameters in each stage of deformation such as acoustic emission rate, energy rate, and wave shape, and so on, which is the most direct means of detecting and predicting rock burst. Prediction of rock burst is to mainly determine the possibility of rock burst in a large area in the next period[3]. For a long time, the analysis of acoustic emission time series monitored and forecasting trends has become the primary means of predicting rock burst [4-5]. Gray system and the neural network model are the most important prediction methods [6]. But prediction results show that the gray model is suitable for exponential growth in time-series, and in dealing with geotechnical engineering timeseries prediction problem, gray model should be used with caution [7-8].In this paper, the rock burst prediction method by means of neural network theory and catastrophe theory is discussed on the basis of the test data of rock burst of acoustic emission. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 11–19. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
12
Y. Xu and D. Xu
2 The Establishment of Forecasting Model of Time Series Neural Network 2.1 The Introduction of BP Neural Network Error Back Propagation(BP) neural network is one of the more mature and widely used models. It is a former feed-forward network, mainly composed of input layers, hidden layers and output layers [9]. Network structure is shown in Fig.1.
Fig. 1. BP network topology
BP network is a kind of guiding learning network, which relies on adjusting the connection weights between layers of neurons (nodes) to "memory" the learning samples. And it automatically creates the highly nonlinear mapping relationship between the inputs and outputs by studying a large number of samples. It has been proved that a three-layer BP network includes m input nodes, 2m +1 hidden layer nodes and n output nodes, which can achieve any non-linear mapping from Rm to Rn [9]. 2.2 Process of Network Training Learning process of BP algorithm is composed of front-propagation and backpropagation. In the process of front-propagation, the input information is dealt with the hidden units layer by layer and transmitted to the output layers. If the output layer can not get the desired output, the back-propagation process is transferred, and the errors between actual value and network output return along the path to the original links. By modifying the link weights of every layer’s neurons, the errors reduce, and then the process of front- propagation is transferred, iterating until the error is less than the value given. (1) Design network structure to adapt to the issue (network layers and nodes of network layers), and choose reasonable network learning parameters (learning rate η and impulse parameters α ) . (2) Set the initial value of network connection weights and thresholds w ji (0) , θ j (0) , which are the random number in the range of [-1,+1].
The Study of Forecasting Model of Rock Burst for Acoustic Emission
13
p ( p = 1,2, " , K ) by the steps of (4) - (8). Sample data include input vector x = ( x1 , x 2 , " , x n ) and desired output vector (3) Deal with every study sample
t p = (t p1 , t p 2 , " , t pm ) . (4) Calculate the actual outputs of nodes layer by layer in forward direction. n
O pj = f j (∑ w ji O pj + θ j ) i =1
,
f ( x) =
1 1 + e−x
(1)
In the Formula 1, f (x ) is an active function of the S-type. (5) Calculate output network errors.
⎤ 1 k ⎡1 m E = ∑ ⎢ ∑ (t pj − O pj ) 2 ⎥ k p =1 ⎣ 2 j =1 ⎦ (6) If
(2)
E ≤ EG (the system average error margin) or iterations number reaches the
specified number, the study ends. Otherwise, the back-propagation turns to step (7). (7) Calculate reversely the error signal of each node in network layer by layer.
δ pj = O pj (1 − O pj )(t pj − O pj )
(3)
m
δ pj = O pj (1 − O pj )∑ δ pl w jl
(4)
l =1
(8) Modify the network connection weights and node thresholds.
Δw ji (t + 1) = η × δ pj × O pj + αΔw ji (t ) Δθ j (t + 1) = η × δ pj + αΔθ j (t )
,
(5)
(9) Turn to step (3). 2.3 Establishment of Time Series Neural NetworkForecasting Model Suppose that there is a time-series X (i ) , i = 1,2, " , N ; and N is the number of observation points. Geotechnical engineering time-series forecasting model can be described as the following:
X (t ) = ϕ[ X (t − 1), " , x(t − p )] In the formula, ϕ [.] is a non-linear function, and p is the order of the model. The building process of geotechnical engineering time-series forecasting model is to find ϕ [.] [10].
14
Y. Xu and D. Xu
From the time series X (i ) ,
i = 1,2, " , N , N − p samples can be constructed.
Input Layer Vector No1 Sample No t − p Sample No N − p Sample
Output Layer Vector
X (1), " , X ( p − 1), X ( p ) X (t − p ), " , X (t − 2), X (t − 1) X ( N − p ), " , X ( N − 2), X ( N − 1)
X ( p + 1) X (t ) X (N )
N − p samples constructed are used for the network shown in Fig.1 for learning and training, and then a stable network structure, connection weights and thresholds can be got. The time series forecasting model of geotechnical engineering is established based on artificial neural network. The input vectors are inputted into the network, using the stable network structure, connection weights and thresholds, so that the samples can be predicted.
3 Catastrophe Cusp Model and Yield Mechanism Cusp theory of the catastrophe theory is the most widely used in the range of mechanics [11]. It has two control variables and a state variable. Its regular function of potential function in the form is as the following:
v ( x) = x 4 + px 2 + qx
(6)
So phase space is three-dimensional, in which x is the state variable, p and q are control variables, and x is the time variable. The corresponding set M of all critical points for E, is called the equilibrium surface (shown in Fig. 2). M satisfying:
grad xV = 4 x 3 + 2 px + q = 0
Fig. 2. Balance surface and control surface
(7)
The Study of Forecasting Model of Rock Burst for Acoustic Emission
15
The graphics of M in ( x, p, q ) space is a smooth surface with a fold, so in different regions, the number of equilibrium position is different. It is easy to be proved that the potential function corresponding to the middle lobe takes the maximum (that is to say, grad x ( grad xV ) < 0 ), whose equilibrium position is unstable; the potential function corresponding to the upper and lower lobe takes the minimum, whose equilibrium position is stable. Clearly, there are vertical tangent points at the curve surface, that is, near the point to satisfy Formula 8 in Fig.2, the number of equilibrium position is different. These points are called sudden change points or singular points, whose projection at the control variables p, q plane constitutes a bifurcation set B (also called bifurcation). This set includes all the state variables, and makes x produce a sudden change.
8 p 3 + 27q 2 = 0 grad x ( grad xV ) = 12 x 2 + 2 p = 0 Eliminate
(8)
x from Formula7 and Formula 8, and the equation is got: 8 p 3 + 27q 2 = 0
(9)
Through the above analysis, the instability discriminant can be drawn as the following:
F = 8 p 3 + 27 q 2
(10)
If control point ( p, q ) changes, corresponding point position changes on the curve surface accordingly. But when the control points trajectory cross over the bifurcation set 8 p + 27 q = 0 , corresponding points must jump across the middle of leaves, that is, rock mass is unstable. The relation is described as the following: 3
If If If
2
F > 0 , the rock mass near measuring point is steady; F = 0 , the rock mass near measuring point is critical; F < 0 , the rock mass near measuring point is unstable.
From Formula 7, it can be seen that the number sequences increase in the form of an exponent, but this kind of exponential function is not suitable for the requirements of catastrophe theory. Because any single variable function can be spread by the Taylor formula, v(t ) can be spread by the Taylor series, and truncated to 5. Then we can get: 4
v(t ) = ∑ ai t i i =0
(11)
16
Y. Xu and D. Xu
∂ i v(t ) In Formula 11, a i = | t =0 , and suppose x = t − n by using of variable ∂t i substitution, we can get:
v( x) = x 4 + px 2 + qx
(12)
For the consistency of the neural network model and the mutation model, it can be obtained:
p = −6n 2 + a 2 / a 5 n = a3 / 5a5
(13)
p = −6 n 2 + a 2 / a 5
(14)
q = 8n 2 − 2na 2 / a 5 + a1 / a 5
(15)
Inserting AE parameters at the moment to predict in Formula 12, adopting the leastsquares method, we can get a i . And then, inserting a i in Formula 13 to Formula 15, we can get the value of p and q , which can be inserted in Formula 10 to calculate the value of F. At last, we can forecast the rock burst circumstances based on the value of F.
4 Application of BP Neural Network – Mutation Model in Rock Burst Prediction of Acoustic Emission To verify the accuracy of the neural network - mutation model in rock burst prediction, we use the data recorded on the experimental to test the model. The white marble specimens, whose integrity is good, are used for testing. In the entire failure process of the specimen, the acoustic emission event rate has a bimodal nature and when the stress reaches about 80% of ultimate stress, it reaches high-value soon until the stress reaches 95% ultimate stress or so. During this time from 80% to 95% ultimate stress, the acoustic emission event rate fluctuations little and has an upward trend. The acoustic emission parameters of white marble specimens after 85% of ultimate stress for some time are shown in Table 1 for every 5 seconds. The corresponding data of rate of acoustic emission time series at 190-260second are used as a model; and the corresponding data of rate of acoustic emission time series at 265-270second are used for testing. We insert the modeling data in the network to operate. The model order is 3, the input layer nodes number using the network is 2, hidden layer node number s is 5, and output layer nodes number is 1. After 15000 iterations, the network converges, and the steady network structure,
The Study of Forecasting Model of Rock Burst for Acoustic Emission
17
connection weights and threshold are got. The fitted and measured values are compared in Table 1. Table 1. Comparison of fitted and measured values Monitoring Time (s)
Monitoring value-AE event rate (time/m)
Predicting value-AE event rate (time/m)
Relative error (%)
190
85
87
2.35
195
90
93
-3.23
200
100
100
0
205
109
103
5.50
210
98
105
-7.14
215
105
107
-1.90
220
111
108
-0.93
225
113
110
2.70
230
109
112
-2.75
235
115
113
1.74
250
115
118
2.60
255
118
120
-1.69
260
124
126
-1.61
Table 2. Predictions comparison Monitoring time (s)
Measured value
Prediction value
Relative error%
265
129
133
-3.10
270
135
132
2.22
From Table 1, the neural network fitting is good, and the relative average error is only 2.62%. The rates of acoustic emission at 265s and 270s are predicted, and compared with the record values to test the correctness of the model. Those results are shown in Table2. It can be seen that for the samples without training, the neural network model has very high prediction accuracy and it is an effective way to solve the problem of rock burst time series. Use AE parameters shown in Table1 at the moment to calculate the value of F for predicting the burst of rock. The result is shown in Table 3.
18
Y. Xu and D. Xu Table 3. Rock burst prediction
Time(s)
p
q
F
Rock burst prediction
245
-5.7
51.69
56096.9
No burst
250
-5.65
-58.55
91091.9
No burst
255
-6.65
-29.30
20837.2
No burst
260
-7.695
-105.36
290512.1
No burst
265
-98.81
-35.08
-7686526.1
burst
From the values shown in Table 3, it can be seen that rock may burst in holes at 265 seconds when the specimens is under load. In reality, at 269 seconds the rock fragments in holes appear, followed by rupture of the specimens. It shows that the forecasting results are very consistent with the actual situation. Several other specimens are used to predict, and the forecast is successful.
5 Conclusion The studies show that artificial neural network method for prediction of rock burst has provided a new and effective method. In this paper, the forecasting model of the rate of rock burst acoustic emission time series has been established by using BP neural network, and it was combined with the catastrophe theory to determine whether the rock burst. The results show that the degree of prediction accuracy is high, and it proves that the prediction model of rock burst is feasible, and has important application value. The model described in this article can forecast the occurrence of rock burst, but it can not predict the level and scale of rock burst. Many factors must be considered comprehensively to predict rock burst, so it is necessary to develop rock burst prediction experts system to grasp comprehensively and accurately the laws of rock burst, and to take appropriate measures to prevent rock burst.
References 1. Wang, W., W.E.Y.: Summary and Development Analysis of Rock Burst Prediction Technology (in Chinese). Mining Safety and Environmental Protection 30(4), 12–15 (2003) 2. Zhang, Y.B., Kang, Z.Q., Jiang, G.H., Xu, D.Q.: Rock Burst Mechanism Based on Rock Damage and Acoustic Emission Theory (in Chinese). Metal Mine 12, 79–82 (2007) 3. Pan, C.L., Cao, X.B., Cao, P.: Rock Burst Prediction Method (in Chinese). Non-ferrous Mining and Metallurgy 13(6), 3–5 (1997)
The Study of Forecasting Model of Rock Burst for Acoustic Emission
19
4. Tang, Q.B., Cao, P.: The Analysis of Rock Burst Prediction Method (in Chinese). Jiang Xi Nonferrous Metals 6, 2–4 (1997) 5. Yang, T., Li, G.W.: Study Based on Prior Knowledge of the Rock Burst Prediction (in Chinese). Chinese Journal of Rock Mechanics and Engineering 19(4), 429–431 (2000) 6. Peng, Q., Zhang, R., Xie, H.P.: Prediction Model for Rock Burst Based on Acoustic Emission Time Series (in Chinese). Rock and Soil Mechanic 30(5), 1436–1440 (2009) 7. Lu, X.Q., Liang, X.H., Lu, X.J.: Neural Network and Its Application to Predication of Nonlinear Time Sequence (in Chinese). Systems Engineering-theory and Practice 6, 97–99 (1997) 8. Zhang, Y.X., Lu, S.L.: Application of Artificial Neural Network to Engineering Time Sequence Forecasting Problem (in Chinese). Journal of NanJing University of Science and Technology 21(6), 522–525 (1997) 9. Li, X.A., Zhang, X.H.: Introduction to Neural Network and Neural Computation (in Chinese). Northwestern Polytechnical University Press, Xi an (1994) 10. Zhang, Y.X.: Primary Research on Forecasting Problem of Time Sequence in Geotechnical Engineering (in Chinese). Chinese Journal of Rock Mechanics and Engineering 17(5), 552–558 (1998) 11. Ling, F.H.: Catastrophe Theory and Its Application (in Chinese). Shanghai Jiaotong University Press, Shang Hai (1987)
Application of Wavelet Neural Network to Prediction of Water Content in Crude Oil Huifen Niu1, Cuiling Liu1, Jinqi Wang2, and Xiaowen Sun1 1
Computer and Information Engineering College, Beijing Technology and Business University, 100048 Beijing, China 2 Earth Sciences College, Daqing Petroleum Institute, 163318 Daqing, China
[email protected],
[email protected],
[email protected],
[email protected]
Abstract. In the course of oilfield development, accurate measurement of water content in crude oil has always been playing important role in practicing development adjustment and enhancing the effects of stimulation operation, and moreover it determines the development perspectives of oilfield. After research of the method of measuring the rate of water content, and the non-linear mapping relation between the rate of water content and impact factors of crude oil, a model of predicting the rate of water content in crude oil about vertical well based on wavelet neural network is proposed. Results of the simulation in MATLAB indicated that the way of WNN (wavelet neural network) has better convergent rate, prediction precision, learning ability and generalization ability than the traditional BP neural network. The method of WNN can predict the water content in crude oil with high precision and it owns much more powerful theoretical guide and much better application effects. It will have a broad application in the future. Keywords: wavelet neural network, BP neural network, crude oil, water content, prediction model.
1 Introduction The water content of oil is the parameter which is not only important in oil gas and oilfield measure of production, but also indispensable in researching the development perspectives of oilfield. If the inspection accuracy of the rate of water content in crude oil can be improved, the production parameters will be more optimized and production accounting will be more reasonable during the storage and transportation process of the oilfield and oil gas[1]. Lots of work on the rate of water content in crude oil has been researched by many domestic and foreign scholars for a long time. In order to improve the measure precision, many kinds of methods have been proposed and many types of equipment about testing the rate of the water content have been designed. But it is hard to set up accurate mathematical models for all situations due to the complex nonlinear relationship of the factors which cause the effect of measuring the rate of the water content in crude and complicated work conditions. With the rapid development of computer and artificial intelligence technology, the prediction models about the rate of Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 21–27. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
22
H. Niu et al.
the water content in crude oil based on ANN(artificial neural network) have been put forward recently[2]. The ANN is suitable for the non-linear prediction, and in theory it can approach the non-linear sequences in any precision. But the ANN is hard to decide the structure of the net in the application, and it like slow convergence of learning and liability of dropping into local minimum [3]. The WNN has been a new analysis method of modeling in recent years, which is established with the combination of the wavelet transformation and the traditional ANN thoughts. It is the progression formed by transformation of shifting and dilation with wavelet decomposition, which has the same attribute of approximating function as wavelet decomposition. Meanwhile, because of introducing two new variables named non-integer dilation factor and shift factor, it has more freedom of motion than wavelet transform, more flexible and effective function approximation capability. It can effectively overcome some defects of normal ANN models. In this paper, after intensive analysis and research on the data of flux and the water content which get from multi-sensors, a prediction model of water content in crude oil based on WNN is established, and compared with the result of the traditional BP neural network.
2 Water Content in Crude Oil Measurement System The oil-water mixture flows between the inner conductor and the outer conductor of the coaxial line sensor, which is used as the measuring device. It also becomes the carrier of the spread of electromagnetic energy. Choosing appropriate parameters of the sensor can ensure the electromagnetic wave in the coaxial line propagate in the form of TEM (Transverse Electromagnetic). It can be got the message of water content by measuring the phase characteristics of the electromagnetic waves spreading in the coaxial line sensor, changing the characteristics of the voltage signal into a frequency signal, and then using the signal amplification circuit and A/D converter. The object of the measurement is well mixed wet crude, whose water content is from 0% to 100% and its gas content is certain[4-6]. The Coaxial-Phase Technology crude oil water content measurement system diagram is shown in Fig. 1.
Fig. 1. Schematic Diagram of Water Content in Crude Oil Measurement System
Application of Wavelet Neural Network to Prediction of Water Content in Crude Oil
23
3 Wavelet Neural Network Model 3.1 Basic Theory The wavelet neural network uses the shifting and dilation of the mother wavelet to constitute the wavelet base, and takes the wavelet as the excitation function of a hidden lay. This network takes the attribute of evaluation index which is used to describe the attribute of the evaluation object as the input vector of the network. And it takes the corresponding evaluation as the output vector of the network. It can batchprocessing by using the conjugate gradient method, and adjust the wavelet exponential and neural weight by self-adapting. Use enough samples to training the neural network, and through the study of the samples the wavelet neural network like a “black box”, it saved the experience of the expertise and some mathematical inference mechanism. And it established the nonlinear mapping relation from the attribute of the evaluation object to the output. In this way, we can use the wavelet which has been trained to evaluate the new object. 3.2 Wavelet Neural Network If towards ψ ( t ) ∈ L2 ( R ) and Ψ ( ω ) which is the Fourier transformation of ψ (t ) meet the admissibility condition of wavelet function[7]: cψ
=
∫
+∞
ω
− ∞
− 1
Ψ (ω )
2
dω
< ∞
(1)
Then ψ (t ) is be called basic wavelet, and it also referred to as mother wavelet. After
the ψ (t ) shift b, then flex a in different delay, it generates the Wavelet bases: ψ
a ,b
(t ) =
⎛ t − b ⎞ ⎟ a ⎝ ⎠
1
ψ ⎜
a
(2)
Where, a, b——the dilation factor and shift factor ofψ a ,b (t ) , a, b ∈ R , a ≠ 0 . Given non-linear time sequence transformation function as wavelet transformation is: 1
WT x ( a , b ) ≤ x (t ),ψ a ,b (t ) ≥
a
∫
+∞
−∞
⎛t −b⎞ x (t )ψ ⎜ ⎟dt ⎝ a ⎠
x(t ) ∈ L2 ( R) , the
(3)
The water content in crude oil sequence can be obtained through the method which is linear superposition, use the finite term of the wavelet levels to approach the real testing data of the water content of crude oil: g (x) =
L
∑
k =1
⎛ ⎜ w kψ ⎜ ⎜ ⎜ ⎝
n
∑
i=1
u
ki
⎞ x (i) − bk ⎟ ⎟ ⎟ ak ⎟ ⎠
(4)
24
H. Niu et al.
Where, g(x)—the prediction value of y; u ki ,
wk —weight coefficient; ak , bk —the
dilation factor and shift factor; n—dimension of input vector; L—number of wavelet basis. The structural diagram of wavelet neural network is shown in Fig. 2 (combined with the actual needs, here gives the situation of two input vectors and one output vector).
Fig. 2. Structural Diagram of Wavelet Neural Network
For N samples of the input and output that is (xt, yt)( t=1, 2, …N), the aim is to
u ki , wk , ak , bk and L to make the gt(x) and the yt u is best fitting. The network parameter ki can be optimized by the energy function of
make sure the network parameter
minimum mean-square error: E
= 0 . 5 [y t − g t ( x ) ]
2
t
(5)
The wavelet basis adopt the Morlet mother wavelet which is : ψ ( T ) = cos( 1 . 75 T ) exp( − 0 . 5 T
2
(6)
)
This wavelet is gaussian wave which is cosine modulation, it has high precision and strong antijamming feature. The calculating method in detail of this network is introduced as follow: 1. 2. 3.
Initialization of the network parameters; Input the training samples xt, and matching desired output value yt; The network self-learning, use the current network parameter to figure out the output of the network:
g
t
( x ) =
L
∑
k = 1
w
k
ψ
⎛ ⎜ ⎜ ⎜ ⎜ ⎝
n
∑
u
ki
x (i ) −
i = 1
a
k
b
k
⎞ ⎟ ⎟ ⎟ ⎟ ⎠
(7)
Application of Wavelet Neural Network to Prediction of Water Content in Crude Oil
4. 5. 6.
Figure out the instantaneous value of gradient vector; Back propagation, modify the network parameter; Calculation the error N
E = 0 .5 ∑
t =1
7.
25
[g t ( x ) −
yt
]2
(8)
If E < E max (one positive value has be preset) or it gets the maximum train epochs, then stop training; else order E=0, return to step (2).
4 The Prediction Model Simulation Prepare oil samples with calibration water content using artificial ration, and then use the three-phase flow dynamic mixed devices to simulate the actual oil well. According to the nonlinear mapping relation between the water content and its influencing factors, taking the frequencies of the coaxial line water detector based on phase and the measured value of the turbine flowmeter as the input vectors of the WNN, taking corresponding known water content in crude oil as the output vector. Get the node number of latent layer by auto-optimization function of the WNN. So the node number of latent layer is intended to be 8 wavelet function to build the WNN prediction model. In order to increase the precision of prediction, we need normalization processing the data firstly by using function (7), because the water content is between 0~1, so it does not need to normalization[8]. x i' =
x i − x min + 0 . 1 x max − x min + 0 . 1
(9)
Choose the former 44 groups of the known data as the training samples, and the remaining 11 groups as the testing samples. Momentum coefficient λ =0.5, learning rate η =0.3, learning error E=0.01, and simulate in matlab7.1. In order to make known the effects of the prediction of WNN, this paper also use traditional BP to predict the water content in crude oil, and compared these two methods. Fig. 3 and Fig. 4 are the fitting chart between the real data and the data predicted by BP and the fitting chart between the true value and the prediction value of WNN. From these two figures we can see that the data predicted by WNN is much more approach the real data. Tab. is the simulated result of BP and the simulated result of WNN. From table 1, we can see that the relative error and mean square error (MSE) of the prediction model based on WNN is much smaller than the relative error and mean square error (MSE) of the prediction model based on BP, the precision of prediction has been remarkably improved. So, we can find out that the predict precision and the data fitting of the prediction model of water content in crude oil based on WNN are both better than the model based on BP neural network.
Ⅰ
26
H. Niu et al.
Fig. 3. The fitting chart between the true value and the prediction value of BP
Fig. 4. The fitting chart between the true value and the prediction value of WNN
Application of Wavelet Neural Network to Prediction of Water Content in Crude Oil
27
Table 1. The Simulated Result of BP Neural Network and The Simulated Result of WNN
number
Measured of water content%
1 2 3 4 6 6 7 8 9 10 11
0 10 20 30 40 50 60 70 80 90 100
BP neural network Prediction Relative of water error% content% 3.10 ∞ 13.56 35.6 24.46 22.3 40.32 34.4 48.62 21.5 53.75 7.5 57.73 5.4 63.16 9.8 70.13 12.3 78.13 13.2 102.16 2.2
MSE
0.5224
WNN neural network Prediction Relative of water error% content% 0 0.0 10. 90 9.0 17.39 13.1 33.12 11.0 37.08 7.3 52.32 4.6 63.12 5.2 69.18 1.2 82.96 3.7 92.12 2.4 101.14 1.14
MSE
0.0562
5 Conclusion This paper proposed the prediction model of water content in crude oil based on wavelet neural network, use the nonlinear wavelet basis replace the excitation function of the neuron in neural network. It combines advantage of the wavelet and the traditional neural network, and through the calculation function such as flexible and shifting to multi-scale analysis the data. The result of the prediction shows that the wavelet neural network prediction model has the following advantages: much better convergence and higher prediction precision than BP network, and have stronger capability of study and popularize. So it can provide the on-line prediction and has a broad application.
References 1. Ming-yi, C., Xin-rong, Y.: Information Merge Technology of Multiple Transducers and Its Application in Moisture Measurement of Oil Products. J. Process Automation Instrumentation 23(8), 2–3 (2002) 2. Shun-tian, L., Yang, S.: The Systems Analysis and Design Based on Matlab-Neural Network. Xidian University Press, Xi’an (1999) 3. Tsaih, R., Hsu, Y.-s., Lai, C.C.: Forecasting S&P 500 Stock Index: Futures with a Hybrid AI System. J. Decision Support System, 161–174 (1998) 4. Jin-qi, W., Xi-fu, Q., Kui-yong, Z.: Coaxial Transmission Line Phase Method for Measuring Water Content of Oil Well. J. Chinese Journal of Scientific Instrument 23(1), 75–76 (2002) 5. Jin-qi, W., Xi-fu, Q., Jian-ming, C.: Fhase measurement of water content in oil well. J. Journal of Harbin Institute of Technology 34(2), 245–247 (2002) 6. Jin-qi, W., Xi-fu, Q., Ying-hua, Y.: Test Study of Water Cut Tool in Oil Well Based on Phase Method. Acta Metrologica Sinica 25(4), 366–368 (2004) 7. Xin, W., Lu, Z., Xiang, L.: Simulation and Application of MATLAB Neural Network. Science Press (2003) 8. Li-qun, H.: Artifical Neural Network Course Book. Beijng University of Posts and Telecommunications Press, Beijing (2006)
Prediction of Urban Heat Island Intensity in Chuxiong City with Backpropagation Neural Network Wujun Xi and Ping He Department of Geography and Tourism Management, Chuxiong Normal University, Chuxiong, China
[email protected],
[email protected]
Abstract. In order to improve prediction accuracy of urban heat island intensity, we chose 9 main influencing factors from 1981 to 2006 and predicted urban heat island intensity in Chuxiong city in 2006 with backpropagation neural network. The predicted value was 2.1815 . Compared with its measured value, residual error was 0.1042, relative error was 4.5588%. It was superior to results of SCGM model and GM(1,1) model. The result shows that backpropagation neural network is effective to predict urban heat island intensity.
℃
Keywords: Backpropagation neural network, Prediction, Urban heat Island intensity, Chuxiong.
1 Introduction An urban heat island (UHI) is an urban area which is significantly warmer than its surrounding rural areas [1]. Urban heat island is one of the most important features of urban climate and atmospheric environment, urban human factors and local meteorological condition cause urban heat island to form [2]. In 1820, The British chemist Lake Howard put forward urban heat island firstly, by observing the urban centre and surrounding town areas [3]. After that, many researchers paid great attention to it, and the attention increased in recent years (Figure 1).
Fig. 1. Academic attention to urban heat island effect in China from 1996 to 2008
Data source: www.cnki.net. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 29–36. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
30
W. Xi and P. He
Correlation study included urban heat island intensity[4-5], Sequential variation of urban heat island[6-7], spatial distribution characteristic and pattern of urban heat island [8-9], monitoring method of urban heat island[10-11], influencing factors of urban heat island[12-15], formation mechanism of urban heat island[16], influence of urban heat island and countermeasures[17-18]. However, the researches of prediction of urban heat island intensity were relatively less, and the method was mainly regression and gray system [18-21]. Professor Zhi CHEN predicted urban heat island intensity in Xian with SCGM model, the relative error was 5.0% [20]. Professor Ping HE predicted urban heat island intensity in Chuxiong with GM(1,1) model, the relative error was 7.97%[21]. When they predicted urban heat island intensity by using regression and gray system, they commonly used urban heat island intensity data, but not considered other factors. In order to remedy the monotony of them, we used backpropagation neural network to synthesize the relevant influencing factors to predict urban heat island intensity in Chuxiong city.
2 A Survey of Chuxiong City 2.1 Natural Situation of Chuxiong City Chuxiong city locates on North-central of Yunnan Province, 101.32°E, 25.01°N, the land area of 4,433 square kilometers, among which, the built-up areas is 23 square kilometers. It has a population of 500,000. The altitude is between 1700 meters to 2300 meters. Its mean annual temperature is 15.6 , and average annual rainfall is 864 millimeters. Chuxiong belongs to subtropical monsoon climate [22].
℃
2.2 Sequential Variation of Urban Heat Island in Chuxiong City According to the professor Ping He's research [6], in the years prior to 1988, the urban heat island intensity in Chuxiong city was below 1 , since then, the intensity was going up over 1 . It increased quickly and reached 2 in 2006 (Figure 2).
℃
℃ ℃
Fig. 2. Variation of annual heat island intensity in Chuxiong city from 1959 to 2000
The maximum value of heat island intensity in Chuxiong city from 1980 to 2000 happened in spring (from March to April), the minimum happened in summer (from June to July) (Figure 3).
Prediction of Urban Heat Island Intensity in Chuxiong City
31
Fig. 3. Variation of monthly heat island intensity in Chuxiong city
3 Prediction of Urban Heat Island Intensity in Chuxiong City with Backpropagation Neural Network 3.1 Influencing Factors of Urban Heat Island in Chuxiong City Because urban human factors and local meteorological condition cause urban heat island to form, so influence factors of urban heat island include relative humidity, air pressure, aqueous vapor pressure, wind velocity, low cloud amount, total cloud amount, sunshine duration, prevailing wind, greenhouse gases, atmospheric pollution, urban underlying surface, urban population, power consumption, coal consumption, number of motor vehicles, industrial pattern, land use type, density of population, water area, road area, building density and urban green coverage ratio, etc[20,21,24,25]. Considering influencing factors’relevancy and data’s acquisition, we chose 4 meteorological elements (relative humidity, wind velocity, total cloud amount, sunshine duration) and 5 human factors (urban population, urban underlying surface area, power consumption, number of motor vehicles and urban green coverage ratio) to analyze urban heat island intensity in Chuxiong city. 3.2 Prediction Process with Backpropagation Neural Network Data preparation. Convert 9 factors and urban heat island intensity data in Chuxiong city from 1981 to 2006 into non-dimensional quantities, all factors’ measured values as reference value 1. Table 1 was the processing results. Table 1. The influencing factors’ values of urban heat island intensity in Chuxiong city from 1981 to 2006 Year
Urban heat island intensity
Urban population
Urban underlying surface area
Urban green coverage ratio
Number of motor vehicles
1981
1.0000
1.0000
1.0000
1.0000
1.0000
1982
1.0000
0.9677
0.9362
1.0784
0.9928
1983
1.0000
1.0622
0.8511
1.0963
1.0870
1984
0.8571
1.2191
1.0851
0.9940
1.1726
1985
0.8571
1.1660
1.0638
1.0273
1.4039
1986
1.4286
1.2674
1.0638
1.1671
1.7280
32
W. Xi and P. He Table 1. (continued) 1987
2.2857
1.2885
1.0638
1.0554
1.9845
1988
2.0000
1.3226
1.3617
1.0656
2.3234
1989
2.1429
1.4746
1.3617
1.0656
2.4559
1990
2.1429
1.5572
1.3191
1.0656
2.8267
1991
2.1429
1.5499
1.7234
1.0656
3.0661
1992
2.1429
1.6971
1.7234
1.2447
3.4622
1993
2.8571
1.7372
2.6809
1.1679
3.9689
1994
2.7143
1.9483
2.8085
1.1765
4.0659
1995
2.5714
1.8281
2.8936
2.1381
4.8725
1996
2.8571
1.9296
2.8723
2.0946
6.1463
1997
3.2857
1.9751
3.0000
2.1475
6.8072
1998
3.0000
2.0821
2.8723
1.8824
7.0374
1999
2.8571
2.1664
2.8723
2.0350
8.0073
2000
2.8571
2.2027
3.5745
1.9241
8.2236
2001
2.5714
2.3715
3.0851
2.0716
8.4146
2002
3.0000
2.2386
3.6170
2.1824
9.0101
2003
2.2857
2.3707
3.6383
2.1483
9.9578
2004
1.8571
2.3615
3.8298
2.1560
11.6458
2005
2.1429
2.4351
4.4681
2.2165
14.0174
2006
2.2857
2.7815
4.2553
2.1876
15.1821
Year 1981
Power consumption 1.0000
Sunshine duration 1.0000
Wind velocity 1.0000
Relative humidity 1.0000
Total cloud amount 1.0000
1982
0.9954
0.8590
1.0000
1.0290
0.9474
1983
1.1382
0.7948
1.0625
1.0145
1.0877
1984
1.1648
0.9771
1.0625
1.0000
0.8421
1985
1.2453
0.8484
0.9375
1.0435
1.0351
1986
1.4415
0.9158
0.9375
0.9565
0.9123
1987
1.3941
0.9151
1.0000
0.9565
0.8772
1988
1.5710
0.8775
1.0000
0.9565
1.0351
1989
1.6595
0.8588
0.7500
0.9565
0.9825
1990
1.8474
0.6352
0.6875
1.0580
1.1053
1991
1.9153
0.8053
0.6875
0.9855
0.9123
1992
2.3170
0.7183
0.8750
1.0000
0.9649
1993
2.1374
0.7223
0.7500
0.9855
0.9649
1994
2.4260
0.8171
0.7500
1.0000
0.8947
Prediction of Urban Heat Island Intensity in Chuxiong City
33
Table 1. (continued) 1995
2.5505
0.7704
0.9375
0.9710
1.0702
1996
3.0175
0.7759
0.8750
1.0435
0.9649
1997
2.8170
0.8852
0.8125
0.9710
1.0000
1998
3.0634
0.7654
0.6875
0.9710
0.9474
1999
3.4123
0.7150
0.6250
0.9855
1.0877
2000
3.3011
0.6787
0.6250
0.9710
1.0702
2001
3.4886
0.6144
0.5625
1.0145
1.2105
2002
3.9863
0.6812
0.4375
0.9855
0.9649
2003
3.9070
0.8734
1.3750
0.9420
0.8070
2004
4.5524
0.6433
1.3750
0.9130
1.0526
2005
5.7407
0.6713
1.5625
0.8841
1.0526
2006
7.3056
0.7999
1.3750
0.9275
1.0175
Data source: Chuxiong city yearbook (1994-2007), Lucheng town statistical station.
Prediction with backpropagation neural network. Analysis tool was DPS Data processing system [23], it provided a rapid way to compute with backpropagation neural network. First, we imported all non-dimensional quantities into DPS Data processing system, then made backpropagation neural network analyze with 9 influencing factors and urban heat island intensity from 1981 to 2005. After repeated experiments, we finally determined essential parameters: the number of hidden network Layer was 1, minimum training ratio was 0.1, dynamic parameter was 0.4, sigmoid was 0.9, error excepted was 0.0001, maximum iteration number was 5000. When iteration number reached about 2300, the result matched error excepted, and the computation stopped, fitting residual error was 0.00094(Figure 4).
Fig. 4. Backpropagation neural network fitting curve
34
W. Xi and P. He
We predicted urban heat island intensity in Chuxiong city with 9 influencing factors in 2006 as observed data, the predicted value was 2.1815 . Compared with the measured value of urban heat island intensity in Chuxiong city in 2006, residual error was 0.1042, relative error was 4.5588%.
℃
3.3 Predicted Result and Analysis Compared with these researchers’ predict results[20,21], our predicted relative error was only 4.5588%. It showed that backpropagation neural network is effective to predict urban heat island intensity. The result is superior to results of SCGM model and GM(1,1) model.
4 Conclusion Prediction the development tendency of urban heat island intensity is more important now. However, the prediction accuracy was not very satisfied. In order to improve prediction accuracy of urban heat island intensity, we chose 9 influencing factors of urban heat island as main influencing factors by analyzing the urban heat island situation in Chuxiong city, the factors included relative humidity, wind velocity, total cloud amount, sunshine duration, urban population, urban underlying surface area, power consumption, number of motor vehicles and urban green coverage ratio. We made our backpropagation neural network analysis with 9 influencing factors and urban heat island intensity from 1981 to 2005, and predicted urban heat island intensity in Chuxiong city with these factors in 2006 as observed data, the predicted value was 2.1815 . Compared with the measured value of urban heat island intensity in Chuxiong city in 2006, the residual error was 0.1042, and the relative error was 4.5588%. Compared with the mentioned researchers’ predict results, our result is superior to the results of SCGM model and GM(1,1) model. It shows that backpropagation neural network is effective to predict urban heat island intensity, and it used many kinds of influencing factors synthetically to analyze urban heat island intensity. It has remedied the defect of predicting with unitary urban heat island intensity values, and has promoted the prediction accuracy.
℃
Acknowledgments. This study was supported by the National Natural Science Foundation of China (Grant No. 40965002), and Natural Science Foundation of Yunnan province (Grant No. 2005D0077M). And Hui Chen made a contribution to data collection.
References 1. Han, S., Guo, J., Huang, S., Bian, H.: Study on the Evolution of Urban Heat Island in Tianjin city (in Chinese). Ecology and Environment 16(2), 280–284 (2007) 2. Zhou, S., Shu, J.: Urban Climatology (in Chinese), p. 315. China Meteorological Press, Beijing (1994) 3. Howard, L.: Climate Deduced from Meteorological, vol. 1(3), pp. 1–24. Harvey and Darton, London (1833)
Prediction of Urban Heat Island Intensity in Chuxiong City
35
4. Wang, J., Wang, K., Wang, P.: Urban Heat (or Cool) Island over Beijing from Modis Land Surface Temperature (in Chinese). Journal of Remote Sensing 11(3), 330–339 (2007) 5. Li, F., Ma, A., Ding, Y., Yang, J., Jiao, J., Liu, L.: Research on Urban Heat Island Effect Based on Landsat Data (in Chinese). Remote Sensing Technology and Application 24(4), 553–558 (2009) 6. He, P., Li, H., Shu, J., Lu, X., Sun, J.: The Analysis of the Urban Climate in Chuxiong of Yunnan: Comparison of Urban Heat Island Effect in China (in Chinese). ACTA Geographica Sinica 58(5), 712–720 (2003) 7. Song, Y., Zhang, S.: The Study on Heat Island Effect in Beijing During Last 40 Years. Chinese Journal of Eco-Agriculture 11(4), 126–129 (2003) 8. Zhang, Y., He, Y., Ma, Y., Liu, Y., Li, Y., Dou, J., et al.: Characteristics of Vertical Distribution of Urban Heat Island Effect in Kunming City (in Chinese). Plateau Meteorology 21(6), 604–609 (2002) 9. Zheng, Z., Liu, W., Wang, Y.: Distributive Character of Urban Heat Island Effect in the Beijing Region (in Chinese). Journal of Nanjing Institute of Meteorology 29(5), 694–699 (2006) 10. Jian, W., Samei, S.: Research on the Method of Monitoring the Urban Heat Island by Remote Sensing Technique of Satellite (in Chinese). Modern Scientific Instruments (12), 27–29 (2004) 11. Rongbo, X., Ouyang, Z., Zhaoming, Z., Xiaoke, W., Weifeng, L., Hua, Z.: Advances in Methodology of Urban Heat Island (in Chinese). Meteorological Monthly 31(11), 3–6 (2005) 12. Cheng, C., Wu, N., Guo, S., Li, S., Liu, D.: A Study on the Interaction Between Urban Heat Island and Vegetation Theory, Methodology, and Case Study (in Chinese). Research of Soil and Water Conservation 11(3), 172–174 (2004) 13. Huang, J., Liu, X., Li, Q.: The Study Of Relationship Between Heat Island Effect and Population in Cities Over South of China (in Chinese). Journal of Tropical Meteorology 20(6), 713–722 (2004) 14. Ma, Y., Tiyip, T., Huang, Y., Yang, J.-L.: Effects of Landscape Pattern Change on Urban Heat-island Effect in Arid Areas—A Case Study in Urumqi (in Chinese). Arid Zone Research 23(1), 172–176 (2006) 15. Zhou, K., Ye, Y., Peng, S., Su, J.: Total suspended particulates and its effects on urban heat island (in Chinese). Ecology and Environment 15(2), 381–385 (2006) 16. Dai, X., Zhang, L., Guo, Z., Wu, J., Li, X., Zhu, Y.: Mechanism of formation of urban heat island effect and its spatial pattern in Shanghai (in Chinese). Acta Ecologica Sinica 29(7), 3995–4004 (2009) 17. Zhu, Z., Wang, M.: The Harms and Countermeasures of Urban Heat Island Effect (in Chinese). Pollution Control Technology 22(2), 94–96 (2009) 18. Wen, J., Bao, J., Zhang, Z.: Ecological Measure Analysis of Reducing Urban Heat Island (in Chinese). Ecological Economy (2), 151–153 (2008) 19. Wei, H., Zhao, Y., Chen, Y.: Grey Assessment and Prediction of the Urban Heat Island Effect in Lanzhou City (in Chinese). Journal of Desert Research 29(3), 571–573 (2009) 20. Zhi, C., Bingfeng, Y., Wangyang, H., Xilian, L., Linxiang, Q.: Grey Assessment and Prediction of the Urban Heat Island Effect in City (in Chinese). Journal of Xi’an Jiaotong University 38(9), 985–988 (2004) 21. He, P., Chen, H., Li, H., Xi, W.: Grey Analysis of the Urban Heat Island Effect Factors of the Medium-sized City of Chuxiong on Yunnan Plateau (in Chinese). Progress in Geography 28(1), 25–32 (2009)
36
W. Xi and P. He
22. Chuxiong Online . A Survey of Chuxiong, http://www.cxs.gov.cn/file_read.aspx?id=14944 23. Tang, Q.Y., Feng, M.G.: DPS Data processing system: Experimental design, statistical analysis, and data mining (in Chinese), pp. 932–935. Science Press, Beijing (2007) 24. Li, Z., Gong, C., Hu, Y., Yin, Q., Kuang, D.: The Progress of the Remote Sensing Research on Urban Heat Island (in Chinese). Remote Sensing Information (8), 100–105 (2009) 25. Wang, X., Bai, J.: The Development Prospect of the Urban Heat Island Effect (in Chinese). Journal of Foshan University (Natural Science Edition) 26(1), 53–56 (2008)
Research on the Fouling Prediction of Heat Exchanger Based on Wavelet Relevance Vector Machine Lingfang Sun, Rina Saqi, and Honggang Xie School of Automation Engineering, Northeast Dianli University, Jilin City, 132012, China
[email protected]
Abstract. Based on the relevance vector machine and wavelet theory, a new machine learning method–wavelet relevance vector is introduced. The method performs wavelet transform to decompose the original series into some filtered series, and a relevance vector machine whose kernel functions is wavelet function models each of them. Then, this method is used to predict fouling thermal resistance of Heat exchanger. Construction of wavelet relevance vector machine prediction model is presented. Simulations show that wavelet relevance vector machine requires dramatically fewer kernel functions and it can get high prediction precision, and work in the paper offers a new method for the research of heat exchanger fouling. Keywords: Fouling thermal resistance, Wavelet Transform, Relevance Vector Machine, Prediction.
1 Introduction Fouling exists in nature, daily life and most of the industrial process. Accumulation of fouling not only degrades heat transfer efficiency of heat exchanger, increases greatly initial capital and maintenance costs, but also causes overheat on certain local areas of heat exchanger, and threaten the safety of heat exchange equipment severely. The economic losses caused by fouling are as high as 0.25 of Gross National Product in a developed country [1]. In our country, the partial fouling losses have reached 0.11 of Gross Domestic Product only in electric power system [2], if considering other industry, the whole fouling losses is more than the developed country’s. So minimizing the harm of fouling becomes an effective avenue of save energy. As the foundation of the fouling research, research on the fouling prediction has become of one main direction since the late 1980s, however, for the many influence factors, fouling prediction based on the traditional mechanics analysis can not satisfy the actual needs. In recent years, the artificial intelligence is applied to fouling research, such as neural network [3,4], support vector machine (SVM) [5] and so on, and gets great results, but these methods also show some limitations at the same time. Tipping improved on the SVM and formulated the relevance vector machine (RVM) [6,7]. The RVM is a probabilistic model similar to the widespread SVM, but the training takes place in a Bayesian framework and predictive distributions of the outputs instead of point estimates are obtained. Relevance vector machine draws Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 37–45. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
38
L. Sun, R. Saqi, and H. Xie
hyper-parameters into run, implements automatic estimation of ‘nuisance’ parameters, and its kernel function doesn’t need to satisfy ‘Mercer’ condition, its calculate complexity is dramatically reduced and the operating speed is advanced. So the RVM has been applied to much industrial process recently [8,9]. In the paper, we constructs the wavelet relevance vector machine(WRVM) regression prediction model with the wavelet transform and relevance vector machine, and uses to analyze and predicts the heat exchanger fouling increase.
2 The Principles of Wavelet Relevance Vector Machine 2.1 Wavelet Transform Wavelet analysis is a multi-resolution analysis method both in time domain and frequency domain. On the low frequency part, the frequency resolution of wavelet transform is high, but the time domain resolution is low. On the high frequency part, the time domain resolution is high, but the frequency resolution of wavelet transform is low. So wavelet analysis is called a mathematic microscope. Wavelet function ϕ (t ) is called mother wavelet. a,b(t) can be obtained through compressing and expanding ϕ (t ) : −1
ψ a ,b ( t ) = a
2
ψ(
t−b ) a
(1)
Where a,b(t) is a successive wavelet; a is scale or frequency factor, a time factor; b R, R is the domain of real number. Wavelet transform is defined as:
∈
WTx (a,τ ) =
1 a
∫
+∞
−∞
x(t )Ψ ∗ (
∈R, a≠0; b is
t −τ )dt a
(2)
The equivalent formula in frequency domain: WTx (a ,τ ) =
a 2π
∫
+∞
−∞
X (ω )Ψ ∗ ( aω )e jωτ d ω
(3)
Fourier transformation of x(t) and ¸ (t) are X(¹) and ¸(¹) respectively. Let frequency factor a and time factor b be discrimination: a=2j and the discrete binary wavelet transform is written as: WT f ( j , k ) = 2
−
j 2
∫ f (t ) Ψ
∗
⎡t − 2 j k ⎤ ⎢ 2 j ⎥dt ⎣ ⎦
τ=2 k. So j
(4)
Where, k is translation factor, and j is wavelet decomposition level. Its binary wavelet reconstruction is formed as follows: f (t ) = ∑ c j ,k ϕ j ,k (t ) + ∑∑ d j , k φ j , k (t ) = A j (t ) + ∑ D j (t ) k
k
j
j
(5)
Research on the Fouling Prediction of Heat Exchanger
39
Where, Aj and Dj is the trend part (background information) and detail part (detail information) respectively. The trend series contains the slowest dynamic, and the detail series contains the dynamic at a certain intermediate scale. 2.2 Relevance Vector Machine
∈
∈
Given a training set {xn , tn }nN=1 , with xi Rn for the input, yi R for the target output, N for the number of training sample. Form this ‘training’, the regression problem is to find a mapping f from the input space to the output space. The purpose of the RVM is to find a relevance regression model based on the Bayesian framework:
t=¶w+¦
(6)
Here, ¹ is parameter vector, ¶ is the N×(N+1) design matrix with Φ = [φ1 , φ2 ,..., φM ]
(7)
Φ nm = K ( xn , xm )
(8)
Wherein
Due to the Bayesian inference, the likelihood of the complete date set can be written as: p(t w , σ 2 ) = (2πσ 2 ) − N 2 exp{− t − Φw
…
2
(2σ 2 )}
(9)
…,
wN). Where, t = (t1, t2, , tN), w = (w1, w2, Generally, with multi-parameter in the regression model, its maximum likelihood estimation may lead to severe over-fitting. And that, on the relevance vector machine, owing to the Bayesian inference makes model more general. The Gaussian prior distributions are chosen: M
p( w, α ) = (2π )− M 2 ∏ α m1 2 exp(−(α m wm2 ) 2) m −1
(10)
Where α = {α1 , α 2 ,...α M } is a vector of hyper parameters that governs the prior over the weight. The value of the hyper parameter is the key to the model sparsity and it also controls the strength of influence of the prior distribution on the each parameter. On the basis of prior distribution and marginal likelihood, the posterior distribution over the weights can be written using Bayes inference: p(w t ,α ,σ 2 ) = = (2π ) − ( N +1) 2 Σ
p ( y | w, σ 2 ) p ( w | α ) ~ N ( μ , Σ) p( y | α , σ 2 ) −1 2
exp{−( w − μ )T Σ −1 ( w − μ ) 2}
(11)
40
L. Sun, R. Saqi, and H. Xie
Where, the posterior covariance and mean are respectively given by: Σ = (σ −2 Φ T Φ + A) −1
(12)
μ = σ −2ΣΦT
(13)
With A = diag (α 0 , α1...α M ) . In practice, we can find that the majority of parameters are automatically set to zero during the leaning process. And the corresponding basis function can thus be ‘pruned’. To the RVM regression model, the non-zero parameters are the relevance vectors (like SV). By integrating the parameters, we obtain the marginal likelihood for the hyper parameters: p (t α , σ 2 ) = N (0, C )
(14)
Here, C = σ 2 I + ΦA−1ΦT . For a new input x∗ and output t∗ , we can then compute the predictive distribution is Gaussian: μ∗ = F μ
(15)
σ ∗2 = σ op2 + F T ΣF
(16)
With F = [Φ1 ( x∗ ), Φ 2 ( x∗ ),..., Φ M ( x∗ )]T . We can consider that μ∗ is the prediction to t∗ , and the distribution of t∗ is given, so we can obtain the Confidence interval under some conspicuous level. If given α ∗ , we can gain its 1 − α ∗ Confidence interval:
( μ∗ − t∗α / 2 ( N − k )σ ∗ , μ∗ + t∗a / 2 ( N − k )σ ∗ )
(17)
With N is the number of the sample and k is the number of parameter.
3 Construction of WRVM Prediction Model WRVM is to hang together the wavelet transform and wavelet kernel function with relevance vector machine. The construction of wavelet relevance vector machine have four steps: (1) decompose the original series into some filtered series with the resolution level 3 (6 time series); (2) reconstruct each sub-time series; (3) let wavelet function to be the kernel function of RVM; (4) predict each sub-time series (trend part and detail part), and combine the results of every series, and then we can acquire the final prediction result. The diagrammatic sketch is shown as Fig. 1. Due to that every sub-time series has its own characteristic, so the each predict model corresponding to the sub-time series is distinct.
Research on the Fouling Prediction of Heat Exchanger
41
Original series
x Wavelet decomposition
d1…
aJ
…
dJ
Wavelet Reconstruction
D1…
AJ
…
DJ
Wavelet Kernel RVM
YD1…
YAJ
…
YDJ
Prediction Result
Fig. 1. Method of WRVM
The capability of non-liner regression model can be evaluated by many measurements. The most important measurement to evaluate the quality of model is the accuracy. In this work, we use mean relatively error: ∧
MAPE =
1 n yt − yt ∑ n t =1 yt
(18)
4 Fouling Thermal Resistance Prediction Based on WRVM 4.1 Data Acquisition and Pretreatment Analyzing the system that we need to predict, make sure the characteristic parameters that can reflect the system’s state. We select the fouling thermal resistance as a way to detect the fouling degree. The data we acquired used for the research were from online monitoring equipment [10]. According to the methods in the reference [1], we found that six measuring parameters influenced fouling, i.e. input temperature of working fluid, output temperature of working fluid, flow velocity of working fluid and three temperatures of heat exchanger tube wall. So we use these parameters forming the inputs of WRVM. The data we collected were from the IDCB893 for every one minute, altogether 180 series sample consisted the discrete fouling thermal resistance data. In the same way, another group data were completed, for 180 series sample, for the prediction. These parameters were chosen to be the inputs of the SVM network. They represent themselves for different significations, so they need to be normalized. The process can be done with the function ‘Prestd’ in the Matlab, it is as below, [pn, meanp, stdp] = prestd (x) Here, ‘pn’ stands for the normalized input vector or matrix; ‘meanp’ and ‘stdp’ stand for the mean and the standard deviation of the x. The sample data were divided into two groups; the former 120 data were used for training, the rest ones were used for predicting. The second group was all used for prediction.
42
L. Sun, R. Saqi, and H. Xie
Hyper parameters and deviation was initialized respectively as follows: initAlpha=( 1 ) 2 N
initBeta=var[t] × 0.1
4.2 Structure of Kernel Function The performance of RVM is mainly decided by its kernel function. How to construct suitable kernel function relating on complicated practical application is an important aspect of RVM all the way. Two wavelet functions are chosen in this paper, Mexico wavelet function and Morlet wavelet function respectively. Mexico wavelet function d
xi − xi2
i =1
a2
K ( x, x ' ) = ∏ [(1 −
) exp( −
xi − xi'
2
)]
(19)
xi − xi' xi − xi' )exp(− )) a 2a2
(20)
2a 2
Morlet wavelet function: 2
d
K (x, x' ) = ∏(cos(1.75× i =1
4.3 Results and Discussion The simulation based on wavelet relevance vector machine includes 5steps. Step 1: Apply the function ‘wavedec’ of Matlab, and then decompose the original time series to 4 subsidiary time series. [ci,li]=wavedec (x1c (:,i),2,'db3') Step 2: Apply the function ‘wrcoef’ of Matlab and carry out wavelet reconstruction to each subsidiary time series. A(:,i) = wrcoef('a',ci,li,'db3',2) Step 3: Let Mexico wavelet function and Morlet wavelet function to be the kernel function of relevance vector machine. Through repeatedly experiments, we can select the most contented output. Step 4: In order to cater for the characteristic of the fouling time series, the parameters is setting as follow: The regression time-delay :d=3 The width of wavelet function: a=0.8 The variance of noise: σ 2 = var( y ) × 0.1 The other parameters:
itermax = 50 ,
α max = 1.0e5 , {α } = (ones (1.N + 1) × ( 1 )) 2 2
Step 5: Predict each sub-time series, and combine the results of every series, we can acquire the final prediction result. Corresponding experiment data and prediction results are shown as Fig. 2 – Fig. 8.
Research on the Fouling Prediction of Heat Exchanger
Fig. 2. Prediction curve of trend part (*) and real fouling cure (-)
Fig. 3. Prediction curve of detail part cd1 (*) and real fouling cure (-)
Fig. 4. Prediction curve of detail part cd2 (*) and real fouling cure (-)
Fig. 5. Prediction curve of detail part cd3 (*) and real fouling cure (-)
43
44
L. Sun, R. Saqi, and H. Xie
Fig. 6. Prediction fouling curve (-) and real fouling cure (*)
Fig. 7. The contrastive curve with 95% confidence interval
Fig. 8. Relative error curve
Through simulation, we can obtain that wavelet theory was used two parts, wavelet kernel functions and wavelet transform respectively. As a result, the precision of wavelet relevance vector is higher than standard relevance vector machine, shown as table 1. So the method reported in this paper plays an important role in improving the precision of relevance vector machine in forecasting time series. Table 1. Performance analysis WRVM The number of RVs/SVs Mean relative error
10 0.000457
RVM
SVM
10
92
0.000465
0.000598
Research on the Fouling Prediction of Heat Exchanger
45
5 Conclusions A research on modeling and Predicting the fouling of heat exchanger based on WRVM was introduced in this work. Simulation analysis indicates that the result of prediction has high prediction precision. The main reason is that the wavelet transform was used to extract the trend series and detail series, and make models respectively. It can avoid the over fitting in short term fluctuate. This method took even short time and it has excellent ability and generalization performance and robustness under the noises background. Acknowledgments. Supported by National Basic Research Program of China (2007CB206904)and the Eleventh Five-Year Science & Technology Project of Jilin Province Office of Education ([2007]NO.53).
References [1] Yang, S.R., Xu, Z.M., Sun, L.F.: Heat Exchanger Fouling and Its Countermeasure, 2nd edn. Science Press, Beijing (2004) (in Chinese) [2] Xu, Z.M., Yang, S.R.: Approach to Costing the Utility Boiler Fouling in China. J. of the CSEE. 24, 196–200 (2004) (in Chinese) [3] Radhakrishnan, V.R., Ramasamy, M.: Heat Exchange Fouling Model and Preventive Maintenance Scheduling Tool. Applied Thermal Engineering 27, 2791–2802 (2007) [4] Sun, L.F., Cai, H.D., Zhang, Y.Y., Yang, S.R., Qin, Y.K.: Research on the Fouling Prediction of Heat exchanger Based on Wavelet Neural Network. In: 3rd IEEE International Conference On Cybernetics and Intelligent Systems, Chengdu, pp. 961–964 (2008) [5] Sun, L.F., Zhang, Y.Y., Zheng, X.P., Yang, S.R., Qin, Y.K.: Research on the Fouling Prediction of Heat exchanger Based on Support Vector Machine. In: 2008 International Conference on Intelligent Computation Technology and Automation, Changsha, pp. 240– 244 (2008) [6] Tipping, M.E.: The Relevance Vector Machine. In: Proc. of Advances in Neural Information Processing Systems. MIT Press, Cambridge (2000) [7] Nikolaev, N.: Sequential Relevance Vector Machine Learning from Time Series. In: Proceedings of International Joint Conference on Neural Networks, Montreal (2005) [8] Lexander, S., Richard, E.: Smooth Relevance Vector Machine: A Smoothness Prior Extension of the RVM. Machine Learning 68, 107–135 (2007) [9] Clodoaldo, A.M., Lima, A.L.V., Coelho, S.C.: Automatic EEG Signal Classification for Epilepsy Diagnosis with Relevance Vector Machines. Expert Systems with Application 36, 10054–10059 (2009) [10] Yang, S.R., Xu, Z.M., Sun, L.F.: Automatic On-line Monitor of Fouling Thermal Resistance and Its Application. J. of Engineering Thermophysics 22, 150–152 (2001) (in Chinese)
Electricity Price Forecasting Using Neural Networks Trained with Entropic Criterion Jianhua Zhang, Jingyue Wang, Rui Wang, and Guolian Hou Beijing Key Laboratory of Industrial Process Measurement & Control New Technology and System, North China Electric Power University, Beijing 102206, China
Abstract. Accurate electricity price forecasting is critical to market participants in wholesale electricity markets. The problem becomes more complex because the acquired data series are non-linear and non-Gaussian. In this paper, Multi Layer Perceptrons (MLP) trained with minimizing error entropy (MEE) algorithm is utilized to forecast electricity price. Compared with the conventional MLP with mean square error (MSE) criterion, the proposed approach can achieve better performance in simulated examples. Keywords: Electricity price forecasting, Neural network, Entropy.
1 Introduction Recently, a group of algorithms have been developed to analyze and forecast electricity prices with different aims and time horizons. Arciniegas and Rueda [1] used Takagi-Sugeno-Kang (TSK) fuzzy inference system in forecasting day-ahead prices in Ontario electricity market. Alicia and Antonio [2] introduced an input-output hidden markov model (IOHMM) by finding the probabilistic model of prices and market information, such as the loads, generation kinds and prices. Rodriguez and Anders [3] invented a hybrid forecast model by back propagation neural networks and neuro-fuzzy system for predicting hourly market clearing price in the energy market. In addition, support vector machine (SVM) was also applied into electricity price forecasting field in [4]. And the forecasting method using rough set theory based on collective concept dealing with uncertainty was also proposed in [5]. Among these different techniques of price forecasting, artificial neural networks has been applied by many researchers because of its ability to learn complex and non-linear relationships that are difficult to model with conventional techniques. The mean square error (MSE) has been almost exclusively employed to train ANN systems [6,7]. However, MSE which concentrates on second order statistics was under the linearity and Gaussian distribution assumptions. Most real problems are governed by nonlinear equations and most random phenomena are far from being normally distributed. In this context, the higher-order statistical behavior of the systems is required. Recently, minimizing error entropy (MEE) has been utilized in the training of some adaptive systems [8-10]. As such, minimization of error entropy is employed as a more robust criterion when MLP is utilized to predict electricity prices. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 47–53. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
48
J. Zhang et al.
2 MLP Trained with MEE Criterion A schematic representation of the MLP with MEE criterion is shown in Fig.1. The weights between different layers of MLP are updated by minimizing the entropy of the error e( k ) between the estimated output and the targets.
y ( n)
r ( n)
^
y ( n)
e(n)
Fig. 1. Training MLP using MEE criterion
In this work, the quadratic Renyi entropy Hˆ 2 (e) of error e( k ) between the desired and estimated output is employed to be the performance index, and it can be obtained from data samples {e1(k), e2 (k),..., eN (k)} by utilizing Parzen window with Gaussian kernel function.
⎡ 1 Hˆ 2 (e) = − log ⎢ 2 ⎣N where the kernel function
κσ ( x) =
N
⎤
N
∑∑ κσ (e (k ) − e (k )) ⎥ i
i =1 j =1
1 2πσ
j
⎦.
(1)
− x2
e
2σ 2
.
For the purpose of computing, the estimation of quadratic Renyi entropy at time k can be calculated by a sliding window of samples [8].
⎡1 Hˆ 2 (e) = − log ⎢ ⎣W where
V ( e) =
1 W
k -1
⎤
∑ κσ (e(i) − e(k )) ⎥⎦ = − log V (e)
(2)
i = k -W
k -1
∑ κσ (e(i) - e(k)) is named as information potential in [9], and
i = k -W
W is the width of sliding window.
Electricity Price Forecasting Using Neural Networks Trained with Entropic Criterion
49
In this work, a q − p − l MLP is used to forecast electricity prices. The weights can be updated by using the steepest descent algorithm as follows.
wk (n) = wk (n − 1) + η ⋅ where
∂V (e(n)) ∂wk
(3)
∂V (e(n)) is the gradient of the information potential with respect to the ∂wk
weights and η the learning rate. Once the weight updating rules are obtained, the MLP can be trained by sampled electricity prices series. The electricity prices can be predicted by generalizing the trained MLP.
3 Price Forecasting by MLP with MEE Criterion The factors that influence the electricity price were investigated in [11, 12] , for example, power demands, transmission and generation outages, system or regional reserves, bidding strategy and market power, system congestion, ancillary services and market rules and so on. The main factors can be extracted by correlation analysis and experience in order to improve the efficiency of forecasting. The correlation coefficient ρ is defined as follows: −
n
ρ=
−
∑ ( x − x)( y − y) i
i =1
i
−
n
−
n
∑ ( x − x) ∑ ( y − y ) 2
i
i =1
−
x= −
y=
i
i =1
1 n
n
∑x
i
(5)
1 n ∑ yi n i =1
(6)
i =1
−
where
(4) 2
−
xi , yi are the ith observations and x , y the corresponding mean value of n
observation series. The correlation coefficients between the physical variables and prediction variables are listed in Table 1 (The historical price and demand data are selected from a real power market. P and L represent the electricity price and load in different days i and different times t). The MLP used to forecast daily electricity prices is shown in Fig.2.
50
J. Zhang et al. Table 1. Correlation coefficients between physical variables and price
Variables P(iˈt-1) P(iˈt-2) P(iˈt-3) P(iˈt-4) P(i-1ˈt) P(i-2ˈt) P(i-7ˈt) P(i-1ˈt-1) P(i-1ˈt+1)
correlation with price P(iˈt) 0.9235 0.8396 0.7451 0.6224 0.8646 0.7078 0.9770 0.7343 0.6249
Variables L(iˈt-1) L(iˈt-2) L(iˈt-3) L(iˈt-4) L(i-1ˈt) L(i-2ˈt) L(i-7ˈt) L(iˈt) L(i-1ˈt-1)
correlation with price P(iˈt) 0.9217 0.8172 0.7420 0.6178 0.8723 0.7738 0.9711 0.9747 0.7248
Fig. 2. MLP applied for forecasting
Several error indicators are analyzed to evaluate the accuracy of the proposed approaches to forecasting electricity prices. They are the mean absolute percentage error (MAPE), absolute percentage error (APE) and sum squared error (SSE). The MAPE criterion is given by:
δ MAPE =
∧ 1 n Y − Y ( ∑ i i ) Y × 100% n i =1
Y=
(7)
1 n ∑ Yi n i =1
(8) ∧
where
n is the total number of hours, Yi is the actual price at hour i, Yi is the
forecasted price at that hour.
Electricity Price Forecasting Using Neural Networks Trained with Entropic Criterion
51
The absolute percentage error (APE) is defined as follows: ∧
δ APE = (Yi − Yi ) Yi ×100%
(9)
The SSE criterion is the sum of the n square differences between the forecasted and the actual ones. The SSE criterion is given by: n
∧
δ SSE = ∑ (Yi − Yi )2
(10)
i =1
4 Simulation and Analysis In order to test the validity of the proposed algorithm, a 9-21-1 MLP is constructed with the historical data of electricity price and demand in certain region from July 15 to August 15, and then the trained MLP is utilized to investigate the daily price for next day-August 16. In this simulation, the learning rate η = 0.9 , the width of sliding window W = 32 , the total number of hours n = 24 . The information potential during training phase is shown in Fig. 3, it can be seen that the entropy of error is minimized gradually. Compared with the conventional MLP (using MSE criterion), the forecasting electricity prices are shown in Fig. 4 and the corresponding APEs are shown in Fig.5. It is clear that APE with MEE criterion is almost smaller than that one with MSE. In addition, Table 2 shown the value of MAPE and SSE during the whole day, it can be seen that the proposed MLP with MEE criterion can obtain better forecasting than conventional MLP.
Fig. 3. Information potential of error
52
J. Zhang et al.
Fig. 4. Forecasting by MLP-MEE and MLP-MSE
Fig. 5. The APE of MLP-MEE and MLP-MSE Table 2. Statistical Analysis of Errors
Criteria Error MAPE SSE
MLP-MEE
MLP-MSE
5.0609% 7.8988
8.1125% 14.1206
Electricity Price Forecasting Using Neural Networks Trained with Entropic Criterion
53
5 Conclusions In this paper, a MLP with MEE criterion is utilized to predict the next 24 hour electricity prices in the real electricity market. The predicted price can be used for obtaining bidding strategies. Acknowledgments. This work is partially supported by National Natural Science Foundation of China under grants (No. 60674051, 60974029). These are gratefully acknowledged.
References 1. Arciniegas, A.I., Rueda, I.E.A.: Forecasting Short-term Power Prices in the Ontario Electricity Market (OEM) with a Fuzzy Logic Based Inference System. Utilities Policy 16, 39–48 (2008) 2. Gonzalez, A.M., Roque, A.M.S., Gonzalez, J.G.: Modeling and Forecasting Electricity Prices with Input/output Hidden Markov Models. IEEE Trans. Power Syst. 20, 13–24 (2005) 3. Rodriguez, C.P., Anders, G.J.: Energy Price Forecasting in the Ontario Competitive Power System Market. IEEE Trans. on Power Syst. 19, 366–374 (2004) 4. Sun, W., Lu, J.-C., Meng, M.: Application of Time Series Based SVM Model on Next-day Electricity Price Forecasting under Deregulated Power Market. In: Proc. 5th Int. Conf. Machine Learning and Cybernetics, Dalian, China, pp. 13–16 (2006) 5. Xiao, Z., Ye, S.J., Zhong, B., Sun, C.X.: BP Neural Network with Rough Set for Short Term Load Forecasting. Expert Systems with Applications 36, 273–279 (2009) 6. Szkuta, B.R., Sanabria, L.A., Dillon, T.S.: Electricity Price Short-term Forecasting Using Artificial Neural Networks. IEEE Trans. Power Syst. 14, 851–857 (1999) 7. Mandal, P., Senjyu, T., Urasaki, N., Funabashi, T., Srivastava, A.K.: A Novel Approach to Forecast Electricity Price for PJM Using Neural Network and Similar Days Method. IEEE Trans. on Power Systems. 22, 2058–2065 (2007) 8. Erdogmus, D., Principe, J.C., Kim, S.P., Sanchez, J.C.: A Recursive Renyi’s Entropy Estimator. In: Proceedings of IEEE Neural Networks for Signal Processing, Martigny, Valais, Switzerland, pp. 209–217 (2002) 9. Erdogmus, D., Principe, J.C.: An Error–entropy Minimization Algorithm for Supervised Training of Nonlinear Adaptive Systems. IEEE Transactions on Signal Processing 50, 1780–1786 (2002) 10. Erdogmus, D., Principe, J.C.: Comparison of Entropy and Mean Square Error Criteria in Adaptive System Training Using Higher Order Statistics. In: Proc. Second International Workshop on Independent Component Analysis and Blind Signal Separation, Helsinki, Finland (2000) 11. Zhu, J.Z., Jordan, G., Ihara, S.: The Market for Spinning Reserve and Its Impacts on Energy Prices. In: Proc. IEEE Power Engineering Society Winter Meeting, Singapore, pp. 1202–1207 (2000) 12. Kian, A., Keyhani, A.: Stochastic Price Modeling of Electricity in Deregulated Energy Markets. In: Proc. 34th Hawaii Int. Conf. on System Sciences, Hawaii, USA, pp. 832–838 (2001)
Least Square Support Vector Machine Ensemble for Daily Rainfall Forecasting Based on Linear and Nonlinear Regression Jiansheng Wu1 , Mingzhe Liu2 , and Long Jin3 1
Department of Mathematical and Computer Sciences, Liuzhou Teachers College, Liuzhou, 545004, Guangxi, China
[email protected] 2 Institute of Information and Mathematical Sciences, Massey University, Albany, 0629, Auckland, New Zealand
[email protected] 3 Guangxi Climate Center, Naning, 530022, Guangxi, China
[email protected]
Abstract. Accurate forecasting of rainfall has been one of the most important issues in hydrological research. In this paper, a novel nonlinear regression ensemble model is proposed for rainfall forecasting. The model employs Least Square Support Vector Machine (LS-SVM) based on linear regression and nonlinear regression. Firstly, Projection Pursuit (PP) technology and Particle Swarm Optimization (PSO) algorithm are used to obtain the main factors of the rainfall, which optimize projection index from high dimensionality to a lower dimensional subspace. Secondly, using different linear regressions extract linear characteristics of the rainfall system, and using different Neural Network (NN) algorithms and different network architectures extract nonlinear characteristics of the rainfall system. Finally, LS-SVM regression is used for nonlinear ensemble model. This technique is implemented to forecast daily rainfall in Guangxi, China. Empirical results show that the prediction by using the LS-SVM ensemble model is generally better than those obtained using other models presented in this study in terms of the same evaluation measurements. The results suggest that our nonlinear ensemble model can be extended to meteorological applications in achieving greater forecasting accuracy and improving prediction quality. Keywords: Projection Pursuit, Particle Swarm Optimization, Neural Network, Least Square Support Vector Machine Ensemble.
1
Introduction
Rainfall prediction is a challenging task in the climate dynamics and climate prediction theory [1]. Accurate rainfall forecasting is one of the most important issues in hydrological research because early warnings of severe weather can help prevent casualties and damages caused by natural disasters [2,3]. With the Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 55–64. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com
56
J. Wu, M. Liu, and L. Jin
development of science and technology as well as intelligent computing in recent decades, many emerging techniques, such as Neural Networks (NNs), have been used in the rainfall forecasting. These efforts enhanced our understanding of intelligent computing and rainfall forecasting [4,5,6,7]. Short-term precipitation in the climate process is a very complex system, which it is not a simple linear or non-linear system. Therefore, it is difficult to effectively establish a model rainfall forecasting by the pure linear or pure nonlinear model. In other words, it is difficult to determine an effective predictor of the impact of weather systems due to the large dimension and noise in the weather system. The model will be complicated and the forecasting ability will reduce if there are many factors. Currently, modelling such a system focuses on: 1. how to extract sufficient feature information and how to reduce irrelevant information from the spatial data for the rainfall forecasting model; 2. how to set the effective forecasting rainfall modeling. In this paper, a novel method is presented for daily rainfall forecasting. This approach combines Least Square Support Vector Machine (LS–SVM), linear and nonlinear regressions. The Projection Pursuit technology and Particle Swarm Optimization (PP–PSO) are used to select the rainfall factors. Three linear regressions extract linear features and three NNs extract nonlinear features. The rainfall data in Guangxi is used as a case study for the development of daily rainfall forecasting model. The rest of the paper is organized as follows. Section 2 describes the model building process. For further illustration, this work employs the method to set up a prediction model for daily rainfall forecasting in Section 3. Finally, some concluding remarks are drawn in Section 4.
2
The Building Process of the LS-SVM Ensemble Model
Once data are collected, determining variables for modelling becomes possible. A large number of meteorological factors depends each others, and too much variables can enlarge the model scale, increase the complexity of the model. Therefore, the best way is to identify the variables that will save modelling time and reduces the problem space. Other reasons for the difficulties in forecasting rainfall are, such as the complexity of atmosphere-ocean interactions and the uncertainty of the relationship between rainfall and hydro-meteorological variables. The characteristic of rainfall is complex system dynamics under the influence of different meteorological factors. 2.1
Feature Extraction Used by PP–PSO
In the late 1970s, the international statistical society has developed a class of new statistical method to deal with and analysis high-dimensional data, which is called Projection Pursuit (PP) [8], which is suitable for dealing with nonlinear, non-normal distribution data to avoid “dimension disaster”, and it has succeeded in practice in many areas [9]. The basic idea of the method lies in: that
LS–SVM Ensemble for Daily Rainfall Forecasting
57
the computer technology is used to project high-dimensional data into the low dimensional subspace by numerically maximizing a certain objection function or projection index, which can extract the original data structure or characteristics, so as to achieve the goal of the study and analysis of high dimensional data. In rainfall system, X={xij , i = 1, 2, · · · , m, j = 1, 2, · · · , n} are m-dimensional factor of weather,where n is the number of the sample. A linear projection from Rm to Rp ,The matrix of the projectional direction is described as follow A = {ast > 0, s = 1, 2, · · · , p, t = 1, 2, · · · , m}
(1)
where p(p < m) is the number of new dimension for the factor matrix, and the row vector of A are orthogonal to each other. Then the new factor matrix has the form Z = A ∗ X = {zsj , i = 1, 2, · · · , p, j = 1, 2, · · · , n} (2) PP searches for a projection A maximizing a certain objection function. While synthesizing the projection value, the dispersal characteristic of projection value Z requests to be: the partial projection point should be as intensive as possible, it can be best to be some point zone, and in the total the point zone should be as wide as possible. The objective function can be written as max Q(a) = S(a)D(a) (3) s.t. A · AT = 1 Because of the complex nonlinear optimization, it is very difficult to solve the solution projectional direction by the traditional method. In order to extract the information, reduce dimension and decrease noise from the original variables, the following optimization models are used to find the solution projectional direction parameters with PSO. The fitness function of PSO is defined as follows: F itness = Q(a)
(4)
The results can be solved by a standard PSO algorithm for Equation 4. 2.2
Extraction of Linear Features by Three Traditional Linear Regression
Perhaps the most popular mathematical model for making predictions is the multiple linear regression model in the traditional mathematical method, which estimates one random variables using known values of multiple variables. There is a continuous random variable called the dependent variable Y , and a number of independent variables, x1 , x2 , · · · , xp . The goal of regression models is to fit a set of data with an equation, the simplest being a linear equation. The linear regression model is given by yˆ = β0 + β1 x1 + β2 x2 + · · · + βn xp + ε
(5)
where ε , the “noise”variable, is a normally distributed random variable with mean equal to zero and standard deviation σ, {βt , t = 1, 2, · · · , p} are the values
58
J. Wu, M. Liu, and L. Jin
of the coefficients. Unknown parameters values should be estimated from the smallest error sum of squares of the samples. Three types of linear regression models have been made in this paper for parsimony variables, such as stepwise multiple linear (SML) regression, partial least squares (PLS)regression and multi recursive regression method (MRR) to capture the linear characteristics of rainfall system. More detailed introductions to the three linear regression model are referred to the related literature [10,11,12]. 2.3
Extraction of Nonlinear Features by Three NNs Methods
NNs provide an interesting technique that theoretically can approximate any nonlinear continuous function on a compact domain to any designed of accuracy. The network learns by adjusting the interconnections (called weights) among layers. When the network is adequately trained, it is able to generalize relevant output for a set of input data. A valuable property of neural networks is that of generalization, whereby a trained neural network is able to provide a correct matching in the form of output data for a set of previously unseen input data. Learning typically occurs by example through training, where the training algorithm iteratively adjusts the connection weights (synapses). With the work about bias-variance trade-off of Bretiman [13], an NNs regression model consisting of diverse models with much disagreement is more likely to have a good generalization [14]. Therefore, how to generate diverse models is a crucial factor. In this paper, there are three methods for generating diverse models, such as multi–layer perceptions based on the Levenberg–Marquard algorithms (MLP–NN), the radial basis function network network (RBF–NN) and General Regression Neural Network (GR–NN) for nonlinear regression model. Readers interested in a more detailed introduction about these neural network algorithms are referred to the related literature [15]. 2.4
Nonlinear Ensemble Based on LS-SVM
LS-SVM is a least squares modification to the Support Vector Machine [16]. The major advantage of LS-SVM is that it is computationally very cheap while it still possesses some important properties of the SVM. In this section we will briefly discuss the LS-SVM method for a regression task. For more detailed information see [16]. We let {xi , i = 1, 2, · · · , N } be the output of linear and nonlinear forecasting predictors, {yi , i = 1, 2, · · · , N } be the aggregated output and the goal is to estimate a regression function f . Basically we define a N -dimensional function space by defining the mapping ϕ = [ϕ1 , ϕ2 , · · · , ϕN ]T according to the measured ˆ = ω T ϕ(x) + b where ω is a weight points. The LS-SVM model is of the form f (x) vector and b is a bias term. The optimization problem is the following: ⎧ N ⎨ min J(ω, ) = 12 ω T ω + γ 12 2i (6) i=1 ⎩ s.t. yi = ω T ϕ(xi ) + b + i , i = 1, 2, · · · , N
LS–SVM Ensemble for Daily Rainfall Forecasting
59
where the fitting error is denoted by i . Hyper-parameter γ controls the tradeoff between the smoothness of the function and the accuracy of the fitting. This optimization problem leads to a solution: ˆ = f (x)
N
αi K(x, xi ) + b
(7)
i=1
where αi is for the coefficients and K(x, xi ) = ϕT (x)ϕ(xi ) is the kernel. A common choice for the kernel is the Gaussian RBF: K(x, xi ) = e
x−xi 2 2σ2
(8)
The above-mentioned method can be summed up as follows: firstly, this paper is to extract the feature of the original meteorological factors from high dimension space to lower dimension subspace,and construct a forecasting variable by PSO algorithm optimization projection index. Secondly, three traditional linear regression methods are used to capture rainfall series linear characteristics. Thirdly, three different NNs models is trained to capture nonlinear patterns. Finally, the LS-SVM method are used for ensemble, which incorporate Linear model with Nonlinear model in an adaptive manner(LS–SVM–LNE). The basic flow diagram can be shown in Fig.1.
Randomly initialize population position and velocities
Initial value of projection directions
Calculate fitnessvalure by the original datd sets
Data sets (the main factor of weather) Yes
S atis fy s toping criteria?
Generate new factors of rainfall
Stepwise Muliple Regression Partial Leas t S quares Regres s ion
Multi Recursive Regression Levenberg-Marquard Neural Network
LS-SVM Ensemble
No Update particle best Update global best
Radial Basis Function Network Network
Update particle velocity Update particle position
General Regression Neural Network
Fig. 1. A Flow Diagram of The Proposed Nonlinear Ensemble Forecasting Model
3
Experiments Analysis
In this paper, PSO parameters are set as follows: the iteration times are 100; the population is 100; the minimum inertia weight is 0.1; the maximum inertia weight is 0.9 and the learning rate is 2.0. NNs parameters are set as follows: the number of neurons in the hidden layer range from 3. The learning rate is 0.9; the momentum factor is 0.7; the iteration times are 1000; the global error is 0.001.
60
3.1
J. Wu, M. Liu, and L. Jin
Empirical Data
Real–time ground rainfall data have been obtained in June from 2003 to 2008 in Guangxi by observing 89 stations, which 144 samples are modeled from 2003 to 2007 in June, other 30 samples are tested modeling in June of 2008. Method of modeling is one-step ahead prediction, that is, the forecast is only one sample each time and the training samples is an additional one each time on the base of the previous training. Due to the complex terrain of Guangxi and inhomogeneous rainfall, the region has been divided into three regional precipitation based on historical precipitation data by the cluster analysis method to reduce the difficulty of forecasting. Statistics for each district in the average daily precipitation is used as the forecasting object. Fig.2 shows three region maps.
Fig. 2. The Group Average Region Map of Guangxi Rainfall
In the first region as an example to show the process of modeling, it is very important to select the appropriate forecasting factors for model, different climatic variability and its effect have been discussed many times in the literature. In this paper, first of all, the candidate forecasting factors are selected from the numerical forecast products based on 48h forecast field, which includes:(1) the 17 conventional meteorological elements and physical elements from the T213 numerical products of China Meteorological Administration, the data cover the latitude from 15◦ N to 30◦ N, and longitude from 100◦ E to 120◦ E, with 1◦ ×1◦ resolution, altogether there are 336 grid points.(2) the fine-mesh precipitation data from the East Asia of Japanese Meteorological Agency, the data cover the latitude from 15◦ N to 30◦ N, and longitude from 100◦E to 120◦ E, with 1.25◦ × 1.25◦ resolution, altogether there are 221 grid points. We can get 60 variables as the main forecasting factors. Finally, a new information matrix can be extract form 60 variables by PP– PSO so we get 3 variables as the predictors in the first region. Fig.3 shows the data–sets of the main impact factors, which 60 variables and 144 samples. Fig.4 shows the data–sets of the predictors including 3 variables and 144 samples by PP–PSO extraction.
LS–SVM Ensemble for Daily Rainfall Forecasting
61
1
1
0.5
0.5
0
0
−0.5 −0.5
−1 1 1
0.5 0.5
0 0
−0.5
0.5 0
−0.5 −1
−1
−1
Fig. 3. The Main Factors Data–sets
3.2
−1 1
−0.5
0
0.5
1
Fig. 4. The Predictors Data–sets
The Performance Evaluation of the Model
In order to measure effectiveness of the proposed method, five types of errors, such as, the Maximum Absolute Error (MAE), the Root Mean Squares Error(RMSE), the Pearson Relative Coefficient (PRC), the Error Value more than 25mm (F1 ) and the Error Value less than 5mm (F2 ) which have been found in many papers, are also used here. Interested readers can be referred to [4] for more details. For the purpose of comparison, we have also built three other ensemble forecasting models: (1) Simple Average Ensemble (SAE-LE) - all the available linear forecasting output; 1 Tˆ1 = (y1 + y2 + y3 ) (9) 3 (2) Simple Average Ensemble (SAE-NE) - all the available nonlinear forecasting output; 1 Tˆ2 = (y4 + y5 + y6 ) (10) 3 (3) Simple Average Ensemble (SAE-LNE) - all the available linear and nonlinear forecasting output; 1 Tˆ3 = (y1 + y2 + y3 + y4 + y5 + y6 ) 6 3.3
(11)
Analysis of the Results
Fig.5. shows the fitting results of all model for 144 training samples, we can see that the fitting results of SL-SVM-LNE model are best in all models, the SAE–LE can basically fit the historical precipitation trends,the fitting results of the SAE-NE and the SAE-LNE are in good agreement with the actual values. The more important factor to measure performance of a method is to check its forecasting ability.
62
J. Wu, M. Liu, and L. Jin 80
140
Rainfall(mm)
Rainfall(mm)
Actural Rainfall SAE−LE Fitting SAE−NE Fitting SAE−LNE Fitting LS−SVM−LNE Fitting
70
60
Actural Rainfall SAE−LE Forecasting SAE−NE Forecasting SAE−LNE Forecasting LS−SVM−LNE Forecasting T213 Forecasting
120
100
50 80
40 60
30 40
20
20
10
0
0
20
40
60
80
100
120
140
5
10
15
20
Day
Fig. 5. Fitting of 144 Samples in June
25
30 Day
Fig. 6. Forecasting of 30 Samples in June
From Table 1, for the rainfall testing case in June, the MAE and RMSE of the LS–SVM–LNE model is also the smallest in all model. The minimum values of MAE and RMSE indicate that the deviations between original values and forecast values are very small. At the same time, from Table 1, the PRC of the LS–SVM–LNE model is the highest in all model. The accurate efficiency of the proposed model is measured as PRC, The higher values of PRC (maximum value is 1) indicate that the forecasting performance of the LS–SVM–LNE model is effective, which can capture the average change tendency of the cumulative rainfall data. The effective information of the proposed rainfall forecasting model is measured by F1 and F2 . If the Error Value is more than 15mm between the original value and the forecast value, the model predictions don’t provide effective information, if the Error Value is less than 5mm, the model predictions provide a piece of very important effective information for the future forecasting. From Table 1, the LS–SVM-LNE model has no unreliable information. Similarly, the LS–SVM-LNE model has reference information at least 86.67%. These results indicate that the proposed model can provide more useful information, avoid invalid information for the future forecasting. Table 1. A Comparison of Result of Ensemble Models about 30 Testing Samples Results Errors MAE RMSE PRC F1 (F1 /30) · 100% F2 (F2 /30) · 100%
SAE–LE
SAE–NE
SAE–LNE
LS–SVM–LNE
T213
79.4627 24.5914 0.5728 12 40.00% 9 30.00%
68.4590 17.7559 0.7543 6 20.00% 13 43.33%
73.7623 18.5157 0.7062 8 26.67% 16 53.33%
49.8746 13.5746 0.8957 0 0.00% 26 86.67%
62.7980 35.4308 0.0129 11 36.67% 10 33.33%
LS–SVM Ensemble for Daily Rainfall Forecasting
63
Furthermore, the same method is used to train precipitation data and predict precipitation at the other two regions. The experimental results also show that LS–SVM–LNE method is better than other methods. From the experiments presented in this paper, the LS–SVM ensemble forecasting model is superior to the traditional linear regression model, the individual NNs model as well as the linear ensemble models for the test cases of daily rainfall.
4
Conclusions
Accurate rainfall forecasting is crucial for a frequent unanticipated flash flood region to avoid life losing and economic loses. This paper proposes a LS–SVM ensemble forecasting model that combines three traditional linear regression model and three different NNs model to predict daily rainfall based on PP–PSO. In terms of the different forecasting models, empirical results show that the nonlinear ensemble model performs the best for the test cases of three regions Guangxi, China on the basis of different criteria. In the LS–SVM ensemble model test cases, the MAE and RMSE are the lowest and the PRC is the highest. This indicates that the LS–SVM ensemble forecasting model can be used as an alternative tool for daily rainfall forecasting to obtain greater forecasting accuracy and improve the prediction quality further in view of empirical results, and can provide more useful information, avoid invalid information for the future forecasting.
Acknowledgment This work was supported in part by the Key Natural Science Foundation of Guangxi under Grant No.0832019Z, and in part by the Department of Guangxi Education under Grant No. 200807MS098.
References 1. Jiangsheng, W., Long, J.: Forecast Research and Applying of BP Neural Network Based on Genetic Algorithms. Mathematics in Practice and Theory 35(1), 83–88 (2005) 2. Jiansheng, W., Long, J., Mingzhe, L.: Modeling Meteorological Prediction Using Particle Swarm Optimization and Neural Network Ensemble. In: Wang, J., Yi, Z., ˙ Zurada, J.M., Lu, B.-L., Yin, H. (eds.) ISNN 2006. LNCS, vol. 3973, pp. 1202–1209. Springer, Heidelberg (2006) 3. Nasseri, M., Asghari, K., Abedini, M.J.: Optimized Scenario for Rainfall Forecasting Using Genetic Algorithm Coupled with Artificial Neural Network. Expert Systems with Application 35, 1414–1421 (2008) 4. Jiansheng, W., Enhong, C.: A Novel Nonparametric Regression Ensemble for Rainfall Forecasting Using Particle Swarm Optimization Technique Coupled with Artificial Neural Network. In: Yu, W., He, H., Zhang, N. (eds.) ISNN 2009. LNCS, vol. 5553, pp. 49–58. Springer, Heidelberg (2009)
64
J. Wu, M. Liu, and L. Jin
5. Yingni, J.: Prediction of Monthly Mean Daily Diffuse Solar Radiation Using Artificial Neural Networks and Comparison with other Empirical Models. Energy Policy 36, 3833–3837 (2008) 6. Govindaraju, R.S.: Artificial Neural Network in Hydrology, I: Preliminary Concepts. Journal of Hydrologic Engineering 5(2), 115–123 (2000) 7. French, M.N., Krajewski, W.F., Cuykendal, R.R.: Rainfall Forecasting in Space and Time Using a Neural Network. Journal of Hydrology 137, 1–37 (1992) 8. Friedman., J.H., Turkey, J.W.: A Projection Pursuit Algorithm for Exploratory Data Analysis. IEEE Transaction On Computers 3(9), 881–889 (1974) 9. Filzmoser, P., Serneels, S., Croux, C., Van Espen, P.J.: Robust Multivariate Methods: The Projection Pursuit Approach. In: Spiliopoulou, M., Kruse, R., N¨ urnberger, A., Borgelt, C., Gaul, W. (eds.) From Data and Information Analysis to Knowledge Engineering Proceedings of the 29th Annual Conference of the Gesellschaft f¨ ur Klassifikation e.V., University of Magdeburg, vol. 8, pp. 270–277. Springer, Heidelberg (2006) 10. Johenson., D.: Applied Ultivariate Methods for Data Analysts, 2nd edn. Thomson Learning Press, UK (1998) 11. Helland, I.S.: PLS Regression and Statistical Models. Scandivian Journal of Statistics 179, 97–114 (1990) 12. Kenneth, D.S.: Using recursive regression to explore nonlinear relationships and interactions: A tutorial applied to a multicultural education study. Practical Assessment Research and Evaluation 14(3), 1–13 (2009) (Retrieved March 3, 2009) http://pareonline.net/getvn.asp 13. Jiansheng, W.: A Novel Artificial Neural Network Ensemble Model Based on K–nn Nonparametric Estimation of Regression Function and Its Application for Rainfall Forecasting. In: 2nd Internatioal Joint Conference on Computational Sciences and Optimization, pp. 44–48. IEEE Computer Society Press, New York (2009) 14. Benediktsson, J.A., Rveinsson, J., Ersoy, O.K., Swain, P.H.: Parallel Consensual Neural Neural Networks. IEEE Transactions on Neural Networks 8, 54–64 (1997) 15. Fredric, M.H., Ivica, K.: Principles of Neurocomputing for Science & Engineering. McGraw-Hill, New York (2001) 16. Suykens, J., Van Gestel, T., De Brabanter, J., De Moor, B., Vandewalle, J.: Least Squares Support Vector Machines. World Scientific Publishing Press, Singapore (2002)
Estimating Portfolio Risk Using GARCH-EVT-Copula Model: An Empirical Study on Exchange Rate Market Zongrun Wang1, Yanbo Jin2, and Yanju Zhou1 1
2
School of Business, Central South University, Changsha 410083, China Department of Finance, California State University, Northridge 91325, USA
Abstract. This paper introduces GARCH-EVT-Copula model and applies it to study the portfolio risk of exchange rates. Multivariate Copulas including Gaussian Copula, t Copula and Clayton Copula were used to describe the structure and extend the analysis from bivariate to any n-dimension. We apply this methodology to study the returns of a portfolio of four major foreign currencies in China. Our results suggest that the optimal investment allocations are similar across different Copula and confidence levels and the optimal investment concentrates in the USD investment. Generally speaking, t Copula and Clayton Copula can better portray the correlation structure of multiple assets than Normal Copula. Keywords: GARCH-EVT-Copula model, Exchange rate, Portfolio risk, VaR and CVaR.
1 Introduction Value-at-Risk (VaR) is one of the most important measures of the market risk and Conditional VaR (CVaR) has emerged as a supplement to VaR. It provides a more conservative measure of losses relative to VaR. Extreme value theory (EVT) focuses directly on the tails and could therefore potentially give us better estimates and forecasts of risk. But applying EVT to the return series is inappropriate as they are not independent and identically distributed. Thus following the approach of McNeil and Frey [1], we use GARCH model to fit the return series and apply EVT to the residuals rather than to the return series. Several researchers home and abroad have made great progress using this method to measure VaR, such as Hans N. E. Bystrom [2], Malay Bhattacharyya [3], Xiao [4] and so on. On the other hand, when measuring the risk of a multi-asset portfolio, one needs to model the relation between the risk factors. This has become an active line of research in the finance field in recent years. Copulas are widely used along this line of research, because it is able to extract the dependence structure from the joint probability distribution function and, simultaneously, to isolate such dependence structure from the univariate marginal distributions. However, most empirical studies have focused on the equity risk using Gaussian Copula and t Copula. Among the few studies that did focus on the exchange rate risk. The Archimedean Copula is rarely used because of implementation issues and the Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 65–72. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
66
Z. Wang, Y. Jin, and Y. Zhou
analysis was limited to two-dimension [5,6]. Tursunalieva [7] extends [5,6] and shows that the SJC-Copula captures the tail dependence between the exchange rates in pre-euro and post-transition periods, while the rotated Gumbel Copula captures the dependence during the transition period. In China, Wu [8] was the first to analyze two-asset portfolio of foreign exchanges using Archimedean Copula. Sun [9] considered the EUR/RMB and JPY/RMB exchange rates, using variety of Archimedean Copula to fit the data and derived the optimal Copula to capture the two variable’s correlation. Wang [10] uses EVT to measure the exchange rate risk and builds Copula-EVT model and calculates the VaR for the exchange rate portfolio of EUR/CNY and JPY/CNY by Monte Carlo simulation. On the other hand, some researchers combine GARCH model and Copula to dynamically study the correlation and risk of financial assets, such as Jondeau [11,12], Tae-Hwy lee [13], Wu[14] and so on. In this paper, we introduce Clayton Copula to apply to the exchange rate portfolio. The contributions of our paper are two folds. First, in addition to Gaussian Copula and t Copula, we use the multivariate Archimedean Copula, in particular the Clayton Copula, to analyze the asymmetric dependence structure among financial asset returns. Second, we extend the bivariate copula methods in a trivariate context, and use GARCH nested copulas with EVT margins to estimate the risk of fourdimensional exchange rate portfolios.
2 GARCH-EVT Model and Copula Parameter Estimates 2.1 GARCH-EVT Model Recognizing the heteroskedasticity and volatility clustering nature of time series, we first use GARCH (1,1) to model the time series. The model is:
⎧⎪ rt = μ + ε t = μ + σ t zt ⎨ 2 2 ⎪⎩σ t = w + αε t −1 + βσ t −1
(1)
Where w > 0, α ≥ 0, β ≥ 0, α + β < 1 . rt is the actual return, μ is the expected return and σ t is the volatility of the returns on day t . zt comes through the stochastic variables, which are the residuals or the innovations of the process. Conventionally these residuals are assumed to follow a normal distribution. However, the conditional distribution of GARCH models has been shown to have a heavier tail than that of a normal distribution, which provides better approximation to actual financial time series. Furthermore, EVT appears to be an appropriate approach for modeling the tail behavior. However, applying EVT to the random variable rt is inappropriate as rt is not i.i.d. Therefore, our marginal model is built on GARCH-EVT model, in which we use GARCH model to fit the historical return data, then model the new series by the GPD distribution in the lower or upper tail and by the empirical distribution in the remaining part. The distribution of innovation is followed by (see details in Stuart Coles [15]):
Estimating Portfolio Risk Using GARCH-EVT-Copula Model −1/ ξ ⎧N ⎧ L ⎫ ⎪ u L ⎨1 + ξ L u − zt ⎬ zt < u L ⎪ N ⎩ βL ⎭ ⎪⎪ F ( zt ) = ⎨φ ( z ) u L < zt < u R ⎪ −1/ ξ R R ⎪ Nu R ⎧ R u − zt ⎫ zt > u R ⎬ ⎪1 − N ⎨1 + ξ R β ⎩ ⎭ ⎪⎩
67
L
(2)
Where β , ξ are the scale and shape parameter respectively, u R ( u L ) is the upper (lower) threshold. In order to get β , ξ exactly, we must choose the proper threshold. 2.2 Copula Parameter Estimates
The following theorem is known as Sklar’s Theorem. It is the most important theorem about copula functions because it is used in many practical applications. Theorem: Let F be an n-dimensional c.d.f. with continuous margins F1 , F2 , it has the following unique copula representation:
xn ) = C ( F1 ( x1 ), F2 ( x2 ),
F ( x1 , x2 ,
Fn ( xn ))
Fn .Then
(3)
We can see that, for continued multivariate distribution, the univariate marginal distribution and the multivariate dependence structure can be separated. Assuming that the dependence structure does not change with time, we select Gaussian Copula, t Copula and Clayton Copula to estimate the joint distribution. For n-variable Gaussian Copula, Let φ∑ be the standardized multivariate normal distribution with correlation matrix ∑ . It can be defined as follows:
C Gu (u1t , u2t ,
, unt ) = φ∑ (φ −1 (u1t ), φ −1 (u2t ),
, φ −1 (unt ))
(4)
Where φ −1 denotes the inverse of the normal cumulative distribution function. From (2) we know the marginal distribution is Fi ( z ) . Based on the historic data {z1t , z2t znt }, t = 1, 2 T ,we set: ut = (u1t , u2t
unt ) = ( F1 ( z1t ), F2 ( z2t )
ξt = (φ (u1t ), φ (u2t ), φ (unt )) −1
−1
−1
Fn ( znt ))
(5)
So we can get the simple expression: C Gu (ut ) = φ∑ (ξt ) . And using MLE method, ∑ is estimated as follows: T ˆ = 1 ξξ ' ∑ ∑ t t T t =1
(6)
To capture the fat tail property, we introduce the multivariate student’s t Copula which shows more observations in the tails than the Gaussian. Let tv , ∑ be the
68
Z. Wang, Y. Jin, and Y. Zhou
standardized multivariate t distribution with correlation matrix ∑ and v degrees of freedom. The t Copula can be defined as follows: C t (u1t , u2t ,
−1
−1
unt ) = tv , ∑ (tv (u1t ), tv (u2t ),
−1
(7)
tv (unt ))
−1
Where tv denotes the inverse of the Student’s t cumulative distribution function. Based on the historic data and the given degree of freedom v , we set: ut = (u1t , u2t −1
unt ) = ( F1 ( z1t ), F2 ( z2t )
ζ t = (tv (u1t ), t (u2t ), −1 v
Fn ( znt ))
(8)
−1 v
t (unt ))
So we have C t (ut ) = tv , ∑ (ζ t ) . The parameter matrix ∑ is also estimated using MLE method. The initial matrix is the correlation coefficient matrix of multivariate normal ˆ Copula function made from (6). We can get the correlation coefficient matrix ∑ of n +1
multivariate t Copula through the following iterative calculation method:
ζ tζ t ' 1 ⎛v+n⎞ T ˆ k = 1, 2, ∑ k +1 = ⎜ ⎟⋅∑ T ⎝ v ⎠ t =1 1 + 1 ζ ' ∑ ˆ ζ t k t v
(9)
ˆ =∑ ˆ . In addition to the fat tails, we can try to Repeat the above process until ∑ n +1 n check whether asymmetry exists in the tails. We use the Clayton Copulas, one of the Archimedean families better known in capturing left tail dependence. The Clayton Copula’s cumulative distribution functions are defined by the following:
C clayton (u1 , u2 ,
, un ) = (u1−θ + u2 −θ +
+ un −θ − n + 1)
−
1
θ
,θ ≥ 0
(10)
Base on the historic data and (2), we have ui = Fi ( zit ) and can directly estimate the parameter θ of n-variable Clayton Copula using MLE method.
3 Simulation Algorithm and Portfolio Risk Analysis 3.1 Simulation Algorithm
ˆ First, for ∑
derived from (6), we do Cholesky decomposition and then
have ∑ = A ' A . Second, draw independent n-dimensional vectors from the multiple normal distribution x = ( x1 , x2 ,
, xn ) ' , xi ∼ N (0,1) , set y = A ' x ,then
z = ( F (φ ( y1 )), F2−1 (φ ( y2 )),
Where Fi −1
, i = 1, 2,
−1 1
, F1−1 (φ ( yn )))
(11)
, n is the inverse of distribution of Fi in (2). Third, repeat the
above step and simulate M times, we can get vector ( z1m , z2 m , , znm ) ' m = 1, 2 M . Then restoring it into (1) we can get M returns at the time t+1. The returns residuals’
Estimating Portfolio Risk Using GARCH-EVT-Copula Model
69
joint distribution is this Gaussian Copula. The returns can be defined by rt +1 = (r1m , r2 m , , rnm ) ' = ( μ1 + z1mσ 1,t +1 , μ 2 + z2 mσ 2,t +1 , , μ n + znmσ n ,t +1 ) ' , Where σ i ,T +1 , μi , i = 1, 2,
, n are calculated by GARCH (1,1) model. ˆ derived from (9) we firstly do Cholesky decomposition. On the other hand, for ∑ After that, generate n random vectors which are i.i.d, then produce random vector s subjecting to Chi-square distribution and independent from x , and set: z = ( F1−1 (t (
,
y1 s/v
y2
)), F2−1 (φ (t (
s/v
))),
, Fi −1 (φ (
yi s/v
)))
(12)
Where Fi −1 i = 1, 2, , n is the inverse of distribution of Fi in (2). Finally, repeat the above step M times, we can also get the vector and the simulating returns via t Copula at the time t+1. A hurdle for practical implementation of any multivariate Archimedean Copula was the absence of an efficient method to generate returns. Marshall and Olkin proposed an alternative method, which is computationally straightforward than the conditional distribution approach. It requires the generation of an additional variable. 1 1 ˆ −1 = t − θ − 1 , Considering the inverse function of Gamma( ,1) ’s LT transformation Ga
θ
it is different from the generator φ (t ) =
1
θ
(t −θ − 1) of Clayton Copula just a constant
and it does not affect generating Clayton Copula. So the algorithms of simulating returns via Clayton Copula are as follows: First, Generate M random variables Ym , m = 1, 2 M which obey the function 1 Gamma( ,1) , where the parameter θ is estimated using MLE. Second, simulate
θ
independent
uniform
random
variables y1 , y2 ,
, yn .Third,
Let uim = Ga(
− ln( yi ) ) , i = 1, 2, Ym
u = (u1 , u2 ,
, un ) is the n-dimension Clayton Copula with the parameter θ , then
, n, m = 1,
M , then the joint distribution of
−1
zim = Fi (uim ), i = 1, 2, , n, m = 1, 2, , M . Finally, similar to the last step of simulating via Gaussian Copula or t Copula above, we can also get M returns via Clayton Copula at the time t+1. 3.2 Portfolio Risk Analysis 3.2.1 VaR and CVaR of the Equally-Weighted Portfolio As we know, the above algorithms have simulated the returns at time t+1 of ndimension time series using the three Copula functions to describe the correlation structure. Now suppose we want to compute the empirical VaR and CVaR of an equally-weighted portfolio. Assuming there are n-dimension assets we can invest and
70
Z. Wang, Y. Jin, and Y. Zhou
1 1 1 the weight in each asset is the same, that is w = [ , , , ]' , according to the M n n n returns samples simulated above, we can get M total returns of the portfolio: Re turns = rt +1 * w = (r1m , r2 m , , rnm ) * w . As a result, we can get the empirical distribution and thus the VaR and CVaR of the equally-weighted portfolio. 3.2.2 Optimal Portfolio with Minimum Risk The discussion above focused on estimating the VaR and CVaR of an equallyweighted portfolio. However, for individual investors, one of the major concerns is to minimize risk of the investment portfolio. As a result, the following algorithm computes the optimal weights of each asset which minimizes the portfolio risk. Assuming that the weight in individual assets within a portfolio is w = [ w1 , w2 , , wn ]' ,
where 0 ≤ wi ≤ 1 and w1 + w2 + + wn = 1 . Needless to say, there are many weight vectors that satisfy these conditions. For each given investment weight vector w , we can get M returns of the portfolio; that is, Re turns = rt +1 * w = (r1n , r2 n , , rnm ) * w and compute VaR of portfolio under a given confidence. After getting all VaR values corresponding to all the weight vectors, we can find the minimum VaR and return back to get the weight in each asset; that is, min VaRα = risk ( w1 , w2 , , wn ) , also we can get the CVaR value under this given investment proportion.
4 Empirical Studies in Foreign Exchange Market We choose four foreign exchange rates including USD/CNY, EUR/CNY, JPY/CNY, HKD/CNY to demonstrate the application of GARCH-EVT-Coupla model. As we know, China has started implementing the new exchange rate policy since July 21,2005. The data series used here are from July 25, 2005 to July 25, 2008.1. We use the daily logarithmic returns defined as rt = ln Pt − ln Pt −1 . The data series give us 732 observations for each exchange rate. 4.1 Summary Statistics
The descriptive statistics of daily logarithmic returns show that all series have heavy tails and they do not follow normal distribution. Take USD/CNY as an example, the ADF test result of USD/CNY indicates that there is no unit root and that the series follow a stationary stochastic process. Then we perform ARCH LM test to find that USD/CNY is a stationary stochastic process, but with heteroskedasticity. And the analysis on the other three exchange rates shows similar properties. As a result, we can use GARCH (1,1) model as in (1) to study the exchange rate movement. 4.2 GARCH-EVT-Copula Application
We first establish GARCH-EVT model to the each RMB exchange rate series and get the marginal distribution of residuals. Second use Multivariate Gaussian Copula, t 1
Data source: http://www.safe.gov.cn/model_safe/index.html
Estimating Portfolio Risk Using GARCH-EVT-Copula Model
71
Copula and Clayton Copula to describe the correlation of the residuals and obtain the Copula parameters. Finally adopt Monte Carlo to simulate returns considering the correlation structure, calculate the risk value under the same investment proportion and find the optimal investment proportion under the given confidence. As described in earlier sections, we can determine the threshold μ . The other parameters can be obtained using the method of MLE. We find that GPD fits the residuals of all series perfectly. After obtaining the parameters of GPD and the residuals i = 1, 2,3, 4, t = 1, 2 T , we substitute zit into (2) and get the marginal distribution ui = f ( zi ) . According to the parameter estimation method in the section 2.1 we can get the parameters of Copulas. That is the correlation matrix ∑ of Gaussian Copula, ∑ and freedom degree v of t Copula, the parameter θ of Clayton Copula. Applying the simulation algorithm described in section 2.2 above, we can simulate the returns at the time t+1 considering correlation structure using the three types of Copula. When the investment proportion of the four foreign exchanges were equal we can calculate VaR and CVaR of the portfolio. In addition, we can find the optimal investment proportion to minimize the portfolio risk. The results indicate that under the same confidence level, VaR(CVaR) calculated from t Copula and Clayton Copula is less than that from Gaussian Copula. That is because t Copula and Clayton Copula consider the tail correlation but the Gaussian Copula does not. In addition, the optimal portfolio weights under minimum portfolio risk are similar across different Copula and confidence levels. The investment concentrates on USD, although the price of USD/CNY has decreased since 2007 and the volatility has increased. However, the volatility of USD/CNY is still smaller than that of EUR/CNY and JPY/CNY, as a result, during the sample period of July 2005-July 2008, USD investment accounted for the largest proportion in the optimal investment under minimum portfolio risk. With higher confidence level, the proportion of USD investment becomes smaller and the proportions of the other foreign currencies become larger. When α = 0.9 , VaR using Clayton Copula is larger than those using t Copula and Gaussian Copula. However, with higher confidence level, Clayton Copula better captures the correlations and results in VaR that is similar to the result using t Copula. As a result, under the higher confidence level, we can choose t Copula and Clayton Copula to calculate the portfolio risk.
,
5 Conclusions In this paper we introduce the GARCH-EVT-Copula model and use it to measure the risk of foreign exchange portfolio with multiple foreign exchanges. When modeling the correlation structure we not only apply the conventional Gaussian Copula and t Copula, but also the multivariate Clayton Copula which was rarely used because the complexity involved in programming. We find that for an equally-weighted foreign exchange portfolio, VaR(CVaR) calculated from t Copula and Clayton Copula is less than that computed from the Gaussian Copula. This is because t Copula and Clayton Copula model the tail correlation but the Gaussian Copula does not. On the other hand, when minimizing portfolio risk, optimal portfolio weights are similar across different copula and
72
Z. Wang, Y. Jin, and Y. Zhou
different confidence levels. The optimal investment tends to concentrate in the investment of USD. With higher confidence level, weight in USD decreases while weight in EUR and JPY increases. In addition, we show that t Copula and Clayton Copula provide better approximation to capture portfolio risk compared to Gaussian Copula. These findings can help investors better manage the exchange rate risk. Acknowledgments. The work was supported by the National Natural Science Foundation of China under the grant No. 70973145 and 70771114.
References 1. Frey, R., McNeil, A.J.: Estimation of tail-related risk measures for heteroscedastic financial time series: an extreme value approach. Journal of Empirical Finance 7, 271–300 (2000) 2. Bystrom, H.N.E.: Managing extreme risks in tranquil and volatile markets using conditional extreme value theory. International Review of Financial Analysis 13, 133–152 (2004) 3. Malay, B., Gopal, R.: Conditional VaR using EVT- Towards a planned margin scheme. International Review of Financial Analysis 17, 382–395 (2008) 4. Xiao, Z., Fu, X.X., Zhang, B.: Dynamic VaR Risk Measures Based on EVT-POTFIGARCH. Nankai management review 11(4), 100–104 (2008) (in Chinese) 5. Patton, A.J.: Modelling time-varying exchange rate dependence using the conditional Copula. Working Paper of London School of Economics & Political Science (2001) 6. Patton, A.J.: Modeling asymmetric exchange rate dependence. International Economic Review 2(47), 527–556 (2006) 7. Tursunalieva, A., Silvapulle, P.: Assessing and modeling the changes in dependence between exchange rates. Working Paper of Monash University (2007) 8. Wu, Z.X., Ye, W.Y.: Risk Analysis of Foreign Exchange Markets by Copula. Chinese management science 4, 1–5 (2004) (in Chinese) 9. Sun, L.J., Bai, M.Y.: The Research of Optimal Archimdean Copula Based Exchange Rate Correlation. Data Analyses 3(2), 91–102 (2008) 10. Wang, Z.R., Chen, C.: The exchange rate risk of Chinese yuan: using VaR and ES based on extreme value theory. Journal of Applied Statistics 37(2), 265–282 (2010) 11. Eric, J., Micheal, R.: Copula-GARCH model of Conditional Dependency of Financial Series: the Copula-GARCH Model. Working Paper (2002) 12. Eric, J., Micheal, R.: The Copula-GARCH model of Conditional Dependency: An international stock market application. Journal of International Money and Finance 25, 827–853 (2006) 13. Lee, T.H., Long, X.D.: Copula-based multivariate GARCH model with uncorrelated dependent errors. Journal of Econometrics 150, 207–218 (2009) 14. Wu, Z.X., Chen, M., Ye, W.Y.: Risk Analysis of Portfolio by Copula-GARCH. Journal of Systems Engineering Theory and Practice 2(8), 45–52 (2006) (in Chinese) 15. Coles, S.: An Introduction to Statistical Modeling of Extreme Values. Series in Statistics, New York (2001)
Forecasting Financial Time Series via an Efficient CMAC Neural Network Chi-Jie Lu1 and Jui-Yu Wu2,* 1
Department of Industrial Engineering and Management, Ching Yun University, Taiwan
[email protected] 2 Department of Business Administration, Lunghwa University of Science and Technology, Taiwan
[email protected]
Abstract. Cerebellar model articulation controller neural network (CMAC NN) has many advantages, such as very fast learning, reasonable generalization capability and robust noise resistance. Thus, CMAC NNs are conventionally used in robot control. To solve financial time series forecasting, this paper presents an efficient CMAC NN scheme. The proposed CMAC NN transforms continuous values of input variables to discrete indexes by using a quantization operator. To enhance generalization ability, the CMAC NN employs high quantization resolution and a large generalization size. To perform many-to-few mappings, the CMAC NN uses an efficient and fast hashing code based on bitwise XOR operator. The proposed CMAC NN was used to Nikkei 225 closing cash indexes collected from Japanese stock market. The forecasting results of the proposed CMAC NN were compared with those of support vector regression (SVR), which is statistical/ machine learning algorithm. Experimental results indicate that the performance of the CMAC NN was better than SVR in the tested case. Therefore, the CMAC NN may be considered as an efficient tool for forecasting financial time series. Keywords: cerebellar model articulation controller, neural network, stock price prediction, support vector regression.
1 Introduction Over the past three decades, there has been a growing interest in financial time series forecasting [1, 2]. Stock index forecasting is regarded as a challenging task of the financial time series prediction process since the stock market is a complex, evolutionary, and nonlinear dynamic system [3]. Many techniques have been applied to the domain of time series prediction, and are traditionally categorized as statistical methods and spectral analysis [4, 5]. Zhang et al. [6] found that the traditional statistical methods based on linear models, such as Box-Jenkins approach, have difficulty in measuring the performance of real systems with nonlinear behavior. Rigoze et al. [7] compared the advantages and drawbacks of four classical spectral analytical methods. These approaches are difficult to apply for general practitioners; Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 73–82. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
74
C.-J. Lu and J.-Y. Wu
that is, prior knowledge of signal processing is required to adjust specific parameters for these spectral analytical methods. Artificial neural networks (ANNs) which can accurately model nonlinear systems. Unlike traditional statistical models, NNs are data-driven and non-parametric models. A cerebellar model articulation controller (CMAC) is a supervised NN. Albus [8, 9] first introduced the CMAC NN based on the functions of the human cerebellum. The CMAC NN has the advantages of very fast learning, reasonable generalization ability and robust noise resistance [10]. It has been successfully used in many applications, such as control [11, 12], fault detection [13, 14] and classification [15, 16]. Moreover, CMAC NNs have been used to predict, such as electricity price forecasting and power system marginal price forecasting [17, 18]. However, this paper attempted to extend the application field to stock price forecasting. This paper presented a CMAC NN scheme for stock index prediction. Moreover, this paper compared the performance of the CMAC NN and support vector regression (SVR), which is a novel machine learning algorithm, and is increasing attention to solve nonlinear regression estimation problems.
2 Literature Review 2.1 CMAC NN A CMAC NN comprises five cells, input space ( X ), sensory cell ( S ), association cell ( A ), physical memory cell ( P ) and output cell ( Y ), and transforms input values into output values using a series of mappings, as follows: (1) X → S A quantization operator is used to transform components of input vectors into discrete quantization indexes. Each component xi of input pattern j is quantized individually by:
⎡⎛ N sij = ⎢⎜ max i min ⎜ ⎣⎢⎝ xi − xi
⎤ ⎞ min ⎥ − 1, i = 1, 2,… , n j = 1, 2,… , Dtotal ⎟⎟ × xij − xi ⎥⎦ ⎠
(
)
sij = 0, if sij < 0 where sij = quantization index of component xi of input pattern j xij = component xi of input pattern j ximin = minimum value of input vector xi ximax = maximum value of input vector xi Ni = quantization resolution of input vector xi n = dimension of input space Dtotal = total number of input patterns
[⋅]
= rounded number
(1)
(2)
Forecasting Financial Time Series via an Efficient CMAC Neural Network
75
The indexes sij depends on ximin , ximax and Ni . Larger values of Ni improve the accuracy of representation, but require a large weight table w . (2) S → A A sensory cell comprises random tables [19]. The size of each random table is calculated by: Ci = Ni + g − 1, i = 1, 2,… , n
(3)
where
Ci = size of the random table of input vector xi g = generalization size The indexes sij are mapped to random table i , and segment mappings are created based on g . The quantization and segment mappings generate natural interpolation, and give the CMAC NN the ability to generalize [20]. A large g increases generalization ability, but reduces the approximation accuracy. (3) A → P → Y A vector a in cell A corresponds to a table w in cell P . The table w stores many weights. An actual output value in cell Y can be computed by the matrix operation:
yo = aT w
(4)
where ⎡ a1 ⎤ ⎢ ⎥ a a = ⎢ 2⎥ , w = ⎢ ⎥ ⎢ ⎥ ⎣ ak ⎦
⎡ w1 ⎤ ⎢ ⎥ ⎢ w2 ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ wk ⎦
y o = actual output, k = size of w and T = transpose of a matrix. 2.2 SVR The SVR is an adaptation of recently introduced statistical/ machine learning theory based classification paradigm namely, support vector machines. For illustrating the concept of SVR, a typical regression problem is formulated. Consider a dataset G = ( xij , yd , j ) ( i = 1, 2,..., n , j = 1, 2,..., Dtotal ), where xij are input patterns and yd , j is desired output. The goal of the regression analysis is to determine a function f (x) , so as to predict accurately the yd , j . The regression problem can be classified as linear and nonlinear regression problems. As the nonlinear regression problem is more difficult to deal with, SVR was mainly developed for dealing with the nonlinear regression problem. SVR algorithms have been successfully applied in time series prediction, such as and financial time series forecasting [21, 22]. When constructing a
76
C.-J. Lu and J.-Y. Wu
forecasting model, one major concern is increasing the accuracy of predictions. This would be especially beneficial in areas like stock price forecasting, where even the smallest improvement in prediction accuracy can have a positive impact on investments. For building SVR forecasting model, the LIBSVM package proposed by Chang and Lin [23] is used in this paper since the package can be easy to find and use.
3 Method A CMAC NN was developed and used to Nikkei 225 closing cash index dataset collected from Japanese stock market. Figure 1 shows the pseudo-code of the proposed CMAC NN, which is described below. begin Training stage: h←0 Step 1: quantize and normalize Step 2: create random tables and weight table While h < epochmax do for j = 1,2,…, Dtrain do Step 3: generate address table Step 4: calculate of actual output Step 5: learning rule endFor Step 6: evaluate training accuracy h ← h +1 endWhile Testing stage: Step 7: measure testing accuracy end
Fig. 1. The pseudo-code of the proposed CMAC NN
Step 1: Quantize and normalize Equations (1) and (2) are used to quantize components of input patterns. Each desired output
yd , j ( j = 1, 2,… , Dtotal ) undergoes normalization, as follows: yd' , j =
yd , j − ydmin ydmax − ydmin
( Emax − Emin ) + Emin ,
j = 1, 2,… , Dtotal
where
yd' , j = normalized desired output j ydmin = minimum value of desired output vector Yd = [ yd ,1 , yd ,1 ,… yd , Dtotal ]T ydmax = maximum value of Yd Emin = minimum value of expected output Emax = maximum value of expected output The values [ Emin , Emax ] are generally set to [0.2, 0.8].
(5)
Forecasting Financial Time Series via an Efficient CMAC Neural Network
77
Step 2: Create random tables and weight table w Equation (3) is used to compute the size of each random table, and the random tables are then created. To control the range of the index of generated address table, each random table consists of uniform random numbers created from the interval [ 0, k / 2 ]. A table w is generated based on k as defined in Section 2, with initial weights set to “0”. To reduce the probability of hash collision, this paper used a large value of k = 10000. Step 3: Generate address table To implement many-into-few mapping, the proposed CMAC NN used a hash coding based on bitwise XOR operator [24]. The detail description of the hash coding can be found in the literature [15]. Step 4: Calculate actual output Equation (4) is used to measure the values yo, j ( j = 1, 2,… , Dtrain ). Step 5: Learning rule Equation (6) is employed to modified the weights in w , as follows: wl (h + 1) = wl (h) +
β ( yd − yo ) g
, l = 1, 2,… , k
h = 1, 2,… , epochmax
(6)
where β = learning rate Step 6: Evaluate training accuracy A normalized root mean square error (NRMSE) is used as a performance index to evaluate a training accuracy, as follows: Dtrain
∑
NRMSE =
j =1
( yd' , j − yo, j )2
(7)
Dtrain
where Dtrain = number of input patterns for training Steps 3–6 are repeated until the termination condition epochmax is met. Step 7: Measure testing accuracy Input pattern j ( j = Dtrain + 1, Dtrain + 2,… , Dtotal ) is used for testing. The actual outputs of recall of the CMAC NN can be calculated using the w obtained in the training stage (Steps 1−6), and then the generalization accuracy of the CMAC NN can be measured by a testing NRMSE.
4 Experimental Results For evaluating the performance of the proposed CMAC NN forecasting model, the daily Nikkei 225 closing cash index dataset is used in this paper. In forecasting Nikkei
78
C.-J. Lu and J.-Y. Wu
225 closing cash index, the Nikkei 225 index future prices are used as forecasting variables since the changes of the future prices lead price changes of the cash market [25, 26]. Four forecasting variables (Singapore Exchange-Derivative Trading Limited, Osaka Securities Exchange, Chicago Mercantile Exchange and the previous day’s cash market closing index) are used for predicting the Nikkei 225 closing cash index. The daily data of futures and cash prices from December 6, 2004 to December 30, 2008 of the Nikkei 225 cash index provided by Bloomberg are collected in this paper. There are totally 1000 data points in the dataset and the daily Nikkei 225 closing cash prices are shown in Fig. 2. The first 800 data points (80% of the total sample points) are used as the training sample while the remaining 200 data points (20% of the total sample points) are used as the testing sample.
19000 17000 Prices
15000 13000 11000 9000 7000 5000 1
100
200
300
400
500
600
700
800
900
1000
Data points
Fig. 2. The daily Nikkei 225 closing cash prices from December 6, 2004 to December 30, 2008
The proposed CMAC NN was coded in MATLAB. This paper employed epochmax = 200 as a termination condition, β = 0.1 and k = 10000. The optimal settings of parameters ( Ni , g ) for the proposed CMAC NN was evaluated by using Ni = {30, 40, 50} and g = {10, 20, 30, 40, 50}. Table 1 lists the model selection results of the proposed CMAC NN. When using a large value of Ni = 50, increasing g decreased the training accuracy, but enhanced the generalization ability of the CMAC NN. When using small values of Ni = 10 and Ni = 20 against various g = {10, 20, 30, 40, 50}, the poor training and generalization accuracies were found. As shown in Table 1, the CMAC NN obtained the best training and testing RMSEs by using the parameter settings ( Ni , g ) = (50, 40), which are high quantization and a large generalization size. For building SVR forecasting model, the LIBSVM package proposed by Chang and Lin [23] is used in this paper. The original dataset is first scaled into the range of [−1.0, 1.0] when using LIBSVM package. The parameter setting of regularization constant (C) and precision parameter ( ε ) which generate the minimum forecasting root mean square error (RMSE) is considered as the best parameter setting. In the modeling of the SVR model, C=1.25 and ε =0.0019 can be obtained. Since C=1.25 is near C=21 and ε =0.0019 is close to ε =2-9, the parameter set (C=21, ε =2-9) is used as the starting point of grid search for searching the best parameters. The testing results
Forecasting Financial Time Series via an Efficient CMAC Neural Network
79
of the SVR model with combinations of different parameter sets are summarized in Table 2. From Table 2, it can be found that the parameter set (C=23, ε =2-9) gives the best forecasting result (minimum testing RMSE) and is the best parameter set for SVR model in forecasting Nikkei 225 closing cash index. Table 1. Model selection results of the proposed Table 2. Model selection results of the SVR CMAC NN model-Nikkei 225 forecasting model-Nikkei 225
The prediction performance is evaluated using the following performance measures, namely, the root mean square error (RMSE), mean absolute difference (MAD), mean absolute percentage error (MAPE), directional accuracy (DA), correct up trend (CP) and correct down trend (CD). The definitions of these criteria can be found in Table 3. RMSE, MAD and MAPE are measures of the deviation between actual and predicted values. Table 3. Performance measures and their definitions
Metrics
Calculation
Metrics
Dtotal
RMSE
RMSE =
∑ ( yd , j − yo , j ) 2 j =1
Dtotal Dtotal
MAD
MAD =
∑ j =1
∑
MAPE =
yd , j − yo , j
j =1
CP
yd , j
⎧1 dj = ⎨ ⎩0 CD =
yd , j − yo , j Dtotal
⎧1 dj = ⎨ ⎩0 CP =
Dtotal Dtotal
MAPE
DA
Calculation 100 DA = Dtotal
CD
⎧1 dj = ⎨ ⎩0
Dtotal
∑ d j , where j =1
(yo , j − yo , j −1 )( yd , j − yd , j −1 ) ≥ 0 otherwise 100 Dtotal ∑ dj n1 j =1
, where
(yo , j − yo , j −1 )( yd , j − yd , j −1 ) ≥ 0 otherwise 100 Dtotal ∑ dj n2 j =1
, where
(yo , j − yo , j −1 )( yd , j − yd , j −1 ) ≥ 0 otherwise
* n1 is number of data points belong to up trend and n2 is number of data points belong to down trend.
80
C.-J. Lu and J.-Y. Wu
The Nikkei 225 closing cash index forecasting results using the SVR and proposed CMAC NN models are computed and listed in Table 4. From Table 4, it can be found that the RMSE, MAD and MAPE of the CMAC NN model are, respectively, 55.36, 35.50 and 0.35%. It can be observed that these values are smaller than those of SVR models. It indicates that there is a smaller deviation between the actual and predicated values using the proposed CMAC NN model. Moreover, compared to the SVR models, the CMAC NN model has the highest DA (directional accuracy), CP (correct up trend) and CD (correct down trend) ratios which are 81.58%, 83.33% and 79.62%, respectively. DA, CP and CD provide a good measure of the consistency in prediction of the price direction. Thus, the proposed CMAC NN model provides better forecasting results than the SVR model in terms of prediction error and prediction accuracy in the tested dataset. Table 4. The Nikkei 225 closing cash prices forecasting results using the proposed the SVR and proposed CMAC NN models Metrics
RMSE
MAD
MAPE
DA
CD
CP
SVR
61.58
43.20
0.39%
78.95%
81.67%
75.93%
CMAC NN
55.36
35.50
0.35%
81.58%
83.33%
79.62%
Models
5 Conclusion Stock index forecasting is a challenging task and has drawn serious attention during past three decades. This paper developed an efficient CMAC NN to forecast Nikkei 225 closing cash index. This paper compared the forecasting results of the proposed CMAC NN with SVR. Experimental results indicate that the CMAC NN was superior to SVR for the tested dataset. Moreover, the CMAC NN is much easier to use than traditional statistical and spectral analytical methods. Thus, the CMAC NN may be considered as financial tool for stock price prediction. Future work is to compare the performance of the CMAC NN with SVR using the dataset TAIEX closing cash indexes collected from Taiwanese stock market.
Acknowledgements The authors would like to thank the National Science Council of the Republic of China, Taiwan for financially supporting this research under Contract No. NSC 982221-E-262-014-.
References 1. De Gooijer, J.G., Hyndman, R.J.: 25 Years of Time Series Forecasting. International Journal of Forecasting 22, 443–473 (2006) 2. Lawrence, M., Goodwin, P., O’Connor, M., et al.: Judgmental Forecasting: A Review of Progress over the Last 25 Years. International Journal of Forecasting 22, 493–518 (2006)
Forecasting Financial Time Series via an Efficient CMAC Neural Network
81
3. Atsalakis, G.S., Valavanis, K.P.: Surveying Stock Market Forecasting Techniques - Part II: Soft Computing Methods. Expert Systems with Applications 36, 5932–5941 (2009) 4. Chatfield, C.: Time-Series Forecasting. Chapman & Hall/CRC, New York (2001) 5. Priestley, M.B.: Spectral Analysis and Time Series. Academic Press, New York (1981) 6. Zhang, G., Patuwo, B.E., Hu, M.Y.: Forecasting with Artificial Neural Networks: The State of Art. International Journal of Forecasting 14, 35–62 (1998) 7. Rigozo, N.R., Echer, E., Nordemann, D.J.R., et al.: Comparative Study between for Classical Spectral Analysis Methods. Applied Mathematics and Computation 168, 411– 430 (2005) 8. Albus, J.S.: A New Approach to Manipulator Control: The Cerebellar Model Articulation Controller (CMAC). ASME Journal of Dynamic Systems, Measurement, and Control 97, 220–227 (1975) 9. Albus, J.S.: Data Storage in the Cerebellar Model Articulation Controller (CMAC). ASME Journal of Dynamic Systems, Measurement, and Control 97, 228–233 (1975) 10. Wong, Y.F., Sideris, A.: Learning Convergence in the Cerebellar Model Articulation Controller. IEEE Transactions on Neural Networks 3, 115–121 (1992) 11. Lin, C.M., Chen, L.Y., Chen, C.H.: RCMAC Hybrid Control for MIMO Uncertain Nonlinear Systems Using Sliding-Mode Technology. IEEE Transactions on Neural Networks 18, 708–720 (2007) 12. Peng, Y.F.: Robust Intelligent Sliding Model Control Using Recurrent Cerebellar Model Articulation Controller for Uncertain Nonlinear Chaotic Systems. Chaos, Solitons & Fractals 39, 150–167 (2009) 13. Wang, S., Jiang, Z.: Valve Fault Detection and Diagnosis Based on CMAC Neural Networks. Energy and Buildings 36, 599–610 (2004) 14. Hung, C.P., Wang, M.H.: Diagnosis of Incipient Faults in Power Transformers Using CMAC Neural Network Approach. Electric Power Systems Research 71, 235–244 (2004) 15. Wu, J.Y., Lu, C.J.: Applying Classification Problems via a Data Mining Approach Based on a Cerebellar Model Articulation Controller. In: 1st Asian Conference on Intelligent Information and Database Systems, pp. 61–66. Dong Hoi City, Vietnam (2009) 16. Lin, C.J., Lee, J.H., Lee, C.Y.: A Novel Hybrid Learning Algorithm for Parametric Fuzzy CMAC Networks and Its Classification Applications. Expert Systems with Applications 35, 1711–1720 (2008) 17. Zhou, H., Chen, J., Wu, H., et al.: CMAC-Based Short-Term Electricity Price Forecasting. In: Sixth International Conference on Advances in Power System Control, Operation and Management, Hong Kong, pp. 348–353 (2003) 18. Qiaolin, D., Jing, T., Jianxin, L.: Application of New FCMAC Neural Network in Power System Marginal Price Forecasting. In: The 7th International Power Engineering Conference, Singapore, pp. 1–57 (2005) 19. Lee, J.: Measurement of Machine Performance Degradation Using a Neural Network Model. Computers in Industry 30, 193–209 (1996) 20. Handelman, D.A., Lane, S.H., Gelfand, J.J.: Integrating Neural Networks and KnowledgeBased Systems for Intelligent Robotic Control. IEEE Control Systems Magazine 10, 77–87 (1990) 21. Huang, C.L., Tsai, C.Y.: A Hybrid SOFM-SVR with a Filter-Based Feature Selection for Stock Market Forecasting. Expert Systems with Applications 36, 1529–1539 (2009) 22. Lu, C.J., Lee, T.S., Chiu, C.C.: Financial Time Series Forecasting Using Independent Component Analysis and Support Vector Regression. Decision Support Systems 47, 115– 125 (2009)
82
C.-J. Lu and J.-Y. Wu
23. Chang, C.C., Lin, C.J.: Libsvm: A Library for Support Vector Machines, http://www.csie.ntu.edu.tw/~cjlin/libsvm 24. Zobrist, A.L.: A New Hashing Method with Application for Game Playing. Technical report, Computer Sciences Department, University of Wisconsin (1969) 25. Lee, T.S., Chen, N.J.: Investigating the Information Content of Non-Cash-Trading Index Futures Using Neural Networks. Expert Systems with Applications 22, 225–234 (2002) 26. Lee, T.S., Chiu, C.C.: Neural Network Forecasting of an Opening Cash Price Index. International Journal of Systems Science 33, 229–237 (2002)
Forecasting Daily Cash Turnover of Bank with EWMA and SVR Wei-min Ma and Wei Lu School of Information Technology of Beijing Normal University Zhuhai Campus, Zhuhai 519085, China
Abstract. This paper present one forecasting method with Exponential Weighted Moving Average(EWMA) and Support Vector Regression (SVR). The daily cash turnover of the banks is time-serial data, banks need to forecasting daily cash turnover for banking reserve. First, the time series is preprocessed with EWMA method. The EWMAs with different coefficients are selected for forecasting features. And then SVR is used in the transformed dataset with EWMA for forecasting. The experimental result shows that the EWMA can improve the forecasting accuracy, and the SVR is more effective than other method such as 1NN and MLP. Statistical correlation of SVR between the forecasted and actual values is much higher than other method such as 1-NN and MLP. Keywords: Time series, EWMA, SV-regression, SRM.
1
Introduction
Forecasting time-series is important in several domains[1][2] and a large number of studies have used classical statistical method for forecasting series behavior. Advanced methods such as neural networks[3], genetic algorithms[4], Markov model[5] and fuzzy methods have also been frequently used[6]. The one nearest neighbor(1-NN) method is one local approximation technique. Unlike global approximators, local approximation techniques are dependent on the number of free parameters. The well-known methods of local approximation for time-series forecasting include chart representations, nearest neighbor methods[7] and pattern imitation techniques. [8] applied local approximation method in dynamic time-serial forecasting. The Multi-Layer Perceptron(MLP)[9] is a kind of neural net that is capable of generating arbitrary decision boundaries. But when the data change remarkably, the forecast precision of these method will descend obviously. This paper present one forecasting method with Exponential Weighted Moving Average(EWMA)[10]and Support Vector Regression(SVR)[11][12] [13] which can forecast the time series steadily. Firstly we use exponential weighted moving average method(EWMA) to preprocess the time series data. Exponential weighted moving average lends more weight to recent data than the simple moving average, which actually drops old data from its sample. And we also calculated the ratio of original data to
Corresponding author.
Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 83–90. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com
84
W.-m. Ma and W. Lu
different EWMA. By comparing, we find that using ratio data can improve the forecasting precision. After the preprocessing, we use SVR method for forecasting. SVMs(Support Vector Machines) are a recently developed class of classifier architectures, derived from the structural risk minimization principle by Vapnik[11]. A promising aspect of the SVM classifiers is that, SVM training involves selecting a small subset of critical data points(known as support vectors) from the original training data base. SVMs are also used to estimate real-valued functions, which is usually denote as SVR. By experiment, we find that SVR is better than 1-NN and MLP. The rest of paper is organized as following: section 2 describes the preprocessing of the original data and forecasting method with EWMA and SVR; the section 3 gives experiment result; and section 4 gives conclusion and future work.
2 2.1
Forecasting Method with EWMA and SVR Domain Description
Our problem is to model the current daily cash turnover of the time-series with past states.The data example is: Table 1. The example of data CDVH …
OULJLQDO Bank Transaction 5DQN No. Date Weekday Records Saturday Sunday Monday Wednesday ... … …. … …
… … … … … …
Daily Cash Turnover …
We denote our data as the form (x, y), where x represent some effective factors such as date, bank No., debit or credit, and so on, y represent daily cash turnover. If we choose to represent daily cash turnover of time-series as vector y = {y1 , y2 , · · · , yn } where n is the total number of points in the series. Often, we also represent such a series as a function of time, e.g. yn = yt , yn−1 = yt−1 , and so on. Then the state of the time-series represents its current daily cash turnover value yt . One simple method of forecasting may be based on identifying the closest neighbor of yt in the past data, say yj , and forecasting yt+1 on the basis of yj+1 . In this paper, we will also refer to states as patterns used in the matching process. The technique of matching structural primitives is based on the premise that the past repeats itself. The dynamic behavior of time-series can be efficiently forecasted by using local approximation. For this purpose, a map between current states and the nearest neighbor past states can be generated for forecasting. The aim of a pattern matching algorithm is to find the closest match of patter ρ in the historical data and use this for forecasting yt+1 . 2.2
Exponential Weighted Moving Average
Exponential smoothing is the most common model-free means of forecasting a future realization of a time series[10]. It requires the specification of a smoothing
Forecasting Daily Cash Turnover of Bank with EWMA and SVR
85
factor which is usually chosen from the data to minimize the average squared residual of previous one-step ahead forecasts. The most basic exponential smoother is the exponentially weighted moving average(EWMA). It has the attraction of being model-free and based on a simple algebraic formula, in terms of a smoothing factor, and therefore enjoys common usage as an ad hoc forecasting procedure. The goal is to estimate the underlying trend in a scatterplot without the use of restrictive models. The exponentially weighted moving average has the advantage of giving the most weight to the most current value. Also it only requires maintaining the value of the previous day’s average rather than the data for the entire period being averaged, so it is very compact. Following above, let y = {y1 , y2 , · · · , yt } be a time series observed of daily cash turnover at equally-spaced time points T1 , T2 , · · · , Tt . We consider the problem of using these data to forecast yt+1 . One of the simplest model-free approaches to this problem is the exponentially weighted moving average(EWMA). It forecasts yt+1 using a weighed average of past observations with geometrically declining weights: t−1 y t+1 = (1 − ω) ω i yt−i (1) i=0
where 0 < ω < 1 is commonly referred to as the smoothing constant. An advantage of using geometric weights is that equ. (1) admits the recurrence formula y t+1 = ωyt + (1 − ω)yt
(2)
with y 1 = y1 . Notice that the sum of the weights in (1)is 1 − ω t which is close to 1 when it is large enough. We use formula (2) to preprocess our bank data. The ω is called EXPONENT, which is the element in the formula that determines the period of the moving average, can set different value. The calculation of the EXPONENT is done by dividing 2 by the number of days in the period being averaged. For example, the EXPONENT for a 200-day moving average is determined by dividing 2 by 200. The result is 0.01 (2/200=0.01). Because our data capacity is not very large, so we let ω takes five values:0.1, 0.2, 0.25, 0.3, 0.4, corresponding to 20, 10, 8, 20/3, 5 days of the moving average. We notate the calculating result of (2) as y(1) for ω = 0.1, y(2) for ω = 0.2, and so on. In respecting that the value of original data or exponential weighted moving average data is very big, we consider using the ratios of them for forecasting. For example, we can create new data y1 = {y11 = y1 /y(1)1 , y21 = y2 /y(1)2 , · · · , yt1 = yt /y(1)t }. We can see that it improves the forecasting precision by latter experiment. 2.3
Support Vector Regression Algorithm
Support vector regression is based on the Structural Risk Minimization(SRM) principle from statistical learning theory [11]. The idea of SRM is to find a hypothesis for which one can guarantee the lowest probability of error. In SVR, this
86
W.-m. Ma and W. Lu
principle is translated to find a separating hyperplane with the largest distance to the closest training samples. Given the training data (xi , yi ), i = 1, · · · , n, xi ∈ Rd , yi ∈ R. SVR maps the input vectors x into a high-dimensional feature space H through some mapping function φ : Rd → H, and constructs an optimal separating hyperplane in this space. In the linear case, φ(x) is an identity function; in the nonlinear case, the mapping φ(·) is performed by a kernel function K(·, ·) which defines an inner product in H. In ε-SV regression Vapnik[11][13], our goal is to find a function f (x) that has at most ε deviation from the actually obtained targets yi for all the training data, and at the same time, is as flat as possible. The function given by an SVR is: f (x) = ω · φ(x) + b. and therefore f (x) =
l
(αi − α∗i )k(xi , x) + b.
(3)
(4)
i=1
where the threshold b is computed as follows: b = yi − ω · φ(xi ) − ε, αi ∈ (0, C) b = yi − ω · φ(xi ) + ε, α∗i ∈ (0, C).
(5)
The SVR acts well in estimating real-valued functions, regression estimation and signal processing[11] [14]. 2.4
The Forecasting Flow
We first preprocess original data with some basic operations, such as separating the data according to bank No., removing outlines, processing missing value, and so on. So the whole processing flow is shown in Figure 1:
2ULJLQDO'DWD
%DVLF3UHSURFHVV
(:0$SURFHVV
0HDVXUH5HVXOWV
11 0/37HVWLQJ 690 7HVWLQJ GDWD 7UDLQLQJGDWD 11
0/37UDQLQJ 690
Fig. 1. Forecasting processing flow chart
Forecasting Daily Cash Turnover of Bank with EWMA and SVR
3 3.1
87
Experiment Results Experimental Data and Performance Measures
The data comes from the bank, we do experiment for every branch, and forecast the daily cash turnover for the debit and credit respectively. We select randomly 4/5 of the data as training set, the rest as testing set. We use the following measures for estimating the accuracy of the forecasting.In these equations, we denote f as the forecasted value , a the actual value, and a the average of the actual value.
i (fi −f )(ai −a) (1) The correlation coefficient SSFFSAA . where SF A = , SF = n−1 2 2 f −f ( ) i (a −a) i i , SA = i n−1 . It measures the statistical correlation between the n−1 forecasted and actual values. (2) Mean absolute error.
|a1 − f1 | + |a2 − f2 | + · · · + |an − fn | n
(6)
It is the average of the difference between forecasted and actual value in all test case; it is the average forecasting error. (3) Root mean-squared error. (a1 − f1 )2 + (a2 − f2 )2 + · · · + (an − fn )2 (7) n If this number is significantly greater than the mean absolute error, it means that there are test cases in which the forecasting error is significantly greater than the average forecasting error. (4) Relative absolute error. |a1 − f1 | + |a2 − f2 | + · · · + |an − fn | |a1 − a| + |a2 − a| + · · · + |an − a|
(8)
It is the total absolute error made relative to what the error would have been if the forecasting simply had been the average of the actual values. (5) Root relative squared error. (a1 − f1 )2 + (a2 − f2 )2 + · · · + (an − fn )2 (9) (a1 − a)2 + (a2 − a)2 + · · · + (an − a)2 It is the total squared error made relative to what the error would have been if the forecasting had been the average of the absolute value. In latter four error measurements, a lower value means a more precise model, with a value of 0 depicting the statistically perfect model.
88
3.2
W.-m. Ma and W. Lu
Primary Results
First we compare the results of different preprocessing data. The results are showed on the table 2. Table 2. The measures of different data Original ratio1 ratio2 &RUUHODWLRQFRHIILFLHQW 5HODWLYHDEVROXWHHUURU 5RRWUHODWLYHVTXDUHGHUURU
ratio3
ratio4
ratio5
In table 2, row original represents using original data, row ratio1 represents using data y1 , row ratio2 of using data y2 , and so on. From the table, we can see that the statistical correlation using ratios increase remarkably than using original data, the error measures using ratios reduce much. And using data y1 can obtain best result, corresponding to take EXPONENT ω = 0.1 in formula (2). It is showed that taking 20 days for smoothing region acts best on our data. We can also show the results of 1-NN, MLP and SVR methods on table 3. Table 3. The measures of different methods 1-NN MLP SVR (SMO) &RUUHODWLRQFRHIILFLHQW 0HDQDEVROXWHHUURU 5RRWPHDQVTXDUHGHUURU 5HODWLYHDEVROXWHHUURU 5RRWUHODWLYHVTXDUHGHUURU
We select the parameter of SVR algorithm, and the table is the best result. From the table, we can see the the forecasting method with EWMA and SVR is much better than other two methods. The correlation of using SVR is much higher than using other two methods, and the error measures of using SVR is the lowest. This prove SVR has better generalization than 1-NN and MLP, for it’s based on Structural Risk Minimization, and other methods are based on Experimental Risk Minimization. The precisions of different methods on some testing data are showed in figure 2: In these figures, the serial number represent testing data set, and forecasting error is defined by error = |yf − ya |/ya (10) where yf is forecasting value and ya is actual value. The forecasting error can reflect the effect of the methods, we can see that the points represent forecasting error of SVR is below these points represent 1-NN and MLP on most data, which proves the SVR does better than the other two methods.
Forecasting Daily Cash Turnover of Bank with EWMA and SVR
89
IRUHFDVWLQJHUURU
11 0/3 695
VHULDOQXPEHU
Fig. 2. The results of three method
4
Conclusion and Future Work
In this paper we present one forecasting method with EWMA and SVR in the bank cash. The experimental results have showed that the SVR works better than 1-NN and MLP methods, and the forecasting precision with SVR is much better. The correlation of SVR is much higher than the other two methods, and the error measures of SVR is the lowest. This is mainly because SVR is based on Structural Risk Minimization, and the other two methods are based on Experimental Risk Minimization. For the data of every branch of bank is small, about 100 records. And SVR is good for small sample problem, so it has better generalization than the other two methods on our data. We also compare the forecasting effect of using original data and ratio data preprocessed by EWMA methods. The experimental results show that the EWMA methods can smooth the data, consequently improve the precision. The statistical correlation of using ratios increase remarkably, and the error measures reduce much. In the future, we will consider adjusting the parameter of the SVR so that we can obtain better forecasting precision. Also we will apply more preprocessing and learning algorithms on our data, and use some statistic measures to compare the results. Acknowledgments. This paper is supported by the teaching quality engineering foundation of Beijing Normal University Zhuhai Campus.
References 1. Last, M., Klein, Y., Kandel, A.: Knowledge Discovery in Time Series Databases. IEEE Transactions on Systems, Man, and Cybernetics 31B (2001) 2. Han, J., Dong, G., Yin, Y.: Efficient Mining of Partial Periodic Patterns in Time Series Database. In: Fifteenth International Conference on Data Engineering, pp. 106–115. IEEE Computer Society, Sydney (1999)
90
W.-m. Ma and W. Lu
3. Refenes, A.N., Burgess, A.N., Bentz, Y.: Neural Neworks in Financial Engineer: A study in Methodology. IEEE Transactions on Neural Networks 8(6), 1222–1267 (1997) 4. George, S.I.: A Search for Hidden Relationships: Data Mining with Genetic Algorithms 5. Tim, O., Firoiu, L., Cohen, P.: Clustering Time Series with Hidden Markov Models and Dynamic Time Warping. In: IJCAI-1999 Workshop on Sequence Learning (1999) 6. Hadjimichael, M., Arunas, P., Kuciauskas, L., Brody, R.: MEDEX: A Fuzzy System for Forecasting Mediterranean Gale Force Winds. In: IEEE Int. Conf. On Fuzzy Systems, pp. 529–534. IEEE press, New Orleans (1996) 7. Bottou, L., Vapnik, V.N.: Local Learning Algorithms. Neural Computation 4(6), 888–900 (1992) 8. Singh, S., McAtackney, P.: Dynamic Time-Series Forecasting Using Local Approximation. In: Proceedings of the IEEE Tenth International Conference on Tools with Artificial Intelligence, pp. 392–399. IEEE press, Los Alamitos (1998) 9. Hampshire II, J.B., Perlmutter, B.A.: Equivalence Proofs for Multilayer Perceptron Classifiers and the Bayesian Discriminant Function. In: Proceedings of the 1990 Connectionist Models Summer School, San Mateo (1990) 10. Gijbels, I., Pope, A., Wand, M.: Automatic Forecasting via Exponential Smoothing: Asymptotic Properties (1997) (manuscript) 11. Vapnik, V.: The Nature of Statistical Learning Theory. Springer, NY (1995) 12. Cortes, C., Vapnik, V.: Support-Vector Networks. Machine Learning 20(3), 273– 297 (1995) 13. Smola, A., Sch, B.: A Tutorial on Support Vector Regression. NeuroCOLT2 Technical Report NC2-TR-1998-030 (1998) 14. Sch¨ olkopf, B., Smola, A.J.: Learning with Kernels. MIT Press, Cambridge (2002)
Financial Distress Prediction Model via GreyART Network and Grey Model Ming-Feng Yeh1, Chia-Ting Chang2, and Min-Shyang Leu1 1
Department of Electrical Engineering, Lunghwa University of Science and Technology, Taoyuan, 33327 Taiwan {mfyeh,unit484}@mail.lhu.edu.tw 2 Taipei Municipal Wan Fang Senior High School, Taipei, 11696 Taiwan
[email protected]
Abstract. This study attempts to use GreyART network and grey model to construct a financial distress prediction model. The inputs used to train the network are the historical data containing 17 different financial ratios of 22 healthy and 5 distressed Taiwan’s listed banks. With the help of the developed performance index, this study also proposes a growing extraction method for financial variables not only to further improve the classification ability in the training and testing phases, but also to use fewer extracted variables to build the financial distress prediction model. Simulation results show that the optimal condition is the one using four extracted variables as inputs and the vigilance threshold of 0.80. Under this condition, the proposed method generates only two clusters with corresponding classification hit rates of 96.30% and 95.24% for the training and testing results, respectively. Keywords: Financial distress prediction model, Grey model, Grey relational analysis, Adaptive resonant theory, GreyART network.
1 Introduction Since Beaver’s pioneering work in 1966, there are a large amount of studies using financial ratios to predict financial failure. Among these researches, Altman was the first one to use discriminant analysis in predicting bankruptcy by combining financial ratios into a single predictor of corporate bankruptcy [1]. The main appeal of this approach was its ability to reduce a multidimensional problem to a single score. Although discriminant analysis was widely used in the financial distress prediction problem, it has the following potential problems: 1) the distressed and viable firms used to distinguish must be linearly separable, and 2) ratios are treated as completely independent [2-3]. An alternative methodology for the analysis of financial distress is based on neural networks. Owing to the learning capability, neural networks could create a perfect mapping between the input and output values for a set of training data. Once the training process is completed, the patterns found by the neural networks can be used to forecast situations where the outcome (e.g., a firm is in distress or health) is unknown. This also demonstrates that the use of neural networks is possible without any of circumscription that binds discriminant analysis. Research studies on using neural networks for bankruptcy prediction started in 1990 [4], and Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 91–100. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
92
M.-F. Yeh, C.-T. Chang, and M.-S. Leu
are still active now. There have been lots of published literatures in the area of financial distress prediction [5-8], which supported the use of the neural networks over the statistical methods. GreyART network [9], a neural-network architecture combining grey relational analysis [10] and Adaptive resonant theory (ART) [11] network as shown in Fig. 1, is designed to find the underlying structure of a given dataset. Owing to the architecture of ART-based network with flexibility and stability, and GreyART network without the problem of selecting an appropriate learning rate, this study adopts GreyART network to construct the proposed financial distress prediction model. Generally speaking, the testing patterns of a neural network are usually given in advance and could be used to ‘post-examine’ the generalization ability of the network. Although those patterns have never been trained in the training phase, the results cannot actually reflect the forecasting ability of the network. Grey model [10] provides a simple and fast scheme to generate the forecasting values from the current and past data. This study adopts the forecasting values obtained by grey model as the testing patterns to solve this problem. Besides, GreyART network in [9] only focuses on determining the optimal clustering results for a given data set and it might not perform well in classifying the financial health of a firm. This study, therefore, develops a new performance index to solve the classification problem by determining the best result the GreyART network can attain in the sense of that performance index. This study also proposes a growing extraction method for financial variables such that the financial distress prediction model could be constructed by some extracted variables, but has better classification ability in the training and testing phases.
2 Preliminaries 2.1 Grey Relational Analysis Grey relational analysis is a similarity measure for finite sequences with incomplete information [10]. Denote the ith N-dimensional input pattern by xi = (xi(1), xi(2), …, xi(N)) and the weight vector of node j in an ART-based network by yj = (yj(1), yj(2), …, yj(N)), j ∈ {1, 2, …, m}. If xi is viewed as a specific reference pattern and all weight vectors are regarded as the comparative patterns, then the grey relational coefficient between xi and yj at the kth component, k ∈ {1, 2, …, N}, is defined as follows:
(
)
r x i (k ), y j (k ) =
Δ i ,min + ξ ⋅ Δ i , max Δ ij (k ) + ξ ⋅ Δ i ,max
(1)
where Δ ij (k ) = xi (k ) − y j (k ) , Δ i , max = max j max k Δ ij (k ), Δ i ,min = min j min k Δ ij (k ), and
ξ ∈ (0, 1] is the distinguishing coefficient controlling the resolution between Δi,max and Δi,min. The corresponding grey relational grade is g (x i , y j ) =
∑ [wk ⋅ r (x j (k ), y j (k ) )] N
k =1
(2)
Financial Distress Prediction Model via GreyART Network and Grey Model
93
where wk ≥ 0 is the weighting factor and ∑kN=1 wk = 1 . Normally, we select it as wk = 1/N for all k. The best comparative weight vector to the input is the one with the largest relational grade. Note that no matter how large the input pattern is, we can guarantee that 0 < g(xi, yj) ≤ 1. 2.2 GreyART Network GreyART network is a kind of ART-based network that incorporates grey relational analysis into ART network [11]. If each input pattern is viewed as a reference while the templates (weight vectors) of all categories are regarded as comparative ones, then the best comparative weight vector can be easily obtained by grey relational analysis. The node corresponding to the best comparative weight vector is denoted as a winner. Additionally, the vigilance test is designed to check the similarity measurement (the grey relational grade) of the winner. The algorithm of GreyART network is presented as follows: Step 1: Present a new training pattern, say xi. By using grey relational analysis, we can select the winner, which has the largest relational grade maxj{g(xi, yj)} among all weight vectors. The winner is denoted as j*. Step 2: The node j* passes the vigilance test if and only if the grey relational grade g (x i , y j ∗ ) ≥ ρ , where ρ is the vigilance threshold.
Step 3: If the winner fails the vigilance test, a new node unit k is created with weight vector yk = xi. Step 4: If the winner passes the vigilance test, adjust the weight vector of the winner j* by ) y (jnew = ∗
(
(
)
) Num cluster j(∗old ) ⋅ y (jold + xi ∗
(
)
Num cluster j(∗old ) + 1
)
) where Num cluster j(old denotes the number of members in cluster j ∗ . ∗
Fig. 1. The configuration of GreyART network
(3)
94
M.-F. Yeh, C.-T. Chang, and M.-S. Leu
After the learning process is terminated, the final weight vectors are regarded as the estimated cluster centroids. Since 0 < g(xi, yj) ≤ 1, the vigilance threshold ρ is therefore limited to the range (0, 1]. 2.3 Grey Model Let x(0) = (x(0)(1), x(0)(2), …, x(0)(N)) be a raw sequence and x(0)(k) ≥ 0, k = 1, 2, …, N. Then the grey differential equation of GM(1,1) model, i.e., the first order and single variable grey model, is defined as [11]
x (0 ) (k ) + az (1) (k ) = b, k = 2, 3, …, N,
(4)
where z (1) (k ) = 12 [ x (1) (k ) + x (1) (k − 1)], k = 2, 3, …, N, and x (1) (k ) = ∑km=1 x ( 0) (m), k = 1, 2, …, N. The unknown parameters a (the developing coefficient) and b (the grey input) in (4) can be obtained by the least square method as ⎡a ⎤ T −1 T ⎢ ⎥ = (B B ) B z , ⎣b ⎦
(5)
T
T ⎡− z (1) (2) − z (1) (3) " − z (1) ( N )⎤ ( 0) ( 0) ( 0) where B = ⎢ ⎥ and z = [x (2) x (3) " x ( N )] . " 1 1 ⎣ 1 ⎦ Once the developing coefficient and the grey input are obtained, the forecasting values of x(0)(k) and x(1)(k), denoted by xˆ ( 0 ) (k ) and xˆ (1) (k ), respectively, are determined by
b b xˆ (1) (k ) = [ x ( 0) (1) − ]e −a ( k −1) + , k = 1, 2, … a a
(6)
⎧ xˆ (1) (k ) − xˆ (1) (k − 1), k = 2, 3, ... xˆ ( 0) (k ) = ⎨ (0 ) k = 1. ⎩ x (1),
(7)
3 Financial Distress Prediction Model This study adopts GreyART network and GM(1,1) model to construct the proposed financial distress prediction model. The inputs applied to GreyART network for classifying the financial health of a firm or bank are the historical financial data. Each resultant centroid indicates a subclass, rather than a class, and several different subclasses could make up each class. Once the training phase is completed, the final clustering results can be used to indicate the distressful conditions in currently viable banks. 3.1 Data Collection and Variable Selection
The data used in this study are collected from the Taiwan Economic Journal (TEJ) Data Bank [12] covering the period 2004-08. Two classes of listed banks are drawn: a
Financial Distress Prediction Model via GreyART Network and Grey Model
95
class identified as financially distressed banks and a class identified as financially healthy banks, namely the ‘distressed class’ and the ‘healthy class,’ respectively. Additionally, the dataset is divided into two separate sets. Data for the 4th quarter of 2004 to 2006 are used as the training set to train the GreyART network, while data for the 4th quarter of 2007 and the 2nd quarter of 2008 are used as the testing (validation) set. The training set contains 27 banks with 5 banks in the distressed class, while the testing set consists of 21 banks with 1 bank classified as in distress in each year. Seventeen financial ratios used in this study are collected from TEJ Data Bank, which reflect the following five dimensions: structure, management, solvency, growth, and profitability. Table 1 contains a detail description of all selected variables, where xi,j represent the raw data of the jth financial ratio for the ith bank. 3.2 Data Preprocess
To reduce the number of training patterns, data for the training set are generated as follows. Let xi,j(t) represent the raw data of the jth financial ratio for the ith bank at year t, and yi = (yi,1, yi,2, …, yi,N) denote the generated pattern of the ith bank. Then each element of yi can be obtained by the next two strategies: i) Healthy class: If the ith bank is in the healthy class, then its jth element is the average value of all corresponding ratios in this bank, i.e.,
(
)
yi , j = xi , j (t1 ) + xi , j (t 2 ) + " + xi , j (t k ) k ,
(8)
Table 1. Description of research variables Dimensions Structure Management Solvency
Growth
Profitability
Description Debt ratio stockholder / total assets Total assets turnover Equity Turnover Current ratio Revenue growth rate Income from operation growth rate Earning before taxes growth rate Total assets growth rate Stockholder’s equity growth rate Total assets turnover growth rate Return on asset Return on equity rate Gross profit rate Operating profit rate Earning before taxes rate EPS (earning per share)
Denotation x i,1 x i,2 x i,3 x i,4 x i,5 x i,6 x i,7 x i,8 x i,9 x i,10 x i,11 x i,12 x i,13 x i,14 x i,15 x i,16 x i,17
96
M.-F. Yeh, C.-T. Chang, and M.-S. Leu
ii) Distressed class: If the ith bank is in the distressed class, then its jth element is the value when the bank occurred the financial distress, i.e., yi , j = xi , j (t d ),
(9)
where td denotes the year that the ith bank occurred the financial distress. After generating the input patterns by (8) or (9), we uses the data preprocessing method proposed by Hsia and Wu [13] to enable each input pattern to be comparable and, at the same time, normalize each pattern. The data preprocessing formula for yi,j is stated as follows. ⎧ yi , j − Δ j ,min , if the larger, the better ⎪ ⎪ Δ j ,max − Δ j ,min ~ yi , j = ⎨ ⎪ Δ j ,max − yi , j , if the smaller, the better ⎪ Δ j ,max − Δ j ,min ⎩
(10)
where Δj,max = maxiyi,j and Δj,min = miniyi,j. After the data preprocessing, we can guarantee that 0 ≤ ~ yi , j ≤ 1. Note that, in Table 1, a financially healthy banks is normally to pursue higher ratios, except the debt ratio (xi,1). 3.3 Learning and Testing Phases
In the learning phase, the inputs applied to GreyART network are those training patterns after data preprocessing. While the learning phase of the network is completed, some templates could be obtained and regarded as the cluster centroids. Thereafter, the cluster centroids are respectively recognized by auditors, experts, or managers to label them as ‘health’ or ‘distress’. Assume that cα,β represents the cluster centroid of the βth cluster belonging to class α. Then an input pattern, say ~ y i , is in * the class α if g (c ,~ y ) = max{g (c , ~ y )}. (11) α ∗ ,β ∗
i
α ,β
α ,β
i
That is to say, an input pattern is in the class that one of its subclasses has the largest relational grade. In the testing phase, all resultant cluster centroids obtained in the learning phase remain changeless and only the first step of GreyART algorithm is required. Each testing pattern is identified as the class that the corresponding template belongs to. The testing patterns described in Sec. 3.1 have never been used to train the network in the training phase. They could be used to post-examine the ‘generalization’ ability of the network. Since those patterns are actual values, the classification results cannot actually reflect the ‘forecasting’ ability for the case of one- or two-year-ahead. As indicated in [14], constructing a GM(1,1) model requires at least four data, i.e., x (0) = (⊗, x(0)(2), x(0)(3), x(0)(4)), where ⊗ represents a arbitrarily positive number. Let x(0)(k) = xi,j(k+2002), k = 2, 3, and 4. That is to say, the raw sequence x (0) in this study is formed by three ‘historical’ data: xi,j(2004), xi,j(2005) and xi,j(2006). Then using (5) to (7) could obtain one- and two-step ahead forecasting values of x (0),
Financial Distress Prediction Model via GreyART Network and Grey Model
97
i.e., xˆi , j (2007) = xˆ (0 ) (5) and xˆi , j (2008) = xˆ ( 0 ) (6). Once the forecasting values are obtained, those forecasting data are regarded as the testing patterns of GreyART network. By this way, the corresponding classification results can actually represent the forecasting results of the proposed financial distress prediction model. Fig. 2 demonstrates the training and testing accuracies with respect to the vigilance threshold, where the number above the point on the training accuracy indicates the number of clusters resulted from that threshold and the testing accuracy includes the results of the actual and forecasting values. As can be seen, the training accuracy is 96.30% for each vigilance threshold. Also, the larger the value of the threshold, the more the created clusters. In one- or two-step ahead prediction case, generally speaking, the testing result with the actual data as inputs (the generalization ability) is better than that with the forecasting values as inputs (the forecasting ability). Besides, the best forecasting result is 57.14% in both cases. It is a worse result. This study therefore proposes a growing extraction method to extract some significant variables from the given 17 variables to improve the forecasting ability. 3.4 Growing Extraction Method In order to overcome the trade-off between the clustering performance and the classification ability, this study utilizes the following index to evaluate the performance of the clustering results. Let Accuracy(ρ) denote the classification accuracy resulted from the vigilance threshold ρ and PQ(ρ) represent the partition quality caused by the same threshold ρ, where the partition quality (PQ) is defined in Hamerly and Elkan [15] as
Fig. 2. Training and testing accuracies with 17 financial ratios
PQ =
nclass ncluster
nclass
α =1 β =1
i =1
∑ ∑ p(α , β ) 2 ∑ p(α ) 2 ,
(12)
98
M.-F. Yeh, C.-T. Chang, and M.-S. Leu
where nclass is the true number of classes, ncluster is the number of clusters found by the clustering algorithm, p(α,β) and p(α) represent the frequency based probability of a point vector in cluster β belonging to class α, and the actual class probability, respectively. Then, the performance index (PI) is defined as follows. PI ( ρ ) = 100 × Accuracy( ρ ) × PQ( ρ ).
(13)
Among those thresholds, the one with the maximal performance index is termed an optimal threshold in this study. The corresponding cluster centroids and the number of clusters are the optimal results in the sense of the performance index (13). The proposed growing extraction method starts with the training input containing only a single variable. That is to say, in the beginning, the input pattern used to train yi = (~ yi , j ), j = 1, 2, …, 17. Hereafter the jth the GreyART network is of the form ~ variable is termed the active variable. Denote the performance index due to such an input form and a specific vigilance threshold ρ as PIj(ρ). Among all possible active variables, the extracted variable is the one with the maximal performance index. In other words, if the j*th variable is the extracted variable, then it must satisfy PI j∗ ( ρ ) = max j PI ( ρ ).
(14)
Assume that the first extracted variable is the ‘return on equity rate’, i.e., j* = 13. To extract the second variable from the remaining variables, the form of input pattern y i = ( ~yi ,13 , ~ yi , j ), j ≠ 13. According to (14) again, the second extracted becomes ~ variable can be determined. Assume that it is the ‘total assets growth rate’, i.e., j* = 9. Subsequently, the 9th variable is inserted into the second component of the input, i.e., ~ yi = ( ~ yi ,13 , ~yi ,9 , ~ yi , j ), j ≠ 9, 13, for consideration. The remaining extraction process could be done by the similar method. Since the number of components is successively increased, that is the reason why the proposed scheme is termed the growing extraction method. Besides, the final input form also could reveal the significance of a variable in order in which it is extracted.
3.5 Results and Discussion The growing extraction method given in Section 3.4 is under a specific vigilance threshold. To search of an optimal result, the vigilance threshold starts from 0.70 and increases by 0.01 in the simulations. Fig. 3 shows the training and testing accuracies corresponding to the vigilance threshold ρ with the value of 0.80. The ‘variable sequence’ given in the top figure represents the extraction variables in order in which they are given. The x-axis represents the number of variables, that is, if the number of variables is n, then it represents that the first n variables in sequence are selected. Besides, the number above the point on the training accuracy indicates the number of resultant clusters resulted from that number of variables. Fig. 3 reveals that no matter how large the number of extracted variables is, the training accuracy rate is 96.30%, i.e., one financially distressed bank is misclassified into the healthy class. However, the larger the number of variables, the more the created clusters. While using the forecasting data as the testing patterns, it can be seen that the forecasting ability with
Financial Distress Prediction Model via GreyART Network and Grey Model
99
fewer extracted variables is generally better than that with more variables in both oneand two-year ahead prediction cases. In our simulations, the conditions resulting in the optimal result are ρ = 0.80 and the inputs with the form constructed by the first four extracted variables (i.e., return on equity rate, total assets growth rate, stockholder’s equity growth rate, and revenue growth rate). Under these conditions, there are only two clusters, one for the healthy class and the other for the distressed class, being generated with the training accuracy of 96.30%. Both one- and two-year ahead prediction cases can attain the same forecasting accuracy of 95.24% (one distressed bank is misclassified into the healthy class) which is much better than the best forecasting result obtained by use of 17 variables as inputs, i.e., 57.14%. That is to say, the forecasting ability is indeed improved with the proposed variable extraction method.
4 Conclusions The proposed financial distress prediction model is based on GreyART network and grey model. In order to further improve the classification ability in the training and testing phases of GreyART network, this study also proposes a growing extraction method for financial variables to extract some important variables. Simulation results show that the best result the proposed method can attain is used by four extracted variables with the vigilance threshold of 0.80. For this case, there are only two clusters, one for each class, being generated with the corresponding classification hit rates of 96.30% and 95.24% for the training and forecasting results, respectively.
Fig. 3. Training and testing accuracies with the growing extraction method
100
M.-F. Yeh, C.-T. Chang, and M.-S. Leu
Acknowledgments. This work was supported by the National Science Council, Taiwan, Republic of China, under Grant NSC 98-2221-E-262-004.
References 1. Altman, E.I.: Financial ratios, discriminant analysis and the prediction of corporate bankruptcy. J. Financ. 23(4), 589–609 (1968) 2. Deakin, E.B.: A discriminant analysis of predictors of business failure. J. Account. Res. 10(1), 167–179 (1972) 3. Blum, M.: Fail company discriminant analysis. J. Account. Res. 12(1), 1–25 (1974) 4. Odom, M.D., Sharda, R.: A neural network model for bankruptcy prediction. In: Proc. Int. Joint Conf. Neural Netw., vol. 2, pp. 163–168. IEEE Press, New York (1990) 5. Atiya, A.F.: Bankruptcy prediction for credit risk using neural networks: a survey and new results. IEEE Trans. Neural Netw. 12(4), 929–935 (2001) 6. Kumar, P.R., Ravi, V.: Bankruptcy prediction in banks and firms via statistical and intelligent techniques – a review. Eur. J. Operation Res. 180, 1–28 (2007) 7. Tsai, C.F., Wu, J.W.: Using neural network ensembles for bankruptcy prediction and credit scoring. Expert Syst. Appl. 34, 2639–2649 (2008) 8. Chen, W.S., Du, Y.K.: Using neural networks and data mining techniques for the financial distress prediction model. Expert Syst. Appl. 36, 4075–4086 (2009) 9. Yeh, M.F., Chiang, S.S.: GreyART network for data clustering. Neurocomputing 67, 313– 320 (2005) 10. Deng, J.L.: Introduction to grey system theory. J. Grey Syst. 1(1), 1–24 (1989) 11. Carpenter, G.A., Grossberg, S.: ART 2: self-organization of stable category recognition codes for analog input patterns. Appl. Optics. 26(23), 4919–4930 (1987) 12. Taiwan Economic Journal Data Bank, http://www.tej.com.tw/ 13. Hsia, K.H., Wu, J.H.: A study on the data preprocessing in grey relational analysis. J. Chinese Grey Syst. 1(1), 47–54 (1998) 14. Li, B.Q.: Three-data modeling of grey system theory. J. Grey Syst. 2(1), 11–20 (1990) 15. Hamerly, G., Elkan, C.: Learning the k in k-means. In: Proc. 17th Ann. Conf. Neural Inform. Process. Syst. MIT Press, Cambridge (2003)
Risk Assessment Model Based on Immune Theory Tao Liu, Li Shang, and Zhifeng Hu Jiangsu Research & Development Centre for Modern Enterprise Information Software Engineering, Suzhou, China
[email protected], {sl0930,huzf}@jssvc.edu.cn
Abstract. As new field of computer intelligence research, Artificial Immune System inspired by biological immune system, provides a strong paradigm for information processing and problem solving. Artificial immune network theory is an important theory of AIS and has already wildly applied in the fields of data clustering, data analysis and robot control. This research proposes firstly to apply artificial immune network theory into risk assessment of managerial field. According to the comparability of the e-commerce risk problem and the biological immune system, presents specific model construction process relating to case study and testifies the result with data from real test. Keywords: Risk, Assessment, Model, Immune network.
1 Introduction This Artificial Immune Systems (AIS) are the information processing technology, computer technology and other intelligence systems based on the theory and application of biological immune system [1]. The study of AIS has become a focus in the research of intelligence system as well as neural networks, fuzzy logic and genetic algorithms. AIS provide a strong paradigm for information processing and problem solving in computer intelligence research [2][3]. Immune theory [4][5]suggested systematic hypothesis on the activity of immune cell, generation of antibody, immune tolerance, self and non-self identification, immune memory and immune system evolution and defined immune system as adjustment network composed by immune cell or molecule. Inspired by immune theory, researchers constructed many artificial immune model [6-8] and implied them into fields like data clustering, data analysis, robust controlling, computer safety, etc [1-2]. However, artificial immune theory is not used in risk fields. This research provides specific model construction process with the study of risk assessment in risk field. Theoretical analysis and test results show that it is an effective new method of risk appraisal.
2 Design of Immune Network Risk appraisal model proposed in this research is constructed on the immune network. In our adaptive immune response network (AIRN) [8], antigen is the original data and antibody is feature of pattern which displays itself as a data item in the network; the system structure is the population of antibody. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 101–108. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
102
T. Liu, L. Shang, and Z. Hu
2.1 Initialization of Network and Antigen Presentation The antigen in the system can be described in detail as AG
= (agi ...ag I )T , and agi
represents the i-th antigen; agij represents the j-th vector of the i-th antigen. And we have
agij ∈ [0,1]m , i = 1,..., I ; j = 1,..., m . The following vector is generated
randomly:
AB = (ab1 ⋅⋅⋅ abk )T , and abij ∈ [0,1]m , i = 1,..., k ; j = 1,..., m are the
new antibodies. In the expression, abi represents the i-th antigen, and abij represents the j-th vector of the i-th antibody. Designate each element in the antigen to each element in the antibody. For any antigen agi and any antibody abj,, the distance between the two can be calculated as follows: 1/ 2
⎛ m ⎞ d (i, j ) = ⎜ ∑ (agit − ab jt )2 ⎟ ⎝ t =1 ⎠
(1)
.
For any pair of antibodies abi and abj, the distance between the two can be calculated as follows: 1/ 2
⎛ m ⎞ s (i, j ) = ⎜ ∑ (abit − ab jt ) 2 ⎟ ⎝ t =1 ⎠ For any antigen
ab j (ab j ∈ AB, j = 1,
ag i (agi ∈ AG, i = 1,
(2)
.
, k ) , the stimulation sl(i, j) it receives from the
, I ) is:
⎧⎪1 d (i, j ) sl (i, j ) = ⎨ ⎪⎩0
d (i, j ) < Tsl
.
(3)
other
In (3), Tsl is the threshold value of stimulation from antigen to antibody, and I
k
Tsl = ksl ∑∑ d (i, j ) [( I × k ) / 2] , Tsl represents the threshold of the stimulation i =1 j =1
from the antigen to the antibody, and
ksl ∈ (0,1] is a constant.
2.2 Clone and Mutation When the stimulation to antibody from antigen arrives certain degree, the immune system is activated and begins to clone antibody. The number of cloned antibody abi stimulated by antigen agi is:
⎧⎪ ⎢ sl (i, j ) ⎥⎦ d (i, j ) > Tsl . c1 (i, j ) = ⎨ ⎣ other ⎪⎩0
(4)
Risk Assessment Model Based on Immune Theory
In (4):
103
⎣⎢i ⎦⎥ represents the closest natural number to sl(i, j). Abji is the set of clones of
the antibody abj as a result of the stimulation from the antigen agi, and the set of clones of the antibody abj as a result of the stimulation from the antigens is:
C j = Ab j1 ∪ Ab j 2 ∪
∪ Ab jc1 (i , j ) =
c1 ( i , j )
∪
Ab jt .
(5)
t =1
The corresponding set of antibodies AB generates the set of new antibodies C through cloning:
C = C1 ∪ C2 ∪
k
∪ Ct = ∪ Ct .
(6)
t =1
The clone cell needs mutation process to bestow the newly born cell higher affinity with the selected antigen. Suppose Mut (i, j ) = d (i, j ) is the new antibody mutation rate from antibody abj stimulated by antigen agi. Suppose the mutation set of C ′j , then antibody abj stimulated by antigen group is
c′j = c j + mut (i, j ) × (c j − agi ) .
Suppose the mutation set of antibody abj
stimulated by antigen group is C ′j , then the antibody set from mutation is:
C ′ = C1′ ∪ C2′ ∪
k
∪ Ct′ = ∪ Ct′ .
(7)
t =1
The mutation ensures the diversity in the newly generated antibodies. This is an inherent characteristic of the learning mechanism of the immune system. The system improves its recognition rate of antigens through repeated cycling of the above process. The clone suppression starts right after clone mutation is completed. For the set C ′ , the intimacy level between the antigen and the antibody and the antibody cells are computed again after the mutation. Meanwhile, the suppression threshold between the antigen and the antibody Td, and the suppression threshold between the antibodies, Ts , are computed as follows: Td p
constant,
= kd
1 p min iI=1 d (i, j ), kd ∈ (0,1] , and kd is a ∑ p j =1
p
Ts = k s ∑∑ s (i, j ) [ p × ( p − 1) / 2] k s ∈ (0,1] and ks is a constant. i =1 j =1
During the clone suppression stage, the antibody cells with higher values of the intimacy level (as compared with the threshold Td) are deleted at first, and then the memory cells with smaller intimacy levels (as compared with the threshold Ts) are deleted according to the intimacy levels between the antibody cells. The set of
104
T. Liu, L. Shang, and Z. Hu
antibodies upon the completion of this stage is C ′′ , whose elements satisfy the following conditions:
⎧ab j ab j = ⎨ ⎩0
∃agi d (i, j ) < Td
⎧ab j ab j = ⎨ ⎩0
∃abi s (i, j ) > Ts
other
other
.
(8)
.
(9)
After the stage of clone suppression, the new set of antibody C ′′ is smaller than C ′ . Clone selection is then carried out at this stage. Among the antibodies in C ′′ , ξ % of the mutated cells are chosen to act as the memory cells for the antibody, according to their intimacy level with the antigen. ξ % can be specified beforehand. The chosen mutated cells are then put into M, the set of memory cells. 2.3 Termination of Network Training Among the antibodies after clone mutation, the ones with higher antigen stimulation level morph into the memory cells with longevity, while the ones with lower antigen stimulation level die out as the antigens die out. Since the recognition among the antibody cells themselves could activate the autoimmune response, the antibodies that are capable of self recognition have to be suppressed each time after new antibody memory cells are added in order to suppress autoimmune. Denote the number of elements in the memory cell set M (M={m1,m2,…,mz; M ∈ ℜ }) as p1. Define the intimacy maturity threshold of the memory cell as: Z
p1
p1
Ts′ = k s′ ∑∑ s (i, j ) [ p1 × ( p1 − 1) / 2] .
(10)
i =1 j =1
In (10):
k s′ ∈ (0,1] and is a constant. Therefore, after the process of the intimacy
level maturity, the elements in the set of memory cells M should satisfy the following conditions:
⎧⎪ab ab j = ⎨ j ⎪⎩0
s (i, j ) > Ts′ other
i =1,2,..., p1 ; j =1,2,..., p1 .
(11)
During the network training, a certain number of iterations could be specified as the termination condition. As an alternative, the stimulation level between the antigen and the memory cell can be measured upon the completion of each iteration to serve as the termination condition. In other words, for any given antibody agi, an antibody abj can be found in the set of memory cells so that the condition d (i, j ) ≤ Ts′ is satisfied.
Risk Assessment Model Based on Immune Theory
105
3 Definition of Risk Measure Risk assessment of managerial field is quite similar with those in the biological immune system (BIS), both needs to sustain the stability of the system in a dynamic environment. Considering the corresponding relationship between antibody consistence and pathogeny invasion degree, to establish mathematical models of elements such as self body, non-self body, antibody and antigen, establish the risk appraisal model based on antibody consistence. With this model, the type and degree of attacks to risk can be calculated. There are diversified risks and different risk sources would bring different damages. Therefore, when assessing the risks, all the factors and their corresponding importance must be considered. Moreover, risk is calculated according to the antibody information under the stimulation of all kinds of risk factors. When there is no risks, number of antibody is stable; when risk increases, number of antibody increase sharply and vice versa. Thus risk can be judged by the number of antibody stimulated by risk factors. Suppose r or R is the facing risk and 0≤r(R) ≤1. when r(R) =1, it indicates the most great risk for current system; when r(R)=0 means no risk for current system, and bigger r(R) implies higher risks. The degree of r(R) can provide information for antirisk policy-making. Define ni is the number of antibody caused by i risk found by the system, βi is the degree of danger of i attack, ci is the number of antibody under normal network. Risk calculation of the system under i risk:
ri = 1 −
1 . 1 + lg( ni − ci + 1)
(12)
Risk calculation of the whole system:
Ri = 1 −
1 1 + lg( β i ∑ ( ni − ci ) + 1)
.
(13)
i
There exist many indicators which are hard to explain exactly in the risk appraisal system. However, this research solved the problem.
4 Case Study This research used e-commerce risk assessment as an example. E-commerce risk appraisal indicator system is a serial of integrated indicators which are inter-related and can sensitively reflect the situation and problems. The design of the system follows principles of sensitivity, flexibility, integrity and with combination of quantitative and qualitative indicators. With the literature review, this research constructs e-commerce risk appraisal indicator system based on AIRN theory Risk quotient of e-commerce risk is the top level of the system (A). And then the ecommerce risk appraisal. indicator system has four first level indicators: indicator for
106
T. Liu, L. Shang, and Z. Hu
risk caused by the incompleteness of national control system (B1), indicator for the risk in the food production and processing channel (B2), indicator for the risk in the food distribution channel (B3) and indicator for the risk in the food consuming process (B4). All these four first level indicators can be assessed by its second level indicators (C) respectively. Risk appraisal indicators are in Table 1. Table 1. E-commerce risk appraisal indicator system
Final Result (A)
Risk quotient of e-commerce
First Level Indicators (B)
Second Level Indicators (C)
Risk in network(B1)
Physical security (C11) Threaten of network virus(C12) attack of hacker (C13)
Risk caused by transaction (B2)
Validity of electronic contract (C21) Flaw of electronic payment (C22) Preservation of virtual property (C23)
Risk in supply chain collaboration (B3)
Risk in law (B4)
… …
price fluctuations (C31) Uneven in profit distribution (C32)
…
confusion caused by unification policy (C41) Non-performance of market regulatory functions (C42) Financial services imperfect (C43)
…
The value of risk appraisal indicator can be decided by expert evaluation and statistic. Expert evaluation is to select a certain number of consultants and experts who would appraise every indicator according to its level and give specific reference value; statistic method would refer to all the data and draw the value of risk indicator by statistical instrument. In this research, 5 indicators such as Law and Regulation and Operation of Government Body are decided by expert method while the others are by statistic method. Relevant e-commerce risk indicator data from 2008 to 2009 in a company are divided into 45 groups. After test, these data is normally distributed. The other data are also generated and satisfy the normal distribution. Firstly, normalize data. Then calculate the risk indicator of e-commerce risk using the presented model. Threshold value of immune network is set as ksl=1,kd=1,ks=1,k’s=1. Fig.1(a) is the threat degree of risk happened in the attack of hacker; Fig.1(b) is the threat degree of risk happened in the confusion caused by unification policy; Fig.1(c) is the e-commerce risk degree appraisal when there is no threat for risk sources; Fig.1(d) is the e-commerce risk degree appraisal when there is threat for risk sources.
Risk Assessment Model Based on Immune Theory
107
Fig. 1. (a) Threat degree of risk only caused by network virus; (b) Threat degree of risk only caused by Flaw of electronic payment; (c) E-commerce risk degree appraisal when there is no threat for risk sources; (b) commerce risk degree appraisal when there is threat for risk sources
From Fig.1(a)- (d), it can be learned that risk appraisal model based on AIRN can reflect the risk of e-commerce risk and dynamically calculated its degree. Compared with neural network appraisal, this appraisal does not need the training sample of risk in advance. While compared with ambiguous appraisal, this can avoid subjective judgment in the process.
5 Conclusions Artificial immune theory has developed into a new branch of computer intelligence research. This research explores the feasibility of constructing risk assessment model based on immune network theory, designs plans for risk assessment and constructs ecommerce risk assessment model with the study of risk assessment in e-commerce risk field. The result indicated that this model is practical, reliable and capable of providing strategic decision-making tool for policy maker of e-commerce risk supervision. Further, this research abstracts and expands the conception of antibody consistence in biological immune system and applies into e-commerce risk appraisal. Lastly this research establishes e-commerce risk appraisal model based on immune network. Test results show that this method is an effective new method of ecommerce risk appraisal. The risk appraisal model in this research has deep influence to self-nonself, antigen, antibody, immune cell, clone selection, immune memory. It can not only be applied in the e-commerce risk appraisal and restructuring of mathematic model but also in pattern recognition and combinatorial optimization.
Acknowledgment This research was supported by the grants of the National Science Foundation of China (No.60970058), the grants of Natural Science Foundation of Jiangsu Province
108
T. Liu, L. Shang, and Z. Hu
of China (No.BK2009131) and is also sponsored by Qing Lan Project of Jiangsu Province.
References 1. Timmis, J., Hone, A., Stibor, T., Clark, E.: Theoretical Advances in Artificial Immune Systems. Theoretical Computer Science 1, 11–32 (2008) 2. Dasgupta, D.: Advances in Artificial Immune Systems. Computational Intelligence Magazine 4, 40–49 (2006) 3. Castro, D.E., Zuben, V.: The Clonal Selection Algorithm with Engineering Applications. In: The Genetic and Evolutionary Computation Conference, pp. 36–37. Morgan Kaufmann Publishers, San Fransisco (2002) 4. Jerne, N.K.: The immune System. Scientific American 1, 51–60 (1973) 5. Jerne, N.K.: Towards a Network Theory of the Immune System. Annual Immunology 125C, 373–389 (1974) 6. Li, Y.Y., Jiao, L.C.: Quantum-inspired Immune Clonal Algorithm. In: 4th International Conference on Artificial Immune Systems, pp. 304–317. IEEE Press, New York (2005) 7. Mo, H.W., Zuo, X.Q., Xu, L.f.: Immune Algorithm Optimization of Membership Functions for Mining Association Rules. In: Garijo, F.J., Riquelme, J.C., Toro, M. (eds.) ICNC 2006. LNCS, vol. 4222, pp. 92–99. Springer, Heidelberg (2006) 8. Tao, L.: Study on Risk Management of E-business Based on Immune Principle. Report of Post Doctorate, Huazhong University of Science and Technology (2007)
Short-Term Load Forecasting Based on Bayes and RS Yanmei Li School of Business and Administration, North China Electric Power University, Baoding 071003, China
[email protected]
Abstract. In the many influencing factors of load forecasting, some are related to each other, and some are independent, so it is not very necessary of all factors to the conclusions. The paper adaptively selects input features making use of Bayes method and Rough Set, and chose a group of input characteristic set which report most out the reason of output change to train the BP Neural Network model for forecasting, which has been testified to be valid. Keywords: Bayes, Load Forecasting, Rough Set, BP Neural Network.
1 Introduction When building up the short-term load forecasting model, we should first make sure the input of the model.The factors and datas which influence the forecasting result are very much, and not all the influencing factors are necessary.In the factors, some are interrelated, and others are irrelated. How to select the factors from the large numbers of infecting factors and datas which have strongger influences on forecasting value and constitute an effective inputting variable set function are the chief matter before establishing the forecasting model. The paper selects the inputting characteristic value of forecasting model through Bayes and RS to predigest the inputting characteristic values. After simplfying and abstracting the samples, making sure the number of neurons of the inputting values,then optimizing the neurons of hidden layer, through which the training speed and accuracy can be increased. The method has been proved to be valid through an example.
2 Bayes Decision Theory Bayes decision theory concept and method are used for fields such as engineering, management science already by broad field, adaptively selects input features step with Bayes method as follows: 2.1 Ascertains a Priori Probability Distribution The priori probability P (ω j ) represents the estimation to the probability distribution of the variable ω j , it has reflected a priori knowledge to the variable, has included Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 109–116. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
110
Y. Li
practical experience and subjective judgment etc. And that a priori probability scatters in load forecasting , is what be remained to be chosen influencing factor and their probability aggregation , restricting condition is whose necessary probability greater than zero, whose combination is 1. The load at the same time, forecasting (namely "big close small distant" distance according to load forecasting nearest segment of period affects maximal) and "identical date load type similarity" characteristic, the priori probability distribution is Pper = αP dis ⎧ ⎪ P equ = βPdiv . ⎪⎪ P per = γPequ ⎨ ⎪ P + P + P + P =1 ⎪∑ per ∑ dis ∑ equ ∑ div ⎪⎩ Pper > 0, Pdis > 0, Pequ > 0, Pdiv > 0
(1)
In the formula, Pper and Pdis represent the distance from influencing factor of forecasted point far or close; and respectively, Pequ and Pdiv represent the priori probability of influencing factor that having identical or different date type; Take α , β , γ value range being (1, ∞ ) , may look at concrete conditions but fix. When α = 1, β = 1, γ = 1 , the priori probability distribution is an uniform one. 2.2 Ascertains a Likelihood Function The likelihood function has been a condition probability essentially it has reflected sample information, whose function value has been called likelihood rates, has been.
(
)
P x1 , x 2 ," , x N ω j . During the period of load forecasting, if the input variable is selected as ω j , we may make use of N samples x n ( n = 1, " , N ) to calculate the Error E according to the follow formula. Ei =
Rvalid _ j [ f train ] =
1 n ∑ Rvalid _ j [ f train ] . n j =1
1 N valid
∑
x j ∈Z valid
[y
j
(2)
( )] × 100% .
− f train x j yj
(3)
Rvalid _ j [ f train ] be relative mean error in style, from training collection to get regression function f train relativity on effective set; N valid be an effective set Z valid all together sample book number; y j be that actual load value; f train x j be forecasting load value.
( )
Short-Term Load Forecasting Based on Bayes and RS
111
2.3 Calculates Posterior Probability
(
)
The formula of posterior probability P ω j x1 , x 2 ,", x N is
(
)
P ω j x1 , x 2 , ", x N =
(
)( ) . ω )P (ω )
P x1 , x 2 ," , x N ω j P ω j
∑ P(x , x j
1
2
i =1
,", x N
j
(4)
j
Among them, a priori probability and likelihood function action input vectors, a posteriori probability is output vector. Because a posteriori probability has synthesized a priori knowledge and sample book information, ultimateness being to decide which group of influencing factors to choose being input vector is standard. A posteriori probability is increasingly big, the probability that input vector is pitched on is increasingly big. 2.4 Chooses Input Vector ω j According to the actual characteristic being unlike area load, choose M may affect bigger factor composition to load waiting for choosing influencing factor, again out of, the random chooses different influencing factor combination, forms input vector ω j .
3 Rough Sets Reduction Algorithm 3.1 Use the Dicernibility Matrix to Get the Least Reduction The Discernibility Matrix is proposed by the Poland Warsaw famous mathematician Skowron. Utilizing this tool, we can give expression to the whole cannot distinguish relations which consist in the complicated information system. One presupposition to use the dicernibility matrix to get the least reduction is to deal with the antipathic register at first at the data in table pretreatment stage, that is, the discermibility matrix not deal with the incompatibility record. With the pretreatment methods, if the records of the conflict by dividing the number of the total number of records, we can get a measurement of the roughness and this measurement can be a character of the data sheet. Suppose S = (U , R , V , f ) is a information system, R=CYD is property collection, subset C = {ai / i = 1,2, " , m} and D = {d } is Conditional attribute set and decisionmaking attribute set, U = {x1 , x2 ," , x n } is on domain, a k (x j ) is sample, on the
j
a k . Definition system get the dicernibility matrix is
x j is the value
[ ]
M (S ) = mij
n×n
, it’s
i and
the list element is ⎧a k ∈ C , a k (xi ) ≠ a k (x j ) ∧ D(xi ) ≠ D(x j ) . mij = ⎨ i, j = 1,2,", n ⎩ φ , D( xi ) = D(x j )
(5)
112
Y. Li
So in the dicernibility matrix the element collection to the
mij is capable of distinct the whole
xi and x j ; But if xi and x j belong to the same decision-making
class, the value of the
mij is φ in the dicernibility matrix.
For every dicernibility matrix M (S ) corresponding the only distinguish function
f M ( S ) (a1 , a 2 , " , a m ) = ∧{∨ mij ,1 ≤ j < i ≤ n, mij ≠ φ }.
(6)
3.2 Reduction Algorithm Based on the Attribute Dependability Strike all about a history of NP-hard problem, so enlightening information to use simplified calculation to identify the optimal or sub-optimal reduction can be achieved is obviously a method. Many basic steps of heuristic algorithms are incepted by the nuclear of the information systems or table, then some measure of the importance of attributes, choose the most important attributes to the nuclear, until the termination conditions, and we have the information systems or a decision table reduction. To be more exact, includes a reduction of the attribute set. A information system for all the attributes of the decision-making is not as important as in the rough set theory, attribute importance of the dependencies can be reflected. Decision-making property D to the property R (R belongs to C), the dependencies γ (R, D ) defined: γ ( R, D ) =
card (POS R (D )) card (POS C (D ))
(7)
Obviously, 0 ≤ γ (R, D ) ≤ 1, γ (R, D ) give a measure of the D decision-making on the attributes of the dependencies. It reflect the importance of the property R to D. Under the premise known conditions R, a property a ∈ C − R to the decision-making D’s importance SGF (a, R, D ) may be defined as follows: SGF = (a, R, D ) = γ (R + {a}, D ) − γ (R, D )
(8)
SGF = (a, R, D ) reflect that after property a plus R, the rise degree of the dependence between R and D. And in fact, the stronger for the impact of a property attribute R and D dependencies, the greater value of SGF = (a, R, D ) is. 3.3 Reduction Algorithm Based on Conditional Information Entropy This reduction algorithm does not require distinction matrix, directly from the point of view of information theory, the idea of entropy calculated using various attributes the importance of the load. Reduction act and algebra is the same in the information entropy reduction in the importance of also need to set a threshold value greater than the threshold value of the property that is affected relatively large load factors, and
Short-Term Load Forecasting Based on Bayes and RS
113
less than the threshold value of the attribute is considered to be redundant and could be deleted. In this method for the calculation of the im nt attribute set, that is, the various factors affecting load; D represent decision-making set, that is, the load of the forecast day; RC, RD represent separately the equivalence relation in the decision-making domain of the attribute set conditions C and portance of attributes ai: I (a i , D
C represent D;
)=
⎡R ⎤ H⎢ D ⎥ ⎣ RC ⎦
⎡ ⎡ RD ⎤ ⎡ RD ⎤⎤ ⎡RD ⎤ ⎢H ⎢ ⎥ − H ⎢ ⎥⎥ / H ⎢ ⎥ ⎣ RC ⎦⎦ ⎣ RC ⎦ ⎣ ⎣ RC−A ⎦
(9)
represent RD to RC’s condition entropy.
3.4 Attribute Reduction Based on Genetic Algorithm Adaptive genetic algorithm is a method of random search, the search methods, not by a unitary structure or direction, it will more than individual as a possible solution and to consider the search space within the scope of the overall sample, thus leading to the possibility of greater convergence Global Optimal Solutions, Therefore, it was the introduction of the genetic algorithm Rough Set Attribute Reduction. Algorithm by using computer simulation of biological evolution, groups continues to be optimized, in the process of change and to identify the optimal solution. In the genetic algorithm, the design of fitness function is the core of the entire GA algorithm steps, as several genetic operator rely on the adaptation of the chromosome and therefore fitness function of design objectives, determines to a large extent iteration convergence direction, but the rough set of attribute reduction is to achieve the smallest reduction attribute set. In this way, the attribute set to meet in ensuring the accuracy of certain circumstances, at least to the number of attributes that ultimately the results that are needed to meet the classification requirements of the attribute set. Therefore, the design function to the ultimate goal should contain the following two objective classification must meet quality requirements usually must be about SR. function: covered by the reduction of the number of properties to minimize. Then provisions for the fitness function: score(r ) = m − l r + k m R for which chromosome the corresponding attribute set, D for decision-making attributes, the attribute set dependence on R. This function will attribute dependence on the introduction of fitness function, and rely on the attributes of that decision attributes of the corresponding attribute set chromosome depended reflects the attributes of taxonomic capacity.
②
①
4 Case Analysis Artificial neural network has been applied extensivly in load forecasting and decisionmaking because its strong parallel processing function, approximation to arbitrary
114
Y. Li
function, self-learning, self-organization capabilities and self-adaption capabilities, and also consider many influencing factors such as climate, temperature etc. However,ANN has some deficiency and shortage. The learning speed is fixed, so the convergence rate is low and the network need longer training time. For some complicated problem, the more inputting nodes will increase the complexity of the network., which will extend the training time. Although the weight can be converged into some value, it is not necessary the minimum one of the error plane, maybe it will produce a local minimum. Rough Set may eliminate some redundancy through reduction, which is in favour of the classification and simplification of the information, but the calculation is based on some basic concept such as approximation, plus field, minus field and boundary field etc, which will bring deviation by boundary problem. The BP neural network has strong fault-tolerance, but it has no ability of screening and compression on priori knowledge,so the RS and the BP nework are complementary.The paper reduces the inputting variables og BP network by Bayes and RS, then selects training samples and bring the to the network to train, and the trained network will be used to forest. In this paper, a southern city of the 2005 data were analyzed to August 10, 2005 to August 19 points for the entire sample for the study and active load to August 20 for the entire load for the test samples were forecast in August on the 21st load. 55 selected by experience on the condition variable attributes, which is 12 load data, that is, on the 10th to the 19th day the whole point of load; The remaining 43 non-load data, including weather, the date type, sunshine duration, maximum temperature, minimum temperature, average temperature, the biggest humidity, humidity, such as minimum 43 factors, including the date and type of rest days (including weekends and the statutory rest), the weather conditions on the provision of meteorological information is divided into 17 types. The neural network input vector, using the above-mentioned were all rough intensive SR algorithm primaries to the impact of load reduction factors. Table 1 lists a variety of reduction algorithm results. Can be seen, after attribute reduction, the input vector be simplified. Table 1. Comparision of using single attribute reduction method
Method Before reduction
The left variables 42
Hidden layer units 12
Time/min
RSME/%
54
2.7317
Bayes Discernibility matrix Conditional information entropy
30
10
13
2.1478
35
11
12
1.9956
19
14
4
1.3234
Attribute dependability
28
8
10
1.6535
Genetic algorithm
27
13
8
1.8718
Short-Term Load Forecasting Based on Bayes and RS
115
Table 1 has carried out a form from the function forecasting accuracy and training time the forecast to Table 3 kinds method in two aspects comparing, has forecast accuracy among them adopt average proportional error (η ) to be the analytical index , ( P − P ) ⋅ 100 ⎤ ⎡ ⎢∑ ⎥ whose definition as follows: P ⎣ ⎦ Among them, PAi be to forecast Ai
η =
Fi n
n
value, PFi be actual value, n is to forecast number of times. We can see from Table 1, making use of Bayes reduction, the input variables reduces from 43 to 31, calculating time reduces to 13min, the average relative error decrease from 11.11% to 5.82%, not only the attribution was reduced in the maximal degree, but the computing time is shortest relatively, and the calculation error is minimum. It is shown from the training time and accuracy,. The method in the paper is better than traditional BP Neural Network, so it is suitable for the forecasting samples in this area. First of all input and output variables on the regulation of a processing, data processing interval [0,1] between the data. Regulation is one of the many ways here by the following formula: xˆ =
x − xmin . x max − x min
In a 3-layer BP neural network model to predict the hidden layer neurons of the first set to the number of n1 = n + m + a , Among them, m is the output neuron number, n is the number of input neurons, a [1,10] between the constants. Network layer neuron transfer function using S-type tangent function, the output layer neuron transfer function using S-type logarithmic function, because the output function in the interval [0,1] between the output precisely meet the requirements. The learning algorithm uses a faster convergence Levenberg-Marquardt dynamic numerical optimization algorithm. Training error indicators set to 0.01, taking into account the network structure more complicated, the number of training set for 1000, at last we get the comparison results between the way adopted in the paper and traditional BP Neural Network. It shows from the Table 1 that comparing with traditional BP Neural Network, the method based on Bayes and RS not only improves forecasting accuracy, but also iecreases training period significantly. In all the reduction ways of the paper, the conditional information entropy method has the least time and the high precision, and also the minimum forecasting error, which shows that the method put forward in the paper has some application prospect. The simulation data above is all calculated on a same computer with main frequency 2.55GHz and memory 1GB.
5 Conclusion This paper has been submitted one kind of short-term load forecasting method based on Bayes and RS. On the premise of every factor data test result being indicated, being method's turn to be able to think that the forecast is connected with load in synthesis, reach higher forecast accuracy, and be that one kind of effective short period load forecasts method within shorter training time.
116
Y. Li
References 1. Fang, R.: Short-term load forecasting of power system with rough sets and support vector regression. Journal of Huaqiao University (Natural Science) 28, 252–255 (2007) 2. Zhu, W., Shan, M.: A neural network model of risk analysis for public construction project. Journal of Harbin Engineering Unicersity 27, 142–147 (2006) 3. Ying, J., Hua, G., Liu, Y.: Power syetem load forecasting based on v-support vector regression. Journal of Northeast Danli Unicersity (Natural Science) 27, 43–47 (2007) 4. Chen, S.: Several attributes reduction algorithm analysis based on rough set. Wuhan Institute of Industrial Journal 24, 118–121 (2005) 5. Wang, L., Li, Z., Wen, J.: Short-term load forecasting based on support vector regression. Relay 33, 17–21 (2005) 6. Pan, F., Cheng, H., Yang, J.: Power system short-term load forecasting based on SVM. Power System Technology 28, 39–42 (2004) 7. Jiang, Y.: Multi-factor forecast models based on rough sets and BP neural network. Computer Engineering 33, 154–155 (2007) 8. Zhang, Q., et al.: Short-term load forecasting based on rough sets attribute reduction algorithm and support vector machine. Grid Technology 30, 56–60 (2006)
Nonlinear System Modeling with a New Fuzzy Model and Neural Compensation Cruz Vega Israel and Wen Yu Departamento de Control Automtico, CINVESTAV-IPN A.P. 14-740, Av.IPN 2508, Mxico D.F., 07360, Mxico
[email protected]
Abstract. Normal fusion procedure of neural networks and fuzzy systems is to use neural learning techniques to train membership functions of fuzzy system. If no mechanistic prior knowledge can be used, fuzzy systems should be obtained from a set of data. The same data are usually used to train these fuzzy systems in the framework of fuzzy neural networks. But modeling accuracy cannot be improved extraordinary, because neural training and fuzzy modeling use the same data set. In this paper, we propose a new modeling idea, the fuzzy system will not be changed, and modeling error between real plant and the fuzzy system is compensated by a neural network. Another contribution of this paper is fuzzy model is generated automatically by kernel smoothing technique. The third contribution of this paper is a new learning approach for neural compensator is proposed, which assures stable and faster learning. Keywords: fuzzy modeling, neural compensation, satbility.
1
Introduction
Fuzzy system is a popular intelligent modeling method , which cannot only model linguistic variables based on fuzzy rules, but also is simple and highly intuitive , especially for complex industrial processes. But fuzzy model did not seem good under testing conditions for the sake of inadequate experimental data. The production rules cannot produce a good precision in case of lack of information. The structure (number of fuzzy rule) also influences identification capability and accuracy [1]. Resent results show that the fusion procedure of neural networks and fuzzy systems seems to be very effective for nonlinear system modeling [2]. A general combination method is to use neural training for a fuzzy model [3]. From prior knowledge, the structure of fuzzy system can be determined. After neural training, the membership functions of fuzzy systems are changed to match the training data. The process of fuzzy rule extraction for nonlinear systems modeling is called structure identification. Two popular methods can be used to construct fuzzy systems: 1) from experts [4], 2) from observed data [5]. In many industrial processes, no mechanistic prior knowledge can be used to construct fuzzy systems. It is difficult to obtain mathematical models for these process. A common approach is to partition input and output data, it is also called fuzzy Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 117–127. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com
118
C.V. Israel and W. Yu
x (k )
Plant
y (k )
Final model: yˆ (k ) + Δˆ (k ) Δ (k ) e (k )
Fuzzy model
yˆ (k )
G1 ω1 G2
Kernel smoothing
Gm
ω2
Δˆ (k )
ωm RBFNN
Fig. 1. Fuzzy model with neural compensation
grid [3], such as fuzzy C-means clustering [6], mountain clustering [6], and subtractive clustering [7]. Besides clustering approaches, fuzzy models can also be obtained by neural networks method [3], genetic algorithms [4], SVD-QR [8] and support vector machine (SVM) technique [9]. Kernel smoothing is an alternative modeling technique [10], it can find a regression function such that function is best-fit match to data at those data points. Normally, modeling accuracy of kernel smoothing is better than clustering methods, it is faster than SVM and SVD-QR. And also kernel smoothing method is very simple. The idea of kernel regression is putting a set of identical weighted function called Kernel local to each observational data point. The kernel will assign weight to each location based on distance from the data point. The kernel basis function depend only to the radius or width (or variance) from the ‘local’ data point X to a set of neighboring locations x. Kernel regression is a superset of local weighted regression and closely related to moving average and K nearest neighbor (KNN), radial basis function (RBF), neural network and support vector machine (SVM). The first successful kernel smoothing approach is NadarayaWatson estimator [16], which used kernel as a weighting function to estimate f as a locally weighted average. Backpropagation and gradient algorithm are always used for neural network training. Slow convergence and local minimum are main drawbacks. Generally, we have to make some modifications such that learning processes are stable [12]. By using passivity theory, a time-varying learning rate is stable without robust modification [2]. Although the stability of neural identification is guaranteed, the optimal properties are lost. Sometimes they cannot even reach local minima. The time-varying rate defined as η (k) = kc (c is a small positive constant) can give optimally fast convergence in the sense of the misjudgment going to zero proportional to k1 , see [14]. But it results in slow convergence to bad solutions when c and k are small. Therefore, the ”search then converge” approach is pro1 posed by [13], where η (k) = η0 2 . But these two kinds of algorithms c 1+a1 k +a2 ( kc ) cannot guarantee stability.
Nonlinear System Modeling with a New Fuzzy Model
119
In this paper, a new multiple fuzzy models is given, see Fig1. First we use a windows with length of N to collect data, a kernel smoothing support method is applied to obtain an approximator in the time interval [τ1 , τ2 ]. Since the kernel smoothing model has similar form as a fuzzy system, fuzzy rules are constructed and the corresponding fuzzy systems are made. If the same data are usually used to train these fuzzy systems. The modeling accuracy cannot be improved extraordinary, because membership functions training training and fuzzy structure generation use the same data set. In the second stage, the fuzzy system will not be changed, and the modeling error between the real data and fuzzy system will be compensated by a neural network. Then we propose a new learning rate, η0 such as η (k) = 1+a1 c +G(x . It takes the advantages of [2] and [13] and assure k ) k stable and faster learning.
2
Generating Fuzzy Systems via Kernel Smoothing
Consider a nonlinear discrete plant represented by y(k) = f [x (k) , θ] + e (k) T x (k) = [y (k − 1) · · · , y (k − ny ) , u (k − d) , · · · , u (k − d − nu )] ∈ n×1
(1)
and f (·) is an unknown nonlinear difference equation representing the plant dynamics, u (k) and y (k) are measurable scalar input and output, d is the delay, θ is an unknown parameter vector associated with an appropriate but yet to be determined model structure, e (k) is a bounded observation noise, ny and nu are the lengths of output and input, ny + nu = n. In fact, (1) is NARX model [22]. A generic fuzzy model is presented as a collection of fuzzy rules in the following form (Mamdani fuzzy model [15]) Rj : IF x1 is Aj1 and x2 is Aj2 and · · · xn is Ajn THEN y is B j
(2)
We use l(j = 1, 2, · · · , l) fuzzy IF-THEN rules to perform a mapping from the input linguistic vector x = [x1 , · · · , xn ] ∈ n to the output y. From [1] we know, by using product inference, center-average and singleton fuzzifier, the output of the fuzzy logic system can be expressed as ⎛ n ⎞ ⎛ l n ⎞ l y = ⎝ wj μAj ⎠ / ⎝ μAj ⎠ = W ϕ [x (k)] (3) j=1
i=1
i
j=1
i=1
i
where μAj is the membership functions of the fuzzy sets Aji , wpj is the point at i which μBpj = 1. W = [w1 , · · · wl ] , ϕ is l−dimension vector function, the element ϕi =
n i=1
μAj / i
n l j=1
i=1
μAj i
(4)
120
C.V. Israel and W. Yu
If we use Takagi-Sugeno fuzzy model [21][20], the fuzzy rules are Rj : IF x1 is A1j and x2 is A2j and · · · xn is Anj THEN y = pj0 +pj1 x1 +· · · pjn xn (5) where j = 1 · · · l. The output of the fuzzy logic system can be expressed as y =
l
piq0 + piq1 x1 + · · · piqn xn ϕi
(6)
i=1
where ϕi is defined as in (4). (6) can be also expressed in the form of the Mamdani-type (3), y = W ϕ [x (k)] (7)
1 with W (k) = p0 · · · p1n p111 · · · pl11 · · · pl0 · · · pln . Since the Takagi-Sugeno fuzzy model (6) has the same mathematical expression as the Mamdani fuzzy model (3), in this paper we only discuss Mamdani fuzzy system. Generally the fuzzy neural networks (3) can not match the given nonlinear system (1) exactly, the nonlinear system (1) can be represented as y(k) = W ϕ [V, x (k)] + ε (k)
(8)
where ε (k) is defined as modelling error. The identified nonlinear system (1) can also be written as
y(k) = W 0 ϕ V 0 , x (k) + f(k) (9) where fk is modeling error, V 0 and W 0 are sets of known parameters chosen by the user. In general, f(k) ≥ |ε (k)| . From (9) we know the modeling error ft depends on the structure of the fuzzy neural network. Assume we have data of observations [y (k) , x (k)] , taken at time points k ∈ [τ1 , τ2 ]. We assume that the data come from the model f (x) = φ (x) + where φ (x) is an unknown smooth mean response curve, and is error. Our goal is to find the kernel smooth estimate φˆ at some pre-specified time point k. Kernel regression of statistics was derived independently by Nadaraya and Watson [16] with a mathematical foundation given by Parzen’s earlier work on kernel density estimation. Kernel regression is the estimation of the functional relationship between two variables y and t on the interval {0 ≤ ti ≤ T } , i = 1···N yi = y (ti ) + ξ where ξ is a random noise variable with the mean equal to zero. The Nadaraya– Watson kernel regression estimate at a time t is N 2 (t − ti ) 1 i yi k (t − ti ) yi = N (10) , k (t − ti ) = √ exp − 2σ 2 σ 2π i k (t − ti )
Nonlinear System Modeling with a New Fuzzy Model
121
Here the Gaussian density of statistics is the kernel function. For nonlinear system (1) modeling, (10) is yˆ (k) = k [x (k)
N y k[x(k)−x(ti )] i N i i k[x(k)−x(ti )] 2 i ) − x (ti )] = σ√12π exp − x(k)−x(t 2 2σ
(11)
where yˆ (k) is estimated value of y(k) = f [x (k)] . Statistical analysis of the Nadaraya-Watson kernel regression is difficult because it is defined as the ratio of two random variables. In many important applications in signal processing and automatic control, which usually has a constant pdf. With these type of random variables, the summations involving theNkernel function are equivalent to a Monte Carlo integration, N i k (t − ti ) → T . The Nadaraya-Watson kernel regression becomes Priestley-Chao regression, which is defined N N yˆ (ti ) = yi k (t − ti ) (12) T i For system identification yˆ (k) =
N N yi k [x (k) − x (ti )] T i
(13)
The nonlinear plant y(ti ) = f [x (ti ) , θ] + e (ti ) , i = 1 · · · N has only N sparse data in the time interval τ1 ≤ ti ≤ τ2 . We use these data to construct fuzzy rules. For the data set [y(ti ), x (ti )] , we extract fuzzy product rules in the form of (2). x (ti ) is center of the Gaussian function μAj , y(ti ) is the center of the Gaussian i function μB j . Compared (13) with (3), kernel smoothing is equivalent to fuzzy modeling if each data corresponds to one fuzzy rule. However, this kind of fuzzy structure cannot be applied, because there are too many fuzzy rules from kernel smoothing when n is large. A possible way is design a wide kernel function to cover more data, such that these data correspond a rule. A practical method is full width at half maximum. The maximum width is defined as σmax = 2 2 ln (2)σ, σmax is chosen by the total number of the data, for N example σmax = 10 , the width parameter of Gaussian function is σ = √N . 20 2 ln(2)
∗ ∗ We use center points of each Gaussian function xj , yj , j = 1 · · · v to construct the membership functions. For the data set [y (k) , x (k)] , k ∈ [1, N ] , we extract fuzzy product rules in the form of (13) or (2). x∗j is center of the Gaussian function μAj , yj∗ is the center of the Gaussian function μB j . In the time interval i [τ1 , τ2 ] , we use the following automatic kernel smoother to construct fuzzy rules. Step 1: Initialization: select a and σmax . Moreover, calculate kernel. Step 2: Solve kernel smoothing problem according to (13), which has the same form as the fuzzy system is (3). Step 3: Membership function training: we will discuss it in next section
122
C.V. Israel and W. Yu
Step 4: Check phase: check if the network construction should be changed. If no, continuous training go to Step 3. Otherwise, terminate the training precess, go to network construction phase (Step 2 ).
3
Neural Compensation for Fuzzy Models
Since there exists modeling error Δ (k) between the plant output y (k) and the fuzzy model output yˆ (k), a Radial Basis Function (RBF) neural network Δˆ (k) is used to compensate this error, see Fig.1. Define neural compensation error as e (k) = Δˆ (k) − Δ (k)
(14)
Radial Basis Function (RBF) neural networks have recently gained considerable attention. The advantages of the RBF approach, such as the linearity in the parameters and the availability of the fast and efficient training methods, have been noted in several publications, see [11]. The neural compensator is m 2 Xi (k) − ξi ˆ Δ (k) = (15) ωi Gi (X (k)) + b Gi [X (k)] = exp 2σi2 i=1 where m is the nodes number in hidden layer, ωi is the weight, ξi and σi are T the center and the width of Gi (t), respectively, X (k) = x (k)T , Δ (k) . b is a threshold, it can be combined with the first term as w0 = b, G0 (x) = 1, so m Δˆ γ (k) = ωi Gi (x). Finally, the RBF neural compensator is i=0
Δˆ (k) = Ωk G [X (k)]
(16)
where Ωk = [ω1 · · · ωm ] , G is m−dimension vector function. We use this RBF neural networks to compensate the modeling error of the fuzzy model. According to the Stone-Weierstrass theorem, this general nonlinear smooth function (14) can be written as ˆ (k) = Ω ∗ G [X (k)] − μ (k) Δ (k) − Δ where Ω ∗ is optimal weight, μ (k) is the modeling error. Since G is bounded function and the output of the plant is assumed bounded, μ (k) is bounded as μ2 (k) ≤ μ, μ is an unknown positive constant. Neural compensation error can be represented as k G [X (k)] + μ (k) e (k) = Ωk G [X (k)] − Ω ∗ G [X (k)] + μ (k) = Ω
(17)
k = Ωk −Ω ∗ . The following theorem gives a stable fast learning algorithm where Ω of the RBF compensator. The following theorem gives a stable gradient descent algorithm for neural modeling.
Nonlinear System Modeling with a New Fuzzy Model
123
Theorem 1. If we use the RBF neural networks (16) to compensate fuzzy mechanistic model error Δ (k), the following gradient updating with time-varying rate can make the final modeling error e (k) bounded (stable in an L∞ sense) Ωk+1 = Ωk − ηk G [X (k)] e (k) η0
where ηk =
2
1 + G [X (k)] + fication error satisfies
c k
, 0 < η0 ≤ 1, c > 0. The average of the identi-
J = lim sup T →∞
where μ = max μ2 (k) .
(18)
T 1 2 2+c e (k) ≤ μ T 1 − η0
(19)
k=1
k
Proof. We select Lyapunov function as n 2 T Ω Vk = Ω = ω i2 = tr Ω k
(20)
i=1
From the updating law (18) k − ηk G [X (k)] e (k) k+1 = Ω Ω 2 2 ΔVk = Vk+1 − Vk = Ω k − ηk G [X (k)] e (k) − Ωk 2 k G [X (k)] = ηk2 e2 (k) G [X (k)] − 2ηk e (k) Ω
Using (17) and 0 < η ≤ 1, 0 ≤ ηk ≤ η0 ≤ 1, ηk = 2
1 + G [X (k)] +
c k
η0 2
1 + G [X (k)] +
≤1+1+c
c k
,
ΔVk = ηk2 e2 (k) G [X (k)]2 − 2ηk (e (k) [e (k) − μ (k)]) = ηk2 e2 (k) G [X (k)]2 − 2ηk e2 (k) + ηk 2e (k) μ (k) 2 2 2 2 ≤ ηk2 e2 (k) G + ηk e (k) + ηk μ (k) [X (k)] − 2ηk e (k)
(21) 2 = −ηk e2 (k) 1 − ηk G [X (k)] + ηk μ2 (k) η0 (1 − η0 ) e2 (k) + η0 μ2 (k) ≤ −ηk (1 − η0 ) e2 (k) + ηk μ2 (k) ≤ − 2+c 2 2 2 2 Since n min w i ≤ Vk ≤ n max w i and n × max w i are K∞ i , n × min w functions, and (2 + c) e2 (k) is an K∞ -function, η02 μ2 is a K-function. From (17) and (20) we know Vk is the function of e (k) and μ (k) , so Vk admits the smooth ISS-Lyapunov function. The dynamic of the identification error is input-to-state stable. The ”INPUT” is corresponded to the second term of the last line in (21), i.e., the modeling error μ (k) , the ”STATE” is corresponded to the first term of the last line in (21), i.e.., the identification error e (k) . Because the ”INPUT” μ (k) is bounded and the dynamic is ISS, the ”STATE” e (k) is bounded. (21) can be rewritten as ΔVk ≤ −
η0 (1 − η0 ) 2 η0 (1 − η0 ) 2 e (k) + η0 μ2 (k) ≤ − e (k) + η0 μ 2+c 2+c
(22)
124
C.V. Israel and W. Yu
Summarizing (22) from 1 up to T , and by using VT > 0 and V1 is a constant, we obtain T 0) 2 VT − V1 ≤ − η0 (1−η k=1 e (k) + T η0 μ 2+c η0 (1−η0 ) T 2 2 k=1 e (k) ≤ V1 − VT + T η0 μ ≤ V1 + T η0 μ 2+c (19) is established. Remark 1. (18) is the gradient descent algorithm, which the normalizing learning rate ηk is time-varying in order to assure the identification process is stable. This learning law is simpler to use, because we do not need to care about how to select a better learning rate to assure both fast convergence and stability. No previous information is required.
4
Simulation Comparisons
The process of engine at idle has time delays that vary inversely with engine speed and is time-varying due to aging of components and environmental changes such as engine warm-up after a cold start. The measurement of system outputs occurs asynchronously with the calculation of control signals. We assume that the occurrence of plant disturbances, such as engagement of air conditioner compressor, shift from neutral to drive in automatic transmissions, application and release of electric loads, and power steering lock-up, are not directly measured. The dynamic engine model is a two-input and two-output system [17]: · · P˙ = kP mai − mao , N˙ p = kN (Ti − TL ) · (23) mai = 1 + km1 θ + km2 θ2 g (P ) ·
mao = −km3 Np − km4 P + km5 Np P + km6 Np P 2 The system outputs are manifold press P (kPa) and engine speed Np (rpm). The control inputs are throttle angle θ (degree) and the spark advance δ (degree). Disturbances act to the engine in the form of unmeasured accessory torque Td · · (N-m). The variable mai and mao refer to the mass air flow into and out of the manifold. mao is the air mass in the cylinder. The parameter τ is a dynamic transport time delay. The function g (P ) is a manifold pressure influence function. Ti is the engine’s internally developed torque, TL is the load torque. The parameters for an engine model with 1.6 liter, 4-cylinder fuel in N 2 P < 50.6625 √ 1 jected are: g (P ) = + , TL = 263.17 0.0197 101.325P − P 2 P ≥ 50.6625 ·
Td , mao = mao / (120N ) , Ti = −39.22 + 325024mao − 0.0112δ 2 + 0.635δ + 2π 2 2π 0.000102N 2, kP = 42.40, kN = 54.26, km1 = 60 (0.0216 + 0.000675δ) Np − 60 0.907, km2 = 0.0998, km3 = 0.0005968, km4 = 0.0005341, km5 = 0.000001757, τ = 45/Np . Difference technique is used to get the discrete-time states of the · system (23). (23) can be written as x = f (x, u) with x = [P, Np ]T , u = T [θ, δ, Td ] .The discrete-time model for P is P (k + 1) = f1 [P (k) , Np (k) , θ (k) , δ (k) , Td (k)]
Nonlinear System Modeling with a New Fuzzy Model
125
100
50
0
0
200
400
600
800
1000
0
200
400
600
800
1000
100
50
0
Fig. 2. The training and testing results
In order to train the neural model, we select inputs as δ = 30 sin (0.06k) , θ is a square wave with amplitude 20, θ (k) = 20Square(0.04k), Td is a constant, T Td (k) = 10, x0 = [10, 500] . 6 fuzzy rules from kernel smoothing are obtained.The time interval the the hidden layer selection N is chosen as N = 300. We start from Ωk ∈ 1×15 . After then (18) is applied to update the weight Ωk . The final training result is shown in Fig.2. The testing signals are changed as δ = 10, θ (k) = 40 cos(0.05k), Td (k) = 1. The testing result is shown in Fig.2. Now we compare the performances of our fuzzy model with neural compensation (FNC) with neural networks identification (NN) [11], ANFIS fuzzy model (FS) [1], and fuzzy neural model [2] (FNN). The multilayer neural networks as in [11] is Π5,15,1 (one hidden layer with 15 nodes), and its learning rate of backpropagation is fixed as η = 0.05. For FS and FNN, we use 15 rules. The results are shown in Table 1, where the modeling error is calculated as RMS.
Table 1. Comparisons of four identification methods networks case modeling error hidden nodes rule number
NN FS FNN FNC training testing training testing training testing training testing 0.24 0.52 0.21 0.49 0.06 0.43 0.21 0.41 15
15
15
15
15
15
6
6
126
5
C.V. Israel and W. Yu
Conclusion
This paper proposed a new fuzzy model for nonlinear system modeling. Although the fuzzy system plus neural networks based method is widely used for modeling nonlinear systems, there are some new contributions in this paper: the fuzzy model is obtained from kernel smoothing, not from prior knowledge; a new fast training algorithm for the neural compensator is proven to be stable.
References 1. Wang, L.X.: Adaptive Fuzzy Systems and Control. Prentice-Hall, Englewood Cliffs (1994) 2. Yu, W., Li, X.: Fuzzy identification using fuzzy neural networks with stable learning algorithms. IEEE Transactions on Fuzzy Systems 12, 411–420 (2004) 3. Jang, J.S.: ANFIS: adaptive-network-based fuzzy inference system. IEEE Transactions on Systems, Man and Cybernetics 23, 665–685 (1993) 4. Rivals, I., Personnaz, L.: Neural-network construction and selection in nonlinear modeling. IEEE Transactions on Neural Networks 14, 804–820 (2003) 5. Leski, J.M.: TSK-fuzzy modeling based on ε-insensitive learning. IEEE Transactions on Fuzzy Systems 13, 181–193 (2005) 6. Mitra, S., Hayashi, Y.: Neuro–fuzzy rule generation: survey in soft computing framework. IEEE Transactions on Neural Networks 11, 748–769 (2000) 7. Chiu, S.L.: Fuzzy model identification based on cluster estimation. Journal of Intelligent and Fuzzy Systems 2, 159–163 (1994) 8. Chiang, J.-H., Hao, P.-Y.: Support vector learning mechanism for fuzzy rule-based modeling: a new approach. IEEE Transactions on Fuzzy Systems 12, 14–25 (2004) 9. Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines. Cambridge Univ. Press, Cambridge (2000) 10. Wand, M., Jones, M.: Kernel Smoothing. Chapman & Hall, London (1995) 11. Narendra, K.S., Cheng, X.: Adaptive control of discrete-time systems using multiple models. IEEE Transactions on Automatic Control 45, 1669–1686 (2000) 12. Suykens, J.A.K., Vandewalle, J., De Moor, B.: NLq theory: checking and imposing stability of recurrent neural networks for nonlinear modelling. IEEE Transactions on Signal Processing 45, 2682–2691 (1997) 13. Darken, C., Chang, J., Moody, J.: Learning Rate Schedules for Faster Stochastic Gradient Search. In: Proceedings of the IEEE-SP Workshop, pp. 3–12 (1992) 14. Ljung, L., Soderstrom, T.: Theory and Proactice of Recursive System Identification. The MIT Press, Cambridge (1983) 15. Mamdani, E.H.: Application of fuzzy algorithms for control of simple dynamic plant. In: IEEE Proccedings-Control Theory and Applications, vol. 121, pp. 1585– 1588 (1976) 16. Nadaraya, E.A.: On estimating regression. Theory of Probability and its Applications 9, 141–142 (1964) 17. Puskorius, G.V., Feldkamp, L.A.: Neurocontrol of nonlinear dynamics systems with Kalman filter trained recurrent networks. IEEE Transactions on Neural Networks 5, 279–297 (1994) 18. Priestley, M.B., Chao, M.T.: Non-parametric function fitting. J. Royal Statistical Soc. Ser. B 34, 385–392 (1972)
Nonlinear System Modeling with a New Fuzzy Model
127
19. Prokhorov, D., Wunsch, D.: Adaptive critic designs. IEEE Transactions. on Neural Networks 8, 997–1007 (1997) 20. Takagi, T., Sugeno, M.: Fuzzy identification of systems and its applications to modeling and control. IEEE Trans. Syst., Man, Cybern. 15, 116–132 (1985) 21. Baruch, I.S., Lopez, R.B., Olivares, J.-L., Flores, J.M.: A fuzzy-neural multi-model for nonlinear systems identification and control. Fuzzy Sets and System 159, 2650– 2667 (2008) 22. Chen, S., Billings, S., Grant, P.: Recursive hybrid algorithm for nonlinear system identification using radial basis function networks. Int. J. Control 55, 1051–1070 (1992)
A Research of Fuzzy Neural Network in Ferromagnetic Target Recognition Caipeng Wu*, Jiahao Deng**, and Yanli Yang*** National Key Laboratory of Mechatronic Engineering and Control, Beijing Institute of Technology, Beijing 100081, China
[email protected],
[email protected],
[email protected]
Abstract. Based on deeply analyzing the characteristic of the battlefield ferromagnetic targets, according to the problems of magnetic detection system, for instance single detection pattern, low detection resolution and poor antiinterference performance, the Giant Magneto-Impedance(GMI) micro-magnetic sensor in combination with the technology of fuzzy neural networks(FNN) were carried as the core of the magnetic detection system. Take advantage of GMI sensor and FNN to realize accurate recognition of the target in the range of nanotesla magnetic field. In this paper, equable magnetization rotation ellipsoid is used to simulate the tank and military truck, taking the triaxial magnetic moments and semi-focal length, that is M x , M y , M z , c as recognition characteristic quantity , and the FNN is used to recognize the tank and military truck including the categories and motion directions. The method reaches good recognition effect through experimental verification, and it has significance to improve detection range and recognition accuracy. Keywords: Ferromagnetic target recognition, Adaptive neuro-fuzzy inference system, Giant Magneto-Impedance, Micro-magnetic sensors.
1 Introduction The tank, armored vehicle, and military truck are the important ferromagnetic target on the battlefield, and the magnetic detection system that is based on geomagnetic field disturbance generated by the ferromagnetic substance makes use of various magnetic measuring methods to locate and recognize targets. However, the majority of the magnetic fuse detection systems are confronted with the problem of neardistance detecting and poor target identification ability at present, so the early burst and error burst situations happen from time to time. *
**
***
Caipeng Wu is a Ph.D. candidate in Beijing Institute of Technology. His research area is target detection, guidance and control technology. Jiahao Deng is currently a professor in Beijing Institute of Technology. His main research interests include target detection and signal processing, sensing and control technology. Yanli Yang is a Ph.D. candidate and his research interests are target detection and signal processing.
Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 129–136. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
130
C. Wu, J. Deng, and Y. Yang
There are two problems mainly in realizing distant reliable detection according to ferromagnetic moving target on the battlefield. First, the most of the present magnetic fuse systems based on metallic eddy effect, electromagnetic induction or hall effect, exist bad effect in high temperature, poor detection accuracy and so on when conduct target detection; Second, when detection range increase, the high sensitive magnetic sensor is necessary, and the target signal is very weak, so we need reduce detection threshold to measure target signal, which is easily reached by natural or artificial disturbance, then the system has high false alarm probability. In order to solve the above two problems, this paper will design a novel magnetic detection system adopting micro-magnetic sensor based on Giant Magneto-Impedance (GMI) effect and fuzzy neural networks (FNN), then make use of it to detect and recognize ferromagnetic target on the battlefield, especially for tank.
2 Magnetic Detection System Establish The detection system mainly consists of a three-axis GMI magnetic sensor, signal conditioning circuit and the target identification module taking FPGA (Field Programmable Gate Array) as core, as shown in Fig.1.
Fig. 1. Magnetic Detection and Target Recognition System Block Diagram
Taking the three-axis GMI sensor as the core of the sensitive element, detect x (East is positive), y (North is positive), z (Upward is positive). The so-called GMI effect, is referred to a significant change in the ac impedance of a soft magnetic conductor with an ac current when subjected to an applied dc magnetic field, which is announced by Panina and Mohri for the first time in 1994. The GMI sensor initial prototype has been successfully developed abroad, the performance of which achieves the following indexes: detection accuracy reaches 1 μ Oe, reaction speed can reach 1MHz, and the average power consumption is approximately 10mW. Compared with a Giant Magneto-Resistance (GMR) that has a sensitive of ~1%/Oe, the field sensitivity of a typical GMI sensor can reach a value as high as 20~100%/Oe [1-3]. The system control and signal processing module is composed of FPGA and CPLD (Complex Programmable Logic Device). With improvement of FPGA capacity, performance, cost efficiency, and parallel processing ability, it can become an significant hardware realization method for FNN, compared with software implementation which is beneficial to improve real-time performance of the fuse system. The CPLD with design flexibility and high hardware integration is applied to assisted control logic, so it will complete chip selection signal, buffering function of A/D and FPGA, some level conversions function, and other control logic.
A Research of Fuzzy Neural Network in Ferromagnetic Target Recognition
131
3 Fuzzy Neural Network Fuzzy Neural Network, FNN for short, is referred to combine fuzzy logic with neural network appropriately, and the combination starts from 1988, absorbs the merits of the both [4]. It can fully develop the advantages of parallel processing and adaptive learning in neural network, and it has reasoning and decision function using fuzzy logic to human knowledge. If we take the FNN system as a recognition method to fuzzy and complex magnetic characteristic signals from ferromagnetic target on battlefield, we can obtain a magnetic detection system with high recognition rate. TS fuzzy neural network proposed by Takagi and Sugeno is a type of neural network that is the most widely used and deeply researched at present. The FNN based of TS type can realize Adaptive Neuro-Fuzzy Inference System(ANFIS), which is a method using data modeling, that is to say the fuzzy membership functions and fuzzy rules come from the known data’s learning, but not from experiences or intuition. For the ANFIS of the n inputs and single output, x = [ x1 x2 … xn ] T is the inputs defined by fuzzy linguistic variables, and the fuzzy rules proposed by Takagi and Sugeno can be expressed as follows [5, 6]:
: x is A
IF
1
j 1
and … and xn is Anj ,THEN: y j = q j 0 + q j1 x1 + " + q jn xn
Where Ai j (i = 1, 2," , n; j = 1, 2," , mi ) is fuzzy subset of the xi , μ
Ai
j
(1)
( xi ) is its
membership function. Every prescriptive relevance weight is shown as that:
μj = μ
j A1
( x1 ) μ
j A2
( x2 ) " μ
j An
( xn )
(2)
Then the network output can be written as follows: m
y = ∑μj yj
(3)
i =1
4 Magnetic Field Analysis of the Ferromagnetic Target In previous magnetic detection systems, the waveform characteristics of target magnetic signals, such as amplitude, waveform width and slope, were used in target detection [7]. When the projectile-target distance is relatively near, because of the higher signal-noise ratio, we can use frequency filtering and other signal preprocessing method to realize target detection effectively, however, for the situation of farther projectile-target distance and low signal-to-noise, if we only use some time-frequency characteristics of the target signal to wave filter, it is difficult to ensure high recognition accuracy of the detection system. In contrast, using modeling method to research ferromagnetic target fully utilizes the time-frequency characteristics of magnetic signal, the spatial distribution characteristic of magnetic vector and the motion features of target, so it can be called a holographic technique of the target field. If we can build the stable reliable model of ferromagnetic target, utilize the model parameter that is the most refined essential characteristic of target magnetic
132
C. Wu, J. Deng, and Y. Yang
field as the characteristic quantity of target recognition, and combine GMI micromagnetic senor as the core of detection system, the target detection distance will be improved significantly at the situation of high correct recognition rate. 4.1 Target Magnetic Field Model Building
、
The most typical ferromagnetic targets of land battlefield are tank armored vehicle and military truck, all of which are magnetic objects having a certain geometry. Taking tank as an example, the common model building methods include uniform magnetized rotary ellipsoid with array of magnetic dipole modeling method, rectangular parallelepiped with hollow rotary ellipsoid model and a single rotary ellipsoid method. The advantage of former two mixed modeling methods is that they can improve fitting precision on the near field of tank, but the deficiency is the model parameters are too complicated and not stable enough. While for a single rotary ellipsoid method, although the fitting precision on the near field of tank is poor, with distance growing, it becomes higher and higher, and the model parameters are more stable. So when the projectile-target distance is relatively far, we can use a single rotary ellipsoid to simulate the magnetic field of tank that is shown in Fig.2, and take the model parameters as the characteristic quantity of target recognition.
Fig. 2. Modeling schematic diagram for tank magnetic field
In the model of Fig.2, the parameters are defined as follows: the coordinate origin is located in the center of tank, the semi-major axis of the ellipsoid is a = L / 2 ( L is equal to the longitudinal length of the tank), the short half axis is expressed as b = W ⋅ H / 2 ( W , H respectively represent the transverse length, the height of the tank), c is the half focal length and equal to a 2 − b 2 [8].So on the ellipsoid coordinate system, the three-axis components of magnetic field intensity on point P( x, y, z ) can be expressed as: H x = M x a x + M y a y + M z a z , H y = M xbx + M y by + M z bz , H z = M x cx + M y c y + M z cz
(4)
In (4), [ M x , M y , M z ] is the magnetic moment of ellipsoid model, and the coefficient matrices are [ax , a y , az ]T [bx , by , bz ]T and [cx , c y , cz ]T that can be calculated through model geometric parameter.
A Research of Fuzzy Neural Network in Ferromagnetic Target Recognition
133
4.2 Target Characteristic Quantity Extraction
The values of magnetic moment that are obtained in different point are not the same, due to the complex shaped parts of tank and the inhomogeneous distribution of ferromagnetic substance, but we can adopt least square method to get the estimated value of model parameters. In the practical projectile target crossing process, the stepwise regression method is used to estimate the magnetic moment parameters [ M x , M y , M z ] of ferromagnetic target real-timely, and the practical projectile target
—
crossing velocity can be calculated through the method. By comparing the real-time model parameters calculated by the above method and that of knowledge base, the result of accurate target recognizing can be attained.
5 FNN Recognition System Design The ANFIS is a modeling method based on mass data, in which the membership functions and fuzzy rules come from learning the knowing data, so it is known as a recognition system based on data rules. The so-called “data rules” means unify organically the input data (target feature) and the rules that are the “IF-THEN” in the ANFIS to form the rule base, which can perform target recognition. 5.1 Fuzzy Rule Base Construction
According to the analysis above, the “ M x , M y , M z , c → y ” is the expected input-output data couple, where M x , M y , M z , c are the magnetic target model parameters according to rotary ellipsoid modeling, y is the result of target recognition. In fuzzy system, the input and output spaces should be divided into some fuzzy spaces, where there are corresponding membership functions. The input and output parameters need to be localized in the spaces having the biggest membership functions, which based on the corresponding membership of M x , M y , M z , c at different intervals, then a rule is generated from the every satisfying input-output data couple. The confidence principle can be used to solve those conflicting rules, that is selection the rules having higher confidence. Finally, the mapping relation can be determined according to combination of fuzzy rules [9-11]. 5.2 Application Example of Ferromagnetic Target Recognition
The received signals of magnetic detection system on land battlefield are mainly from tank target, common military truck, industrial interference or circuit noise, and natural interference signal, especially for the geomagnetic storm and pulsation. For the latter two interference signals, although the frequency range of signals is so wide that it can cover that of the target magnetic field completely, they can’t pass the examination of rotary ellipsoid model and the fitting accuracy is poor, because their magnetic field distribution can’t be simulated by a uniform magnetic rotary ellipsoid; whereas the
134
C. Wu, J. Deng, and Y. Yang
military truck is very similar to tank in geometry shape, material composition and weight, so the two types of targets is the distinguished emphases. Take a three bits binary array T (T1, T2 , T3 ) as the output of FNN, and there are eight possible outputs, which respectively direct at the four driving directions of tank and military truck. For improving the reliability of target recognition system, we have measured the actual magnetic field values of a certain main battle tank and a “Liberation” truck, and according to different measurement points, in the space of 20m × 15m × 10m including target, we selected 400 groups sample data that is 100 groups data selected in one direction, in which 200 groups data constitute training group, and the other 200 groups data is the testing group of the FNN. All of the data need to be preprocessed to obtain the model parameters M x , M y , M z , c at the four directions, and normalize the
—
model parameters as the inputs of FNN. Some data having been processed are shown in Table 1. Table 1. Partial training samples and the expected outputs
Model parameters(input) Mx
My
Mz
c
0.452 0.437 0.484 0.474 0.075 0.041 0.168 0.162 0.989 0.952
0.851 0.857 0.999 0.999 0.522 0.521 0.379 0.378 0.322 0.018
0.075 0.152 0.797 0.869 0.041 0.172 0.667 0.699 0.240 0.766
0.061 0.079 0.873 0.920 0.011 0.112 0.775 0.796 0.157 0.793
…
T (T1, T2 , T3 )
000 000 100 100 001 001 101 101 011 111 …
5.3 Realization Process of FNN
According to the data of Table 1, we can construct the ANFIS to complete the recognition of tank and truck targets in species and directions, which includes input layer, fuzzy layer, reference layer and conclusion layer. The inputs of FNN is M x , M y , M z and c , and the output is the recognition result. In MATLAB fuzzy logic toolbox, the function of genfis1() is used to train the ANFIS and generate the initial value of TS type ANFIS, and utilizing the function of anfis() to train the initialized network structure. The optimum parameters of membership functions can be calculated through data learning effectively, and make the designed Takagi-Sugeno type FNN simulate the expected or actual relationship of input-output. Fig.3 is the graph of membership functions about M x , M y , M z , c before and after training.
1
Before Training 1
0.5
0.5
0
c
Mx,My and Mz
A Research of Fuzzy Neural Network in Ferromagnetic Target Recognition
0
0 1 0 After Training 1
0.5
0
Mz
My
0.5 0
0.5
1
1
0.5
0.5
0
0
0.5
0
1
0.5
1
0
0.5
1
0
0.5
1
0.5 0
1
c
Mx
1
135
Fig. 3. Membership Function about M x , M y , M z and c Before and After Training
The error change curves of training and testing data is shown in Fig.4. -3
6
x 10
A B
5
training error testing error
B 4 3 2 1 0 0
A 10
20
30
40
50
60
70
80
90
100
Fig. 4. Training and Testing Error Change Curves
From Fig.4, we can see both of the training and testing errors is smaller than 0.002 after function training of 100 times, and the fitting effect is good. The testing error is bigger than the training error, but the correct recognition rate is up to 95%.
6 Conclusion In this paper, according to the problems of magnetic detection system, such as the near detection distance and poor target recognition ability, we proposed the solution that used the new generation magnetic sensor—GMI sensor having better performance as
136
C. Wu, J. Deng, and Y. Yang
the detecting element, and took fuzzy neural network as the recognition tool of ferromagnetic target. It is important to make above research for improving the distance and recognition accuracy of magnetic detection system. Acknowledgments. The work was supported by the National Natural Science Foundation of China under the grant No.60874100, and the Innovation Foundation of Aerospace Science and Technology (CAST200834).
References 1. Panina, L., Mohri, K.: Magneto-impedance effect in amorphous wires. Appl. Phys. Lett. 65, 1189–1191 (1994) 2. Mohri, K., Uchiyama, T., Panina, L.: Recent advances of micro-magnetic sensors and sensing application. Sensors & Actuators A: Physical 59, 1–8 (1997) 3. Kentaro, T., Yichi, H., Masayoshi, E.: Three-axis magneto-impedance effect sensor system for detecting position and orientation of catheter tip. Sensors & Actuators A: Physical 111, 304–309 (2004) 4. Sugeno, M., Yasukawa, T.: A fuzzy logical based approach to qualitative modeling. IEEE Trans. on Fuzzy Systems 1(1), 7–31 (1993) 5. Hairong, S., Pu, H., Lihui, Z.: A New Method to construct Fuzzy Systems Based on Rule Selecting. In: ICMLC 2004, vol. 3, pp. 1855–1858 (2004) 6. Takagi, T., Sugeno, M.: A robust stabilization problem of fuzzy control system and its application to backing up control of truck-trailer. IEEE Trans. on Fuzzy System. 2(2), 119–134 (1994) 7. Chunsheng, L., Qian, X., Shenguang, G.: A modeling algorithm for detection of moving on-water magnetic objects. Journal of China Ordnance 2(26), 192–195 (2005) 8. Pingxian, Y., Shenguang, G.: The physical field of warship, pp. 25–60. Weapon Industry Press (1992) 9. Zhenyuan, J., Ying, Z., Yingdong, S., Wenyan, T.: The application of NN in recognition of vehicles. Journal of Natural Science of Heilongjiang University 1(26), 39–42 (2009) 10. Tao, Z., Tingjin, L., Xuehai, Z.: The recognition method of small underground objective based on fuzzy neural network. Journal of Projectiles, Rockets, Missiles and Guidance 4(27), 316–319 (2007) 11. Hitoshi, L., Masafumi, H.: Adaptive fuzzy inference neural network. Pattern Recognition 10(37), 2049–2057 (2004)
Multiple T-S Fuzzy Neural Networks Soft Sensing Modeling of Flotation Process Based on Fuzzy C-Means Clustering Algorithm Jiesheng Wang, Yong Zhang, and Shifeng Sun School of Electronic and Information Engineering, Liaoning University of Science & Technology, Anshan 114044, China
[email protected],
[email protected],
[email protected]
Abstract. Inspired by the idea of combining multiple models to improve prediction accuracy and robustness, a soft sensing modeling of flotation process based on multiple T-S fuzzy neural networks and fuzzy c-means clustering algorithm (FCM) is proposed. Firstly, the model adopts principal component analysis (PCA) to reduce dimensions of the input variables data composed of texture characteristics of floatation froth image and process variables. FCM algorithm is used for separating a whole training data set into several clusters with different centers and each subset is trained by T-S FNN. The degrees of membership are used for combining several models to obtain the finial soft sensing result. Simulation results show that the proposed modeling is effective in the prediction of indexes and meets the requirement for optimization computation for the flotation process. Keywords: Flotation Process, Texture Characteristic, Multiple T-S Fuzzy Neural Networks, FCM, Soft Sensing, PCA.
1 Introduction In the flotation process, technical indicators such as concentrate grade and flotation recovery of economic are the key to production process control targets. However, it can only be adjusted and be based on an off-line analysis. Because large-time delays, it is difficult to achieve the quality of direct closed-loop control [1]. This estimating problem of process quality indicators of off-line measurement is solved effectively by using inferred estimates (soft-sensing) technology [2]. At present, the single model structure which uses the sample data is most used in the non-linear soft sensor modeling. If we do not restrict the size of the model and the training data is sufficient, a satisfactory model structure and the approximation accuracy are got which are based on neural networks or fuzzy systems modeling. However, as the expansion of the training area, the systemic states become more and more complexity. It will result in low efficiency model optimization and the reducing of accuracy of the model prediction and the significantly ability of robustness and generalization. By considering the divide and conquer, and task decomposition, a number of model structure are proposed by many scholars in order to improve Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 137–144. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
138
J. Wang, Y. Zhang, and S. Sun
prediction accuracy and robustness, such as Multiple Neural Networks (MNN), Stacked NN (SNN) and Distributed NN (DNN), and so on [3-4]. A method which based on principal component analysis (PCA) and fuzzy c-means clustering algorithm (FCM) method of the multi-T-S model structure (the PFM T-S algorithm) is presented in this paper. Simulation studies have shown that the method can significantly improve the model accuracy and robustness.
2 FCM-Based Multiple T-S Fuzzy Neural Network Model Considering a multi-input single-output (MISO) system, the training sample set can be expressed as D = {Y , X | i = 1, 2, " , m} . Y = { y | j = 1, 2, " , n} is the output matrix ( n is the number of the training datum and m is the number of input variables). X = [ x , x , " , x ]′ (i = 1, 2, " , m ) is the input vector. If D is divided into c clusters {D , D , " , D } , it can establish c of sub-model {M , M , " , M } by using a subset {D | i = 1, 2, " , c} . Fig. 1 is the structure diagram of PFM T-S FNN model [4]. j
i
i
1i
1
2
2i
ni
c
1
2
c
i
Fig. 1. PFM T-S Fuzzy neural network model structure
2.1 FCM Clustering Algorithm FCM clustering algorithm [5] divides the feature points of the feature space X = ( x , x , " , x ) into c categories ( 2 ≤ c ≤ n ) , the i class cluster center is 1
2
expressed as
n
vi , any feature point
x
j
belong to the i
category of
membership u ( 0 ≤ u ≤ 1) , and uij Satisfy the following conditions: ij
ij
c
∑u
ij
=1
for
j = 1, 2, " , n
(1)
i =1 n
0<
∑u j =1
ij
for
i = 1, 2, " , c
(2)
Multiple T-S Fuzzy Neural Networks Soft Sensing Modeling of Flotation Process
139
The objective functions of FCM algorithm as follows: c
n
∑∑ (u )
J m (U , V ) =
m
ij
i =1
x j − vi
2
(3)
j =1
Where m > 1 is the index weight to influence the degree of fuzzy membership degree matrix. Clustering problem is to make (3) the smallest degree of membership matrix U = [ u ] and center V = ( v , v , " , v ) . Using FCM, a prior parameters of the classification needed to be determined is the number of data sets. The cluster validity function is used in this paper described as follows [6]: ij
c×n
1
V p (U , c ) =
2
c
c −1 1 n 1 c −1 c 1 n max(uik ) − ∑ ∑ ( ∑ max(uik , u jk )) , K = ∑ i ∑ n k =1 i K i =1 j =i +1 n k =1 i =1
(4)
The clustering criterion is combined with the fuzzy division of compactness and the separation. To get the optimal fuzzy partition, it is combined with the ISODATA clustering algorithm and the proposed criterion function. The algorithm is described as follows: Step 1: cmax ( c ≤ n ): the maximum number of cluster center. T : maximum number of iterations. Index weights m and stop threshold ε > 0 ; m ax
Step2: Center on the clustering number c = 2, 3, " , c , initialize fuzzy clustering max
Center: V = { v , v , " , v 0
10
20
c0
}.
Step 3: For t = 1, 2, " , T , according to the following formula, calculate membership degree matrix and the cluster center:
uij ,
t
⎡ 1 =⎢ ⎢ xi − v j ⎣
⎤ ⎥ 2 ⎥ ⎦
1
⎡ 1 ⎢ ∑ j =1 ⎢ x − v j ⎣ i
a −1
c
⎤ ⎥ 2 ⎥ ⎦
1
a −1
(5)
n
v j, t =
∑ [u i =1 n
ij ,t
]m xi (6)
∑ [uij,t ]m i =1
Where i = 1, 2," , n, j = 1, 2," , c . If V − V < ε , switch to the next step, otherwise repeat step 3; Step 4: According to the formula (4) Calculate V (U , c ) ; If c < c max , switch to t
t −1
p
Step 2; Otherwise, stop iteration. cb is the optimal number of clusters. cb Satisfy the following formula:
140
J. Wang, Y. Zhang, and S. Sun
Vp (U , cb ) = min{Vp (U , c )}, c = 2, 3, " , cmax
(7)
2.2 Flotation Foam Image Texture Feature Selection and Extraction
The state of the production process flotation can be reflected by flotation foam statistical characteristics of image texture, the iterative changes of different gray values which distribute in the space forming the image texture, a gray level relationship exist between the two pixels (the gray space-related features).LCM reflects the relationship of changing characteristics, which image gray on the direction, the adjacent interval and of the magnitude image. That f ( x, y ) pixel of gray-scale I for proceeding, and declination θ , distance δ , gray-scale of the j of Pixel ( x + Δx , y + Δy ) of the probability of simultaneous P (i , j , δ , θ ) .That means the joint distribution statistical form of two-pixel. It can be expressed as follow: P (i , j , δ , θ ) = {[( x , y ), ( x + Δx , y + Δy )] | f ( x , y ) = i
(8)
f (x +Δx, y +Δy) = j; x =0,1,", Nx −1; y =0,1,"Ny −1}
(9)
Where i , j = 0,1, " , L − 1 , x , y is pixel coordinates in the image, L is the gray series of the image, N and N indicated rows and columns of the image number. According to the definition above, GLCM is defined by the i row and column j element. That x
y
all the θ directions of the image and separation as δ , the gray is on the difference frequency of the i and j pixel. θ connects the two clockwise and the pixel x Axis angle. Its value is usually 0o, 45o, 90o and 135o. Three characteristic quantities of the calculation of the statistical characteristics texture energy (E), entropy (ENTS), and moment of inertia (I) are based on the gray co-occurrence matrix, are defined as follows [7]:
E=
L −1 L −1
∑∑ Pˆ (i, j ) i=0
∑∑ Pˆ (i, j) log
ENTS = −
I =
2
Pˆ (i, j )
(11)
L −1 L −1
L −1
∑n { ∑ ∑ 2
n=0
(10)
j=0
i =0 j =0
Pˆ (i , j )}
(12)
n= i− j
Three texture parameters of the flotation froth image can be calculated by using Equation (10) - (12). They can reflect the amount of pulp in the reagent addition and indirectly reflects the state of the production in the flotation process. This article use
Multiple T-S Fuzzy Neural Networks Soft Sensing Modeling of Flotation Process
141
image texture characteristics and process variables as the input variables consisting of PFM T-S fuzzy neural network soft-sensor model of the input variables. 2.3 Dimensionality Reduction Based on PCA
If input vector dimension of T-S model is too large, it would make a great network topology and training become very complex. We can use PCA on the T-S model to reduce model high-dimensional of input vector dimensionality. Using Principal Component Analysis [8], we selected the process of input vector principal component of the T-S model as follows: (1) Data collection operation, a number of the image texture statistical characteristic energy (E), entropy (ENTS), and moment of inertia (I) and the corresponding time period to the ore grade, the ore flow, the ore concentration, give mine size and pharmaceutical flow measurements X ∈ R in flotation process, m : process number of variables (in this case 8), n : a sample of data vectors. In order to avoid the impact of different dimensions of process variables on the results and ease calculate. It is , standard necessary to normalize the data processing. Using medium vector deviation vector σ , the normalized process variables are defined as follows: n ×m
μ
xij = ( xij − μ j ) / σ j (i = 1, " ,n,j = 1, " ,m)
(13)
Define X is Hutchison normalized process variable vector. Covariance matrix is ∑ . (2) Calculate the m characteristic values λ of the covariance matrix ∑ and the j
orthogonal eigenvectors
Pj
of the corresponding units;
(3) Calculate the j principal component t : j
t j = XP j
(14)
(4) Calculate the principal component model: T T T Xˆ = t1 P1 + t2 P2 + " + t j Pj
Where
tj ∈ R
n
is the principal component,
Pj ∈ R
m
(15)
is the load vector, and
Pj
is the
characteristic roots of matrix ∑ of the matrix t arranged in descending. Each pair j , P are the corresponding eigenvectors P by the corresponding eigenvectors of
j
j
j
Eigen values λ of the descending order. The first pair intercepted the largest amount of information of the load vector of all the decomposition and principal component vectors, the rest of inferior race and so on. T = [t , " , t ] is the main element matrix, P = [ P , " , P ] is the load matrix, and then the equation above can be expressed as: j
1
1
m
m
T Xˆ = T P
(16)
142
J. Wang, Y. Zhang, and S. Sun
(5) Calculate the previous k (1 ≤ k ≤ m ) variance contribution rate ηk : k
η k = ∑ λi i =1
m
∑λ
(17)
i
i =1
Then determine the main element k of the PCA model number. Because there is a high correlation between the variables, if general previous k ( k << m ) principal component of the cumulative contribution rate more than 85%, we can think that k principal component reflect the main information of the process. We can choose the k vectors corresponding to the main element in the T matrix as the training input vector of the TS fuzzy neural network. In this paper, by using principal component analysis cope with input vector data of soft-sensor model flotation process, the variance contribution rate of the former four principal components will have reached 92.54%, So choose the previous four principal components as an input of each T-S model. It dot not only preserve the information characteristics of the original various, but also simplify the model structure. 2.4 T-S Fuzzy Neural Network Model
Fuzzy data-based modeling problem can usually be described as follows[6]: Given n pairs of input-output model P ( x, y ) and modeling error indicators ε > 0 . By Optimization fuzzy model F (θ , w, p ) .we can obtain the minimum number of fuzzy rules p and the optimal model parameters (Including the rules antecedent membership function parameters θ and the rule consequent linear parameters w ), which makes the modeling error function E = y − F satisfy the inequality E (θ , w) < ε . According to s historical data sets Victoria {P1 , P2 ," , Pn } , a MISO fuzzy model can be expressed as having the following form of fuzzy rule sets:
:
Ri IF x 1 is A i1 and x 2 is A i2 " and x s is A is , Then y i = zi ( x ) Where,
x = ( x1 , x2 , " , x s ) ∈ U 1 × U 2 × " × U s
(18)
Are linguistic variables; Ai j is fuzzy sets on
the domain of discourse U ∈ R ( j = 1, 2, " , s ) ; Ri (i = 1, 2," , p ) :the i rule; yi ∈ V : the i rule output. z i (x) has the following two forms: Single-valued ( zi ( x ) = bi ), called as j
the Mamdani model; Linear function ( zi ( x ) = bi 0 + ∑ j =1 bi j x j ), called as Takagis
Sugeno (T-S) model. T-S used in this paper fuzzy neural network model diagram as shown below. The model input X r , r = 1," , 4 , each T-S model input is divided into three membership function X r u , u = 1," , 3 ; we use concentrate grade and flotation recovery as a model output ( f h , h = 1, 2 ). For each T-S sub-model, we can use gradient descent method of least squares mixed model to adopt the before and after the pieces of parameters: First, we fix antecedent parameters, prior to the calculation, and identify the last pieces by using least squares linear parameter; Second, we use gradient descent method to adjust the non-linear parameters of fuzzy rules, the algorithm in the reference [9] as described.
Multiple T-S Fuzzy Neural Networks Soft Sensing Modeling of Flotation Process
143
3 Industrial Experiments Collect historical data of production of the flotation process and choose 700 group historical data with uniform and representative. Then we divided the pre-processing datum into two parts. The first part used as the training data by using PMM T-S model and a simple model of T-S model in the paper, then we learn and train the model. The second part is used to verify the soft-sensor model performance. Fig. 2 and Fig.3 are the prediction output of concentrate grade and flotation recovery.
Fig. 2. The predictive output of concentrate grade
Fig. 3. The predictive output of flotation recovery
It can be seen from Fig. 2 and Fig.3 that the multi-TS FNN soft-sensor model is based on PCA and FCM algorithm is better than a single TS soft-sensor model at economic and technical indicators (concentrate grade and flotation recovery). It has higher prediction accuracy and generalization capabilities. The forecast error of concentrate grade is between ±0.3 percent and the flotation recovery of the forecast error is between ±0.1 percent. The simulation results show that the higher prediction accuracy of the model-based prediction model is helpful to stabilize the flotation of
144
J. Wang, Y. Zhang, and S. Sun
pharmaceutical operations and improve operational recovery rate. It has a certain practical value for the flotation process and laid the foundation for optimal control.
4 Conclusions In this paper, a multi-T-S fuzzy neural network soft-sensor model based on principal component analysis and fuzzy c-means clustering algorithm is presented. According to extrapolation of the actual operating data estimates, it shows that the method has a good tracking rate and high estimation accuracy and fully meets computing requirements of the real-time optimal flotation reagent addition system. Acknowledgments. This work is supported by the Scientific Research Foundation of the Higher Education Institutions of Education Bureau of Liaoning Province, China (Grant No. 20060432) and the Program for the Innovative Research Team of Education Bureau of Liaoning Province, China (Grant No. 2008T091).
References 1. Zhang, Y., Wang, J.S., Wang, W.: Application of Expert Control Method to Flotation Process. Control and Decision 19, 1271–1274 (2004) (in Chinese) 2. Zhu, X.F.: Soft-sensing Technique and Its Applications. Journal of South China University of Technology 30, 61–67 (2002) (in Chinese) 3. Wu, J., Xu, Y., Ou, H.Y.: Modular Fuzzy Logic System Based on Principal Component Regression Analysis. Journal of Xi’an Jiaotong University 36, 1311–1314 (2002) (in Chinese) 4. Zhong, W., Yu, J.S.: Study on Soft Sensing Modeling Via FCM-based Multiple Models. Journal of East China University 26, 83–87 (2002) (in Chinese) 5. Bezdek, J.C.: Pattern Recognition with Fuzzy Objective Function Algorithm. Plenum Press, New York (1981) 6. Chen, M.Y., Linkens, D.A.: Rule-base Self-generation and Simplification for Data-driven Fuzzy Models. Fuzzy Sets and Systems 142, 243–265 (2004) 7. Bartolacci, G., Pelletier Jr., P., Tessier Jr., J.: Application of Numerical Image Analysis to Process Diagnosis and Physical Parameter Measurement in Mineral Processes. Minerals Engineering 114, 734–747 (2006) 8. Ipek, H., Ankara, H.: The Application of Statistical Process Control. Minerals Engineering 12, 827–835 (1999) 9. Takagi, T., Sugeno, M.: Fuzzy Identification of Systems and Its Applications to Modeling and Control. IEEE Transaction on Systems, Man, and Cybernetics 15, 116–132 (1985)
A Using Reliability Evaluation Model for Diesel Engine Based on Fuzzy Neural Network Ying-Kui Gu and Kai-Qi Huang School of Mechanical and Electronical Engineering, Jiangxi University of Science and Technology, Ganzhou 341000 China
[email protected]
Abstract. Using reliability evaluation for diesel engine is a complex, dynamic and uncertain process. In order to make an objective and right evaluation, and offer a stronger decision-making tool for designer and user of diesel engine, the neural-network-driven fuzzy reasoning mechanism of using reliability evaluation was developed based on the detail analysis of engine using reliability in the case that there is no sufficient quantitative information or the information is fuzzy and imprecise, where a feedforward neural network was used to replace fuzzy evaluation in the fuzzy system. Applications show that the evaluation result can be used as references for the improvement of reliability and maintainability of engines, and for the establishment of maintenance strategy. Keywords: Neural network, fuzzy logic, fuzzy reasoning, reliability evaluation, multiple attribute decision making.
1 Introduction As the momentum of automobile, engineering machine, ship and power plant units, diesel engine plays an important role in the national economy. With the fast development of science and technology, diesel engine is increasingly developing towards large scale, high speed, high precision, and its working performance has been constantly improved, the automation degree is also higher. On the one hand, the productivity of labour as well as quality of product is greatly improved, and the cost of production as well as consumption of energy is reduced. But on the other hand, once certain part or certain link of diesel engine breaks down, there will be shut down or off production, thus enormous economic loss would be directly or indirectly caused. Therefore, it is very imperative to evaluate the using reliability of diesel engine. However, it is very difficult to evaluate the using reliability of diesel engine because of the complexity, fuzziness and dynamic uncertainty of evaluation process. Fuzzy logic and neural network provide stronger tools for decision-making modeling and analyzing under the fuzzy and uncertain environment [1-3]. In this paper, we presented an approach to evaluate the using reliability of diesel engine using fuzzy logic and neural network. First, the using reliability indices for diesel engine were Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 145–152. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
146
Y.-K. Gu and K.-Q. Huang
analyzed in detail. Second, we developed a fuzzy-logic-based reasoning mechanism to analyze the using reliability under the fuzzy and imprecise design environment. Then, a neural-network-driven fuzzy reasoning system was developed to improve the reasoning efficiency, where the neural network was used to replace fuzzy evaluation in the fuzzy reasoning system. Finally, a case was used to illustrate the proposed neural-network-driven fuzzy reasoning.
2 Using Reliability Indices Analysis for Diesel Engine The main using reliability indices for diesel engine are listed as follows. (1) Mean Time to First Failure (MTTFF) The point estimate value Tˆf of MTTFF and confidence lower limit value T fL of interval estimation are as follows. 1 n Tˆ f = [∑ t i + ( N − n)t . n i =1
(1)
n
T fL =
2[∑ t i + ( N − n) ⋅ t i =1
χ 12−a (2n + 2)
.
(2)
where N is the output of diesel engine. n is the number of diesel engine that break down (not include minor failure). t i is the time to first failure of ith diesel engine, km (h). t is test cut-off distance (or time), km (h). a is significance level, the suggested value a = 0.1 . (2) Mean Time Between Failure (MTBF) The point estimate value Tˆb of MTBF and confidence lower limit value TbL of interval estimation are as follows. Nt . Tˆb = n TbL =
χ
2 1− a
2 Nt . ( 2 n + 2)
(3)
(4)
where N is the output of diesel engine. n is the number of diesel engine that break down (not include minor failure, that is not include 4 kinds of faults). t is test cut-off distance (or time), km (h). a is significance level, the suggested value a = 0.1 . (3) Use Availability (UA) N
A=
∑U
i
i =1
N
∑ (U i =1
i+
Gi )
=
MTBF . MTBF + MTTR
(5)
A Using Reliability Evaluation Model for Diesel Engine Based on Fuzzy Neural Network 147
where N is the output of diesel engine. U i is running time of ith diesel engine, h. Gi is the stopping time of ith diesel engine because of fault (include failure diagnosis time, preparing time, repairing time), h. MTBF is mean time between failure, h. MTTR is mean time to repair, h. (4) Equivalent Failure Rate (EFR) D=
1000 4 ∑ ε j rj . Nt j =1
(6)
(5) Mean Maintenance Cost Ratio of Each Year (MMCR) When reliability of diesel engine is measured from economical efficiency, it can be measured by mean maintenance cost ratio of each year. M =
C ma int enance × 100% . C price
(7)
where C ma int enance is the mean maintenance cost of each diesel engine for the first year, unit: yuan. C price is the price of diesel engine, unit: yuan. (6) Comprehensive Evaluation Value (CEV) Q = 100 −
(MTBF) 0 Nt 0
r
∑K e . i =1
(8)
i i
where (MTBF) 0 is the target value of reliability on diesel engine’s product or the MTBF value of congeneric advanced diesel engine’s product, unit: h. N is output of diesel engine . t 0 is test cut-off distance (or time), km (h). k i is coefficient of injury about ith fault, the specific value can be obtained from following table, ei is coefficient of injury about ith fault, ei =
2 ⋅ t0 , where t i is accumulative work t0 + ti
mileage of diesel engine for ith fault (or time), km (h). Each index can be obtained by reliability trial or cumulative observation, while using reliability on diesel engine can be synthetically measured by all the indices, thus hierarchy model of using reliability on diesel engine can be established as shown in Figure 1. Evaluation result
MTTFF
1# engine
MTBF
UA
2# engine
EFR
CEV
MMCR
3# engine
…
n# engine
Fig. 1. Hierarchical respresentation of using reliability evaluation for engine
148
Y.-K. Gu and K.-Q. Huang
3 The Using Reliability Evaluation Method Based on Fuzzy Logic Reliability evaluation process is a complex problem. There exist many factors, which influence the evaluation result of the engine reliability. In order to simplify the problem and also to illustrate the approach, only the following three important indices are considered in this paper, i.e. (1) use availability, (2) mean maintenance cost ratio of each year, (3) comprehensive evaluation value. All these indices are fairely vague and fuzzy and can be represented linguistically. Thus, fuzzy sets and fuzzy logic are ideally suited for representing these indices. These indices form the input of the model and the output are the desirability of the reliability evaluation results. Figure 1 illustrates the hierarchical linguistic representation. Since these representations are fuzzy and approximate, neural network learning is a necessity to improve this approximate model.
Fuzzy membership
VL
L
M
H
0.25
0.5
0.75
VH
1 0.5
0
1.0 UA, MMCR, CEV
Fig. 2. Membership functions of reliability indices
The indices are represented lingusitically in Figure 2. each index is divided into several different grades based on a universe of discourse of [0,1]. Each grade is represented by a triangular membership function. The three indices, use availability, mean maintenance cost ratio of each year and comprehensive evaluation value has the linguistic grade: VL (very low), L (low), M (medium), H (high), VH (very high) respectively. The problem is to evaluate the engine reliability based on the three indices. Notice that this is a multiple criteria problem and, in general, the three indices cannot mutually compensate. Therefore, the output variable can also be divided into different fuzzy levels. The connections between the input indices and the output evaluation results can be made by the use of fuzzy rules. Table 1 illustrates these fuzzy if-then rules. For example, Rule 1: if the use availability is VH, and the mean maintenance cost ratio of each year is VL and the comprehensive evaluation value is VH, then the grade of the output is A1. Rule 2: if the use availability is VH, and the mean maintenance cost ratio of each year is VL and the comprehensive evaluation value is H, then the grade of the output is A2.
A Using Reliability Evaluation Model for Diesel Engine Based on Fuzzy Neural Network 149 Table 1. Fuzzy if-then rules I/O Sample 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
Input (if) MMCR VL VL L L VL M M L M VL VL L L VL M M L M VL VL L L VL M M L M
UA VH VH VH VH VH VH VH VH VH H H H H H H H H H M M M M M M M M M
Out (then) Ranking Classic A1 A2 A3 A4 A5 A6 A7 A8 A9 B1 B2 B3 B4 B5 B6 B7 B8 B9 C1 C2 C3 C4 C5 C6 C7 C8 C9
CEV VH H VH H M VH H M M VH H VH H M VH H M M VH H VH H M VH H M M
The expression if-and-then is in fuzzy logic terms. To simplify the illustration, only three levels of each index with a total of 27 rules are considered in Table 1. the different output grades for these 27 rules are shown in Figure 3. The output ranking has three classes designated as A, B and C, and each class has five grades. For example, class A is divided into A1, A2, A3, A4 and A5.
Fuzzy membership
Class
C5 B5(C4) B4(C3) B3(C2) A5(B2) A4(B1) A3
A2
A1
1 0.5
0
0.6
0.65
0.7
0.75
0.8
0.85
0.9
0.95
Fig. 3. The membership function for output ranking score
0.1
150
Y.-K. Gu and K.-Q. Huang
The fuzzy rules or the interconnections between the input and the output can first formulated approximately based on the vague knowledge of the actual system. These approximate interconnections can be refined for any given system by actual learning using neural network. The training sample sets can be obtained from the actual system experts.
4 BP-Neural-Network-Driven Fuzzy Evaluation Model Because back-propagation neural network has the ability to learn by examples, it has been used in pattern matching, pattern classification and pattern recognition. Therefore, it can be used to establish the neural-network-driven fuzzy evaluation model for engine reliability. The BP network is constructed in three layers. Nine nodes in input layer, five nodes in hidden layer, and one node in the output layer. In the training process, a hybrid learning algorithm that combined unsupervised learning and supervised gradient-descent learning procedures was used to build the fuzzy rules and train membership functions. Through assigning values to the input variables, calculating the membership function measures of the input nodes, and achieving the output membership function using fuzzy reasoning, the correct data set can be achieved. All the fuzzy rules are involved and can be encoded in the neural network. When a set of input data is provided to the neural network, these encoded rules are activated to different degree at the same time. The output membership function can be generated by the neural network computation, and it is related to all the fuzzy rules. To obtain the training samples, the center of the area defuzzification method is used. The center of the area defuzzification equation for discrete system can be represented by [4,5]
∑ f μ (f ) n
j
F=
j =1 n
0
j
∑ μ0 ( f j )
.
(9)
j =1
where n is the number of quantization levels, and μ 0 ( f j ) is the membership at the quantized level j. using the above equation and the fuzzy if-then rules in Table 1 with the help of Figure 3, the 54 training samples can be obtained.
5 Case Study Take the reliability evaluation of YC6100 as an example to illustrate the established BP-neural-network model. There are 10 engines waiting for evaluating their reliability. The reliability index and its evaluation value for these engines were listed in Table 2.
A Using Reliability Evaluation Model for Diesel Engine Based on Fuzzy Neural Network 151 Table 2. The input and output of the training sample I/O Index sample 1# engine 2# engine 3# engine 4# engine 5# engine 6# engine 7# engine 8# engine 9# engine 10# engine
Use availability VH 0.92 0 0.88 0 0.95 0 0.86 0.78 0.75 0.65
H 0.08 0.89 0.12 0.93 0.05 0.96 0.14 0.22 0.25 0.35
M 0 0.11 0 0.07 0 0.04 0 0 0 0
Input Mean maintenance cost ratio of each year VL L M 0.85 0.15 0 0.67 0.33 0 0 0.95 0.03 0.83 0.17 0 0 0.92 0.08 0.74 0.26 0 0 0.76 0.24 0.68 0.32 0 0.65 0.35 0 0.62 0.38 0
Output Comprehensive evaluation value VH 0.85 0 0.87 0.82 0.79 0 0.75 0 0.65 0
H 0.15 0.87 0.13 0.18 0.21 0.88 0.25 0.82 0.35 0.86
M 0 0.13 0 0 0 0.12 0 0.18 0 0.14
Value
Rank
0.9321 0.8564 0.9023 0.8765 0.9125 0.8128 0.8956 0.7986 0.7887 0.7469
1 6 3 5 2 7 4 8 9 10
Based on the analysis in Section 2 and the established neural network model in Section 3 and 4, and the network precision of training parameter is 10-6 and the study rate is 0.001. The ranking order are expressed as: R1 > R5 > R3 > R7 > R4 > R2 > R6 > R8 > R9 > R10. The output result shows that the 1# engine has higher using reliability.
6 Conclusions In order to decrease the influence of the fuzzy and uncertain factors during evaluation process, the fuzzy logic and neural network were combined to establish the evaluation model for diesel engine reliability. The evaluation result of engine using reliability can be obtained by training the network. Applications show that The result can be used as references for the improvement of reliability and maintainability of engines, and for the establishment of maintenance strategy, and it also lay a theoretical foundation for studying the improvement of evaluation method for the engine reliability as well.
Acknowledgments This research was partially supported by the Jiangxi Provincial Natural Science Foundation under the contract number 2007GQC0654 and the Science Foundation of Education Commission of Jiangxi Province under the contract number GJJ09245.
References 1. Gu, Y.K., Yang, Z.Y.: TS-neural-network-based maintenance decision model for diesel engine. In: Liu, D., Fei, S., Hou, Z.-G., Zhang, H., Sun, C. (eds.) ISNN 2007. LNCS, vol. 4491, pp. 557–565. Springer, Heidelberg (2007)
152
Y.-K. Gu and K.-Q. Huang
2. Su, J., Ao, C.L., Jiao, Y.: Evaluation of tractor reliability based on Fuzzy Neural Network. Journal of Northeast Agricultural University 39, 99–103 (2008) 3. Hsu, C.H., Jiang, B.C., Lee, E.S.: Fuzzy Neural Network Modeling for Product Development. Mathematical and Computer Modelling 29, 71–81 (1999) 4. Gu, Y.K., He, X.W.: Neural-network-driven fuzzy optimum selection for mechanism schemes. In: Liu, D., Fei, S., Hou, Z., Zhang, H., Sun, C. (eds.) ISNN 2007. LNCS, vol. 4492, pp. 275–283. Springer, Heidelberg (2007) 5. Sun, J., Kalenchuk, D.K., Xue, D., Gu, P.: Design candidate identification using neural network-based fuzzy reasoning. Robotics and Computer Integrated Manufacturing 16, 383–396 (2000)
Fuzzy Sliding Mode Control with Perturbation Estimation for a Piezoactuated Micromanipulator Qingsong Xu and Yangmin Li Department of Electromechanical Engineering, Faculty of Science and Technology, University of Macau, Av. Padre Tom´ as Pereira, Taipa, Macao SAR, China {qsxu, ymli}@umac.mo http://www.sftw.umac.mo/~ yangmin/
Abstract. This paper is concentrated on control system design for an XY parallel micromanipulator with piezoelectric actuation. The decoupled property of the manipulator enables the employment of single-inputsingle-output controllers for the two working axes. To compensate for nonlinear hysteresis effect stemming from piezoelectric actuator, the dynamics with Bouc-Wen hysteresis model is derived and identified for the system. Afterwards, a real-time fuzzy sliding mode control with perturbation estimation (FSMCPE) along with fuzzy switching control is proposed to attenuate the chattering phenomenon. Experimental studies reveal that the designed FSMCPE is superior to conventional SMCPE in terms of both positioning and tracking performances of the micromanipulator system, which validates the effectiveness of the presented control system design as well. Keywords: Micropositioning, parallel manipulator, fuzzy sliding mode control.
1
Introduction
Recently, XY parallel micromanipulators have been paid extensively attentions [1,2] due to their promising applications in micro-/nano-manipulation fields such as scanning probe microscopy, biological cell manipulation, and so on. Nevertheless, majority of existing manipulators have either an uncomplicated structure along with a coupled motion or a decoupled motion which is at the expense of a complicated structure. To benefit both controller design and prototype fabrication in practice, a parallel micromanipulator with both a decoupled motion and an uncomplicated structure is desirable. Moreover, it is suggested [3] that the micromanipulator is preferred to be designed with both input and output decoupling properties in virtue of actuation isolation and decoupled output motion. This totally decoupling property is necessary for some situations where the platform is under-actuated or sensory feedback of end-effector positions is not allowed [4]. One of such XY parallel micromanipulators has been recently designed Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 153–160. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com
154
Q. Xu and Y. Li
by the authors. The objective of the current research is to design an appropriate controller to enable a sub-micron accuracy for the micromanipulator. In view of the advantages of high stiffness as well as high output force, piezoelectric actuator (PZT) is widely used for the drives of micromanipulators. However, a piezoactuated manipulator is a highly nonlinear system due to the hysteresis and creep behaviors introduced by the PZT. In the literature, various types of approaches have been exploited to alleviate the hysteresis effect. Particularly, sliding mode control (SMC) has been employed in the piezoactuated manipulators since the SMC is an effective and simple way to deal with model imperfection and uncertainties for nonlinear systems. As a nonlinear control strategy, SMC drives the system’s state trajectory onto a specified sliding surface and maintains the trajectory on this surface for the subsequent time. However, in conventional SMC design, a priori knowledge of the bounds on system uncertainties has to be acquired. As a consequence, the controller based on the above knowledge tends to be overconservative, which may result in poor tracking performance and undesirable oscillations in control signal. To overcome such drawback, SMC with perturbation estimation (SMCPE) has been proposed [5] as an enhanced version of the conventional SMC. It is shown that the need to know the uncertainty bounds is removed in the SMCPE and is replaced with a strategy of online estimation of perturbations. Nevertheless, due to the discontinuity of the signum function, high-frequency chattering phenomenon may occur in the control input, which is a severe problem especially when the state trajectory is close to the sliding surface. To attenuate the chattering phenomenon, the boundary layer technique is usually adopted by replacing the signum function with saturation function in the switching control term. The chattering can be avoided with the saturation function. As the cost, sliding mode performances will be compromised at the same time [6]. Additionally, in selecting the boundary layer, a tradeoff between the chattering and tracking error is always required. In this research, an alternative approach employing a fuzzy SMCPE controller with fuzzy switching control is proposed to reduce the chattering effect in the conventional SMCPE. The efficiency of the presented fuzzy SMCPE controller will be demonstrated by experimental studies performed on an XY micromanipulator prototype in the following sections.
2
System Description
The experimental setup of the XY parallel micromanipulator is shown in Fig. 1, where the XY stage is fabricated from the alloy Al 7075. In particular, the stage has a compact size owing to a two-layered mechanism design. It is actuated with two 20 μm-stroke PZT (model PAS020 produced by Thorlabs, Inc.) through displacement amplifiers. A PCI-based D/A board (PCI-6703 from National Instruments Corp.) is adopted to produce an analog voltage, which is then amplified by a two-axis piezo-amplifier (BPC002 from the Thorlabs) to provide a voltage of 0–75 V for the drives of the PZTs. The output displacements of the mobile platform are measured by two laser displacement sensors (Microtrak II,
FSMCPE for a Piezoactuated Micromanipulator
Plant
Real-Time Controller
Prototype
PZT
D/A (PCI-6703)
Piezo amplifier
Laser sensor
PC (Malab/Simulink, Real-Time Windows Target Toolbox)
155
A/D (PCI-6034E)
Sensor controller
Fig. 1. Schematic diagram of apparatus connection
head model: LTC-025-02, from MTI Instruments, Inc.), whose analog voltage outputs are read simultaneously by the PC through a PCI-based data acquisition board (PCI-6034E from the National Instruments). It can be calculated that the resolution of the position detection system is 0.038 μm. Besides, the software of Matlab/Simulink with Real-Time Windows Target toolbox is employed to implement the real-time control of the micromanipulator system. Open-loop tests show that a workspace range of 138×128 μm2 can be attained by the XY micromanipulator, and the cross-talk between the two working axes is less than 5%. Thus, the two-axis motions are well-decoupled, which allows the employment of two SISO controllers for x- and y-axes, respectively.
3
Dynamic Modeling and Identification
It is known that a piezoactuated system is nonlinear mainly attributed to the complicated hysteresis effects. Due to the decoupled property of the XY stage, the x- and y-axes can be treated independently. For brevity, only the x-axis motion is described in this paper. The dynamic model of the entire micropositioning system with Bouc-Wen hysteresis model can be established as follows: m¨ x + bx˙ + kx = k(du − h) + p h˙ = αdu˙ − β|u|h ˙ − γ u|h| ˙
(1) (2)
where the parameters m, b, k, and x represent the mass, damping coefficient, stiffness, and x-axis displacement of the XY stage, respectively; d is the piezoelectric coefficient, u denotes the input voltage, and h indicates the hysteretic loop in terms of displacement whose magnitude and shape are determined by parameters α, β, and γ. Besides, p denotes the overall perturbation of the system arising from model parameter uncertainties, unmodeled dynamics, and other unknown terms. By neglecting the perturbation, the seven parameters (m, b, k, d, α, β, and γ) of the system model with Bouc-Wen hysteresis can be identified simultaneously by an optimization algorithm. Specifically, with a 0.2-Hz multi-amplitude sine-wave voltage applied to the PZT, the primary output motion of the stage is recorded and collected for the purpose of model identification. By employing
156
Q. Xu and Y. Li
the particle swarm optimization (PSO) [7], the following model parameters are identified for the x-axis motion: m = 0.1040 kg, b = 1.8625×104 N s/m, k = 1.9920×105 N/m, d = 1.8347×10−5 m/V, α = 0.0786, β = 0.0995, and γ = 0.0008. Because the Bouc-Wen model cannot represent nonsymmetric hysteresis exactly, there exist a maximal error of 7.0% between the identified model and experimental result. In order to compensate for the model errors and other uncertainties, a SMCPE is designed below.
4
Conventional SMCPE Design
Based on the perturbation estimation strategy [5], the system model (1) becomes: m¨ x(t) + bx(t) ˙ + kx(t) = k[du(t) − h(t)] + pest (t) + p(t)
(3)
with the estimated perturbation: x(t) + bx(t) ˙ + kx(t) − k[du(t − T ) − h(t − T )]. pest (t) = m¨
(4)
where p(t) = p(t) − pest (t) represents an error between the system’s real perturbation and its estimation. To design a SMCPE controller, an error coordinate is defined as: e(t) = x(t)− xd (t), where xd represents the desired position trajectory, and t denotes the time variable. In view of the linear second-order dynamic system, a first-order sliding surface is defined as s = e˙ + λe, where λ (λ > 0) is a design parameter. It can be proved that, the tracking error e satisfies lim e(t) = 0 and the sliding variable t→∞ meets the sliding condition ss˙ < 0 if the control law is given by: u = ueq + usw =
1 pest m m b ( − λ)x˙ + (x + h) − + (¨ xd + λx˙ d ) − η sgn(s) (5) kd m d kd kd
p| ε where η > | kd + kd , the last term in (5) represents the switching or corrective control usw and the rest is the equivalent control term ueq . Besides, sgn(·) represents the signum function, η is a positive switching gain, and ε is an arbitrary positive constant. Additionally, inserting (4) into (5), gives
1 m u(t) = u(t − T ) + [h(t) − h(t − T )] − [¨ e(t) + λe(t)] ˙ − η sgn(s(t)). d kd
(6)
Furthermore, due to the discontinuity of the signum function sgn(s), chattering may occur in the control input. To alleviate the chattering phenomenon, the boundary layer technique is usually adopted by replacing the signum function in switching control term usw in (6) with the saturation function: sgn(s) for |s| > δ sat(s) = (7) s/δ for |s| ≤ δ where the positive constant δ represents the boundary layer thickness, which ensures that s is always bounded by δ. The chattering can be avoided with the
FSMCPE for a Piezoactuated Micromanipulator
157
saturation function. Nevertheless, sliding mode performances will be compromised as the cost. Besides, in the selection of parameter δ, a tradeoff between the chattering and tracking error is required. In the next section, an alternative approach employing a fuzzy SMCPE controller with fuzzy switching control is proposed to reduce the chattering effect.
5
Fuzzy SMCPE Design
In this research, the sliding variable s which measures the closeness to the sliding surface is employed as the input variable to design a fuzzy switching controller, which will replace the switching control term usw in the control signal and regulate this control term on-line to improve the control performance. A fuzzy controller consists of three modules as described below. Fuzzification: The input variable s is imported into the fuzzy controller and classified by the input membership functions in the fuzzification module. Specifically, the input space is classified into five fuzzy sets, which are specified by the selected triangular membership functions. A membership function associated with a fuzzy set maps an input value to its appropriate membership value. The degree of an input variable belonging to a fuzzy set is denoted by a membership value between 0 and 1. Particularly, the input fuzzy sets are: F (s) = {Fs1 , Fs2 , · · · , Fs5 } = {N B, N M, ZO, P M, P B}
(8)
where the labels of fuzzy sets N B, N M , ZO, P M , and P B represent negative big, negative medium, zero, positive medium, and positive big, respectively. Fuzzy inference engine and fuzzy rules: The input information is converted into the output by the fuzzy inference engine module through the fuzzy rules. In this research, the Mamdani’s fuzzy inference method is adopted. The output space is classified into five fuzzy sets: F (u) = {Fu1 , Fu2 , · · · , Fu5 } = {N B, N M, ZO, P M, P B}
(9)
where the meanings of fuzzy sets labels N B, N M , ZO, P M , and P B are the same as the input fuzzy sets. Besides, the triangular-type membership functions are chosen for the output sets. Selecting appropriate fuzzy rules is the major task in fuzzy controller design. According to the sliding surface, switching control effort is determined to satisfy the sliding condition ss˙ < 0, so that the system state is forced to move toward the origin along the sliding surface. From empirical knowledge of sliding mode controller design, a large switching control term will drive the state trajectory to approach the sliding surface s = 0 quickly. However, it tends to excite large chattering at the same time. Thus, when the sliding variable s is negative or positive large, the switching control term usw should be increased accordingly, and vice versa. Hence, the rules which determines the switching control output (6−i) can be established as Ri : If s is Fsi then usw is Fu , for i = 1, 2, · · · , 5.
Degree of membership
Q. Xu and Y. Li
NB
1
NM ZO PM
0.4
PB
0.3 (a)
0.5
0.2 0.1
0 −2
−1.5
NB
1
0.5
−1
−0.5 NM
0 s
0.5
1
ZO
PM
PB
1.5
2
usw
Degree of membership
158
0
(c)
−0.1 −0.2
(b)
−0.3 0 −0.6
−0.4
−0.2
0 usw
0.2
0.4
0.6
−0.4 −2
−1.5
−1
−0.5
0 s
0.5
1
1.5
2
Fig. 2. Input (a) and output (b) membership functions, and the control signal (c) of the fuzzy controller
The fuzzy values of the rule parameters N B, N M , ZO, P M , and P B are tuned by the trial and error procedure through experiments, and the input and output membership functions are depicted in Figs. 2(a) and 2(b), respectively. The fuzzy inference engine performs a mapping from the input fuzzy sets to output fuzzy sets based on the established IF-THEN fuzzy logic rules. Defuzzification module: The input for the defuzzification process is the output fuzzy sets obtained above, and the output is a single crisp number which represents the output of the fuzzy controller. Using the centroid defuzzification method, Mamdani (minimum function) implication in the rule base, and maximum aggregation method (μu ), the output of the fuzzy controller can be derived as: N i=1 usw μu uf s = (10) N i=1 μu where N is the total number of fuzzy rules. The control signal of the fuzzy controller with respect to the input variable is plotted in Fig. 2(c), which exhibits a shape similar to the saturation function. By replacing the switching control in (6) with the fuzzy controller output, a fuzzy SMCPE control input can be expressed as: 1 m u(t) = u(t − T ) + [h(t) − h(t − T )] − [¨ e(t) + λe(t)] ˙ + uf s (s(t)). d kd
6
(11)
Experimental Studies
Based on the fuzzy SMCPE controller designed above, several experiments are conducted to demonstrate the performance of the micromanipulator. The implemented closed-loop sampling time is 0.003 seconds for the real-time control. Concerning the fuzzy SMCPE (FSMCPE) controller, the sliding surface is defined with λ = 2 × 104 . For comparisons, a conventional SMCPE with saturation-function switching control is also implemented with the parameters
FSMCPE for a Piezoactuated Micromanipulator
14 SMCPE
25
FSMCPE
12
Desired SMCPE FSMCPE
20
Position x (μm)
Position x (μm)
10 8 6 10.5 11
4
10
10 2
9
9.5 4
8 0
7 0
−2 0
0.5 1
4.5
4
15 10 5 0
5
1 2 3 Time (sec.)
−5 0
5
1
(a)
5
4
5
FSMCPE 1.5
Control effort (V)
Position error x (μm)
4
2 SMCPE FSMCPE
0.5 0 −0.5
1 0.5 0
−1 −1.5 0
2 3 Time (sec.)
(b)
1.5 1
159
1
2 3 Time (sec.)
(c)
4
5
−0.5 0
1
2 3 Time (sec.)
(d)
Fig. 3. (a) Step responses obtained with SMCPE and fuzzy SMCPE (FSMCPE); 0.4Hz sinusoidal motion tracking results (b), tracking errors (c), and FSMCPE control effort (d)
of λ = 2 × 104 , δ = 0.50, and η = 0.32, which gives a switching control signal shape like the one produced by the fuzzy controller as shown in Fig. 2(c). First, a 10-μm step response is experimentally tested. The control results with the two types of controllers are depicted in Fig. 3(a). It is observed that the conventional SMCPE produces a response with a longer settling time. In contrast, the fuzzy SMCPE controller gives a rapid response, which is at the expense of a larger overshoot value. In addition, at the steady-state, both controllers can maintain the positioning error within ±0.5 μm. Second, the sinusoidal motion tracking performance of the micromanipulator is verified. Given a 0.4-Hz sine-wave signal with peak-to-peak (p-p) amplitude of 20 μm, the tracking results of the micromanipulator are shown in Figs. 3(b)–(c). It is seen that the SMCPE can only suppress the p-p error within ±1.3 μm. On the contrary, the FSMCPE achieve a p-p error of ±0.8 μm, which means that the tracking error has been reduced by 38% with respect to the conventional SMCPE control. The low-level tracking error also implies that the hysteresis has been substantially compensated by the FSMCPE approach. In addition, the control effort of FSMCPE as plotted in Fig. 3(d) indicates that the chattering phenomena have been eliminated. It is noticeable that above performances for the parallel micromanipulator are obtained within the input rate of 0.4 Hz. The limitation mainly lies in the relatively low sampling frequency due to the hardware restrictions. Additional
160
Q. Xu and Y. Li
experiments show that the sub-micron accuracy of the stage is degraded as the increasing of input rate. Once a higher bandwidth is achieved, a high-rate tracking control can be implemented even with the same control method proposed in this paper.
7
Conclusions
The studies presented in this research shows that the fuzzy sliding mode control with perturbation estimation technique can improve the tracking performance of a piezo-driven XY parallel micromanipulator. Based on a relatively simple Bouc-Wen hysteresis model, the dynamic model of the system is established and identified by PSO approach. The fuzzy controller is employed to replace the conventional switching control in SMCPE so as to overcome the limitation of boundary layer technique. Experimental results show that the hysteresis effect has been significantly reduced and chattering phenomenon has been suppressed by the proposed fuzzy SMCPE substantially, which leads to a sub-micron accuracy motion tracking of the micromanipulator. In the future works, a highbandwidth control will be carried out to improve the positioning performance of the system by increasing input rate. Acknowledgments. The authors appreciate the fund support from the research committee of University of Macau under grant no.: UL016/08-Y2/EME/ LYM01/FST and Macao Science and Technology Development Fund under grant no.: 016/2008/A1.
References 1. Pham, H.H., Chen, I.M.: Evaluation of resolution of flexure parallel mechanisms for ultraprecision manipulation. Rev. Sci. Instrum. 75(9), 3016–3024 (2004) 2. Li, Y., Xu, Q.: A novel design and analysis of a 2-DOF compliant parallel micromanipulator for nanomanipulation. IEEE Trans. Automat. Sci. Eng. 3(3), 248–254 (2006) 3. Li, Y., Xu, Q.: Design and analysis of a totally decoupled flexure-based XY parallel micromanipulator. IEEE Trans. Robot. 25(3), 645–657 (2009) 4. Awtar, S., Slocum, A.H.: Constraint-based design of parallel kinematic XY flexure mechanisms. ASME J. Mech. Des. 129(8), 816–830 (2007) 5. Elmali, H., Olgac, N.: Sliding mode control with perturbation estimation: A new approach. Int. J. Contr. 56, 923–941 (1992) 6. Kim, S.W., Lee, J.J.: Design of a fuzzy controller with fuzzy sliding surface. Fuzzy Sets and Systems 71(3), 359–367 (1995) 7. Xu, Q., Li, Y.: Error analysis and optimal design of a class of translational parallel kinematic machine using particle swarm optimization. Robotica 27(1), 67–78 (2009)
A Credit Risk Rating Model Based on Fuzzy Neural Network Ke-Jun Zhu1, Pin-Chang Chen1,2, and Yu-Teng Chang2 1
China University of Geosciences, Wuhan Hubei 430074, China 2 Yu Da University, Miaoli 36143, Taiwan
[email protected], {chenpc,cyt}@ydu.edu.tw
Abstract. This study integrates the characteristics of credit risk rating and artificial intelligence technology into a credit risk rating model based on fuzzy neural network. The combination of fuzzy theory and neural network provides a good foundation for credit risk rating, making this model with fewer parameters, faster learning and less training samples. This study confirms that fuzzy neural network is an effective method for credit risk rating. The results of this study can solve the shortcomings in existing credit risk rating model and provide more information for decision-making reference. Keywords: fuzzy theory, neural network, credit risk rating.
1 Introduction Credit risk has significant influence on the financial market. It can directly affect various activities in modern economy [1]. When rating the credit risk of companies or individuals, an accurate and efficient prediction model is important to financial institutions, so as to determine an effective loan solution to reduce the rate of non-performing loans [2]. This study examines the shortcomings and deficiencies in models and methods of current credit risk rating models, and combines the characteristics of credit risk rating with artificial intelligence technology to develop a credit risk rating model based on fuzzy neural network. This study is intended to construct a fuzzy neural network model through input and output data sets, and optimize the parameters in membership function through a separate back-propagation algorithm. According to the combination of fuzzy theory and neural network, the function is not based on intuition or experience, but on any given values, which is especially important for this proposed model with complex characteristics [3]. In addition, the membership function and fuzzy rules are acquired automatically, so that the application object of fuzzy inference is not limited to the models with pre-determined characteristics of some model variables [4]. The traditional neural network has no specific weight of physical meaning, but given the physical meaning of inference parameters in fuzzy theory, which increases the transparency of the black box Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 161–171. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
162
K.-J. Zhu, P.-C. Chen, and Y.-T. Chang
and expresses neural network’s inference function appropriately [5]. The fuzzy neural inference has a problem that there are too many fuzzy rules, and it is inferior to back-propagation neural network in multi-dimensional images [6]. Fuzzy neural network can effectively utilize the advantages of both fuzzy theory and neural network, and make up for their shortages. The use of fuzzy neural network for credit risk rating not only can improve the learning and expression abilities of the rating index system, but also improve the accuracy of rating at the same time, achieving the purpose of more efficient credit risk rating [7].
2 Literature Review Fuzzy theory and neural network are two distinct areas of research, but they have the same purpose because they both are used to study artificial intelligence. They are equivalent in theory, but in practice, they have their own advantages and disadvantages. Fuzzy theory focuses on human brain’s macro functions, which can be expressed with language and concept. According to the membership function and serial fuzzy inference rules defined by human, it deals with all kinds of fuzzy information with the logical inference [8]. Neural network focuses mainly on the microstructure of the human brain, and studies the generation and formation of human’s wisdom from physical structure. Through the nonlinear dynamics on neural network with self-learning, self-organizing and self-adaptive functions, it processes the information which cannot be verbalized [9]. Fuzzy theory and neural network are two new methods for information processing, they are self-adaptive estimators that are not related to the model, with a certain degree of expertise in dealing with uncertain information and the information contained noise [10]. Fuzzy theory and neural network have their own advantages and disadvantages. The advantages of neural network are its self-learning and self-adaptive abilities as well as its parallel processing capacity and good fault tolerance. However, in the expression of knowledge and interpretation of rules learned, there are obvious deficiencies and its learning speed is usually unsatisfactory. The advantages of fuzzy theory include it can expresses knowledge naturally and easily, deal with uncertain information, express knowledge with fuzzy and natural language, and easy to use expertise. Fuzzy theory can realize fuzzy inference of knowledge with simple computing. However, fuzzy theory has poor learning ability, and it is very weak in knowledge acquisition [11]. Fuzzy theory and neural network are correlated and complementary in many aspects. It is theoretically proven that fuzzy theory can approach a nonlinear function with any precision, while neural network has the imaging capacity. It means that both two have close connection. If fuzzy theory and neural network are combined, which supplements each other, field of information processing will rise to a new realm. In such a combined fuzzy neural network, neural network simulates brain’s topology, that is, the “hardware”; and fuzzy theory simulates the thinking capacity of fuzzy information processing, namely “software” [12]. As a result, fuzzy neural network absorbs the essence of the biological cybernetics, it is distributed and parallel in storage and processing of information. In this way, all kinds of neurons support and supplement information mutually based on information sharing. It can collect inputs from nearby neurons connected to the input, calculate the
A Credit Risk Rating Model Based on Fuzzy Neural Network
163
output and then pass to higher or the other neurons, thus accomplish stronger fault tolerance and association ability. The overall performance is not affected severely when partial neurons are damaged. The output is not distorted seriously if input signal is subject to a certain degree of noise pollution. It proves that fuzzy neural network has a unique ability and can be widely used in many research fields [13].
3 Methodology Due to the complexity and high nonlinearity of credit risk, there is a wide deviation between traditional linear model and actual system. This study is intended to use non-linear and generalization capacity according to fuzzy theory and neural network, establish a credit risk rating model based on fuzzy neural network, which can achieve risk assessment and prediction. The model is based on the financial risk model and company scale rating, coupled with the consideration on companies’ fundamental factors, so that the rating factors are comprehensive, and the results are more scientific and accurate. When constructing the financial risk model, due to neural network’s self-learning, self-adaptive capacity, parallel processing capability and good fault tolerance, it can obtain more accurate non-linear outputs from various financial indexes. Fuzzy theory expresses knowledge naturally and easily, which is able to process uncertain information. It can express knowledge with fuzzy and natural language, as well as use expertise to assess the company scale. Therefore, both fuzzy theory and neural network are used in credit risk rating to obtain fewer parameters, faster learning and less training samples. The research methodology of this study is described as follows: Step 1: Collect examples of industrial enterprises, substitute companies’ credit-related data into the method proposed by this study to confirm the effectiveness. Step 2: Build a financial risk model with back-propagation neural network. Compared with traditional methods of statistical analysis, neural network is more accurate and with lower predictive risk and smaller error. SPSS statistical software is used to carry out factor analysis on company balance sheet and income statement. Step 3: Build company scale rating with fuzzy theory. Based on the concept of corporate risk rating model, the company scale can be divided into four types, which are small, medium, large, and extra large. Such vague language is particularly suited to be realized with the concept of fuzzy theory. Based on expertise, parameters of the model are adjusted continuously according to the actual output, so as to determine the scope of the scale rating. Step 4: Carry out credit risk rating with fuzzy neural network. Based on the financial risk rating from Step 2 and scale rating from Step 3, the fundamental factors are added in the final credit risk rating along with the description of the index system of corporate risk rating. Such input is companies’ total score rated by the experts from quality, strength, environment, credit status, critical incidents, and leader’s quality, and then it is naturalized into a value among [0,1]. A corporate credit risk rating model is constructed based on the above-mentioned rules and fuzzy neural network. Step 5: Analyze the data and propose a conclusion.
164
K.-J. Zhu, P.-C. Chen, and Y.-T. Chang
4 Data Analysis 4.1 Acquisition and Preprocessing of Sample Data Since business environments and scopes in different industries vary greatly, their financial and non-financial indexes are not comparable. Thus, the same industry with short-term loan as sample data are selected by this study to avoid the problems mentioned above. The data of this study are collected from the credit department of a bank. A total of 250 samples are collected. The sample data has to be preprocessed first. In the actual application, the sample cannot be increased unlimitedly, so the selected samples must be representative. Since the data obtained by this study has larger capacity, there are a wider range of indicators. The double and triple standard deviation tests are conducted to remove data anomalies, and 250 sample data are obtained. Among those, 200 are selected as training sample data and 50 are test sample data. As this rating model is applied as a corporate risk rating for commercial banks through a more comprehensive system of indexes, it is obvious that application of fuzzy neural network is a relatively new attempt. However, too much input information in the fuzzy neural network will lead to explosive growth of training time. This study focuses on the financial risks and scale, and then applies fuzzy neural network to obtain the final results of credit risk rating. 4.2 Construct the Financial Risk Model Based on Back-Propagation Neural Network Compared with the traditional methods of statistical analysis, artificial neural network is more accurate with lower predictive risk and smaller error. SPSS statistical software is used to perform factor analysis on company’s balance sheet and income statement. The main purpose is to determine the smallest number of factors which can interpret the relationship between observed variables. According to the characteristics of eigenvalue criteria, under the current conditions of sample data, the number of factors is 5. Also, from the factor loading matrix, it is clear that the economic implications of each factor are obvious; the specific results are the sample data in this study. In a three-layer back-propagation neural network model, the first layer is the input layer and the number of neurons is 5. The input layer includes the data input of profitability, growth capacity, operating capacity, short-term solvency and long-term solvency, respectively. The second layer is the hidden layer. The third layer is the output layer and the number of neuron is 1. After completing the model training, it can be used to solve practical financial risk rating. The MATLAB software is applied for back-propagation neural network, the transfer function of hidden layer is tansig, the transfer function of neuron in output layer is logsig, this is because the elements in target vector are located between [0, 1], meeting the output requirements of logsig. Back-propagation neural network structure is as shown in Fig. 1.
A Credit Risk Rating Model Based on Fuzzy Neural Network
165
Fig. 1. Back-Propagation Neural Network Structure
4.3 Construct the Company Scale Rating Based on Fuzzy Inference System According to expertise, company scale has a direct impact on business credit. It is determined by the company's total income and total assets. In this study, the Mamdani fuzzy theory with fuzzy generator and eliminator is applied to build the fuzzy inference model. According to the concept of corporate risk rating model, the company scale can be divided into four types, which are small, medium, large, and extra large. Such vague language is particularly suited to realize with the fuzzy inference system. Based on expertise, the parameters of the scale rating model are adjusted continuously according to the actual output, so the scope of scale rating is determined: 0 small 0.27; 0.27<medium 0.587; 0.587
≦
≦
≦
≦
≦
(1) Establish corresponding data sets into fuzzy inference system for storage, modification and management of the fuzzy theory (2) Determine the input and output linguistic variables and linguistic values (a) Two inputs: the total assets and total income; (b) One output: company scale; (c) The corresponding fuzzy sets: T (total assets) = {small, medium, large, extra large}; T (total income) = {low, medium, high, extra high}; T (company scale) = {small, medium, large, extra large}; (3) Determine the membership function of linguistic value, including the types and parameters of the membership function, as shown in Table 1. (a) gaussmf (Gaussian membership function): (1) (b) gauss2mf (two-sided-Gaussian membership function):
166
K.-J. Zhu, P.-C. Chen, and Y.-T. Chang
(2)
(c) trimf (triangle membership function):
(3)
Table 1. Types and Parameters of the Membership Function Linguistic Variable total assets
total income
company scale
Linguistic Value small medium large extra large low medium high extra high small medium large extra large
Membership Function gaussmf gauss2mf gauss2mf gaussmf gaussmf gauss2mf gauss2mf gaussmf trimf trimf trimf trimf
Parameter [0.8, 0] [3.6, 2.75] [0.8, 2.75] [8, 27.5] [12, 27.5] [16, 80] [0.8, 0] [3.6, 2.75] [0.8, 2.75] [8, 27.5] [12, 27.5] [16, 80] [-0.1, 0.0846, 0.2635] [0.2667, 0.4538, 0.6385] [0.632, 0.7778, 0.9154] [0.9154, 1, 1.1]
(4) Determine the fuzzy rules Fuzzy inference rules are usually derived from expert knowledge, fuzzy rules can be specified as follows: (a) If (total assets is small) or (total income is low) then (company scale is small) (b) If (total assets is medium) and (total income is not low) then (company scale is medium) (c) If (total assets is not small) and (total income is medium) then (company scale is medium) (d) If (total assets is large) and (total income is high) then (company scale is large) (e) If (total assets is large) and (total income is extra high) then (company scale is large)
A Credit Risk Rating Model Based on Fuzzy Neural Network
167
(f) If (total assets is extra large) and (total income is high) then (company scale is large) (g) If (total assets is extra large) and (total income is extra high) then (company scale is extra large) (5) Determine fuzzy computing, including fuzzy inference method, fuzzy method, and defuzzification methods (a) Fuzzification of input The fuzzification of input is similar to a corresponding query table or function calculation. The first step to establish a fuzzy inference system is to choose the system input variables and to determine these inputs which must be definite values based on corresponding membership function. The fuzzy inference system uses seven fuzzy inference rules, which regarded inputs as sets of different fuzzy senses. Before using these fuzzy rules, input variables should be fuzzified to obtain their corresponding fuzzy sets. (b) Synthesis operation of input fuzzy sets When the input is fuzzified, the level that it meets corresponding fuzzy rules is known. However, one condition of fuzzy rules is multi-input, so composition operation is required to consider and analyze comprehensively. The input could obtain a value to express its satisfactory level on the rules, and then the value is used in output function. Any reasonable and sound fuzzy synthesis method can be realized with “and” and “or”. In the realization of this model, “and” operation uses the min method, “or” uses the max method. (c) Fuzzy implication The input for calculation of fuzzy implication is from previous synthesis operation of input fuzzy sets, the single value obtained is the fuzzy set. Output is the conclusion fuzzy set derived based on fuzzy rules. Fuzzy implication is the expression problem of fuzzy rules. The MATLAB software contains two methods of calculation. This study selected the min method. (d) Synthesis of output Synthesis of output is the process to integrate the fuzzy sets output from all fuzzy rules. The method of synthesis is not related to the order as well as the synthesis order of rule does not affect the results. After tests, the max method is taken. (e) Defuzzification The input of defuzzification is used as fuzzy set, as well as the output is a value. After the fuzzy inference, the membership function for the scope of output variable is obtained, so it is necessary to carry out defuzzification to make the output a specific value. (6) Obtain the model output Model output is obtained through the training sample data. The input and output surface of fuzzy inference can clearly reflect the fuzzy rules. 4.4 Application of Fuzzy Neural Network in Corporate Credit Risk Rating (1) Model construction with fuzzy neural network Companies’ financial risk and scale ratings are obtained based on fuzzy theory and neural network. Companies’ fundamental factors are added in the final credit risk rating, such input is companies’ total score rated by the experts from quality, strength,
168
K.-J. Zhu, P.-C. Chen, and Y.-T. Chang
environment, credit status, critical incidents, and leader’s quality. And then, it is naturalized into a value among [0,1]. In this way, the rating model for corporate credit risk can be constructed based on fuzzy neural network. On the basis of Takagi-Sugeno inference rules, the model has three inputs, which are financial risk, fundamental risk and company scale. The model has one output, which is the default probability of companies. The default probability refers to the degree of likelihood that the borrower of a loan or debt will not be able to make the necessary scheduled repayments or to fulfill relevant obligations. Under normal circumstances, it means the one-year default probability. It is based on a large number of historical data and obtained with logistic regression analysis. Before loaning to the lender, commercial banks will decide the amount of loan based on the default probability. When conducting a rating, such output data should carried out training for the model, and expertise can be used for reference. The network architecture of fuzzy neural network model has three inputs, one output; the fuzzy layer contains 12 neurons, fuzzy rules layer has 60 rules, which constructs the fuzzy neural network model. (2) Model training Fuzzy toolbox of the MATLAB software provides a command function ANFIS, which obtains membership function parameters of fuzzy inference system through self-adaptive neural network learning. When training the fuzzy system model with training samples, ANFIS can construct fuzzy sets and fuzzy rules adaptively based on samples and relevant given parameters, and adjust the parameters of membership function. Modification in membership function curve of input variables before and after identification fully reflects the self-learning ability of this fuzzy neural network. According to the model, the training data is the mean square error curve, perf = 6.1421e-005, and it demonstrates a good training effect as shown in Fig. 2.
Fig. 2. Model Training
A Credit Risk Rating Model Based on Fuzzy Neural Network
169
(3) Model testing After completing model training, the test sample is applied into the trained ANFIS fuzzy neural network for corporate credit rating. There are 50 test samples which tested with the MATLAB software. The error dependence graph between the actual values and output values is shown in Fig. 3. The average error is 0.0094.
Fig. 3. Error Dependence Graph
4.5 Data Analysis and Assessment According to historical data of the bank, the companies in this study are divided into six levels, according to the default probability which are distributed as follows: 0 very small 0.008; 0.008
≦
≦
≦
≦
≦
≦
≦
Table 2. The Rating Classification of Output Values and Actual Values No 1
Output 0.0092
Actual 0.0132
0.6230
Rating quite small suspicious
2
No 26
Output 0.0075
0.6452
Rating quite small suspicious
27
0.0221
3 4
0.0023 0.6418
very small suspicious
0.0034 0.6484
very small suspicious
28 29
0.9465 0.0104
5
0.8294
loss
0.8237
loss
30
0.8342
Rating quite small quite small loss quite small loss
Actual 0.0064
Rating very small
0.0258
general
0.9484 0.0120
loss quite small loss
0.8367
170
K.-J. Zhu, P.-C. Chen, and Y.-T. Chang Table 2. (continued)
No 6
Output 0.0145
7 8 9
0.8561 0.0015 0.0093
10
0.0094
11 12 13 14
0.4683 0.1018 0.2365 0.9811
15 16 17 18
0.0449 0.0748 0.0052 0.0221
19 20
Rating quite small loss very small quite small quite small attention attention attention loss
Actual 0.0157
Rating quite small loss very small very small
No 31
Output 0.0698
Rating general
Actual 0.0309
Rating General
32 33 34
0.0073 0.1258 0.6674
very small attention suspicious
0.0032 0.1322 0.6633
very small attention suspicious
35
0.0514
general
0.0455
general
0.4600 0.0812 0.2729 0.9737
quite small attention attention attention loss
36 37 38 39
0.1892 0.0545 0.1284 0.0159
0.2452 0.0559 0.1325 0.0081
0.0300 0.0756 0.0035 0.0293
general attention very small general
40 41 42 43
0.5276 0.0776 0.8485 0.1475
attention general attention quite small suspicious general loss attention
0.9477 0.0037
general attention very small quite small loss very small
attention general attention quite small attention attention loss attention
0.9454 0.0010
loss very small
44 45
0.2510 0.0142
21 22
0.7986 0.0049
suspicious very small
0.7894 0.0057
46 47
23
0.0145
0.0161
24 25
0.8765 0.6412
quite small loss suspicious
suspicious quite small quite small loss suspicious
0.8781 0.0024 0.0078 0.0088
0.8721 0.6438
0.5436 0.0681 0.8346 0.1459
0.6802 0.7654
attention quite small suspicious suspicious
0.2204 0.0107 0.6708 0.8001
attention quite small suspicious suspicious
48
0.3505
attention
0.3807
attention
49 50
0.0431 0.1333
general attention
0.0467 0.1458
general attention
5 Conclusions Due to the complexity and high nonlinearity of credit risk, there is a wide deviation between traditional linear model and actual system. This study uses non-linear and generalization capacity based on fuzzy theory and neural network, establishes a credit risk rating model on a basis of fuzzy neural network, which can achieve risk rating and prediction. This model is based on the financial risk model and company scale rating, coupled with the consideration on companies’ fundamental factors, which makes the rating factors more comprehensive, the rating results more scientific and accurate. When constructing the financial risk model, due to neural network’s self-learning, self-adaptive capacity, parallel processing capability and strong fault tolerance, it can obtain more accurate non-linear outputs from various financial indexes. Fuzzy theory expresses knowledge naturally and easily, it is able to process uncertain information, which expresses knowledge with fuzzy and natural language and is easy to use expertise, so it is appropriate to rate the company scale. The combination model of fuzzy theory and neural network provides a good foundation for credit risk rating with fuzzy neural network, making this model with fewer parameters, faster learning, as well as less training samples.
A Credit Risk Rating Model Based on Fuzzy Neural Network
171
This study proposes the combined model based on fuzzy theory and neural network. This model fully utilizes the advantages of both methods, and it rates companies’ credit with a more comprehensive rating index system. On the one hand, it makes the rating index more comprehensive; on the other hand, it makes the model as simple and reliable as possible. In addition, the powerful MATLAB software is used as the development tool in this study, which completes the leap from theory to practice. The results of this study demonstrate an effective model for credit risk rating. This study has combined with practice and theory, verified the use of theoretical models, so as to facilitate the future inferences and follow-up research. Future studies should focus on improving the model proposed in this study, and expanding the sample set to improve generalization ability. It is intended for future researchers to learn and research for a more comprehensive rating index system, as well as to improve the algorithm for better training speed and accuracy.
References [1] Forest, L., Aguais, S.: Implementing a comprehensive credit risk management system: the case study of Hanvit Bank. Hanvit Bank 16(2), 16–30 (2001) [2] Eklund, T., Larsen, K., Berhardsen., E.: Model for analysis credit risk in the enterprise sector. Norges Bank 72(3), 99–106 (2001) [3] Kosko, B.: A Dynamical Approach to Machine Intelligence: Neural Networks and Fuzzy Systems. Prentice-Hall, Englewood Cilffs (1992) [4] Zadeh, L.A.: Fuzzy sets. Information and Control 8, 338–353 (1965) [5] Wang, B.K., Bodnovich, T., Selvi, Y.: Neural network applications in business: A review and analysis of the literature (1988-1995). Decision Support Systems 19, 301–320 (1997) [6] Gomez, F., Schmidhuber, J., Miikkulainen, R.: Accelerated neural evolution through cooperatively coevolved synapses. The Journal of Machine Learning Research 9, 937–965 (2008) [7] Jackson, P., Perraudin, W.: Regulatory implications of credit risk modeling. Journal of Banking and Finance 24(1), 1–24 (2000) [8] Zhang, H., Tam, C.M., Shi, J.: Application of fuzzy logic to simulation for construction operations. Journal of Computing in Civil Engineering 17(1), 38–45 (2003) [9] Haykin, S.: Neural networks: a comprehensive foundation, 2nd edn. Prentice Hall, Upper Saddle River (1999) [10] Kuo, R.: A decision support system for stock market through integration of fuzzy neural network and fuzzy delphi. Applied Artificial Intelligence 12(6), 501–520 (1998) [11] Feng, Y., Lei, Y., Li, Y.: Research on economic simulation based on neural network. In: Proceedings of 2001 International Conference on Management Science & Engineering, Harbin, P. R. China, August 2001, pp. 18–20 (2001) [12] Kuo, R., Wu, P., Wang, C.: An intelligent sales forecasting system through integration of artificial neural networks and fuzzy neural networks with fuzzy weight elimination. Neural Networks 15(7), 909–925 (2002) [13] Leng, G., McGinnity, T., Prasad, G.: An approach for on-line extraction of fuzzy rules using a self-organizing fuzzy neural network. Fuzzy Sets and Systems 150, 211–243 (2005)
Interval-Valued Fuzzy Control Wenyi Zeng1, and Jiayin Wang2 1 College of Information Science and Technology, Beijing Normal University, Beijing, 100875, P.R. China [email protected] 2 School of Mathematical Sciences, Beijing Normal University, Beijing, 100875, P.R. China
Abstract. In this paper, we introduce the concept of interval-valued fuzzy control. Based on the idea of the interpolation mechanism of fuzzy control, we propose the inference algorithm of interval-valued fuzzy inference and mathematical model of interval-valued fuzzy control, investigate the interpolation mechanism of interval-valued fuzzy control. Finally, we use a simulation experiment of interval-valued fuzzy control to illustrate our proposed algorithm reasonable. Keywords: Interval-valued fuzzy set, interpolation mechanism, intervalvalued fuzzy inference, interval-valued fuzzy control.
1
Introduction
Since the fuzzy set was introduced by Zadeh [16], many new approaches and theories treating imprecision and uncertainty have been proposed. Some of these theories, such as the intuitionistic fuzzy sets theory pioneered by Atanassove [1, 2], the generalized theory of uncertainty (GTU) introduced by Zadeh [20], are extensions of the classic fuzzy set theory, and have allowed people to deal with uncertainty and information in a much broader perspective. Another wellknown generalization of an ordinary fuzzy set is the interval-valued fuzzy set, which was first introduced by Zadeh [17, 18, 19]. Since this was introduced, many researchers have investigated this topic and have established some meaningful conclusions. For example, Deschrijver [8] investigated the arithmetic operators of the interval-valued fuzzy set theory, and Zeng [21, 23] investigated decomposition theorem and representation theorem of the interval-valued fuzzy set, respectively. Moreover, some researchers has pointed out that there is a strong connection between Atanassov’s intuitionistic fuzzy sets and the interval-valued fuzzy sets. For more details, readers can refer to [7]. Fuzzy inference is an important topic in fuzzy set theory. Many researchers have investigated it from different points of view and proposed many different mathematical models of fuzzy inference. These different fuzzy inference models
This work is supported by grants from the National Natural Science Foundation of China (10971243). Corresponding author.
Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 173–183. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com
174
W. Zeng and J. Wang
and inference algorithms have been applied rapidly in many areas such as fuzzy control and pattern recognition according to their feature. For example, Wang [14] proposed true-value flow inference theory and inference algorithm, Zeng [22] proposed inner product truth-valued flow inference algorithm, Wu [15] investigated the principle and method of fuzzy inference. Applying fuzzy inference into fuzzy control, scholars have done a lot of meaningful works. For example, Li [11] proposed the interpolation mechanism of fuzzy control and applied fuzzy control in the research of inverted pendulums. Li [12, 13] investigated adaptive fuzzy controllers based on variable universe and the relationship between fuzzy controllers and PID controllers, respectively. Since the interval-valued fuzzy set was introduced, researchers have started to investigate fuzzy inference of interval-valued fuzzy sets. Some researchers have proposed some different approximate reasoning methods of interval-valued fuzzy sets. For example, Gorzalczany [9, 10] proposed fuzzy inference methods of interval-valued fuzzy sets and discussed its some properties, respectively. Bustince [3] proposed approximate reasoning models of interval-valued fuzzy sets based on the interval-valued fuzzy relations. Chen [4, 5] investigated the bidirectional approximate reasoning methods of interval-valued fuzzy sets based on the similarity measure of interval-valued fuzzy sets and direction index of matching function. Chun [6] applied the similarity-based bidirectional approximate reasoning method in decision-making systems. In this paper, we introduce the concept of interval-valued fuzzy control, propose the inference algorithm of interval-valued fuzzy inference and the mathematical model of interval-valued fuzzy control, investigate the interpolation mechanism of interval-valued fuzzy control. Finally, we use a simulation experiment of interval-valued fuzzy control to illustrate our proposed algorithm reasonable. The organization of our work is as follows. In section 2, some basic notions of interval-valued fuzzy set are reviewed. In section 3, we propose the mathematical model of interval-valued fuzzy control and investigate the interpolation mechanism of interval-valued fuzzy control. In section 4, we use a simulation experiment of interval-valued fuzzy control to illustrate our proposed algorithm reasonable. The conclusion is given in the last section.
2
Some Notions
Throughout this paper, we use X and Y to denote the input discourse set and the output discourse set, respectively, IVFSs(X) and IVFSs(Y ) stand for the set of all interval-valued fuzzy subsets in X and Y , respectively. A and B express the interval-valued fuzzy set in X and Y , respectively, the operation “c” stands for the complement operation of interval-valued fuzzy set or fuzzy set, ∅ stands for the empty set. Let L([0, 1]) be the set of all closed subintervals of the interval [0, 1]. Especially, for an arbitrary element a ∈ [0, 1], we assume that a is the same as [a, a], namely, a = [a, a]. Then, according to Zadeh’s extension principle [16], − + for any a = [a , a ], b = [b− , b+ ] ∈ L([0, 1]), we can popularize some operations such as , , and c to L([0, 1]), and have a b = [a− b− , a+ b+ ],
Interval-Valued Fuzzy Control
a
t∈W
b = [a− at = [
b − , a+
t∈W
a− t ,
b+ ], ac = [1 − a+ , 1 − a− ],
t∈W
at = [
a− t ,
t∈W
175
a+ t ] and
t∈W
a+ t ], where W denotes an arbitrary index set. Further-
t∈W
more, we have a = b ⇐⇒ a− = b− , a+ = b+ , a b ⇐⇒ a− b− , a+ b+ and a < b ⇐⇒ a b and a = b; then there exists a minimal element 0 = [0, 0] and a maximal element 1 = [1, 1] in L([0, 1]). We call a mapping, A : X −→ L([0, 1]) an interval-valued fuzzy set in X. For every A ∈ IVFSs(X) and x ∈ X, then A(x) = [A− (x), A+ (x)] is the degree of membership of an element x to the interval-valued fuzzy set A. Thus, fuzzy sets A− : X → [0, 1] and A+ : X → [0, 1] are called low and upper fuzzy sets of the interval-valued fuzzy set A, respectively. For simplicity, we denote A = [A− , A+ ]. F (X) and P(X) stand for the set of all fuzzy sets and crisp sets in X, respectively. Specially, if X = {x1 , x2 , . . . , xn }, then A(xi ) = [ai1 , ai2 ], for every xi ∈ X, i = 1, 2, · · · , n. Thus, we have A = {(x1 , [a11 , a12 ]), (x2 , [a21 , a22 ]), . . . , (xn , [an1 , an2 ])} where [ai1 , ai2 ](0 ≤ ai1 ≤ ai2 ≤ 1, i = 1, 2, · · · , n) is the degree of membership of an element xi to the interval-valued fuzzy set A. If A, B ∈IVFSs(X), then the following operations can be found in Zeng [21]. A ⊆ B if and only if ∀x ∈ X, A− (x) ≤ B − (x) and A+ (x) ≤ B + (x), A = B if and only if ∀x ∈ X, A− (x) = B − (x) and A+ (x) = B + (x), (A)c = [(A+ )c , (A− )c ], (A ∩ B)(x) = [A− (x) ∧ B − (x), A+ (x) ∧ B + (x)], ∀x ∈ X, (A ∪ B)(x) = [A− (x) ∨ B − (x), A+ (x) ∨ B + (x)], ∀x ∈ X.
3
Interpolation Mechanism of Interval-Valued Fuzzy Control
Recently, Li [11, 12, 13] has pointed out that some common fuzzy inference algorithms in fuzzy control can be attributed to some kind of interpolation method, and has proved that fuzzy control algorithm based on the implication operator R(a, b) = 0 ∨ (a + b − 1) can be attributed some kind of interpolation system. To introduce the nomenclatures, we will consider a reasonably simple intervalvalued fuzzy control system with one-input and one-output. Let X = [a, b] be the universe of input variable x, and Y = [c, d] be the universe of output variable y. Denoting A = {Ai }(1≤i≤n) , B = {B i }(1≤i≤n) , Ai ∈ IVFSs(X), B i ∈ IVFSs(Y ), and A, B are linguistic variables, thus, we can form the fuzzy inference rules as follows: if x is Ai then y is B i where i = 1, 2, · · · , n; x ∈ X, y ∈ Y .
176
W. Zeng and J. Wang
For a given interval-valued fuzzy set A , according to Zadeh’s compositional rule of inference, then we have the mathematical model of interval-valued fuzzy inference in the following. Model n
B =
A ◦ Ri =
i=1
n
A ◦ (Ai → B i )
i=1
where Ri (x, y) = (Ai → B i )(x, y) = [Ri − (x, y), Ri + (x, y)], Ri − (x, y) = max(0, Ai − (x) + Bi − (y) − 1), Ri + (x, y) = Ai + (x) ∧ Bi + (y), i = 1, 2, · · · , n. Hence, the conclusion of interval-valued fuzzy inference can be determined as follow. − + B = [B , B ] −
where B (y) =
n
n
(A− (x) ∧ Ri − (x, y)), B (y) = +
x∈X i=1
(A+ (x) ∧
x∈X i=1
Ri + (x, y)). In general, the input to the fuzzy logic system is crisp or real quantity, thus, it must be fuzzified. We will adopt the following most frequently used fuzzification system for every x = [x , x ], 1, x = x A (x) = 0, x = x
Then, we will obtain an interval-valued fuzzy set A , A = [A− , A+ ], where A − = A + = A . Using the above fuzzification system, we have B = [B − , B + ], where n n − + Ri − (x , y), B (y) = Ri + (x , y), To obtain a crisp quantity, B (y) = i=1
i=1
the interval-valued fuzzy set B will be defuzzified by the use of the following centroid method: − + yB (y)dy yB (y)dy + − Y Y , y = y = + − B (y)dy B (y)dy Y
Y
where
n
−
B (y) =
−
−
(max(0, Ai (x ) + Bi (y) − 1))
i=1
B (y) = +
n
(Ai + (x ) ∧ Bi + (y))
i=1
Thus, for the interval-valued fuzzy control system with one-input and oneoutput, we find that the interval-valued fuzzy control system can be transformed
Interval-Valued Fuzzy Control
177
two kinds of general fuzzy control systems. Namely, they are two different kinds of fuzzy inference rules, respectively. These two kinds of fuzzy inference rules are illustrated as follows. ifxisAi − thenyisBi − , i = 1, 2, · · · , n and ifxisAi + thenyisBi + , i = 1, 2, · · · , n
4
Simulation Experiment of Interval-Valued Fuzzy Control
For convenience, we use the mathematical model of controlled object, x˙ + x = u(t), to do the simulation experiment, where x(0) = −1, u(t) is out action. The control target is to trace pulse signal, 1, t > 0, r(t) = 0, else For u(t) ≡ 0, we can obtain the response curve for being controlled object, then the response curve for being controlled object is listed as Fig. 1. t
In our experiment, we use ID controller, u(t) = KI
e(τ )dτ , where KI =
0
10 is proportion coefficient, e(t) = r(t) − x(t) is system error, the response curve 3 for the system is listed as Fig. 2. 0
−0.1
−0.2
Response curve
−0.3
−0.4
−0.5
−0.6
−0.7
−0.8
−0.9
−1
0
1
2
3
4
5 Time (s)
6
7
8
9
Fig. 1. The response curve for being controlled object
10
178
W. Zeng and J. Wang
2
1.5
Response curve
1
0.5
0
−0.5
−1
0
1
2
3
4
5 Time (s)
6
7
8
9
10
Fig. 2. The response curve for the system on ID
In the following, we give two kinds of general membership functions, their membership functions are listed as follows and their membership function curves are listed as Fig. 3. For k > 1, ⎧ k x−a ⎪ ⎪ , x ∈ [a b], ⎪ ⎪ ⎨ b−a k
w minus mf (x, [a b c]) = x−c ⎪ , x ∈ [b c], ⎪ ⎪ ⎪ ⎩ b−c 0, else ⎧x−a ⎪ ⎪ ⎨ b − a , x ∈ [a b], trimf (x, [a b c]) = x − c , x ∈ [b c], ⎪ ⎪ ⎩ b−c 0, else In the design of interval-valued fuzzy controller, we choose input variable as t error integral, x(t) = e(τ )dτ , output variable as control quantity u(t), where 0
input variable universe X = [−3, 3], output variable universe U = [−10, 10]. Fuzzy partition of input variable universe is listed as Fig. 4. We use five intervalvalued fuzzy sets, A1 , A2 , A3 , A4 , A5 , to denote such linguistic variables as “NL”, “NM”, “ZR”, “PM”, “PL”, respectively. These membership functions are listed as follows, respectively. + A1 (x) = A− 1 (x), A1 (x) , + A− 1 (x) = w minus mf (x, [−3 − 3 − 1.5]), A1 (x) = trimf (x, [−3 − 3 − 1.5]), + A2 (x) = A− 2 (x), A2 (x) ,
Interval-Valued Fuzzy Control
179
1
Membership Function
k=1 k>1
0.5
0
a
b x
c
Fig. 3. Membership function curves + A− 2 (x) = w minus mf (x, [−3 − 1.5 0]), A2 (x) = trimf (x, [−3 − 1.5 0]), − A3 (x) = A3 (x), A+ 3 (x) , + A− 3 (x) = w minus mf (x, [−1.5 0 1.5]), A3 (x) = trimf (x, [−1.5 0 1.5]), − A4 (x) = A4 (x), A+ 4 (x) , + A− 4 (x) = w minus mf (x, [0 1.5 3]), A4 (x) = trimf (x, [0 1.5 3]), − A5 (x) = A5 (x), A+ 5 (x) , + A− 5 (x) = w minus mf (x, [1.5 1.5 3]), A5 (x) = trimf (x, [1.5 1.5 3]).
Similarly, for the fuzzy partition of output universe, we use five intervalvalued fuzzy sets, B 1 , B 2 , B 3 , B 4 , B 5 , to denote such linguistic variables as “NL”, “NM”, “ZR”, “PM”, “PL”, respectively. Their membership functions are listed as follows, respectively. B 1 (u) = B1− (u), B1+ (u) , B1− (u) = w minus mf (u, [−10 −10 −5]), B1+ (u) = trimf (u, [−10 −10 −5]), B 2 (u) = B2− (u), B2+ (u) , B2− (u) = w minus mf (u, [−10 − 5 0]), B2+ (u) = trimf (u, [−10 − 5 0]), B 3 (u) = B3− (u), B3+ (u) , B3− (u) = w minus mf (u, [−5 0 5]), B3+ (u) = trimf (u, [−5 0 5]), B 4 (u) = B4− (u), B4+ (u) , B4− (u) = w minus mf (u, [0 5 10]), B4+ (u) = trimf (u, [0 5 10]), B 5 (u) = B5− (u), B5+ (u) , B5− (u) = w minus mf (u, [5 10 10]), B5+ (u) = trimf (u, [5 10 10]).
180
W. Zeng and J. Wang
Membership Function
1
0.66
k=1.5
0.33
0 −3
−1.5
0 x
1.5
3
Fig. 4. Psedu-fuzzy partition of A− and A+ on input universe X
Then, based on five kinds of fuzzy inference rules in the following, we can construct the rule base for interval-valued fuzzy inference. Rule 1: if x(t) is A1 then u(t) is B 1 ; Rule 2: if x(t) is A2 then u(t) is B 2 ; Rule 3: if x(t) is A3 then u(t) is B 3 ; Rule 4: if x(t) is A4 then u(t) is B 4 ; Rule 5: if x(t) is A5 then u(t) is B 5 . According to the interpolation mechanism of interval-valued fuzzy control, we can obtain the control function curve for the interval-valued fuzzy inference system: u = F (x) = F − (x), F + (x) where F − (x) =
5 i=1
+ A− i (x)yi and F (x) =
5 i=1
A+ i (x)yi , yi are the peak points of
fuzzy sets both Bi− and Bi+ on output universe. For the convenience, we order k = 1.5, then we can obtain their control function curve on interval-valued fuzzy controller which is listed as Fig. 5, and the response curve for the system on interval-valued fuzzy controller which is listed as Fig. 6.
Interval-Valued Fuzzy Control
181
10
u
5
0
u− u+ −5
−10 −3
−1.5
0 x
1.5
3
Fig. 5. Control function curve on interval-valued fuzzy controller 2
u− u+ 1.5
Response curve
1
0.5
0
−0.5
−1
0
1
2
3
4
5 Time (s)
6
7
8
9
10
Fig. 6. The response curve for the system
5
Conclusion
Considering the importance of fuzzy inference and fuzzy control, in this paper, we introduce the concept of interval-valued fuzzy control. Based on the idea on the interpolation mechanism of fuzzy control proposed by Li [11], we propose the mathematical model of interval-valued fuzzy control and investigate the interpolation mechanism of interval-valued fuzzy control. Firstly, we investigate the mathematical model of interval-valued fuzzy inference and obtain an interval-valued fuzzy set as the conclusion of the interval-valued fuzzy inference. Secondly, we use an interval to denote the response value of the interval-valued
182
W. Zeng and J. Wang
fuzzy control, and obtain an interval belt of the response curve of interval-valued fuzzy control, not as a single value and a single curve. Finally, we use a simulation experiment of interval-valued fuzzy control to illustrate our proposed algorithm reasonable. We believe that the design of interval-valued fuzzy controller will induce more and more fuzzy technology. These technology will be extensively applied in many fields such as pattern recognition, image processing, approximate reasoning, fuzzy control, and so on.
Acknowledgements This work is supported by grants from the National Natural Science Foundation of China (10971243). And the authors are grateful to the anonymous referees for their great comments and valuable suggestions which helped us improve this paper.
References [1] Atanassov, K.: Intuitionistic fuzzy sets. Fuzzy Sets and Systems 20, 87–96 (1986) [2] Atanassov, K.: Intuitionistic Fuzzy Sets: Theory and Applications. PhysicaVerlag, Heidelberg (1999) [3] Bustince, H., Burillo, P.: Mathematical analysis of interval-valued fuzzy relations: Application to approximate reasoning. Fuzzy Sets and Systems 113, 205–219 (2000) [4] Chen, S.M., Hsiao, W.H., Jong, W.T.: Bidirectional approximate reasoning based on interval-valued fuzzy sets. Fuzzy Sets and Systems 91, 339–353 (1997) [5] Chen, S.M., Hsiao, W.H.: Bidirectional approximate reasoning for rule-based systems using interval-valued fuzzy sets. Fuzzy Sets and Systems 113, 185–203 (2000) [6] Chun, M.G.: A similarity-based bidirectional approximate reasoning method for decision-making systems. Fuzzy Sets and Systems 117, 269–278 (2001) [7] Deschrijver, G., Kerre, E.E.: On the relationship between some extensions of fuzzy set theory. Fuzzy Sets and Systems 133, 227–235 (2003) [8] Deschrijver, G.: Arithmetic operators in interval-valued fuzzy set theory. Information Sciences 177, 2906–2924 (2007) [9] Gorzalczany, M.B.: A method of inference in approximate reasoning based on interval-valued fuzzy sets. Fuzzy Sets and Systems 21, 1–17 (1987) [10] Gorzalczany, M.B.: An interval-valued fuzzy inference method -some basic properties. Fuzzy Sets and Systems 31, 243–251 (1989) [11] Li, H.X.: Interpolation mechanism of fuzzy control. Science in China, Ser. E 41(3), 312–320 (1998) [12] Li, H.X.: Adaptive fuzzy controllers based on variable universe. Science in China, Ser. E 42(1), 10–20 (1999) [13] Li, H.X.: Relationship between fuzzy controllers and PID controllers. Science in China, Ser. E 42(2), 215–224 (1999) [14] Wang, P.Z., et al.: Mathematical theory of truth-valued flow inference. Fuzzy Sets and Systems 72, 221–238 (1995) [15] Wu, W.M.: Principle and Method of Fuzzy Inference. Guizhou Science and technology Press, Guizhou (1994) (in Chinese)
Interval-Valued Fuzzy Control
183
[16] Zadeh, L.A.: Fuzzy sets, Inform. Control 8, 338–353 (1965) [17] Zadeh, L.A.: The concept of a linguistic variable and its application to approximate reasoning(I). Inform. Sci. 8, 199–249 (1975) [18] Zadeh, L.A.: The concept of a linguistic variable and its application to approximate reasoning (II) 8, 301–357 (1975) [19] Zadeh, L.A.: The concept of a linguistic variable and its application to approximate reasoning(III). Inform. Sci. 9, 43–80 (1975) [20] Zadeh, L.A.: Toward a generalized theory of uncertainty(GTU)—-an outline. Information Sciences 172, 1–40 (2005) [21] Zeng, W.Y., Shi, Y.: Note on interval-valued fuzzy set. In: Pal, N.R., Kasabov, N., Mudi, R.K., Pal, S., Parui, S.K. (eds.) ICONIP 2004. LNCS, vol. 3316, pp. 20–25. Springer, Heidelberg (2004) [22] Zeng, W.Y., Li, H.X.: Inner product truth-valued flow inference, International Journal of Uncertainty. Fuzziness and Knowledge-Based Systems 13(6), 601–612 (2005) [23] Zeng, W.Y., Shi, Y., Li, H.X.: Representation theorem of interval-valued fuzzy set, Internat. J. Uncer. Fuzzi. Knowloedge-Based Systems 14, 259–269 (2006)
Research on Fuzzy Preference Relations-Based MAS for Decision Method Weijin Jiang1 and Qing Jiang2 1
School of Computer and Electronic Engineering, Hunan University of Commerce, Changsha 410205, China [email protected] 2 School of Computer, South University of Technology, Guangzhou 510092, PRC Abstract. The use of multiple intelligent agents in the decision support systems has become common. However, there are many questions in this field, such as the consensus of the agents and the method on choosing the best decision. In order to solve these problems, this paper provides a method, which uses the fuzzy aggregation operator to aggregate the opinions of the agents, ranks the alterative with the fuzzy preference relations, and then chooses the best alternative. This approach can be widely used in many fields. Keywords: Agent, Fuzzy preference relations, Fuzzy satisfaction function.
1
Introduction
The use of multiple intelligent agents in the decision support systems has become common. Distributional artificial intelligence technology took one kind of establishment in the distributional control structure foundation intelligent method integration method, by its unique superiority by the people was being gradually known and takes [1-5]. But also brings some questions along with it. First, because person's thought has the inherent fuzziness, the person is accustomed to uses the natural language description correlation concept, when the agent provides a quantity, the explicit policy-making result [6-8], the person very difficultly makes to it conforms with the intuition the judgment, Moreover at very many times, the agent also is according to incompletely, the indefinite information makes the decisionmaking, in this kind of situation [9], the agent very is also difficult to provide a precise policy-making result, therefore in this article the agent decision-making result indicated with the fuzzy collection [10], causes its output is not a simple numeral, but is a subordination degree, namely a fuzzy concept. The second question lies in: Because various agent looked the question the angle quite same not less than, can obtain also is different, like this very possibly causes the decision-making which some main bodies makes to be mutually contradictory. Then, how obtains to conform to the objective policy-making result then to become a key question [11]. Some literature proposed through neglects achieves the result with the majority main body policy-making result contradictory main body policy-making result method the uniformity. But in very many situations this method is not suitable. The third question is: How obtains finally the reasonable policy-making plan. In the article agent use fuzzy collection expresses the policy-making result [12-15], then becomes a difficulty to the fuzzy number arrangement. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 185–192. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
186
W. Jiang and Q. Jiang
Above in order to solve the problem, the author proposed one kind of fuzzy policymaking result polymerization algorithm, and through fuzzy relates by chance to the polymerization result carries on the arrangement [16], obtains finally the reasonable policy-making plan. This article content arrangement as follows: The second part of introduction fuzzy intelligence main body and its the polymerization algorithm, the third part proposed based on fuzzy number arrangement method which fuzzy relates by chance, the fourth part of simulation result, finally summarizes the full text.
2 Fuzzy Agent Since 1965 Zadeh proposed the fuzzy collection theory, the fuzzy collection obtained the widespread application in many domains. The fuzzy collection distinguishes to the classics gathers mainly lies in a subordination function. The classics gather discuss use two values functions to take it subordinates a function:
⎧1x∈A ⎩0x∈A
μA = ⎨
A fuzzy collection subordination function finally will map [0, 1] on, namely: . The such fuzzy collection may express some fuzzy concepts well, for example: Young, middle age, concept and so on old age. Thus even more conforms to person's thinking mode. The fuzzy collection subordination function type has very many kinds, quite commonly used has the triangle to subordinate a function, a trapezoid subordination function, a subordination function. This article supposition already knew policy-making in the policy-making question plan collection S= {S1, S2... Sm}, various agent embarks from own angle to each policy-making plan puts forward the proposal, then the so-called fuzzy agent decision-making is refers, various agent does not produce the accurate judgement, which plan uses, but is the use fuzzy collection theory produces the fuzzy judgement to various policy-making plan. In fuzzy process, subordination function selection count for much. This article is the simplification operation, a selection triangle subordination function. A typical triangle subordination function like figure 1 shows.
μ 1
a
c
b
x
Fig. 1. Triangular fuzzy numbers
In order to coordinate various agent the work, this article first the supposition various intelligences main body all can real-time produce the policy-making result.
Research on Fuzzy Preference Relations-Based MAS for Decision Method
187
The author proposed the method is: Various main bodies after analyze the data which itself obtains, produces own to the corresponding policy-making plan policy-making opinion, the natural policy-making opinion produces by the fuzzy collection form, in this foundation, the use fuzzy polymerization operator gets together each agent to the specific policy-making plan policy-making opinion, finally, carries on these polymerization results the arrangement, like this may obtain the best policy-making plan, but this plan also is the multi- agent unification policy-making result. Then the question key lies in seeks an appropriate polymerization operator. Generally says, fuzzily gets together the operator is a solid mapping function:
∫ : [0,1]
→ [0,1] [8]
∫
below has the nature:
∫ ( 0)
= 0 ∫ (1) = 1 , Among
them, the parameter 0, 1 is the n Uygur vector. Many literature proposed the corresponding polymerization operator, for example, Yanger proposed OWA [6] (Ordered Weighted Averaging) the method, this operator use language classifier has carried on the fuzzy number polymerization; D. DuBois [7] and so on proposed in Multicnieria decision uses the Choquet integral method to take the polymerization operator, this operator use fuzzy measure, the use fuzzy Choquet integral method carries on the polymerization computation. These methods have obtained the success in many domains, but they all have a very big flaw, namely computation complex is very high, like this in the timely request quite high situation, the military situation, cannot answer the purpose in particular. Such author considered uses the simple weighing method, concrete method as follows: (1) Acts according to various agent the important degree difference, entrusts with a power for each agent ωi ; (2) Carries on the power value normalized processing: '
ωi =
ωi'
∑ω i =1
;
, i=1…n
n
' i
(3) Supposes the ith agent to the kth policy-making plan policy-making result ~
is R ki Then all agents to the kth policy-making plan policy-making polymerization result is: ~
R ki =
)
ck ,
~
~ ~ ⎤ 1 ⎡ n ( 1 R ) 1 . Among for parameter, R + μω − μ ki = ak, bk, ki ∏ i ⎥ μ ⎢⎣ i = 1 ⎦
(
(
)
R ki = aki, bki, cki .
3 The Model of Multi-agent Decision-Making In obtains the main body after all policy-making plan policy-making result, next step must choose the best policy-making plan, takes the final policy-making result. Then
188
W. Jiang and Q. Jiang
must carry on the arrangement to various policy-making plan policy-making result. The fuzzy number arrangement does not look like fine arrangement such simple, is direct. Has very many literature to the fuzzy number arrangement to propose many methods [2-3], these methods mostly concentrate to two aspects: (1) defines a arrangement function, it will blur the number to map in the real axis, then will carry on the arrangement; (2) the second method is establishes one kind of fuzzy function, the use fuzzy by chance function carries on the arrangement by chance. Revolves many fuzzy numbers arrangements method which these two aspects proposed, although has all obtained the success in the literature, but does not have one method all to obtain satisfaction in each kind of situation the arrangement result. The main question lies in the arrangement result which some algorithms produces is violates the intuition [4], and has some methods to have the certain limit to the fuzzy number shape, For example Nakamura and so on proposes based on the arrangement method which fuzzy relates by chance[3]. Compares says, person and so on KeonMyung Lee proposed the use degree of satisfaction function carries on the fuzzy number arrangement the algorithm first is lets the person in the intuition accept, and it has not requested, the result to the fuzzy number shape which obtains also is satisfying, therefore this article mainly uses this method to carry on the fuzzy number arrangement. Under separately introduces several main concepts. The fuzzy degree of satisfaction function is one kind of fuzzy measure, it produced two fuzzy numbers to conform to some comparison relations degree, here comparison relations was the arithmetic comparison relations which in the usual significance said{< = >} Under produces the correlation definition.
,, 。
Defines 1: Fuzzy collection Aγ-resolution collection D gamma (A), definition as follows:
D γ (A) = {x | x = ηγ , x ∈ Supp(A ), n ∈ N} Among
them,
Supp
(A)
is
;
a fuzzy collection A collection, N is the integer collection. Is worthy of looking
Supp(A) = {x | μ A (x) > 0} at from its definition, D γ (A) will continual real number field separate.
Defines 2: Fuzzy degree of satisfactionis ϕ , Supposes A, B is two fuzzy numbers,
ϕ(ARB) )expresses A, B satisfies relates R the degree, Obviously has: ϕ(ARB) ∈ [0,1] . Here about R is{< = >} ϕ (ARB) =1 expresses A, B completely satisfies relates R, On the contrary, ϕ ARB =0 expresses A, B completely does not satisfy relates R.
Then
,, 。
(
)
Defines 3: Fuzzy degree of satisfaction function Sγ
(A
j
Research on Fuzzy Preference Relations-Based MAS for Decision Method
S γ (Ai < Aj) =
j
min(y ~ γ )
y= j
x =d
∑ ∑μ I
Ai
x =i y= j
( x ) ⊗ μ A i ( y) (1)
J
∑∑μ
189
Ai
( x ) ⊗ μ A i ( y)
Among them, Imin=minDγ (Ai), Imin=maxDγ (Ai), Jmin=minDγ (Ai), Imax=maxDγ (Ai), ⊗ for T- nom operator, in article, Chooses min operator which Zadeh proposed. Similar may define: Defines 4: Fuzzy degree of satisfaction function Sγ j
min(y ~ γ )
y= j I
x=d J
∑ ∑μ
S γ (Ai > Aj) =
∑∑μ x =i y = j
S γ (Ai = Aj) =
x =d J
∑∑ μ
(
x =i y = j
j
( x ) ⊗ μ A i ( y) (2)
Ai
j min(y ~ γ )
y= j I
i
Ai
( x ) ⊗ μ A i ( y)
Defines 5: Fuzzy degree of satisfaction function Sγ
∑ ∑μ
(A >A ),defines as follows:
Ai
(A =A ),defines as follows: i
j
( x ) ⊗ μ A i ( y) (3)
Ai
( x ) ⊗ μ A i ( y)
) (
) (
)
Defines by above obviously has: Sγ AiAj =1. In this section, will define in the above fuzzy degree of satisfaction function foundation Fuzzy preference relations. The supposition various intelligences main body to various policy-making plan policy-making opinion polymerization result is: A={A1 A2 …An} Ai is the agent to the policy-making plan i decision-making opinion polymerization result, n is the policy-making plan integer which mutually competes. Then fuzzy relates the definition is by chance:
, ,
,
Defines 6: About fuzzy degree of satisfaction function Sγ Fuzzy preference relations Rs is:
R s ( Ai , A j ) = S γ ( Ai > A j ) +
Rs : A × A → [0,1],
Fuzzily by chance relates satisfies:
(1)R (A ,A )+R R (A ,A )=1, ∀A , A (2)R (A ,A )=0.5, ∀A ∈ A 。 s
i
j
s
i
j
s
i
j
i
j
∈ A
1 S γ ( Ai = A j ) (4) 2
;
i
By definition which fuzzy relates by chance below, may obtain the conclusion. As shown in table 1:
190
W. Jiang and Q. Jiang Table 1. Conclusion
Rs (Ai, Aj) value scope
Conclusion
1
The plan i completely surpasses plan j
[0.5, 1]
The plan i surpasses plan j
0.5
The plan i and plan j is equal
[0, 0.5]
Plan j surpasses the plan i
0
Plan j completely surpasses the plan i
After had fuzzily has related by chance, obviously was allowed through fuzzy to relate by chance to polymerization result A carries on the arrangement, obtained the best policy-making plan. The literature [4] center has produced the sort algorithm, but considered the question the particularity, namely this article main goal lies in found the best policy-making plan, but certainly not nice other plans fit and unfit quality orders. Such literature [4] center produces the sort algorithm appeared too complex; thereupon here proposed simple sought the most superior plan the algorithm. Algorithm 1 Seeks most superior policy-making plan. Step1: Input intelligence main body policy-making polymerization result A, as well as parameter γ; Step2: Calculates fuzzy degree of satisfaction function Sγ, and fuzzy relates Rs by chance; Step3: To all Ai, Aj A calculates O (Ai) = min R i ( A i , A j ) ;
∈
Step4
: N = ⎧⎨⎩A
i
i= j
| 0(Ai) = max 0(A j )⎫⎬, alternative = i ; A J ∉A ⎭
Step5: Returns to most superior policy-making plan alternative. Algorithm conclusion.
4 Simulation and Discussion In order to confirm the algorithm the accuracy, a this article structure simulation environment, the supposition in a policy-making question, altogether had 6 agent, its important degree respectively is: (0.8, 0.5, 0.4, 0.6, 0.2, 0.7), After the normalized weight is: (0.25, 0.16, 0.13, 0.19, 0.06, 0.21), the policy-making marquis chooses the plan altogether to have 4, namely S= {S1, S2, S3, S4}, after the stroke operation, the decision system chooses the final fuzzy appraisal to various marquises which the plan obtains for (A1, A2, A3, A4), Respectively is: A1= (0.2, 0.3, 0.55), A2= (0.12, 0.48, 0.9), A3= (0.45, 0.6, 0.65), A4= (0.71, 0.9, 1.0), its result like figure 2 shows. Obtains fuzzily through the fuzzy degree of satisfaction functional calculus relates like table 2 to show by chance.
Research on Fuzzy Preference Relations-Based MAS for Decision Method Table 2. Fuzzy Preference Relations(
191
γ=0.01)
Rs
A1
A2
A3
A4
O
A1
0.500
0.618
0.200
0.007
0.007
A2
0.380
0.500
0.010
0.150
0.010
A3
0.800
0.990
0.500
0.000
0.000
A4
0.993
0850
1.000
0.500
0.850
Simulation result plan A4 most melts, then the final policy-making plan is A4. ha rt
3
1
0.2
0.4
0.6
4
0.8
1.0
Fig. 2. A1,A2,A3, A4fuzzy value result
5 Conclusions The multi- agent technology development, causes the multi- main bodies technology to be allowed to enhance policy-making the accuracy and the policy-making efficiency, displays the multi- main bodies decision-making well the resources superiority. This article in sums up experience in the foundation, proposed based on multi- agent decision-making technology which fuzzy relates by chance, causes policy-making the result to be more reasonable, the experiment indicated, this method has the certain confidence level. This article provides the method may in the widespread domain application. Because its policy-making efficiency greatly enhances, the timely enhancement, enables it to have the very good application prospect in the tactical decision-making. Waits for question which further studies: Various agent weight factor determination, at present has the subjective factor; Various agent fuzzy number production, namely the fuzzy subordination function expression also has the certain subjectivity.
192
W. Jiang and Q. Jiang
Acknowledgments This work is supported by the Society Science Foundation of Hunan Province of China No. 07YBB239, and the Important Natural Science Foundation of Hunan Province of China No. 06JJ2033.
References 1. Bredin, J., Kotz, D., Rus, D., Maheswaran, R.T., Imer, C., Basar, T.: Computational markets to regulate mobile-agent systems. Autonomous Agents and Multi-Agent Systems 6(3), 235–263 (2003) 2. Lind, J.: Issues in Agent-Oriented Software Engineering. In: The First International Workshop on Agent-Oriented Software Engineering (AOSE 2000) (2000) 3. Fortemps, P., Roubens, M.: Ranking and defuzzification methods based on area Z compensation. Fuzzy Sets and Systems 82, 319–330 (1996) 4. Scherrer, B., Charpillet, F.: Cooperative Co-learning: A Model-based Approach for solving MultiAgent Reinforcement Problems. In: Proceedings 14th IEEE International Conference on Tools with Artificial Intelligence, pp. 463–468 (2002) 5. Nakanura, K.: Preference relations on a set of fuzzy utilities as basis for decision making. Fuzzy Sets and Systems 20, 147–162 (1986) 6. Jiang, W.J., Wang, P.: Research on Distributed Solution and Correspond Consequence of Complex System Based on MAS. Journal of Computer Research and Development 43(9), 1615–1623 (2006) 7. Liu, j., Jing, H., Tang, Y.Y.: Multi-agent oriented constraint satisfaction. Artificial Intelligence 136(1), 101–144 (2002) 8. Yager, R.R.: Quantifier Guided Aggregation Using OWA Operations. International Journal of Intelligent Systems 11, 49–73 (1996) 9. Weicai, Z., Jin, L., Mingzhi, X., et al.: A multi agent genetic algorithm for global numerical optimization. IEEE Transactions on System, Man, and Cybernetics 34(2), 1128– 1141 (2004) 10. Bredin, J., Kotz, D., Rus, D., Maheswaran, R.T., Imer, C., Basar, T.: Computational markets to regulate mobile-agent systems. Autonomous Agents and Multi-Agent Systems 6(3), 235–263 (2003) 11. Busoniu, L., Babuska, R., De Schutter, B.: A Comprehensive Survey of Multiagent ReinforcementLearning. IEEE Transactions On System, Man, And Cybernetics-Part C: Application And Review 38(2), 156–172 (2008) 12. Gou, Y., Huan, J.Z., Rong, H.: Adaptive grid job allocation with genetic algorithm. In: Future genre. comp. sys., vol. 21, pp. 151–161. Elsevier Press, London (2005) 13. Matsui, M.: A generalized model of convey-serviced production station (CSPS). Journal of Japan IndustrialManagement Association 44(1), 25–32 (1993) 14. Yamada, T., Satomi, K., Matsui, M.: Strategic selection of assembly system under viable demands. Assembly Automation 26(4), 335–342 (2006) 15. Dubois, D., Grabisch, M., Modave, F., et al.: Relating decision under uncertainty and multieniteria decision making models. In: IJCAI, Nagoya, Japan (1997) 16. Jiang, W.J., Wang, P., Zhang, L.: Research on Grid Resource Scheduling Algorithm Based on MAS Cooperative Bidding Game. Chinese Science F 52(8), 1302–1320 (2009)
Study of Stochastic Demand Inventory Routing Problem with Soft Time Windows Based on MDP Wei Zeng and Qian Zhao Institute of Systems Engineering, Huazhong University of Science and Technology, Wuhan, Hubei 430074, P.R. China [email protected]
Abstract. Compared with related studies of stochastic demand inventory routing problem (IRP), this paper takes the constraints of soft time windows into consideration. Specially, penalties for violating time window limits are included in value function; upper limit of time window is dependent on inventory level. These assumptions of IRP are more practical. Firstly, the paper represents stochastic demand inventory routing problem with soft time windows (IRPSTW) as a discrete time Markov decision process model. Then the paper proposes a solution with considering constraints of soft time windows. Finally the application of this solution is illustrated with a numerical example. Keywords: Inventory route problem, Soft time window, Stochastic demand, Markov decision process.
1 Introduction Inventory routing problem (IRP) that is a problem of how to make decisions regarding integrated optimization of inventory and transportation, is a core problem to be solved when implementing vendor managed inventory (VMI) [1]. Generally, the IRP has the following three problems to be solved [2]: ─ Frequency and time of replenishment. For a given planning horizon, when to replenish at each customer need to be decided. If planning horizon is not given, it is necessary to decide how frequently to execute replenishment service at each customer. ─ How much to replenish at customers. The supplier needs to decide how much to replenish at each customer. ─ Delivery route. The distribution vehicles need to be arranged optimally on the transportation route and service the customers in proper order. The first two problems are related to inventory control, and the last one is a vehicle routing problem (VRP). This paper studies the stochastic demand inventory routing problem with soft time windows (IRPSTW), while most related studies do not consider the constraints of time windows. For example, toward stochastic demand and short-term IRP, Federgruen et al study a kind of single product and single day planning horizon IRP. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 193–200. springerlink.com © Springer-Verlag Berlin Heidelberg 2010
194
W. Zeng and Q. Zhao
The IRP is divided into inventory allocation problem and traveling salesman problem (TSP) based on nonlinearity integer programming model [3]. Golden et al defined demand urgency as the ratio of inventory capability to practical inventory level. First, the supplier calculates each customer’s demand urgency and chooses a customer to be delivered directly based on demand urgency. Then formulate and solve a traveling salesman problem (TSP) by iteration [4]. Jaillet et al use a rolling horizon framework to solve the IRP with a period of two weeks [5]. Minkoff et al represents the stochastic demand and long-term IRP as a Markov decision process (MDP). To overcome the calculation difficulty brought from huge state space, they put forward a heuristic algorithm based on the idea of decomposition [1], [6]. Anily et al study replenishment strategies of indefinite horizon IRP in order to minimize long-run average transportation and inventory costs. A collection of regions (sets of retailers) is specified which cover all outlets. Each time of the retailers in a given region receives a delivery, this delivery is made by a vehicle who visits all other outlets in the region as well [7]. Similarly to this paper, Li et al study the IRP with constraints of hard time windows, which means the IRP with constraints of time windows should be inviolable. In fact, it is generally unpractical to assume that the constraints of time windows are inviolable [8]. The paper studies the stochastic demand inventory routing problem with soft time windows. In other words, the penalties for violating time windows are taken into consideration. If your paper is intended for a conference, please contact your conference editor concerning acceptable word processor formats for your particular conference.
2 Problem Definition 2.1 Problem Hypothesis In order to facilitate to research stochastic demand IRP under the constraints of soft time windows, we have the following assumptions without loss of generality: • The entire logistics system delivers only a kind of production. • The demand of each customer is mutual independent and its probability distribution is deterministic without change over time in a planning period. • The supplier monitors the customer's inventory level and knows each customer's demand probability distribution. • Out of stock is allowed. Demand loss occurs in case of stockout. • The vehicle fleet size is limited; all vehicles have the same maximum capacity and the same cost characteristics. • Ignoring the time of loading and unloading goods. No demand occurres from acquiring the customers’ inventory level to making a decision on delivery quantity d i to the customer i . • Each vehicle can perform several delivery tasks per day. Each customer may receive several deliveries every day. • Do not consider inventory cost of distribution center. • Transportation costs is only depended on travel distance.
Study of Stochastic Demand IRPSTW Based on MDP
195
• Customer’s inventory cost per day is proportional to the maximum storage capacity of that day. • Storage capacity of distribution center is infinite. • Each customer's inventory state is discrete. • Consider the constraints of soft time windows. Specially, delivery vehicles should delivery goods to customer within a specified time window. If violating the constraints of time windows, should pay penalties. 2.2 Problem Description Consider products are delivered from a distribution center to N customers using a fleet of M homogeneous vehicles. The storage capacity of each customer is Ci (i = 1, 2,..., N ) . The maximum capacity of each vehicle is CV . Delivery vehicles
will deliver products to clients within a specified time window[ ETi , LTi ] , ETi is the earliest start time customer allows and LTi is the latest start time. If delivery time violates the time window limit, there will be a penalty for it. The process is modeled in discrete time t = 0, 1,..., and the discrete time periods are called days. Customers' demands on different days are independent random vectors with a joint probability distribution F . Distribution center makes decisions every day regarding which customer’s inventory to replenish, how much to deliver at each customer and how to combine customers into vehicle routes. Suppose everyday’s decisions should be completed to ensure that all vehicles available at the beginning of the next day. The revenues and costs of the logistics system include: a reward of r ( di ) when the supplier delivers quantity d i to customer i ; the travel cost ci , j on the arcs (i, j ) of the distribution network that is the shortest distance between the customer i and j; penalty pi ( si ) for inventory shortage assuming the shortage quantity at customer i is si ; each customer's inventory holding cost hi ( xi + d i ) assuming the inventory at customer i is xi at the beginning of the day and quantity d i is decided to deliver at customer i ; for customer i , assuming the penalty is mi if delivery time violates lower time limit and the penalty is li for violating upper time limit. The objective is to choose an optimal distribution policy that maximizes the expected discounted value (revenues minus costs) over an infinite time horizon.
3 MDP Model In the paper the IRP is formulated as a discrete infinite-stage Markov decision process model. The MDP model is described as follows: 1. Decision phase: Each decision phase is defined as a day, which is denoted as T = {t | 0,1, 2,...} . 2. State space: The state variable x = {x1 , x2 ,..., xN } represents the current inventory level at each customer. Thus the discrete state space is
196
W. Zeng and Q. Zhao
χ = [0,1,..., C1 ] × [0,1,..., C2 ] × ... × [0,1,..., C N ] Let X it ∈ {0,1,...Ci } denote the inventory level at customer i at time t . Let Xt ≡ ( X1t , X 2t ,...X Nt ) ∈ χ denote the state at time t . 3. Action space: For any state x , action space A( x) is the set of any feasible decisions in state x . Any feasible decision a ∈ A( x) made in state about
x contains information
─ which customer’s inventory to replenish? ─ how much to deliver at each custom? ─ how to combine customers into vehicle routes?
The set A( x) is determined by various constrains, such as the limited number M of vehicles that can be used each day, the limited quantity CV that can be delivered by each vehicle, the maximum inventory levels Ci that are allowed at each customer i and so on. For any decision a , let d i (a ) denote the quantity of products that are delivered at customer i while executing the decision a . 4. Transition probability: Let U it denote the demand of customer i at time t . Then the amount of products used by customer i at time t is denoted as min { X it + di ( At ),U it } . Thus the inventory shortage at customer i at time t is
given by Sit = max {U it − ( X it + di ( At )),0} , the inventory level of customer i at time t + 1 is given by X i ,t +1 = max { X it + di ( At ) − U it , 0} . Markov transition matrix
Q is determined by the known joint probability distribution F of customer’s
demands U it . For any state x ∈ χ , any decision a ∈ A( x) and any subset B ∈ χ , u ( x, a, B ) ≡ {U ∈ R+N : (max { x1 + d1 (a ) − U1 , 0} ,..., max { x N + d N ( a ) − U N , 0} ∈ B} ,
then Q ⎡⎣ B x, a ⎤⎦ ≡ F [u ( x, a, B ) ] . Thus P ⎡⎣ X t +1 ∈ B X t = x, At = a ⎤⎦ = Q ⎡⎣ B x, a ⎤⎦ ≡ F [u ( x, a, B ) ]
5. Rewards: Let g ( x, a ) denote the expected single stage reward if the process is in state x and decision a ∈ A( x ) is executed. For any decision a and arc (i, j ) , let kij ( a ) denote the number of times that arc (i, j ) is traversed by a vehicle while
executing the decision a . Let gi denote whether delivery task i is below the lower limit of time window. g i is set to 1 when task i is below the lower limit of time window, otherwise g i is set to 0. Let qi denote whether task i exceeds the upper limit of time window. qi is the set to 1 when task i exceeds the upper limit of time window, otherwise qi is set to 0.
Study of Stochastic Demand IRPSTW Based on MDP N N g ( x, a ) ≡ ∑ r ( d ( a )) − ∑ c k ( a ) − ∑ h ( x + d ( a )) i i ij ij i i i i =1 i =1 (i , j ) N N − ∑ E ⎡ p (max U − ( x + d ( a )), 0 ) ⎤ − ∑ ( g m + q l ) ⎥ F ⎣⎢ i i0 i i i i ii ⎦ i =1 i =1
{
197
(1)
}
6. Objective function: The objective is to maximize the expected total discounted value over an infinite horizon. Let α ∈ [ 0,1] denote the discount factor. Let ∏ denote the set of policies. Policy π ∈ Π prescribes a decision π ( x ) ∈ A( x ) based on the information contained in the current state x of the process only. And for any policy π and any state x ∈ χ , the expected value V π ( x) can be expressed as: ⎡∞ ⎤ V π ( x ) ≡ E π ⎢ ∑ α t g ( X t , π ( X t )) X 0 = x ⎥ ⎣ t =0 ⎦ π = g ( x, π ( x)) + α ∑ V ( y )Q ⎡⎣ y x, π ( x) ⎤⎦
(2)
y∈χ
Then the optimal expected value can be given by: ⎧ ⎫ V ∗ ( x) = supV π ( x) = sup ⎨ g ( x, a) + α ∑V * ( y)Q ⎣⎡ y x, a ⎦⎤ ⎬ a∈ A( x ) ⎩ π ∈∏ y∈χ ⎭
(3)
∗
If a policy π ∗ which makes V π = V ∗ exists, then it is the optimal policy.
4 Solve the Model Because the state space of MDP model is too large, the traditional solving methods for MDP will not be adopted in this paper. Otherwise, an algorithm similar to reference [1] will be proposed. Its algorithm flowchart is shown in Fig.1. Compared with reference [1] and related studies of IRP, this paper takes the constraint of time window into consideration. Given a current state x = ( xi , x j ) , the constraints of time window of customer i and customer j are determined according to the inventory level of each customer. Suppose that travel route is given, arrival time ti at customer i can be calculated by the travel distance and travel speed of vehicle. Comparing ti with the upper and lower limit of time windows, whether delivery task i violates the constraint of time window can be determined. When customer i and customer j are combined into one vehicle route, there are two cases: arrive at customer i from the distribution center, then arrive at customer j ; arrive at customer j from the distribution center, then arrive at customer i . The optimal sequence of visiting can be determined by comparing these two cases. However, when the number of customers on a vehicle route increases, the problem will be more complicated to solve. Considering that there are usually not too
198
W. Zeng and Q. Zhao
many customers on a vehicle route, it is assumed that no more than 3 customers will be combined on a vehicle route in the following example.
Solving IRP
Building approximation value function
Dividing customer set into subsets Given a policy, simulating the entire IRP to determine parameters of sub-problems
Solving the sub-problems
Determining approximate function through subproblems joint optimization
Building the approximate function
Solving optimal policy of each stage
Solving the policy of direct distribution Adding the rest customers into the direct delivery routes
Solving value function and select the optimal policy
Stopping selecting the optimal policy of each stage
Solving IRP ends
Fig. 1. Algorithm flowchart
5 Example Study Given the following example, all customers are served by a distribution center. The inventory and order costs of the supplier are not considered. The logistics system has 3 customers whose maximum inventory level is 6. There are 2 vehicles whose capacity is 6. It is assumed that each vehicle at most executes one delivery task per day. The example data is shown in Table. 1 and Table.2. In addition, CV = 6 , α = 0.9 , N = 3 , M = 2 , C12 = 80 , C13 = 60 , C23 = 40 ,
V = 90km / h , where Ci , j denotes the distance between customer i and j, for example
Study of Stochastic Demand IRPSTW Based on MDP
199
C13 represents the distance between customer 1 and 3. Let the upper limits of time windows dependent on the customers’ inventory level. In table 2 according to inventory level 0, 1, 2, 3, 4, 5 and 6, upper limits of time windows are given respectively. Table 1. Example data i
Ci
1
fi
ci
ri
pi
hi
0
70
20
45
5
0
0
140
15
45
5
0.5
0.5
120
25
20
5
0
1
2
3
4
5
6
6
0
0.5
0.5
0
0
0
2
6
0
0
0
0.5
0.5
3
6
0
0
0
0
0
Table 2. Time window limits lower limit of time window
0
1
2
3
4
5
1
8:00
8:20
8:30
8:40
20:00
20:00
2
8:10
8:20
8:25
8:40
9:00
3
8:00
8:20
8:20
8:30
8:50
i
6
Lower limit penalty
Upper limit penalty
20:00
20:00
50
120
9:30
20:00
20:00
50
260
9:20
10:00
20:00
50
210
upper limit of time window
Without loss of generality, only the computational result of state (5, 1, 1) is given here. The optimal value of state V (5,1,1) is -3120.75, the optimal policy is that the distribution vector is (1, 2, 5), which means delivery quantity at customer 1, 2 and 3 is 1, 2, 5 respectively; there are two vehicle routes, 0-1-3 and 2-0, which means customer 1 and customer 3 are combined into the same route, customer 2 is on another route. If there is no constraint of time window and other parameters of the example remain unchanged, the optimal policy is that the distribution vector is (0, 3, 5) and the optimal routes are 0-2, 0-3 in the state (5, 1, 1). Compared the example’s computational results with constraints and without constraints of time windows, the obvious influence on the solution of IPR with constraints of time windows is shown.
6 Conclusion Compared with the researches of stochastic demand IPR, this paper takes the influence of constraints of soft time windows into consideration. The penalty for violating the time window limit is included in the value function; upper limit of time
200
W. Zeng and Q. Zhao
windows is dependent on inventory level. These assumptions of the IRP are more practical. Firstly, the stochastic demand inventory routing problem with soft time windows (IRPSTW) is formulated as a discreet time Markov decision process model in this paper. Based on the algorithm similar to reference [1], a solution with considering constraint of soft time window is proposed. According to the research, the constraints of soft time windows have obvious influence on optimal policy, so it is meaningful to do the research of IRP with constraints of time windows. Acknowledgements. This research is supported by the Natural Science Foundation of China (Grant No. 70971048).
References 1. Kleywegt, A.J., Nori, V.S., Savelsbergh, M.W.P.: Dynamic Programming Approximations for a Stochastic Inventory Routing Problem. Transportation Science 38(1), 42–70 (2004) 2. Qu, W., James, H.B., Iyogun, P.: An Integrated Inventory Transportation System with Modified Periodic Policy for Multiple Products. European Journal of Operational Research 115(2), 254–269 (1999) 3. Federgruen, A., Zipkin, P.: A Combined Vehicle Routing and Inventory Allocation Problem. Operations Research 32(5), 1019–1036 (1984) 4. Golden, B., Assad, A., Dahl, R.: Analysis of a Large Scale Vehicle Routing Problem with an Inventory Component. Large Scale Systems 7(2), 181–190 (1984) 5. Jaillet, P., Bard, J.F., Huang, L., Dror, M.: Delivery Cost Approximations for Inventory Routing Problems in a Rolling Horizon Framework. Transportation Science 36(3), 292–300 (2002) 6. Minkoff, A.S.: A Markov Decision Model and Decomposition Heuristic for Dynamic Vehicle Dispatching. Operation Research 41, 77–90 (1993) 7. Anily, S., Federgruen, A.: One Warehouse Multiple Retailer Systems with Vehicle Routing Costs. Management Science 36(1), 92–114 (1990) 8. Zhao, D., Li, J.: An algorithm for Stochastic Demand Inventory Routing Problem with Time Windows (in Chinese). Science Paper Online, http://www.paper.edu.cn
An Agent-Based Approach to Joint Procurement Modeling with Virtual Organization Chih-Yao Lo1,2, Hung-Teng Chang2, Yu-Teng Chang2, and Hsiu-Yun Hu2 1
School of management, Huazhong University of Science & Technology 2 Department of Information Management, Yu Da University {jacklo, cht, cyt and 97310022}@ydu.edu.tw
Abstract. The research aims at profit maximization, and firstly, according to the purchase methods for one-to-many purchase situation, the research installs purchase agent system model of virtual organization to solve the problems of purchase possibly faced by the virtual organization on supply chains. The research uses the results of value allocation model of virtual organization researched by AHP to install the module of value distribution agent system. In the last step, the research uses the joint procurement cycle and quantity estimation model made